zin1953 wrote:At least superautos take one "M" out of the equation.

If only.... I stood in line Sunday at a coffee kiosk at the Getty, watching as a high-end La Cimbali superauto was allowed to continually froth a pitcher of milk for several minutes. By the time they were done with it, they had reduced the milk to its subatomic components. Yes, I ordered the iced tea.

another_jim wrote:The rule is that we don't speak ill of other coffee people. If a coffee or piece of gear isn't good, we don't review it. This is neither advertising, which is praising things regardless of their quality; nor is it journalism, which is reporting other people's reactions, regardless of their or the reporter's expertise.

Nor is it accurate critical review. I didn't mean to offend, Jim, but if you're not going to tell the good and the bad, then it isn't journalism at all, and I'm not sure if Marshall intended (but I am sure that he'll tell us) to include what you do in the thread.
What you do is great, but it doesn't, as you say, qualify as journalism. It is decidedly one-sided. And it tends to be more akin to advertising than anything else, as it won't do anything but praise? Leaving out the points that may harm sales while trumpeting the ones that will boost sales is sort of the trademark of advertising, ain't it?
It seems like there are some grey areas and crossovers that are confusing me a tad, and I'm sort of lost as to what 'technically' is considered which.

In any case, what I was trying to say is that if a journalist doesn't feel comfortable telling us what is what, then he's not in a journalistic endeavor.
Folks like yourself and Dan, et al, are not jurnalists in my narrow POV, but buddies that I'm extremely jealous of that get to play with the next cool toys that they drool over, and tell us what hapless geeks we are for wanting them so badly and not being able to have it all! (OK, so this is somewhat exaggerated *and* hyperbolic!)

zin1953 wrote: <snip>
One more problem with reviewing coffees that I have yet to bring up: the date of the review.

In wine, just because the 2009 Cabernet Sauvignon from Château Cache Phloe might be stunning doesn't mean the 2010 or 2011 vintages will be. In coffee, it's the same thing. Being an agricultural crop, sometimes Mother Nature gives up better beans than in other years. A roaster may be able to compensate for this in a blend, just as producers of non-vintage Champagnes can, but those who offer single types of beans (e.g.: Columbia) will have a more difficult time, and just as with single vineyard wines, single origin coffees from a single farm can vary significantly from crop-to-crop.

But there are far too many roasters and beans to review them all each and every single year. Thus, some reviews will be out-of-date, and thereby give the consumer false or misleading information about currently available coffees. Institutional hazard, and one I know NO WAY of avoiding.

Per my own comment above, I agree about outdated reviews, and in fact would go further & say that IMHO, many if not most of the better blends change enough over time to require regular revisit. Heck, even if a blend is unchanging, other coffees change, and therefore it changes by context/comparison.

But, that said, I think things changing over time is always a problem with reviews (equipment OEMs make changes from year to year too that can materially affect product performance & render a review outdated), and that the easiest improvement the average coffee drinker can access is still better beans. I also think that having a steady supply of new reviews, notwithstanding the inability to review everything every year, is of interest both to the coffee freaks who read forums like this all the time and to interested but less focused consumers (heck, Car and Driver doesn't review every car model from the major OEMs every year, and there are far fewer of those, nor Wine Spectator every new year's wines). The limitations of coffee reviewing are substantial, there's little doubt of that, but I still wonder if we wouldn't all be better off with more focus on the bean & less on the boiler.

Psyd wrote:I didn't mean to offend, Jim ... Leaving out the points that may harm sales while trumpeting the ones that will boost sales is sort of the trademark of advertising, ain't it?

You have to understand my posts in order to offend me.

We don't leave out points, the reviews are of machines we buy or would buy. If there are negative points, we post them, even if they show up later, e.g. the early demise of Elektra Semi pstats, which is now part of the review (including ae downgrade of the point score). If that had been apparent earlier, it would have been included in the initial review.

I've even retracted a recommendation, that of the Versalab grinder, whose flaws are, IMO, fatal. However, I would have far preferred not recommending it in the first place.

Much more importantly, many today, including you in the post above, fail to understand the most basic truth about objectivity: you have to be generous before you can be objective. To understand any human endeavor requires a sympathetic understanding of what the people doing it intended. The failure of most journalists to sympathetically understand means they are incapable of even seeing the facts, never mind passing judgment on them. This is why a lot of the media is held in well deserved contempt nowadays; people read the stories and a part of them asks: "what if this reporter had reported on me in this fashion?"

By this standard, an objective negative review is something exceedingly difficult to produce. One can judge and then dismiss thing, saying it misses the mark; but that isn't a review, merely a warning. Since a review requires a sympathetic understanding of the thing, and of its creation; if the thing went wrong, it requires a sympathetic understanding of how the flaws occurred. Therefore, a true review that is negative requires a degree of patience unlikely to be forthcoming from anyone except editors, friends, competent bosses, or dedicated teachers. The type of so-called negative review one gets from critics is merely an exercise in comedic vituperation, they are warnings for people to stay away wrapped in comedy writing.

As to us espresso gear reviewers warning people away from what we think are bad products; we do that all the time in the forums. We just don't pretend these warnings are reviews.

Marshall wrote:If only.... I stood in line Sunday at a coffee kiosk at the Getty, watching as a high-end La Cimbali superauto was allowed to continually froth a pitcher of milk for several minutes. By the time they were done with it, they had reduced the milk to its subatomic components. Yes, I ordered the iced tea.

Ah well, another theory blown out of the water! Yes, ladies and gentlemen, you STILL need to train people on a superauto!

(BTW, Marshall, I think I would have ordered the iced tea regardless . . . )

another_jim wrote:We don't leave out points, the reviews are of machines we buy or would buy. If there are negative points, we post them, even if they show up later, e.g. the early demise of Elektra Semi pstats, which is now part of the review (including ae downgrade of the point score). If that had been apparent earlier, it would have been included in the initial review.

I've even retracted a recommendation, that of the Versalab grinder, whose flaws are, IMO, fatal. However, I would have far preferred not recommending it in the first place.

I thought coffeecuppers.com only reviewed beans. Or are you talking about the reviews here on HB?

I have been a lurker here for a month or so and have learned a lot from the experience of others who frequent HB, but this has been one of the most interesting posts to follow, and the first that I have followed from the beginning. For the last few days I have been excited to check the forums a few times a day to see if anything has been added to the thread. This is why I love this site, knowlegeable people discussing an interesting issue in an intelligent manner.

Welcome to HB, and congratulations for leaving "lurk mode" and posting! I hope you'll continue to participate . . .

I may be talking here out of turn. Meaning, I don't offer enough to this community to really be a critic on this topic. But, here goes:

1. In order for a "amateur or casual reviewer" to get serious, Marshall is right they have to stop looking through rose colored glasses. That takes a substantial investment. Me personally, I have a commercial HX Faema S87 machine. Even with that I haven't spent the money on things like a Scace Device and such. In ORDER TO stop looking through those rose colored glasses a reviewer would need to get real scientific with his review process, but ALSO he would need to invest in possible up to three or maybe more machines.

The Problem with that is money, with this economy, and wives!

2. The solution to the above is being paid to review stuff. Not that the money would skew your results, but that the money would help you maintain and acquire more equipment. Don't think I am being silly here. I have just been offered money to do some work like this and I tried to turn it down and he insisted. We will be working together on some projects in the future. I just did a review for another company and it takes a considerable amount of time out of your schedule to do it right and then do the proper write up.

If your wanting to develop better reviews, I think we need to look at the ball from a different angle.

When it comes to professional espresso (and coffee) review journalism, I want nothing less than a comprehensive genius, weekly column front page of NY Times would be nice.

When it comes to "amateur" reviews, don't be so hard on yourselves. This community is nice with some good leaders in the mix, and I hope they keep up the good work... and of course there's tons more cool work to be done. 'Most everyone here should pump more effort into their reviews.

another_jim wrote:You have to understand my posts in order to offend me.
As to us espresso gear reviewers warning people away from what we think are bad products; we do that all the time in the forums. We just don't pretend these warnings are reviews.

Yeah, I think a bit is getting lost in translation. Are the 'we' that you speak of, you and I here on HB? If that's the case, I'm not sure (as I said in the post that you quoted) if that's quite the target of Marshall's criticism. I'll let him distinguish if he cares to, but my take (in a nutshell) was that we are the only alternative because professional journalists aren't up to snuff. To suggest that 'we' are adequate really doesn't address his lament that those that criticise professionally aren't covering their bases. Or maybe it's the percocet. Possibly the recent head trauma. I could just be getting this all wrong.
And the definition of review vs warning is troubling me a bit. Granted, I get most of my 'reveiw' and criticism' definitions from the entertainment world, reviews of movies and musical performances, criticism of plays and concerts, etc, but they're still called a reviewer if they say that it's bad and I shouldn't go see it, and they're still called the theatre critic if they only have bad things to say about it. I'm not sure why they are free to say the good and the bad, and with espresso machinery somehow it's different. Like you say, I might understand these subtleties if I understood your posts.
What is it that I'm missing? Go slow and simple, I really am recovering from a head trauma.

I didn't mean to get down on you, Chris. I don't know either what the thread is about.

Regular newspapers' or consumer reports type reviews are never taken seriously by enthusiasts. The people doing these reviews are "professional testers," rather than experts in what is being reviewed. If they are such great "professional testers," why don't we allow them to do our medical tests; if they can do washing machines one day and cameras the next, why not us on Wednesdays?

The idea that these "professional, objective, 83 point checklist, I have a voltmeter from Radio Shack," reviews are meaningful was always a non-starter. Newspapers had food, theater, and movie critics, along with sports writers serving the same role for the local teams. That took care of the passions of all the larger groups in town. With the internet, we now can also address the passions of the smaller groups, including us espresso riff-raff. So I say again, our reviews shouldn't pretend to be objective; we should fall in love or grow to hate machines, since this is our hobby. Yeah, Marshall will get La Spaziale fanboys dissing his De La Corte the same way Celtics fans diss the Lakers. But so what?

There is the other side of this as well. Look at it from the POV of the reviewer...

The closer you get to the top of the pyramid, the more zealous the zealots become, and the fewer zealots there are to serve. So right off the motivation is low. And dealing with the zealots can be... well.. trying to say the least. As an example, you may remember a recent discussion over the 7000-whatever-dollar that the GS? LM machine cost, and when I mentioned that the amateur grinding done on the frame under the drip tray was ridiculous, my comments were brushed aside [with something like, '..you have to expect that with a low-production item such as this,' etc.]... BULL!

Serving the needs of fewer people, some of whom would appear as if they have made up their mind as to what they want to purchase and are looking more for justification for their planned purchase than information to assist purchase. As one who has written a few reviews, serving the needs of such
"readers" doesn't sound too tempting to me...

Looking at it from that perspective, we are lucky to have resources such as HB, CG, and the few other such places to find out about the that top 10% of the pyramid. If it wasn't for the internet, how many of us would know about Vibiemme, the Silvia, or so many other such machines? I probably would have ended up with a Solis SL70 from the local kitchen store ad the Cuisinart grinder from Costco... I shudder to think about it.

zin1953 wrote:To say that "espresso review journalism is seriously broken," and ask how it can be fixed implies several things, not the least of which is that people care. No one does.

Thank you.

Malachi, just yesterday I was wishing you would still describe shots you're tasting.

zin1953 wrote:To say that "espresso review journalism is seriously broken," and ask how it can be fixed implies several things, not the least of which is that people care. No one does.

malachi wrote:Thank you.

They don't care (a dubious conclusion, but I'll assume it is true), because:

1. No one is interested in the quality of a machine before or after buying it?

2. Everyone thinks they are getting a broad range of well-informed reviews?

3. Ignorance is bliss?

SL28ave wrote:Malachi, just yesterday I was wishing you would still describe shots you're tasting.

I blush.

There are plans afoot.

Marshall wrote:They don't care (a dubious conclusion, but I'll assume it is true), because:

1. No one is interested in the quality of a machine before or after buying it?

2. Everyone thinks they are getting a broad range of well-informed reviews?

3. Ignorance is bliss?

1 - They are - and there are million marginally informed sources to tell them. And how would you differentiate the one out of the million who actually is informed.

2 - Yes. See above.

3 - If behavior is any indication then sadly yes.

Randy G. wrote:There is the other side of this as well. Look at it from the POV of the reviewer...

People rarely consider what goes through the reviewers mind. I have reviewed many (more than 30) prosumer machines over the years and it's a difficult job. I certainly don't do it for the money and spend at least 3 weeks using the machine day in and day out (because some of them grow on you over time) and over 40 man hours doing the various write ups (which are secured so the retailer cannot change them), many of which the consumer never sees. People wonder why the bad reviews are not on the web....simple answer:

When I review a machine the brief is if it's no good, be honest, as we don't want to stock it, we have only ordered the one for you to review and if necessary we will ebay it out at cost . So it is pointless me doing a "sales"/glowing review on a machine that's not very good, it doesn't help anyone. The reason negative reviews are not often seen on retailers web sites (a few have been archived), is because it's not worth the aggravation for the Retailer, if the manufacturer sees the review (why should the retailer bother)...and believe me, some of the manufacturers don't appreciate constructive criticism.

The other problem is that many of the prosumer machines are pretty good (considering they are electromechanical devices dealing with steam and hot water all day long, usually with little/no maintenance and often with poor quality water. They have to be assessed in terms of their price vs the price of other machines, so a £700 machine will get more leeway than a £4000 one. I think if I had a GS3 to review, it would probably get a lot of focus on certain issues that are not great when paying that amount of money (e.g. I would have been singularily unimpressed at the lack of a ball-joint steam wand, or the ability to turn off the 3.5 litre steam boiler etc..). Similarly, I cannot expect the same quality standards from an Expobar Pulser for £450.

http://coffeetime.wikidot.com/cho...n-espresso-machine

This link is an article I wrote, which pretty much put into perspective what goes on from my point of view. It's advice for the consumer, but so often ignored....equally reviews are not often read properly.

e.g. take the Dalla Corte review I did (Marshal linked to it in one of his threads), the machine gets pretty hot and I mention that you can't turn the steam boiler off and it's a mod I would do. The perceptive reader will realise that although not a weakness in the machine per se....it's:

1. A modification they should do sooner than later (Marshal) and it's really easy to do because XP700 pressurestat switches the full element current within the relay on the top. It's also completely reversible (if you find a place where you don't need to drill a hole) and should only take a competent person less than 1 hour. http://coffeetime.wikidot.com/ma-ter-xp700-pressurestat ....really though it would less than $10 for the manufacturer to make it a standard feature.

2. Something that will stop it getting so hot and lengthen the lifetime of components

So these things and other areas didn't pump my nads when considering the cost of the machine, are bought out in the review, but are not really negative points as such (well except to me personally) and it's still a great espresso machine.

God help us if reviewers gets as picky and single minded as the purchaser/potential purchaser, we need to remain objective and fair to both the manufacturer and the consumer. Wearing rose tinted glasses is not helpful and the level of knowledge of the reader (which covers a considerable range) has to be considered when writing any review.

So just a view from one of those people on the receiving end of threads like this one

P.S. ....the "Sad state of espresso review journalism", may be in part, the fault of the consumer, if consumers only bought from places with detailed and frank reviews, reviews would get better and better....but they don't. Often consumers go to box shifters to get the cheapest price...box shifters don't pay to have reviews done and certainly don't encourage manufacturers to improve their machines.

DaveC wrote:e.g. take the Dalla Corte review I did (Marshal linked to it in one of his threads), the machine gets pretty hot and I mention that you can't turn the steam boiler off and it's a mod I would do. The perceptive reader will realise that although not a weakness in the machine per se....it's:

1. A modification they should do sooner than later (Marshal) and it's really easy to do because XP700 pressurestat switches the full element current within the relay on the top. It's also completely reversible (if you find a place where you don't need to drill a hole) and should only take a competent person less than 1 hour. http://coffeetime.wikidot.com/ma-ter-xp700-pressurestat ....really though it would less than $10 for the manufacturer to make it a standard feature.

2. Something that will stop it getting so hot and lengthen the lifetime of components

Thanks for the tip, Dave. If that diagram continues to baffle me (which I expect it will), I'll take it to a friend who will know what to do.

By the way, I hope you did not think I was criticizing your review or style of review. I thought it was very helpful, which is why I linked to it.