zin1953 wrote:To say that "espresso review journalism is seriously broken," and ask how it can be fixed implies several things, not the least of which is that people care. No one does.

Before this goes too far afield, let me reiterate what I said, rather than have this one line floating completely out of context . . .

In the context of my original post on Page One, "No one" means the population of people who drink coffee on a regular basis. Obviously the passionate participants of home-barista.com care, but we are already preaching to the converted. We are but a very small community compared to the number of people who regularly drink some sort of espresso-based beverage -- tiny, compared to the number of people who visit Starbucks at least once a week!

In this (and this is where the whole wine analogy began), we chat among ourselves and run the very real risk of presuming all those "Starbucks people" would happily join us if only they knew . . .

On wine boards, especially those where Parker is considered a god, far too many people either make "newbies" feel uncomfortable by using "winespeak" instead of English; by making people feel there actually are such things as "stupid questions"; and not welcoming them with open arms and not being willing to answer the most basic of questions in a simple, straight-forward, and helpful way.

It can often feel to the casual participant/lurker as though everyone has the secret decoder ring but them!

So when I wrote "no one cares," it is in that context: obviously we do care; a number of us do care; but that actual number -- no matter its real size -- is but a small drop in the proverbial ocean of coffee.

To come to sites such as home-barista.com, one already has to enjoy espresso-based drinks enough to actually sit down and search the site(s) out. That already displays significantly more interest than most of the planet! To actually register and post a first question on this site takes courage. And to participate on even a semi-regular basis takes an even greater commitment, both of time and of passion.

So when Marshall writes,

Marshall wrote:They don't care (a dubious conclusion, but I'll assume it is true), because:
1. No one is interested in the quality of a machine before or after buying it?
2. Everyone thinks they are getting a broad range of well-informed reviews?
3. Ignorance is bliss?

I first have to say this is a faulty interpretation of what I said, but no doubt the fault is mine, not Marshall's. However . . .

1. "No one" is interested in the quality of a machine before or after buying it because "no one" (i.e.: the vast majority of the population on Planet Earth) is interesting in buying a machine, period. (Do you read every issue of Consumer Reports, or do you only look at it when (e.g. it's time to replace the refrigerator or the stove?) If and when someone becomes interested in buying a machine, then (and only then) will that individual become interested in the quality of machines which are potential candidates for purchase. After buying it, it's too late, and no one wants to hear they just spent $_______ on "an Edsel."

2a. To the best of my knowledge, there is no published magazine one can buy at a newsstand or via subscription that reviews espresso machines and coffee makers exclusively. So aside from an occasional "comprehensive" review in something like Consumer Reports (the "White & Patio" programmable coffee maker at $19.99 is a CR "Best Buy", much better than the "Vickers" model 303), you get an occasional mention in some food/cooking magazine . . . and most (though not all) would fall into the super-auto category, recommending the reader go to Sur La Table or Williams-Sonoma, rather than places like Chris' Coffee, 1st-Line, EspressoParts NW, etc.

2b. So, again, IF the individual is already "into" espresso enough to be looking at a high-end consumer, or even "prosumer" machine, they already know not to go to Starbucks, Costco, Williams-Sonoma, or Sur la Table . . . so they come here, other similar sites, or maybe stumble upon WLL¹. Thus, they seek out information from "fellow consumers like them," who are perhaps a bit more knowledgeable -- or at least, who already own such types of machines -- for additional information. And even if they read the formal reviews (Buyer's Guide to _______) here, or the "smackdowns," or articles/discussions like the Titan Grinder Project, they (generally) will STILL post questions directly to the board, asking about this or that machine.

3. I don't know anyone who will fork out $3K in total ignorance, Marshall. Heck, I'm not sure I know anyone who would fork out $600 for a new Silvia in total ignorance. This is not a serious issue.

Cheers,
Jason

¹ I always found it more likely that "newcomers" would find Whole Latte Love's site before finding someone like 1st-Line, Chris Coffee, etc. YMMV.

DaveC wrote: People wonder why the bad reviews are not on the web....simple answer: When I review a machine the brief is if it's no good, be honest, as we don't want to stock it, we have only ordered the one for you to review and if necessary we will ebay it out at cost . So it is pointless me doing a "sales"/glowing review on a machine that's not very good, it doesn't help anyone.

There is a night-and-day difference between being a journalist and being a reseller reviewing machines for a commercial website. As a reseller, that person has a vested interest in the review even before the machine shows up for testing and review. That review of a machine is done based on an entirely different set of standards which are based on commercial needs— it's an advertisement. There is nothing wrong with that, and no one expects anything different, but there is a dramatic difference between that and a journalist reviewing a machine. If no other factor is considered, I would not expect to go to a reseller's website and see a machine with a list of reasons why I should not buy it, and exactly why it is a bad machine.

In other words, a good journalist is expected to report the bad machines as well as the good ones, and the problems of a machine as well as its benefits in a balanced and objective manner. I don't expect to see that on a reseller's website.

Randy G wrote:But by your own statement, you are not a journalist- you are a reseller who has a vested interest in the review even before the machine shows up for testing and review. Your sort of review of a machine is done based on an entirely different set of standards which are based on your needs. There is nothing wrong with that, and no one expects anything different, but there is a dramatic difference between a journalist reviewing a machine and a reseller reviewing a machine. If no other factor is considered, I would not expect to go to a reseller's website and see a machine with a list of reasons why I should not buy it, and exactly why it is a bad machine.

I think you drawing inferences from my post that are completely incorrect, perhaps you should re-read what I said and read what's on my NON commercial Wiki.

1. I am NOT a reseller
2. I have no vested interest in the review
3. The review is not done on a set of standards based on MY needs, I do it for the consumer
4. I do not make part or all of my living from Coffee (not even the smallest part)

I pick and choose whether I will review a machine or not....I usually won't review low end non prosumer stuff and superautos, although I did review a couple of highly expensive, but appalling superauto machines (and regretted wasting any time on them at all), I in fact cut the review short after a few days and refused to waste any more time on them. The reason why, is I simply don't want to waste my time reviewing these type of machines. Reviews are secured and will not be changed (and I have had manufacturers want me to change them), unless I decide there is an error, the retailer concerned can choose to use them or not, that's up to them.

I get paid whether the review is good or bad, long or cut short (the money is the same), much of the money I get, goes to charity or coffee projects, even though it's not very much.....i think if I count up the hours I spend reviewing and the money I get minus my costs, it works out to less than 80 cents per hour. For all that, I produce a review, specsheet, manufacturers feedback (so they can improve their machine) and a decent user guide.

I sell nothing, I am retired and everything I do that's coffee related, I do to try and help the consumer (including the reviews). I make NOTHING from coffee.....and the reviews are so much work, I feel I go to extraordinary lengths to try and be thorough. Currently I couldn't care less if I never do another one again, especially if people read them and think as you do!

I did wonder whether to bother posting at all in this thread when I first read it.....as uninformed/incorrect comments like yours only makes me regret having posted at all.

Oh Marshal, no I did not take your comments personally, simply putting a viewpoint from a reviewers perspective.

There is a HUGE difference between a professional journalist (who reports facts), and the sort of writer who "reviews" things. Such writers are not journalists, even though they may belong to the Guild; rather, they are critics or reviewers -- and these each are two very different things.

Don't confuse "journalist" and "reporter".

zin1953 wrote:There is a HUGE difference between a professional journalist (who reports facts), and the sort of writer who "reviews" things.

Well, actually there is often little or no difference. Many critics also serve as reporters for their publications, and many full-time critics started as full-time reporters. To further confound your pigeon holes, some journalist/critics are also historians. In today's rapidly shrinking journalism world, you had better be prepared to wear more than one hat!

If you're in L.A. on the second Saturday of any month, come hang out with me at the Culinary Historians of Southern California, where you'll meet several of my critic/journalist/historian friends.

Randy G. wrote: I would not expect to go to a reseller's website and see a machine with a list of reasons why I should not buy it, and exactly why it is a bad machine.

It's not quite the same, but Sweet Maria's steps up to the edge of this when they describe limitations and caveats in their roaster descriptions. Their candor is something I'd like to see more frequently from vendors.

Marshall, you've made my point!

Marshall wrote: . . . many full-time critics started as full-time reporters.

Exactly They are no longer reporters! Furthermore . . .

Marshall wrote:To further confound your pigeon holes, some journalist/critics are also historians.

Yes, and when they are scholars-in-residence at (e.g.) the Hoover Institution or the Kennedy School of Government, they are no longer journalists, either!

Marshall wrote:In today's rapidly shrinking journalism world, you had better be prepared to wear more than one hat!

Nothing wrong with wearing more than one hat, Marshall, but you can't wear more than one at a time! This is why opinion columnists (almost) universally stop all reportage. At the very least, it clearly moves to the Editorial page, rather than the news section (for print), or you bounce back-and-forth but on two different networks (one example would be Keith Oberman on broadcast network NBC doing sports; Keith Oberman on cable network MSNBC doing his political "Countdown").

In the fields of film, art, music, etc., there is a clear distinction made between the role of a reviewer and that of a critic. That said, it's quite common for one person to wear both hats (albeit one at a time) -- for example, Roger Ebert writes movie reviews in the newspaper and for television, but he writes critiques for film journals. Reading examples of both, one would never confuse the two -- even though the public-at-large often uses the word "critic" and "reviewer" interchangeably.

But no one I know would ever call, for example, Robert Parker, a journalist. He is a wine reviewer (although some may [incorrectly, IMHO] refer to him as a critic), and an author. Similarly, I do not consider someone like Ken Davids a journalist; he is a reviewer.

That said, I think the semantic bull$#!+ has gone on long enough. Whatever you want to call the individual making the comments, I think we can both agree that he (or she) had better be qualified and consistent IF the individual does it professionally and expects the public to pay for his/her opinions.

Cheers,
Jason

So who would be someone that should be doing reviews of machines?

Obviously someone who does not make their living building, importing or selling machines. The reviewer should be someone in the coffee business and probably someone who has at least at some time been a pro barista or roaster. In my opinion this would be an unbias and skilled reviewer if they have some cupping skills.

shadowfax wrote:I have had the privilege of meeting and getting to know a few extremely skilled professional baristas and roasters in the past few months that I have lived in Houston; I can tell you that they are pretty biased guys with very varied opinions.

That's a fair comment. Many baristas get sentimental about the machines they "drive" every day, especially if they've learned how to turn the machine's quirks to their advantage.

Machine technicians are probably the best source of relatively impartial reliability information, But I don't think many of them put a high priority on espresso in the cup, if they even drink it at all. Besides which, they don't work much on consumer machines.

I used to work for a winery. I used to work for a major (chain) wine retailer. I used to work for a major (independent) wine retail. I used to work for a major (national) wine wholesaler. I used to work for a small, high-end importer-wholesaler.

It is impossible to have no bias whatsoever. We are, after all, still human beings (the last time I checked). Human being have opinions. The key is NOT finding someone without bias; the key is finding someone who has the knowledge and ability to give a fair and unbiased evaluation.

Hypothetical: I dislike Cabernet Sauvignon. (It's not true, but like I said, it's a hypothetical.) But as a professional wine judge and wine writer, I need to be able to taste, evaluate and write descriptive tasting notes on _____________ Cabernet Sauvignon, knowing enough about the type of wine to accurately describe the wine, its characteristics, its qualities and whether or not it is a good example of a Cabernet Sauvignon.

I now have an HX machine that is plumbed-in. I am a much happier camper now that I don't have to refill the reservoir daily. OK, I admit it: I prefer plumbed-in to pourover. Does that mean I will unfairly "ding" a pourover in a review? How can you criticize a machine for being something it's not? Does that make any sense? If you are reviewing equipment, you compare that piece of equipment to other pieces of equipment in the same class/category. That is, you evaluate a 1-group, pourover HX "prosumer" machine as to how it stacks up to other 1-group, pourover HX "prosumer" machines, not how it compares to an LM FB80. If you are evaluating a 1-group, plumbed-in "prosumer" HX model, your basis of comparison is not a GS/3 or even a La Spaziale Vivaldi II -- it's how it stacks up to other 1-group, plumbed-in "prosumer" HX models. Evaluating a 3-group professional machine? Yeah . . . compare it to other 3-group professional machines.

It doesn't matter that I generally prefer French red Burgundy to California Pinot Noir, or California Pinot Noir to California Valdigue. If I am called upon to judge Valdigue in a competition, or write an article about a winery that produces a Valdigue, that won't stop me from giving the wine a fair evaluation. Reviewing espresso machines should be no different.

Cheers,
Jason

I now have an HX machine that is plumbed-in. I am a much happier camper now that I don't have to refill the reservoir daily. OK, I admit it: I prefer plumbed-in to pourover. Does that mean I will unfairly "ding" a pourover in a review?

'Kay, I think what's being discussed here (and feel free to ignore me if I've interpreted this wrongly) is some more semantics than the OP intended. I think the comparison between reviewer and journalist was meant to insinuate that a decent reviewer should be operating with a journalist's integrity.
No one is without bias, and exposing your own in a review will help with that integrity. Yep, you will probably ding the pour-overs in your reviews, but showing that your preference is why you do that allows the consumer to replace any 'points' lost in the 'score' if they prefer a pourover or don't mind it. If you prefer a standard trans and ding a car for only coming in auto, those that prefer auto will ignore that ding, as long as you've explained that it's a preference. Explaining why pourovers aren't as handy, and that plumbed are a bit less, well, portable and renter-friendly, it allows the consumer to see your bias, and still make choices based on his needs.
Bias ain't a bad thing. Hidden bias is. A good journalist isn't one without bias. A good journalist is one that recognises and accepts his biases, and is willing to share them with his audience. A good reviewer is similar to that journalist in that respect.

Sorry, Chris. You're wrong.

Psyd wrote:Yep, you will probably ding the pour-overs in your reviews

No. I won't. Because it is a pourover.

To quote from erobertparker.com, it is "important for the reader to gauge a professional critic's overall qualitative placement of a wine vis-à-vis its peer group." As I said above, you review a machine for what it is, not for what it isn't. It is impossible to "ding" a 110v pourover, 1-group HX because it's not a plumbed-in, 230v, 3-group double-boiler. Different category of machine. The same is true whether you're talking about a Silvia, a Caravel, or a Speedster by Kees van des Westen.

The question that ANY professional reviewer will ask is NOT, for example, "How does this Pavoni Europiccola stack up against a 2-group Synesso Cyncra?" It's "How does does Europiccola stack up against other, similar lever machines designed for home use?" (Yeah, OK, you may be able to find someone with a 2-group Mirage Idrocompresso in their home, but clearly it was not designed with that purpose in mind. It IS a professional, commercial machine.)

How does the machine being reviewed stack up against other machines in its category? THAT is the question.

Psyd wrote:If you prefer a standard trans and ding a car for only coming in auto, those that prefer auto will ignore that ding . . .

So YOU'RE the one! I always wondered who took those Car & Driver reviews seriously. You know the ones I mean -- where they pit a Ferrari Testarosa's acceleration against that of an F/A-18 Hornet . . . certainly C&D doesn't; they're just having fun.

Chris, in your above example, the reviewer would certainly point out that the (e.g.) 2012 Porsche Pferd only comes with an automatic transmission. The reviewer may even say, "Now that VW has purchased Ford, they seem to be underestimating the US market by not providing a standard transmission in this German-designed successor to the Mustang . . . " but how can you "ding" the car for what it isn't? How can you say, Yeah, well, the automatic accelerates from 0-60 in 6.8 seconds, but if it had a manual, it would do 0-60 in only 6.4??? How do you know that? Where did that "6.4" come from? How can the reviewer "mark" down the Porsche Pferd for that?!?!?!?

You cannot, just as you can't fault a Cabernet Sauvignon for not being a Pinot Noir, or an HX for not being a DB.

Now then, if someone is not writing a professional review, but merely participating in a conversation on a bulletin board/discussion group, that is a different issue. One can readily say, Well, personally, I prefer a plumbed-in model, but ____________________ makes a pretty good pourover . . . But that's a comment, an observation, not a professional review.

Cheers,
Jason

You're not really sorry, and I'm not really wrong. But that's really great comment (or is it just an opinion? Maybe a review?!?)
rap! Now we're getting into the meaning of the word 'ding'.
If the reviewer suggests that the two being compared both lack a certain level of performance, they are both being 'dinged'. Comparing apples to apples is a technique. Comparing an unknown apple to a Fuji, a MacIntosh, and a Jona Gold gives the reader a comparison to a known quantity. Either a superior known, or an inferior known, or a roughly equivalent known will give a comparison that is usefull.
These aren't excluded from the 'Church of Critique' they are merely techniques. Some that you like, some that you don't, but suggesting that the only techniques that are acceptable are the ones that you prefer is simply untrue, and not particularly useful in this thread, or in any comparative or critical review.
And no, I did bike reviews. We chose the top of the line/most familiar to compare to, because it was something that most bike enthusiasts had a great idea about what it was like, and its reputation. It was our baseline, if you will.
Do we really want to keep going down this line of discussion? It's been fun, but it's about to get really boring really fast...

Chris, are you really saying you're going to criticize a Schwinn Stingray in a professional review for not being a Cannondale 21-speed racing bike? Does that make sense?

This isn't a "mine's bigger than yours" type of discussion; I agree, that would get boring rather quickly. I reviewed wines (and wineries) in various magazines, newspapers etc. for some two decades. You reviewed bikes -- I presume the reviews were also for publication -- so we both have professional experience. I'm just stunned that you would use the same criteria for a Stingray and a 21-speed racer . . . (or, in case you meant motorcycles, insert a Honda 50 and a Honda Gold Wing as an example of two very different bikes).

I have no idea how to do that -- how to make that kind of comparison. Even Consumer Reports will separate (e.g.) flat screen televisions by the size of the screen . . . I cannot see how one can criticize a Cabernet Sauvignon for not being a Chardonnay, or a Gaggia Baby for not being a 2-group LM GS/5.

Perhaps you can explain it to me someday over a glass a wine or an espresso if I ever get to Tuscon . . .

Cheers,
Jason

Chris, are you really saying you're going to criticize a Schwinn Stingray in a professional review for not being a Cannondale 21-speed racing bike? Does that make sense?

Yes, if you don't limit your definition of 'criticise' to the negative-only one. 'Criticise' is often used correctly to be defined as analysis, and comparison. The literary definition (which is the one that we were discussing) is most often interpreted this way. Having a much accepted and widely known pinnacle to compare to give a broader point of comparison. If you were to compare a red that you were reviewing to a great, but only pressed and kept in one tiny cellar in East Lower Bumfudgia, you'd be making a baseline for comparison that you might be the only one to be able to understand, and have failed in your critique of the wine.
A critic will expose his criteria for good (which he has to have, and which will always be considered a bias) and then describe the article that is being reviewed in the light of those criteria. Anything else is just snobbery and false elitism, or advertising. Seeing as no two people have the same palate, having an idea of what the reviewer sees as 'good' gives the person reading the review an idea of the benchmark used to compare. Comparison is probably the most useful tool in a critic's box.
Try using the literary definition of 'criticise' and you'll probably be more comfortable with my statements.

Perhaps you can explain it to me someday over a glass a wine or an espresso if I ever get to Tuscon . . .

I'll make the espresso, you're buying the wine.
Look for any Review of Arturo Fuente's 'Curly-Head' line. They're cheap cigars made by a reputable manufacturer, using the leftovers from the table where some of the finer cigars are made. The tobacco is remarkably good, but the skill level of those rolling is along the lines of an apprentice who is starting out. I happen to like these more than most any other cigar, because I happen to like a harder draw on my cigar, and most cigar smokers want an almost 'hollow-tube' draw, requiring no effort at all. It's considered a sign of a superior cigar.
While ninety-percent of those reviewing the Curly-Heads will 'ding' them for their draw, I'll know by that very 'ding' that that will be the cigar that I want. It doesn't mean that the cigar critic is wrong to compare these to the 'Shakespeare' or the 'Hemingway', though.

Another day at Home-barista.com

Psyd wrote:While ninety-percent of those reviewing the Curly-Heads will 'ding' them for their draw, I'll know by that very 'ding' that that will be the cigar that I want. It doesn't mean that the cigar critic is wrong to compare these to the 'Shakespeare' or the 'Hemingway', though.

Seriously flawed analogy. However, I agree with Marshall . . .

Marshall wrote:Another day at Home-barista.com

Cripes! And I originally just wanted to get the semantic argument out of the way so that we could get back to the issue at the heart of the thread. Ooops... 'Kay, This horse is officially dead. Back to the original discussion...