Jack,

Yes, agreed. The espresso shot tasting I did was too subjective, I need to do it again this time blind tested.

I actually did not write up anything about comparing the 3 grinders for several weeks, but I had been asked in PM's by multiple people here to please do it, I was against doing it as I was not sure I could do a fair comparison, I have never done such work before. I caved in and gave it my best shot, spending several hours trying to compare them as both espresso and drip grinders.

Before I did the review I asked a "Team H-B" member for advice on how to do the review, and was told to do the blind tests just as you suggested, that is the reason I ordered a matched set of new HQ baskets for the Strega. Interesting that he also did warn me that no matter what results I got, I should be prepared for the testing and results to be challenged, so he did nail that part, and it is my bad for not listening to him and testing blind. At the time I was doing the initial tests I did not have anyone to help with the testing, so I just did some more informal tasting while I tried to do lots of photos and such. Next time I will just taste, no lights and cameras.

I will get back to doing the Rosco v. Pharos blind taste tests, it is a great idea. I will post it here once I do it.

Jeremy, there is no need to be hard on yourself. You've started a cool thread and most people have really enjoyed your input. It's clear that results are going to be subjective to some extent, but it's still a valuable contribution.

I don't think anyone expects HB members to be running a scientific lab and if they do maybe they are looking in the wrong place.

I also am wary of the notion that differences between A and B are usually just imagined. In that case might as well just drink swill. No need prove your opinions by calling in a panel of tasters to do a blind tasting. That's nice when it happens and in general blind tasting is a great idea. It does show flaws in our perceptions and help to justify what we say but it's not a requirement for posting your thoughts on HB. You didn't make any outrageous claims and therefore don't require such stringent evidence to support your opinions.

I for one don't think you need to apologize or go on the defensive for taking the time to explore different equipment and posting your personal results.

I don't mean to be harsh on anyone - I just want to see objective results. We often have threads on HB such as "A vs. B", etc, which implies comparisons in taste, functionality, etc. People reading these forums are often in the process of deciding how much money to spend, and what to spend it on. It would be useful to them to see objective taste tests. I have yet to see the results of the simple test I describe, where A is correctly identified 90-95% of the time in a blind taste test (of course I may have missed a few posts).

I know, for example, that I probably have a poorly-developed espresso-tasting palate. I can tell a) tea from b) espresso, and a) properly-ground and prepared espresso from b) very poorly-ground, poorly extracted espresso.

But I don't know if I can tell Pharos from Rosco espresso (assuming both properly prepared). What I would like to know before spending $2-300 to upgrade is that at least some people can reliably distinguish between them.

When I did my one and only comparision it was between a La Pavoni and a Ponte Vecchio Export. I soon found that the within-machine variation in flavor was greater than the between-machine variation, so it was very difficult for me to say I preferred the flavor of one to the other.

peacecup wrote:If the taster can correctly identify the grinder 9 out of 10 times one would be approaching scientific rigor (i.e. 90% confidence).

I'm not a statistics guru, but according to Jim's writeup A note on comparison tests, it would only take 5 trials to statistically demonstrate an obvious difference. For minor but nonetheless real differences, it would take many more trials than 10 to statistically prove the difference exists with a confidence level of 95%. Jim sums it up nicely:

another_jim wrote:As far as coffee tests are concerned, my guess is that people are hardly interested in making a change if it turns out not to beat their current set up at least 6/10 times. Unfortunately, even that requires more trials (75 to 200) than any of us amateur testers can easily handle. So, if we discover no difference, it's somewhere from 30/70 to 70/30, and if we do discover a difference, it's because its more extreme than that.

Agreed, doing a large number of trials is not easy. And being able to identify one grinder's coffee 90 out of 100 times would be a remarkable achievement, especially with more than two variables. The confidence interval would be the probability that the result lies within a certain range, not the percentage of correct guesses.

Personally I would be happy to pick a Rosco from a Pharos or a Nemox Lux from a Robur 7 out of 10 times. That would be meaningful to me. If I ran that experiment a number of times, a 90% confidence interval might be guessing correctly 6-8 times, for example. 95% confidence could be 5-9 correct guesses. And I could have 100% confidence in getting 0-10 correct guesses for any given trial.

Still I certainly look forward to any blind trials that Jeremy does - it would be very interesting to see if the initial opinion does hold up to scrutiny!