Summary:

Sherman Chong and Jim Schulman broke off an experiment to confirm a consistent difference in the taste of paired shots, one ground with a hopper, the other single dose, when it became clear that there was no discernible taste difference, consistent or otherwise, between them.



Experimental Protocol:

We tested a Compak WBC using a blend of Brazilian, Guatemalan, and Kenyan coffees at various roast levels, and a Mazzer Super Jolly with a Vienna Roasted Sulawesi. The grinders were set to produce the same 1.5 ounce volume, in the same 30 second shot time, at the same 15.5 gram dose, in the same Faema style baskets. The grind setting was adjusted to achieve this for both grinders alternatively equipped with small, full hoppers, or with 16 gram single doses in the throat, weighed down with a tamper.

About 15 to 20 grams of purge coffee was ground between grinder changeovers. Grinders were changed over every two shots, In other words, the first round was hopper grinding first, then single dose, the second round, single dose first, then hopper grinding, etc.

The baskets were unobtrusively marked. The person grinding flipped a coin to determine which basket went with which grind option. Then the person pulling the shots shuffled the baskets. So by the time the shots were pulled, neither person knew which coffee was which. Both testers tasted each round.

The shots were pulled on a pair of Elektra Semi-Automaticas at the same time. The shots were preceeeded first by a cooling flush, then a single pulled a minute earlier, and then a cleaning flush, in order to settle the machines into their optimum performance.

There is a slight difference in the pump performance of the two Elektras, so that the shots layer differently, and the crema alone on one machine tastes stronger than on the other. The difference completely disappears once the shots are swirled and rested for 15 seconds. This is what we did when tasting.

Once each grinder was properly calibrated, the plan was to pull four rounds of shot pairs per grinder. The first two rounds were to familiarize oneself to with taste differences. The final two rounds were to be used to attempt the correct identification of the coffees. A perfect record by one taster for both grinders, or on one grinder by both tasters, or seven out of eight correct guesses pooled over both tasters and grinders would have been a significant result. Six out of eight pooled would have been somewhat suggestive; but not a significant result.



Outcome

In the first two rounds of both grinders, neither Sherman Chong nor Jim Schulman could in any way distinguish the shots. "They could have come from the same cup," Sherman remarked. Since we could not taste a difference, there was no point in going on to test whether the differences we had tasted were real. We abandoned the tests on both grinders after two rounds a piece.



Discussion

As usual, the best information came from the surroundings, not the test itself.

It was extraordinarily difficult to dial in the grinders, especially the SJ. It took me three tries to dial in on the Compak, and it took Sherman five on the SJ. When the grind settings drifted apart, getting them synchronized again was not to be done with linearly proportional adjustments. On the Compak, I noticed no difference in the degree of finickiness in adjusting hopper and single shot grinding, but I did notice they were different. Sherman had slightly harder time setting up the single dose.

I speculated previously that single dose grinding may turn out to be more finicky. I don't think the set up problems we had setting up bore this out in any convincing way. What they did bear out is this:

• Keeping the grind dialed in correctly works differently when doing single dose and when doing hopper based shots.

People who perceived differences in the grind-modes' taste outcomes were probably making a different sequence of compensating adjustments over the days they were testing, with one sequence more successful than the other. I have no doubt that if I were to do such a sequence of shots with several days on one mode and then the other, I would end up with significantly better shots in one than the other too. My point is that it will be impossible to replicate such results, since they will depend on the particular coffees, the dosing, the weather, the staling, and umpteen other purely local variables.

Interesting results, Jim.

Thanks to you and Sherman for testing this and reporting back.

I have a couple of questions: if I understand correctly you weren't using 2 x SJ and 2 x WBC but grinding the shots in both modes in advance (with a purge shot in between) and then doing a blind tasting. Is that correct?

And the dose was always 16g, or was it sometimes 15.5g?

I'm just trying to get my head around the protocol.

People who perceived differences in the grind-modes' taste outcomes were probably making a different sequence of compensating adjustments over the days they were testing, with one sequence more successful than the other. I have no doubt that if I were to do such a sequence of shots with several days on one mode and then the other, I would end up with significantly better shots in one than the other too. My point is that it will be impossible to replicate such results, since they will depend on the particular coffees, the dosing, the weather, the staling, and umpteen other purely local variables.

I think the latter is a very good point, and it is one of the reasons why I am somewhat cautious about claiming any definitive results from any single set of tests. In my own case regarding the former and recounted ad nauseam elsewhere (so I'll spare everyone a third time ) I found my impression of the "in between shots" was confirmed very strongly by the changeover shots (although expecting the reverse to happen). The differences I perceived were less marked on the SJ than the Nino. It could, however, of course be that the latter is such a through-and-through dedicated Grind-On-Demand design that it is unsuited to these tests.

In particular, Jon and Nicholas have suggested a quick purge shot might be a good idea when using the Nino in single-shot mode, since it is likely (and my "black hole" findings support this) that there is some grind retention in single-shot mode on the Nino and this is likely to have skewed the results in favour of hopper grinding on that grinder, since I generally purged a single if the grinder had been sitting for a while.

As observed elsewhere, I was also dosing lower and found the differences more marked for classic "Italian" singles. I touch on this not simply to comment on differences in the testing, but equally to criticise my own "protocol" (such as I had one), since any stale grinds would have amounted to a larger percent of the total.

I cannot be certain why with the SJ in hopper grinding I perceived nuances in beans that I had noticed on the Nino in the past but never on the SJ when single-shot grinding etc., but I could not definitively rule out the weather or the bean batches etc. I have just been using the SJ for decaf since the second Nino has arrived, so I cannot comment further on this either.

Happy New Year!

Mike

Thanks to Jim and Sherman, for performing the test. I know it's not easy to do!
I will watch with interest as others debate, but have nothing to add at this time

michaelbenis wrote:And the dose was always 16g, or was it sometimes 15.5g?

I'm just trying to get my head around the protocol.

I had to re-read carefully to clarify for myself.

There was only one of each grinder. The grinder was used to prepare two baskets, one from single dose grinding and the other from a loaded small hopper. The dose in the basket was always 15.5g but they did load a small extra amount into the throat of the grinder (16g total) for the single dose grinding

First, and most important of all, thank you for taking the time to plan, perform and report on this test.

The test procedure and controls seems to be very thorough and adequate. There is no doubt, what so ever, in my mind that both tasters are gifted, qualified, knowledgeable and capable.

another_jim wrote:Sherman Chong and Jim Schulman broke off an experiment to confirm a consistent difference in the taste of paired shots, one ground with a hopper, the other single dose, when it became clear that there was no discernible taste difference, consistent or otherwise, between them.

With all due respect, and I have the up most respect for Jim, the fact that Jim Schulman does not taste something does not mean that there is no discernible difference!

There are a few members, for which I also have the up most respect, that posted that they perceived a taste difference. Their situation may be different, as you acknowledge yourself, than the test conditions described above.

another_jim wrote:People who perceived differences in the grind-modes' taste outcomes were probably making a different sequence of compensating adjustments over the days they were testing, with one sequence more successful than the other. I have no doubt that if I were to do such a sequence of shots with several days on one mode and then the other, I would end up with significantly better shots in one than the other too. My point is that it will be impossible to replicate such results, since they will depend on the particular coffees, the dosing, the weather, the staling, and umpteen other purely local variables.

Although your hypothesis here may be fully correct, there is no definite way to associate your results and assumptions to the other member's results. But whatever the reasons for the different results, we should allow for the possibility that something may be there but we cannot just see it yet. The fact that, as you pointed out yourself, it is unlikely to replicate the tests conditions implies that one's results does not invalidate the other's results.

Please Jim* do not take this post the wrong way. I am not disputing your results. I am not disputing (from the peanut gallery) your opinions or your capabilities. I am just trying to keep the door open (in my mind at the very least). You have such a high status here (and rightly so) that your posting such a definite result may be taken as a definite reality by some.

All in all, it is not the results posted that I am discussing here but the way they are posted. What I am getting from your's and other's posts on the subject so far, is that the "one dose grinding versus the hopper full grinding" effect is definitely not clear cut, for whatever reason, just yet.

Final disclaimer: I am all tangled up into a single grinder, 3 different beans and some serious issues of results consistencies (probably because of grinder dialling-in and lack of Barista skills issues) that I cannot conclude to a taste opinion on the aforementioned subject just yet. In fact I have already come to the two opposite results probably again for lack of thoroughness in the experimenting.

Thank you again.

Happy, cheerful and tasteful New Year to all.

*or anybody else that may take offence at this post.

Jim and Sherman, thanks for taking the time to perform this experiment.

Whale wrote:The test procedure and controls seems to be very thorough and adequate. There is no doubt, what so ever, in my mind that both tasters are gifted, qualified, knowledgeable and capable.

<snip snip snip>

In fact I have already come to the two opposite results probably again for lack of thoroughness in the experimenting.

You may want to try the "solo tester" strategy I used during the Titan Grinder Project testing; it demands less equipment than Jim/Sherman used (excerpted below). I placed the baskets on a lazy Susan opposite each other, closed my eyes, and spun it a few times. Then I asked one of my kids to place one of the baskets in the portafilter and lock it into the group. I then scored the espresso for flavor profile, texture, and appearance using SCAA barista competition standards. To avoid palate fatigue, I would do no more than 4-6 rounds per session.

another_jim wrote:The baskets were unobtrusively marked. The person grinding flipped a coin to determine which basket went with which grind option. Then the person pulling the shots shuffled the baskets. So by the time the shots were pulled, neither person knew which coffee was which. Both testers tasted each round.

The shots were pulled on a pair of Elektra Semi-Automaticas at the same time.

HB wrote:You may want to try the "solo tester" strategy I used during the Titan Grinder Project testing; it demands less equipment than Jim/Sherman used (excerpted below). I placed the baskets on a lazy Susan opposite each other, closed my eyes, and spun it a couple times. Then I asked one of my kids to place one of the baskets in the portafilter and lock it into the group. I then scored the espresso for flavor profile, texture, and appearance using SCAA barista competition standards. To avoid palate fatigue, I would do no more than 4-6 rounds per session.

Thank you for the suggestion. Sounds reasonable but as I wrote I am all tangled up right now and cannot even repeat two shots in a row... I am getting frustrated and impatient with my results.

I have decided to go back to the basics. Single coffee, WDT, weighing, calibrated espro tamping, timing, bottomless P/F and measured brew ratio. When I learn to pull shots again (if I can manage that!) I'll try doing taste tests again!

Whale wrote:Although your hypothesis here may be fully correct, there is no definite way to associate your results and assumptions to the other member's results.

As a philosophical point: This is always the case when the boundary conditions of the test change, and it's a black box test, so that the effect of boundary conditions is unknown. Generalizable results are a product of successful predictive theories, not more data. We're not even close here.

As a practical point, if you compare using a grinder for several days in one mode and then the other, you are not directly comparing the grinds, you are directly comparing the process, that of making espresso with or without a hopper. In such a case, the changes in particle distribution are not the only ones, you also deal with how often the grind changes, how easy it is to compensate, and differences in the way the basket packs. The result of such a test will definitely tell you what is best for you, but it'll only be a cautionary story for others, not a command.

As far as my tasting ability; it is just average by coffee pro standards; Sherman has more raw tasting acumen. The ability to discern small differences is a product of both raw acumen and experience; I'm very good at triangle tests among African coffees, and not so good with Centrals, due to the difference in the frequency with which I drink from these regions. If you do an equipment taste test, it's best to use a style of coffee with which you are very familiar.

In sum, the tasting convinced me that for any shot you can produce with a hopper, you can produce the same tasting shot without one, or vice versa, providing you take enough pains. But the difficulties of setting up convinced me that the process for staying consistent with a hopper and without one are different enough so that one or the other can be a lot harder for changing combinations of coffee, machines, and grinders.

I just wanted to say that I appreciate all the work that Jim and many others have put into this topic over the past weeks.

As a relative newbie who does measured single doses to aid the learning process, but is always wondering about the hopper alternative, this has been an important topic for me.

For all the active participants in the debate I'm sure there are many others like myself carefully taking notes on the sidelines. It's why I enjoy this site so much. I have had the urge to comment at various times, but know I don't really have anything useful to say except "thanks".

Jim, I am confused about your method used for "single dose grinding"

First, from your previous topic: Single dose versus hopper grinding: hypothesis about the exact difference.

The argument for a wider distribution is a little more flimsy than the one for finer grinds with a hopper. There's two points:

* With single dosing, all the beans are relatively unobstructed, while with hopper grinding, some of the beans are unobstructed, and some are obstructed. Hence with hopper grinding, the bean face more varied circumstances as they go through the burrs. This creates a wider distribution.
* A wider distribution around the same mean will get slower flow due to better packing. So the hopper grind is likely to be both finer and more widely dispersed

I assumed that you were talking about single dosing with "no external weight" to maintain pressure against the beans...such as Jon stating he would "close the sliding door in the hopper" to keep them from flying out...so in essence no mass to pack the beans into the burrs.

Your testing method here uses a tamper?

The grind setting was adjusted to achieve this for both grinders alternatively equipped with small, full hoppers, or with 16 gram single doses in the throat, weighed down with a tamper.

So, how does this single dosing allow the beans to "move about unobstructed"? Am I missing something?
The outcome is rather what I would have expected with the test conditions.

danetrainer wrote:Am I missing something? The outcome is rather what I would have expected with the test conditions.

The question was whether single dose grinding differs from hopper (multiple dose grinder). Most single dose grind aficionados use a tamper, especially Mazzer owners, or equivalent weight on the bean column if it's not tamper friendly:


From What does single dosing lose?

For what it's worth, my guess is the importance of weighted versus unweighted for single dose grinding will diminish for slow RPM grinders like the Compak K10 versus speed demons like the Mazzer Robur.