Blind tasting comparison of Niche Zero with EK43, and other tests
Last weekend, a couple of us HBers in Melbourne (Sam, Luca, John, Julian and Shauvik) got together to test the Niche Zero.
In the gathering, we have the following line-up:
(i) ACS Vesuvius as the espresso machine that does the shot pulling duty
(i) Niche Zero - the star of the day
(ii) Mazzer Kony (dosered - stock) - a few years old by now but has only been used domestically
(iii) EK43 with coffee burrs - Luca has spent quite a bit of hours in aligning the grinder using the marker method
We also have other grinders available : Comandante C40 MK3, Helor 101, Baratza Forte with steel burrs. But unfortunately we did not get to go through all of them due to time constraint. Photo: Calm before the storm...Part 1 - The not-so-useful refractometer test
One of the very first tests we did was to pull some espresso shots and measure the TDS, just because Luca has a VST refractometer and I was curious if it would give a different conclusion than what I got previously with Atago refractometer. Our main focus was not to get the refractometer readings, but rather to dial in the grinders and to get a feel for the machine and grinders combination.
As for the beans of choice for this exercise, Luca had some mixed 'garbage' sacrificial beans laying around which he thought didn't taste good and/or slightly older. To set the context, I tasted some of the shots and my, some of his 'garbage' beans were actually pretty drinkable for my standard! Photo: One of the shots pulled using Luca's 'garbage' beans. The roast was definitely on the lighter side but I thought some of the shots were pretty aromatic. By the way, the timer doesn't reflect the actual shot time - it kept going after the shot has been cut. Table: The results recorded in the session. Note that these shots were on the long side, as they were pulled to match the EK43. Longer shot typically yields higher extraction yield (EY). Outcome:
It did appear that the EK43 consistently extracted a little more than either the Kony or the Niche, but not by much. The Kony and the Niche Zero were almost indistinguishable in terms of EY%, which is not surprising given they have the same burr set.
Note that we initially got almost 23% EY from the Kony for its second shot of the day. Turned out the Kony retained quite a bit of older coffee (it's a stock grinder which means the anti-static and finger guard were still in place) and took about 3 doses to get the old coffee in the grinder to be completely exchanged.
Honestly, the result wasn't all that surprising and the observations mirrored exactly what I saw at home (using Atago refractometer). A few caveats though: (i) Due to me (the barista) being in a new environment/machine and that we weren't being extremely meticulous with the measurement, I would put the margin of error to be somewhere around a generous +/- 1% EY
(ii) the sample size was small and too limited to draw any useful conclusion (n=2 for each). We were more keen on the blind taste test to consider spending any more effort than this.
So this observation supports the idea TDS measurement is usually a pretty meaningless measurement by itself
unless something has gone extremely wrong e.g. massive under-extraction (which you could also tell by taste in most cases).
All that this has proven is that the Niche can extract coffee just as well as any other capable grinder, at least on paper. But does it? Part 2 - Espresso blind taste test - dark roast (Niche vs EK43)
Our plan was to do blind taste tests for both dark roast and light roast. We started with the dark roast first because we thought it's easier to get the E61 grouphead hotter for the light roast (just by flushing) but cooling down (for darker roast requiring lower temp) would be a much slower process.
We originally purchased 1 kg of commercial dark roast (not shown) for the dark roast tests. However, it turned out that the roast was darker than most of us prefer and the coffee was on the fresh side ( 5 days post roast). Lucky for us, John brought some of his roasts (which was on the medium-dark spectrum) to be used for the dark roast espresso comparison. Coffee for the blind taste test :
(Left) Lighter espresso roast from Genovese - Ethiopian Guji Hambella (Roaster's note: jasmine aroma, sweet flavours of melon, tangerine, black tea, and cherry).
(Right)The darker roast is a blend roasted by John on his North TJ-067 roaster (Sam's note : chocolatey but not ashy, good body, a classic comfort blend with hints of brightness)Methodology:
For the espresso comparison, we set the dose to 18g using a VST 18g basket, since that seems to be the 'standard' dose for many HBers. Our first task was to decide on the brew ratio to be used for the blind taste. So we pulled a couple of shots with the EK43 at a variety of brew ratio and extraction time, and then tasted the coffee. We all agreed that the shot tasted pretty good at 18g in/26g out (1:1.5 brew ratio, in about 35s). We then dialed in the Niche Zero to reach the same extraction parameters (18g in 26 g out, in about 35s). Photo: Something that has been observed over and over again : darker roasts do provide visually more attractive crema, with abundant tiger mottling/stripes and more vivid colour.
For the actual blind taste test, we pulled 4 shots consecutively, as quickly as I can into labelled cups (masking tape taped at the bottom). To normalize the temperature effect in influencing the taste evaluation, I pulled the shots in the order of -> Niche, EK, EK, Niche.
All four cups were shuffled until we couldn't tell which cup came from which grinder/order.
Our tasks were divided into two parts:
Level 1: Can we distinguish which two cups came from the same grinder (pairing test)?
Level 2: Can we guess which grinder the shots came from, and any taste preference? Photo: This is the format how we blind tasted the shots from two different grinders - four shots placed side-by-side and shuffled. Note that in this picture we were doing the long black tasting (too busy sipping espresso to be taking photos!)
The first part (level 1) 'seemed' fairly easy to us initially
(that is before the answers were revealed), since we had tasted quite a few of shots from each grinder prior to the actual blind test and we all thought the differences were obvious : a big flat grinder and a conical grinder, it can't get any more different than this. Identifying the grinder in a blind test would require a certain level of familiarity with both the grinders, so I wasn't expecting most of us to guess them correctly. Outcome:
To our utmost surprise, most of us weren't able to even get past the first level. Out of the five of us (1/5), only Luca managed to pair the cups correctly. Even though Luca got them right, he commented that he too was struggling to put forth his guesses confidently since the taste differences were so subtle (and perhaps that this is not his usual preferred roast level).
There were two issues that became apparent in this blind test - 26g of espresso isn't really enough for the five of us (duh!), and that the layering effect of espresso may be affecting our judgement. So we decided to stir the espresso in the subsequent tests. Part 3 - Long Black/Americano blind taste test - dark roast (Niche vs EK43) Methodology:
Since we ran into the shot volume issue described above, this time we decided to dilute the espresso into long blacks for the blind test. Also, we were almost running out of the dark roast coffee at this point. So to play it safe, we pulled two shots instead of four: one from the Niche, one from the EK43. Each shot was stirred and split equally into two cups (four half-espresso in total), which were diluted with equal amount of hot water.
All the four cups were again shuffled for blind tasting. Similar to the first blind taste test, we first try to pair the cups and then try to guess which grinder the shot came from. Outcome:
Perhaps the dilution was making the shot more distinguishable, or that we got really lucky, 4/5 of us managed to pair them correctly (maybe our palate is still working after all) AND 4/5 of us also guessed the grinder correctly. However, we all agreed the taste difference were so small, and getting them correct could've been by chance. Most of us had no strong preference of one over the other. Photo: To make sure what we were tasting was not due to the difference in TDS, we measured the coffee afterwards, and they came up to be fairly close : 1.38% and 1.41% TDS.
Winner: Tie Part 4 - Espresso blind taste test - light roast (Niche vs EK43) Methodology:
For the light roast espresso test, we used a commercially available light roast meant for espresso (not a super light roast as you would get for filters). The dialing-in method was the same as with the dark roast. First, we dial in by tasting the shots at different brew ratio using the EK43 . Once we got the taste/balance that we're happy with, we then match the flow rate using the Niche Zero. This time, the brew ratio we settled upon were close to 1:2 brew ratio (18g in and 36-38g out). As these are longer shots, shot volume was much less of an issue for the blind tests.
So again, four successive shots were pulled as quickly as possible, shuffled and blind tasted. Outcome:
This time, 2/5 of us got them right. Photo: Luca's 'custom-made' whiteboard for making notes during the blind tests.
Now, this is surprising to me. With a lighter roast level, I thought it would've been a piece of cake to pick up the difference and to pair the cups right. As a reference, all the five of us are definitely knee-deep in this coffee hobby, probably as much as most HBers out there (three of us were espresso drinkers and home roast, one was obsessed enough to take on Super Popcorn roaster project, and another one is a Q-grader/brewer cup champion, who regularly travels the world to taste/seek excellent coffee). I was totally expecting us to do better than 2/5, even if by fluke. Needless to say, we were quite humbled by the results. Food for thought:
Does that mean the Niche Zero pull shots that were indistinguishable to the EK43? I don't believe this is the case. A more likely explanation is that the magnitude of difference was really so small that it could be easily overshadowed by other factors. During the blind tests, the shots were stirred/cooled down and the crema had mostly dissipated (in a wipe-mouthed cup), so these might've reduced the difference in the taste. Most of us don't have regular experience in blind cupping. So maybe we could improve with more practice, maybe not...
However, this puts forth a very compelling case : it is likely most average home baristas won't be able to tell the difference either, without being trained and spending a lot of time in learning the minute differences.
More importantly, when we were dialing in and optimizing the extraction, we could clearly taste the coffee going from almost undrinkable (not dialed in correctly) to excellent (after dialing in). So, our consensus was - if you want to achieve excellent coffee/espresso, you may be better off by improving your techniques, rather than putting the blame on the machinery. The dialing in process showcased a lot more taste-able differences in the shots than when different equipment was used. So there is no point in over-obsessing about equipment, provided you have a decent grinder, which the Niche is undoubtedly one.
Lastly, I was quite surprised by the potential effect of mindset (either subconscious or preconceived notions) on our taste perception. When we were tasting the shots non-blinded, we all thought we could tell the difference easily. Yet, for some reasons, we struggled to different extents during the blind tests. This has been a humbling and eye-revealing exercise and I highly recommend you to do the same if given the opportunity. Part 5 - Brew tests (Niche vs EK43 vs Comandante)
At this point, our palates were pretty much shot and were fatigued from the previous tastings. Julian & Shauvik also had to leave early so all we could muster next was a non-blind side-by-side brewed coffee tasting in cupping bowls. We used three different grind settings from the Niche, two from the Comandante and one from EK43. The grind size was just a touch finer than for drip coffee. Outcome:
One conclusion that was apparent to us : if we are serious about improving the coffee quality, we would be much better off spending money on seeking better coffee that we like and optimizing the extraction parameters, rather than putting that $1-3k price difference into equipment/gadgets. All of the grinders produced a similar level of coffee quality (none stood out as particularly bad or good, other than one cup that was ground too finely and over-extracted).
Winner : Again, none Photo: We used cupping 'bowls' with slightly finer than usual grind for the brewed coffee comparison. Some final conclusions:
- The Niche did unbelievably well in this session and held its fort against a vastly different, more expensive grinder (the dear of the current coffee trend).
- The Niche was really easy to dial in, and the minimal retention makes dialing in a cinch. Even when I was in a completely foreign environment with an unfamiliar machine, I was able to dial in the Niche to the target brew ratio within 3 attempts (often 1-2).
- The holy grail in espresso is arguably to achieve consistency. In this regards, without a doubt, the extractions from the Niche Zero grinder were consistent enough throughout the session. When pulling the consecutive shots for blind tests, I was fully expecting having to re-pull some of the shots (in case the extraction time was different/off), but that was not the case at all. For reference, the workflow I've used was to grind into the provided cup, pour the grind into the basket using a funnel, WDT and tamp.
- Last but not least, for average (or even advanced-level) home baristas, I do see the Niche Zero as being a more than adequate grinder : with its easy work flow for single dosing, approachable price-tag, consistency in performance and excellence in taste quality. If the manufacturer is able to keep up the quality and the Niche can maintain its performance reliably for at least 5 years, I don't think it's a stretch to call the Niche Zero a game-changer for home barista/consumers.
This gathering wouldn't have been possible without the people behind it: From left to right: Shauvik, Luca, Julian, JohnThe barista (Sam) gets special treatment with a photo all to himself! Well almost, that's Luca behind attempting to steal the limelight ;P (thanks Shauvik for the photos)Credits: Thanks again to Luca for hosting us, to Shauvik for some of the photos, and to everyone (including John and Julian) for volunteering your time in contributing to this enjoyable afternoon!