Behmor 1600 coffee roaster - group taste test

Discuss roast levels and profiles for espresso, equipment for roasting coffee.
User avatar
HB
Admin
Posts: 21983
Joined: 19 years ago

#1: Post by HB »

Friday before last, a group of the regulars at Counter Culture Coffee's espresso lab convened to cup samples of the same coffee roasted using the five Behmor 1600 coffee roaster profiles. The green coffee was provided by Counter Culture Coffee, specifically their Harfusa Yirgacheffe. Mike roasted the five 8 ounce samples earlier in the week and delivered them in plastic bags. Nathan at Counter Culture provided a sample of their roast of the same coffee rested for the same number of days.

Test Strategy

Given the limited time for our panel, we chose a very simple single elimination strategy, what we called "pick the loser." Each round the tasters would pick the sample that would be eliminated; the rest would advance to the next round. That is:
  • Round 1, Flight 1: coffees 1, 2, and 3
    Round 1, Flight 2: coffees 4, 5, and 6

    Round 2, Flight 1: two coffees from Round 1, Flight 1
    Round 2, Flight 2: two coffees from Round 1, Flight 2

    Final round: winner of Round 2, Flight 1 versus winner of Round 2, Flight 2.
Mike was not present and has not revealed the roast profiles of the Behmor samples. To randomize the samples, I transferred the five Behmor samples and the one Counter Culture Coffee sample to identical bags, discreetly marked on the bottom with their original letter designation (Mike used A..E). I then did the shell game mixup of the bags and prominently relabelled the front of each bag 1..6. Thus nobody knew which bag contained which sample.

For each sample, we prepared three cups and cupped them using Counter Culture's standard form (fragrance, aroma, break, brightness, flavor, body, aftertaste). In addition to taste notes, each taster selected a "loser" for each round.

Results:

Round 1, Flight 1: coffees 1, 2, and 3

We expected this taste test to be difficult, but it proved otherwise. From samples 1, 2, and 3, only 2 advanced to the next round by unanimous vote. My own notes for the losers included damning terms like earthy, flat, bitter, wet hay... not a good start.

Round 1, Flight 2: coffees 4, 5, and 6

While not as bad as the other flight, it was nearly unanimous that only sample 5 should advance. Sample 4 garnered one vote, so the others grudgingly agreed to advance it to the next round.

Round 2, Flight 1: coffee 2

No need to repeat since its competitors were eliminated by clear majority vote.

Round 2, Flight 2: coffees 4 and 5

Since there was only three competitors remaining and time was running out, we decided to advance both of these without another taste test.

Final round: coffees 2, 4, and 5

At last a close call! Originally there were six tasters, but as we were running late, one of the attendees for the public Friday cupping arrived. Thus the seven tasters sampled the coffees and there was a near split decision for the winner:
  • 3 votes for coffee 2
    4 votes for coffee 5
    0 votes for coffee 4
I believe our hosts felt a short moment of trepidation as we checked if their roasted coffee was among the finalists. With a sigh of relief, indeed coffee 2 was their sample. We only had a few minutes to discuss the rationale behind our votes. I was among the tasters that picked the Behmor sample #5 over Counter Culture's sample #2. It was a lighter roast and I thought it captured the varietal characteristics of a Yirgacheffe. Others preferred the higher body and superior balance of Counter Culture's rendition.

Closing Comments:

I expected it would be difficult to discern the different roasts, but it turned out to be easy: Of the five samples from the Behmor, only two were drinkable. I will defer to Mike for an explanation of how the samples were roasted.
Dan Kehn

lparsons21
Posts: 124
Joined: 17 years ago

#2: Post by lparsons21 »

Interesting results. Thanks for doing this. I for one, can hardly wait to see what the profiles of the Behmor roasts that were not good.
Lloyd

mike
Posts: 54
Joined: 19 years ago

#3: Post by mike »

Sorry, my doggone post got eaten last night, and I had to go to bed. Sick as a dog here.

SO....... here is the scoop. I'll pull out my notes later, but the critical mapping for you guys to understand is that the letter labels were also randomized, and didn't sequentially indicate the profile.

Coffee 5 was by coincidence P5, and coffee 4 was P1.

If all other things were equal, before the experiment I would have expected some sort of trend of preference based on profile shape. Seeing the two most extreme profiles in the #1 and #3 positions suggests that the profile on 8oz batches is NOT a significant factor.

I had to keep my mouth shut after the roast phase, because the times to 1c on all the profiles really didn't vary by much more than 2 minutes, and every roast was in a reasonable range. Also, it was impossible to get each of the samples to the exact same roast level because the Behmor has such a long coast on the cool and there is no way to probe the bean mass for temperature. So, variations in final roast level almost surely drowned out any subtle differences between the profiles.

As I explained to Dan though, the purpose of the experiment was to see if we could reproduce any of the really poor results, especially baked beans, correlated to selection of profile. And, at least in this experiment, I don't believe that profile did turn out to be significant. I do think there is some theoretical basis to say that P2 and P1 are probably the better profiles on the machine.

With this experiment behind me and re-reading a lot of the posts here and at CG, I'm becoming more and more convinced that the much more significant factor is charge load size. Which to some people is going to be obvious because 21 minutes to 1c for a 1lb load is NOT in a reasonable range. And under those conditions, profile might be a factor of some magnitude, but the charge load remains the critical one.

The next experiment will obviously be to do loads of 4oz, 8oz and 16oz under the same profile, and blind taste them. If we can reproduce bad roasts, and they correlate to increased batch size, then we have it and it simply becomes an understood limitation of the Behmor.

The other limitation that became really glaring during this test is the difficulty in reproducing roast levels or hitting a specific one due to the coast and limited ability to observe the bean mass temps/etc. In using the Behmor you've got to be a little bit tolerant of not being able to exactly hit your desired finish, or be willing to do at least one experimental load into 2c so that you can note the time periods and approximate when you want to finish your real load. And for $300, I can live with that.

I'll probably think of more later, but that's the critical stuff I think guys.

User avatar
JonR10
Posts: 876
Joined: 19 years ago

#4: Post by JonR10 »

mike wrote: Coffee 5 was by coincidence P5, and coffee 4 was P1.
This is the shocker for me :shock:

But that's why they play the game, right?


Thanks to Mike for organizing, et al. for participating.....
A very enlightening experiment!

-Jon

User avatar
HB (original poster)
Admin
Posts: 21983
Joined: 19 years ago

#5: Post by HB (original poster) »

mike wrote:Coffee 5 was by coincidence P5, and coffee 4 was P1.
What surprised me is how NOT "plug and play" easy these home roasters must be. The cupping quality of the Behmor roasted coffee was all over the map -- three out of five were unacceptable. On the other hand, it was only one test and there could be operator error in play. Now that we have the format down, we can run another test any Friday that Mike feels like it.
Dan Kehn

mike
Posts: 54
Joined: 19 years ago

#6: Post by mike replying to HB »

Yep, and by operator error you want to include my difficulties with getting the roast levels consistent, feel free. Some of those roasts came out really light.

One of the original premises was that I wanted to keep the final roast level to City+ so that roast would not obscure any subtleties. The side effect was that I lost the "marker" of hearing the beginning of second crack you have with darker roast targets, and as a result trying to hit City+ across 5 different profiles was a crapshoot at best.

Again though, if there was any trend relative to profile type, I'd worry more, but it was pretty clear to me that at 8oz loads profile type was not a highly significant factor.

I'm open to suggestions on how to do the charge load experiment.

As much as I'd like to do the 3 charge loads under both P1 and P5, that is a lot of time and a lot of coffee to waste. I'm leaning toward P1 since there seems to be less controversy over that profile than the others, and experts like Tom at SM also back using that profile. I think we also need to adopt a darker roast level so that we have the marker of second crack and more consistency in finish roast level. I'm thinking I'll stop the roaster on the first few cracks of second. That should eliminate some of the unwanted variables. I also want to get Agtron readings (exterior and spread) on them to see any correlation between those, roast length, and taste/preference.

Anything else I missed guys?

lparsons21
Posts: 124
Joined: 17 years ago

#7: Post by lparsons21 »

HB wrote:I was among the tasters that picked the Behmor sample #5 over Counter Culture's sample #2. It was a lighter roast and I thought it captured the varietal characteristics of a Yirgacheffe. Others preferred the higher body and superior balance of Counter Culture's rendition.
This is something I observed in my early test roasts. A Sidamo roast that ended up lighter than planned turned out to be one of the best Sidamo roasts I had ever had. All the nuance was there. Usually the berry note doesn't develop for a few days after the roast and then dies off in a very few days. With the one done on the Behmor, the note was fully there the next day and lasted about 6 days.
Lloyd

User avatar
RapidCoffee
Team HB
Posts: 4995
Joined: 18 years ago

#8: Post by RapidCoffee »

HB wrote:What surprised me is how NOT "plug and play" easy these home roasters must be. The cupping quality of the Behmor roasted coffee was all over the map -- three out of five were unacceptable.
Two really big surprises for me:

:shock: #1: The bar is so low. Three of five unacceptable? If I had to toss 60% of my home roasts, quite frankly, I'd quit home roasting in a New York minute.

:shock: #2: The bar is so high. One of the Behmor roasts was judged superior to a Counter Culture roast. If 20% of my home roasts were on par with CCC, one of the best commercial specialty coffee roasters in the biz, I'd be in home roaster heaven!

I do not have a Behmor, but I've been roasting with a small electric drum roaster (AeroRost plus variac) for the past three years. These results confirm my experience: it's possible for the home roaster to achieve true excellence using low end equipment, but it takes practice and a bit of luck. Drum roasters in particular make it hard to monitor the roast visually (or by temperature probe), so consistency will always be an issue. And no matter what roaster you use, home roasting will never be a "set it and forget it" activity.

Thanks to all involved for taking the time and effort to run this test. I look forward to more experimentation with this new home roaster.
John

User avatar
woodchuck
Posts: 311
Joined: 18 years ago

#9: Post by woodchuck »

John, I don't think it was a question of the bar being too low or too high. We selected the roasts we liked the least and moved the rest to the next round. We could have used the same model on a number of commercial roasts and in the end we would still have a loser each round. In terms of besting a commercial roaster. I think some people just liked the taste of the lighter roast better. Personally I enjoyed the darker roast from CCC. I think commerical roasters differentiate themselves from the home roasters in repeatability as much as taste. I know I enjoy CCC's coffees one because they taste good and two because I always know what I'm getting.

Cheers

Ian

User avatar
RapidCoffee
Team HB
Posts: 4995
Joined: 18 years ago

#10: Post by RapidCoffee »

woodchuck wrote:John, I don't think it was a question of the bar being too low or too high. We selected the roasts we liked the least and moved the rest to the next round. We could have used the same model on a number of commercial roasts and in the end we would still have a loser each round.
I'm just responding to Dan's descriptions of the Behmor roasts: "earthy, flat, bitter, wet hay", "of the five samples from the Behmor, only two were drinkable", "three out of five were unacceptable". That gives a much stronger (negative) impression than your relatively mild comment.

So... what's it gonna be, guys? 3 of 5 were unacceptable, or all were OK but you had to pick a loser each round?
John

Post Reply