How much does roasting affect cup score?

Discuss roast levels and profiles for espresso, equipment for roasting coffee.
baldheadracing
Team HB

#1: Post by baldheadracing »

I've always had the opinion that I should buy the best greens that I can, as my lack of roasting skills means that my roasting can only reduce cup quality.

Recently Scott Rao did a blog post on score inflation, and he did have a couple paragraphs of interest about roasting for cupping:
How roast quality affects score

When Ryan Brown and I ran Facsimile, a cupping-oriented subscription service, I occasionally sent multiple sample roasts of the same coffee to Ryan. Ryan preferred that all samples he received were not marked in any identifiable way (Not even country of origin). Every sample roast we cupped was well within the range of "typical" third-wave roast quality, ie nothing was grossly flicked or underdeveloped.

Our scores for a given coffee roasted a variety of ways would land in a 1.5-point range. Since then I have assumed semi-competent roasting could influence a cup score by up to 1.5 points. One way to look at that would be a "perfect" roast would capture the full potential of a coffee, while a flawed-but-not-awful roast would cause a deduction of up to 1.5 points.
- from https://www.scottrao.com/blog/2022/9/26 ... o-find-88s

FWIW, I did sign up for the Facsimile roasted+green subscription, and was happy that I was unable to distinguish between cups from the Facsimile roasts and cups from my iRoast2 (using a profile that Tom @ Sweet Maria's made to approximate his Probat sample roaster. All I had to do was match Facsimile's roast colour.) This experience begs the question, "Is a cupping roast the way to capture the full potential of a coffee?"

(I typically roast a little longer/darker than a cupping roast as I usually roast for espresso.)
-"Good quality brings happiness as you use it" - Nobuho Miya, Kamasada

Milligan

#2: Post by Milligan »

What is interesting is that a mediocre roaster that is good enough to not cause roast defects but not good enough to highlight the coffee in its best light isn't completely hamstringing a coffee. A 1.5 point difference is nothing to sneeze at (especially at the upper end) but it isn't a staggering change. An 83 "good" coffee and an 84.5 "got-a-couple-notes-here" coffee isn't a massive swing when we are talking about the dichotomy of pitting the world's best roaster against an informed hobbyist. A point swing like that at the upper end can be the difference between middle of the pack and a gold medal at an event though. Or a huge difference at an auction. So it certainly does matter.

On another note, it does slap us roasters a bit to wake us up and say there is more to roasting than heating the coffee without adding roast defects. 1.5pts is a good amount of sway to make the minutiae of a dial-in past simply removing roast defects a very worthy process.

Very neat how he actually said 1.5 points instead of keeping it less definitive.

More importantly, how this manifests with cupping scores to sell seems ethereal at this point. Is it fair to compare a cupping score given to a green coffee roasted by an individual using the same profile on 50 samples in a row on a beat up Probat sample roaster at origin to a coffee roasted on the latest and greatest roaster while given special attention with multiple roasts by an exporter looking to cup it the highest? If we are to buy partially based on cupping score and roast affects the score that much, then perhaps it is best to think of the score as, for example, "83 +/- 1.5." Because that could be a score for the best it will ever taste or the worst it could taste without roast defects.

You could be getting a bargain if they roasted it poorly to score or you better have amazing roast skills to ever achieve the rating they gave the coffee. :D

baldheadracing (original poster)
Team HB

#3: Post by baldheadracing (original poster) »

It's important to keep in mind this is within the context of cupping roasts. Regardless, the 1.5 point spread from roasts done be someone one hopes knows how to roast well tells me to not take cupping scores as a single number.

Regardless, there's a bit of a boom on in sample roasters after Ikawa came out - now there is the electric Probat and Roest and the upcoming Nucleus Link ... these machines can run profiles so are the scores the coffee or the roast? This may make comparisons across green coffee vendors even more difficult.
-"Good quality brings happiness as you use it" - Nobuho Miya, Kamasada

User avatar
Almico
Supporter ★

#4: Post by Almico »

baldheadracing wrote: "Is a cupping roast the way to capture the full potential of a coffee?"
Not to me. A cupping roast is a way to evaluate a coffee in order to make a buying decision. All coffees, whether they are intended to eventually be a dark or light roast, are roasted exactly the same way. This way I am tasting the coffee and not the roast. Once I determine the acidity, body and sweetness, as well as lack of agricultural defects, I can make a decision whether or not that it's a coffee I can work with.

FWIW, I am not inclined to roast acidity, body or sweetness into a coffee via the roast. The green either has those qualities, or it doesn't. I do not believe in roasting to "bring out the best in a coffee" other than to avoid roast defects. I prefer letting the coffee speak for itself.

My job as a roaster is 80% picking good coffee and 20% roasting it well. That is what cupping is for to me.
★ Helpful

User avatar
another_jim
Team HB

#5: Post by another_jim »

Almico wrote:My job as a roaster is 80% picking good coffee and 20% roasting it well. That is what cupping is for to me.
Alan is describing how roasters should work and most actually do work; while only the coffee gods and his twitter followers knows what fantastic roasting discoveries Rao has made in the last five minutes. But the quoted sentence is not quite right for home roasters or anyone producing a roast for a specific drinker.

To me, some coffees taste better at lighter roasts, some at darker roasts. I think each coffee has its best roast, and that roast can be light or dark depending. So my standard cupping roast is a little darker than most, since I want to get a sense of how the roast flavors develop. Because the greens I buy are mostly pre-vetted and free of gross defects; I don't need to do the light cupping roasts that pinpoint defects and ripeness problems. Instead, I try to figure out at what roast the bean will show best.

If you are an Italian roaster producing a bar blend with a very specific taste, you will first do defect cupping, then roast the coffee for its intended role in the blend. My guess is that they have a book of alternative possibilities that is as thick as a chess openings manual.

The way you cup and roast is determined by your intent. As a home roaster, you can ask yourself what each coffee wants to be; or you can create blends and roasts that suit your own tastes and purposes. For instance, when you want a big after dinner demi--tasse, don't do fast, light roasts.
Jim Schulman

baldheadracing (original poster)
Team HB

#6: Post by baldheadracing (original poster) »

baldheadracing wrote: This experience begs the question, "Is a cupping roast the way to capture the full potential of a coffee?"
So I didn't word that quite right. What I meant to say: Is a cupping roast the way to get an understanding of what a coffee is capable of; of the coffee's potential characteristics?

Most home roasters do not have samples to use to make buying decisions. What a commercial roaster can do and how a commercial roaster uses samples to make buying decisions is just not a process that is possible, however desirable. One reads the marketing from the importer/distributor and opinions of others who have already roasted that particular coffee and that is all that one can go by to make a buying decision. So now one has 5 - 10 - 20 pounds of green. What's next?

Is it a cupping roast? Is it a slightly darker roast, as Jim uses? Is it a roast where you pull samples from the roast as time passes, as Neal Wilson does? I'm not suggesting that there is one right approach.

As for Rao, I deliberately left out the rather provocative thesis of his blog post to consider just one point. As home roasters generally can't make buying decisions based on roasting samples, home roasters have to consider what the vendor gives us, and one part of that is cupping scores and score sheets. Ryan Brown, who wrote "Dear Coffee Buyer," scores the same coffee in about a 1.5 point range depending on which cupping roast Rao did. Leaving Rao out of it, I might predict that if I did a half-dozen cupping roasts of the same coffee and scored them/had them scored, then a range of 1.5 points would not be unusual ... or would it?
-"Good quality brings happiness as you use it" - Nobuho Miya, Kamasada

User avatar
another_jim
Team HB

#7: Post by another_jim »

baldheadracing wrote:if I did a half-dozen cupping roasts of the same coffee and scored them/had them scored, then a range of 1.5 points would not be unusual ... or would it?
When is this a relevant question? Who would ask it?

As a home roaster, I buy a one pound bag of coffee and try it. If I like it, I buy more; if I don't, I dump it. That's a yes or no, not a point count. Is it possible that I've blown the sample the sample roast and made a mistake? I suppose it is; but I don't find it plausible, since I'm also paying attention while I roast, especially a new coffee, smelling and sampling during the roast. If there's any doubt, I try again. A pro roaster will do the same if it's about buying a single coffee.

If it's about buying, lets say, a Kenya Kirinyaga or a washed Yrgacheffe, you'd be cupping several samples and picking the best. Here the question becomes if sample roasting variance could affect the ranking. My guess is that a roaster spending thousands of dollars on green will be fairly confident that his roasts will tell him what's best. It gets even more critical if you are a greenie roasting the sample box for an auction when you'll be bidding tens of thousands to buy import lots. Trust me, they are not worrying about if their sample roasting variance is affecting the cup scores and their willingness o bid. If they did, they'd can the roaster.

So how does this question arise in the first place? If you are selling green coffee, e.g. Sweet Maria's, you'll have a cupping routine that all your buyers will take with a very large pinch of salt. Caveat emptor: get a sample or buy a one pound bag first. The real cheek is when a greenie like 90 plus sells large lots, no samples, and makes out that their roasts are so scientific that if you disagree with their scores, there's something wrong with you. Again, caveat emptor.
Jim Schulman

User avatar
Almico
Supporter ★

#8: Post by Almico »

I think home roasters, me included, fall into the trap of believing there is such a thing as a ubiquitous 88 point coffee, or an 85 point or a 90 point coffee, and if only we could improve our cupping skills enough, we would finally be able to arrive at this score. Nothing could be farther from the truth. Cupping scores are in the pallet of the beholder.

baldheadracing (original poster)
Team HB

#9: Post by baldheadracing (original poster) »

another_jim wrote:When is this a relevant question? Who would ask it?

As a home roaster, I buy a one pound bag of coffee and try it. ...
How does one decide on that one pound bag of coffee? Home roasters - well, some of them - would make that choice based in part on vendor-provided cupping scores. If a vendor has a half-dozen washed Yirgs, then some will pick the one with the highest score. If there were six different vendors each offering a washed Yirg, then some will pick the one with the highest score. While I would expect a single vendor to be consistent across their own scores, I would not be so sure about comparing between vendors' scores - for a multitude of possible reasons. Roasting variability may be one of those reasons; if so, then roasting variability may apply within a vendor as well. That's what the question (partially) answers. Whether the answer is relevant or not is a judgement call. People may want to believe in absolutes, but Heisenberg was uncertain :wink:.

(I haven't made a purchase decision based on scores in quite a while, but I remain tempted.)
-"Good quality brings happiness as you use it" - Nobuho Miya, Kamasada

User avatar
another_jim
Team HB

#10: Post by another_jim »

I'm not into regret. If I have a good washed Yrg, I enjoy it. I don't worry I might have missed a better one. But yeah, if you are looking to source the best, you're dealing with a population that is too big for any one person to try. So you'll have to rely on comparing the cupping scores from different people. Top roasters don't do this; they narrow to their choice to suppliers they trust, and then cup the limited samples for themselves.
Jim Schulman