I'm curious if other's who are more familiar with the structure of these big competitions can chime in with some thoughts on how the scoring structure works. Mainly, I'm wondering, if the highest achievable score is now 719, and competitors are for the most part extremely polished in their technical approach and not likely to loose points there, why aren't these "unicorn" coffees (mostly exotic geishas) putting their scores in the highest ranges? I'm sure it's more complex than my current understanding.

If memory serves me correct, the top rankings this year were in the very high 400's with some just over 500. Does anyone perceive that as a sign that these superstar geishas may not be the best choice for an espresso based competition? It's not an indictment of these geishas we all love, but just a curiosity. I get it that you need to bring all the bang and pizazz possible. And I don't think scores were higher in years past, when geishas weren't common, but perhaps the scoring structure has evolved and a direct comparison isn't easily made.

Anyway, I'm just curious if anyone else thinks that it's actually harder to squeeze out higher flavor scores when the most ubiquitous flavor descriptors are lime and florals? Could there be a claim that there's a bit of palate fatigue going on?

I realize that other varietals still make their way to the top too.

Would be interesting if judges shared notes, even if they have to maintain privacy of competitors.

I had a similar question in mind watching the Brewers Cup where geshas competed with some naturals I can't remember.

I wondered if people spent too much $$ on their raw coffee and not enough effort on trying to pull something great or at least intriguing out of a less expensive raw resource. I know in food and movies, as the years pass, the more I look for highly intriguing experiences and the more bored I get with formulas, even if they are great ones.

TomC wrote:I'm curious if other's who are more familiar with the structure of these big competitions can chime in with some thoughts on how the scoring structure works. Mainly, I'm wondering, if the highest achievable score is now 719, and competitors are for the most part extremely polished in their technical approach and not likely to loose points there, why aren't these "unicorn" coffees (mostly exotic geishas) putting their scores in the highest ranges? I'm sure it's more complex than my current understanding.

If memory serves me correct, the top rankings this year were in the very high 400's with some just over 500. Does anyone perceive that as a sign that these superstar geishas may not be the best choice for an espresso based competition? It's not an indictment of these geishas we all love, but just a curiosity. I get it that you need to bring all the bang and pizazz possible. And I don't think scores were higher in years past, when geishas weren't common, but perhaps the scoring structure has evolved and a direct comparison isn't easily made.

Anyway, I'm just curious if anyone else thinks that it's actually harder to squeeze out higher flavor scores when the most ubiquitous flavor descriptors are lime and florals? Could there be a claim that there's a bit of palate fatigue going on?

I realize that other varietals still make their way to the top too.

Over the years the total points receivable from the Judges have declined.

Between 2012 and 2016, the highest points receivable from a Sensory Judge has dropped from 179 to 162 per judge. There are four Sensory Judges.

The same is true of a Technical Judge; from 77 points to 71. Before 2016, there were two Technical Judges. Now there is only one. You can confirm this on the Head Judge's score sheet.

2012 = 870 points possible
2016 = 719 points possible

The most important thing to observe here is, there is now MORE weight carried by Sensory points than by Technical points.

A WBC caliber barista shouldn't lose many points on a Technical score sheet. Now that there is only one Technical Judge, there are fewer "easy" points to be had (certainly not "easy", but I believe I'm making my point).

Please let me know if you have any other questions/observations.

happycat wrote:Would be interesting if judges shared notes, even if they have to maintain privacy of competitors.

I had a similar question in mind watching the Brewers Cup where geshas competed with some naturals I can't remember.

I wondered if people spent too much $$ on their raw coffee and not enough effort on trying to pull something great or at least intriguing out of a less expensive raw resource. I know in food and movies, as the years pass, the more I look for highly intriguing experiences and the more bored I get with formulas, even if they are great ones.

I don't see a question here but allow me to share some info on the WBrC score sheets as well.

In previous years there were three Open Service Judges who assigned points and one Head Judge who did not.

The number of Judges hasn't changed, but the Head Judge now assigns points for "Overall Workflow" and "Sensory Uniformity". As a result the Open Service Judges no longer assign an "Overall Impression" score in the Presentation section of their score sheet; they only assign a "Taste Description" score and "Customer Service" score.

By doing this, competitors are now judged for their technical abilities, not just their presentation and the quality of their cup.

The multipliers in the Cup Score section of the Open Service score sheet have been shifted from the "Flavor", "Balance", and "Overall" scores to the "Acidity", "Body", and "Balance" scores.

The reason for these changes is to de-emphasize the points given to a Cup due to the varietal and instead to emphasize the attributes of the coffee within the context of itself.

For the most part, the types of coffees in the top 6 are similar to previous years. A mix of washed and natural Geishas with a stellar Ethiopian from Ninety-Plus (Odd-Stienar won with an Ethiopian last year.)

sdbrewer wrote: ...
By doing this, competitors are now judged for their technical abilities, not just their presentation and the quality of their cup.
...
The multipliers in the Cup Score section of the Open Service score sheet have been shifted from the "Flavor", "Balance", and "Overall" scores to the "Acidity", "Body", and "Balance" scores.

The reason for these changes is to de-emphasize the points given to a Cup due to the varietal and instead to emphasize the attributes of the coffee within the context of itself.

Thanks for sharing this. If I get it, sounds like efforts to curb score inflation and refocus on more challenging aspects. I am guessing this addresses Tom's point about higher technical ability and high grade coffees... How to separate people at that level in a meaningful way to isolate their contributions.

My main question just centers around what the judges may be seeing as "geisha fatigue" not much more.

happycat wrote:Thanks for sharing this. If I get it, sounds like efforts to curb score inflation and refocus on more challenging aspects. I am guessing this addresses Tom's point about higher technical ability and high grade coffees... How to separate people at that level in a meaningful way to isolate their contributions.

Just to clarify, my first post is in reference to the World Barista Competition and the second is in reference to the World Brewers Cup.

With regard to the World Brewers Cup, there has been a push to introduce scoring based on technical aspects of brewing, as this explicit evaluation has been absent in the past. In recent years the competition has been criticized for being "less about who can brew better" and "more who can source the better coffee."

http://shop.gardellicoffee.com/blogs/be ... -to-fix-it

I can't imagine the challenge that faces the US Brewers Cup competitors when faced with someone like Goldsworthy with Klatch's backing when it comes to sourcing

TomC wrote:I'm curious if other's who are more familiar with the structure of these big competitions can chime in with some thoughts on how the scoring structure works. Mainly, I'm wondering, if the highest achievable score is now 719, and competitors are for the most part extremely polished in their technical approach and not likely to loose points there, why aren't these "unicorn" coffees (mostly exotic geishas) putting their scores in the highest ranges? I'm sure it's more complex than my current understanding.

TomC wrote:My main question just centers around what the judges may be seeing as "geisha fatigue" not much more.

The multipliers on the Sensory Judges' score sheets have always made it difficult to knock it out of the park. Since the Technical scores have been played down, the competitors scores seem lower compared to past competitions.

Unless they recently judged the Best of Panama Competition, I doubt any of the judges are suffering from "geisha fatigue". It's a rare occurrence when one has the opportunity to evaluate so many skillfully extracted geishas from a range of origins and harvests.

TomC wrote:I can't imagine the challenge that faces the US Brewers Cup competitors when faced with someone like Goldsworthy with Klatch's backing when it comes to sourcing

No doubt! A disciplined competitor + great sourcing is a difficult combination to beat.