Excellent writeup Abe! This was my third tour as sensory judge, first time as USBC judge. The thread SCAA Barista Competition - Competitor Debriefing documents my thoughts after last fall's SERBC; I would like to add a comparison of the SERBC and USBC competitors from my admittedly brief experience.

First of all, the quality of the judges' certification excelled over the last two I've attended, thanks to the hard work of Brent Fortune and co-presenters Marcus Boni and Jeff Taylor. Brent / Marcus were exceedingly well prepared, managed time effectively, and tabled side discussion when necessary without stifling comments. Even the written tests assured you had paid attention (at the first SERBC workshop they handed me a pencil as I came through the door - "What, an essay test? Are you kidding?!?"). The information presented in the USBC workshop was topical, complete, and succinct. Day two with the "mock" competitions and calibration exercises were well organized and ran darn close to schedule. (By the way, did anyone else notice that the head judges were scoring the candidate judges on several categories? I happen to glance over and saw my name... and poor marks for legibility; I cleaned up my act for the next few rounds ).

So, what separates the US Barista Competitors from the Regional Competitors?

Abe was lucky to have a very strong showing on his flight, including two finalists. I had a similar experience at my first regional, with the winner Ryan Jensen in my flight, two of the six finalists, and an exhibition competitor from Intelligentsia, Stephen Rogers, who returned to this year's USBC. My second year as judge at the SERBC was a mixed bag; below is a closeup of one of the pours during my round:


It didn't go as planned

Which leads me to the first noteworthy distinction: Not surprisingly, the expected level of consistency is much higher. Abe suggests that the winner's circle is reserved for those who regularly pull 4's ("very good"). I've not seen the final scores, but would guess that making it from the first round to the second round requires consistent 2.5's (between "average" and "good") and a couple threes.

USBC competitors use their time more effectively. For example, they would clean their station with a quick brush here, a brief wipe there, ending with almost no cleanup work. Some competitors casually straightened a tray or restacked some cloths as the last minute ticked away in hopes of getting extra "attention to details" points.

Fluidity of the presentation. A few competitors were clearly gifted public speakers, and most managed to interlace their running commentary with the necessary task of prepping their drinks. The best baristas were able to "connect" with the audience and the judges by sharing their knowledge and interests in coffee while staying focused. Those who struggled would face their station while speaking, or look downward when addressing the judges, costing them presentation points.

The Drama Behind the Numbers

The full competition ranking was published yesterday. We do not have the score sheets that detail the competitors score for every single component, just the final score for each competitor in the various rounds. The real drama is in the full score sheet, and I wish the USBC made it public. The final number misses the whole dance. Oh well, let's see if we can learn something from the final numbers.

Matt Riddle, who took the championship, would not have made it to the final if not for a clerical error by the judges. I'm referring to that famous overtime penalty error with Ryan Dennhardt. This is not to diminish in any way Matt's accomplishment, which he won fair and square. Clerical errors do happen in competitions and I believe the judges made the right decision allowing seven finalists instead of the usual six because of that mishap.

Billy Wilson was awarded the highest score in the competition. He got it in round one, 835 points. However, every round is a new competition, and sadly for him, that score does not carry any weight to the next round.

There has been a total re-shuffling of ranking from the semi finals to the finals. What accounts for that? It is a combination of a few factors:

1. The coffee - it is a day older. For some coffees this is good, for others it is not so good. Competitors must bring with them coffee of different roast dates to ensure that they have a batch that is at its peak for every day of the competition.
   
2. The barista - every day is a new day. We all have our good and bad days.
   
3. The judges - mostly on the sig, but on standard drinks also. When the difference between first and second place is 25 out of 1200 points, every tiny thing counts. There is an element of subjectivity in the scoring which cannot be neutralized, and shouldn't be. The scoring rules try hard to objectify every possible aspect of scoring, but this is a sensory experience at the end of the day, which is influenced by personal preferences. There is nothing the barista can do about it other than give his best shot, and let the dice role. The fact that there was such a small point difference between first and second place is a tribute to the judges ability to calibrate their scoring.

Matt and the rest of the finalists did a great job in this competition. Their score tells the story, they raised the level at the finals from prior years. I believe that this year, for the first time, we will see a U.S. Finalist in the WBC.

Dramas are part of any competition and this is a main reason we enjoy watching them. To increase the level of drama in the competition, I think the full score sheet should be published after every round. I do not see any unfavorable consequences that should keep it from being public in real time.

Abe Carmeli wrote:Dramas are part of any competition and this is a main reason we enjoy watching them. To increase the level of drama in the competition, I think the full score sheet should be published after every round. I do not see any unfavorable consequences that should keep it from being public in real time.

That runs the risk of influencing the latter round judges, doesn't it?

Nobody knew the rankings of the seven finalists except the scorekeepers. If it were public knowledge prior to the finals that Matt was in fact the 7th finalist, could some judges be predisposed to view him less favorably than a competitor that ranked well ahead of him in the prior round? I would like to think that the answer is no, judges don't pre-judge, but keeping rankings out of the public eye until the end of competition seems prudent, especially given that the scores can be so narrow.

Just for the record...

It wasn't a "scoring error."

(NOTE: my following comments are completely hearsay. They are accurate to the best of my knowledge, but they do not reflect any official USBC committee statements and are completely unauthorized)

The USBC committee (or at least part of it) received a formal written letter of complaint, pursuant to section 8.0 of the USBC competitor rules and regulations (PDF file). Ryan Dennhardt was approximately 32 seconds over-time, resulting in a 40 point penalty (5 points from each of the four sensory judges for each 30-second block over... 32 seconds = two deductions... 40 points). The Dennhardts claimed that because of the actions of runners and judges, Ryan was unfairly prevented from proceeding with his presentation. The USBC Event Manager and USBC committee members reviewed the videotape from Ryan's presentation (because it was available), and made the decision to re-evaluate Ryan's competition time and restored his 40 points that were deducted due to time penalty.

These are the facts as I know them to be.

Carry on.

HB wrote:That runs the risk of influencing the latter round judges, doesn't it?

Nobody knew the rankings of the seven finalists except the scorekeepers. If it were public knowledge prior to the finals that Matt was in fact the 7th finalist, could some judges be predisposed to view him less favorably than a competitor that ranked well ahead of him in the prior round? I would like to think that the answer is no, judges don't pre-judge, but keeping rankings out of the public eye until the end of competition seems prudent, especially given that the scores can be so narrow.

It is a good point Dan but I disagree. You could run the same argument at the beginning of the competition. Some baristas are known to be excellent in their craft even before the competition begins. If the judges are biased by ranking, they will be biased there too. In my short experience, ranking played no role in the scoring. Give the judges some credit here. Generally speaking, publishing the score after every round is common practice in the great majority of sport art and food competitions. From musicians to athletes to chefs. This is not only done to enhance the drama, but to give the competitors important feedback. A clue where they should improve their performance in the next round.

Nick wrote:Just for the record...

It wasn't a "scoring error."

Thanks for the correction Nick. I wish they'd come out by now and officially tell us what happened there.

Abe Carmeli wrote:It is a good point Dan but I disagree. You could run the same argument at the beginning of the competition. Some baristas are known to be excellent in their craft even before the competition begins. If the judges are biased by ranking, they will be biased there too. In my short experience, ranking played no role in the scoring. Give the judges some credit here. Generally speaking, publishing the score after every round is common practice in the great majority of sport art and food competitions. From musicians to athletes to chefs. This is not only done to enhance the drama, but to give the competitors important feedback. A clue where they should improve their performance in the next round.

Scores? An interesting point and idea. On the other hand, there's enough for competitors to worry about... having scores published after each round would frankly create an additional layer of... stress? That being said, I think it's an intriguing idea that could be feasible.

Scoresheets? No chance. At least not during the competition. There is a lot of information about a fellow competitor that could feasibly be used to one's advantage. However, if I remember correctly, there are certain national championships (Australia comes to mind) that might release the full scoresheets after the competition. On the other-other hand... maybe doing so wouldn't be such a good idea, considering you want to send your US Champ to the WBC without having their scores published for... you get the idea.

Nick wrote: However, if I remember correctly, there are certain national championships (Australia comes to mind) that might release the full scoresheets after the competition. On the other-other hand... maybe doing so wouldn't be such a good idea, considering you want to send your US Champ to the WBC without having their scores published for... you get the idea.

I didn't think about that one, but in practice I don't think it should be a concern. It assumes that one can't improve from one competition to the other, that the WBC judges care to read the minutia detail of the competitors' past performances in their nationals, and that they will also be biased by it. I think all those assumptions are false. From my brief experience with that crowd, they couldn't care less.

Nick wrote:On the other hand, there's enough for competitors to worry about... having scores published after each round would frankly create an additional layer of... stress? That being said, I think it's an intriguing idea that could be feasible.

It comes down to who the competitions are for. The current format is designed for the baristas themselves, not for an audience. For an audience, it would be something like this:

The competitors start the final round in inverse finishing order from the qualifying rounds, some points carry over. As each competitor performs, the voice over tells the crowd how many points are needed for her or him to get into first place. The scoring is displayed live, with voiceovers "nailed that," or "oh what a mistake!"

A sports or competition audience feeds off stress.

another_jim wrote:It comes down to who the competitions are for. The current format is designed for the baristas themselves, not for an audience. For an audience, it would be something like this:

The competitors start the final round in inverse finishing order from the qualifying rounds, some points carry over. As each competitor performs, the voice over tells the crowd how many points are needed for her or him to get into first place. The scoring is displayed live, with voiceovers "nailed that," or "oh what a mistake!"

A sports or competition audience feeds off stress.

Haha! The funny thing is, we've talked about this. A play-by-play. It's an idea that's been thrown around among WBC folks and USBC folks.

Logistically, it's tricky. Let's see if the idea (for Long Beach 2007) takes hold for a closed-circuit feed of the USBC on monitors around the SCAA show floor... or at least, a monitor outside of the show-floor. That would provide a possible venue for a play-by-play to take place. We certainly couldn't have that happen right at the competition area.

You're right in saying that the competition "is designed for the baristas themselves, not for an audience." The fact is, we're sort of in the 2nd-generation of barista competitions. The audience-friendly competitions will emerge someday soon... either by changing and developing the current WBC/USBC's, or creating something altogether new.