USBC Barista Competition - Competitor Debriefing

[Abe Carmeli]

It is Seattle, May 2005, and I'm sitting front and center watching the 2005 World Barista Championship Finals. As I was salivating at the looks of some of those drinks, I felt like the family dog watching a dinner party in progress: he knows it's good and he knows he ain't getting any.


Judging a straight shot

So what does it take to be on the guest list for that party? A few days off work, 2 days of intensive judges certification workshop, and a little luck passing the dreaded sensory test. With all that behind me, I was looking forward to my first stint as a sensory judge in the 2006 USBC.

My turn came on day two of the competition. After a quick judges meeting in the morning, we were all set to go. It didn't dawn on me at that time that I'm about to have 15 espressos in 1.5 hours.

I looked at my flight's roster and smiled. On paper, it was arguably the strongest flight of the day, with some famous baristas leading the list. (There are five contestants in a flight). But as we all know, pedigree means nothing once the clock starts running. You are judged by your performance on stage, on that day, and not by anything else. We've seen star baristas bomb before. As to the judges, I was the only new judge on the panel, joined by very experienced dudes, some with WBC competition and numerous USBC credits. It felt good to be around that bunch. I was looking forward to some eye popping drinks.

I intentionally do not mention the contestant names in this report, and I also mixed their gender & order of appearance to keep it from becoming a personal commentary. So let's get on with it.

The first contestant just flew by. From the outside, it may look like the hardest part in judging would be, well, judging. (drink & performance evaluation). But in practice, the hardest part was keeping records. The contestant must pass a list of required criteria, and as a judge, one needs to document the nuances of his/her performance. It helps with giving the final score, but it is also used to justify a particular score if challenged by the head judge. The head judge can override any score given by any judge, and the lowest and highest scores on any criteria require his/her approval. Record keeping comes in handy in debriefing as well, when a contestant asks the judges for feedback.

As I summed up her total score I felt empathy with her performance. She was unprepared & particularly nervous. To give you an idea how nervous one can get in those competitions, in some instances the competitor's hand was shaking so violently, that they almost tipped the cup when serving their drink. One judge commented to me afterwards, that in the WBC, the Japanese contestant had to balance the drinks tray on the table to avoid spilling them all over the judges.

Back to our contestant, one must admire those baristas that go out there and compete. Many of them are not born performers, and the thought of being on stage for anything, is as appealing as eating glass. But they still go out there and give their best shot.

The only way to shine on stage is to rehearse the performance until you go through it with the same ease as driving. The actions are all automatic & smooth. We are all able to drive while thinking of other things than driving itself. It is that kind of smoothness or almost mindless performance contestants should be striving for. When this is achieved, the barista can turn his attention to excelling.

Another overall problem in her presentation was that throughout her entire performance, it sounded more like she is talking to herself than to the judges. Very quickly, that particular challenge separated the leaders from the pack. The enemy in this case is time. They go through their audio presentation while preparing their drinks with their back turned to the judges. Fifteen minutes sound like a long time, but those fly by so quickly, and in most cases, performers finished with no more than one or two seconds to spare, and some went over by almost a minute.

Those who had it together, were confident that they can do both and still make time. They came to the judges table & talked about each drink before they prepared it. In some cases, the signature drink was prepared right on the judges' table. Those rare treats had a positive impact on us, and added to the contestants' overall presentation & attention to details score.

We are off to the second barista. In between the performances, the judges convene backstage to finish up their score sheet, quickly grab a palate refresher (a cracker), and return back to stage for the next contestant. That short break takes about three to five minutes.

The second barista was better prepared, and generally appeared more comfortable on stage. He served better drinks overall and improved his chances to advance to the semi finals.

Coming up next was the show stopper of the flight. It was such a pleasure to watch an almost flawless performance. She delivered understated passion wrapped in a professional presentation with great flow. She was indeed driving, or as others would call it, "in the zone". The espresso was very good, the cappa smooth & creamy with great milk/coffee balance, and she did all that with such ease that makes you wonder what are those contestants so nervous about? (It is easier said than done, and takes many long months of hard work to get there.)

That performance was followed by another stellar show and the two ended up with a very similar score. Closing the flight was an average performance, and before I knew it, it was all over.

The first question I was asked after I handed my final score sheet was "were you nervous?" The answer is not at all. I've been critiquing and judging my own espresso for years. I feel at home in that environment, and I had no problem doing it in the USBC setting. The only thing that required some effort on my part was record keeping, but after the first contestant this became a non issue. You get into the rhythm of things.


The Sensory judges. From left to right: Tracy Allen, the silver fox (Head Judge); Abe Carmeli; Jeff Babcock; Mike Perry & Spencer Turer

So, what does it take to become a US Barista Champion?

1. Don't even try it until you can consistently deliver drinks that will be rated as 4 (very good) by a USBC judge. The criteria for a 4 are known and published, and use an experienced Certified USBC judge to rate your drinks if you can.
   
2. Work on your performance until you do it without thinking. Start with the basics without the commentary; just make the drinks in a row. Once this is automatic, start introducing your commentary. Your commentary is extremely important, and use a wide audience to critique it. Filling up dead air with nonsense should be avoided. Go for quality not quantity, and say something interesting about your coffee, your blend, your drink, who you are. Stay away from over the top sermon on the mount commentary. We had one of those in the competition (not on my flight) that sent me cringing looking for cover. Passion for your work should come through but it should be understated. Nuances go a long way. A class act is built on restraint and nuances and not on all-out "I'm the king of the world".
   
3. Be bold with your standard drinks. Particularly with your coffee. The sensory judges know espresso. They can recognize a boring cup when they see one, even if it is balanced and sweet. If you are going out there, go to win it, and to add something to our espresso experience. Another standard cup is not what will get you a five and it will add nothing to our experience.
   
4. Select your grinder after a careful examination of all available alternatives. The grinder, and not the espresso machine, is the barista's most important tool. Billy Wilson used a Mazzer Jolly, I assume it is because he was familiar with it. Matt Riddle and the rest of the Intelligentsia contestants used a Mazzer Robur. The major difference between those two is the burr set: Flat vs. Conical. Choose the grinder that works best with your coffee. This will come through in your straight shot score, perhaps the barista's ultimate challenge.
   
5. Before you make your drinks approach the judges and engage them with your selection. Say something about the coffee and why you chose it, tell an anecdote, tell them something, even small that they don't know, and keep them interested. Some contestants brought the coffee they are using to the table, and placed it in front of the judges to touch see and smell. Good move. Be engaging. As much as possible, prepare at least some of your Sig drink at the judge's table. This has a big impact on your presentation score and sometimes that makes the difference between a 1st & 2nd place
   
6. Work on your signature drink. The Sig was a problem across all contestants. They were universally mediocre to really bad. One of the criteria that judges keep in mind is that the USB champion will be representing the US in the WBC. The Sig at that level makes a huge difference in the score. I have not seen the score results of Matt Riddle and Billy Wilson (who finished 1 & 2) but I'm willing to go out on a limb here and say that Matt won it on the Sig.
   
7. In preparing a signature drink, stay away from cold drinks. It is no accident that all USBC champs in the past three years served their signature drink warm. Judges are not accustomed to consume espresso as a cold drink and it is the espresso taste we are looking for. When it is cold, it often reminds me of a Starbucks Frappuccino, no matter how delicate & elaborate the preparation was. You don't want to be within 1 mile from a Starbucks drink.
   
8. Consult with a dessert chef when you are preparing your Sig. No, you don't want to prepare a desert, but these guys know more than anyone else how to combine variety of ingredients to deliver a particular taste profile. Every major city in the US has a culinary institute. Choose a chef with a good theoretical background in food chemistry. Promise him you will mention his contribution if you win and give him invaluable publicity. Offer to give an espresso class to his students. He may take you as a pet project, or as a curiosity but I'm sure you will find a receptive audience.

Tricks I learned from the pros

If I learn only one thing from an espresso jam I consider myself lucky. During the judges certification workshop, we were served drinks by two USBC Champions, Bronwen Serna & Phuong Tran. Both terrific baristas.

Avoiding Crema Bubbles

Bronwen is famous for her cappuccinos, and here's what makes her shine: Often, a cappa will show crema bubbles. This may not affect the taste so much as it affects the visual. To avoid crema bubbles she drops a little foam to the cup and immediately starts swirling and banging it on the counter as one would do with a steaming pitcher. After she is happy with the result she pours the rest of the milk.

A perfect cappuccino by Bronwen Serna served during the judges certification workshop.

Pouring Even Cappas

This one comes from Intelligentsia. Ellie, Amber & Matt all used it. The problem with using one pitcher to pour two cappuccinos is that the first pour will have more foam than the second. To avoid it, pour some of the foam into a 2nd pitcher before you pour your first cappa. Then use that foam to top off your 2nd pour.

A Few Technical Observations

Distribution & Leveling Techniques

The main two classical techniques are Stockfleth and Schomer. One is a circular motion and the other is an X pattern linear motion. I observed that almost all contestants performed some version of a linear motion as opposed to a circular one, with one contestant using a saw motion to distribute and level the puck.

Dosing

All of the contestant whacked the doser handle while the grinder was running, and did not wait for it to finish grinding before they dosed. This may reduce clumping but it is also a good way to ensure that they do not waste coffee, for which the will be penalized in the scoring.

Tamping

Almost all the contestant use two fingers to balance the tamper and achieve leveled tamp.

Parting shot

This has been a fantastic experience. Other non-professional coffee enthusiasts that were certified as USBC judges are Dan Kehn & Bob Roseman of Team HB; and Kitt Johnson. In addition to all that fun, I spent close to five days with great people who are as certifiable as I am, and for a brief period of time - I felt normal . By the end of the day I've had close to 30 espressos, counting all the other shots I had on the show floor. Sitting down for dinner with Jim Schulman, Bob Yellin & Marshal that night, I had a problem stopping my hands from shaking. A small price to pay for a terrific experience.


Phillip Search preparing his signature drink at the judges table. Notice the beautiful table layout and decoration


Say what? You gonna put four eggs in my espresso?? Billy Wilson giving a flawless performance.


The Sig


The Judges certification workshop. From left: Spenser Turer (in checkered shirt); Nimo; Kitt Johnson; Dan Kehn, Bob Roseman, and Scott.


Phuong Tran, 2005 USBC Champion preparing her Sig at the judges' workshop


Mock competition at the judges' workshop

* All competition pictures are courtesy of Sean Lennon. Is there anything that dude does not do well??

[another_jim]

Great post! Have you considered cross-posting it to the Barista Guild forum?

[HB]

Good idea Jim, I added a link back to the USBC related threads on the BGA here, as did Ben (e61brewski).

[Ken Fox]

Great post, Abe, I feel like I was there, and wish I had been.

ken

[e61brewski]

The Sig was a problem across all contestants. They were universally mediocre to really bad.

care to elaborate? at the regional level, it's easy to see from the audience what sigs are really terrible (milkshakes, lots of syrup in a standard drink, etc.). but, speaking for myself, i saw a lot of really creative stuff at the usbc that at least sounded good. why the mediocre review? i believe you, i'm just curious about what set the good drinks apart from the bad when, from the audience, the distinction wasn't all that clear.

and since there's a mild debate going on about billy's egg whites, what did you think?

[another_jim]

The main problem with eggwhite is that at soft peak, they have a great mouthfeel, but can muffle the taste, same as milk. At stiff peak, there's more volume and less muffling, but the mouthfeel is lousy. This makes it a very tough ingredient to use. The classic case is chocolate mousse. The diet versions replace the whipped cream and some of the beaten eggyolk with extra eggwhite. It's almost impossible to get one that isn't chalky or tastes dead.

I've admired Billy for taking big risks at each of the USBCs -- his performances are always a highlight for the audience. But given that actual performances on the day of competiton will usually not be utterly perfect; I have a feeling that the competitors taking risks go into the finals as underdogs -- they have to hit on all stops to win. An analogy is Heather Perry's circus pour she used two years ago at Atlanta. It came off perfectly in the semis, but was slightly muffed in the finals. Bronwen won.

[Abe Carmeli]

The Sig was a problem across all contestants. They were universally mediocre to really bad.

care to elaborate? at the regional level, it's easy to see from the audience what sigs are really terrible (milkshakes, lots of syrup in a standard drink, etc.). but, speaking for myself, i saw a lot of really creative stuff at the usbc that at least sounded good. why the mediocre review? i believe you, i'm just curious about what set the good drinks apart from the bad when, from the audience, the distinction wasn't all that clear.

I can't talk about specific performances that I judged, since the score sheet is not a public record. The problem with the sigs was that many of them looked good on paper, but not on the palate. In judging the sig, we first try to ascertain whether the drink we taste meets the taste profile presented to us by the barista. Did he deliver what he said he would? In that part of the score, we ignore our personal preference and it is almost a technical score. However, to get the extra points the drink must be tasteful. It needs to impress us on the palate. Multiple ingredients in a drink do not guarantee a good result. The bottom line was that many of those were boring on the palate, the ingredients clashing, or not dominant enough, lacking complexity, or too much of it to lose focus. As to the general question of using egg whites in a sig, I think Jim gave an excellent answer. I will add one observation to it: The notion of drinking raw eggs is not very appealing to most people. As a general rule, I would avoid eggs as an ingredient.

[another_jim]

I will add one observation to it: The notion of drinking raw eggs is not very appealing to most people. As a general rule, I would avoid eggs as an ingredient.

An observation on this and the Barista board thread. Billy heated his eggs in a doubleboiler. If he got to 150F to 160F, he met the normal standard for safe eggs.

[HB]

Taste issues aside, I believe pasterized eggs are readily available these days. I don't recall Billy mentioning that he used them though.

[luca]

Hey Guys,

I was lucky enough to judge in our state barista competition and what y'all are saying held true for the signature drinks here. The main problem was that the espresso was seldom a dominant flavour. The winner's signature drink was fantastic - yes, he used egg, but to make a sort of a thin espresso-based custard. Very clever.

Abe, your comments really hit the nail on the head. I've only got a few more things to add:

-Explain your blend, but if you use descriptors, make sure that what is in the cup matches up to them. Some competitors used every single descriptor under the sun - fruity, chocolatey, caramel, bright, spicy, mandarin, cassia bark ... A good description can add to the overall impression if you can deliver. An overly complicated description will do the opposite if what you serve up is dreck.

-Have a few run-throughs before. I doubt that this would ever really be a problem at national level but, at state level, some of the competitors clearly hadn't had much practice. C'mon, bosses, get behind your staff!

-Select a blend that is easy to work with. You have no idea how the machine is going to behave. (Our local LM distributors decided not to sponsor the event, so ECM stepped up to fill the void. I've got to say that they tweaked the hell out of the machines and did a great job) Don't create a blend that only tastes good on a Synesso set to 198.5F.

Cheers,

Luca

[HB]

Excellent writeup Abe! This was my third tour as sensory judge, first time as USBC judge. The thread SCAA Barista Competition - Competitor Debriefing documents my thoughts after last fall's SERBC; I would like to add a comparison of the SERBC and USBC competitors from my admittedly brief experience.

First of all, the quality of the judges' certification excelled over the last two I've attended, thanks to the hard work of Brent Fortune and co-presenters Marcus Boni and Jeff Taylor. Brent / Marcus were exceedingly well prepared, managed time effectively, and tabled side discussion when necessary without stifling comments. Even the written tests assured you had paid attention (at the first SERBC workshop they handed me a pencil as I came through the door - "What, an essay test? Are you kidding?!?"). The information presented in the USBC workshop was topical, complete, and succinct. Day two with the "mock" competitions and calibration exercises were well organized and ran darn close to schedule. (By the way, did anyone else notice that the head judges were scoring the candidate judges on several categories? I happen to glance over and saw my name... and poor marks for legibility; I cleaned up my act for the next few rounds ).

So, what separates the US Barista Competitors from the Regional Competitors?

Abe was lucky to have a very strong showing on his flight, including two finalists. I had a similar experience at my first regional, with the winner Ryan Jensen in my flight, two of the six finalists, and an exhibition competitor from Intelligentsia, Stephen Rogers, who returned to this year's USBC. My second year as judge at the SERBC was a mixed bag; below is a closeup of one of the pours during my round:


It didn't go as planned

Which leads me to the first noteworthy distinction: Not surprisingly, the expected level of consistency is much higher. Abe suggests that the winner's circle is reserved for those who regularly pull 4's ("very good"). I've not seen the final scores, but would guess that making it from the first round to the second round requires consistent 2.5's (between "average" and "good") and a couple threes.

USBC competitors use their time more effectively. For example, they would clean their station with a quick brush here, a brief wipe there, ending with almost no cleanup work. Some competitors casually straightened a tray or restacked some cloths as the last minute ticked away in hopes of getting extra "attention to details" points.

Fluidity of the presentation. A few competitors were clearly gifted public speakers, and most managed to interlace their running commentary with the necessary task of prepping their drinks. The best baristas were able to "connect" with the audience and the judges by sharing their knowledge and interests in coffee while staying focused. Those who struggled would face their station while speaking, or look downward when addressing the judges, costing them presentation points.

[Abe Carmeli]

The Drama Behind the Numbers

The full competition ranking was published yesterday. We do not have the score sheets that detail the competitors score for every single component, just the final score for each competitor in the various rounds. The real drama is in the full score sheet, and I wish the USBC made it public. The final number misses the whole dance. Oh well, let's see if we can learn something from the final numbers.

Matt Riddle, who took the championship, would not have made it to the final if not for a clerical error by the judges. I'm referring to that famous overtime penalty error with Ryan Dennhardt. This is not to diminish in any way Matt's accomplishment, which he won fair and square. Clerical errors do happen in competitions and I believe the judges made the right decision allowing seven finalists instead of the usual six because of that mishap.

Billy Wilson was awarded the highest score in the competition. He got it in round one, 835 points. However, every round is a new competition, and sadly for him, that score does not carry any weight to the next round.

There has been a total re-shuffling of ranking from the semi finals to the finals. What accounts for that? It is a combination of a few factors:

1. The coffee - it is a day older. For some coffees this is good, for others it is not so good. Competitors must bring with them coffee of different roast dates to ensure that they have a batch that is at its peak for every day of the competition.
   
2. The barista - every day is a new day. We all have our good and bad days.
   
3. The judges - mostly on the sig, but on standard drinks also. When the difference between first and second place is 25 out of 1200 points, every tiny thing counts. There is an element of subjectivity in the scoring which cannot be neutralized, and shouldn't be. The scoring rules try hard to objectify every possible aspect of scoring, but this is a sensory experience at the end of the day, which is influenced by personal preferences. There is nothing the barista can do about it other than give his best shot, and let the dice role. The fact that there was such a small point difference between first and second place is a tribute to the judges ability to calibrate their scoring.

Matt and the rest of the finalists did a great job in this competition. Their score tells the story, they raised the level at the finals from prior years. I believe that this year, for the first time, we will see a U.S. Finalist in the WBC.

Dramas are part of any competition and this is a main reason we enjoy watching them. To increase the level of drama in the competition, I think the full score sheet should be published after every round. I do not see any unfavorable consequences that should keep it from being public in real time.

[HB]

Dramas are part of any competition and this is a main reason we enjoy watching them. To increase the level of drama in the competition, I think the full score sheet should be published after every round. I do not see any unfavorable consequences that should keep it from being public in real time.

That runs the risk of influencing the latter round judges, doesn't it?

Nobody knew the rankings of the seven finalists except the scorekeepers. If it were public knowledge prior to the finals that Matt was in fact the 7th finalist, could some judges be predisposed to view him less favorably than a competitor that ranked well ahead of him in the prior round? I would like to think that the answer is no, judges don't pre-judge, but keeping rankings out of the public eye until the end of competition seems prudent, especially given that the scores can be so narrow.

[Nick]

Just for the record...

It wasn't a "scoring error."

(NOTE: my following comments are completely hearsay. They are accurate to the best of my knowledge, but they do not reflect any official USBC committee statements and are completely unauthorized)

The USBC committee (or at least part of it) received a formal written letter of complaint, pursuant to section 8.0 of the USBC competitor rules and regulations (PDF file). Ryan Dennhardt was approximately 32 seconds over-time, resulting in a 40 point penalty (5 points from each of the four sensory judges for each 30-second block over... 32 seconds = two deductions... 40 points). The Dennhardts claimed that because of the actions of runners and judges, Ryan was unfairly prevented from proceeding with his presentation. The USBC Event Manager and USBC committee members reviewed the videotape from Ryan's presentation (because it was available), and made the decision to re-evaluate Ryan's competition time and restored his 40 points that were deducted due to time penalty.

These are the facts as I know them to be.

Carry on.

[Abe Carmeli]

That runs the risk of influencing the latter round judges, doesn't it?

Nobody knew the rankings of the seven finalists except the scorekeepers. If it were public knowledge prior to the finals that Matt was in fact the 7th finalist, could some judges be predisposed to view him less favorably than a competitor that ranked well ahead of him in the prior round? I would like to think that the answer is no, judges don't pre-judge, but keeping rankings out of the public eye until the end of competition seems prudent, especially given that the scores can be so narrow.

It is a good point Dan but I disagree. You could run the same argument at the beginning of the competition. Some baristas are known to be excellent in their craft even before the competition begins. If the judges are biased by ranking, they will be biased there too. In my short experience, ranking played no role in the scoring. Give the judges some credit here. Generally speaking, publishing the score after every round is common practice in the great majority of sport art and food competitions. From musicians to athletes to chefs. This is not only done to enhance the drama, but to give the competitors important feedback. A clue where they should improve their performance in the next round.

[Abe Carmeli]

Just for the record...

It wasn't a "scoring error."

Thanks for the correction Nick. I wish they'd come out by now and officially tell us what happened there.

[Nick]

It is a good point Dan but I disagree. You could run the same argument at the beginning of the competition. Some baristas are known to be excellent in their craft even before the competition begins. If the judges are biased by ranking, they will be biased there too. In my short experience, ranking played no role in the scoring. Give the judges some credit here. Generally speaking, publishing the score after every round is common practice in the great majority of sport art and food competitions. From musicians to athletes to chefs. This is not only done to enhance the drama, but to give the competitors important feedback. A clue where they should improve their performance in the next round.

Scores? An interesting point and idea. On the other hand, there's enough for competitors to worry about... having scores published after each round would frankly create an additional layer of... stress? That being said, I think it's an intriguing idea that could be feasible.

Scoresheets? No chance. At least not during the competition. There is a lot of information about a fellow competitor that could feasibly be used to one's advantage. However, if I remember correctly, there are certain national championships (Australia comes to mind) that might release the full scoresheets after the competition. On the other-other hand... maybe doing so wouldn't be such a good idea, considering you want to send your US Champ to the WBC without having their scores published for... you get the idea.

[Abe Carmeli]

However, if I remember correctly, there are certain national championships (Australia comes to mind) that might release the full scoresheets after the competition. On the other-other hand... maybe doing so wouldn't be such a good idea, considering you want to send your US Champ to the WBC without having their scores published for... you get the idea.

I didn't think about that one, but in practice I don't think it should be a concern. It assumes that one can't improve from one competition to the other, that the WBC judges care to read the minutia detail of the competitors' past performances in their nationals, and that they will also be biased by it. I think all those assumptions are false. From my brief experience with that crowd, they couldn't care less.

[another_jim]

On the other hand, there's enough for competitors to worry about... having scores published after each round would frankly create an additional layer of... stress? That being said, I think it's an intriguing idea that could be feasible.

It comes down to who the competitions are for. The current format is designed for the baristas themselves, not for an audience. For an audience, it would be something like this:

The competitors start the final round in inverse finishing order from the qualifying rounds, some points carry over. As each competitor performs, the voice over tells the crowd how many points are needed for her or him to get into first place. The scoring is displayed live, with voiceovers "nailed that," or "oh what a mistake!"

A sports or competition audience feeds off stress.

[Nick]

It comes down to who the competitions are for. The current format is designed for the baristas themselves, not for an audience. For an audience, it would be something like this:

The competitors start the final round in inverse finishing order from the qualifying rounds, some points carry over. As each competitor performs, the voice over tells the crowd how many points are needed for her or him to get into first place. The scoring is displayed live, with voiceovers "nailed that," or "oh what a mistake!"

A sports or competition audience feeds off stress.

Haha! The funny thing is, we've talked about this. A play-by-play. It's an idea that's been thrown around among WBC folks and USBC folks.

Logistically, it's tricky. Let's see if the idea (for Long Beach 2007) takes hold for a closed-circuit feed of the USBC on monitors around the SCAA show floor... or at least, a monitor outside of the show-floor. That would provide a possible venue for a play-by-play to take place. We certainly couldn't have that happen right at the competition area.

You're right in saying that the competition "is designed for the baristas themselves, not for an audience." The fact is, we're sort of in the 2nd-generation of barista competitions. The audience-friendly competitions will emerge someday soon... either by changing and developing the current WBC/USBC's, or creating something altogether new.