SCAA Barista Competition - Competitor Debriefing

[HB]

(Split from SCAA Barista Competition - SERBC 2005 by moderator...)

After the competition, the baristas were invited to meet with the judges to review their scores. I was fortunate to be among the sensory judges for three of the six finalists (Lem Butler, Mandy Catron, Claudia Raymo-Quirk). Understandably most of the baristas want to talk with the senior judges, not a local nobody like me (awww-w ), so I'll instead share my thoughts with you. None of these points are revelatory, but I hope they give you some insight into what judges are looking for.

Note: All of these comments are based on actual events. Names have been (mostly) omitted.

• Read the rules & regulations
  
  Yes, it would seem obvious, but clearly some competitors skipped this important step. For example, no additives to the coffee (e.g., cayenne pepper, ginger) are allowed during the brew process because it can taint the machine's internals with hard-to-remove flavors. It is also worth reading the judges' rules & regulations. Why? That's the next comment...
  
• Make the judges' job easy
  
  Know what the judges are looking for by reading the regulations. Be aware of that the judging protocol includes stipulations that aren't typical for social situations. For example, the judges must wait for a sign from the competitor that they are allowed to start sampling, either by explicitly saying something like "Please enjoy your drink" or clearly indicating that he/she is moving onto the next part of their work (e.g., "Now I am going to start your signature drinks."). Because the judges are required to listen intently to the competitor, eye contact during key points is essential. Sometimes competitors would chat with the MC about non-essential points (e.g., respond to questions like "What is your favorite drink?"). Providing clear clues that an important point is being made helps the judges score your performance more accurately. Also keep in mind that judges are strongly encouraged to jot down notes about their evaluation should the head judge or competitor later ask for clarification. Providing a "lull" in the conversation immediately after serving gives judges proper time to evaluate the drink and take notes. If the competitor begins speaking to the judges after serving, they are required to stop and listen. If a judge is engrossed in writing notes on the score sheet, they may miss the beginning of the competitor's commentary. Again, keep the important points clearly framed in your presentation so the judges don't miss them.
  
• Dress for the occasion
  
  If I got a dollar every time somebody said "five star restaurant" when explaining what type of performance merits top scores, I would be able to afford an evening at such an establishment. Given this expectation, you would think that competitors would choose their apparel accordingly, but some were attired in jeans with untucked and rumpled shirts. The ladies generally understood the message better than the gentlemen baristas, which helps their "professionalism and passion" score.
  
• Bring everything for the setting
  
  The judges' table wasn't a finished piece really, more like a plank of painted plywood. I suggested to organizers that a nice marble top would really look sharp, to which they replied that it was the competitor's responsibility to determine the look. Their point is well taken and the majority provided a nice linen tablecloth and cloth napkins. Lem Butler's setting was a surprise -- although he was dressed to the 9's, he used brown paper napkins and nothing else. At first glance I thought they were handtowels hastily grabbed from the men's room. Another common faux pas was the choice of waterglasses: huge 16+ ounce glasses filled with three ounces of water. Not only does it look out of place compared to the setting, it also potentially means being marked down for poor service for not refilling glasses.
  
• Don't serve drinks you know are seriously flawed
  
  Every competitor watches the clock and several did go over the allotted 15 minutes. Even so, the maximum overtime deduction is 20 points, which is easily lost in the taste evaluation scores. A couple times I saw the look on the face of the competitor as they placed the drinks on the tray and it was clear they were making a mental calculation -- they knew that two or more of the drinks had sat way too long, or the extraction was terribly off the mark. I wanted to say out loud, "Remake it! Remake it! You still have enough time!" which of course isn't permitted. There's 60 points riding on each espresso for the four sensory judges, 48 points on the taste evaluation alone. Going up to two minutes over will cost 20 points, but losing even 1 point on the taste evaluation multiplied by the number of judges and weighting is 32 points. Bottom line: If you know it is no good, don't serve it even if you're short on time (up to 17 minutes).
  
• The unvarnished truth is in Part V, Judge's Total Impression
  
  Workshop instructors pointed out that judges have more freedom in this score, since it is subjective by its very nature. Call it that je ne sais pas factor if you like, but this part is where the judge can show how they would "vote" overall. If a given drink was disappointing but overall the competitor was exceptional, a judge can send that message of approval in this score. Conversely, if the performance was deficient, this section will show it unfiltered by the particulars of one drink.

[barry]

Make the judges' job easy

Know what the judges are looking for by reading the regulations. Be aware of that the judging protocol includes stipulations that aren't typical for social situations. For example, the judges must wait for a sign from the competitor that they are allowed to start sampling, either by explicitly saying something like "Please enjoy your drink" or clearly indicating that he/she is moving onto the next part of their work (e.g., "Now I am going to start your signature drinks.").

also remember the tech judges will be in your personal space. some competitors are really good about working with techs (see http://www.intelligentsiacoffee.com/roasting/wholesale/03-17-2005 for a photo of me getting up close with matt riddle at the usbc); and some aren't. we all gotta be there, so it's in the competitors' best interests to accommodate us.

Don't serve drinks you know are seriously flawed

Every competitor watches the clock and several did go over the allotted 15 minutes. Even so, the maximum overtime deduction is 20 points, which is easily lost in the taste evaluation scores. A couple times I saw the look on the face of the competitor as they placed the drinks on the tray and it was clear they were making a mental calculation -- they knew that two or more of the drinks had sat way too long, or the extraction was terribly off the mark. I wanted to say out loud, "Remake it! Remake it! You still have enough time!" which of course isn't permitted. There's 60 points riding on each espresso for the four sensory judges, 48 points on the taste evaluation alone. Going up to two minutes over will cost 20 points, but losing even 1 point on the taste evaluation multiplied by the number of judges and weighting is 32 points. Bottom line: If you know it is no good, don't serve it even if you're short on time (up to 17 minutes).

the difficult thing is all four drinks need to be remade, lest the "good ones" go bad waiting for the remake.


--barry "and close the fridge door!"

[HB]

Thanks Barry for chiming in with the technical judge's viewpoint! I have a few more tidbits to share, please feel free to add to this list.

• Plan for disaster
  
  Black holes spontaneously form around competitions and warp the time-space continuum. Competitors who easily finished in 11 minutes during countless practice sessions realize in competition that they have three minutes and they're only starting their signature drinks. One way of mitigating this effect is disaster recovery planning. For example, have extra cups at the ready in case you're caught by a series of bad pours. If something does go wrong at the last second (e.g., you knock over the sugar jar while serving), remember that the clock cannot be stopped until all four judges are served. Place the four drinks down and then start cleanup, even if the presentation suffers. Judges are less likely to mark down minor tiding up, but time overages are mandatory mark downs. In other words, think about how you will respond when things go wrong.
  
  If I were planning for a competition, I would also schedule "gremlin" trial runs. That's where you turn your back for a moment so your sadistic friend can mess with the grinder, move your ingredients or tools around, etc. Your job then is to recover and still finish on time. One competitor had two grinders where the second was specifically tuned for their speciality drink. Problem was the competitor was so nervous, they started with the wrong grinder, panicked, then starting re-dialing in. In such cases, a checklist and labels are your friends.
  
• Plan your preparation time
  
  Technical judges are brutal about the startup appearance of your station. They figure that you've had a leisurely 15 minutes to polish and clean, so there's no excuse why it shouldn't be impeccably organized. Jeff Taylor mentioned in the workshop and the debriefing that top competitors will have their tray(s) labeled with the exact location of every item. They do not want to waste seconds fumbling for a spoon. Additionally they will have a checklist of every item that should be on their tray and its proper location in the station. Details like vacuuming the grinder, dusting with an air canister, and verifying that even the towel edges are straight and facing forward don't escape their attention. Setup is one of the few areas where you have complete and absolute control -- there's no blaming a wandering grinder setting on the HVAC. With careful planning, you can pick up points that the majority of the competitors will leave on the table (literally).
  
• Stay in character
  
  This is really picky, but remember the "five-star experience" that everyone keeps reminding the judges about? That level of professionalism and courtesy may come natural to seasoned waiters and waitresses at fine restaurants, but let's be honest, it isn't the norm at your average coffeeshop. It's very easy to lapse into "how ya doin'" familiarity, especially if you're dressed like it's a workday. Friendliness is definitely encouraged. Choose a balance between formality and friendliness that's consistent with your own personality. To give a concrete example, below is my writeup of Sammy Piccolo's performance in the 2005 WBC:
  
  

  I was sitting in the front row next to Abe Carmeli and Andy Schecter for the first third of the finals. Andy had to dash off to the airport... and he missed a great presentation by Sammy. The clock ticked down almost halfway and he was only serving his first espressos. I quietly commented to Abe, "He's running late. If Sammy finishes in time, I'll be impressed."
  
  Well, I was indeed impressed. While the first 2/3rds of his routine was leisurely and chatty, he transformed into a controlled cyclone for the last four minutes, banging out his "three hemispheres" specialty drinks in barely three minutes (most competitors spent almost half their time just on those four drinks). He threw up his arms and called "TIME!!!" with less than 10 seconds on the clock. It was a great juxtaposition of five-star cafe service and rush hour hustle.

  
  Sammy demonstrated an interesting phenomena: Judges can be just as impressed or even more impressed by a competitor who overcomes problems and still finishes strong compared to another competitor who performs swimmingly. Some of the murkier scores like "understands grinder" and "passion and professionalism" are truly brought to light when things go wrong (not to suggest that you should intentionally screw up just to demonstrate your ability to recognize problems and adapt accordingly ).

[barry]

Plan your preparation time

Technical judges are brutal about the startup appearance of your station. They figure that you've had a leisurely 15 minutes to polish and clean, so there's no excuse why it shouldn't be impeccably organized. Jeff Taylor mentioned in the workshop and the debriefing that top competitors will have their tray(s) labeled with the exact location of every item. They do not want to waste seconds fumbling for a spoon. Additionally they will have a checklist of every item that should be on their tray and its proper location in the station. Details like vacuuming the grinder, dusting with an air canister, and verifying that even the towel edges are straight and facing forward don't escape their attention. Setup is one of the few areas where you have complete and absolute control -- there's no blaming a wandering grinder setting on the HVAC. With careful planning, you can pick up points that the majority of the competitors will leave on the table (literally).

this is especially critical in the finals. everyone who makes it to the finals should be very capable, so the judges will be looking for any little justification to deduct points. i remember at the usbc that everyone was so top notch that sometimes it came down to the "little stuff" to determine who was doing a better performance.

if the competitor messes up, then points come off. how well the competitor recovers helps determine how many points come off. don't screw up on purpose to show your "recovery ability". if you have to adjust your grinder during competition, absent an external factor which would necessitate such a change, then you'll get dinged. if you *don't* adjust your grinder when you really need to, then you'll get dinged *more*.

[Abe Carmeli]

A pour that didn't go as planned

Ouch, that one really hurts. I'm sure this is the worst shot that contestant ever pulled, and it happened in competition. These kind of colossal meltdowns are more likely due to poor competition training and prep work.

[barry]

These kind of colossal meltdowns are more likely due to poor competition training and prep work.

that hardly qualifies as a "colossal meltdown" in competition.

[Abe Carmeli]

that hardly qualifies as a "colossal meltdown" in competition.

I would call it that if I were pulling that shot in my kitchen. That must be true to the 10th power in a competition. I guess my colossal meltdown is not yours.

[barry]

unlike the pass/fail grading in your kitchen, the judges have a 0 to 6 scale to use. if nothing under a 6 is acceptable, then what's the point in competing? in your kitchen, you can take your time and put your full concentration on the task at hand, and take 5 minutes to make your morning double. in competition, all that goes out the window. most competition shots range between 2 and 4, with a rare 5. i've yet to see a sensory judge pass out from flavor ecstasy, so i'd have to say there have been even fewer 6s.

based on that photo, i'd give a 1.5 or maybe a 2.


--barry "be happy that the competitor hasn't perspired into your drink"

[Abe Carmeli]

most competition shots range between 2 and 4, with a rare 5. i've yet to see a sensory judge pass out from flavor ecstasy, so i'd have to say there have been even fewer 6s.

based on that photo, i'd give a 1.5 or maybe a 2.


--barry "be happy that the competitor hasn't perspired into your drink"

It is a shame, and I must say unacceptable. Something is off here. If time is the culprit, then more time should be given to the competitors to prepare their drink. But I assume that the time limitations are built to simulate a real working coffee house environment. In such case, the problem is poor preparation by the competitors. It is highly unlikely that that shot is even remotely the best that competitor can do, in his own coffee shop and under the same time restrictions. We would not accept such a poor performance from any top tier professional in the arts, sports, or in the kitchen. These guys are professionals in their fields and they are competing for a regional championship. They need to prepare better for those competitions. As long as such performances are accepted as the norm, the norm will stay at that level. That whole field is in its infancy, and it will take some time until that awareness sinks in, but we need to call it as it is.

[barry]

In such case, the problem is poor preparation by the competitors. It is highly unlikely that that shot is even remotely the best that competitor can do, in his own coffee shop and under the same time restrictions. We would not accept such a poor performance from any top tier professional in the arts, sports, or in the kitchen.

you are extrapolating more from that photo than is, perhaps, warranted. we don't know if those shots were rejected or not; we only have that one instant in the photo. we don't know if that's the very tail end of an otherwise lovely pour; perhaps the competitor just got distracted for a moment, and at that moment, the photo was taken. we don't know if those shots tasted bad or just not great.

if the shots were served, it is probable that the competitor lost points because of their quality. that, to me, does not constitute a meltdown. you're assuming the competitor was 6 point material all along. sorry competition isn't like that, with coffee or anything else. for every 1st place competitor, there's going to be a last place one. does this mean the competitors who aren't at the very peak of performance shouldn't bother to try? the barista competitions are very demanding, on purpose, but not so demanding as to discourage people from competing, on purpose.


--barry "even the best baseball players miss the ball over 60% of the time"

[Abe Carmeli]

you are extrapolating more from that photo than is, perhaps, warranted. we don't know if those shots were rejected or not; we only have that one instant in the photo.

I'm actually not referring to the photo anymore, but to your comment that the average shot in a competition is between 2-4 points (out of max 6). It is unlikely that professional baristas that pull hundreds of shots a day cannot muster a very good or an excellent shot. They can, they just can't do it in a competition. Have you noticed how many personal, world and Olympic records are shattered in the Olympic games? How many athletes break at the very least their own personal best in the competition? This is what those competitors should be shooting for. It's all about preparation and training. They are way behind. You want an example closer to coffee, check out baking competitions (which are much more difficult). You do not see such poor performances there.

--barry "even the best baseball players miss the ball over 60% of the time"

Yes, but they also miss the ball in practice. The question is: can you deliver your best performance in competition? In sports the answer is yes in many cases. There lies the difference.

[HB]

you are extrapolating more from that photo than is, perhaps, warranted. we don't know if those shots were rejected or not; we only have that one instant in the photo.

I was one of the sensory judges for that particular preliminary round. To put it delicately... that photo alone accurately captures what you would expect. But in all fairness, the regionals are the first stepping stone of the overall competition and as for a regional competition of any nature, there is a wider range of skills and finesse under the stress of performance. This brings me back to your earlier point:

everyone who makes it to the finals should be very capable, so the judges will be looking for any little justification to deduct points. i remember at the usbc that everyone was so top notch that sometimes it came down to the "little stuff" to determine who was doing a better performance.

Despite that some of the drinks were disappointing, I admire those who are willing to work in front of an audience and seven judges peering at their every move. I'm barely willing to try it myself in the Unsanctions Pro-Am. Honestly, I would be very pleased if I could perform as well on stage as I do lounging in my kitchen.

[barry]

I'm actually not referring to the photo anymore, but to your comment that the average shot in a competition is between 2-4 points (out of max 6). It is unlikely that professional baristas that pull hundreds of shots a day cannot muster a very good or an excellent shot. They can, they just can't do it in a competition.

i'll agree 100% that every single competitor i've judged could have done better, but that's the nature of anything, isn't it? there's a huge difference between standing on the sidelines saying/thinking "oh, he should have done this" and actually doing it. coaches coach and players play, and there's always room for improvement. the majority of shots in competition fall between 2 and 4 (especially in regionals). i would say there are more 5s at the USBC and WBC. remember that there're no preliminaries for the regionals, so baristas of every level of experience/knowledge can be in the line-up. there have been several times, while judging, when i just wanted to scream out "NO! don't do that!", but, alas, i am a judge, not a coach.

also, there are complications in the way the contests are run and judged which can have a negative impact on scoring. all four drinks must be served at the same time. i have watched a number of beautiful shots die in the cups while waiting for milk or the second set of shots. we can't judge the shot based on the pour alone; they are judged as they are presented (espresso), or as they appear just before the milk is added (espresso base). a shot that might be a 5 or even a 6 as it pours into the cup, drops to a 3 or 4 (or worse) before it is judged.

[Abe Carmeli]

also, there are complications in the way the contests are run and judged which can have a negative impact on scoring. all four drinks must be served at the same time. i have watched a number of beautiful shots die in the cups while waiting for milk or the second set of shots. we can't judge the shot based on the pour alone; they are judged as they are presented (espresso), or as they appear just before the milk is added (espresso base). a shot that might be a 5 or even a 6 as it pours into the cup, drops to a 3 or 4 (or worse) before it is judged.

That particular rule has bugged me to no end. Why is it so important to serve all judges at the same time, especially when it is clear that it compromises the drink? There is room for improvement in the contest rules for sure.

[HB]

Why is it so important to serve all judges at the same time, especially when it is clear that it compromises the drink?

I asked the same question to the head judge, Jeff Taylor. He replied the competition isn't about optimizing the rules to maximize the quality of the drink or recreating a cafe production line, it's to simulate the mythical "five star" experience. As a sensory judge, I wouldn't mind being served out of order, but it would detract from the service aspect. I suggested that baristas should consider pulling all the shots at the same time and apparently at one time they did just that. However they soon discovered that the time out of the portafilter affected the quality of the shot too, plus they got dinged on technical points, so everyone reverted back to sequential shots.

Hey Barry, what about popping out the baskets and prepping a couple? It's not an uncommon practice among home baristas... are there competition rules against it? I know competitors aren't allowed to introduce new baskets or modify the machine, but technically they aren't.

[Abe Carmeli]

Why is it so important to serve all judges at the same time, especially when it is clear that it compromises the drink?

I asked the same question to the head judge, Jeff Taylor. He replied the competition isn't about optimizing the rules to maximize the quality of the drink or recreating a cafe production line, it's to simulate the mythical "five star" experience.

Jeff's reply does not add up. That mythical Barista exists only in the delusional mind of whoever wrote those rules. If indeed it were about the service and not the drink, how come the allocated points for service performance is so minuscule in comparison with the technical and sensory evaluation? The whole thing is half baked and inconsistent. Who cares about the service if the shot sucks?

[barry]

That particular rule has bugged me to no end. Why is it so important to serve all judges at the same time, especially when it is clear that it compromises the drink? There is room for improvement in the contest rules for sure.

it bugged me, too, until i realized that holding the bar just slightly out of reach always meant there was room for improvement. i've only seen one or two competitors who've properly managed their time wrt espresso prep/brewing. making things difficult lets the judges see how well the competitor deals with the challenge. if you're really really good, you can grind/dose/tamp/load and start brewing in about 20 seconds (or less), which means the longest the 1st set has to sit before serving is about 20 seconds. the first year i was judging, i could tech judge with one stopwatch because no one would get the 2nd set going before the 1st set finished. after spending much of last year encouraging competitors to improve this aspect of their performance, at this year's USBC i had to use two stopwatches. the overall level of performance at the USBC improved remarkably between 2004 and 2005, and i expect to see similar improvements in 2006. in 2004, it wasn't uncommon to see competitors using only one of the three groups available. in 2005, iirc, there were a couple of competitors who used all three groups. i still think there is room for improvement in several areas.

[HB]

Who cares about the service if the shot sucks?

I can't speak for the authors of the rules, but I bet they would say that a truly talented barista will overcome the competition's limitations (e.g., time, equipment, and service rules) and still produce a superior shot. I did notice that one barista who advanced to the finals learned a lesson in the preliminaries on this very topic -- the consequences of spending too much time between the two sets -- and did indeed adapt in the finals.

[barry]

Hey Barry, what about popping out the baskets and prepping a couple? It's not an uncommon practice among home baristas... are there competition rules against it? I know competitors aren't allowed to introduce new baskets or modify the machine, but technically they aren't.

against the rules to pop out baskets.

[Abe Carmeli]

against the rules to pop out baskets.

Why am I not surprised... As far as producing the drink, I think the competition should allow some leeway to deviate from common practices. There is no room for innovation otherwise.