SCAA Barista Competition - Competitor Debriefing - Page 2

[barry]

against the rules to pop out baskets.

Why am I not surprised... As far as producing the drink, I think the competition should allow some leeway to deviate from common practices. There is no room for innovation otherwise.

every sport has common hardware rules, innovation or not. check out the rules for just about any form of auto racing. in the USBC, competitors are not allowed to change the equipment in any way. otherwise folks would be using custom steam tips, custom showerscreens, custom baskets, etc, etc. perhaps the rules will change in the future, as this is a continually evolving event, but for right now, everyone has to play by the same rules.

[Abe Carmeli]

every sport has common hardware rules, innovation or not. check out the rules for just about any form of auto racing. in the USBC, competitors are not allowed to change the equipment in any way. otherwise folks would be using custom steam tips, custom showerscreens, custom baskets, etc, etc. perhaps the rules will change in the future, as this is a continually evolving event, but for right now, everyone has to play by the same rules.

What I fear is that we are stifling the very spirit we are trying to develop. The last thing you want is for espresso making to become a dogma. Tennis has a lot of rules, but none of them is related to how the player hits the ball, what technique he uses. As I agree that some things must be standardized in the competition, grinding into the basket is hardly changing the equipment. The basket is not really part of the P/F, it is attached to it with a clip. The only change here is in the Barista technique.

[barry]

i think there's a little concern about starting down a slippery slope wrt messing with equipment. also, pulling baskets out to dose/tamp is hardly practical in a professional setting (however handy it may be for some for brewing at home).

abe, consider getting in on judging, so you can really see how things work and maybe even help improve them.

[Abe Carmeli]

abe, consider getting in on judging, so you can really see how things work and maybe even help improve them.

I have been thinking about it. The NE Regional is held in New York next year. I will give it a try.

[Nick]

If you don't mind me jumping in on this discussion,

actually, to be completely honest, I'd rather not have Abe as a judge, if that's the sort of perspective that he's going to come in with.

The fact is, competing is really, really difficult. Think you're hot stuff in your kitchen? To put things into perspective: Abe, you've pulled hundreds and thousands of espresso shots, right? You've experimented and somewhat developed some sort of technique (which I guess involves popping out the baskets for you). Good.

Now, imagine that you're at the competition as a competitor. You have 15 minutes to set up all of your gear, so you'd better be damn sure you have every piece of everything totally laid out in your mind and on your cart. You're wearing clothes that you normally wouldn't ever be barista'ing in. 15 minutes to prepare, on a machine and grinder that you've had no more than 15 minutes (during open practice time) to spend any time on. You don't know the way the water temps will fluctuate, but you're on a Marzocco Linea 3AV, which means about ± 6*F. The machine-prep people who cleaned your station before you flushed the hell out of the machine, which dropped the brew boiler temps a good bit. Sure hope it'll recover by the time you're up!

Dial in your grind, pull some shots, lay out your stuff, and get cleaning. Then your set-up time is over, and you stand and wait for 8 minutes for the last competitor to finish, and the judges to finish deliberating over their performance.

Then you get up to go. Keep in mind, the grind that you set has been sitting for 12 minutes since you last pulled a shot. Now. Get to work. Hope you're not nervous!

---
This is but only a small nibble off of all of the unique challenges that the barista competitor faces.

I'd love to see a (at least) mock-competition requirement from all judges before they're certified to judge. The judges shouldn't be coming from a perspective of, "Okay barista... impress me." They should come in with some level of respect, understanding how hard it is to compete, and be like, "It's my honor to be your judge."

[Abe Carmeli]

I'd love to see a (at least) mock-competition requirement from all judges before they're certified to judge. The judges shouldn't be coming from a perspective of, "Okay barista... impress me." They should come in with some level of respect, understanding how hard it is to compete, and be like, "It's my honor to be your judge."

Nick,

Yes, I am disappointed with the competition. It needs a serious overhaul for the very reasons you've mentioned. I have a lot of respect for people who compete in front of an audience. I have done it myself numerous times in a different discipline, and I know very well how it feels to be on the receiving end of it. But, perhaps because I've been there, I know what it can be, and the USBC just ain't it. Please do not take it as an insult, but rather as a comment from a person who cares about it, and would like to see it reach its potential.

You want my respect as a competitor? Then make your best effort to earn it. There are no discounts there. But this is not really the competitor's fault alone. The organization needs to look at the conditions they put the competitors in. Some examples for improvements? Give the competitors 2 days before the competition to work on the equipment. Let them feel comfortable. Machine prep causing problem in temperature? change the procedure to avoid it. Time is too short? perhaps add a few minutes to the rules and see if it improves performance. In later years, once the training wheels are off the competition, gradually reduce the time back to its original.

The USBC is new in the competitive arts circuit, but such competitions have been held in other disciplines for centuries. There is a lot to be learned from them. They are in many cases a great force in developing the discipline itself. I'd like to see that happen to espresso. We will all benefit from it.

[barry]

I'd love to see a (at least) mock-competition requirement from all judges before they're certified to judge.

Mine are simple:

1) don't puke or wet yourself;

2) don't sweat in my drink;

3) serve me something tasty.

[Nick]

Yes, I am disappointed with the competition. It needs a serious overhaul for the very reasons you've mentioned. I have a lot of respect for people who compete in front of an audience. I have done it myself numerous times in a different discipline, and I know very well how it feels to be on the receiving end of it. But, perhaps because I've been there, I know what it can be, and the USBC just ain't it. Please do not take it as an insult, but rather as a comment from a person who cares about it, and would like to see it reach its potential.

Duly noted.

However, there are a few different elements here that I'd wanna mentally separate: the competition, and the competitors, and the general "state of the barista craft" in the country and in the world.

The competition? If it's fair, and after all is said and done, you really do see the best baristas rise to the top and win these competitions, then what else do you need? Whether you give them an hour, or 5 minutes, it shouldn't matter. If it's fair, and it rewards true excellence... that, the competitions do indeed accomplish.

It's flawed for sure. There are a growing number of folks (some who DO have some access to the WBC governing body) who are proposing a comprehensive overhaul of the WBC/USBC competitions. The biggest problem with the competitions are the judging, the skill of the competitors, and the "spectator sport" element.

Some examples for improvements? Give the competitors 2 days before the competition to work on the equipment. Let them feel comfortable. Machine prep causing problem in temperature? change the procedure to avoid it. Time is too short? perhaps add a few minutes to the rules and see if it improves performance. In later years, once the training wheels are off the competition, gradually reduce the time back to its original.

Why put training wheels on at all? We're not talking about children. What's the point of just giving all of the competitors a 50 point boost? The scoring is relative anyway.

The USBC is new in the competitive arts circuit, but such competitions have been held in other disciplines for centuries. There is a lot to be learned from them. They are in many cases a great force in developing the discipline itself. I'd like to see that happen to espresso. We will all benefit from it.

No argument there.

What I would add though is that not only has espresso not been around for centuries, but this level of barista craft is really in its infancy and is being developed worldwide as we speak.

It's not like the Food Network Wedding Cake Challenge that I just watched on TV a couple of nights ago, in which from a community of thousands of highly skilled pastry chefs around the country, you take the elite few and marvel at their brilliance and years of training and experience. As an M.C. at regional, USBC, and WBC competitions, I ask almost every barista how long they've been a barista. The range is from 6 months to 5 years, with the average being about 2.5 years. Once we have an industry and a culture with a good-sized pool of 10-15 year veterans of premium espresso, it'll be fascinating to see how this particular "competitive art" has developed over the years.

Do take solace/comfort though, in the knowledge that there are indeed, people who are aware of the various issues... and are working to do something about it.

That being said, the competitions aren't for everyone. Right now in this country, there are few (if any) other opportunities for a passionate barista to do something that goes beyond working behind her or his shop's espresso bar... so we see many competitors who frankly seem to have no clue about competitions. As the barista profession develops (thanks in most part to the Barista Guild of America ), we'll see the caliber of competitor improve dramatically.

[Nick]

Mine are simple:

1) don't puke or wet yourself;

2) don't sweat in my drink;

3) serve me something tasty.

Whatever Barry.

We all know your #1 thing is: Close your refrigerator door.

[lennoncs]

Whatever Barry.

We all know your #1 thing is: Close your refrigerator door.

How many competitors demonstrate good food safety practices?

Closing the door to the fridge may just be an annoyance thing but it begs the question, at least in my house where a health dept employee lives, is food safety even considered at the competitions? I dunno, I have not been to one.

Regards,
Sean

[barry]

How many competitors demonstrate good food safety practices?

if something is obvious, then it will cost the competitor points. if something is dramatically wrong, then i wouldn't be surprised to see those drinks disqualified. i've had discussions with other judges about these issues (especially after someone perspired into my drink), so it is on our minds, somewhat, during competition.

[HB]

This year I didn't have the energy or time to write-up the local SCAA barista competition like last year. I did however jot down some notes about the 2006 Southeast Regional Barista Competition's signature drinks, and once again, I would like to share some of my thoughts on how competitors might improve their game.

This time around, Abe and I had the honor of serving as finals sensory judges. Honestly I was surprised to see our names on the roaster since I assumed "home enthusiast" was an unspoken Scarlet Letter among the pro ranks. It could have been the lack of available USBC-certified judges, or their desire to give motivated judges outside the coffee profession a break. Whatever the reason, I'm grateful to event organizer Michelle Campbell and head judges Marcus Boni and Spencer Turer for (what I'll assume was) their vote of confidence.

Hearing that some competitors at the WBC complained that sensory judges didn't sample their drinks fully, I drank deeply... every cup... at least three times. The runners were irritated with my dawdling and Marcus did make note of my pondering, but I believe the extra time added confidence to my scores (especially as some defects become much more evident when the drink cools). The downside was that I blew the top off my acquired tolerance for caffeine. Tomorrow is the first day that I hope to have espresso again.

Now that my body is on the mend, my debriefing notes:

• Talk about the coffee, not yourself - There are a number of required elements of the barista's performance. For a good Professionalism/Dedication/Passion score, you must demonstrate knowledge of your coffees. Why did you select the blend components? What does each contribute to the cup? Some competitors shared interesting personal stories at the expense of coffee-related discourse, others dumb-downed their descriptors to the point of platitudes. I wondered if some competitors have decided it's too risky to be specific.
  
• Palate cleansers, please! - Or at least water! The Ideal Palate Refresher offers specific suggestions. I emptied the presented glass every time and still craved soda crackers. It helps the judges between courses and demonstrates your appreciation for this important ritual.
  
• Presentation cues - It would be helpful if commentary directed to the sensory judges were delineated from those directed to the audience or technical judges. We're instructed to listen intently to the barista, but sensory judges are also told to take notes to support their scores. Clear signals of a transition between the explanation of the coffee characteristics and the preparation elements would help assure the right judges get the message.
  
• Make a final checkout list - This oversight is likely limited to the regionals where invariably a competitor or two stops the clock when they have remaining time. Enough time to clean up their station and increase their technical score. I assume it's one of those "d'oh!" moments combined with the urgent desire to finish; put a checklist near the timer that reminds you of last-minute point getters.
  
• Stick around for the debriefing - few baristas took the opportunity to question the judges' scoresheets. It's a good learning experience for all involved; I know that I asked some competitors, "Why did you do XXX? You surely knew it would be a mandatory markdown." The answer: "I was running out of time". To wit, I've asked Marcus for a pie-chart that shows the point breakdown. It's worth knowing what you're giving up in order to meet the 15 minutes. The better course of action when things go bad may in fact be to take the time deduction.

This competition was the first where signature drinks were served first by several competitors. I don't have a strong preference on the presentation order, though it is more difficult to keep your taste acuity sharp if the signature drink is particularly spicy. Again, palate cleansers are appreciated; Abe brought his own and munched during the calibration period between flights.

PS: HB members Bob Barraza and Kevin Kratwald were also SERBC judges this past weekend. Gentlemen, you are welcome to add your debriefing suggestions.

[another_jim]

Hearing that some competitors at the WBC complained that sensory judges didn't sample their drinks fully, I drank deeply... every cup... at least three times.

This may not be a good idea; and the people doing the complaining may not know a lot. I'm reading my way through the tasting literature, not just coffee, but all sorts of things, including the food science stuff.

There's not a lot they agree on; but the one thing they do is that the first and perhaps second taste are the most accurate. After that the taste and odor sensing nerves overload; odor and bitter/savory receptors bind the molecules and are out of commission for a few minutes, while the sweet, sour and salt ion based taste sensors need to recharge their batteries. The basic drill recommended thoughout is one or two small sips, then a rest before moving on. In this sense, barista competition with its pauses between courses and competitors may actually be more accurate than, say, cupping 10 coffees in one sitting.

[Abe Carmeli]

Just a few comments:

Know your coffee

With the exception of perhaps 2 baristas in the SERBC, none of the competitors have a chance of passing the first round in the USBC. The problem was not so much the new faces in the competition, but the fact that many of them knew very little about coffee, and are poorly trained in evaluating their own drinks. A great number of them were using the same blend (Toscano), perhaps because it is the only coffee they know. Some explained what we should taste in the coffee before they served their espresso. In most cases, none of those flavors we expected were in the cup. I assume some of them did taste their coffee during practice, and you would ask yourself why didn't they detect it? The answer is: they are clueless. With very little training on how to cup coffee, they were reading the label on the box, and not their taste buds.

Steaming

The cappuccinos were universally milky. You would think with so much milk there will be some sweetness to them as a trade-off, but there wasn't. This one goes back to evaluating your drinks. I am not sure there was a universal problem with steaming technique, but the end result was lack of sweetness, and not enough coffee taste. Only in two cases I had both. One competitor used a separate blend for cappuccinos, and it did make a difference.

[Bob Barraza]

This was my first attempt at judging, hence no frame of reference, just first impressions. I am glad that I did the Judges Calibration course. It was quite intense, and I learned precisely what to look for in the competition as well as tips for my own home brewing. I would encourage others to participate as well.

As complex and subjective as tasting can be, I think that the process of breaking down the attributes of the drink help make the process more objective. For example, the espresso is judged separately for appearance/color, consistency/persistency of crema, as well as taste and mouth feel. Knowing this, it surprised me how poor the crema was in the drinks that I was served. Even finalists presented me espresso in the first round which had little or no crema. Perhaps this problem is exacerbated by the protocol itself which requires that four singles be served at the same time. Most of us are used to pulling doubles, ristrettos, and we serve each drink immediately.

At the end of the day, my hat goes off to all competitors. It is quite a challenge to serve twelve drinks within the timeframe, particularly when you are on stage with seven judges nit picking every step, usually with unfamiliar equipment. Perhaps an unrealistic protocol, however, it does seem to highlight excellence from mediocrity.

[miKe mcKoffee]

Great thread, thanks! Helps prepare my mindset for next months NWRBC and my first judging.

[Omniryx]

It seems to me that some of the same ideas are being shouted about over in the Ken Fox "who gives a sh**" thread are being talked about here with measured gentility. Much nicer over here.

Nick said something a few posts back that perked up my ears. Here it is:

actually, to be completely honest, I'd rather not have Abe as a judge, if that's the sort of perspective that he's going to come in with.

I don't want to lift his commentary out of the perspective in which he intended it but it seems to me that this kind of thinking represents a certain rigidity of mindset that may or may not be helpful to the competitions in general. This is the same concern that I raised in the other thread about the downside of "calibrating." It potentially diminishes the value of new ideas, perspectives, and insights.

Competitions at this point seem marked by artifice. They aren't focused on making the absolutely best coffee of which the barista is capable, as Abe has clearly pointed out. They aren't focused on barista-life-as-lived, as others have demonstrated. So why the "five star" approach if it is totally outside the range of occupational expression of virtually all the contestants? Just to make it hard? And why do most of the judges and organizers seem so resistant to alternative ways of doing things?

[Abe Carmeli]

Nick said something a few posts back that perked up my ears. Here it is:

actually, to be completely honest, I'd rather not have Abe as a judge, if that's the sort of perspective that he's going to come in with.

Funny thing, I ended up judging him after all (he won the SERBC under my watch). I'm not sure how strongly Nick stands behind that statement today, but it was amusing to be reminded of it. I forgot all about it.

And to the main point of your post. In general I am for relaxing the rules on how to make the drinks, and judge it in the end in the cup. I would like to see more innovation coming out of the USBC. Treat it like the world baking competition. They are judged on the cleanliness of their procedure and on the baked goods - taste & visual. They can get there any way they want within the time limits of the competition. One major improvement is simple: Do not require all 4 drinks to be served to the judges at the same time. Espresso should be consumed immediately after it is prepared. Having those drinks sit at the barista station for two minutes before the judges taste it is unfair to the baristas.

[Nick]

Nick said something a few posts back that perked up my ears. Here it is:

actually, to be completely honest, I'd rather not have Abe as a judge, if that's the sort of perspective that he's going to come in with.

I don't want to lift his commentary out of the perspective in which he intended it but it seems to me that this kind of thinking represents a certain rigidity of mindset that may or may not be helpful to the competitions in general. This is the same concern that I raised in the other thread about the downside of "calibrating." It potentially diminishes the value of new ideas, perspectives, and insights.

Competitions at this point seem marked by artifice. They aren't focused on making the absolutely best coffee of which the barista is capable, as Abe has clearly pointed out. They aren't focused on barista-life-as-lived, as others have demonstrated. So why the "five star" approach if it is totally outside the range of occupational expression of virtually all the contestants? Just to make it hard? And why do most of the judges and organizers seem so resistant to alternative ways of doing things?

You dirty bastard! You're a jerk! You have no idea what you're talking about! How dare you attack me!



Seriously though...

My post was not simply an open opinion... it was in response to the fact that Abe's posts were not in alignment with USBC judging guidelines. Opinions are opinions, but if you're judging, you should follow the judging guidelines. That said, I have no issue with all of the "If I ruled the world, I'd change the competitions like this..." ideas. However, everyone has an opinion, and certain opinions do, necessarily, have more impact on the competitions than others... which is why you'll see the new WBC rules-changes very soon, to take effect at USBC-sanctioned regionals that occur after the first of the year.

[HB]

It seems to me that some of the same ideas are being shouted about over in the Ken Fox "who gives a sh**" thread are being talked about here with measured gentility.

Agreed. I'll respond to your question here as they seem more relevant (a sort of "judge debriefing").

The judges do independently fill out the scoresheet completely, but the calibration is the head judge's opportunity to challenge your scoring. Their challenge may take the form of pointing out something you missed. They don't require you to modify your score thereafter, but that's up to each judge's discretion. You are required to get the head judge's approval for any zeros.

You'll note that I acknowledged that all the judges may be busting their butts to be fair and objective. But you can see the problems created by the procedure you cite above. Imagine that I am a brand spanking new judge; the ink barely dry on my credentials. A head judge with a forceful personality takes exception to my scoring. How likely is it that I am not going to be swayed by her or his "suggestions"? Especially if I want to be on another judging panel...

And what if the new judge saw something that the head judge missed instead of the other way around? Is this impossible? Are head judges incapable of error?

Of course, one might reply that this is precisely why there is a head judge; to teach noobs how to do their work. But that assumes that there is only one standard or perspective for judging and that the head judge has the ultimate handle on it. One might counter that fresh perspectives, attitudes, and insights would be a good and healthy thing.

The head judges are often the same people who run the certification workshop. While some scores are more subjective than others, scoring consistency is a prime objective of the workshop. For example, the scoring of crema consistency and color should not vary much depending on the judge in question; if four judges evaluated the same cup, I would challenge any deviation more than than 1/2 point. On taste and tactile balance, well, that's harder to dispute, though I would be surprised to hear about disagreement in excess of one point for the same cup.

Nick and others have hinted about changes on the horizon for SCAA barista competitions. I know nothing about them, but I'm confident they'll be for the better.

It seems to me that a higher level of perceived credibility would be maintained if, once a judge was qualified, her or his judgments were not subject to "calibration." Of course, if there was significant variance among scores when they were announced, individual judges might be called upon to explain their reasoning. Would that be a bad thing?

I've never felt any pressure to change a score, though I have been asked to justify them on occasion. That comes with the learning process and is part of the reason I volunteer. The more I think about it, the more appealing Abe's suggestion to add instant scoring sounds. However, a lot of the so-called calibration is really the head judge making certain nobody screwed up. For example, leaving a score blank. There are also several scores that are Yes/No (e.g. correct cup, all drinks served at once, served with accessories) and it would be an embarrassing oversight if they were inconsistent, barring unusual circumstances (e.g., a competitor spills a drink and chooses not to serve one judge).