Dan Kehn said:

The judges do independently fill out the scoresheet completely, but the calibration is the head judge's opportunity to challenge your scoring. Their challenge may take the form of pointing out something you missed. They don't require you to modify your score thereafter, but that's up to each judge's discretion. You are required to get the head judge's approval for any zeros.

You'll note that I acknowledged that all the judges may be busting their butts to be fair and objective. But you can see the problems created by the procedure you cite above. Imagine that I am a brand spanking new judge; the ink barely dry on my credentials. A head judge with a forceful personality takes exception to my scoring. How likely is it that I am not going to be swayed by her or his "suggestions"? Especially if I want to be on another judging panel...

And what if the new judge saw something that the head judge missed instead of the other way around? Is this impossible? Are head judges incapable of error?

Of course, one might reply that this is precisely why there is a head judge; to teach newbies how to do their work. But that assumes that there is only one standard or perspective for judging and that the head judge has the ultimate handle on it. One might counter that fresh perspectives, attitudes, and insights would be a good and healthy thing.

It seems to me that a higher level of perceived credibility would be maintained if, once a judge was qualified, her or his judgments were not subject to "calibration." Of course, if there was significant variance among scores when they were announced, individual judges might be called upon to explain their reasoning. Would that be a bad thing?

As the editor of espressomap.com, I'll note that had I started this project 3 or 4 years ago, most of the map would be empty. Now you've got nearly 120 top-notch cafes around the continent, with more being added weekly.

When I've spoken with most of the owners, their common commitment to quality stems in part from attending or participating in regional competitions, where they come away motivated and more passionate about making better espresso in their locales.

Omniryx wrote:Were the competitions dependent on admission revenues, the planners and operators would be a great deal more responsive.

Of course if the SCAA Barista Competitions did charge admission there'd be the corresponding deluge of complaints from people having to pay just to watch! Not saying your comments aren't valid as far as making it a better experience for the audience.

Mike said:

Of course if the SCAA Barista Competitions did charge admission there'd be the corresponding deluge of complaints from people having to pay just to watch! Not saying your comments aren't valid as far as making it a better experience for the audience.

Oh sure. You can't win with things like that. And there are those who would say that pandering to the paying customers would lower the quality of the competition. That doesn't seem to happen with Wimbledon, though.

Dan has sent an email to Nick and to me requesting that we cease and desist. Having just now gotten back online (I am, afterall, in France, and do not have constant internet access) I see there have been numerous responses to my last post. I am going to stop responding to any and all things in this thread in accordance with Dan's wishes.

My only final comment is that I continue to believe the great bulk of what I posted in my intial post, and found only the title (poorly chosen as an attention getter, something made clear in the first sentence of that first post) out of bounds. So be it.

I'm not going to even read this thread from now on since the temptation to do something that might aggravate our great patron, Dan, is not something I want to deal with.

ken

I think that many of Will's criticisms are right on.

I have long felt that the Barista Competitions leave something to be desired as a spectator activity. Heck, even I can barely stand to watch them for more than a few hours!

I've thought about this a lot; I've watched sporting events, food network contests, etc. etc. with this in mind and the thing that makes them interesting is either 1. inside knowledge of what is going on or 2. someone using their inside knowledge to explain what is going on (color commentary) or 3. rooting for the home team.

I hear Will's comment about the casual spectators not being included, and it's something I always struggle with as an MC. The insiders are there because they are into it. The friends and family are there to root for their home barista. The spectator naturally feels left out, if they belong to neither of those camps.

I believe the answer is "color commentary". In fact, I did this at the first-ever SERBC... I would constantly be saying things like "the competitor has chosen to use tulip shaped demitasses. She probably feels that this accentuates the aroma of the espresso, which is one of the things the judges are looking for" or whatever. I think it made a better show. However, very understandably, some felt that this patter distracted the judges and the competitor.

At this point, I will say that so far the contests HAVE been "all about the competitor". The organizers of the USBC, WBC, and regional competitions all do it primarily out of love and respect for the professional baristas of the world. (I do believe that barista-ing is a profession.) When a judgement call needs to be made, the usual question is: "what would make this a better experience for the competitors?" These are young, passionate people who deserve to be recognized and celebrated, and for this reason the whole competition tries amazingly hard to be fair and supportive to the competitor.

It is with these points in mind that I speak to Will's excellent criticisms:

1. The camera coverage sucks. Spectators cannot see what is going on, especially with the judges hovering like angels over the Christ child.

Okay, okay. Religious iconography aside, I agree fully. We tried to fix this at the 2005 SERBC, and hired a broadcast-quality film crew to document the event. The event was broadcast live on a big screen, and also on a flatscreen in the foyer. The bad news is that that little idea cost us $8K or so, which we never were able to recover. The good news is we turned some of that footage into a cool movie about the Competitions!! (available here)

2. Too much importance is placed on spiffy table settings, suave movements around the machine, a glib line of patter, and (God help us) mopping the mess off the counter.

I actually disagree with this one. A clean, well presented culinary experience is central to the food arts, if you ask me. The idea is to get a sense of the barista's aesthetic presentation and ability to relate to customers. Both are important, in my view. If it helps any, though, the scoring form emphasizes drink quality over presentation.

3. The judging system is ridiculous. The notion that judges need to go into a huddle in a corner and "calibrate" before announcing their scores removes any semblance of independent impartiality. Perhaps the judges are, in fact, trying as hard as ever they can to be fair; they will never, ever be perceived to do so as long as they decline to score and announce their scores independently. Even the illuminati have begun to have serious reservations, as Mark documented in some detail recently.

The judging system is not perfect. The calibration phase exists to try to eliminate the "Russian Judge" factor, where a judge is unfairly disjointed from the rest of the pool. Since there are so few taste judges, if one judge arbitrarily slams a competitor it can have a deep impact on their points performance. Calibration does not exist to try to sway judges, just to make sure that they are all using the same measure. I do, however, believe that the judges are very accountable for their scores, each competitor has the opportunity to go over their scores individually with the judges at the end of the competition. The judging system is constantly improving, though...and needs to. Mistakes have been made in the past. When you say "announce", do you mean actually?

4. Signature drinks are absurd. People bemoan having to make half-caf hazelnut lattes with soymilk yet they will cheerfully concoct and serve up a combination of city-roasted Agajanian RuffRuff blended with apricot jam, crushed pistachios, and cubes of lime jello, garnished with melted Nutella? Can no one see the hilarious inconsistency here?

The sig drink has been a part of the contest since the very beginning in the '90s... and used to have even MORE emphasis than it does today. Before the "third wave", in the days when there were only a few espresso blends available to baristas, the specialty coffee industry defined itself by crazy, invented drinks. One company I worked for had an "Artista Barista" contest for the most creative coffee drink. This had a place, and was instrumental in building the specialty coffee industry into what exists today. I agree, however, that the sig drink is inconsistent with the notion of "letting the coffee speak for itself", the belief the top baristas ascribe to today. It is for this reason that many champ-baristas make subtle objections to the signature drink during this phase: for example, Nick won the SERBC by basing his sig drink on the simplest presentation of the coffee imaginable. I believe that the sig drink will (and should) continue to decrease in emphasis. On the other hand, what with all the blowtorches and such, it DOES create interest for the spectator, and allows for some creative expression by the barista.

5. The quiet game of "I'm on the inside and you are on the outside" combined with the endless choruses of "O Come Let Us Adore Us" are entertaining for about an hour and then boring as hell.

While I really hope it doesn't come across that way, I understand that the competitions can seem pretty alienating. I constantly struggle with this while I am MCing, and I try my dangdest to try to lift whatever veil exists, perceived or real. I don't see the comps as quite so self-serving, but I can see how they can seem that way, especially with such a small family of competing baristas/companies. I'm at a loss to know how to fix this one, honestly.

Peter G

PeterG wrote:
I actually disagree with this one. A clean, well presented culinary experience is central to the food arts, if you ask me. The idea is to get a sense of the barista's aesthetic presentation and ability to relate to customers. Both are important, in my view. If it helps any, though, the scoring form emphasizes drink quality over presentation.

Peter,
I think most competitors will agree with you on this point, as do we. And we think it's a point that will see greater acceptance in by the general public as more become aware of the competition's purposes and goals.

But, as we experienced at the MARBCs this past February, it appears very subjective even if it may not be in fact. One judge wrote on their scoresheet that our competitor's outfit did not match the tablecloth. Were points deducted for that? We don't know. But why even write that down?

That was just dumb and got our ire up because we didn't think that would ever be a criterion. Nobody told us this was both a barista competition AND Project Runway.

Thankfully it wasn't an issue at the nationals. But it's that type of thing that brings the point on importance of presentation into question - what exactly does "presentation" mean?

I'd also add that some companies can afford much nicer "stuff" for presentation (table covers, cups/saucers & accoutrement) than the average indie still using LaMarzocco demitasse or the standard-issue brown jobs.

As an improvement, I'd recommend is a set of kiosks behind the audience during the semis where other baristas (perhaps from the same shop?) prepare the competitor's sig drinks right after the competitor is finished - or maybe throughout? (I don't think I'd ask the competitors to do this themselves - too nervewracking as it is).

I'd also suggest that the six finalists can't use their standard, practiced sig drinks. Do it like they do at the CIA Exams - once they know they're a finalist, present the "basket of stuff" - including the coffees - and let them figure out what they're going to make with it. They've got a night to think about it and a morning to practice, right?

Now that's bringing culinary knowledge into play (can you tell I spend more time in the kitchen than behind the bar?)

I'm not one to attack/defend the judges' rules on scoring, because they're opt to change anyway, but I thought I would comment on some things:

But, as we experienced at the MARBCs this past February, it appears very subjective even if it may not be in fact. One judge wrote on their scoresheet that our competitor's outfit did not match the tablecloth. Were points deducted for that? We don't know. But why even write that down?

That was just dumb and got our ire up because we didn't think that would ever be a criterion.

There are no spots on the technical or sensory judges' sheets to give/take away points for something like that. I believe the head judge will make a broad assessment on overall presentation, but since i wasn't a candidate to be head judge, i only skimmed over their score sheet.

I will say that I made several comments on score sheets....not to affect the score, but to remind me exactly who I was judging at the time. Being a judge is a lot more hectic than it looks. There's a lot of movement backstage, and obviously time is not a luxury. In the judges' meeting, they stressed that, for many competitors, it is still a learning experience, and they encouraged us to make as many constructive comments on the sheets as possible. While on stage, I was really just trying to focus as much as possible on the drinks themselves, and then in my time backstage, i was going back and adding any comments or suggestions that I had. Adding a comment like that on a score sheet may have helped me remember a detail.....

Dunno. I hope you took advantage of the judges' commentary time at the end of the competition. I was a judge at SERBC, and was actually pretty surprised at how many competitors DIDN'T come up to me afterward for more feedback. That was actually supposed to be my favorite part. I got certified as a judge because i wanted it to be as constructive a learning experience as possible for each competitor, and I thought the feedback time was a great idea.

I'd also add that some companies can afford much nicer "stuff" for presentation (table covers, cups/saucers & accoutrement) than the average indie still using LaMarzocco demitasse or the standard-issue brown jobs.

Again it comes back to "the gourmet espresso experience." I have an idea of what the organizers are talking about when it comes to presentation, and I suppose it's arguable as to how important that really is. Not going to argue that point, but I will say that at a past competition (in which i was a spectator and nothing more) I saw a competitor's tablecloth setup.....he/she had obviously bought it brand new, I suppose to "impress" the judges. Brand new, and completely unironed. You could see where the packaging label was previously. It looked....well, it looked bad. Sure, a new cloth is better than an old stained one, but I will say that the judges aren't impressed by just throwing money into a setup. It's not a bankroll contest. Instead of looking at it as "What is the fanciest, most expensive stuff I can get for this comp?" I would recommend looking at it as "How can I take what is readily available to me, or what I can afford, and create the most unique visual experience to compliment how much damned sweat, blood and tears I put into my training?"

I do agree that the visual presentation assessment is highly subjective, and can be explained more specifically in the future. I assume that's why it is reserved only for the head judges' opinion, and not the sensory/technical judges.

As an improvement, I'd recommend is a set of kiosks behind the audience during the semis where other baristas (perhaps from the same shop?) prepare the competitor's sig drinks right after the competitor is finished - or maybe throughout?

My first thought was: you are taking the attention away from the competitors.

I'd also suggest that the six finalists can't use their standard, practiced sig drinks. Do it like they do at the CIA Exams - once they know they're a finalist, present the "basket of stuff" - including the coffees - and let them figure out what they're going to make with it. They've got a night to think about it and a morning to practice, right?

Damn. Now that's an idea!

Good discussion. I like. However, can anyone tell me why this discussion is going on at three different forums and numerous blogs......and NONE of them are the SCAA website forum?? If change is ever to be made, isn't that where it'll happen?

dankbean wrote:Good discussion. I like. However, can anyone tell me why this discussion is going on at three different forums and numerous blogs......and NONE of them are the SCAA website forum?? If change is ever to be made, isn't that where it'll happen?

Not a bad question. Maybe because this forum has 40% more members than the SCAA forum? Hey Dan, cut & paste the whole thread!

And on the Barista Guild forums Barista Competition forum closed to non-members.

dankbean wrote:I was a judge at SERBC, and was actually pretty surprised at how many competitors DIDN'T come up to me afterward for more feedback. That was actually supposed to be my favorite part. I got certified as a judge because i wanted it to be as constructive a learning experience as possible for each competitor, and I thought the feedback time was a great idea.

In the past I've noted most competitors seek out the head judges. But at this last competition, I saw little interest in feedback among the competitors. No idea why.

Compass Coffee wrote:Not a bad question. Maybe because this forum has 40% more members than the SCAA forum? Hey Dan, cut & paste the whole thread!

The number of board members means very little, it's sustained contributors that defines a site (a topic that A fool and his money touches on). The SCAA isn't on the bleeding edge of exploiting the web. For example, they still insist on faxing competition volunteer forms and their response to e-mail is spotty. I've learned that if you want to contact someone from the SCAA, pick up the phone.