lennoncs wrote:How many competitors demonstrate good food safety practices?

if something is obvious, then it will cost the competitor points. if something is dramatically wrong, then i wouldn't be surprised to see those drinks disqualified. i've had discussions with other judges about these issues (especially after someone perspired into my drink), so it is on our minds, somewhat, during competition.

This year I didn't have the energy or time to write-up the local SCAA barista competition like last year. I did however jot down some notes about the 2006 Southeast Regional Barista Competition's signature drinks, and once again, I would like to share some of my thoughts on how competitors might improve their game.

This time around, Abe and I had the honor of serving as finals sensory judges. Honestly I was surprised to see our names on the roaster since I assumed "home enthusiast" was an unspoken Scarlet Letter among the pro ranks. It could have been the lack of available USBC-certified judges, or their desire to give motivated judges outside the coffee profession a break. Whatever the reason, I'm grateful to event organizer Michelle Campbell and head judges Marcus Boni and Spencer Turer for (what I'll assume was) their vote of confidence.

Hearing that some competitors at the WBC complained that sensory judges didn't sample their drinks fully, I drank deeply... every cup... at least three times. The runners were irritated with my dawdling and Marcus did make note of my pondering, but I believe the extra time added confidence to my scores (especially as some defects become much more evident when the drink cools). The downside was that I blew the top off my acquired tolerance for caffeine. Tomorrow is the first day that I hope to have espresso again.

Now that my body is on the mend, my debriefing notes:

• Talk about the coffee, not yourself - There are a number of required elements of the barista's performance. For a good Professionalism/Dedication/Passion score, you must demonstrate knowledge of your coffees. Why did you select the blend components? What does each contribute to the cup? Some competitors shared interesting personal stories at the expense of coffee-related discourse, others dumb-downed their descriptors to the point of platitudes. I wondered if some competitors have decided it's too risky to be specific.
  
• Palate cleansers, please! - Or at least water! The Ideal Palate Refresher offers specific suggestions. I emptied the presented glass every time and still craved soda crackers. It helps the judges between courses and demonstrates your appreciation for this important ritual.
  
• Presentation cues - It would be helpful if commentary directed to the sensory judges were delineated from those directed to the audience or technical judges. We're instructed to listen intently to the barista, but sensory judges are also told to take notes to support their scores. Clear signals of a transition between the explanation of the coffee characteristics and the preparation elements would help assure the right judges get the message.
  
• Make a final checkout list - This oversight is likely limited to the regionals where invariably a competitor or two stops the clock when they have remaining time. Enough time to clean up their station and increase their technical score. I assume it's one of those "d'oh!" moments combined with the urgent desire to finish; put a checklist near the timer that reminds you of last-minute point getters.
  
• Stick around for the debriefing - few baristas took the opportunity to question the judges' scoresheets. It's a good learning experience for all involved; I know that I asked some competitors, "Why did you do XXX? You surely knew it would be a mandatory markdown." The answer: "I was running out of time". To wit, I've asked Marcus for a pie-chart that shows the point breakdown. It's worth knowing what you're giving up in order to meet the 15 minutes. The better course of action when things go bad may in fact be to take the time deduction.

This competition was the first where signature drinks were served first by several competitors. I don't have a strong preference on the presentation order, though it is more difficult to keep your taste acuity sharp if the signature drink is particularly spicy. Again, palate cleansers are appreciated; Abe brought his own and munched during the calibration period between flights.

PS: HB members Bob Barraza and Kevin Kratwald were also SERBC judges this past weekend. Gentlemen, you are welcome to add your debriefing suggestions.

HB wrote:Hearing that some competitors at the WBC complained that sensory judges didn't sample their drinks fully, I drank deeply... every cup... at least three times.

This may not be a good idea; and the people doing the complaining may not know a lot. I'm reading my way through the tasting literature, not just coffee, but all sorts of things, including the food science stuff.

There's not a lot they agree on; but the one thing they do is that the first and perhaps second taste are the most accurate. After that the taste and odor sensing nerves overload; odor and bitter/savory receptors bind the molecules and are out of commission for a few minutes, while the sweet, sour and salt ion based taste sensors need to recharge their batteries. The basic drill recommended thoughout is one or two small sips, then a rest before moving on. In this sense, barista competition with its pauses between courses and competitors may actually be more accurate than, say, cupping 10 coffees in one sitting.

Just a few comments:

Know your coffee

With the exception of perhaps 2 baristas in the SERBC, none of the competitors have a chance of passing the first round in the USBC. The problem was not so much the new faces in the competition, but the fact that many of them knew very little about coffee, and are poorly trained in evaluating their own drinks. A great number of them were using the same blend (Toscano), perhaps because it is the only coffee they know. Some explained what we should taste in the coffee before they served their espresso. In most cases, none of those flavors we expected were in the cup. I assume some of them did taste their coffee during practice, and you would ask yourself why didn't they detect it? The answer is: they are clueless. With very little training on how to cup coffee, they were reading the label on the box, and not their taste buds.

Steaming

The cappuccinos were universally milky. You would think with so much milk there will be some sweetness to them as a trade-off, but there wasn't. This one goes back to evaluating your drinks. I am not sure there was a universal problem with steaming technique, but the end result was lack of sweetness, and not enough coffee taste. Only in two cases I had both. One competitor used a separate blend for cappuccinos, and it did make a difference.

This was my first attempt at judging, hence no frame of reference, just first impressions. I am glad that I did the Judges Calibration course. It was quite intense, and I learned precisely what to look for in the competition as well as tips for my own home brewing. I would encourage others to participate as well.

As complex and subjective as tasting can be, I think that the process of breaking down the attributes of the drink help make the process more objective. For example, the espresso is judged separately for appearance/color, consistency/persistency of crema, as well as taste and mouth feel. Knowing this, it surprised me how poor the crema was in the drinks that I was served. Even finalists presented me espresso in the first round which had little or no crema. Perhaps this problem is exacerbated by the protocol itself which requires that four singles be served at the same time. Most of us are used to pulling doubles, ristrettos, and we serve each drink immediately.

At the end of the day, my hat goes off to all competitors. It is quite a challenge to serve twelve drinks within the timeframe, particularly when you are on stage with seven judges nit picking every step, usually with unfamiliar equipment. Perhaps an unrealistic protocol, however, it does seem to highlight excellence from mediocrity.

Great thread, thanks! Helps prepare my mindset for next months NWRBC and my first judging.

It seems to me that some of the same ideas are being shouted about over in the Ken Fox "who gives a sh**" thread are being talked about here with measured gentility. Much nicer over here.

Nick said something a few posts back that perked up my ears. Here it is:

actually, to be completely honest, I'd rather not have Abe as a judge, if that's the sort of perspective that he's going to come in with.

I don't want to lift his commentary out of the perspective in which he intended it but it seems to me that this kind of thinking represents a certain rigidity of mindset that may or may not be helpful to the competitions in general. This is the same concern that I raised in the other thread about the downside of "calibrating." It potentially diminishes the value of new ideas, perspectives, and insights.

Competitions at this point seem marked by artifice. They aren't focused on making the absolutely best coffee of which the barista is capable, as Abe has clearly pointed out. They aren't focused on barista-life-as-lived, as others have demonstrated. So why the "five star" approach if it is totally outside the range of occupational expression of virtually all the contestants? Just to make it hard? And why do most of the judges and organizers seem so resistant to alternative ways of doing things?

Omniryx wrote:Nick said something a few posts back that perked up my ears. Here it is:

actually, to be completely honest, I'd rather not have Abe as a judge, if that's the sort of perspective that he's going to come in with.

Funny thing, I ended up judging him after all (he won the SERBC under my watch). I'm not sure how strongly Nick stands behind that statement today, but it was amusing to be reminded of it. I forgot all about it.

And to the main point of your post. In general I am for relaxing the rules on how to make the drinks, and judge it in the end in the cup. I would like to see more innovation coming out of the USBC. Treat it like the world baking competition. They are judged on the cleanliness of their procedure and on the baked goods - taste & visual. They can get there any way they want within the time limits of the competition. One major improvement is simple: Do not require all 4 drinks to be served to the judges at the same time. Espresso should be consumed immediately after it is prepared. Having those drinks sit at the barista station for two minutes before the judges taste it is unfair to the baristas.

Omniryx wrote:Nick said something a few posts back that perked up my ears. Here it is:

actually, to be completely honest, I'd rather not have Abe as a judge, if that's the sort of perspective that he's going to come in with.

I don't want to lift his commentary out of the perspective in which he intended it but it seems to me that this kind of thinking represents a certain rigidity of mindset that may or may not be helpful to the competitions in general. This is the same concern that I raised in the other thread about the downside of "calibrating." It potentially diminishes the value of new ideas, perspectives, and insights.

Competitions at this point seem marked by artifice. They aren't focused on making the absolutely best coffee of which the barista is capable, as Abe has clearly pointed out. They aren't focused on barista-life-as-lived, as others have demonstrated. So why the "five star" approach if it is totally outside the range of occupational expression of virtually all the contestants? Just to make it hard? And why do most of the judges and organizers seem so resistant to alternative ways of doing things?

You dirty bastard! You're a jerk! You have no idea what you're talking about! How dare you attack me!



Seriously though...

My post was not simply an open opinion... it was in response to the fact that Abe's posts were not in alignment with USBC judging guidelines. Opinions are opinions, but if you're judging, you should follow the judging guidelines. That said, I have no issue with all of the "If I ruled the world, I'd change the competitions like this..." ideas. However, everyone has an opinion, and certain opinions do, necessarily, have more impact on the competitions than others... which is why you'll see the new WBC rules-changes very soon, to take effect at USBC-sanctioned regionals that occur after the first of the year.

Omniryx wrote:It seems to me that some of the same ideas are being shouted about over in the Ken Fox "who gives a sh**" thread are being talked about here with measured gentility.

Agreed. I'll respond to your question here as they seem more relevant (a sort of "judge debriefing").

Omniryx wrote:

HB wrote:The judges do independently fill out the scoresheet completely, but the calibration is the head judge's opportunity to challenge your scoring. Their challenge may take the form of pointing out something you missed. They don't require you to modify your score thereafter, but that's up to each judge's discretion. You are required to get the head judge's approval for any zeros.

You'll note that I acknowledged that all the judges may be busting their butts to be fair and objective. But you can see the problems created by the procedure you cite above. Imagine that I am a brand spanking new judge; the ink barely dry on my credentials. A head judge with a forceful personality takes exception to my scoring. How likely is it that I am not going to be swayed by her or his "suggestions"? Especially if I want to be on another judging panel...

And what if the new judge saw something that the head judge missed instead of the other way around? Is this impossible? Are head judges incapable of error?

Of course, one might reply that this is precisely why there is a head judge; to teach noobs how to do their work. But that assumes that there is only one standard or perspective for judging and that the head judge has the ultimate handle on it. One might counter that fresh perspectives, attitudes, and insights would be a good and healthy thing.

The head judges are often the same people who run the certification workshop. While some scores are more subjective than others, scoring consistency is a prime objective of the workshop. For example, the scoring of crema consistency and color should not vary much depending on the judge in question; if four judges evaluated the same cup, I would challenge any deviation more than than 1/2 point. On taste and tactile balance, well, that's harder to dispute, though I would be surprised to hear about disagreement in excess of one point for the same cup.

Nick and others have hinted about changes on the horizon for SCAA barista competitions. I know nothing about them, but I'm confident they'll be for the better.

Omniryx wrote:It seems to me that a higher level of perceived credibility would be maintained if, once a judge was qualified, her or his judgments were not subject to "calibration." Of course, if there was significant variance among scores when they were announced, individual judges might be called upon to explain their reasoning. Would that be a bad thing?

I've never felt any pressure to change a score, though I have been asked to justify them on occasion. That comes with the learning process and is part of the reason I volunteer. The more I think about it, the more appealing Abe's suggestion to add instant scoring sounds. However, a lot of the so-called calibration is really the head judge making certain nobody screwed up. For example, leaving a score blank. There are also several scores that are Yes/No (e.g. correct cup, all drinks served at once, served with accessories) and it would be an embarrassing oversight if they were inconsistent, barring unusual circumstances (e.g., a competitor spills a drink and chooses not to serve one judge).