The SCAA Sensory Skill
[Psych] Test


One of the Speciality Coffee Association of America's (SCAA) most visible activities for the public at large is the regional, national, and worldwide barista competitions they organize. An SCAA barista competition is an event designed to encourage and recognize professional achievement in the art of espresso beverage preparation and service. Competitors prepare and service 12 coffee beverages— one espresso, one cappuccino, and one signature drink of their own creation —for four judges in a space of 15 minutes.

Many volunteers serve as judges, including myself. There are three types of judges: Technical judges who score the correctness of the preparation of the drinks, sensory judges who score the drinks themselves, and a head judge who oversees the overall process to assure it conforms to established competition rules. I had served as a sensory judge at the regional levels twice before and considered judging at the national level, which requires a more rigorous certification process, including a sensory taste test.

I loathed the idea of taking such a test. Loyal Overextracted readers may remember my meeting 2003 Italian Barista Champion Andrea Lattuada, as recounted in the thread Performance Anxiety (emphasis added):

...I attended a group dinner during the past SCAA conference and we were sat next to each other. We made small talk, asking what brought us to the conference, where we hailed from, that sort of thing. Later he shared some of his irritation with the recent judge certification, as he was among those who failed the taste discernment test that was introduced this year. Being an Italian barista national champion a couple years back, he figured it would be a shoe in. Others were surprised by their not making the cut too (I met another disgruntled past WBC-qualified judge in the Elektra booth the next day).

Around the same timeframe, Mark Prince published more details on this "dreaded" sensory skills test in 2005 USBC Judges' Certification, where he reported squeaking by on the first attempt (100 / 90 / 70 for Parts I, II, and III). Hearing several other first-hand stories of difficulties encountered by pros in the industry, I figured the chances of me passing were between slim and none. But with the 2006 USBC coming to Charlotte— a mere 90 minutes away —and the opportunity to meet and learn from pros, how could I let the chance slip by, even if it meant a little humiliation over the results of a test?

Failing and then Passing the Test

I convinced HB moderators Abe Carmeli and Bob Roseman to sign up for the judges' certification workshop. Day 1 was dedicated to lecture and review of the rules and regulations; Day 2 would start with the sensory skills test, and then finish with calibration exercises and mock competition judging. Michelle Campbell, who coordinates the SCAA barista competitions, warned us about eating spicy foods prior to the test, so it was unexciting fare for dinner and an even duller breakfast. The next morning Joseph Rivera presented an interesting lecture on the science of taste, interlaced with suggestions on the best approach to the test. He underscored that taste bud density was a genetic thing and worrying about performance was pointless: You either have 'em or you don't. Then it was time to pass out lots of tiny cups...

Standard eye chart To make a long story short, I took the test and failed (100 / 80 / 64 for Parts I, II, and III; 70 is the cutoff for Part III). Abe and Bob passed with very good scores. You can imagine my mood that evening! I knew there was a risk of failure, but why only me? Loss as a shared experience would (almost) be funny. As the only one to miss the bar, I appreciated the encouragement and sympathies of my friends, but failed to appreciate the humor of the situation. Despite my poor showing, I did well enough in the practicals to qualify as a sensory judge. I was also invited to try the sensory skills test again the next day ("Ugh! Another boring dinner?!?" I thought).

That afternoon and into the evening, I received lots of advice on the sensory skills test, which I will summarize later in this article. But what prompted me to write about my experience wasn't sharing tidbits on navigating the SCAA's sensory skills test, it was a comment from Peter Guiliano during one of our regular espresso labs at Counter Culture Coffee several weeks later.

He likened the sensory skills test to an eye exam, which the majority of people don't consider challenging. Similarly, few question the quality of their hearing. Yet many have doubts about their sense of taste, assuming that only trained professionals or naturally gifted super-tasters can accurately convey sensory taste descriptors. He explained that a significant part of the cupping events open to the public is convincing attendees to simply trust their own senses. I know exactly what he means! To the uninitiated, the flowery language that enamors cuppers becomes a "clubby" nomenclature.

Whether taking the sensory skills test or trying your hand at cupping, the best advice I received applies to both: Don't psych yourself out, don't second guess. Trust your sense of taste as naturally and confidently as you trust your eyesight.

Test Format

Trying out the test structure

The Coffee Cupper's Handbook by Ted Lingle documents how to setup the test. If you want to try out the structure of the test, you could approximate it based on the description to the left. Diminishing intensity levels by half would approximate the experience (e.g., one / one-half / one-quarter teaspoon of lemon juice to a cup of distilled water for intensities 3, 2, 1; one-half / one-quarter / one-eighth teaspoon salt or sugar, etc).

Before getting into test-taking advice, let's briefly cover the structure of the test itself. It is composed of three parts. The first two parts are samples of distilled water with three different intensities of either sweet, sour, or bitter (salty). The last part is a mixture of two or three sweet / sour / bitter components.

  • Part I: Reference Solutions

    To familiarize you with the differences between the three levels of intensity, you are presented with three groups of three cups representing the low, medium, and high intensity samples of sweet, sour, and bitter. You simply order the samples by intensity within their appropriate group (i.e., sweet intensity 1, sweet intensity 2, sweet intensity 3). Once the testees are finished, the answers are self-corrected. Everyone gets 100% on Part I.

  • Part II: Blind Reference Set Identification

    This is the same format as Part I: Nine cups representing three groups at three intensity levels. Unlike the first time around, the samples are not identified as sweet, sour, or bitter.

  • Part III: Mixtures of Reference Solutions Identification

    You are presented with eight cups. Four of the cups have two of the three possible mixtures of sweet, sour, and bitter; four of the cups have all three. You must identify the intensity level of each (low, medium, high).

  • Exam Criteria

    Minimum passing scores for this exam are 100% / 80% / 70% respectively; Scores above 100% / 100% / 80% are considered outstanding. According to the SCAA, twenty-five percent of those taking the test will pass.

The last round is the one that trips up most people. Joseph explained that you lose two points for every incorrectly identified intensity level (e.g., you mark "sweet 1" (low) when it is in fact "sweet 2" (medium)) and four points for failing to detect sweet / sour / bitter. In Part III, you should have four blanks among the three intensity scores of the eight samples.

Increasing Your Odds of Passing

Finally the collected advice on performing your best on the SCAA Sensory Skills Test:

  • Go with your first impression.

    Don't change what you've written. Taste once and move on. Don't overthink it. Time and time again, I heard variations of the same advice from those who've taken and passed the test. Analytical types like me will find it very difficult to resist the temptation to revisit prior decisions.

  • The effect one component has over another is an important clue.

    The first two parts were tastes in isolation (sweet, sour, bitter), which may lead you to look for the same intensities in Part III.

    I mentioned to Jim Schulman the difficultly I had detecting sweetness in Part III. He reminded me that cuppers know sweetness can reveal itself as an attenuation of bitterness or sourness. In retrospect, it seems self-evident. For example, my wife makes a tasty vinaigrette that includes vinegar and some salt and sugar. Tasting the sweet / sour / salty ingredients individually certainly does not have the same perceived intensity as them together.

  • Move along briskly, especially in Part I and II.

    Part I is for reference, so there's no need to dwell over the samples, especially considering that it is self-scored and that you'll repeat the same exercise in Part II. Tasting and re-tasting dulls your mental acuity and risks overworking your taste buds for the Part III Challenge. If you must retry samples, set aside the highest intensity one and focus on the other two. Approximate number of tastes for Part I: 9 + (2 * 3) = 15.

    The Coffee Cuppers' Handbook My lack of organization for Part II wasted time and added frustration because I scored each sample individually instead of grouping them. Retaking the test the second time, I drew lines across the back of the page to make three sections (sweet, sour, and salty). I tasted each sample once and placed it in the correct group. The high intensity sample is a sure thing, so it was placed rightmost in the appropriate section. The only remaining task was to place the low and medium intensity samples in the correct order. Tasted in sequence, it's fairly easy to confirm the difference. Perhaps it is not as obvious as the high intensity sample, but I believe you could invert the order of one or two within a group and still pass Part II (Joesph did not cover the point deductions in Part II). Maximum number of tastes for Part II: 9 + (2 * 3) = 15.

  • Be willing to leave less than four blanks in Part III.

    Remember that there's eight samples, four with two components, four with three components. You know going in that there must be four blank ("0") intensities on the evaluation. However, the markdown for incorrectly noting "not present" is 4 points, but the markdown for noting the lowest intensity when in fact there's none is only 2 points.

    You shouldn't have any difficulty discerning medium intensity (2) from not present (0), so realistically you would be off by no more than 1. In other words, if you think that a component is present but you're wavering, I believe your overall score will be better if you mark 1, not 0, even if that results in only three blanks. Do however consider the earlier advice about the effects of one component over another. It may increase your level of certainty. For example, if a sour intensity seems to fall between 2 and 3, it may be that the sample includes sweet intensity 1.

My original score was 100 / 80 / 64, six points shy of passing in Part III. Frankly, completing the test a second time, I wondered if I would come up a few points short again, but I passed with a score of 100 / 100 / 76. The SCAA claims only 25% of the candidates pass, but I suspect that is based on ALL who've taken the test, most of whom probably don't repeat it at the cost of another $35 if they fail. Although they say that it's mostly about genetics, I bet following the advice above would increase anyone's odds of passing on the first try.

Good luck!

Epilogue

Below are excerpts of two esteemed HB members on the subject of the SCAA Sensory Skills Test:

I think that we, as a culture, are generally too insecure about our senses of taste and smell. I talk to dozens of folks every week who say something to me like "I just can't tell the difference between coffees" and "My palate isn't sophisticated enough to detect the differences you describe". I always call BS on that. I usually ask: "Can you tell the difference between Coke and Pepsi? Do you have a preference?" The answer is usually "yes" to both. I then reply; "The difference between those two brands of soda is much subtler than the difference between a mediocre Kenya and a great one." To put it in the context of our discussion: if you can taste the difference between Coke and Pepsi (differences in sweetness and acidity) and you can tell the difference between salted and unsalted soda crackers, you probably have the physical ability to pass the sensory skills test. The obstacle, therefore, is largely mental. (Ted Lingle and Joseph Rivera would probably argue with me on this score, but I'll stick with it.)

Somehow, we've decided as a culture that perceiving and describing differences in wine, coffee, scotch, perfume, etc. is best left to the super-initiated and skilled. Most people, when they hear what I do for a living, say to me: "You must have an amazingly gifted palate." While I would love to think that this is true, I actually believe that my sense of taste or smell is no better or worse than, say, 80% of the people out there. I have tasted coffee with those who are considered the best in the industry, and they are in the same boat. The only thing we do differently is 1. pay attention, 2. trust our senses, and 3. document our results.

In thinking about this topic, it occurred to me that the sensory skills test measures not only the ability to physically taste, but the ability to confidently assert your results. And, in thinking about it, this is just as important to the espresso judge as the ability to taste is.

—Peter Giuliano, Director, Counter Culture Coffee


As Peter says, there's a real misapprehension about tasting. In general, and there's lots of data on this, foodies, wine experts, chefs, and other food and drink professionals are not supertasters, but rather average to above average tasters. Supertasters are almost as unsuitable as sub-tasters to judge food, since they are so sensitive that anything except plain water is too sour, bitter, salty, and sweet. They're the ones keeping themselves alive on rice cakes and bean sprouts.

The main qualification for tasting is a mature palate, that is, one that doesn't go yuck, like a baby's palate, whenever it gets something unfamiliar or unexpected, but tries to figure out what it is. This happens pretty automatically as one gets beyond adolescence. Growing up in a home where there's lots of variety and good food helps a lot; but I know many people who discovered their liking for fine tastes later in life.

My guess, especially after meeting lots of people who post about coffee and consider themselves poor tasters, is that everyone who's drunk a cup or shot of good coffee and gone "wow, this is something special" is perfectly capable of cupping and passing taste tests.

—Jim Schulman, coffeecuppers.com

Excellent advice from knowledgeable coffee cuppers. If you would like to judge in SCAA barista competitions, passing the sensory taste test has advantages: Your judge's certification is extended from one year to two, and you only have to take (and pass) the test once in your life. So, are you willing to try the SCAA's "Pepsi Challenge"? I bet your odds are much better than you think.