This time Rao's right

Want to talk espresso but not sure which forum? If so, this is the right one.

#1: Post by Aguirre »

I usually don't like Scott Rao's style and statements, but this time he nailed it. This is a very appropriate rant :

User avatar
Team HB

#2: Post by homeburrero »

I don't like his subtitle much:
...Scientists are often wrong...
which implies that he may be anti-science, but I know that that was not at all his intent here. But of course, especially when talking about podcast/news article/Instagram/conference talk 'science', much of it is dubious even if it comes from a scientist or scientific institution.

Of the tons of comments, I think this one sums it up nicely:
cgncr wrote:@whereisscottrao the slides/presentation of what I think is the same research is available from SCA website, but presented in 2018. Podcast #40 on the SCA website podscasts section. I also had the unfortunate chance to go through it. The dark and light roast they compared were NOT the same coffee. One is a blend from Starbucks and the other is a "single origin" from Peet's (or vice-versa, but who cares). They underextracted almost 100% of the flat bottom brews, some as low as 8% EY (!) and about 50% of the Melitta Cone brews. To be hitting that low of extraction means coffee wasn't even getting wet. They also present R values of 0.3-0.4 as significant correlation between bitterness and TDS. They have not proved anything expect 1) Different coffees taste different and extract different 2) Coarser grinds extracts less than finer grind 3) Different EY taste different 4) Big flat bottom extracts less than small Melitta basket at same grind size, all of which have been known since 50s.
Here's a link to SCA podcast #40: ... 8-lecture/

More discussion of this research is in this thread: Davis Coffee Center Tests Flat vs. Conical Filters
nínádiishʼnahgo gohwééh náshdlį́į́h


#3: Post by cunim »

Of course scientists are often wrong. They are supposed to be. I would start the first lecture in an upper year course by saying "Everything I am about to teach you is wrong, but it's the best guess we have at the moment". Drove the med students crazy because they didn't have time for "philosophy" and just wanted to know answers for the exam.

Scientists are people who apply the scientific method to test hypotheses (not facts, hypotheses). Some people apply the method properly and report valid data. These people are scientists, just like a skilled woodworker is a carpenter. Some people do not apply the method correctly, in which case it is not science and the people who do it are incompetent, ignorant, evil, or some combination of those things. Sadly, much of what the public perceives as science is, in fact, conducted and reported by non-scientists.

Scientists and quacks, they're all wrong. However, a well designed experiment allows you to create a better hypothesis for subsequent testing. In contrast, quasi-science is quite likely to tell you that what you already believe is, in fact, true. Comforting, isn't it?


#4: Post by OldNuc »

^^There you go, popping peoples bubble. :lol: They used to teach this in Jr. High science class.

User avatar
Team HB

#5: Post by RapidCoffee »

cunim wrote:However, a well designed experiment allows you to create a better hypothesis for subsequent testing.
Exactly! Science encourages skepticism and questioning. You can say "Rao's right" all you want, but that's just your opinion. You can claim that your personal experience differs, but that's just anecdotal evidence, which is essentially worthless in the scientific realm.

If you believe this experiment is flawed, then design your own (double blinded, statistically significant) study, carry it out, and publish the results. That's how science works.

Aguirre (original poster)

#6: Post by Aguirre (original poster) »

RapidCoffee wrote: If you believe this experiment is flawed, then design your own (double blinded, statistically significant) study, carry it out, and publish the results. That's how science works.
Myself, as a graduated scientist, try to avoid "believing" at all costs. Therefore, it's not a matter of "believing" that this experiment is flawed. The reason why I say Rao's right is, beyond an opinion, an observation that anyone with reasonable knowledge about coffee brewing will definitely share.

If the people conducting this experiment really wanted to contribute to the community with meaningful findings, they should have invested time in researching and understanding important aspects that would be, by any standards, mandatory. Not even using the same coffee for comparison, how can this be serious?

Definitely, if I ever decide to take the time to work on a scientific experiment about any aspect of coffee brewing, I'd for sure pay attention to the basics and avoid wasting mine and other people's time.

Again, I am a scientist and didn't feel offended or attacked by Rao (at least not this time :)). He was simply criticizing some bad work. Scientists should, and would, agree with him.

Team HB

#7: Post by samuellaw178 »

I personally think the responses have been a bit harsh...It can be easy to dismiss something when you have the benefit of hindsight on your side and are looking at things in retrospect.

I don't doubt the group who conducted the experiments are scientists with years of experience in experimental design, but they might not be coffee-extraction scientist and possibly had no hands-on experience in obsessing/making coffee at home. Given what they know then (before the study), that's probably their best stab at the matter.

Prior to embarking the experiment, the scientists most likely didn't know the bed height/shape can affect the extraction time/yield. I can't imagine the scientists being given the brewers a month in advance to play with before proposing and designing the experiment. The experiment could possibly be their first and last chance. In an ideal world (unlimited resource, no deadline), you could conduct unlimited preliminary experiments and refine the experimental design before the 'official' experiment. This may not always be the case especially in a corporate research environment.

If you're not a coffee veteran/expert who obsesses about coffee extraction, the logical variable to control will likely be the grind size. 'Conventional' wisdom says that the grind size affects the extraction and that's what they controlled. But who would've expected you have the bed height/TDS % curve-ball thrown at you, which only became obvious after the study?

About the comments on using two different coffees for different roast levels, seeing that they are sourcing the coffee from Starbucks/Peets, perhaps there are some feasibility/political restrictions that we are not aware of? Any experienced scientist will have attempted to minimise any confounding factors (I have no doubt the scientists had considered this given their credential - but sometimes some compromise has to be made based on feasibility). Perhaps they didn't know how or did not have the resource to get the same coffee roasted two ways, in the required quantity and timeline?

From their perspective, they might be seeing themselves as selecting a 'representative' light roast and a 'representative' dark roast for the study, which can be a totally valid approach for scientific research. For casual/mass coffee drinkers, light roasts surely taste different enough from dark roasts, but between light roast or dark roast itself, the difference may have been much smaller/insignificant. Since the objective of the study is to compare flat vs conical brewer (not dark vs light roast), this may be an acceptable compromise although non-ideal. If I read correctly, they are comparing conical vs flat brewers given a roast level - not "light roast+flat brewer" vs "dark roast+ conical brewer".

The study may have not answered the flat vs conical question conclusively, but surely there're still some learnings that can be be gained from the study.

Just my 2 cents... :P
★ Helpful

User avatar
Randy G.

#8: Post by Randy G. »

Some of this reminded me of a situation I created when teaching 6th grade. We were covering proportions as a sort of pre-algebra lesson, so the steps to solve for the unknown were regimented and showing all work was required. I told the class each day that, yes, there were easier ways to solve for "x" but this was like learning to use a hammer and later you will use it to build a house.

One student received a zero on a homework assignment and then told me that father had shown an easier way.

"What does your father do for a living?" I asked.
"He sells furniture."
I told the student, "Tell your father that I promise not to sell furniture if he promises not to help you with the math homework."
The parent conference was interesting, to say the least.
* 22nd Anniversary 2000-2022 *


#9: Post by cunim »

OldNuc wrote:^^There you go, popping peoples bubble. :lol: They used to teach this in Jr. High science class.
Bubbles. Don't even start me on the origins of the easter bunny.

You had a good jr high. There are so many parts of the world where faith-based teaching is primary before children reach the university level. In North America, my impression is that the majority of first year students have only the vaguest understanding of what science is. We then spend years trying to teach these open minds (hopefully) what good science is.

But enough of that. Here, I'd be happy to learn what good espresso is.


#10: Post by OldNuc »

The real problem here is the production of highest quality --top shelf espresso is an art, not a science. This is the same with highest quality --top shelf cooking.