another_jim wrote:"For the treatment of vegetative myopathy, Simglobulon significantly outperformed placebos in the FDA approval trials."

Now Jim, you're not being paid by the Simglobulon people are you? You are so right regarding drug trials. The companies often quote "relative risk reductions" that look great, but absolute reductions are minuscule.

Jim,

I'm so glad you posted this. Your succinct post is actually a good substitute for the "article" we talked about writing on the "Scientific Method," but which we never could get around to writing. Scientific study of common issues is both simpler and more complicated than it appears at first glance. In order to have even a chance of answering an important question, the question you are asking (and hence what you are studying) must be distilled down to something very simple and direct; otherwise, your "results" are apt to be as fuzzy as the question you were trying to answer.

I have tried to live by two major principles when it comes to coffee studies I have done and participated in. The first principle, is to try to design a study that answers a question that real people want the answer to. An example of such a question would be, "does freezing so seriously damage coffee that using it as a method of preservation should be absolutely discarded, because the previously frozen coffee is obviously and grossly inferior to fresh and never frozen coffee??" This is not an unreasonable question to ask, given what has been said by many notable people about the use of freezing to preserve coffee. Here, we aren't asking whether frozen coffee is 9% worse than fresh-never frozen; in order to answer THAT question, you would have to compare more espresso shot pairs than anyone I know would have the patience for.

The second principle has been to try to design the actual test to be as simple as possible, and few things are more simple than comparing two cups of espresso at the same time and coming up with a preference (or a lack of preference) by asking one's self the simple question, "which one is better?"

Statistics are useful for some things and not so useful for others. In the "not so useful" category I would place the subjective experience of tasting two cups of espresso at the same time, and coming to some sort of a conclusion on the overall comparison of a relatively small number of shot pairs. No doubt it is true that you would need a HUGE NUMBER of such comparisons to detect a small but real difference between the two conditions being tested (in this case, previously frozen coffee vs. never frozen coffee). It is also true, however, that after one has the chance to taste a much smaller number of such paired cups, and to then learn right after forming an opinion, which cup was from frozen coffee and which was from fresh -- an open minded person is simply forced to acknowledge that he prefers one condition one time, and the other condition the other time, and in reality the inter-shot differences overpower the differences one observes between shot pairs.

Taking this "previously-frozen" vs. "fresh, never frozen" example a step further, a taster in such a study (Jim and myself in this case) is forced to conclude that (1) freezing doesn't do that much "damage" to coffee, certainly nothing within the magnitude alleged by its detractors; and, (2) it sure seems to do a pretty good job of preserving coffee, when what would otherwise be undrinkable (4 month old coffee left out exposed to the environment at room temperature) is virtually indistinguishable from fresh, never frozen coffee.

And with all its limitations, a small (repeat) study such as we (Jim and myself) have just completed, proves, at least to US, that freezing is a very reasonable method for preserving coffee, and the previously frozen coffee (within the limits of how our coffee was handled) is a close enough approximation to fresh that freezing should be considered a very reasonable way to extend the shelf life of coffee, certainly better than any other way we have to deal with coffee that would otherwise oxidize into oblivion if left at room temperature.

Getting back to Jim's above post, the study that we have just done proves the merit of what Jim is trying to explain. One benefits from considering statistics in the design of any study. If one expects or wishes to consider the possibility of a large difference being present in the two studied "conditions," the "thing" being studied has to be simple if one is going to detect the difference within a reasonable number of comparisons. If on the other hand one hopes to find a subtle difference, then this has implications for both the mechanics of the test itself, and for the feasibility of testing it in the first place. Another way of saying this is that one has to decide if a subtle difference is sufficiently important to detect, that it would be worth spending several weeks in tasting hundreds of shot pairs in order to detect it. In day to day espresso preparation, my palate is simply not THAT discerning that I'd be willing to invest the time and effort in trying to detect a difference of that magnitude. There are others here who may feel differently, and I'd encourage them to conduct such a study if they thought it was worth doing.

ken

Yes.....

But which is better?

michaelbenis wrote:Yes.....

But which is better?

The one you paid more for, of course.