WBC Procedure for Measurement of Brewing Water Temperature - Page 2

[gscace]

That's not what Schomer used - it's not what was used to create most if not all of the profiles you're talking about.
The Scace has been around for what... 3 months?

The shape of a profile using the Scace device is very similar to the profiles you get when placing a t/c inside the cake. In other words, you see the same sharp increases in temp during the 1st 8 - 10 sec. resulting in one shot stability readings of +-3 degrees if you just throw out the 1st 3 sec. whereas, throwing out the 1st 8 - 10 sec. results in much lower stability readings of +-.5 degrees.

Again, why would you want to throw out ramp up data that must have an effect on the cup. We should be examining all of the data that is reasonable, not just the pretty stuff. Usually the ugly stuff is more interesting anyway.

-Greg

[gscace]

I'm just trying to figure out how someone like Schomer and others can say their machines are stable to within .5F - 1F. It seems the only way anyone can make this claim is to ignore the 1st 10 readings. If you only ignore the 1st 3 readings, there is no way to get any lower than 5F - 6F variance within a shot (i.e. max temp - min temp ignoring 1st 3 readings).

Doug

It's actually pretty clear how they can do it. They set up groundrules that allow them to do so. According to your post, Schomer bases his data on the last 15 seconds of the 25 second shot. We decided to include all data except that data produced by a thermometer that is substantially not in agreement with the actual temperature because of response time issues. We did this because this data is relevant in our opinion. Note that David Schomer is not an author of the standard.

-Greg

[gscace]

Can anyone give me just one example of any machine that has a intrashot variance (max temp minus lowest temp ignoring readings 1 - 3) of less than 5F (as tested according to WBC testing procedures)?

Certainly my LM Linea is better than 5 degrees. Data I have was collected using a datalogger, and the data was reduced according to the standard.

-Greg

[gscace]

On the 2 machines that I have tested (both dual boiler rotary pump models) the 1st reading always seems to be somewhere between 180 -185 while the 4th reading seems to be between 195 - 198 on a consistent basis. That being the case it would seem the "one shot stability" rating is highly dependent upon the machine's set point. Set points near 198 will have very small intra shot variances while set points like 203.5 will have large variances. This leads me to conclude that as it is currently defined and measured "one shot stability" seems meaningless.

As I emailed you, and as is described in the technical rationale for the standard, the first 3 seconds are not used because the answer reported by the thermometer is not the same as the actual temperature, for reasons described in the standard. Beyond that, the stability data is real stability data. The temperature rampup is certainly reasonable to look at. Rampup rate can be influenced by how hot the dispersion block and screen are, the group temperature relative to the water temperatue, and other factors specific to the machine. So it is not meaningless at all, particularly since these relatively cold or hot parts influence the temperature of water hitting the coffee. It's important to remember that what we are trying to measure is the temperature of the water that hits the coffee, period. Artificial boundaries limiting the data used, such as "it's not as stable as the temperature later in the shot" should not influence the data.

Have you made measurements at 198 degrees setpoint temperature, and 203.5 degrees setpoint temperature to see what the influence of setpoint will be? I would be interested in your results. I think the ones you sent me were for only 201 degrees. I would suspect that temperature setpoint may have some influence in the stability value, but conceptually that doesn't bother me much. If you notice, the WBC standard prescribes a uniform measurement and data reduction method only. It doesn't specify the groundrules for acceptance / rejection, or tell you at which setpoint temperature you have to do this stuff at. It can't because it has to be somewhat flexible wrt brewing temperature setpoint to suit coffees that may be used for a competition. If stability values change due to changes in setpoint, then that may be useful information for someone who is designing a machine or evaluating one for use. The folks selecting a machine for use in competitions may end up making a decision partially based on uniformity of stability results over a variety of brewing temperatures, and we don't want to limit them by the standard. Again, we want good, technically sound data without preconceptions.


-Greg

[gscace]

well, if you take the scace thermofilter out for 15 sec. then do a 2 sec. flush then put the p/f back in and start the shot I don't see how to get the 4th second up to the set point especially if the setpoint is high like 204. Seems like most machine profiles start out at somewhere in the 180-190 range and that greatly depends on how fast you are getting the p/f back in.

Doug:

One of the things we strived to do in writing the WBC standard was to get the data with as little outside interference as possible. So we standardized a couple of things that are used across the board by reasonable baristas - the group cleansing flush after brewing, for instance. Other than that, we ignore things that mask results, like heating / cooling flushes. In our opinion, the holy grail is a machine that produces the same average brew temp every time, with the same stability profile every time it is used. By removing artificial practices that cover up machine deficiencies, we can see how well machines measure up to our ideal. In practice one does not use a machine in the way that we test them. We cool or heat groups. We warm dispersion blocks.

-Greg

[another_jim]

I've been thinking about the nitty-gritty of this standard for a while. It has the advantage of being precise, reproducible, simulating varying traffic conditions very well, and encompassing intra and intershot stability with a single statistic for each. However, if I were an HX manufacturer, I'd reject it and refuse to submit my machines to this test. They could justly claim that the inter-shot procedure is systematically biased in favor of dual boilers.

No matter what the HX design, the equilibrium shot temperature will depend on the shot making pace. If the pace is slow, the temperature over a long series will settle at a higher level than if the pace is fast. This has to be, since at the slower pace, the HX is heating up more. You are dumping the first four very slow measures for inter-shot calculations. But you still have the pace accelerating from 2 min 35 seconds to 45 seconds between shots. This will always cause the brew temperatures to drop for the later shots. (Potentially, they could avoid this with a very large HX, and a too high boiler pressure, so the temperature would rise at a steady pace and be offset by the accelerating shot pace -- but this an even more perverse result, since they would be deliberately mis-setting the machines to show well under this procedure)

The easiest, and I think correct, fix is to allow each manufacturer to specify the amount for the preshot flushes. For instance, Brasilia might specify 6 ounce flushes for the early shots, down to 2 ounces for the shots at 1 to 2 minute intervals, and down to no flush at all for the final 45 second interval shots. If they don't make a specification on this, stick with the existing one. This would be fair to their claim that the machines are stable under different operating conditions given skilled and trained baristas, and would still allow for the two test statistics to be comparable between machine brands. You are still forcing each manufacturer to test the machine in variable traffic conditions, but simply allowing them to specify, in more detail, how their machines are to be operated.

And if one of them sticks a TC in the grouphead, puts on a readout, and specifies a flush to X degrees, more power to them -- provided that the model is part of their regular commercial line, not a wbc one-off.

[malachi]

I understand how this would seem to bias against HX machines, but the reality is that this is all designed for evaluating machines for the WBC - and for the WBC it's unrealistic to have machines that require some specific and varying flush methodology.

[another_jim]

I understand how this would seem to bias against HX machines, but the reality is that this is all designed for evaluating machines for the WBC - and for the WBC it's unrealistic to have machines that require some specific and varying flush methodology.

The test does not reflect WBC use, which is to pull 3 shots at each group at roughly around 2, 5, and 9 minutes into the presentation, with the final two rounds having looser requirements from the taste point of view since they are being used as the base for other drinks. Moreover, if an HX machine is used, you'd want to know how much to flush it for those intervals, and especially for the initial espresso shot. Presumably, the test data would be available, and you could use it as you please.

Suppose someone does submit an HX machine with a grouphead sensor, and the user instruction is to flush until the sensor reads, say 0.5C higher than the desired shot temperature. If you then insist on the flush amounts in the protocol as written, the manufacturer would say you are in effect disabling the final temperature control. Now suppose he resubmits it with the sensor hooked to the autodosing, so the machine flushes automatically to a set temperature, and won't run with the PF mounted till it's there. Do you disqualify the machine entirely? No matter how you answer these questions, the fixed flush part of the standard looks more and more absurd.

Let me be completely blunt here. You can stick with this standard and LM as the sole supplier; and everyone will be happy. But I cannot see any HX manufacturer agreeing to it; and I cannot see it as a good faith step for creating an open competition for manufacturers:

1) The measuring protocol in effect prescribes how the machines are to be operated.
2) One of the co-authors is Bill Crossland of LM.
3) The operating instructions closely follow good barista practice on LMs and only on LMs.

I used to work for a company that bid on process and HVAC control jobs. We called specs like this proprietary, and assumed it was ghost-written by the favored company. We loved it when we were doing the ghost writing; and made a big stink when it was another one. In the end, unless the fix came from upper management, the engineer in charge of writing the spec usually ended up fired.

[gscace]

I understand how this would seem to bias against HX machines, but the reality is that this is all designed for evaluating machines for the WBC - and for the WBC it's unrealistic to have machines that require some specific and varying flush methodology.

The test does not reflect WBC use, which is to pull 3 shots at each group at roughly around 2, 5, and 9 minutes into the presentation, with the final two rounds having looser requirements from the taste point of view since they are being used as the base for other drinks. Moreover, if an HX machine is used, you'd want to know how much to flush it for those intervals, and especially for the initial espresso shot. Presumably, the test data would be available, and you could use it as you please.

Suppose someone does submit an HX machine with a grouphead sensor, and the user instruction is to flush until the sensor reads, say 0.5C higher than the desired shot temperature. If you then insist on the flush amounts in the protocol as written, the manufacturer would say you are in effect disabling the final temperature control. Now suppose he resubmits it with the sensor hooked to the autodosing, so the machine flushes automatically to a set temperature, and won't run with the PF mounted till it's there. Do you disqualify the machine entirely? No matter how you answer these questions, the fixed flush part of the standard looks more and more absurd.

Let me be completely blunt here. You can stick with this standard and LM as the sole supplier; and everyone will be happy. But I cannot see any HX manufacturer agreeing to it; and I cannot see it as a good faith step for creating an open competition for manufacturers:

1) The measuring protocol in effect prescribes how the machines are to be operated.
2) One of the co-authors is Bill Crossland of LM.
3) The operating instructions closely follow good barista practice on LMs and only on LMs.

I used to work for a company that bid on process and HVAC control jobs. We called specs like this proprietary, and assumed it was ghost-written by the favored company. We loved it when we were doing the ghost writing; and made a big stink when it was another one. In the end, unless the fix came from upper management, the engineer in charge of writing the spec usually ended up fired.

Actually Jim, we had philosophical reasons for writing it this way. We think that the ideal machine would exhibit the same thermal performance off idle compared to full-tilt boogie. We think it's reasonable to write the standard in a way that allows us to examine off idle performance without flushing. It's not fair to force competitors to play on something that doesn't require specialized experience with the machine to begin with unless everyone gets lots of time on the machine. Why does art of running an inferior machine trump the art of the coffee? The machine for next year was selected partially on that basis. Currently, no manufacturer of hx machines installs flushing thermometry in its machines, so it's a moot point for now. Currently there is no good information re flushing specific machines. Are you suggesting that this get turned into more of a research project than it already is? IF a machine manufacturer doesn't like the results after the tester performed the manufacturer's flushing ritual, are there then going to be claims of "you didn't do it right"? From our perspective it's just better to do it as we wrote.

I disagree that the deck is stacked by the authors. The authors of the standard are reasonable people to do the job. I work in applied thermometry. John Sanders is head of the WBC technical standards committee. Barry Jarrett is very capable, understands instrumentation and has experience with a wide variety of machinery. Bill is a good mechanical engineer with wide experience who agreed to help work on this. From my perspective, his working for LM doesn't mean squat wrt this standard. He knows a lot about coffee machines in general. In our view, and in our experience so far, the measurement method does expose the warts of various machinery. Our work is also supported by the other members of the technical standards committee, who adopted the standard.

Your item 3 is just plain incorrect. If you perform this measurement procedure on a LM Linea you will learn that this procedure is not good barista practice on that machine as well. However, the procedure shows what the weaknesses of the Linea are, gives you a clue about what to do to compensate and also gives you good information about the ultimate capability of the Linea.

You should know that one heat exchanger machine submitted to the WBC had extremely good reproducibility over all duty cycles, when measured using this method. In fact, it had the best reproducibility of all machines submitted. It was not selected for other reasons.

Jim, have you made some measurements of machinery using this method? If so, I'd love to see your results. If you have extensive experience with the machine, does the measurement procedure give you information about how to use the machine that reflect your experience?

-Greg

[another_jim]

Jim, have you made some measurements of machinery using this method? If so, I'd love to see your results. If you have extensive experience with the machine, does the measurement procedure give you information about how to use the machine that reflect your experience?

-Greg

Fair enough. I don't have the Scace device, but I do have Schomer baskets. I'll use those to run tests on the Tea and maybe the Peppina, if I can rig the otherwise useless single basket with a TC and microconnector that will go through the trousered PF. I'll follow your protocol first. Then I'll play manufacturer, research a bit, and follow the same protocol, substituting an equilibration flush of X seconds for the 2 seconds (just one time, e.g. 3.5 seconds, not a different one for each round -- I appreciate that this would be a pita, and also not prove much, since one cannot really start a timer at shot end, consult a table, etc etc). I don't have a logging recorder, but will take readings every 2 seconds to 0.1C resolution (I'll preprint a form so I just need to dot the number, rather than write it down, and get some practice with dry runs).

The Schomer basket is buffered compared to the Scace device, so the under temp early in the shot will be worse, and the stability later better. That means the results won't be comparable to yours. However, the alternative flushes should be comparable.

Given the construction of the Peppina, it may blow away everything you've tried so far

I'll post all the data, and various ways of describing it, and then report back here. I have a busy weekend, so this will probably happen Monday.

[barry]

However, if I were an HX manufacturer, I'd reject it and refuse to submit my machines to this test. They could justly claim that the inter-shot procedure is systematically biased in favor of dual boilers.

as one of the folks who spent a lot of time on the phone in deep discussions about the development of the protocol, i'll throw in my $.02:

the protocol isn't biased for or against any machine or machine system. the objective presented for machine performance was stable, repeatable brew temperatures over a variety of duty cycles. the protocol reflects that objective. we knew the HX people might have kittens over this, and really went out of our way to make sure the protocol fairly measured machine performance. we also knew that the protocol would show the cold start of marzoccos. the duty cycle variation was specifically intended to show weaknesses in machine performance, for both types of machines. note the wbc/usbc competition schedule has a variable duty cycle: the competitor can use it as much as they like during setup, and then the machine sits for a few minutes (an unknowable period), and then it is put back in service at an unknowable duty cycle.

No matter what the HX design, the equilibrium shot temperature will depend on the shot making pace. If the pace is slow, the temperature over a long series will settle at a higher level than if the pace is fast. This has to be, since at the slower pace, the HX is heating up more. You are dumping the first four very slow measures for inter-shot calculations. But you still have the pace accelerating from 2 min 35 seconds to 45 seconds between shots. This will always cause the brew temperatures to drop for the later shots.

recall that it was precisely this phenomena which pushed me down the path of machine modification, and recall that the manufacturer denied any such problem existed.

The easiest, and I think correct, fix is to allow each manufacturer to specify the amount for the preshot flushes.

an easy fix, no doubt, except that no manufacturer specifies that any such flush is required during the routine operation of their machines. this harks back to coffeegeek's test procedure, which only follows "out of the box" performance, not how the machine performs after tweaks and surfs and other machinations which are not specified by the manufacturer in the operating instructions. if the manufacturers want to stand up and say, "yeah, our machines are only stable if you do x, y, and z," then x, y, and z may become part of the protocol.

in essence, you're suggesting the hx manufacturers be allowed a self-determined handicap because their machines won't perform even up to their own specifications and marketing. how about getting the manufacturers to make machines which will perform up to the desired specs? the goal was to find out how machines really perform, not how they perform after the barista spends ten years becoming intimate with the quirks and foibles of the machine. yeah, i can start my MGB w/o any problem, but valet attendants invariably cannot (it's that manual choke thing).

[barry]

The test does not reflect WBC use, which is to pull 3 shots at each group at roughly around 2, 5, and 9 minutes into the presentation, with the final two rounds having looser requirements from the taste point of view since they are being used as the base for other drinks.

i can't recall seeing that dynamic with any regularity (or even at all). maybe that's the ideal dynamic, but reality is much much different. i've even watched people just use one group for the whole competition.

[another_jim]

in essence, you're suggesting the hx manufacturers be allowed a self-determined handicap because their machines won't perform even up to their own specifications and marketing. how about getting the manufacturers to make machines which will perform up to the desired specs? the goal was to find out how machines really perform, not how they perform after the barista spends ten years becoming intimate with the quirks and foibles of the machine. yeah, i can start my MGB w/o any problem, but valet attendants invariably cannot (it's that manual choke thing).

I'm still not getting this. I'm suggesting that specifying a flush of two seconds is arbitrary and unrelated to to the functionality of the test. Since it is arbitrary, you'll get questions of "why 2 seconds". Your answer is "for consistency." So why not 3 seconds or 8 or whatever. And why be consistent in this regard? It's unrelated to what is being tested, so it's like specifying how high the machine should be. If a manufacturer wants some other timing, I don't get how it would in any way affect the procedure.

You may be right, manufacturers may be so arrogant that they'll say their machines can violate the laws of thermodynamics. But I suspect this is more the department of the sales reps; and the manufacturers wanting to underwrite the WBC (they'll be lining up when it catches on more) will take a close look at the entry requirements.

[barry]

I'm still not getting this. I'm suggesting that specifying a flush of two seconds is arbitrary and unrelated to to the functionality of the test. Since it is arbitrary, you'll get questions of "why 2 seconds". Your answer is "for consistency." So why not 3 seconds or 8 or whatever. And why be consistent in this regard? It's unrelated to what is being tested, so it's like specifying how high the machine should be. If a manufacturer wants some other timing, I don't get how it would in any way affect the procedure.

as i mentioned on a.c, yes, there is some arbitrariness in the "2 seconds". we had to have a time for the wbc required screen flush. one second seemed too difficult to handle during testing, three seconds seemed too long (and might have an influence on the thermal performance), and we didn't think we could get away with saying "blip the pump". this is not a thermal flush; this is a screen flush only. it has nothing to do with marzoccos. it has nothing to do with heat exchangers. no manufacturers currently specify a flush of any sort, so you're creating issues that aren't there.

and yeah, we spent a lot of time discussing this sort of stuff.

[malachi]

To be clear - this is not a temp-related flush but rather a cleanliness-related flush?

[barry]

To be clear - this is not a temp-related flush but rather a cleanliness-related flush?

iirc, that was my understanding from our discussions last winter. i've reread what was published here and it says "to simulate a temperature equalization flush" in section 5.2(B), which does not appear in my draft copy from last winter. my comment notes to greg talk about how my experience has been that flushes after the portafilter is removed tend to be screen cleaning flushes and those before the portafilter is reinserted tend to be thermal flushes, so perhaps there was a misunderstanding between us on the roll of the reinsert flush. iirc, those comments had more to do with whether two flushes were needed at all, and i think further phone conversations included a discussion of the wbc rules allowing screen cleaning flushes either before or after, just so long as it happened somewhere, and that we decided on two flushes to account for the before and after, since the rules aren't clear. i'm almost positive the 2 seconds was determined as a comfortable compromise between the too-short 1 second and the too-long 3 seconds (in reference to a screen cleaning flush). i would like to hear from greg about what his notes say, cuz i really can't see him agreeing that 2 seconds could be considered a useful simulation of any sort of temperature related flush.

of course, my memory might be whacked, in which case i apologize in advance.

[malachi]

If simulating cafe situations, then it would indeed be to mimic a cleaning flush (given the published description). That "pulse flush" is best practices in cafes and is, as noted, designed for cleanliness purposes rather than temp purposes.

[gscace]

iirc, that was my understanding from our discussions last winter. i've reread what was published here and it says "to simulate a temperature equalization flush" in section 5.2(B), which does not appear in my draft copy from last winter. my comment notes to greg talk about how my experience has been that flushes after the portafilter is removed tend to be screen cleaning flushes and those before the portafilter is reinserted tend to be thermal flushes, so perhaps there was a misunderstanding between us on the roll of the reinsert flush. iirc, those comments had more to do with whether two flushes were needed at all, and i think further phone conversations included a discussion of the wbc rules allowing screen cleaning flushes either before or after, just so long as it happened somewhere, and that we decided on two flushes to account for the before and after, since the rules aren't clear. i'm almost positive the 2 seconds was determined as a comfortable compromise between the too-short 1 second and the too-long 3 seconds (in reference to a screen cleaning flush). i would like to hear from greg about what his notes say, cuz i really can't see him agreeing that 2 seconds could be considered a useful simulation of any sort of temperature related flush.

of course, my memory might be whacked, in which case i apologize in advance.

You're right. I recall the discussions and it's bad writing on my part.

-Greg

[wogaut]

Hi Greg,

I really welcome finally having a standardized test procedure, that hopefully most people will use!

A question about the idle time between shots:

the first idle time is shown as 10 minutes. Is that before or after the first shot of the 14 repeats?
If it's before, I'm confused since you write about having the machine get to equilibrium for an hour or so (after backflushing?) and then having it sit idle for another 10 minutes doesn't do anything.

I was looking at your graph in the GS3 temperature thread, and it looks as if the first shot occurs after 10 minutes.

Your clarifications are highly appreciated!

Wolfgang

[gscace]

Hi Greg,

I really welcome finally having a standardized test procedure, that hopefully most people will use!

A question about the idle time between shots:

the first idle time is shown as 10 minutes. Is that before or after the first shot of the 14 repeats?
If it's before, I'm confused since you write about having the machine get to equilibrium for an hour or so (after backflushing?) and then having it sit idle for another 10 minutes doesn't do anything.

I was looking at your graph in the GS3 temperature thread, and it looks as if the first shot occurs after 10 minutes.

Your clarifications are highly appreciated!

Wolfgang

The first idle time is before the first measurement. We felt that while we specified that the portafilter and thermometer were to remain in the group, and that the machine was to be hot for an hour, we hadn't adequately prevented anyone from flushing or otherwise dicking around immediately before getting the first measurement, so we specified that you had to wait the 10 minutes before making the first measurement. That makes it really clear.

-Greg