I've done the WBC 14 Shot "test" now several times with my Cimbali Juniors, a couple times with my pourover vibe pump machine (circa 1995) and with the Rotary D1 Junior, both on P-stat and PID control. All of the shot series are quite similar, and they differ only by degree. What they show is the inability of the heat transfer mechanism to heat water quickly enough to maintain temperature. The first few shots, with no initial flush, are not going to show how a HEX machine is generally used, but that has been discussed before.

So, before looking at the graphs I'll say that the test by design "excludes successful completion" by a 110v HEX machine, since the first few shots will be too hot and the machine can't heat water fast enough at the end to maintain shot temperature.

The WBC test was not developed for machines like this, which vary from semi-commercial home machines to those designed for catering and small restaurant use. The Cimbali Junior would be at the upper end of that range, but no one would seriously use one in a high volume espresso bar, escept maybe as an auxilliary or backup machine. The marketing department rates these machines at 120 "cups" per hour. Undoubtedly they are talking in the Italian sense of a double spouted PF pulling 2 single shots per minute. I would suspect that this is very optimistic thinking on their part, especially for the 110v model. Having briefly met several of the Cimbali sales reps at the SCAA in Seattle, they did not inspire me to trust them any more than I would a car salesman (apologies to car salesmen).

Even if these machines were really capable of producing shots at this rate, the WBC procedure for the last 6-8 shots greatly exceeds the stated shot production rate of these machines.

I will post below three graphs showing how my machines did on this "test," (not good), and throw out the question, has anyone else tried running this WBC procedure on a 110v Heat Exchanger machine? Few or none of us home barista types would even attempt to produce shots at such a hectic rate as the last 6-8 shots in this series, so the test results aren't applicable to the way we make espresso, and never have been stated to be so.

Here are the graphs:


Pourover S1 Junior at a relatively high pstat setting (I think it was around 1.20-1.35bar):





D1 Junior with Pstat Control:




D1 Junior with PID Control:




ken

HB wrote:Never, but I can guess the results. The section below from WBC Procedure for the Measurement of Brewing Water Temperature in Espresso Coffee Machines spells "game over" for any heat exchanger machine I've used:

• 5.3 Testing pattern:
  Specification: The length of the idle interval for item A of the Test Procedure shall be:
  
  image: http://i6.photobucket.com/albums/y230/h ... table1.jpg

Ignoring the question of flushes for shots 1-4, the temperatures will definitely tank for shots 9-14.

I don't follow the WBC, although I have seen parts of a regional competition. People need to realize that a machine that meets these requirements bears no relationship to a machine that will meet (and exceed) their needs in a home setting. Since this place is called "home-barista.com" I won't address the needs of other types of users.

I ran these tests on my machines because they are considered to be one group commercial machines and I had just spent oodles of moolah on this spiffy Scace device and Fluke, and damn! I'm gonna get my money's worth! So I did it.

I respect the authors of this protocol and I think they did the best they could with what they had to work with. BUT, they have set up a protocol that probably excludes just about every type of machine other than double boilers or commercial single boilers, if such a sort of creature exists (never heard of one). On the other hand, in an attempt to remove one variable from the equation I think perhaps the point has been missed that there are so many other uncontrollable variables that reducing the temperature variable really solves little. For example, can the WBC control how well local milk supplies froth? How about the water supply? What if they run the test one year in an area with soft water and the next year in an area whose water varies greatly? Or the beans used-- do they want to force everyone to use the same beans since maybe some barista is going to take advantage by his/her choice of bean supplier?

I think there are lots of variables and no way to control them all unless the judges want to remove all variables and then you might as well just use the lowest common denominator for all the constituent ingredients and in the end no one is going to want to taste the end product.

Don't, I repeat, DON'T, use this protocol or results from proud owners/users who have used this protocol, for judging suitability of a machine in a home setting. There are temperature stability issues that are potentially important in a home setting and these issues are NOT addressed AT ALL in this protocol. Home espresso users are by definition virtually always low volume users. They pull 2 or 3 or 4 or 5 or 6 double shots in a day, and some of those shots are used in milk drinks. The interval in between shots is variable which means that more shots than not are pulled after an uncertain idle period. The fact that XYZ machine can pull "perfect profiles" 8 shots into a variable 14 shot WBC series is irrelevant to their usage pattern. A machine that excels on such a test is more likely than not to fail when used on an intermittent home usage schedule.

There are a great many machines out there that can be successfully used in the home. Virtually all of them are going to require user intervention in order to get good shots out of them, on the typical intermittent usage schedule of a home user.

ken

As was discussed in the WBC protocol thread this protocol isn't really designed for HX machines . A big reason being the fixed flush volume regardless of idle time. The protocol has little value to me as a home (wannabe) barista. The fact I know I can pull a shot no sooner than 35sec and within 1 minute or so after either initial idle cooling flush or the screen flush and PF wiggle of the previous shot and hit my temp within <1F on a consistent basis based flush & go countdown from end of flash sound is what counts to me. No way can I pull a shot with only 25sec after end of previous shot without temp drop off and I don't care!

Using Thermofilter extensively to dial in my flushes etc. I have a pretty good idea how my Bricoletta would perform following the WBC protocol. No way would the series of shot temps would be even close. Might be interesting to guesstimate what they'd be then run the protocol! Now the protocol could have some value in learning what is really needed to properly operate a particular new to you HX machine, but not as any real meaning as to how well an HX can be used with an experienced person at the controls.

Compass Coffee wrote:As was discussed in the WBC protocol thread this protocol isn't really designed for HX machines . A big reason being the fixed flush volume regardless of idle time. The protocol has little value to me as a home (wannabe) barista. The fact I know I can pull a shot no sooner than 35sec and within 1 minute or so after either initial idle cooling flush or the screen flush and PF wiggle of the previous shot and hit my temp within <1F on a consistent basis based flush & go countdown from end of flash sound is what counts to me. No way can I pull a shot with only 25sec after end of previous shot without temp drop off and I don't care!

Using Thermofilter extensively to dial in my flushes etc. I have a pretty good idea how my Bricoletta would perform following the WBC protocol. No way would the series of shot temps would be even close. Might be interesting to guesstimate what they'd be then run the protocol! Now the protocol could have some value in learning what is really needed to properly operate a particular new to you HX machine, but not as any real meaning as to how well an HX can be used with an experienced person at the controls.

I don't know your machine and I don't know the curves you have produced; if you have posted them, please point me to them.

If a protocol for machines to qualify in a high profile event like the WBC excludes whole classes of machines and only allows double boilers (and I believe that Greg has posted that a HEX machine, presumably a 220v version, did fine on this test) then one would have to question the value of such a test where the winners have been pre-ordained by the chosen methodology.

I've watched parts of a couple of these competitions and although I think they probably do motivate employees in the few good cafes out there, those of us who never or almost never have a chance to go to such places really don't benefit much. If the purpose for developing the protocol was solely for choosing machines for the WBC, then I think this should be entirely divorced from marketing attempts on the part of whichever company "wins" this contest. Of course this is not going to happen because (presumably) fund raising through sponsorship is a major part of the funding for such an event as the WBC.

Since the protocol seems to have been designed to pick a certain class of machine to "win," and then the winner gets to sponsor the event and to use the fact that they "won" in advertising, I think it would be a bit (a lot) more honest to simply have a sponsor for the event, let the sponsor advertise that they are the sponsor and they are supplying the machines for the contest, and leave it at that. This seems to work well for sponsorship in Olympic events. Instead there is this mystique of a competition for choosing the most stable machine and then this machine just ends up having been more or less selected by the selection criteria and so on and so forth.

I happen to like the company that is the sponsor for the upcoming events and I admire what they are trying to make for home users; don't get me wrong on that. But I think this whole process is a bit disingenuous.

As to home users, again, I repeat, temperature stability is a very hard thing to deliver, and much harder to deliver in a low volume setting like a home. The importance of temperature stability has not been established by any sort of scientific means. Simply reading that someone claims they can taste brew temperature differences of 0.1 degree F doesn't prove that it is so. I think it will be proven by those who are conscientious enough to test it and honest enough to report the results they have found, that no machine currently existing can deliver a flat temperature profile through all the possibly good brew temperatures, even if it was true that this was desirable (and I'm not sure of that, either).

ken

Ken Fox wrote:
I don't know your machine and I don't know the curves you have produced; if you have posted them, please point me to them.

If a protocol for machines to qualify in a high profile event like the WBC excludes whole classes of machines and only allows double boilers (and I believe that Greg has posted that a HEX machine, presumably a 220v version, did fine on this test) then one would have to question the value of such a test where the winners have been pre-ordained by the chosen methodology.

I've watched parts of a couple of these competitions and although I think they probably do motivate employees in the few good cafes out there, those of us who never or almost never have a chance to go to such places really don't benefit much. If the purpose for developing the protocol was solely for choosing machines for the WBC, then I think this should be entirely divorced from marketing attempts on the part of whichever company "wins" this contest. Of course this is not going to happen because (presumably) fund raising through sponsorship is a major part of the funding for such an event as the WBC.

Since the protocol seems to have been designed to pick a certain class of machine to "win," and then the winner gets to sponsor the event and to use the fact that they "won" in advertising, I think it would be a bit (a lot) more honest to simply have a sponsor for the event, let the sponsor advertise that they are the sponsor and they are supplying the machines for the contest, and leave it at that. This seems to work well for sponsorship in Olympic events. Instead there is this mystique of a competition for choosing the most stable machine and then this machine just ends up having been more or less selected by the selection criteria and so on and so forth.

I happen to like the company that is the sponsor for the upcoming events and I admire what they are trying to make for home users; don't get me wrong on that. But I think this whole process is a bit disingenuous.

As to home users, again, I repeat, temperature stability is a very hard thing to deliver, and much harder to deliver in a low volume setting like a home. The importance of temperature stability has not been established by any sort of scientific means. Simply reading that someone claims they can taste brew temperature differences of 0.1 degree F doesn't prove that it is so. I think it will be proven by those who are conscientious enough to test it and honest enough to report the results they have found, that no machine currently existing can deliver a flat temperature profile through all the possibly good brew temperatures, even if it was true that this was desirable (and I'm not sure of that, either).

ken

OK, I gotta try to nip this misconception in the bud. The WBC procedure was designed to test machines period. None of us give a rats ass what wins or doesn't win in terms of machine configuration (hx or twin boiler) and if you read the procedure you'll see we don't say how to pick winners or losers. Testing machines without performing big cooling flushes gives lots of useful information that benefits home user or pro because you learn what is deficient in the design of the machine, and you get an idea of what is needed to compensate. The fact that a specific machine may not produce the same temperature at duty cycles shorter than a specific time may be of interest to anyone. The magnitude of correction required to make a machine achieve the same brew temperature can't be determined until you know the baseline without any flush. So you gotta start somewhere.

WRT 110V machines doing well enough on the tests, my Astra Gourmet can keep up at all duty cycles. It suffers from needing cooling flushes to get the temp down like any e-61 machine, but it can keep up with very short duty cycles just fine. I did some calculations on a perfectly efficient system a while ago that showed you needed 800W of heat to brew a double shot a minute, which is the fastest WBC duty cycle (10 sec wait, 15 sec dose / tamp, 25 second shot, 10 sec to knock out the puck and flush the screen). A 50% efficient machine would need 1600W heating element. The Astra's is 2kW, so you'd expect it to keep up. 1400W machines may not.

Also, the LM GS3 can keep up no problem. Not only can it keep up, the machine needs pretty much no flushing ritual at all, according to the data I have and that got posted on HB.

Ken, I disagree with your comment that the importance of temperature stabilty has not been proven. It most certainly has been studied and conclusions have been drawn that form the basis for various standards of espresso. The question is more properly "what is the stability requirement?" and that is a much harder question to answer. The opinion of pro baristas who are very highly regarded in the industry, that have tested the GS3 claim that they can taste the difference in 0.3 F changes in setpoint temperature. My own measurements put the reproducibility of the GS3 at around a half degree. Taking the square root of the sum of the squares of 0.3 and 0.5 gives you 0.6, which is prolly a good guess on what actually makes a meaningful difference. That doesn't mean that one shot brewed 0.6 degrees different is gonna suck compared to another one, it just means you can taste a difference. There are only a couple of machines out there that can produce that level of reproducibility and one of them is still a prototype. So we ain't there yet when it comes to reproducibility. With regard to temperature profile, the argument that flat line is good or bad is pretty meaningless really. It's a long way down the road before anyone's gonna develop a machine with variable profile that can test the argument, if one blindly assumes that we care about the temperature above the cake. The temperature inside the cake varies with time and position in the cake. It is most assuredly not constant during the brew cycle. The reason that we oughtta be concerned with flat line temperature above the cake is that given the technology that we use to pull this stuff off, a flat line is more likely to give you reproducible brew temps from shot to shot that can get within a factor of 2 of the minimum detectable difference temperature.

OK, I better go do some work now.

-Greg

Ken Fox wrote:I think it will be proven by those who are conscientious enough to test it and honest enough to report the results they have found, that no machine currently existing can deliver a flat temperature profile through all the possibly good brew temperatures

The "possibly good" brew temperatures fall within a pretty narrow range (~196F-204F), so I think you're nuts.

Also, I'm conscientious and honest enough to report that I have zero interest in testing your hypothesis.

Dr. Schecterstein:-)

I don't think it would be that hard to test and you certainly have a machine to test it on. What I showed on my testing, with my D1 Cimbali PID'd, is that I could come up with maybe 4 reproducible more or less flat line shot profiles between about 200 and 204F, with less reproducibility below that and above that was not tested (and who'd want to anyway unless you underroasted the beans to begin with )

Maybe with further PID tuning and a more skilled barista:-) it would be possible to get some finer control than that. What I'd like to know is what is state of the art on what other people own, be they prototypes, be they production machines, be they hacked machines like mine.

To me, the key to a low volume (e.g. home) user, as regards temperature stability, is the ability to go to their machine time after time, after varying idle periods, and pull the same temperature profile when they are requesting a certain temperature, e.g. 103 on the display, is what it is, but it is always more or less the same.

Please test this, run appropriate statistics, and report back to us.

ken

p.s. I'm enroute to visiting electronic luddites in Alaska who don't believe the internet exists and I don't think I can respond to your (predicted to be erudite) response for a few days.

gscace wrote:OK, I gotta try to nip this misconception in the bud. (snippage)

WRT 110V machines doing well enough on the tests, my Astra Gourmet can keep up at all duty cycles. It suffers from needing cooling flushes to get the temp down like any e-61 machine, but it can keep up with very short duty cycles just fine. I did some calculations on a perfectly efficient system a while ago that showed you needed 800W of heat to brew a double shot a minute, which is the fastest WBC duty cycle (10 sec wait, 15 sec dose / tamp, 25 second shot, 10 sec to knock out the puck and flush the screen). A 50% efficient machine would need 1600W heating element. The Astra's is 2kW, so you'd expect it to keep up. 1400W machines may not.

Also, the LM GS3 can keep up no problem. Not only can it keep up, the machine needs pretty much no flushing ritual at all, according to the data I have and that got posted on HB.

Ken, I disagree with your comment that the importance of temperature stabilty has not been proven. It most certainly has been studied and conclusions have been drawn that form the basis for various standards of espresso. The question is more properly "what is the stability requirement?" and that is a much harder question to answer. The opinion of pro baristas who are very highly regarded in the industry, that have tested the GS3 claim that they can taste the difference in 0.3 F changes in setpoint temperature. My own measurements put the reproducibility of the GS3 at around a half degree. Taking the square root of the sum of the squares of 0.3 and 0.5 gives you 0.6, which is prolly a good guess on what actually makes a meaningful difference. That doesn't mean that one shot brewed 0.6 degrees different is gonna suck compared to another one, it just means you can taste a difference. There are only a couple of machines out there that can produce that level of reproducibility and one of them is still a prototype. So we ain't there yet when it comes to reproducibility. With regard to temperature profile, the argument that flat line is good or bad is pretty meaningless really. It's a long way down the road before anyone's gonna develop a machine with variable profile that can test the argument, if one blindly assumes that we care about the temperature above the cake. The temperature inside the cake varies with time and position in the cake. It is most assuredly not constant during the brew cycle. The reason that we oughtta be concerned with flat line temperature above the cake is that given the technology that we use to pull this stuff off, a flat line is more likely to give you reproducible brew temps from shot to shot that can get within a factor of 2 of the minimum detectable difference temperature.

OK, I better go do some work now.

-Greg

Greg,

You are a scientist and I doubt that if you reread your last paragraph you would really conclude that XYX pro baristas claiming they can taste 0.3 degree F changes in shot temperature constitutes "proof" in any sort of scientific sense.

I read the instructions for the WBC test as carefully as I think anyone could, and the pace of shots that you describe above is different than I interpreted. For one thing, I thought shots were supposed to be measured over THIRTY seconds, which in reality means you have to run the shot for 31 or 32 data points since the first two measured temps are often the same because the brew water hasn't gotten down to the temperature sensor in the first second. I did not put two identical numbers in for the first two measurements, so I needed an extra number in order to have 30. My interpretation when you have two identical numbers for the first two seconds measured is that one hit the datalog button on the fluke a hair of a second too soon so it was not really data but junk. Also, you are counting one of the ten second intervals twice, I think. In the last 6 shots it is, simulated dump of puck/flush screen (10 seconds), 10 second wait, 15 seconds dose, followed by shot. The way I did this, which is how I read the procedure, amounted to 1 double shot plus two small "cleaning" flushes per 50 seconds, which is well beyond the rated shotmaking duty cycle of my machines. Whether they could have "kept up" with 25 second instead of 30 second or 31 or 32 second shots I don't know. Maybe I tried too hard to follow the rules as I read them.

In any event, I think being able to control brew temperature is cool and desirable. I don't know how many temperature points within the typical range (Andy says it is 196 to 204, and since he's smarter than anyone else I know, that's a range of 8 degrees F). Do we need 3 repeatable temps in this range, 5, 7, 37? I don't know. I do know that I'll have to see some blind tasting results from these pro baristas and others who claim extraordinary tasting acuity by shot temperature, before I will accept as scientific fact a need for 25 or 27 different temps in this range (one every 0.3 degrees F).

ken
p.s. I'll be offline for a few days and won't see your response until then, most probably.