As I mentioned in my latest update to the single group Slayer review, I made some modifications to the WBC protocol for my temperature testing to make it a little more reflective of (demanding) home usage patterns. From the review:

I wanted to preserve the core of the shot-pulling procedure [of the WBC protocol]. Below is an explanation of the workflow; It is unchanged, except inasmuch as the timings are a little more detailed and expressed in a way that I found easy to implement.

Time   Duration  Action
0:00   0:15      Start Datalogging
0:15   0:10      Remove portafilter (shot prep)
0:25   0:02      Flush 2 sec.
0:27   0:03      Insert portafilter
0:30   0:25      Pull shot 1
0:55   0:01      Remove portafilter (flush)
0:56   0:02      Flush 2 sec.
0:58   0:07      Insert portafilter at end of 7 seconds
1:05   x:xx      Idle Interval
y:yy   0:10      Remove portafilter (shot prep)
z:zz   . . .     [ repeat ]

This retains the structure of the WBC preparation spec almost exactly:

• Portafilter out for 15s prior to the shot, with a 2s flush right before (simulated shot preparation)
• 25s shot time rather than a volumetric shot simulation. This was for simplicity, and because 25s shots fall in the water output range specified by the original protocol
• Portafilter out for ~10s immediately after the shot with a 2s "clearing flush" right after removing it.

The changes I made were my choice of shot intervals. Rather than 14, I opted for only 10 shot simulations, using the following intervals between each:

1. 10:00+ ("idle time" before pulling the first shot)
2. 5:00
3. 2:00
4. 2:00
5. 1:00
6. 1:00
7. 1:00
8. 0:30
9. 0:30
10. 0:30

In addition to reducing the total number of shots, I omitted the 10s interval series at the end, instead using a minimum interval of 30s. I opted to use 3 each of the 1:00 and 0:30 intervals at the end. The point of this is to make the test reflective of a home-user's pace.

A copy of this protocol in Excel spreadsheet format is available here.

The results for the Slayer espresso machine are posted in the review. I was hoping to get some comparable data for other machines, and indeed I already have received some privately. I also wanted to start this thread to open a discussion about the protocol.

From a sister topic, Slayer single group - Temperature accuracy and stability issues, forum member jonr posted this comment on my procedure:

jonr wrote:I'll argue that repeat-ability should also be tested over much longer periods than shot-to-shot. For example, do room temps, drafts, barometric pressure, limited warm-up time, input water temp, etc effect results?

The answer is certainly yes, and the question is how much-it's almost certainly only a little bit. I can't say I have a whole lot of interest in this question, because I am not under the illusion that brew temperature is anything more than a number. I am interested in ensuring that for any given session, the machine is consistent. But if I set the brew temperature at 200°F today, and the actual average brew temperature is 201.5°F, and then in 2 weeks when a cold front comes through and the house is 5°F cooler, making the brew temperature 1°F cooler on average, who cares? Adjusting temperature by taste at every session is a core barista competency.

My point is that testing those effects in a controlled way is a significant amount of extra work with minimal added value. This is exactly the kind of thing I want to avoid in trying to make a meaningful protocol that others could repeat on other machines, producing data that is meaningfully comparable.

On a related note, forum member Devin (indend007) has used this procedure to examine the effect of preinfusion on his GS/3 MP and Speedster (here). I thought the shape of the Speedster's temperature profile was surprisingly similar to the Slayer's. I was also impressed at the size of the impact of pre-infusion the GS/3 MP. Check it out if you're interested.

Just wondering - the WBC protocol can incorporate steaming. I suspect that a typical 'home' use scenario with this many shots is always going to involve a lot of milk - and steaming may have a big impact on temperature stability in some 100v-120v machines.

I am not familiar enough with any of this to suggest how to incorporate/modify the WBC steaming protocol. I am just thinking that the steaming requirement would change the comparative results - and thus impact consumer buying decisions.

Just a thought.

For 'top tier' 120V single group machines like the Slayer, Speedster, and GS/3, they do have heat exchangers that rely on the steam boiler to pre-heat the brew boiler water, so losing pressure and temperature in the steam boiler could theoretically impact the temperature stability. That said, all these machines default to giving full priority to the brew boiler, meaning that it gets heat when the heating algorithm calls for heat, regardless of the steam boiler state.

So the main risk at high volume is losing full steam pressure much more than losing brew temperature stabiilty, but I think for machines like this we'd be talking about a pretty preposterous (high-volume catering) volume of steaming before you'd see much of an effect. Moreover, when it comes down to it, if you're steaming enough milk to throw the machine's temperature stability off, the chance that you'd notice in the big milk drinks you're probably making is... virtually nil.

Still, I don't have any gathered data to back my expectation up. I think it would be easy to incorporate a number of 10-15 second steamings into the routine with minimal trouble. I'm open to proposals on that... Not sure when I'll get around to trying it, though.

I did a temperature test on speedster that was setted to group boiler to 200f.
The test routine is same as Shadowfax mentioned, he mailed to me a modified WBC routine description and his slayer test data what you can see on "Slayer review thread".

I really enjoyed with a being simplified routine. And it's not too annoying to testing.
Here my data on KvdW speedster.

It shows quite impressive result, and I found a slow ramp-up time on SPST and I made a comparison thread about it on "The effect of pre-infusion flow-rate on brew temperature profile".














Speedster has very tight temperature line, furthemore go foward to 30sec and 1 minute interval test, gap of the line more narrow.
I have a data on LM GS/3 same routine as well, but it was made on celsius data, so after converting I'll write it up here too.