another_jim wrote:Oy! Now I'll have to buy a logging meter.
The WBC test is an idealized replication of the heavy but highly irregula traffic patterns in cafes. A home machine protocol should be based on a similar idealization of home use.
My own home use and my sense of most others is:
-- Mostly single shots after long (excess of 1 hour) idle periods. Should just 1st shot stability and repeatability be conidered; or also the pita of getting the machine prepped?
-- Occasional multiple shots when testing blends or other enthusiasts drop by (note to handle the common the 2nd shot is always better effect)
-- Occasional multiple milk drinks for guests, followed perhaps by a straight shot for oneself.
-- While the overwhelming use is single shots from idle; people's justification for buying HXs is their capability of handling company and of doing multiple shots consistently. So while the from idle capacity may be weighted highly, the second two should be weighted higher than their actual occurance, since this is the raison d'etre of the machines.
I'm not proposing a test; I'd just like some feedback on whether this is a realistic summary of home use patterns.
I don't have any problem with including all these things in a test, although I would not combine the straight shot performance with the milk drink performance; the results should be reported separately. The solitary straight shot plus multiple straight shot runs could be combined in one test, but the milk drink results are going to be so different and so subjective (my view anyway) that they should be reported separately and the potential purchaser or user could weight them however he or she wanted based upon how they would intend to use the machine. Combining milk and straight shot results is, in my view, just going to produce meaningless scattershot results that no one is going to be able to interpret and that won't highlight the actual differences among machines.
As to what the user has to do before he initiates a shot run after idle (e.g. flushing, nothing, or whatever) this has to be determined by the individual tester. Since we are looking largely at first shot consistency and reproducibility, each machine model is apt to have approaches that will work and approaches that will not. It is hoped that by having a number of us out there doing this sort of testing we could actually come up with one or several methods that work with each machine to get it to perform to its best potential. What I have described on my Cimbali, the low boiler setting miniscule flush technique, is something I discovered by doing just this sort of testing and something I've not read about before. Perhaps people will come up with novel ways of getting the best out of a whole host of machines and this will help out everyone who either owns such a machine or is contemplating buying one. The only requirement as regards the test per se would be that the tester explain what it was he did in order to get the results that he posted. It would be up to those discussing these results in the future to determine if the preparation/flush routine was such a PITA as to render the machine less desireable; simple preparation/flush routines are obviously going to be preferred.
I should note, I should EMPHASIZE, that the purpose of setting up a home user protocol is not to come up with some sort of number (like a report card) that would enable one to say that a Brewtus scored 97 and a Silvia scored 85. I don't see any point in doing machine testing with that sort of endpoint. Instead, the idea would be to come up with some sort of definition of temperature stability in actual random home usage conditions, accompanied by a lot of descriptive material and some shot temperature graphs. One would then have a whole lot of DATA that one could use in comparing machines and trying to figure out how much one would gain if one upgraded from what one has to another machine. You could look at graphs of your own machine optimized for best performance and compare them to other machines. In the end, the person making the comparison is still going to have to consider other usability factors that are beyond the scope of this sort of limited temperature stability testing.
So, for example, you could have straight shot tests run at, say, 4 different temperatures chosen within the band of temps one might seek for straight shots. One could arbitrarily select 198, 200, 202, and 204F, which would more or less cover the range (or 203, 201, 199, 197, I care not). Each of these temps could have a total of say, 10 or 15 shots done with maybe half of them done as single shots after an idle period of 10 minutes or more (or pick 20 or 30 minutes, I just don't think it will matter much in the results and it would make the test take a lot longer to get through hence dissuading a lot of people from actually doing it). The other half of the total number of shots could be two or three two shot series and one or two longer series. All the shots should be superimposed on top of each other in a graph to see how reproducible the shots are regarding shape and temperature, and simple statistics could be applied.
We would probably find that a machine did better with one or two or three of the 4 temps tested and less well with the rest of them. The data could then either be combined into one number or statistic or expressed as results with the 4 different temps tested.
On to a milk test. Here is where I have no idea exactly what one would be testing and how one would express the results. Since tight temperature control of the espresso shot that is going into a milk drink probably isn't all that important, I'm not exactly sure what we would be measuring; frothing power, microfoam quality, speed of recovery? All of these things are going to vary by how the boiler temp is set and what steam tip one chose to use. I think you could more or less determine your results by how you gamed the test at the start by your boiler pressure/temp setting. I think the test parameter needs to be very focused or we are not going to know what to do with the observed results or how to use them to compare machines.
ken
)
