malachi wrote:I totally understand.

In the meantime - is anyone else interested in trying to collaborate on Use Cases with the goal of using them to create a generalized protocol for evaluating brew temperature performance for home-use espresso machines?

So Chris, are you accepting my challenge to pull maybe 10 random first shots from idle on your GS3, at each of several requested temperatures, and plot out the results for each temp? I am 1200 miles from my machine right now and have no idea how it would do, and can't test it until the middle of next week.

You might end up with median shot temps that differ from the requested temps so I'll let you request a different temperature on the panel if necessary in the attempt to get a median temp close to what we are testing.

How does 199, 200, 201, and 203F sound? Or do you have 3 or 4 other temps you'd like to test?

ken
p.s. Sean, want to get in on this with your modded Brewtus?

Ken Fox wrote:There aren't that many people out there with the required equipment (Scace Thermofilter and datalogger) who are going to do this and although interesting I think nothing will come of this discussion. I've decided that I'm going to do some very straightforward and simple testing that can be replicated by anyone who finds it interesting and ignored by everyone else. An example would be datalogging random first shots after variable idle periods at an intended shot temperature; it can be repeated for as many intended shot temperatures as desired. To perform this test one would walk up to an idled machine, do whatever one needs to do with the idled machine first to pull a good shot (in my case, that would be a 45ml flush; for a GS3, perhaps nothing), then reinstalling the Scace device and datalogging a 30 second shot. This will be done to a total of 10 shots after random idle periods of 10 minutes or more. All 10 shot plots will be put on one graph, and it will be labelled as to boiler temp, pstat setting, and intended (or median obtained) shot temp.

Suggest including a few shorter than 10min idle times like maybe 3, 5, & 8min too. Why? Real life home usage. You plan to pull a couple shots and pull the first. Just then the phone rings. 5 minutes later you're back to pull the second shot...

As Chris suggested multiple times come up with the home use cases then the protocol.

As to the other stuff, I'm not trying to encourage anyone to buy a Cimbali nor am I recommending that anyone hack their machine with a PID as I have done; they can look at what I am getting from this modified machine and if it interests them they can pursue this on their own.

This machine has enough steam power to clean all the bathrooms in my house in 3 minutes:-) It is of no interest to me to examine milk drink performance on this machine because it is a waste of my time to "prove" something like this that is already widely known; this machine can froth like there is no tomorrow.

Which sounds to me like you're really not trying to come up with a realistic protocol to examine home machines in general but rather a protocol to validate your machine... Which is fine, go for it.

Ken Fox wrote:There aren't that many people out there with the required equipment (Scace Thermofilter and datalogger) who are going to do this and although interesting I think nothing will come of this discussion.

Which of course is why Jim suggest not using the Thermofilter but rather a temp measuring method readily available to a majority of home users.

malachi wrote:In the meantime - is anyone else interested in trying to collaborate on Use Cases with the goal of using them to create a generalized protocol for evaluating brew temperature performance for home-use espresso machines?

I'm game. Maybe should start a new thread.

miKe mcKoffee wrote:Which sounds to me like you're really not trying to come up with a realistic protocol to examine home machines in general but rather a protocol to validate your machine... Which is fine, go for it.

I don't need to validate my machine; I bought it (and the other one) a while ago and they have met my needs and continue to do so. I have no plans to replace them regardless of the results of any of this testing. Since we are ascribing motives, you appear to want some sort of amorphous testing that is irreproducible and will allow anyone to say anything about their equipment. To me, this is a total waste of time. Earlier in my life I did some scientific research and one thing you learn very quickly is to only test things you have some possibility of successfully testing.

What I have addressed from the beginning of this thread and from earlier ones on similar topics is ONE ISSUE; STRAIGHT SHOT TEMPERATURE STABILITY.

Newer dual boiler machines are being promoted with claims for shot temperature shot consistency based upon the WBC protocol, but selecting out shot sequences that don't come close to what a home user would experience (e.g. "shots 5-14" on the WBC). I'm interested in what is the temperature that you get when you walk up to your machine after an indeterminate idle interval and pull a random shot; what shot temperature do you actually get vs. what you expected to get? Throwing in frothing and other stuff not having been tested on the dual boilers (to date) just complicates matters and doesn't answer the first question. If we answer the first question, then we can go on to answering others. I would not assume, as I have said before, that these dual boilers will do all that well on frothing tests because the 110v versions are going to have the same amperage draw problems that any 110v machine will have, and if they try to brew at an accurate temperature AND froth at the same time, they will either fail to do so or they will flip the circuit breaker.

In answer to the question about varying idle periods to shorter than 10 minutes; on straight shot testing, I have a pretty good idea what my machine will do in its current setup of a low pstat setting with a small flush; the 45ml flush does not perturb the machine much and after more than 2 minutes idle period I'd just use that flush and I think the curves will be similar to what I get either in shot runs or after 10 minutes idle period. The reason I believe this is that the last 15 seconds or so of shots in these shot series I've done and posted show essentially no temperature variation on the Scace device and Fluke. This suggests to me that 45mls pulled after several minutes of idling will not have much effect. This can be tested and I have no problem with testing it.

As to using the protocol that Jim suggests, I think that is fine as long as you are going to use the information to help you work on your own machine. If instead you planned to post it and claim it was accurate enough for comparisons among machines as tested by various individuals with their own differing cheap digital thermometers, I'd say any such comparison is worthless. Mind you, I'm not suggesting that home user types rush out and drop $500 on a Scace and Fluke; this would only be for people who already have bought this equipment or who don't mind making this unjustifiable expense.

I remain interested in comparing various machines (especially older ones that have been modded) to the newer crop of machines to test this ONE SPECIFIC THING; straight shot temperature stability on random shots made after random idle intervals. This is going to be very representative of what people get in low volume usage who are looking for reproducible shot temps on straight shots. This should be of interest to others because mods (such as PIDs) are comparatively cheap to do, much cheaper than going out and buying a new $4500 machine. If someone can do almost as well with a $150 mod on their current machine as they can on a new GS3, then people OUGHT to find this interesting. Of course, I am not the arbiter of what others find to be interesting:-)

I for one do not think that tight shot temperature control matters in milk drinks; to the extent that it is worth testing, I think it should be tested later, after establishing whether any machine can really deliver a shot at a desired temperature when the shot is made from idle conditions. If we don't know whether equipment can make the stable shots without introducing other factors (like frothing) then we don't have enough of a knowledge baseline to tackle other issues like the impact of frothing, for example. Until this can be established we are talking about putting the cart before the horse.

ken

miKe mcKoffee wrote:I'm game. Maybe should start a new thread.

I've started Used Case for Home Espresso Machine Testing Protocol.
Start thinking of some Use Cases that you think are likely to be high probability for this audience.

Ken Fox wrote:So Chris, are you accepting my challenge to pull maybe 10 random first shots from idle on your GS3, at each of several requested temperatures, and plot out the results for each temp?

Ken - honestly, when this looked like it was about trying to come up with a model for quantifiable evaluation that is more applicable to home use than the WBC protocol is - it seemed interesting and valid to me. If this is some sort of "challenge" on the other hand... with all due respect, in general I try to stay out of this sort of pissing match (both in the coffee world and in life in general).

malachi wrote:Ken - honestly, when this looked like it was about trying to come up with a model for quantifiable evaluation that is more applicable to home use than the WBC protocol is - it seemed interesting and valid to me. If this is some sort of "challenge" on the other hand... with all due respect, in general I try to stay out of this sort of pissing match (both in the coffee world and in life in general).

Chris,

The last time I checked, the definitions of "challenge" and "pissing match" were different in my dictionary

I appreciate the work you have done in your review on the GS3 and none of this detracts from that.

I am going to do some random interval shots from idle at various temps, and post the graphs on HB and maybe a.c. as well. This is something I would do anyway in my attempt to learn how to better use my machine now that I have a Scace device and fluke. Since I am going to do this work anyway, I might as well post it since it might help other owners and might encourage owners of other machines to study this issue, for the benefit of all of us.

I am suggesting that you consider doing some of this testing also. I would be surprised if you weren't curious to find out how this machine or any machine you use regularly would function on such a test. It seems a shame to me to have the ability to do this sort of testing but not use the device for it, assuming you have access to a Scace device, that is.

ken

I would not do this test because in my opinion the measuring precision is not good enough for comparing machines. Combined with a good palate, which you are blessed with, this instrument and technique may be helpful to you, but I would not use it as a way of comparing one machine to another. For one thing, the cheapie TCs and meters have large tolerances. While it may be true that a given cheap TC and cheap digital thermometer may have a relatively consistent error, there is no assurance that using a whole bunch of them by a whole bunch of people will produce usable data. I personally doubt it. Since this is really all about taste, I see no "harm," but I would not compare data so obtained.

Ken, if you want a home machine protocol; you will need to measure exit temperatures. Home machines include 49 mm pf lever machines, 52 mm steamtoys, no mm superautos, 53mm and 57mm decentish machines like Saecos, Solises and Quahas, along with the machines we usually talk about. If you want a model of what a good home machine test looks like, check out the Norwegian Specialty Coffee Association's

http://www.kaffe.no/

Go to the orange right hand panel, and click at the bottom where it says, "Vi har testet 47 espressomaskiner for hjemmebruk. Resultatene finner du her. Then click on "resultater" for the tabular breakdowns. The testing protocol is fairly easy to make out despite being in Norwegian.

Malachi, about use cases: I've owned home espresso machines for 30 years, and most people I know own them too. The primary use case is gathering dust somewhere, the second one is making coffee with folger's preground. There's a reason for pressurized pfs and superautos; most home users don't care much about espresso. A test that will make good machines and good technique attractive to them is one that is honest about how much time and skill it takes to make decent espresso with the machine. This is my only caveat with the Norwegian test; by leaving this out, it sort of misses the point. Exact performance measures, as opposed to a simple pass/fail on stability and workability are of little interest to most people.

Finally, just to get everyone's focus set right: a home machine test is one that a vendor can easily do and post the figures. It will need to be fast, easy to understand, and somewhat accurate. Graphing out shot profiles to the nearest 0.1 degree is not in the cards. Think of a test that will clearly show a Krups or Superauto doesn't make espresso since it fluctuates too much; a Silvia can, but takes about 5 minutes of surfing for the first shot, and about 30 to complete the protocol, and that a home HX can run through the protocol I proposed in about 10 to 12 minutes.

If, on the other hand, you all want to find out which of the home semicommercials is slightly better than all the others; I suggest shipping them all off to the race track and awarding 5 brownie points to whichever one wins.

another_jim wrote:Ken, if you want a home machine protocol; you will need to measure exit temperatures. Home machines include 49 mm pf lever machines, 52 mm steamtoys, no mm superautos, 53mm and 57mm decentish machines like Saecos, Solises and Quahas, along with the machines we usually talk about. If you want a model of what a good home machine test looks like, check out the Norwegian Specialty Coffee Association's...

The mistake I made at the beginning of this thread was to speak in terms of measuring temperature stability in a HOME machine or HOME setting. I'm less convinced that this even matters than Chris or Greg or some others although I think it does need to be tested and in order to test it one has to confirm that a machine can produce this sort of temperature controlled shot.

The error in terminology was the word "HOME," because it is pretty damn obvious right now that we are talking about a very small subset of home users who have the equipment, the inclination, or both needed to test these theories and hypotheses.

Most people who care enough to try and who have at least adequate equipment will be able to make espressos and espresso drinks better than most any cafe in their towns; this is in fact a rather low bar to clear.

What I'm looking at is the theoretical possibility of any machine out there being able to produce more or less the same shot profile and temperature when attempted after varying idle periods. This is the hardest test of any machine, used in a home or in a commercial setting, and one I'd expect few if any machines can pass.

Is it a comprehensive test for espresso machines used in the home? Absolutely not. The thing is, that as you state, one already knows suitability of these various home machines from reports of other users and published reviews in sites like this. In normal usage the quality differences among the machines can easily be overcome by good barista technique and using good coffee. So, it really doesn't matter, except when looking at convenience, to the great majority of people who own equipment from the Silvia level on up.

The one thing that we have not been able to test before that we can test now is brew water temperature, reproducibly, in tenths of a degree. We have Greg Scace to thank for that. But as has been discussed repeatedly the Scace device doesn't really give you the actual brew temperature, just what is coming out of the head.

ken