Pump Comparison Project

[malachi]

I've been thinking about this a lot recently.

I think that we should try to figure out a way to do a structured taste test of the results from various pumps.

Barry had a great idea - use a machine with an external pump and rig it with various pump options. This will eliminate other variables.

I would love to see evaluations of various types of pumps as well as various configurations.

For example:
- stock Procon rotary pump,
- Procon rotary pump isolated from mains,
- various stock vibe pumps,
- various modified and stabilized vibe pumps,
- linear hydraulic pump,
- air pump.

I understand this is a big undertaking - so perhaps we could do this in a collaborative manner. We could try to schedule the final testing for the SCAA annual event in North Carolina. This way we could collaborative remotely on planning, development, etc and then have enough people in one spot to actual do the blind taste testing.

What do you all think?

So far all the tests that have been done seem to either be dated, have questionable methodology, be incomplete or be "commissioned" and/or not public.

I think we'd need:

- 1x modified espresso machine (I'd suggest a PID'ed dual boiler like the Marzocco or Synesso if possible to give us further decreased variables),
- Various pumps,
- water system,
- at least 6x different coffees,
- barista supplies.

I'm thinking this would probably need to occur over the course of three days to get the results we need.

What do you all think?

[lennoncs]

Any resources of mine are available for the asking.


Sean

[AndyS]

I would love to see evaluations of various types of pumps as well as various configurations.

For example:
- stock Procon rotary pump,
- Procon rotary pump isolated from mains,
- various stock vibe pumps,
- various modified and stabilized vibe pumps,
- linear hydraulic pump,
- air pump.

Sounds like a great idea, Chris, and long overdue. If it's scheduled to coincide with the SCAA show in Charlotte, it might have to be immediately before or after the regular event, so that people would be free to spend the time testing.

For quite a while I've been preparing to do a comparison with my equipment (Tricked-Out Silvia) fed by:
(1) Procon in various configurations
(2) pressure-regulated vibe
(3) PID-controlled miniature gear pump (will allow digital profiling pressure control with very low flutter).

The third item has been difficult to put together, but I'm making steady progress (thank you, eBay).

Of course my testing might be limited by the fact that I'm not as skilled as you or many other pros, but it would still be fun and instructive.
I'm up here in Rochester, NY, not exactly a hotbed of serious barista activity, but I hope to rope in several relatively experienced tasters. If you flew out here, Chris, I'd give you a free place to stay!

If the PID-controlled gear pump works out I'd be happy to lend it to your testing in Charlotte. On the other hand, who knows, by then a certain "Mr. L" might have something even better to lend.

[malachi]

OK

I think we might want to try and make this happen at the SCAA show unless someone has a better idea.

[another_jim]

Ken and I will redo the comparison of his vibe and rotary juniors from two years ago early 2006 (before Charlotte). He and I would welcome any suggestions on the protocol.

Last time we poured shots simultaneously on both machines and served them to the other person, who was tasting blind. The machines had previously been set to roughly the same pressure and temperature using PF mounted TCs (schomer style) and pressure gauges. We could detect no systematic differences in taste.

My idea for this time is to use a variety of SOs, specifically Terroir's Daterra to represent a clean pulp natural brazil, a Harar to represent a complex DP, and the Rwanda Karaba to represent a high grown wp. All these make excellent espressos, so the job won't be a horror, and it will be representative of high end blending practices.

This will test whether a pump difference exists for some coffees, but not others.

I we also take along the Peppina to provide a lever reference. These shots will be a the same temperature, but not at the same pressure. This will be a less formal test. The basic idea is whether pump differences group (vibe versus rotary/lever) due to the lack of vibration, or (vibe/rotary versus lever) due to the pressure profile.

[malachi]

I'd love to push ahead with organizing a research project for the SCAA show.
I've got one person interested - anyone else?

[Ken Fox]

I'd love to push ahead with organizing a research project for the SCAA show.
I've got one person interested - anyone else?

I'm not going to Charlotte, so my interest doesn't really matter. As Jim posted previously, we'll redo our prior Juniors (vibe vs. rotary) study in a grander fashion, before the SCAA convention.

The more I think about this whole topic the more I think the likelihood of showing a difference in such a study is vanishingly small, and that if the finer coffee establishment has the resources to do a large "pump study," that those resources could be better devoted to something else.

Think about this rationally: why the hell should a machine make different quality espresso based solely on the type of pump, vibe vs. rotary, contained within it? You could almost convince yourself that it should (I was personally convinced myself) in the case of an unregulated vibe pump vs. a rotary, but once you regulate the vibe pump with an overpressure valve, there is no rational reason to assume that the difference would be detectable. Does it really matter how the nearly constant 9bar of pressure are produced? Does the puck care?

Let's take this discussion to a more practical level; all things being equal, almost anyone who could have a rotary pump rather than a vibe pump in their machine would choose to do so. Afterall, vibe pumps are unpleasant to be around, because they make a racket and what's more, they vibrate. The small incremental cost of the pump itself is not going to deter someone who is prepared to spend enough to buy a competent home espresso setup. I think we will see rotary pumps moving more and more into the home espresso machine market, albeit most probably just above the Silvia level of machine.

Commercial machines are now overwhelmingly run by rotary pumps; I don't see any study as likely to reverse this reality. I don't think that anyone rationally thinks that a vibe vs. rotary study will show conclusively that vibe pumps are better, so that likelihood can be dispensed with and we are not going to see vibe pumps regaining the commercial market. If one is studying home machines, unless you have a nearly perfect comparison such as my two juniors or some type of external pump situation, other factors differentiating the machines may well overwhelm any differences attributable solely to pump type, so the comparison will likely be meaningless. The reason that I say it would be meaningless is that few people are going to decide to buy one machine or another based SOLELY on what type of pump is inside it; they are buying a "package" that contains a certain type of pump.

Speaking only for myself, I think that if there is an effort to marshall enough resources and talent to test something, there are other things to test that are more potentially meaningful; for example, tight temperature control and other temperature manipulations that can be done, intrashot.

ken

[malachi]

If we were just doing a vibe vs rotary pump test, I would agree. But once we roll in various other methods of producing pressure - and especially if we can manage pressure profile and stability... well then I think it gets far more interesting.

I don't think we really have a grasp on the effects in the cup of pressure.

Temp, on the other hand, is becoming increasing well understood.

[another_jim]

The devil is in the details.

For the Tea, setting the vibe was a minor adjustment from 11 to 9 bar, since they are factory adjusted by Italian standards. Micheal Teahan says that the Italians think vibe pumps are actually better for crema and should be set to around 10 to 11 bar; Illy wants pod machines adjusted there. I was shocked that Ken's was completely unregulated; that's probably more a sign of Italian contempt for American espresso (see Barry's infamous NS HX) than of their practice in Italy. Given the overwhelming amount of dp brazil (and robusta) in most Italian blends, I think their assessment of vibe pumps being better in some circumstances is sincere.

I see two major issues we can test:

1. Pressure settings in the US in the best cafes have been going down. Is this in response to using more wp coffees in blends? I'd like to redo the test with the Rwanda Karaba, a high grown wp that makes a lovely shot, to see if pump differences show up better. Certainly roasting and temperature differences show up more strongly in wps than dps (thanks Barry for pointing this out to me). One thing speaks against there being a big difference: We both noticed when using Ken's undamped PF gauge that the lower we set the max pressure on the vibe's OPV, the steadier the vibe output became. This was a dramatic change: At around 9 bar, there was no needle flicker at all; whereas at 12 bar, the needle was vibrating so furiously it was basically a shadow centred on the 12 mark, not a needle pointing at it. So the OPV is a major pressure capacitor as well as limiter on these machines.

2. There is a purported difference in the attack of a rotary pump, it being faster off the mark than a vibe. Again the conventional wisdom is that this may be a bad thing; the complex pistons in the E61 are precisely designed to make the ramp gradual. But there are those (including me) who suspect that this may muddy the flavors a bit. Of course, I have no direct evidence. here the old thirds test with the two juniors may work. An early shot difference will be more readily discernable when just tasting the early part of the shot.

The really big issue, pump profiling, we cannot test. I'll take my Peppina along and 12 gram commercial basket along. We'll be able to set identical temperatures (and If I get the Peppi pressure rigged, also the initial pressure) using the same weight of coffee and shot volume for blind tasting. However, I'm very pessimistic about this test, since there's so many other differences.

I think once Andy has his closed loop pressure controlled pumps running, he'll be in the best position to test hypotheses along these lines. He can invite over the Gimme folks, or roughly follow the procedure I used when doing the verslab taste testing -- assume shot to shot variability and just rate paired shots over a long series. (the bigger the difference the smaller the series needs to be to establish significance). It may take months that way, but the results will be very reliable, overarching and averaging a wide range of operating conditions, moods, and blends.

[barry]

2. There is a purported difference in the attack of a rotary pump, it being faster off the mark than a vibe. Again the conventional wisdom is that this may be a bad thing; the complex pistons in the E61 are precisely designed to make the ramp gradual. But there are those (including me) who suspect that this may muddy the flavors a bit. Of course, I have no direct evidence. here the old thirds test with the two juniors may work. An early shot difference will be more readily discernable when just tasting the early part of the shot.

and don't forget potential differences due to pressure waves. maybe.