A detailed comparison study of the Atago and VST refractometers - Page 11

#101: Post by **AssafL** » May 24th, 2015, 2:38 am

TomC wrote:http://socraticcoffee.com/2015/05/measu ... n-part-ii/

Aren't all three perfectly calibrated (as expected throughout this thread)? To within 0.03 (as per the VST II spect and 0.01 as per the VST III spec?

I still think the culprit that all three are not within the defines precision of 0.01 for all three is temperature. Atago waits longer for the temp to stabilize while Misco waits less.

As I said earlier one cannot compare the specs if one doesn't know how the spec is derived.

If Atago, for example, is +~0.15 across the entire temp range (even at T=100C) which seems to be the case from my and Socratic's tests, as well as the Atago spec sheet (which allows up to 100C) it can definitely be that a different protocol (eg VSTs protocol) would show that calibration was +-0.01.

Speculative, but somewhat suggested by the results. Actually it is easy to validate the Atago since it has an additional Brix scale which can be used with a reference solution.

Or one can use the VST reference solutions.

Overall - an expected result from Socratic.

If I rationalize for espresso, which demands even less accuracy and precision, all three are perfect.

#102: Post by **jpender** » June 1st, 2015, 2:53 pm

AssafL wrote:Aren't all three perfectly calibrated (as expected throughout this thread)? To within 0.03 (as per the VST II spect and 0.01 as per the VST III spec?

There's no way to tell from their tests since they had no way to ascertain the accuracy. But the numbers aren't inconsistent with that possibility.

AssafL wrote:I still think the culprit that all three are not within the defines precision of 0.01 for all three is temperature.

The precision for all three isn't 0.01%. It is for the VST III, but for the VST II it is 0.03%. And for the Atago it isn't stated.

AssafL wrote:Precision is least significant digit (0.01%). It always has been for any piece of equipment.

The least significant digit represents the resolution, not the precision. VST could modify their refractometer to display to 0.001% but that wouldn't change the precision.

I don't think the Socratic Coffee experimenters can make statements about precision of the device models. All they can do is talk about the specific units they possess. They have only one of each. To make general comments about Atago vs VST II and III they would need to test a number of each of the refractometers. That said, the Atago they have appears to be on par with their VST III.

Sorry for bumping this thread. :-)

June 1st, 2015, 4:08 pm

The authors of the Socratic Coffee post state several times that the goal of the test wasn't to establish the individual precision of each device. It's also very difficult to draw any real conclusions from the rated specifications of the various VST models and the Atago PAL. VST states precision and accuracy specifications for their refractometers, Misco (manufacturer of the VST) states precision, and Atago states "measurement accuracy". The Atago accuracy statement is also for a much wider temperature range than VST.

FWIW, in the brewed coffee range VST claims an accuracy of approximately ±0.00005 nD, while the Atago's rated Brix accuracy is something like ±0.00014 nD. That's very close to the "warranted" accuracy of the original VST LAB model, and pretty close to the VST's full scale accuracy of about ±0.0001 nD (incidentally the same as Misco's stated internal precision for the Palm Abbe).

#104: Post by **jpender** » June 1st, 2015, 4:41 pm

Bob_McBob wrote:The authors of the Socratic Coffee post state several times that the goal of the test wasn't to establish the individual precision of each device.

That's true, but they did note that the VST II that they tested exhibited greater "variability". That's just a casual way of talking about the precision.

In addition to mentioning the relative variability they also spent some time detailing the differences in average values. But given the specified accuracies of the devices, the fact that two of the three refractometers were in closer agreement could just be luck.

The stated goal of the study was to "examine the measurements" of the devices. So they couldn't measure precision or accuracy or really say anything with much certainty. And yet I think you do get a useful picture from their results.

#105: Post by **doublehelix** » June 1st, 2015, 6:12 pm

another_jim wrote:Aristotle is still cited in professional papers in physics and ethics; those in physics are far less friendly than those in ethics. The water work is still cited because its relevant, the taste tests and brewing recommendations do not get serious citation: This is for some very simple reasons:
-- Normal brewing methods all get gross extractions from 17 to 23 percent quite automatically, regardless of measuring methods. Higher levels require hydrolization of insoluble fibers; lower levels have been deemed unacceptable for as long as coffee has been roasted.
-- the coffee to water ratio varies from 25 to 1 in 1950s diners to 1:1 in extreme modern ristretto espresso shots. Much of this cited research was driven by an industry wide drive to get diners to go from coffee to water ratios of 25:1 to 15:1.
-- Cupping methods are still used today to vet coffees; they are far more precisely specified than other brewing and tasting methods. They are not quite useful for scientific sensory assessment, but they are a lot closer than the other methods used in these studies.
-- A modern sensory assessment never asks about better or worse. It might ask what is most acceptable to a specific population of consumers. Obviously the results are completely worthless for that today.
-- It may also ask about mappings from physical prep variables to sensory variables. In this case, the sensory variables used in cupping would also be the subject of the tests. But the tests would be based on having trained panelists detect the difference/no difference in aroma, acidity, body, etc, in double blind samples.

The reason there has been no scientifically acceptable coffee taste testing is that there is no consensus on how to vary coffee in the manner required. For instance, how do you change acidity without changing flavor, body, aroma, etc? Joseph Rivera has been trying to get the funding for these kinds of endeavors for a long time without much success. The reason is that the only imaginable protocol, the ones he advocates, requires using standardized additives -- i.e. adding a fixed quantity of malic or tartaric acid to a standard brew and seeing how much would is required before, say, 10% and 50% of the panel can taste it. Similarly, one would use precisely measurable artificial fruit or caramel additives to test flavor discrimination, and soluble fiber or tasteless fats for body discrimination. Comparing these results to the corresponding intensities of the actual coffee components would then tell us what we need to know. However, an experimental approach like this is deeply antipathetic to the specialty coffee ethos, which decries any use of additives or flavor substitutes

I've copied Joseph and done some of this at home and during lectures; but the results are so variable that it is clear that it would take a lab to do this validly for enough people to be useful. The cost would not be chump change either: about 6 months work for 3 or so experimenters and a revolving group of maybe 10 panelists, so about 250K minimum. The NCA and its members could fund this; Starbucks might also bite; but for the specialty sector, it's pretty much out of the question.

However, pretending this research already exists, and ascribing it to the 1950s through 1970s brewing studies, is fantastic.

One small consideration here: If dispersed components like fats, oils, etc. are in the tested coffee and transmit light they will contribute to the measured RI. Coffee particles that do not transmit light will not contribute to the RI. (For now, we ignore secondary effects to RI concerning surface-mediated phenomena.) These particles may scatter and absorb light. This means that RI measurements of unfiltered coffee will likely have contributions from dispersed oil, fats, etc. How much oils, fats, etc. remain after filtering depend on the type of filter used. How this all figures for evaluating coffee is out of my wheelhouse.

June 1st, 2015, 6:22 pm

Thanks for the clarification. My problem is that I stick to the "ideal model" level of science, frictionless planes, point particles, etc. That makes for clear, no bs thinking, but also ignoring the gotchas.

June 1st, 2015, 6:40 pm

Very much agreed!!!! There is a lot of discussion/debate on HB and CG on how solid particles, oils etc. affect RI--just wanted to get this out there. In science, there is much virtue, at times, in just considering spherical chickens-- who wants to deal with feathers!!!!

#108: Post by **AssafL** » June 5th, 2015, 12:01 pm

jpender wrote:The least significant digit represents the resolution, not the precision. VST could modify their refractometer to display to 0.001% but that wouldn't change the precision.

While that is true I made the earlier example of an HP8.5 digit DMM where precision isn't spec'ed. One can find the precision of the voltage standard but the precision of the standard doesn't reflect on the precision of the test.

For VST the precision depends (and thus can be specified) on following the protocol. Since Atago did not specify a protocol, they cannot guarantee precision. For example, it is very likely the same coffee measured at 100c in a cold room (ie temp is dropping like a rock) vs a cooled coffee sample at room temp with a same temp refractometer would measure differently.

Hence precision is never a factor for a test equipment and is considered least significant digit.

In specific cases (eg mV ranges, uv and nV ranges) noise may eclipse the least significant digit readings. In those cases you'll see a +-x Count on the specs (e.g. 0.01% +- 10 counts).

Since Atago doesn't state that, the precision would be least significant digit given sound measurement practices.

Just like any other precision instrument we use (pressure, voltage, temperature, etc).

#109: Post by **jpender** » June 5th, 2015, 1:00 pm

AssafL wrote:For VST the precision depends (and thus can be specified) on following the protocol. Since Atago did not specify a protocol, they cannot guarantee precision. For example, it is very likely the same coffee measured at 100c in a cold room (ie temp is dropping like a rock) vs a cooled coffee sample at room temp with a same temp refractometer would measure differently.

First of all, Atago does list a protocol on their web site. Plus they specify an accuracy which would also depend on the details of the procedure. And accuracy is inclusive of precision.

We can always speculate why they didn't provide a more detailed specification. I find that this often correlates with lower cost and quality. So it makes me a little suspicious of the Atago device. It's nice to see it apparently working so well in the Socratic tests.

AssafL wrote:Hence precision is never a factor for a test equipment and is considered least significant digit.

In the case of these refractometers you have VST themselves telling you that the precision is not the same as the resolution.

AssafL wrote:Just like any other precision instrument we use (pressure, voltage, temperature, etc).

I have sensors that come with datasheets that detail their precisions. They do not equal the resolutions. I have measured the precisions of these myself. I also took some time to check the precision of the little scale I use to weigh dried coffee samples, using a particular weighing protocol of course. I thought that step was crucial to determining whether it was suitable for measuring %TDS in the first place. For my scale the precision is not equal to the resolution.

#110: Post by **AssafL** » June 6th, 2015, 10:17 am

Finally well spoken words. Atago does have a process whose target is QA not accuracy. Hence no cooling, filtering etc. under that process you are granted 0.15 accuracy.

Try VSTs protocol and you'll get different results. Hence Socratic coffee not being able to demonstrate a big difference (or actually any difference) between the devices.

Do you want to prove superiority of VST? I'll give a dandy method. Run Atago procedure with Atago and VST procedure with VST and you'll probably see the 0.15% difference. Do it boiling vs fully stabilized for the drama.

jpender wrote:In the case of these refractometers you have VST themselves telling you that the precision is not the same as the resolution.

Actually VST precision of 0.01% happens to be exactly like their resolution.

Do any if your voltmeters have a precision rating? Which one? None of the 5 digits I have nor the 6-8 digits I own come with one.