Socratic Coffee: IMS baskets vs VST baskets

[dominico]

I don't know if I'd call "10 espressos each" a statistically significant sample size, but the results are very interesting nonetheless:

[aecletec]

Big variability kinda makes the result underwhelming...

[samuellaw178]

^...which goes to say VST is more finicky?

What is the generic basket though?

[dominico (original poster)]

boh, HQ maybe?

[Trimethylpurine]

I don't know if I'd call "10 espressos each" a statistically significant sample size, but the results are very interesting nonetheless:

A statistically significant result.... heard that before

A statistically significant difference..... heard that before.

A statistically significant sample size? New one for me.

One could make an argument that if there is a significant difference then the sample size is big enough that you can make meaningful comparisons between/amongst the means, and if it isnt, you cant.

T

[dominico (original poster)]

What I was getting at is that 10 shots each is not a big enough sample size to get a statistically significant result.

[Trimethylpurine]

1, 1.1, 1, 1.2, 1, 1.1

5, 5,1, 5, 5.1, 5.2, 5.1

Bet those are significantly different in Student's T-test. And only 6 samples each.

Three things affect significance.

1) Effect size, the difference between the means.
2) Variance, the difference between each sample and the mean.
3) Replicates, how many of each treatment you have.

If effect size is big then sample size can be smaller or variance bigger and you still get a significant difference.
If variance is small then fewer samples and/or a smaller effect size is needed to get a significant difference.
If replication is high then variance can be higher and/or effect size smaller and you will still get a significant difference.

h

[samuellaw178]

I suspect most knew what Dominick was talking about, at least I did - though it may not be the correct jargon. For us who have rusty statistics, can you tell/calculate how many sample is needed for Socratic's results to be statistically significant, say with a 95% confidence?

[dominico (original poster)]

It would help to compare apples to apples. Descending into pedantry and providing an example dissimilar to the findings being discussed does not move the discussion forward.

There are too many variables that go into each "sample": replication would be real low.

I don't see how you can claim any sort of significance with samples of this type in a study where they went through not even a pound of coffee.

[aecletec]

I suspect most knew what Dominick was talking about, at least I did - though it may not be the correct jargon. For us who have rusty statistic, can you tell/calculate how many sample size is needed for Socratic's results to be statistically significant, say with a 95% confidence?

I'm not sure exactly what you're asking, whether it's about p-values, confidence intervals, or something else but this isn't an issue of n, usually.
While my stats is a bit rusty I think to get what what you're wanting - that is a powerful result - one would need quite a bit of knowledge about the data or make many assumptions to get to the point of knowing how many samples you need - it's easier to say... well, it's not many shots and there's high variability so let's not treat this as Truth... specifics of word meanings etc for the layman is a bit distracting because it's clear words are being misused at a technical level when we can't even see the data or assumptions!