Cupping has even more standards than HTML code, and even less compliance. Every roaster and lab I've cupped at uses a slightly different roast, protocol, and scoring form -- so much so that it takes someone a few days of cupping to get used to each. The COE, where some of these factors vary country to country, diplomatically calls this "calibration".

Therefore:
-- cup a lot
-- come up with the procedure that works for you.

.... That's how the real pros do it. If you boast how your cupping conforms to "industry standards," they'll laugh at you

Is standardization even desirable? I do not believe it is; here's why:

Most scoring systems consists of 0 to 10 scores in several categories. If there are five categories, people will add 50 to get to 100; if there are eight categories, they'l add 20. etc. A great coffee scores 10 in each category, but there is a difference how an average coffee is scored, with lower scores when there are few categories, and higher scores when there are many. The idea is that a C grade coffee will get about 65 to 75 points; a basic clean specialty grade coffee about 80 to 85, and a fine coffee 85 and more. This means that in a five category plus 50 system, you give the C grade 4 points and the best coffees 7.5 to 10; whereas in a ten category plus zero system, you'd give the C grade 6 to 7, and the best coffees 8.5 to 10.

This set of scoring systems is open to imbalances when there are many judges with backgrounds in different scoring systems using the same one -- the ones who give a small differences between their highest and lowest scoring coffees have less influence than the ones who give large differences.

Quite comically, both the COE and SCAA judge the judges by how close they score to the average consensus. This means that the experienced judges, who enjoy the gig, game the system by scoring all entries within a very narrow range of scores close to the average. Given this ill considered QC standard; they will always rate as great judges even if they happen to score the worst coffee highest and the best lowest, as long as they keep all their scores close to average.

The consequence is that the judges the COE and SCAA considers most qualified are those with the least influence on the outcome. Only the clueless judges, who score in an unrestrained manner, and whose scores show real love and hate, decide the winners and losers. But the quality control system ensures they either change their ways or do not get invited back. I'm not certain what the consequence is; but if each contest's deciding judges are the newbies, it seems there would be a great deal of randomness.

The take away is that the scores of a successful roaster who cups regularly will be more reliable than the high end competition scores, regardless of the protocol; since whatever the protocol, it will be applied consistently, by the same group of people, who know their market.

Is there evidence for this scenario? There is, hundreds of thousands of dollars worth ...

If you look at the jury scores and auction prices at coffee auctions, they are uncorrelated except for the few superstar coffees. This means the buyers treat the jury score versus their own taste in exactly the way called for given this description. If the jury scores were really of higher quality than each roasters assessment, the buyers would show them more deference in their bidding. The complete disregard buyers have for the jury scores is why I think the auctions remain valuable. When you buy COE and other auction coffees, look at who bid and how much, not the jury score.

These simple facts should be enough be inoculation enough against believing in the magic of standardized cupping apps.