wa2.gif (4241 bytes)

abut9.gif (3095 bytes)

abut12.gif (3207 bytes)
abut10.gif (3636 bytes)

abut11.gif (4039 bytes)


Scoring wines: how consistent is it?

Iím a scientist by training, so whenever I see a data point, my instinct is to want to know more about it. Most importantly, I want to know how significant it is. How much variability is there in measurement? Is it a solid data point or a soft one?

So, when I see a criticís rating of a wine, Iím keen to know how seriously to take this figure. If the critic scored a particular wine 92/100, what is their margin of error? If they tasted the same wine on 10 separate occasions, what would the spread of scores be like?

Since I started scoring wines out of 100, Iíve asked myself the same question. Iím aware that the senses of taste and smell are relatively imprecise, and are influenced by factors such as context and visual cues. So it is interesting to revisit a wine after a month or two and see whether my scores match up. Iíve not done this rigorously, but several times Iíve scored a wine on a second occasion not recalling how I scored it the first time round, and then gone back and compared notes.

This is important. Itís clear that the utility of any scoring system rests on the variability in the scoring process. If I taste the same wine on separate occasions and there is a large variability in my scores, then these scores are clearly of little use to readers, who will be tasting the same wines in rather different contexts.

The good news is, Iím finding that my scoring is relatively robust. In the business end of my scoring range, 80Ė98 (the latter is the highest Iíve yet scored a wine), the variation is no more than 2 points on separate occasions. Itís not perfect, of course, but it means that the scores (which I never intend to be taken too seriously) are a useful guide.

More importantly, I find the verbal descriptions tend to tally quite well, although they are often phrased quite differently. Iíve written before, partly tongue in cheek, about how wine writers should be tested. Perhaps itís not such a bad idea. At least for critics who spend their time writing wines, it would be nice to know the variance in their scoring systems. After all, if we are, as wine writers, hopelessly inconsistent, then weíd do well to stop wasting our own and our readersí time, and go and find an occupation that better matches our own particular giftings.

Back to top