|
Scoring
wines: how consistent is it?
So, when I see a critic’s rating of a wine, I’m
keen to know how seriously to take this figure. If the critic scored a
particular wine 92/100, what is their margin of error? If they tasted
the same wine on 10 separate occasions, what would the spread of
scores be like? Since I started scoring wines out of 100, I’ve
asked myself the same question. I’m aware that the senses of taste
and smell are relatively imprecise, and are influenced by factors such
as context and visual cues. So it is interesting to revisit a wine
after a month or two and see whether my scores match up. I’ve not
done this rigorously, but several times I’ve scored a wine on a
second occasion not recalling how I scored it the first time round,
and then gone back and compared notes. This is important. It’s clear that the utility of
any scoring system rests on the variability in the scoring process. If
I taste the same wine on separate occasions and there is a large
variability in my scores, then these scores are clearly of little use
to readers, who will be tasting the same wines in rather different
contexts. The good news is, I’m finding that my scoring is
relatively robust. In the business end of my scoring range, 80–98
(the latter is the highest I’ve yet scored a wine), the variation is
no more than 2 points on separate occasions. It’s not perfect, of
course, but it means that the scores (which I never intend to be taken
too seriously) are a useful guide. More importantly, I find the verbal descriptions
tend to tally quite well, although they are often phrased quite
differently. I’ve written before, partly tongue in cheek, about how
wine writers should be tested. Perhaps it’s not such a bad idea. At
least for critics who spend their time writing wines, it would be nice
to know the variance in their scoring systems. After all, if we are,
as wine writers, hopelessly inconsistent, then we’d do well to stop
wasting our own and our readers’ time, and go and find an occupation
that better matches our own particular giftings. |
I’m a scientist by training, so whenever I see a
data point, my instinct is to want to know more about it. Most
importantly, I want to know how significant it is. How much
variability is there in measurement? Is it a solid data point or a
soft one?