How consistent are wine reviewers?
Interesting article in the Telegraph titled Consumers warned over consistency of wine reviews. It's based on a survey undertaken at the California State Fair wine competition at Sacramento, in which the judges were assessed for the reliability of their palates. You can read the original article from the Journal of Wine Economics here.Tasting wine blind in competition settings is difficult, and few do it really well. It's important we know how reliable tasters are in these sorts of settings, because then we know what sort of confidence we can have in the results.
This is the first time I've seen these sorts of data collected, and the results are quite sobering. If we, as the 'wine trade', are to be taken seriously, then these are the sorts of studies we should be encouraging.
How about doing this sort of exercise at the International Wine Challenge, or the Decanter World Wine Awards? Or entering exactly the same wine into the competition under several different names? We should be eager to see how well we are doing, because this would reassure us, or help us improve.
Labels: blind tasting, wine science
posted by Jamie @ 11:59 AM
4 comments
links to this post
4 Comments:
Yeah, very interesting and sobering / worrying at the same time. It is something that I've suspected since seeing just how many chardonnay wines the judges had to get through at the Royal Adelaide Show. And I've often thought that if you want to get a medal, you just have to enter a lot of shows.
It's good to see some hard data and proper statistical analysis, thanks for flagging this up, Jamie.
At the end of the day, I think you can say that it's likely that golds or trophys from the "better" shows, eg. Bruxelles or Decanter or Sydney Blue Gold almost certainly represent great wines and another fairly safe bet is to follow the advice of a critic you respect. But this approach does not stop a huge number of potentially great wines going unreviewed or "just" getting a silver.
I have found that it is very difficult to rank wines on quality if the tasting is totally blind and out of context. Try picking 10 wines at random from Chateau Margaux to Yellowtail and then get a wine expert friend to rank on quality alone, and then look to see if there is any correlation between price and perceived quality.
Below a salutary quotation from the 19th c when they weren't afraid to call a hum a bug.
I firmly believe that tastings are as mere snapshots of opinion. Great wines, as Michael Broadbent averred, have an arc of development, now dumb or introverted, now glowing and eloquent.
Since wines cannot (and, arguably, should not) be consistent why should wine reviewers?
THE PERILS OF BLIND TASTINGS:
"I remember three constant frequenters of the docks...who used to express themselves in their peculiar oracular way, so authoritatively, that I resolved to put their judgment to the test.....I used to tell the cooper to draw two glasses from the same pipe, and to hand them as if they were from different numbers. I may say that the trick upon them was invariably successful, for they were sure after tasting, and retasting, and much profound thought, to pronounce the verdict that, although similar, one possessed rather more of this, or that, than the other. I kept my own counsel, but was convinced...that in wine-tasting and wine-talk there is an enormous amount of humbug."
---T.G. Shaw, Wine, The Vine and the Cellar,
Thank you for sharing this piece of information, Jamie.
Actually, i've always considered that, with the same jury members, and the same wines, just changing the sequence folowing which the jury judges will change their perceptions in a noticeable manner.
Specially true with astringence and acidity.
Michael Moisseeff
Really good points - all three! It's important that the wine trade addresses these issues.
Post a Comment
Links to this post:
Create a Link
<< Home