How accurate are our assessments?
One question keeps coming back to me. When we judge or rate wine as professionals, how accurate are our assessments?
When it comes to sensory work by scientists, ins studies they are looking to get statistical data on the wines being tasted, they are much more rigorous in their approach than most wine professionals looking to criticise or assess wine. For example, a study might have compared vineyard treatments, and then to assess wine quality they could do microvinifications of the different plots in the experiment. They are looking for statistically significant differences between young wines that might be quite close in character.
For a start, they limit the number of samples tasted in any one session. And they give their sensory panelists gaps between each wine, where they rinse their mouths and allow their palates to reset. What is known as carry-over is a real thing: the previous wine can influence the perception of the next wine. For this reason tasting order is randomised – this is something that many competitions try to account for, for example by having panellists each taste in a different order.
It’s because of the rather imprecise nature of sensory perception that we need such approaches. We are not measuring devices giving a read-out based on what is there. Instead, we model the world around us, and our perceptions are multisensory. A lot of processing goes on in the background before we are consciously aware of the world around us.
When I taste a wine, I like to think that the vast experience gained from tasting thousands of wines a year in all sorts of different settings, and including wines from all the wine growing regions of the world (most of which I’ve visited), means that my perceptions have a solid base to them, and I can make sense of the wine in front of me. Many professionals would say the same. But, at the same time, I’m aware that some of my assessments are more accurate than others. Sometimes, I feel I’m really getting the wine; other times, it feels like I’m straining a bit to try to get closer to understanding what I’m tasting. One day I taste with clarity; another day it seems much harder.
It would be really helpful to test my palate in order to confirm this hunch I have, but this is hard to do. I think that for most professionals, we’d be uneasy about having our palate tested because we have quite a bit to lose if it turns out it’s not as reliable as we’d like. But there is more to functioning well and reliably as a taster than olfactory and gustatory acuity, otherwise we’d just test lots of young people and hire the ones with the most discriminatory palates. There’s the actual nature and acuity of the perception, and then understanding and making sense of it. And knowing how we tend to perform in different settings is a welcome (and perhaps humbling) perspective.
I do think there’s a case for testing everyone’s smell and taste performance and then telling people who are in the worst performing group – far below the average – that this isn’t the job for you. It may happen that with age, some people lose their sense of smell enough that they should retire from wine judging and assessment. But for most, I suspect we’d be in a broad middle ground where we can taste well enough that, with the right experience and understanding, we can function highly in this profession. There’s no evidence that those with super-sharp olfactory acuity would be at a great advantage, because they might end up having problems with lots of wines that to the vast majority are absolutely fine.
There’s another issue, though: the context of tasting. If I write a note on a wine and give a score, perhaps I should also note the occasion of tasting and the confidence I put in this assessment.
If I’m at a busy trade tasting, and it’s quite crowded, and I’ve been given a small pour by someone who looked at me in my t-shirt and thought that they’d save the wine for more important people, then my assessment will be less reliable than when I’m sitting down in a more controlled environment with more than half a mouthful of wine to assess. Or I might be in a masterclass where the wines were poured an hour earlier (by the time we got round to tasting them) in glasses which have a bit of polish taint. I trust this verdict less than a setting where the wine is freshly poured into a clean glass.
If I open a bottle and drink it with friends over an hour or two, then I think my assessment is much better than if I’ve just tasted a small pour. If the wine has been Coravined or dispensed from a preserving device like an Enomatic machine, then it’s still possible to assess it, but ideally the wine should be assessed without being forced through a thin needle under pressure, with argon or nitrogen affecting the dissolved gases in the wine. Similarly, my assessment of a wine from a bottle opened one or two days previously is less certain than from a fresh pour.
If I’m tasting a large line up of 100 wines under a bit of time pressure, then it’s harder to be certain of an assessment than if the line up is 30 and there’s plenty of time. If I’m tasting in the winery, then the context can help me understand the wine better, but being in a nice setting with nice people can make you score wines a little higher than they merit, which is something to be aware of.
I think that care has to be taken to help the taster understand the wine as clearly as possible: serving it at the correct temperature, with a calm environment, with a clean glass that’s of a suitable size and shape. Sometimes it is good to start blind (without knowing the wine’s identity), but information is useful and can help make sense of the sensory properties of the wine.
As a wine journalist I taste wines in all sorts of contexts and settings, and I appreciate this, and would be able to write about far fewer wines if I only tasted them in laboratory settings. But I remain interested in the question of how accurate my assessments are, and would love to have my palate tested in various ways, in different tasting environments. What we do when we taste analytically is quite complicated: we are effectively untangling information that our brains have worked hard to bind together into a seamless conscious perception, and this takes practice, experience and tasting ability to do reliably. We should remain humble in the face of wine.