The humbling of Robert Parker
Spending a couple of high-intensity days in Washington, as I’ve just done, is enough to send anybody dreaming of booze. And so it’s quite lovely to read Dr Vino’s missive from the latest Executive Wine Seminar, which featured not only 15 spectacular 2005 Bordeaux wines, but also Robert Parker, tasting them blind.
Parker has rated all these wines, of course. But would his blind ranking bear much if any relation to his official ranking? And how many of the wines could he successfully identify? The answers, I wasn’t at all surprised to hear, were “no”, and “zero”.
To take just one example, Parker identified wine #8, the mainly-Merlot L’Eglise Clinet from Pomerol, as being the mainly-Cabernet Cos d’Estournel from Saint-Estèphe.
Writes Dr Vino, charitably:
A final issue is about points and the nature of blind tasting, a capricious undertaking if there ever were one. Although Parker did not rate the wines yesterday, his top wine of the evening (Le Gay) was the lowest rated in the lineup from his most recent published reviews… For all the precision that a point score implies, it is not dynamic, changing with the wines as they change in the bottle nor does it capture performance from one tasting to the next.
Blind tasting is not easy, but that does not mean it’s not useful. With all due respect to Bob Millman, it’s not that tasting blind is “judging from ignorance”; it’s that ignorant judges do poorly in blind tastings.
Millman is the person who runs the Executive Wine Seminar tastings, and he’s much less constructive on the subject of blind tastings than Matthews, and it’s not hard to see why: after all, it does rather seem as though Matthews is saying that Parker is an ignorant judge.
Parker isn’t an ignorant judge, of course. And I daresay he actually did well in this tasting, in terms of judging the wines purely as he was drinking them. He just did badly in terms of identifying what they were, or giving them this time around the same ranking that they got the last time he ranked them. Wine is not a fungible commodity, where one bottle is always the same as the next — quite the opposite. But the fact that wine changes, from bottle to bottle and from month to month, rather defeats the purpose of magazines such as Wine Spectator.
I asked Matthews what he considered a “good judge” to be, and whether there were any downsides to tasting blind. He wrote back:
On judges: Yes, in my opinion, judgment is a quality that lies along a spectrum. “Ignorant” judges lack the context to make useful distinctions; “good” judges have enough experience and understanding to apply appropriate criteria to the wine in the glass. All judges begin in ignorance. If they work hard — taste widely, concentrate intently, read and travel and interview the experts — they may become good. At Wine Spectator, it’s less a matter of “choosing” judges than training them. Before an editor qualifies as a wine critic, he or she undergoes a long and intensive period of apprenticeship with us. The “empirical data” we look for is their consistency of judgment, breadth of understanding and sensitivity to the qualities of the wines they are tasting (faults, structure, flavor descriptors, etc.) Once they do qualify, they are given responsibilty for specific wine types, and as they prove their consistency and expertise, they take on larger tasting beats.
On “downsides”: The challenge in evaluating wine is eliminating externalities that can bias judgment (such as price and reputation) without eliminating so much context that judgment is impossible (since good wine is supposed to reflect its vintage, terroir, etc). Carefully-calibrated single-blind tasting is the methodology that long experience has convinced us is the best and fairest approach. But enjoying wine is a different matter, and in that situation, blind tasting can rob us of information that can supply both pleasure and edification. At dinner parties, I like to serve a wine blind, to get an “unbiased” reaction, then unveil it, so we can learn both from our reactions, and from the wine.
In a world where Robert Parker fails to perform on the criterion of “consistency of judgment”, even in the context of a “carefully-calibrated single-blind tasting”, this does ring a little hollow. Parker didn’t invent the guess-this-wine game that many tastings become, but he is partly responsible for the idea that people who are good at that game are necessarily the people best qualified to judge wine. It’s definitely fun to see him hoist on his own petard once in a while.