Those random wine medals

By Felix Salmon
September 1, 2009
abstract, full paper) that there's really no such thing as "wine quality", if you're tasting blind:

" data-share-img="" data-share="twitter,facebook,linkedin,reddit,google" data-share-count="true">

Bob Hodgson provides yet more evidence (abstract, full paper) that there’s really no such thing as “wine quality”, if you’re tasting blind:

An analysis of over 4000 wines entered in 13 U.S. wine competitions shows little concordance among the venues in awarding Gold medals… An analysis of the number of Gold medals received in multiple competitions indicates that the probability of winning a Gold medal at one competition is stochastically independent of the probability of receiving a Gold at another competition…

For the 375 wines entered in five competitions, one would expect by chance alone (for p = 0.09), 234 wines receiving no Golds, 116 receiving a Gold in just one competition, 23 receiving Golds in two competitions, two receiving Golds in three competitions and no wine receiving Golds in more than three competitions. The observed frequencies closely mirror these numbers.

The more I look at empirical studies such as this one, the more I’m convinced that if you’re tasting blind, there’s no correlation between perceived quality and just about anything. On the other hand, we almost never taste blind in real life — and when you know what you’re drinking, there’s are very strong correlations between perceived quality and lots of things, such as provenance, price, and even whether the wine has a screw cap or a cork.

Which is why it’s perfectly rational to order expensive wine in a restaurant: since you know what you’re drinking, and how much it costs, there’s a very good chance that you’ll enjoy the more expensive wine more and the less expensive wine less. If you tasted them blind, on the other hand, there would be no correlation there at all.

18 comments

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/

But would you enjoy it if you knew that price and quality do not correlate? It’s a placebo efffect no?

Posted by Martin Jol | Report as abusive

It may be in principle rational to order expensive wine in a restaurant knowing that you’ll enjoy it more; but that is not to say that the there is a linear cost-benefit relationship.

Posted by Joao Costa | Report as abusive

Taleb has an equation for this….probly

Posted by otto | Report as abusive

It may come down to status signaling, where the purchase of the more expensive item is simply to show that one can afford to purchase it, not because of any inherent superiority of the item. I recall something similar being discussed during the Eliot Spitzer scandal.

Posted by Ken | Report as abusive

Of course, knowing that the price and the quality don’t correlate, you might have higher expectations of the cheap wine in a restaurant than you would otherwise have, causing it to taste better.

Additionally, since you perceive that you are getting the same quality for a lower price, the cheap wine will also be perceived as a very good deal – and I believe that this will ALSO cause you to more favorably assess the quality of the wine when you drink it*.

Maybe this expectations engineering has something to it…

*I don’t remember where I read this – it may have been in Dan Ariely’s book, though.

Posted by Dan | Report as abusive

That’s much too strong a conclusion to draw from that study. It seems more reasonable to conclude that within (perhaps broad) bands of quality, either a) people have a very hard time consistently making fine distinctions within a quality band or b) variation across judges in idiosyncratic preferences swamps what the “average” quality ranking would be. Not all wines get sent to wine competitions; we would tend to think that producers send their best plonk, no?

That is not to deny the importance of status consumption as being a very important driver of perceived quality, and that one way to signal (to yourself and others) that you are consuming status is to pay a lot for the wine. But if you paid $150 for Two-Buck-Chuck, I think you would know.

Posted by JBS | Report as abusive

As you likely know, Brian Wansink has been exploring this kind of stuff for years. When I try to explain to people that their expectations and the cues they receive from ambiance actually determine how they believe their food and wine tastes, they don’t believe it and continue to insist against scientific evidence that the place has gone downhill or this one is the best ever, etc. They don’t understand that experience has been shown over and over to truly be relative.

As a musical type, I relate to this through pitch. I have relative pitch. When you hear a singer say, “Give me an A,” that singer is setting his or her pitch for that song. And the more you know, the more you realize that pitch is never really absolute, but that some people have really great memories for what has been defined to be these notes. An A in opera in Paris used to be about 415 cycles – which made it much easier to sing there and to hit high notes and to hit them with volume. As halls got bigger, A went up because higher cycles means brighter tone that penetrates. (And kills the voice.) Standard symphony tunings have been 442 or 444 – with stories of higher – for quite a long time. That extra few cycles is really noticeably bright. When the orchestra tunes, they’re making sure everyone has tuned to the same A, not making sure that someone isn’t in tune – these are professionals, not tone deaf amateurs, but they need to be tuned to whatever has been decided to be A that night.

And then if you get deeper you realize that the scales we play are made up, that every note in the commonly used scale is tempered, meaning that it’s off by a few cycles, because if you tune to make say D perfect then some notes get really far off. Old days used specific tunings for specific kinds of music. And a violinist with a really good ear can get ticked off at other instruments because they can’t tune to a perfect e.

The same thing happens with wine. Have a nice conversation? Better food. Have a bad service problem? The food may be bland. Same wine, different prices, different scores. Same food, different presentation, different scores. Charge more for the same product – as luxury makers realized in ancien France – and people think your product is better. Something not selling on your menu? Mark it up, not down.

Posted by jonathan | Report as abusive

If it works for colleges, why not wines?

From today’s Economix blog:

“Traditional economics would suggest that raising the price of an item (such as a college education) would reduce demand for it. But instead this study found that raising tuition — as well as instructional expenditures — actually improves the demand to attend liberal arts schools and schools in the bottom half of the top 50. For example, for liberal arts colleges ranked 26th to 50th, a $1,000 increase in tuition and fees was associated with a 12.9-point increase in SAT scores and a 3.5 percent increase in the proportion of top freshmen admitted.

This is because such costs “serve as markers of institutional quality and prestige,” the authors write.”

Posted by Mark Beauchamp | Report as abusive

I am not a wine snob, but know several. You either like it or you do not. Cost is not a factor. If I find a wine on a restaurant menu that I know and like, I just decide if I want to pay for it. Usually, I look for something I am not familiar with that is less costly. I am usually happy with my pick.

Posted by Norma Peratt | Report as abusive

So restaurants and bars should serve randomly from their inventory, charge a price that covers average cost, and offer “red wine” “white wine” “scotch” “bourbon” “gin” etc (maybe get fancy with the wine, go for “bordeaux” “merlot” “rhone” “plonk” or whatever).

I clearly remember from my days working in quasi fancy restaurants instances of snooty customers sending back a scotch, saying “I ordered Pinch, this is not Pinch” (do they still make Pinch?) so the bartender would send them a replacement made with bar scotch, which was deemed accurate. Then one of us would get to drink the real Pinch!

Posted by bdbd | Report as abusive

I’ve done some work on this, and I believe the paper only shows that these state fair judges are consistent. Professional critics like Parker and Wine Spectator are remarkably consistent in their tastings.

Posted by dk | Report as abusive

I had a job years ago selling stuffed animals door to door (yes, it sucked). I could double my sales by saying that customers were limited to 2 of any item that they seemed to take a fancy to. Instead of buying 1, by introducing the threat of scarcity, I could trigger some kind of hoarding mechanism and get them to buy 2. Fun, but kind of depressing as well even after all these years.

Posted by GaryD | Report as abusive

>>Of course, knowing that the price and the quality don’t correlate, you might have higher expectations of the cheap wine in a restaurant than you would otherwise have, causing it to taste better.<<

Actually, one thing about the bottom of the wine list in many (not all) restaurants is that here is where the sommelier has gone out of his or her way to seek out a wine with good qualities that does not cost the restaurant a fortune. Perhaps this selection should best be made via blind tasting. Of course there are low-cost restaurant wines that are simply awful, but in this world awash with excess grapes, inexpensive yet interesting wines should be (relatively) easy to find.

Posted by JD | Report as abusive

when one drinks wine with meals and friends for the purposes of pleasure as a matter of course…the ‘selection decision’ becomes of marginal import. Salut!

Posted by crocodilechuck | Report as abusive

Well, “cheap guy” says … “not me.”

It seems like more reason to throw a good dinner party.

(Consider that part of the Julia Child resurgence, in this time of restaurant decline, if you will.)

Wine is a living thing – not a sterelized x-rayed can of drink.
It’s taste can be temporarily or definitely altered by transportation, conditions of storage etc…
In wine countries, it is’nt much of a problem for everyday-wine, a plain product. Top products used to be rather sundays wines only, years ago, before the US$ speculation.
Some wines have some kind of magic. It makes you so happy – just the way very good cooking does.
Sometimes the very good cooking is that of famous & expensive chefs, sometimes it is that of friends, or a very gifted wife.
But this magic – both of food and wine – has become a thing of misleading fame, corrupt trading, gross speculation – please never read anything signed P*rker about wines, it is as truthfull as a C*A report.
How do you get good magic for little money?
Ha ha! I think you would need some magic of yourself.
It does’nt mean no good & afordable wines are made any longer.
But maybe they can’t make their way in your country.
Ha ha!

Posted by renvoie-la-soupe | Report as abusive

Unfortunately the paper does not separate out the different varietals. I am convinced that all chardonnays taste the same, so I am not surprised by the random result in that case. Cabernets are a different matter.

Posted by kUAS | Report as abusive

Also, these statistical samplings of blind tastings are nearly always conducted with people who know nothing about wine. Duh. If you put a panel of eight year olds in front of a screen and ask them rate the difference between Scooby Dooby Doo and Lawrence of Arabia will they tell you something you didn’t already know?

Posted by Lawrence Osborne | Report as abusive