Posted by: oysterculture | February 4, 2009

Wine judges, do we trust them?

pouring, hopefully the best wine (photo from

pouring, hopefully the best wine (photo from

… and if we do, should we?  When I select my wine, the scores posted along with the description of the wine certainly influence my decision.  My assumption is that the educated folks making the decisions are professionals after all, and as for me, I’m imbibing purely for pleasure.  Sure, I know what I like, but I am not confident enough to voice my opinion when pushed for details.  In truth, I wilt like yesterday’s parsley if questioned by someone I perceive as more knowledgeable.  But should I be so hard on my abilities?  I’ve consumed wine for a number of years, and really enjoy the experience.  My husband, while he drinks, is content with a glass of wine at dinner. 

If I had my way, and for quest of scientific knowledge only,  I would open a few bottles of zin to do comparison tasting.  But being a properly raised mid-western gal I have waste issues, as I know my husband would not keep up his end.  (My dream job belongs to Jeffery Steingarten at Vogue)   Then there is the matter that we get up at 5 am to run.  If you’ve never tried it, I do not recommend early rising after a delightful evening of sipping the juice.

Back to my original question, a study was performed at the California Wine Fair that showed only 30 of the 65 judges achieved consistent results.  Each judge received a flight of 30 wines, in which were embedded triplicate samples poured from the same bottle.  Only 10% of the judges replicated their selections in a single award category.  Put another way, 90% of the judges could not repeat their work!  90% – that’s huge!  To get a better picture, 80% of those judges scored the same wine from as a gold medal winner to no medal.

Given that ratings can directly affect the price charged for a wine I wonder why this issue is not raised, or the vineyards challenge the rankings.  Before you say, ah but its just the California judges that have this problem, I have news for you.  The American Association of Wine Economists AAWE agrees that this problem is systemic across the industry.   (Bet you didn’t know there was an American Association of Wine Economists)

Given the issue, and implications, the industry is trying out various solutions.  One suggestion for improving the rating system was to take the average of a small group of judges.  When this method was used, the results proved more consistent.  I do not know about you, but my self confidence soared after reading this news.  Now, I know that the experts face the same issues, they can be overwhelmed as well.



  1. There’s a book called “The Wine Trials” you’d probably like.

    Also, Gary Vaynerchuk’s is pretty good. He’s always saying things like, “Don’t trust my palate, or anybody else’s, trust your own palate.”

  2. Thanks for the heads up, I’ll definitely check out the sources you identified.

  3. Judging wine is definitely subjective because everyone has such different taste buds. Robert Parker is a good example – he has such different opinions from other wine experts and yet he has so much influence over the wine industry. This is a fascinating subject.

  4. I agree it is facinating. But the topic is vast, some folks focus on a specific wine, or color, etc. The more I unearth, the more I discover I do not know. Just means, I have to keep studying, and I’m definitely game!

  5. I’m always skeptical of reviews and ratings – for wines and restaurants. There seems to be so many factors involved that could sway a judge/critic one way or the other.

    Wine is always daunting for me and I find it difficult to make a selection. I know what I like and what I don’t, but is it considered good?

    I find that I rely on recommendations by people I know rather than scores. Taking advantages of tastings has made me more confident about wine as well.

  6. I agree, subjectivity is king. A pop culture comparison is the “best dressed at the Oscar’s or some other event – one critic might pan said dress and the another compliment the actress on her creative daring. (ok, I’ve let you in on a guilty secret, I’m a sucker for those reviews)

    This website, gives an alternative idea to people’s tastes that I find interesting. The author claims that people have various degrees of sensitivity and which reflects their wine preferences.

    I agree, it boils down to if you like it, its good. I had a prof in B School that used to lobby for Diageo and had access to wines of any price point. His preference typically fell in the $15-$30 range. He said that wines over that were lost on him. Me, its not the price point, except at the lower end. There, I can definitely tell a difference there, but more of the different vineyards and the variences of the same grape between them.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s


%d bloggers like this: