Ratings and scores

Anything to do with Port.
Post Reply
PhilW
Dow 1980
Posts: 2510
Joined: 14:22 Wed 15 Dec 2010
Location: Near Cambridge, UK

Ratings and scores

Post by PhilW » 15:43 Fri 03 Mar 2017

There are a number of different rating/scoring systems used for wine and port, many of which will be familiar to people, including:
  • Wine Spectator : Score 0-100 (though with <75 rarely used)
  • Jancis Robinson : Score 12-20
  • Platter : Score 1-5
and I'm sure many others.

I've always felt uncomfortable giving absolute scores, preferring to rate in terms of poor, ok, good, verygood, excellent, or in shorthand:
  • poor, ok, g, vg, ex
This does not allow enough discrimination for me, so I also use half-way rating such as "g/vg"; at some point, especially when considering a lot of lovely bottles at a tasting, that was also not enough granularity, so I also now use + and -. Thus, for example the steps from "g" to "vg" are:
  • g, g+, g/vg, vg-, vg
I have been happy with this for several years now, so this is pretty much fixed, and used in all my tasting notes.

Any rating/scoring system is only useful if its meaning is clear. Translation from a score to a rating category such as "good" or "very good" is a start, but even then different people might use such terms differently. Personally, the categories I use have the following meaning:
  • faulty: Believed to have a fault (or otherwise be extremely poor and would never buy)
  • poor: Disliked, would definitely not buy
  • ok: Acceptable drinking, but would not buy
  • good: Enjoyable, will sometimes buy at good price
  • very good: Very enjoyable, will buy
  • excellent: Extremely enjoyable, keen to buy

PhilW
Dow 1980
Posts: 2510
Joined: 14:22 Wed 15 Dec 2010
Location: Near Cambridge, UK

Re: Ratings and scores

Post by PhilW » 15:44 Fri 03 Mar 2017

Every now and then I considered switching to a score system, but have not done so as it feels non-intuitive to me (which surprises me, to be honest); However, it can be difficult to then compare others scores/ratings with your own; for example, if I rate a port as "vg-", Alex scores it as 90, and Wine Spectator rate it as 87, who rated it most highly?

To try and answer this, and as part of past consideration of using a score myself, I attempted an alignment of scores/ratings/meanings between several systems. The results were interesting (to me, at least) and in recent conversation I was persuaded to share them with the group, so the following picture shows alignment of my ratings against Alex's scores, and my ratings against Wine Spectator's scores.

PW_vs_AHB_ratings.jpg
PW_vs_AHB_ratings.jpg (69.63 KiB) Viewed 700 times
PW_vs_WS_ratings.jpg
PW_vs_WS_ratings.jpg (76.24 KiB) Viewed 700 times

Clearly were I to adopt the Wine Spectator scoring system my scores would generally be lower than if I adopted Alex's scale; which is ironic since WS seem to only use the very high end of the scale most of the time, implying almost all wine to be utterly fabulous. In any case, Alex's scale seems to align well with my own (albeit that he has more rating categories than I, but without use of modifiers such as +/- which I use) and the scores seem more naturally appropriate somehow against the meanings for my taste. So, were I to use a points system, I would align fairly well with Alex, but I am now able to approximately translate between my own ratings and Alex's scores.

LGTrotter
Dalva Golden White Colheita 1952
Posts: 3684
Joined: 17:45 Fri 19 Oct 2012
Location: Somerset, UK

Re: Ratings and scores

Post by LGTrotter » 19:48 Fri 03 Mar 2017

Wow! That is some infographic you've got going there. Most pleasing.

I have gone to a 100 point scale which I have not calibrated precisely but it feels like it has a bit more give in it than a straight five point scale. I do not claim it is 'better' than other scales and accept that it is preposterous (and moreover meaningless) to say this Graham 77 is 91 points and this champagne is also 91 points. But I find it useful, the text is what I read first then the points/rating after. I think Alex has become the go to guy for notes partly because of their consistency and partly because he has tried so many and posts all the notes. He also seems to write the note in an orderly fashion. The number is no more than a nuanced estimate, at least for me.

I saw a wine forum doing a palate calibration test recently, where everyone goes and buys some readily available plonk and then rates it, the idea presumably being that everyone can compare how they rate the wine against the other members. I would suggest a port that literally everyone has; Smith Woodhouse 63.

Glenn E.
Quinta do Vesuvio 1994
Posts: 3349
Joined: 22:27 Wed 09 Jul 2008
Location: Seattle, WA, USA

Re: Ratings and scores

Post by Glenn E. » 20:26 Fri 03 Mar 2017

I use words first, then convert those words to numbers. I just find it easier to think "wow, this is excellent!" instead of "hmm... I think this deserves around 90-93 points." Once I have the words I can convert them... basically just like you're doing above. I've also found that having several scales helps a lot - one is "quality words", one is "would I buy that?", and one is "how much would I drink?" Note that "would I buy that" is very dependent on finances, so often isn't the most reliable indicator. It's also dependent on consumption, and since I drink less Port than most other Port lovers I tend to prefer to save my money to only drink the good stuff.

In the past my numbers have been slightly higher than Alex's while my words have sounded far better. Your charts show that very clearly compared to my scale. I may adopt his ranges for the 80s, as I often struggle there and could use more precision.

79 and lower: poor. Wouldn't buy. Might not finish the glass. There are theoretical levels worse than this one, but they're not really relevant.
80-84: fair, drinkable, meh. Wouldn't buy intentionally. I'll probably pass if offered another glass.
84-86: possible new range for "fine/good." Which would leave 87-89 as very good.
85-89: good to very good. Might buy as cellar defender if cheap enough. I'll probably accept if offered another glass.
90-93: excellent. Sometimes also "very nice" which looks a lot like "very good" but is expressed differently in my head and so is a higher rating. My baseline for buying. I might ask for another glass.
94-96: outstanding. I get teased that "outstanding" and "excellent" mean the same thing, but they do not - one definition of outstanding is "stands out amongst its peers in excellence" so to me that means it stands out amongst otherwise excellent wines. E.g. it is better than excellent. I'll probably ask for more while also asking where you got the bottle so that I can get some, too.
97-99: magnificent. Lots of glorious, hyperbolic words fit this level. I'll be eyeing the bottle to make sure I get more before it's gone. Sadly at this level price often becomes a hindrance for purchasing, but I'll want to buy it even if I can't afford it.
100: no words. Speechless. A life-altering experience. At minimum, changes my perspective on Port. I'll over-spend my budget to get these, but only to an extent. (I still haven't spent more than $1000 on a bottle... I think $750 is my current most expensive. The S. Leonardo "100".)

Another way I describe some of the levels is that 94-96 is the limit of mechanical perfection. Meaning that the winemaker made no mistakes and the wine is as well-made as it can be. 97-99 requires something more - magic in the glass. The sum must be greater than the parts. 97-99 will speak to you, whereas 94-96 will just be great. 90-93 may have a few quibbles... it's not quite sweet enough, there's a bitter edge, etc. 85-89 may have several quibbles or may have a small flaw.
Glenn Elliott

User avatar
CranBurgundy
Cheap Ruby
Posts: 12
Joined: 04:42 Fri 03 Mar 2017
Location: Noo Joizey

Re: Ratings and scores

Post by CranBurgundy » 21:07 Fri 03 Mar 2017

Rate using the binary system:

0 = don't buy again
1 = buy again

Voila! :lol:

Okay, seriously - I use CellarTracker's scoring system because that's where I post tasting notes.

98 - 100 Extraordinary
94 - 97 Outstanding
90 - 93 Excellent
86 - 89 Very Good
80 - 85 Good
70 - 79 Below Average
50 - 69 Avoid

As with most scales, I don't know why it's a 100 point system if the first 50 numbers aren't used, but what do I know? I assume it has something to do with grade inflation. :mrgreen:
Purple dranking cretin

User avatar
AHB
Quinta do Noval Nacional 1962
Posts: 11447
Joined: 13:41 Mon 25 Jun 2007
Location: Berkshire, UK

Re: Ratings and scores

Post by AHB » 00:10 Sun 05 Mar 2017

Glenn E. wrote:
20:26 Fri 03 Mar 2017
Another way I describe some of the levels is that 94-96 is the limit of mechanical perfection. Meaning that the winemaker made no mistakes and the wine is as well-made as it can be. 97-99 requires something more - magic in the glass. The sum must be greater than the parts. 97-99 will speak to you, whereas 94-96 will just be great.
You've just put perfectly into words something which had been bubbling away in my subconcious for ages. That's exactly it. 94-96 is a great wine, a really impressive wine. But 97-99 just has that extra soul, the magic that sings to you, that makes you weep when the glass is empty and there is no more in the bottle; that ingredient which makes you wish you wrote your tasting notes more quickly so others didn't empty the bottle before you came back for your second taste.

I like it, thank you!
Top Port in 2017 (so far): Graham Stone Terraces 2015 and Quinta do Vesuvio 1994
2016 Port of the year: Cockburn 1908

User avatar
AHB
Quinta do Noval Nacional 1962
Posts: 11447
Joined: 13:41 Mon 25 Jun 2007
Location: Berkshire, UK

Re: Ratings and scores

Post by AHB » 00:13 Sun 05 Mar 2017

CranBurgundy wrote:
21:07 Fri 03 Mar 2017
Rate using the binary system:

0 = don't buy again
1 = buy again

Voila! :lol:

Okay, seriously - I use CellarTracker's scoring system because that's where I post tasting notes.

98 - 100 Extraordinary
94 - 97 Outstanding
90 - 93 Excellent
86 - 89 Very Good
80 - 85 Good
70 - 79 Below Average
50 - 69 Avoid

As with most scales, I don't know why it's a 100 point system if the first 50 numbers aren't used, but what do I know? I assume it has something to do with grade inflation. :mrgreen:
I use the "100 point" system mainly because it gives me 25 real points to play with. People who use a 20 point system - which shouldn't be much different in its range to my 25 point system only tend to use the last 6 points or so and then have to add half points and such because 6 points doesn't give them enough wiggle room to differentiate between impressions of two wines.

But most importantly, I like people scoring because it allows me to see how they position wines against each other in relative terms.
Top Port in 2017 (so far): Graham Stone Terraces 2015 and Quinta do Vesuvio 1994
2016 Port of the year: Cockburn 1908

Post Reply