Video Game Reviews - The Ten-Point Scale And Inflated Scores

Couchpotato

Part-Time News-bot
Joined
October 1, 2010
Messages
36,180
Location
Spudlandia
Forbes has a new editorial discussing game reviews, and inflated scores.
There’s a perception out there among many gamers that video game review scores are inflated or biased. Why do so many games get scored between 7 and 10 on a 10-point scale? What happened to the rest of the numbers?

And why do some games—especially ultra-hyped AAA franchises—get such high numbers?

There’s a few things going on here, I think. First of all, the 10-point scale is deceptive.
More information.
 
Joined
Oct 1, 2010
Messages
36,180
Location
Spudlandia
Why why why, oh, why?

In most cases the answer is - because journalists got payed.
 
Joined
Apr 12, 2009
Messages
23,459
Why why why, oh, why?

In most cases the answer is - because journalists got payed.
That's the obvious answer and it's not always the correct one Joxer.
 
Joined
Oct 1, 2010
Messages
36,180
Location
Spudlandia
I don't buy into the "it's too subjective" rhetoric. If you have a good grasp of a game, and of what other games and especially other games within the genre deliver, then you can slap a grade on it. The grade is always optional - if there's a 2-4 page review that doesn't align with the score in your opinion, go with the written word.

It only gets ugly/ murky when other people try to pick apart your scoring; because they don't know half the things that make you like/ dislike the game that weren't put into writing.
 
I agree with the article.

Regardless of swag-swayed reviews, our perception is to dismiss anything below a 6; so instead of a 1-10 grading system, it's more like a 6-10 point system.
 
Joined
Nov 10, 2008
Messages
5,978
Location
Florida, USA
One might argue endlessly where a well-produced and narratively effective game like Last of Us belongs on the 7-10 scale, but review score inflation in general is not a phenomena I believe in anymore than I believe in flying saucers or libertarian economics.

The reason reviewed games usually don't get below a 6 is that most truly bad games are simply never reviewed (let alone played by more than a handful of unfortunate consumers). There's just so much shovelware out there that rightfully doesn't get any attention at all, and the handful of new releases each month which are reviewed by all the major publications are bound to be relatively decent games. Also, standards in AAA game design have risen considerably over the years, as the development/production process has been streamlined and professionalized in ways that were simply unheard of in, say, the early 1990s. This obviously does not mean that your average developer is likely to create a ton of games which fit the preferences of RPGWatch regulars; that's a whole separate issue…
 
Joined
Apr 2, 2011
Messages
72
Location
Sweden
The worst thing, imho, just is this :

At one point, readers just get *so much* used to these inflated numbers,
that games beneath a certain "soft wall" are just not bought anymore.
 
Joined
Nov 5, 2006
Messages
21,908
Location
Old Europe
…if there's a 2-4 page review that doesn't align with the score in your opinion, go with the written word.
That's the correct approach to most reviews BUT I can tell you one thing for sure, that most people won't ever read half of a two pages review and instead, go straight to the conclusion paragraph. Most of the crowd will go directly to the final note ! That's a major cause of frustration for a reviewer and the only way to avoid it is by writing short articles (not everybody's able to do that well though). It's very rare when people commenting about your work are able to quote the article honestly, as showing that they actually read the whole text. Instead you get partial quotes that don't actually reflect about what the article meant and that adds to the frustration. One of the main problem IMHO about written reviews is that people tend to think that while a person write reviews in a magazine they really are journalists but that's rarely the truth ! This is a real job and it's very demanding. Many writers are occasional ones (as I am). Contrary to what the people think, it's a work that mostly takes away the pleasure from playing games and it also takes time (a lot of it !) to learn how to be done well. Apart from being really gifted from the start, it takes time to understand how to synthesize weeks of gameplay into a couple of paragraphs and how to express the meaningful from the mundane parts of it. What I would say to people criticizing and even accusing writers of being corrupted, which is 99% of the time false, is to take the time to reflect and think about the amount of time needed before putting down those two pages of review and also to really take time to actually read it as a piece of work. Think about how you would have put it but not as a pastime, taking a few minutes from a week-end, but as a full time job that repeats week after week. Be intelligent and be tolerant !
 
Joined
Nov 13, 2012
Messages
499
Location
Chapelle Guillaume
Interesting article and I agree 10 point scales are odd because in general we're all sort of pre-programmed to view the point-scale in terms of letter grades. 5 Point scales always made better sense to me. Anyway, here is my take on why so many reviews seem inaccurate:

- Reviewer dislikes the genre of the game he is reviewing. This happens so often these days that every now and then the reviewer admits as much in the review itself. "I really don't like RPGs but I was assigned this review and I have rent and a car payment to make so here goes…"

- Reviewer spends an amount of time with the game that is inappropriate considering the scope of the game. I remember reading a Two Worlds 2 review where right at the start of the review the writer stated something similar to, 'I spent 10 hours with TW2…' STOP. No need to read any further than that. 10 hours with a game like TW2? As a reviewer you're not even qualified to comment giving the game just 10 hours of time. Might as well apply to NASA and state on your application you hate math and science and have a clinical fear of flying.

- Reviewer makes too many comparisons to other games and/or conducts her analysis based on personal expectations that were never promised by the developer instead of judging the game primarily based on its stated features. It's ok to compare a little bit and it's ok to say things like, 'I wish this game had xyz' but so many reviewers these days indulge in every sort of comparison and/or expectations and spend little time analyzing the game to its stated features. These kinds of reviews read more like a user forum and much less like a 'professional' review. This trend is particularly rampant with online reviews.

- Lack of coordination and consistency by publishers to marry reviewers to appropriate genres. This is why you'll sometimes see bizarre scoring results over time between similar games and even with games that are generally considered improved sequels over predecessors scoring lower. I read a Skyrim review that praised the game up and down for 3 pages and then realized the reviewer gave it a lower score than the same publisher gave to Oblivion. This result is debatable either way but personally I thought it was odd as their Oblivion review seemed much more critical of that game but yet it received a higher score than Skyrim.

- Reviewer/Publisher is given 'gifts' or is otherwise bribed in some way (I do believe this happens a lot despite all the eyes rolling just now…). Even though Metacritic user reviews are an exercise in what bi-polar is all about, I thought D3 provided a fantastic example of 'funny business' going on behind the scenes when the first few metacritic "reviews" offered spectacular numbers for D3 while on the user side quite the opposite unfolded. In the case of D3, the poor user scores gave a more accurate picture over the review scores by a very wide margin. Things that make you say, 'hmmmmm….'
 
Last edited:
Joined
Oct 18, 2006
Messages
2,897
Location
Oregon
"…BUT I can tell you one thing for sure, that most people won't ever read half of a two pages review and instead, go straight to the conclusion paragraph. Most of the crowd will go directly to the final note !"

Yep, see that all the time. This goes right into the old "fool and his money are soon parted" folder. There's only so much you can do for folks like that. But then, if they don't care enough to read the article, do they REALLY care that much about what game they are getting? (Despite their screaming on the reviewer's/game company's forums afterward.)

I really agree with MadGamer - but I think there's another explanation to the Diablo 3 business and it points to another problem with reviews. Those reviews came out day 1, didn't they? The reviewers were reviewing on servers that had much, much lower populations. What's more, the "gold copies" often used in reviews also often do not have the DRM included. All this is in the name of getting your review out there first which certainly gets the reviews read more but it also really hurts the accuracy, particularly in games where server performance is a factor.
 
Joined
Aug 3, 2008
Messages
8,238
Location
Kansas City
well, we could all make excuses as to why games journalism is so hard... 9_⑨

...or we could, like, not... I mean seriously, we're gonna justify using only three points of a ten point scale?? how do movie reviewers do it, bad movies are better than straight to video and youtube releases ahhh it's so confusing :p
 
Joined
Sep 16, 2011
Messages
97
Games below 50% are usually not fit to be released at all. Games like Gothic 3: Forsaken Gods (37%) and there's usually a story about how it got to the shelves more exciting than the game itself.
 
Joined
Oct 26, 2006
Messages
6,027
I don't see the problem at all.

Games below 50% are very rare - and they should be.

Most games today - certainly for the mainstream - are above 60% - and they better be, or they won't make a dime.

Yes, the most popular games have inflated scores - mostly based on hype - but that has nothing to do with the 10 point scale.

People are obsessed with using all points of the scale, but that's because they're not thinking straight.

Anyway, that's my position.
 
People making excuses for there not necessarily existing games on the lower half of the scale should recall titles such as Dungeon Lords and it's 6.8 gamespot score :p

Plenty of games have received lower than 50 - but they're still very rare overall.

As for Dungeon Lords - I quite like it and would have given it 5-6/10 myself.
 
I'm with DArtagnan on this one - yes, games with lower scores are rare, but they should be.

Developers actually put a lot of effort into their games. This isn't high school where lower grades are somewhat common because quite a few students don't give a crap. Most developers do, so their games end up being average or above. Every now and then a completely rubbish game comes along and then it gets 1-4, but it would be very odd if these games were common.
 
Joined
Oct 18, 2006
Messages
7,583
Location
Bergen
People making excuses for there not necessarily existing games on the lower half of the scale should recall titles such as Dungeon Lords and it's 6.8 gamespot score :p

There are games that could score below 5/10 if critics would bother to review them.Check Total biscuit's why do I own series for 3 examples of such games.
 
Joined
May 21, 2012
Messages
1,436
Location
Sto plains
sorry for reviving this but I call bullshit. Maybe if only 3 points on a ten point scale are worth using then we can all finally admit that that number scores are fucking stupid. if 9 = must have 8 = good for the genre and 7 or lower = utter garbage, then why mot just give games pass/fail and then, yknow, force people to READ if they want to UNDERSTAND, and not sell tenths of points to advertisers?
 
Joined
Sep 16, 2011
Messages
97
Back
Top Bottom