|
Your continuous donations keep RPGWatch running!
RPGWatch Forums » Comments » News Comments » Video Game Reviews - The Ten-Point Scale And Inflated Scores

Default Video Game Reviews - The Ten-Point Scale And Inflated Scores

June 16th, 2013, 00:30
Forbes has a new editorial discussing game reviews, and inflated scores.
There’s a perception out there among many gamers that video game review scores are inflated or biased. Why do so many games get scored between 7 and 10 on a 10-point scale? What happened to the rest of the numbers?

And why do some games—especially ultra-hyped AAA franchises—get such high numbers?

There’s a few things going on here, I think. First of all, the 10-point scale is deceptive.
More information.

"We must stand strong my fellow watchers against the horde of red trolls."- The Couchpotato
Couchpotato is offline

Couchpotato

Couchpotato's Avatar
LazyGamer
RPGWatch Team

#1

Join Date: Oct 2010
Location: Potato Land
Posts: 9,430

Default 

June 16th, 2013, 00:30
Why why why, oh, why?

In most cases the answer is - because journalists got payed.

Toka Koka
joxer is offline

joxer

joxer's Avatar
The Smoker
RPGWatch Donor

#2

Join Date: Apr 2009
Posts: 6,092

Default 

June 16th, 2013, 00:32
Originally Posted by joxer View Post
Why why why, oh, why?

In most cases the answer is - because journalists got payed.
That's the obvious answer and it's not always the correct one Joxer.

"We must stand strong my fellow watchers against the horde of red trolls."- The Couchpotato
Couchpotato is offline

Couchpotato

Couchpotato's Avatar
LazyGamer
RPGWatch Team

#3

Join Date: Oct 2010
Location: Potato Land
Posts: 9,430

Default 

June 16th, 2013, 05:03
I don't buy into the "it's too subjective" rhetoric. If you have a good grasp of a game, and of what other games and especially other games within the genre deliver, then you can slap a grade on it. The grade is always optional - if there's a 2-4 page review that doesn't align with the score in your opinion, go with the written word.

It only gets ugly/ murky when other people try to pick apart your scoring; because they don't know half the things that make you like/ dislike the game that weren't put into writing.

"In Grimwhoah, you can ride on turtles."
Sacred_Path is offline

Sacred_Path

Sacred_Path's Avatar
Basement Horror

#4

Join Date: Nov 2012
Location: Germany
Posts: 894

Default 

June 16th, 2013, 07:39
I agree with the article.

Regardless of swag-swayed reviews, our perception is to dismiss anything below a 6; so instead of a 1-10 grading system, it's more like a 6-10 point system.
Drithius is offline

Drithius

Drithius's Avatar
Misbegotten Alien

#5

Join Date: Nov 2008
Location: Florida, USA
Posts: 2,446

Default 

June 16th, 2013, 09:11
One might argue endlessly where a well-produced and narratively effective game like Last of Us belongs on the 7-10 scale, but review score inflation in general is not a phenomena I believe in anymore than I believe in flying saucers or libertarian economics.

The reason reviewed games usually don't get below a 6 is that most truly bad games are simply never reviewed (let alone played by more than a handful of unfortunate consumers). There's just so much shovelware out there that rightfully doesn't get any attention at all, and the handful of new releases each month which are reviewed by all the major publications are bound to be relatively decent games. Also, standards in AAA game design have risen considerably over the years, as the development/production process has been streamlined and professionalized in ways that were simply unheard of in, say, the early 1990s. This obviously does not mean that your average developer is likely to create a ton of games which fit the preferences of RPGWatch regulars; that's a whole separate issue…
Demiath is offline

Demiath

Watcher

#6

Join Date: Apr 2011
Location: Sweden
Posts: 70

Default 

June 16th, 2013, 12:31
The worst thing, imho, just is this :

At one point, readers just get *so much* used to these inflated numbers,
that games beneath a certain "soft wall" are just not bought anymore.

“ Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius – and a lot of courage – to move in the opposite direction.“ (E.F.Schumacher, Economist, Source)
Alrik Fassbauer is offline

Alrik Fassbauer

Alrik Fassbauer's Avatar
TL;DR

#7

Join Date: Nov 2006
Location: Old Europe
Posts: 15,966

Default 

June 16th, 2013, 12:57
Originally Posted by Sacred_Path View Post
…if there's a 2-4 page review that doesn't align with the score in your opinion, go with the written word.
That's the correct approach to most reviews BUT I can tell you one thing for sure, that most people won't ever read half of a two pages review and instead, go straight to the conclusion paragraph. Most of the crowd will go directly to the final note ! That's a major cause of frustration for a reviewer and the only way to avoid it is by writing short articles (not everybody's able to do that well though). It's very rare when people commenting about your work are able to quote the article honestly, as showing that they actually read the whole text. Instead you get partial quotes that don't actually reflect about what the article meant and that adds to the frustration. One of the main problem IMHO about written reviews is that people tend to think that while a person write reviews in a magazine they really are journalists but that's rarely the truth ! This is a real job and it's very demanding. Many writers are occasional ones (as I am). Contrary to what the people think, it's a work that mostly takes away the pleasure from playing games and it also takes time (a lot of it !) to learn how to be done well. Apart from being really gifted from the start, it takes time to understand how to synthesize weeks of gameplay into a couple of paragraphs and how to express the meaningful from the mundane parts of it. What I would say to people criticizing and even accusing writers of being corrupted, which is 99% of the time false, is to take the time to reflect and think about the amount of time needed before putting down those two pages of review and also to really take time to actually read it as a piece of work. Think about how you would have put it but not as a pastime, taking a few minutes from a week-end, but as a full time job that repeats week after week. Be intelligent and be tolerant !
Gloo is offline

Gloo

Gloo's Avatar
Sentinel

#8

Join Date: Nov 2012
Location: Chapelle Guillaume
Posts: 313

Default 

June 16th, 2013, 19:02
Interesting article and I agree 10 point scales are odd because in general we're all sort of pre-programmed to view the point-scale in terms of letter grades. 5 Point scales always made better sense to me. Anyway, here is my take on why so many reviews seem inaccurate:

- Reviewer dislikes the genre of the game he is reviewing. This happens so often these days that every now and then the reviewer admits as much in the review itself. "I really don't like RPGs but I was assigned this review and I have rent and a car payment to make so here goes…"

- Reviewer spends an amount of time with the game that is inappropriate considering the scope of the game. I remember reading a Two Worlds 2 review where right at the start of the review the writer stated something similar to, 'I spent 10 hours with TW2…' STOP. No need to read any further than that. 10 hours with a game like TW2? As a reviewer you're not even qualified to comment giving the game just 10 hours of time. Might as well apply to NASA and state on your application you hate math and science and have a clinical fear of flying.

- Reviewer makes too many comparisons to other games and/or conducts her analysis based on personal expectations that were never promised by the developer instead of judging the game primarily based on its stated features. It's ok to compare a little bit and it's ok to say things like, 'I wish this game had xyz' but so many reviewers these days indulge in every sort of comparison and/or expectations and spend little time analyzing the game to its stated features. These kinds of reviews read more like a user forum and much less like a 'professional' review. This trend is particularly rampant with online reviews.

- Lack of coordination and consistency by publishers to marry reviewers to appropriate genres. This is why you'll sometimes see bizarre scoring results over time between similar games and even with games that are generally considered improved sequels over predecessors scoring lower. I read a Skyrim review that praised the game up and down for 3 pages and then realized the reviewer gave it a lower score than the same publisher gave to Oblivion. This result is debatable either way but personally I thought it was odd as their Oblivion review seemed much more critical of that game but yet it received a higher score than Skyrim.

- Reviewer/Publisher is given 'gifts' or is otherwise bribed in some way (I do believe this happens a lot despite all the eyes rolling just now…). Even though Metacritic user reviews are an exercise in what bi-polar is all about, I thought D3 provided a fantastic example of 'funny business' going on behind the scenes when the first few metacritic "reviews" offered spectacular numbers for D3 while on the user side quite the opposite unfolded. In the case of D3, the poor user scores gave a more accurate picture over the review scores by a very wide margin. Things that make you say, 'hmmmmm….'

If I'm right but there is no wife around to acknowledge it, am I still right?
Last edited by TheMadGamer; June 16th, 2013 at 19:15.
TheMadGamer is offline

TheMadGamer

TheMadGamer's Avatar
SasqWatch

#9

Join Date: Oct 2006
Location: Oregon
Posts: 1,976

Default 

June 16th, 2013, 21:47
"…BUT I can tell you one thing for sure, that most people won't ever read half of a two pages review and instead, go straight to the conclusion paragraph. Most of the crowd will go directly to the final note !"

Yep, see that all the time. This goes right into the old "fool and his money are soon parted" folder. There's only so much you can do for folks like that. But then, if they don't care enough to read the article, do they REALLY care that much about what game they are getting? (Despite their screaming on the reviewer's/game company's forums afterward.)

I really agree with MadGamer - but I think there's another explanation to the Diablo 3 business and it points to another problem with reviews. Those reviews came out day 1, didn't they? The reviewers were reviewing on servers that had much, much lower populations. What's more, the "gold copies" often used in reviews also often do not have the DRM included. All this is in the name of getting your review out there first which certainly gets the reviews read more but it also really hurts the accuracy, particularly in games where server performance is a factor.
Zloth is online now

Zloth

Zloth's Avatar
I smell a… wumpus!?

#10

Join Date: Aug 2008
Location: Kansas City
Posts: 2,731

Default 

June 16th, 2013, 22:43
well, we could all make excuses as to why games journalism is so hard… 9_⑨

…or we could, like, not… I mean seriously, we're gonna justify using only three points of a ten point scale?? how do movie reviewers do it, bad movies are better than straight to video and youtube releases ahhh it's so confusing
qpqpqp is offline

qpqpqp

Watcher

#11

Join Date: Sep 2011
Posts: 86

Default 

June 16th, 2013, 23:52
Games below 50% are usually not fit to be released at all. Games like Gothic 3: Forsaken Gods (37%) and there's usually a story about how it got to the shelves more exciting than the game itself.

Mankind must put an end to war or war will put an end to mankind. - John F Kennedy
An eye for an eye, and soon the whole world is blind. - Mahatma Gandhi
The world is my country. To do good is my religion. My mind is my own church. This simple creed is all we need to enjoy peace on earth. - Thomas Paine
JemyM is offline

JemyM

JemyM's Avatar
Okay, now roll sanity.

#12

Join Date: Oct 2006
Posts: 6,028
Send a message via ICQ to JemyM Send a message via MSN to JemyM

Default 

June 17th, 2013, 08:32
I don't see the problem at all.

Games below 50% are very rare - and they should be.

Most games today - certainly for the mainstream - are above 60% - and they better be, or they won't make a dime.

Yes, the most popular games have inflated scores - mostly based on hype - but that has nothing to do with the 10 point scale.

People are obsessed with using all points of the scale, but that's because they're not thinking straight.

Anyway, that's my position.
DArtagnan is offline

DArtagnan

DArtagnan's Avatar
Waste of potential

#13

Join Date: Oct 2006
Location: Denmark
Posts: 14,366

Default 

June 17th, 2013, 09:36
People making excuses for there not necessarily existing games on the lower half of the scale should recall titles such as Dungeon Lords and it's 6.8 gamespot score
Drithius is offline

Drithius

Drithius's Avatar
Misbegotten Alien

#14

Join Date: Nov 2008
Location: Florida, USA
Posts: 2,446

Default 

June 17th, 2013, 09:55
Originally Posted by Drithius View Post
People making excuses for there not necessarily existing games on the lower half of the scale should recall titles such as Dungeon Lords and it's 6.8 gamespot score
Plenty of games have received lower than 50 - but they're still very rare overall.

As for Dungeon Lords - I quite like it and would have given it 5-6/10 myself.
DArtagnan is offline

DArtagnan

DArtagnan's Avatar
Waste of potential

#15

Join Date: Oct 2006
Location: Denmark
Posts: 14,366

Default 

June 17th, 2013, 10:17
I'm with DArtagnan on this one - yes, games with lower scores are rare, but they should be.

Developers actually put a lot of effort into their games. This isn't high school where lower grades are somewhat common because quite a few students don't give a crap. Most developers do, so their games end up being average or above. Every now and then a completely rubbish game comes along and then it gets 1-4, but it would be very odd if these games were common.
Maylander is offline

Maylander

SasqWatch

#16

Join Date: Oct 2006
Location: Bergen
Posts: 5,419
Send a message via MSN to Maylander

Default 

June 17th, 2013, 10:19
Originally Posted by Drithius View Post
People making excuses for there not necessarily existing games on the lower half of the scale should recall titles such as Dungeon Lords and it's 6.8 gamespot score
There are games that could score below 5/10 if critics would bother to review them.Check Total biscuit's why do I own series for 3 examples of such games.
Nameless one is offline

Nameless one

Nameless one's Avatar
Keeper of the Watch

#17

Join Date: May 2012
Location: Europe
Posts: 1,132

Default 

June 23rd, 2013, 01:37
sorry for reviving this but I call bullshit. Maybe if only 3 points on a ten point scale are worth using then we can all finally admit that that number scores are fucking stupid. if 9 = must have 8 = good for the genre and 7 or lower = utter garbage, then why mot just give games pass/fail and then, yknow, force people to READ if they want to UNDERSTAND, and not sell tenths of points to advertisers?
qpqpqp is offline

qpqpqp

Watcher

#18

Join Date: Sep 2011
Posts: 86
RPGWatch Forums » Comments » News Comments » Video Game Reviews - The Ten-Point Scale And Inflated Scores
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 03:24.
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Copyright by RPGWatch