|
Your donations keep RPGWatch running!
RPGWatch Forums » Comments » News Comments » Video Game Reviews are Broken @ Kotaku

Default Video Game Reviews are Broken @ Kotaku

November 4th, 2007, 23:01
Kotaku has an editorial up on the subject of game reviews and why they don't seem to work properly, their theory laying the blame on the common concept of the numerical score:
…But after reading countless video game reviews over the years,…it's become more and more obvious that the video game review system is dated, limiting and even, at times, unintentionally condescending….
…If there is no such thing as a perfect game, then why the hell are you scoring out of 100? It's not just PC Gamer that thinks this way—most publications, even those who do give out "perfect" scores, do so begrudgingly. It's as if the developer has somehow cheated and broken their system.
The movie reviewers solved this problem a long time ago. That's why most adopted a simpler rating system in which a 4-star movie didn't imply "perfection" but supreme excellence. In most cases, games are penalized through being divided by a sum that they can never possibly reach. What does that make a 94 or a 9.5 then…is that our mortal interpretation of perfection? Is that the closest we can fly to the sun before our wings melt and we're doomed to playing Spongebob Squarepants XVI for eternity?…
…The fundamental problem with game reviews is that they're analyzing products, not pieces of art. Or more clearly stated, art reviews decide if something is worth your time; game reviews decide if something is worth your money. Let's go back to our convenient film comparison for a moment. When do reviewers ever complain that a movie is only an hour and a half long? They don't, because length as a value proposition is generally never affiliated with art…
Games are viewed as consumable goods meant to entertain for X hours at X amount per X dollars. There's an interest in durability (replay value, multiplayer), functionality (controls, camera), interface (HUD, menus), sex appeal (graphics) and accessibility (difficulty level)…
We really do direly need the video game review industry because there is just too much volume for gamers to get prioritize[d] alone, even with word of mouth, message boards, etc. We need people and organizations that can plow through multitudes of games to pick out the gems and crap on the…crap.
But we need them to analyze the subject in a critical light, judging a game's intrinsic value over its dollar value.
More information.
magerette is offline

magerette

magerette's Avatar
Hedgewitch

#1

Join Date: Oct 2006
Posts: 7,929

Default 

November 4th, 2007, 23:01
Bah, this could easily be fixed if reviewers stopped using decimal values or ratings out of 100. All the decimal values like 9.6/10 do is make it hard on a reviewer when a clearly better game comes out.
"Uh-oh, I rated Shooter 2 a 9.9/10, but Shooter 3 is clearly better! I'll have to rate it 9.95/10 - that should do it."
None of this really matters, since all readers really look for is where it generally falls: is it a 9-10, 8, 7, 6, or below? Reviewers should rate using simple ranges like Crap, Average, Above-Average, Good, Great, and Excellent instead of breaking it down to such tiny, nitpicky fractions.
Saber-Scorpion is offline

Saber-Scorpion

Saber-Scorpion's Avatar
Eyup.

#2

Join Date: Sep 2007
Location: Virginia, USA
Posts: 70

Default 

November 4th, 2007, 23:11
How do you put down impressions into something *anybody* can understand ?

We must decide upon a unified language first.

Numbers *can* be such a language, or at least most people seem to believe that.

“ Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius – and a lot of courage – to move in the opposite direction.“ (E.F.Schumacher, Economist, Source)
Alrik Fassbauer is offline

Alrik Fassbauer

Alrik Fassbauer's Avatar
TL;DR

#3

Join Date: Nov 2006
Location: Old Europe
Posts: 16,036

Default 

November 4th, 2007, 23:27
It occurred to me when I got a paper back from class and I was looking at votes at NWVault that numbered games reviews more closely resemble test scores and class grades than a "5 being average" scale. Less than 50% means failure here in NA.

Developer of The Wizard's Grave Android game. Discussion Thread:
http://www.rpgwatch.com/forums/showthread.php?t=22520
Lucky Day is offline

Lucky Day

Lucky Day's Avatar
Daywatch

#4

Join Date: Oct 2006
Location: The Uncanny Valley
Posts: 3,214

Default 

November 4th, 2007, 23:36
Exactly. Everyone (in general) sees it the same way, otherwise scores at Gamerankings or Metacritic from gamers would cluster around 5/10 — but they don't. In the main, they're remarkably close to the critics' scores.

-= RPGWatch =-
Dhruin is offline

Dhruin

Dhruin's Avatar
Keeper of the Watch
Super Moderator
RPGWatch Team

#5

Join Date: Aug 2006
Location: Sydney, Australia
Posts: 11,968

Default 

November 4th, 2007, 23:41
Just to add my two cents, I always thought that the best system for grading a game was the system PC Games used. (This was way back in the day where there were five or six magazines covering the PC scene. Later on PC Games would be rolled into PC Accelerator, which a year or two later was rolled out of existence, but that's another story…) It was simple, easy, and understandable- A-F, with pluses and minuses for precision. Just like in grade school. A was great, B was good, C was not great but playable if you really liked the concept/genre/developer/etc., D was bad, F was simply unacceptable. No E because you could add a single stroke to an F and… uhh… get spanked LESS severely by Dad, I guess.

But the thing is, you KNEW how it worked. Sure you'd get into arguments on how game X was scored too high or too low, but everyone understood what the score MEANT- none of this "Shouldn't 5.0/5.0 mean 'gaming nirvana'?" or "Is average 5.0 or 7.0?" crap. The first rule of writing is to talk in a language your audience understands. But no, we have to quantify instead of qualify because it's "more accurate", so we wind up confusing everyone with our accuracy.

If I ever make a game review site, I'll be using a 10.00 system and just generate scores with a random number generator. Play it up, too- trumpet the advantages of PUDSS, the Phenominally Useless Decimal Scoring System, which with much less effort produces scores equal to or better then any other gaming site on the web with accuracy. Go into a long-winded discussion about how it incorporates all aspects of game design into a byzantine scientific formula modified by genre, system, and relative hype index (RHI), then "rolls a hundred-sided die and inserts a decimal point to determine the final score." Add a site feature that allows you to rank all reviewed games based on score, then point them to the page expounding on PUDSS when they complain that Classic Game Y got 3.84 while Crapware Game Z got a record 9.76 for the genre.

I mean seriously, guys, does anyone actually read the damn review anymore?

BellosTheMighty

Guest

#6

Posts: n/a

Default 

November 5th, 2007, 00:05
Actually, YES!! I do. Check out our reviews here, while we use a numerical score, each number has a corresponding comment.

If God said it, then that settles it!!

Editor@RPGWatch
Corwin is offline

Corwin

Corwin's Avatar
On The Razorblade of Life
RPGWatch Team

#7

Join Date: Aug 2006
Location: Australia
Posts: 10,615
Send a message via Skype™ to Corwin

Default 

November 5th, 2007, 01:10
Reviews ought to provide readers with information they can use to make a decision. But some readers want more than that. They prefer to be told what decision to make. In that respect, numerical scores are really not much more than ornate "thumbs up / thumbs down" decisions.

The sad truth is there are quite a few people who struggle with decision-making throughout their lives and rely on help every step along the way.
Squeek is offline

Squeek

Squeek's Avatar
connoisseur of tidbits

#8

Join Date: Nov 2006
Location: Orange County, California
Posts: 1,807

Default 

November 5th, 2007, 08:59
The scoring system is really not among my major beef with game reviews. It is a really minor problem compared to reviewers having too little time to spend on each game, previews being taken way too seriously, or reviewers simply not understanding the type of game they are testing (my favourite is one reviewer who I think tried to play Europa Universalis III as a "standard" RTS, declared war against a bigger country without building up an army in advance, and then complained about an unbalanced game allowing France to outproduce him).

As for the ratings it is rather silly to use a 0-100 scale when a) there is no way to make subjective judgments with that kind of precision (any statistician will be able to tell you that too fine a scale makes for bogus results), b) only the last third of the scale is used. But those are really minor nitpicks when there is a consensus over what the ratings mean. Complaining about that would be like attacking flat earthers not for using the flat earth theory, but for using the imperial system of measurements!
Zaleukos is offline

Zaleukos

Zaleukos's Avatar
Bum

#9

Join Date: Nov 2006
Posts: 1,872

Default 

November 5th, 2007, 12:28
I don't mind 0-100, since a game has so much more stuffs to consider compared to a movie. I think the new gamespot way of rating is very interesting giving the game medals, both good and bad ones it gives you a quick overview and also shows why a game has a certain rating!
GothicGothicness is offline

GothicGothicness

GothicGothicness's Avatar
SasqWatch

#10

Join Date: Oct 2006
Posts: 4,366

Default 

November 5th, 2007, 14:36
I think the author has a good point here:

The fundamental problem with game reviews is that they're analyzing products, not pieces of art. Or more clearly stated, art reviews decide if something is worth your time; game reviews decide if something is worth your money.
about reviewers seeing their job as recommending a purchase instead of analyzing what a game actually is and how it achieves it's goal.

Yes, one of the things you need to know is if it's worth buying, but presenting the game on the same level as a plasma TV or a refrigerator misses the reason why we play games to begin with( the experience) and ends up focusing on appearance, (graphics) and how a game compares to other games instead of what it does that makes it work or not work.

Where there's smoke, there's mirrors.
magerette is offline

magerette

magerette's Avatar
Hedgewitch

#11

Join Date: Oct 2006
Posts: 7,929

Default 

November 5th, 2007, 15:16
Why not use school grade terms for that ?

“ Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius – and a lot of courage – to move in the opposite direction.“ (E.F.Schumacher, Economist, Source)
Alrik Fassbauer is offline

Alrik Fassbauer

Alrik Fassbauer's Avatar
TL;DR

#12

Join Date: Nov 2006
Location: Old Europe
Posts: 16,036

Default 

November 5th, 2007, 15:48
I don't think this depends on the rating system. In fact, sites such as metacritic.com use the same rating system in various works. Personally, when I am interested in a work, I read its reviews rather than just numbers, which may allow me to have fewer problems with rating systems. However, I cannot but agree that I am unhappy with the way video games to be evaluated, too. Generally speaking, the writing skills of reviewers of video games don't seem to match those of the reviewers of films/books…
Dusk is offline

Dusk

Sentinel

#13

Join Date: Jul 2007
Posts: 278

Default 

November 5th, 2007, 17:07
The article is littered with false premises, broken analogies and unsupported statements.
The fundamental problem with game reviews is that they're analyzing products, not pieces of art. Or more clearly stated, art reviews decide if something is worth your time; game reviews decide if something is worth your money.
It's not a problem, fundamental or otherwise, because:
a) for the most part that's not what game reviews actually do, and
b) even if they did that would be part of the solution not the problem, and
c) it's a false analogy in anycase.

This “games are art” concept is so flawed I don’t even know where to begin. I hope “games are art” advocates don’t go harassing tech support upon encountering game breaking bugs or poor system performance. Remember, art can’t be judged by such uncouth requirements as “fit for purpose” – that’s for tawdry things like consumer products.

Originally Posted by Dusk View Post
Generally speaking, the writing skills of reviewers of video games don't seem to match those of the reviewers of films/books…
Doubtless part of it is lack of technical skill, however they’re also writing for a different audience. A literary review is only of interest to those who are literate, whereas, judging by posts on mainstream forums, the average next-gen gamer is n0t biG on teh r3aDin R thiKin lol omg. I blame SMS messaging, the government, the parents, lack of corporal punishment, and Oblivion.
MudsAnimalFriend is offline

MudsAnimalFriend

Watchdog

#14

Join Date: Feb 2007
Posts: 150

Default 

November 5th, 2007, 17:18
Interesting article and I don't really disagree with any of it. But an even bigger problem than trying to find some kind of improved scoring system is the whole 'exclusive' deals many publishers live by.

Exclusivity breeds all kinds of business deals that have a lot more to do with making money than providing an honest review, whatever the scoring system might be.

If I'm right but there is no wife around to acknowledge it, am I still right?
TheMadGamer is offline

TheMadGamer

TheMadGamer's Avatar
SasqWatch

#15

Join Date: Oct 2006
Location: Oregon
Posts: 2,022

Default 

November 5th, 2007, 18:44
Yeah, I'd be more concerned with the currying of favour between publishers and reviewers.

Even the ostensibly innocent stuff. A writer visits the development team for an article or somesuch. It's easy to be more forgiving or positive when you've had a few laughs with the team, when perhaps you really shouldn't be.
Dyne is offline

Dyne

Watchdog

#16

Join Date: Oct 2006
Location: United Kingdom, London
Posts: 122

Default 

November 5th, 2007, 23:48
The real problem with game reviews is that most reviewers simply aren't trustworthy. The scale used for reviews is a very minor problem. Inflated reviews from biased, fanboyish journalists is the reason why my faith in the business is at an all-time low.
doctor_kaz is offline

doctor_kaz

Keeper of the Watch

#17

Join Date: Oct 2006
Posts: 622
RPGWatch Forums » Comments » News Comments » Video Game Reviews are Broken @ Kotaku
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 11:06.
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Copyright by RPGWatch