Edge Magazine - Publishers Overrating The Importance Of Reviews

[skavenhorde]

There is an article done over at Edge Magazine on how review scores are one of the least important factors when it comes to buying games. Genre is the most important factor followed by how people liked an ealier series entry, word of mouth, how the game looked in the store, and feelings about the publisher. The last two factors were critics' scores and scores on aggregator sites like Metacritic. Here is a snippet from the aritcle:

Respondents were asked to identify the most important factors they consider when choosing which games to purchase. Using a scale of 1 (very unimportant) to 5 (very important), the genre of a given game was deemed to be most important, earning an average rating of 4.20.
The second most important factor, how much people liked an earlier series entry, earned an average rating of 4.06, demonstrating the value of strong game franchises. The third most important factor was the price of the game (rating of 3.82), followed by word of mouth about the game (3.70), how the game looked when seen in store, online or in advertisements (3.55), and feelings about the publisher's previous titles (3.33). Rounding of the list were critics' reviews of the game (3.19) and scores on review aggregator sites like Metacritic (3.11).

More information.

 

[Grandor Dragon]

Keep in mind that gamers were polled on this, which means that the data you get are based on self-reflections…how gamers see themselves, or even how they would like to see themselves based on their principles. If you assume that the decision to buy or not to buy games is psychologically complex and dependent on many factors, it would not be surprising to get a different weighting.

To do a proper study, you would create a large database on let's say a hundred games with their sales numbers, review scores, number of preview pages, advertising and PR budget, some measure of how much gamers liked the previous titles (in that case you could use a poll I think), etc. You then run some sort of univariate multifactorial statistical test to determine your predictors.

What I want to say is that from all the feasible studies you can imagine on that question, the study described in the article could be one of the worst.

 

[northreign]

That's why there should be a demo.

 

[Alrik Fassbauer]

Oh. That took long to find it.

Everyone knows that - at least among gamers - reviews are less important.

The casual buyer, however, is a different thing.

 

[Hindukönig]

Casual buyers even care less about reviews, as they pick up the games at the store casually. They usually don't try to inform themselves about the games they buy.

 

[zakhal]

> genre of a given game was deemed to be most important, earning an average rating of 4.20

Of course. I would never buy say NHL game. Som genres are harder to decide though like driving games or simulators.

>The second most important factor, how much people liked an earlier series entry, >earned an average rating of 4.06,

This is true too. If there was earlier entry then it has big influence especially if I liked it.

> The third most important factor was the price of the game (rating of 3.82), followed >by word of mouth about the game (3.70),

This must be an age/work status thing. I never think about price.

>how the game looked when seen in store, online or in advertisements

This is not relevant. Most games look alike nowadays. They are 3d with nice graphics. If the graphics are a bit worse it matters not if the game is good.

>feelings about the publisher's previous titles (3.33).

Well I would not buy just anything from bioware. I wouldnt buy a bad game. I never bought their first game!

>Rounding of the list were critics' reviews of the game (3.19) and scores on review >aggregator sites like Metacritic (3.11).

I would put critic reviews on the same place as user reviews. Equally important. Both of them would be number three on my list.

 

[khaight]

I typically use factors like genre, previous installments and developer to build a 'watch list' of games I might be interested in buying. Magazine and web previews can also put a game onto the 'watch list'. Reviews, metacritic scores and word of mouth can lead to my not purchasing a game on the 'watch list' (I'm looking at you, Alone In The Dark), but almost never lead to my buying something that wasn't on the list. In other words a bad review can be a deal-breaker, but a good review isn't likely to convince me to buy something I wasn't already contemplating.

 

[skavenhorde]

Wonder what the outcome would be if they asked the question: What influences you on NOT buying a game?

I bet reviews would be a lot higher up there on the list, next to type of publisher, and word of mouth. I never did understand all consuming hatred for one publisher other than EA. After Ultima 9 I was almost ready to just say forget it, but it's not like they care about just one person and they have made some other decent games before and after Ultima 9. Who cares what a certain publisher has done in the past, it is what they do in the future that interests me.

Maybe review scores might not be a major factor in buying a game, but it doesn't mean that publishers should ignore reviews. They still should use every trick they know to get a higher score so that their game doesn't get put on the "Not going to buy it no matter what" list of consumers.

 

[TheMadGamer]

Who cares what a certain publisher has done in the past, it is what they do in the future that interests me.

Consider the Jowood/PB debacle. The ongoing speculation of who was *more* at fault with the very buggy release of Gothic 3?

Then you consider the even more horrendous buggy... no... broken release of Foresaken Gods which PB had nothing to do with.

Then you consider the highly bug-free (PC version) release of Risen by PB of which Jowood had nothing to do with.

The result, at least for me, is that I will be highly cautious about purchasing Arcania, pretty much because for me, all signs point to Jowood for releasing buggy/broken games.

I hope I'm wrong. But in this particular case, consideration of publisher is high on my list.

 

[LuckyCarbon]

These rankings pretty well reflect my buying decision process.

1. Genre. I buy nearly all RPGs that come out and very little outside of the genre.

2. Sequels if they were good, get automatic consideration. Though since I rarely go outside of the RPG genre and I buy most RPGs....does a sequel really make a difference.

3. I'm more likely to consider a non-RPG if it's super cheap off a steam deal. If I really want a game the difference between $30 & $50 isn't going to stop me. PS: publisher's I do prefer $30.

4. Word of mouth is important, I really consider this review source #1. I would have never tried (and loved) TF2 if so many friends didn't recommend it.

5. I don't care much about how it looks as long as it's "modern", the UI is more important to me, especially if it's a potential console port.

6. I think the developer plays a larger role than the publisher for me. But maybe that's what they meant since most people don't pay as close attention to the industry as I do? Regardless who publishes: Bethesda, won't touch em. Bioware, I buy everything.

7. If a score on a AAA game is high, it's probably paid for, especially if it comes out the day of release. Metacritic is laughable since is combines scores from newspapers that don't know crap about games and write very general articles for non-gamers. While a bad review will make me read further, a good review is meaningless to me.

 

[wolfing]

To me it's all about 'features'. I read previews/reviews and dismiss the fluff trying to concentrate on the core aspects:
- Party based or single character
- Open world or storyline
- Turn based/semi-turn/action
- is it a clickfest (i.e. Diablo clone)

Then I check screenshots, no matter how 'good' the game seems to be, I just can't go too far back in terms of graphics. Baldur's Gate/Betrayal at Krondor is about as far ago as I can play.
In very few instances have I tried a game against my guidelines because of reviews/recommendations, and ended up liking it. Perfect examples: the Gothics, The Witcher, Titan Quest.
On the other hand, I've found great games that would go under the radar because they have the features I like, like King's Bounty and Drakensang, I had them in my "must buy" list long before they were released, and loved both.

 

[Zephyr]

Reviews of any sort, user or publications, mean little to me. Very low on the list if at all.
I have a mental check-list I go through. I prefer single-player, open world, real time combat RPGs. After Baldur's Gate and the like, I don't want to fiddle with all the party, turn-based combat, inventory shuffling, etc. That's why the Gothics are my favorites. Yes, even G3 for all it's faults. Same with FO3. Oblivion was too unfocused for me, it never really grabbed me the way Morrowind did.

 

[wolfgrimdark]

I only give reviews any weight if they are really BAD reviews - then that really tends to make me consider. Usually really bad means buggy, poorly done, low quality work. On the other hand a good review doesn't have to much weight for me. I do like reviews that go over the mechanics, features and detail in a game - and that will weigh heavy on my decision, but not the score.

Overall I tend to go in the same direction, except price which doesn't come into it all - Genre first, then if it a sequel to a game I enjoyed, features/mechanics and "look" of the game, publisher then review.

I don't really put to much into word of mouth, taste in games is often just to subjective.

 

[khaight]

I don't really put to much into word of mouth, taste in games is often just to subjective.

Word of mouth is a useful supplement to reviews, particularly for AAA titles. Assassin's Creed, as an example, got very good reviews; it was only in the word of mouth that the crippling repetitiveness of much of the gameplay became evident.

 

[txa1265]

Way I look at it is this: Transformers 1 and *especially* 2, each of which is amongst the worst movies of their year, are proof that people really have no desire for quality, substance or enlightenment. Mindlessness and proper hype rules.

 

[skavenhorde]

I don't really put to much into word of mouth, taste in games is often just to subjective.

The opinions here are one of the reasons I started visiting rpgdot and later rpgwatch. I've bought many a game and passed on a few because of something one of the staff or certain members here said about a game. Generally, our tastes are similar. So word of mouth might be subjective, but it still pulls more weight from someone who shares your interests than a review from someone I don't know at all.

I never would of tried Puzzle Quest without Magerette's glowing opinion of it. That game looked way too silly for me, but it turned out to be really fun. Another is Depths of Peril. That game looked way too much like another Diablo clone for me to be interested in it, but a few glowing comments from Dhruin and some of the others changed my mind on it. I gave it a shot and it was another fun game that I didn't expect to be.

 

[Cassius]

When the tone, level of literacy, and objectivity of the average professional review is on par with the average gamefaqs fanboy post why would anyone care about reviews? Look a the metacritic review summaries for any heavily marketed AAA game: "Game of the Year", "RPG of the Epoch", "Better than sex", "I write like a hyper-active five-year-old on a sugar high". Yaaaaaaaaawn. Opinion in general is an oversaturated market on the Internet, but sober, impartial opinion is as rare as tits on a bull.

 

[holeraw]

As I see it, a reviewer is someone who gets paid to play the games I won't like so I won't have to. Still no matter how 'professional' the reviewer is, some subjectivity, both on his and my part, can't be avoided. I think a review is most useful when I have some familiarity with the reviewer's views - if I trust his taste, the likelihood of me liking something that he praises is, naturally, higher… but even if we are not usually in agreement, if I'm familiar with his reviews I can fairly safely deduce from his comments whether I will like the game or not. (and that doesn't only go for games of course - I recently stopped buying a music magazine after a certain reviewer that I respected stopped writing for it)

It's with the internet making everything so much more accessible that the importance of a review seems to fade - too many reviewers to keep track of, too easy to find more in depth and extensive opinions from other gamers and the demo is only two clicks away… Back in the 90s however, a review was perhaps the most crucial factor in choosing a game to buy (I didn't even care much for genres nor to remember company names back then)

oh.. first post… hello

 

[DArtagnan]

Well, I have a few issues with reviews in general.

One would be that they've moved away from the ideal of objectivity and more towards personal "style" and flair than in the old days. So it seems to me, anyway. The price of having gaming become part of popular culture, perhaps.

Another would be the level of control the market-forces seems to exert over the media, and based on the various things we've heard throughout the years - such as the extremely damning Gamespot travesty with the reviewer that got fired, means I can have absolutely no trust in the words of reviewers connected with major sites. As the money aspect has taken over, corruption is too likely for me to appreciate the "honesty" of anyone in that world.

Lastly, I think it's extremely rare to find reviewers who understand the history of gaming and the various genres. They don't seem to understand the nuances of tradition and catering to certain smaller markets. It's all about casual gamers these days, which I find tremendously sad for people like me, who're primarily enthusiasts.

 

[Tragos]

Of course there is the question of how much someone "trusts" reviewers , specially when presentation is an exclusive ...

Personally i go to official forums and read all negative comments, if i can deal with the problematic parts of the game i buy it .

I feel that the genre being that important indicates that there is not much space left for innovations or hybrid games.