Rampant Games - A Tale of Two RPG Reviews

Gorath

Prime Evil
Staff Member
Moderator
Original Sin Donor
Original Sin 2 Donor
Joined
August 30, 2006
Messages
7,830
In his blog post A Tale of Two RPG Reviews Jay Barnson explains why he isn't fond of sites which are hiding the reviewer's identity. In the intro he uses the Pillars of Eternity reviews by the Codex and by us as positive examples to illustrate why you want to know the author and his background.

I am still not far enough to really comment, *BUT* I find the difference between two reviews by hardcore, old-school sites pretty intriguing for Pillars of Eternity:
More information.
 
Joined
Aug 30, 2006
Messages
7,830
How you receive a game has a lot to do with your expectations.

A game that is objectively good, can be judged as bad if your expectations were too high.

A game that is objectively mediocre, can be judged as very good if your expectations were too low.

It's hard to have zero expectations, because everything is hyped, early accessed and discussed very much nowadays.
 
Joined
Oct 18, 2006
Messages
20,056
Location
Germany
I generally dislike reviews that pan a game for not being unique or innovative enough… for supposedly not 'standing out'. While those abnormal elements may be memorable, they often aren't the defining qualities that make for an enjoyable game. Surprise plot twists are often welcome, but just being different for the sake of it is just catering to a niche market.
 
Joined
Mar 22, 2012
Messages
5,531
Location
Seattle
So really, this is a comment on the usefulness of aggregate reviews.

TBH I find the Metacritic user review aggregation terrific but Rotten Tomatoes almost useless. I used to use the Yahoo user reviews for the individual commentary but it seemed like they had to shut them down because they were flooded by paid reviewers.

Amazon had the same problem, such as the last Dune book (in the timeline) that was outlined by Frank Herbert and penned by his kid and Kevin Anderson. You couldn't find a bad review of it because those reviewers were getting deleted (so said some of the reviews that managed to slip through). This indicates there was a commercial bias in the reviews themselves.

Which also speaks of Jays reviews from "the old days". didn't we (and don't we still) have constant complaints of bias in AAA games?
 
Joined
Oct 19, 2006
Messages
5,215
Location
The Uncanny Valley
I generally dislike reviews that pan a game for not being unique or innovative enough… for supposedly not 'standing out'. While those abnormal elements may be memorable, they often aren't the defining qualities that make for an enjoyable game. Surprise plot twists are often welcome, but just being different for the sake of it is just catering to a niche market.

And pushes the art forward.
 
Joined
Feb 23, 2009
Messages
522
I had a friend who did game reviews for a particular high-profile site in the early 2000s that I don't think is around anymore. For one major game (not AAA, but still a decent ad-buy for the website), she gave it a score that was something like a full point or so below it's aggregate average of 8.7 or something.

She came under tremendous pressure from her editors to revise her score more towards the existing average. She was sent patches for the game, and asked to take certain other aspects of the game under consideration. Of course, it was couched in the perspective of, "it's your call," so they wouldn't change the score without her leave, but they definitely encouraged her to change her call.

Afterwards (I don't remember if she changed it or not, but I think she did), she was no longer given the opportunity to review games from any company that was a significant advertiser. I think she was a little happier with that, anyway, but she quit doing game reviews a few months later. The experience burned her out hard.

But from the website's perspective, she was quite replaceable. The only thing that mattered was that they wouldn't be considered an outlier - and to do that, they had to come in with a reasonable range of the aggregate. And once that aggregate score has been set (by the first reviews out there... which are often not by people who finished the game), the "lesser" sites and magazines have to fall in line or run the potential loss of credibility and advertising dollars.
 
Joined
Oct 9, 2007
Messages
624
Kinda just reinforces that a review is an opinion that happens to get published and that hopefully has some coherency to the writing - but ultimately, it's just the opinion of someone, and really holds no more weight than my, your, or any other random person's opinion.

Sometimes I factor reviews into how I perceive a game, sometimes not. I potentially investigate a variety of sources when I'm "researching" a game I might want to buy.

As HiddenX was saying, expectations, and IMO personal tastes (reviewers reviewing genres they don't like boggles the mind), and other factors can play a part too.
 
Joined
Apr 22, 2013
Messages
633
Location
Arizona
I had a friend who did game reviews for a particular high-profile site in the early 2000s that I don't think is around anymore. For one major game (not AAA, but still a decent ad-buy for the website), she gave it a score that was something like a full point or so below it's aggregate average of 8.7 or something.

She came under tremendous pressure from her editors to revise her score more towards the existing average. She was sent patches for the game, and asked to take certain other aspects of the game under consideration. Of course, it was couched in the perspective of, "it's your call," so they wouldn't change the score without her leave, but they definitely encouraged her to change her call.
We had an advertising campaign once for Mars: War Logs and also did a review. Our reviewer thought the game had some serious things holding it back, resulting in a somewhat negative review.
The publisher came back to me and was not amused. I shrugged and told them that running an ad campaign is not what keeps this site floating. The review is what it is.
We never did business again :)
 
Joined
Aug 30, 2006
Messages
11,223
I used to love Scorpia reviews from Computer Gaming World. I miss getting her reviews and that magazine. Some things from the 80's and 90's rocked.
 
Joined
Aug 28, 2010
Messages
2,861
Location
Wolf Light Woods
Is this actually a thing?

I think there's a *level* of objectively good. Or at least levels of objectively bad.

But the longer I've been playing games (with an analytic eye, though not usually as a reviewer), the more I've come to the belief that beyond that, it really is just opinion. Good reviewers can explain and describe their opinion well. Bad reviewers depend on hyperbole or false objectivity.

It is my experience that every game is a mix of objectively good and objectively bad... how those elements get weighted in the mind and heart of a player is subjective. And then beyond that, there is a ton of subjective matter that may resonate with a particular crowd (or the mass market), or may not. If I feel the good significantly outweighs the bad, and the game resonates with me, then I'm inclined to say, "It's a GREAT game!" And if I do a good enough of a job explaining why and how it does that, you can get a good idea about whether or not you can do the same.

But throwing my opinion of "it's a great game!" labeling it a 5/5 and throwing it on a stack of other opinions to come out with an aggregate numerical rating doesn't do a whole lot of good, and IMO can do more harm than good during the industry's obsession with these numbers, and a tendency for groupthink to take hold.

Interestingly, a few months ago I saw a movie review for a film that absolutely panned it... for all the reasons (fortunately, this reviewer could explain his rationale) that I thought I'd love it. It was because of this bad review I watched the film, and it was one of my favorite films last year.

But aggregated, that opinion only moved the film's aggregate review more towards the center... the "meh" zone.
 
Joined
Oct 9, 2007
Messages
624
Another issue is the various aspects of a game. Aside from story alone, you have game mechanics, graphics, optimization, sound design, etc. If any one or several are lacking, it can bring a game down regardless of how well other aspects are executed.
Nowadays, everyone seems enamored with graphics. If a game doesn't possess dazzling eye-candy, many consider it crap.
IMO, whizz-bang graphics are only gravy, not my meat and potatoes. Sure, I appreciate a good looking game, but that's not why I play it. Decent story, good writing, smooth game play, and I'm happy.
 
Joined
Jan 23, 2009
Messages
601
Location
Minnesota
I think the base of a game, the mechanics and technical aspects of a game can be objectively good or bad to a big degree. Now depending on the kind of game this has a bigger or smaller impact.

If a racing sim is easy to get into, Experts say that the cars control like real ones, and the graphics are top notch, and it has some features providing some entertainment on the long run, you could basically already say that it's gets like 100% of the "checklist" aspect, which might be around 70-80% of the total value. Now the rest is the personal impression of the game.

Other genres have less technicality, and less check list based features. For example the Telltale Adventures, which I wouldn't even call Adventures but Interactive movies.
Now if the engine is decent, and the few riddles there are, are more or less logical, there isn't much more you could say about it. Most of the rest is personal taste.

I guess that RPGs are somewhere in the middle. Having a lot of technical/mechanical aspects, which can work or not, and lots of personal aspects. And of course people don't just have a different opinion within the personal aspects, but also a different weighting between mechanical and personal aspects. E.g. I weight the mechanical aspects extremely high and therefore in my Conclusion Video regarding Blackguards I gave it a thumbs down. People who put less weight on it, will rate it differently.

But I guess what HiddenX actually meant is
objectively = unbiased.

One game where I saw a lot of biased Opinions, just a couple of hours ago, is Civilization Beyond Earth. This game had big ambitions and people expected a better Civ 5 merged with Alpha Centauri and got a game which was a mix of that, but did not live up to the expectations. But that doesn't make it a bad game. It's still a good game. It's just not as good as we wished it had been.
Same with pillars. If the game provided everything the Developers made you expect like more weight to attributes and skills in scripted interactions and similar stuff, it would have been an even greater game. But nevertheless it is a good game.

I mean it's like creating a Car, then one year later put out an improved version of the Car which should get a better rating than the first version. Now does it become a worse car if the developers promised that the improved version can fly but then it turns out it cant? Don't think so.
 
Joined
Jun 2, 2012
Messages
4,699
I beg to differ with many posters here, while also agreeing with them in most of the things they say - the difficult area for objective analysis is the human element.

You can have objective game scoring, it just takes too much aptitude than the average person is willing to invest. It requires you to be able to step back from the experience and have a properly open-minded approach. Sounds easy enough, but why don't we see this? For all the reasons mentioned above, the human element in the system prevents the system from functioning because the human element has needs and requirements beyond the stated goal.

For example, take games like Dragon Age or Pillars of Eternity. If you were to objectively score them on their success at providing interesting combat, any correctly objective review would mark them down very heavily, possibly as much as 0.7 off the final 10, so the games cannot be, objectively, a 9.5 or above before you've even closely analysed anything else.

The objective part of the combat is the extent to which the game succeeds in what it was trying to achieve - not the subjective notion of whether you enjoyed the combat.

Now take a near universally accepted crappy game, Sacred 3, and apply the same rationale - did the game's combat achieve what it was supposed to - as in, work faultlessly with the associated mechanics, provide enough of an interesting and varied experience for the kind of person who likes that form of combat? And, objectively, Sacred 3 could well have a better combat score for the purposes of review.

Because you would be buying the game based on the description of the play-style and intent, not just because it's something that claims to be an RPG.

Though rarer, a fan of the Sacred 3 method is generally a happy camper, they got what they paid for. The people who bought PoE and DA:I would be fully justified in feeling they did not get what they paid for, because both these games advertised highly tactical combat as a sales promotion and both apply mechanics/gameplay which is/are underutilised for this purpose. And that's what a review should be, a buyer's guide.

For all three games, however, there's also the human element of the titles being associated with other games - they are all three supposedly sequels. A fully objective review doesn't care about this, sequel or not, you should have researched objective reviews before you purchased the game - tough puppies if your beloved has metamorphed into Frankenstien's Monster, that's not objective, that's you being depressed because something you like isn't there.

The objective scoring system just looks at what the game prioritises and scores based on how it implements its priorities. And, yes, boring is an objective reasoning, because boring is a fault of a known element. People might have different boredom thresholds, but boring is still pretty obvious to anyone with a sense of empathy.

The best time to make an objective review is a few days after completing the game for the first time, while every thought you had while playing is still fresh, but you've also had time to cool down from the exhilarating/tedious end boss.

However, humans are humans. If objectivity produces a world where Sacred 3 scores higher than Pillar of Eternity, are you absolutely sure you wish to follow true objectivity?

Is 100% objectivity as equally incorrect as 100% subjectivity…? (dun, dun, dahhhh)
 
Joined
Nov 1, 2014
Messages
4,778
Yep, totally agree (while I can't judge wether DA:I or Sacred 3 are in itself good or bad games).

But what I wanted to get at before is that 100% objectivity is only possible for a X% of the game.
Each game has elements which cannot be measured objectively.
Most obvious thing would be the setting: Do you like/dislike a post apocalyptic setting?
Other things would be: Do you like/dislike Manga-Graphics in the Game? Do you like/dislike the writing? Do you like/dislike the scope of the story? Do you like/dislike intrigues in the story? Do you like/dislike that it is RTwP instead of Pause? And so on.
Of course you could just exclude that completely out of the review. But I think that would kill the enthusiasm you could bring into a review which might actually make it sound interesting. Also I think someone who states that he likes a post apocalyptic setting and says so could also be helpful by saying that he likes how good the atmosphere is brought to the game and so on. Stuff which is almost impossible without having this subjective "chain".

I think 100% objetivity is possible. Same as 100% subjectivity.

But these reviews would probably not very interesting to read.

A Review which is 100% objective could be:
This is my 100% objective review of Pillars of Eternity.
Pillars of Eternity has 6 races and 11 Classes. This is more than in most other games.
Hence the Game receives a score of 100% as these are the only two things which are measured in this review.

And a 100% subjective review would be:
This is my 100% subjective review!
I got the game from somewhere and played it a while. It sucks! 0%!
 
Joined
Jun 2, 2012
Messages
4,699
RampantCoyote made a good point in his article:
If you know a reviewer and his (subjective) opinions about games for a longer time, you can estimate how you would like the game reviewed by this reviewer.

I know for example Darth Roxor for a long time and Corwin even longer. I *know* their preferences in games. I know mine, too. Extrapolating from older reviews and postings I can decide how much I agree with them when I read a new review.

I have nothing against subjectivity in reviews as long as the reviewers tells me what his preferences and measuring criteria are.
 
Last edited:
Joined
Oct 18, 2006
Messages
20,056
Location
Germany
My take on reviews is similar to Jay's. I prefer to get to know the reviewer, then I'll use that experience to determine whether I always agree, or disagree with that person. Reviews are a personal opinion dependent on the reviewers personal preferences; once you're aware of them, it makes judging reviews much easier. Like others, I despise 'paid for' reviews and usually those by people who dislike the genre.
 
Joined
Aug 31, 2006
Messages
12,827
Location
Australia
I would wholeheartedly disagree with any suggestion that there could ever be an "objective" score or review for any game, much the same way I would object to the idea that there could ever be any "objective" review of a piece of music, a painting, etc. To claim that something is objectively good (or bad) necessarily requires that there is a universally (or at least nearly universally) accepted set of criteria for determining those things. But there isn't. Hence, getting to know the idiosyncracies of a particular reviewer when determining whether or not, or to what extent, to take their views into account is, in my opinion, pretty important.
 
Joined
May 14, 2015
Messages
3
Back
Top Bottom