Wanted: dynamic (RP)Game rating system

Morrandir

SasqWatch
RPGWatch Donor
Original Sin 2 Donor
Joined
May 6, 2013
Messages
4,968
Location
Germany
Considering the recent discussion about ratings for D:OS or Skyrim, I again feel the need for something like a dynamic rating system.

I want reviewers to separately rate the components of a game being something like combat, plot, exploration, visuals, sound/score, usability, quests, dialogs, companions etc.
But instead of a static score being derived from these sub-ratings everyone should
have the possibility to give each of these components a separate weighting in some easy-to-use web interface, making the final score a dynamic result, depending on the weightings.

Of course the are hardly chances any of the big reviewers want to implement a system like this as they need to have a simple static score that they can publish.

But nevertheless I do want to have it! :)
 
Joined
May 6, 2013
Messages
4,968
Location
Germany
It would be good for RPGWatch as a rating system.
 
Joined
Jul 22, 2012
Messages
4,388
RPGwatch already has a very good system, but I'm not saying it can't be improved.

General dynamic game system simply can't be the same for each genre. One of things mentioned is quests. It's okay to have it of course, but many games don't have quests at all. :)

If you'll limit it only on RPGs, well, is it for singleplayer or MMOs then? You can't put a measure on MMO lags and servers in singleplayer games, but also you can't talk about savegame system in MMOs since they don't have one.

In a way, after determining what genre and playersnumber a game is, the system could actually be built. But as every other it will contain one thing deeply flawed - subjectiveness.

While every one of us can agree that some visuals are lame, some very original and some abnormally spectacular, what about stories? I assure you, there can be no consensus on stories, there is no such story everyone will just adore just as there is no story everyone would hate. Also, some people are very annoyed with many details, some on the other hand read only if there are so many details to get lost with and who cares for the base story.

It doesn't stop there. Music. I'm not saying my taste is perfect and when I say something rocks, that's what it is. There are some people like me who just can't be tortured by simplistic 3chords max or no music at all stuff. There are some people who believe parallel quints are the most beautiful thing they've heard in their life. And then there are people who would love if Bach never existed as they can't understand nor enjoy the complexness of his work. Finally some people want to play games without any music at all. What to do on that part?

What about voiceovers? I've seen unbelievable amount of nitpick posts "that's not brit, it's australian!". Um… So what? What I want to say, some are obsessed by accents, some like me just want to feel emotions from a voice (that's why I adore Claudia Black for example although her accent is different from usual USA movies, words she says are crystal clear and understandable).

Uggh. I'll get a headache…
I'm not saying it's impossible to make this system, I'm saying constructing it will be a tough job.
 
Joined
Apr 12, 2009
Messages
23,459
RPGwatch already has a very good system, but I'm not saying it can't be improved.

General dynamic game system simply can't be the same for each genre. One of things mentioned is quests. It's okay to have it of course, but many games don't have quests at all. :)

If you'll limit it only on RPGs, well, is it for singleplayer or MMOs then? You can't put a measure on MMO lags and servers in singleplayer games, but also you can't talk about savegame system in MMOs since they don't have one.
Make it dynamic again. There would be a pool of components, which may or may not fit the game. If it doesn't fit, it won't be considered in the review.

In a way, after determining what genre and playersnumber a game is, the system could actually be built. But as every other it will contain one thing deeply flawed - subjectiveness.
Of course subjectiveness is an inherent problem in recommendation systems. But you can limit its impacts by having averaged ratings of multiple reviewers and letting the users select whose reviews will be considered in the rating calculation. Users could select reviewers with a similar taste. This process may be automated as well, if you don't have a static set of reviewers, but letting the users do reviews/ratings as well.

Making the system itself isn't difficult and can even easily be done in excel.
Yeah, modeling the system is the more difficult part. Implementation wouldn't be much of a problem, though I wouldn't choose Excel. ;)
 
Last edited:
Joined
May 6, 2013
Messages
4,968
Location
Germany
To decide if I should buy a certain game or not.

Of course a system like that, like every recommender system, won't be perfect. But it would lower the chances of mispurchasing.
 
Joined
May 6, 2013
Messages
4,968
Location
Germany
To decide if I should buy a certain game or not.

You actually use reviews for that? Ok :)

But I'm not seeing how this is any different than something like Metacritic - except the sample would be tiny.

The thing about the Watch is that we're few people - and I'd say it's more useful to read our comments and go by what you know of our tastes.

A "weighted" system like that will only overcomplicate something that has no need for it.

Of course, that's just my opinion.
 
You actually use reviews for that? Ok :)
Sometimes, yes. Not for the big titles I'm hyped about and read a lot of stuff myself, but for smaller ones I use reviews indeed.

But I'm not seeing how this is any different than something like Metacritic
Two main differences as described before:
  • individual weighting of components
  • only reviews of selected reviewers are considered
The thing about the Watch is that we're few people
If I could get weighted ratings from 2-3 RPGWatch reviewers with a similar taste, that would be awesome. I don't need hundreds.

- and I'd say it's more useful to read our comments and go by what you know of our tastes.
Maybe. But also way more time consuming. Time is precious. My personal freetime most likely will decrease even more in the next years. :(

I know that this isn't something everybody needs, so most likely it wont be implemented anywhere.
But nevertheless I do want to have it! :)
 
Joined
May 6, 2013
Messages
4,968
Location
Germany
Sure it's not weighted in MC - but you do get a very large sampling and combined with what you probably already know about a game, it's pretty useful if other opinions are what you tend to use.

Time? You mean the time you spend reading threads anyway? Hehe, ok :)

Well, as I said, I think it will overcomplicate matters - but it means nothing to me - so if people want it, I say go for it :)
 
Sure it's not weighted in MC - but you do get a very large sampling and combined with what you probably already know about a game, it's pretty useful if other opinions are what you tend to use.
Well, my personal experience is, that I can in no way rely on ratings especially from MC.

so if people want it, I say go for it :)
I don't think enough people want it. If I'm wrong here, I'd even consider helping to actually implement it - whatever that may tell about my freetime management…
 
Joined
May 6, 2013
Messages
4,968
Location
Germany
Well, my personal experience is, that I can in no way rely on ratings especially from MC.

It depends on what you mean by "rely on". In most cases, you get a pretty accurate picture of what critics and users think of a game, represented by a score.

If you expect to agree with such an average, that would be your mistake :)

I tend to think of games as "what they're going for". I use my knowledge of the game, and then I look at the scores as how successful they are when it comes to what they went for, combined - of course - with more research.

In that way, it's a very useful tool.

Well, it would be, except I don't care to spend time doing research very often. I just accept that I'm taking a chance. But that's me :)
 
I think it would be great to have system that rates the games on individual parts ( well we are kind of doing it already on here in the pro/cons kind of things )

But let's say this is just for RPG's. It could be something like this as a very simple start.

Quests
Story
Exploration
Combat
Character Progression
C&C
——
Graphics
Sound
Music
UI

Then let's say I care the most about a good story, I could go into the review database and sort by story… and find exactly the games with highest score in that deparments immideitly. Which would be impossible to do with the tradional ratings system and with metacritics too. Of course it'd still not be perfect and still be subjective for example "the game of the generation" unmodded would get something like this from me.

Quests ( 4 / 10 ) There are some okay ones.
Story ( 0 / 10 ) There is a story ?
Exploration ( 5 / 10 ) Well it is generic but it is huge.
Combat ( 3 / 10 ) Simply horrible except for sneaking / archery
Character Progression ( 2 / 10 ) Not really working at all
C&C ( 4 / 10 ) Lack of meaningful choices
——
Graphics ( 4 / 10 ) Characters are dead ugly, environments have a generic look.
Sound ( 5 / 10 ) Sometimes annoying sometimes good
Music ( 6 / 10 ) Some scores are good.
UI ( 2 / 10 ) Horrible.
 
Joined
Oct 25, 2006
Messages
6,292
This is a pretty good demonstration of overcomplication :) Because what does a good story mean? I don't have to tell you that people think very very differently when it comes to a good story. This can be said for most anything.

Clearly GG has a very, shall we say, unique way of giving a fair and useful score.

Some people would be more interested in being of use to others - and then giving their idea of an objective score, and some people would rather just use their emotions when evaluating something.

This is no different, really, than when HiddenX tries to establish what an RPG really is.

Sure, you can work and work and come up with some kind of estimation with a ton of exceptions - but is it really worth it?

Obviously, some people think so - so there :)
 
GothicGothicness - Dart is right - you can get scores for the categories

Quests
Story
Exploration
Combat
Character Progression
C&C
——
Graphics
Sound
Music
UI

easily by just answering a bunch of YES/NO questions in the CRPG Analyzer checklists. Nothing complicated about it. Creating the lists was a lot of work, using them is easy.

PS:
Only difference: C&C and quests are integrated in the main category Story.

PS:
Even more information you get with the CRPG-Meter, but this system requires a lot of work and thinking from the reviewer.
 
Joined
Oct 18, 2006
Messages
19,818
Location
Germany
Well, I know a lot of RPG's wth good C&C and quests but not a good story :p
 
Joined
Oct 25, 2006
Messages
6,292
Well, I know a lot of RPG's wth good C&C and quests but not a good story :p

Indeed, which proves my point - as most people are a bit more balanced and don't use their emotional response when trying to create something for others to use as a guide :)

This is why you'll see sensationalist pieces - like is common on QT3 and Tom Chick - because he's emotionally invested around half the time, and can't be objective to save his life in those situations. Some places like to "stay calm" and attempt objectivity - which is only as useful as the level of insight the reviewer has, both in terms of the gaming industry - but also design, history of game evolution and the human mind.

Which is why these "averages" really don't amount to much of anything.
 
Also, I have to say there's a very disturbing amount of "established" reviewers who don't seem to care about what a game is trying to do - and instead simply compare it to other games that seem similar.

Another reason not to invest too much in critics :)
 
The CRPG Analyzer (Version 1.00) for Divinity Original Sin

The three core categories Character Development, Exploration and Story that need to be applied and quantified to determine if an interactive computerized game can be labeled as a Computer Role Playing Game (hereafter referred to as CRPG) are listed to show the necessary component elements and qualifying factors.
Any proposed or purported CRPG must contain all three core categories and their essential necessary Must Have conditions fulfilled to achieve the (minimal) CRPG status.

These core categories must maintain some form of progressive nature that will improve from when the game starts and leads to a conclusive game ending.

Each core category and the auxiliary category Combat also have related Should Have conditions; the reviewer should make a comment if a Should Have condition is not fulfilled.

So we have these scenarios to reflect the broadness of the genre:

  • At least one Must Have condition is violated => the game is not a CRPG.
  • All Must Have conditions are fulfilled => the game is at least CRPG'ish or a CRPG light.
  • All Must Have and some Should Haves conditions are fulfilled => the game is a CRPG that needs to be further qualified with (sub-genre) tags and/or reviewer comments.
  • If all necessary Must Have and all Should Have conditions are fulfilled there's no further discussion necessary => the game is a true CRPG.
Optional elements are listed in the Nice to Have (NtH) list. With it you get precise information which optional CRPG elements are implemented in the game. A general game info questionnaire is added too, to do some rating.


I. The Checklist:

Character Development
Describes ways to create, change or enhance your characters in order to increase their effectiveness in the game.

  • Must Have
    C1: you can control and role-play one (=Avatar) or more (=Party) unique characters (-> not only uniform units) yes
    C2: you can progressively develop your characters' stats and/or abilities (-> e.g. through an in game value (usually exp. points) gained by quests, exploration, conversation, combat, …) yes
    C3: Checks against character stats and/or character abilities/skills are necessary to make progress and finish the game yes
    C4: you can equip and enhance your characters with items you acquire yes
  • Should Have
    C5: you can create your characters yes, two of them
    C6: the player needs preplanning for the development of the character(s) yes, you can't learn everything in one play-through
    C7: the primary means of problem solving, gameworld interaction and overcoming challenges is the tactical use of character/party skills/abilities (-> the player's physical coordination skills are secondary) yes

Exploration
Includes how you can move through the game world, as well as everything you can find, see, manipulate or interact with, like locations, items and other objects.

  • Must Have
    E1: your character(s) can interact with the gameworld and find new locations by exploring. yes
    E2: your character(s) can find items that can be collected in an inventory (-> there have to be more item types than quest items, weapons, ammunition and consumable stat boosters.) yes
    E3: your character(s) can find information sources (-> e.g. NPCs, entities, objects that provide info) yes- a lot
  • Should Have
    E4: there are NPCs in the game yes, many
    E5: you can choose a path (-> there is at least some branching) yes - nonlinear game
    E6: your character(s) can manipulate the game world in some way (-> e.g. pull levers, push buttons, open chests, …) yes
    E7: the gameworld can affect your character(s) (-> e.g. weather, traps, closed doors, poisoned areas, …) yes
    E8: there are initially inaccessible areas in the gameworld that can only be reached by enhancing your characters' abilities, solving quests or puzzles (-> e.g. unlock locked areas, overcome obstacles, repair bridges, dispel barriers, …) yes

Story
Concerns all narrative elements like setting, lore, plot, characters, dialogue, quests, descriptions, storyline(s) and similar, including how you can interact with them.

  • Must Have
    S1: your character(s) can get information from information sources (-> e.g. hints, goals, quests, skills, spells, training, …) yes
    S2: your character(s) can follow quests (-> there is at least one main quest) yes, lots of quests
    S3: your character(s) can progress through connected events and play their role yes
  • Should Have
    S4: the story is influenced by your decisions and your characters' actions and stats/abilities/skills.yes
    S5: your character(s) can interact with information sources (-> e.g. NPC conversation, riddle statue question, …) yes
    S6: your character(s) can make choices in those interactions yes
    S7: at least some of these choices have consequences yes, many irreversible choices
    S8: advancing in the story requires thinking of the player (-> e.g. irreversible choices, moral dilemma, riddles, …) yes
Combat
Describes how combat (or more general: conflict resolving) corresponds with elements of Character Development, Exploration and Story.

  • Should Have
    F1: Combat efficiency is in some way tied to character stats or abilities (-> e.g. amount of damage, chance to hit, weapon access, …) yes
    F2: Combat works with some random elements (game internal dice rolls) yes
    F3: Combat should provide some challenge (-> e.g. preparing, use of tactics or environment possible) yes

Hints:

  • A game that fulfills conditions in the categories Character and Exploration but not in Story could be a Dungeon Crawler or a Rogue-Like
  • A game that fulfills conditions in the categories Exploration and Story but not in Character could be an Adventure game, a Strategy game or a Shooter.
  • A game that fulfills conditions in the categories Character and Story but not in Exploration could be a Simulation game or a Linear CRPG

Bottom Line
D:OS is a Western CRPG with tactical turn based combat that can be played in single player and co-op. It's a fantasy setting, you solve a lot of quests, and besides save the whole existence of everything. Sounds serious but a lot of Larian humor is sprinkled all over the game.
The turn based combat is interesting, challenging and offers many different tactics and options. Enemies are not scaled. Exploring is rewarding and essential to solve the game. A fair amount of puzzles and riddles are implemented, too.
Like the list above is showing all important elements of a full fledged CRPG are there. Some things like the GUI and balancing could be implemented a bit better, but overall a very good game for fans of games like Ultima 7, Baldur's Gate 2 or Fallout.
I have rarely seen a game with so many character options, secrets, crafting possibilities and different combat elements. Put this together with a high quest density, c&c and alternative quest solutions and there is the final score:

9/10 -> Two thumbs up!!
 
Last edited:
Joined
Oct 18, 2006
Messages
19,818
Location
Germany
Back
Top Bottom