The context and long-term relevance of scores as a means to refence quality

Nereida said:

I was just defining the game without much of a deep meaning. Merely speaking of it as a game that I liked, but to the general consensus is not a great game, which I have no trouble accepting.

Click to expand...

So you lack the willpower and conviction of your own taste and preference and are happy to let people bully a perfectly good game out of the conversation?

lackblogger said:

That's an awesome set of stats there Ivan, and, yes, its amazing how quick and simple it is to put together relevant and interesting stats, quicker even than writing out a long textual post.

Now, what are your conclusions from your findings? From the stats you've generated did you see a direct pattern between score and the longevity of popularity and quality assessment over time? Or did you find that score provided a pretty random an not very relevant guide to what people will probably remember as good/great games?

Click to expand...

There is a red trend line in the charts. If I'm reading it right, there is a slight trend of games that have a higher metacritic score getting more votes in our poll. I'm not sure if that would be considered a significant trend by statisticians, but I can see a clear upward trend. If we think of the top games of the decade poll as a measure of popularity/longevity, I would say that the more popular and fondly remembered games tend to have better scores.

Now, even I as a layperson can see that my analysis is not perfect. The sample size of games is too small (just 50 games) and I think our poll is also not a good measure of popularity. I think it also had a small number of voters and that is seen in the fact that a lot of games got the same percentage of votes. There is also an issue of Witcher 3 getting much bigger amount of votes than the next game and I don't know if that (or any other game on the list) would be considered an outlier and discarded.

JFarrell71 said:

Somebody is bound to get the last word by 2025.

Click to expand...

I'm sorry if this conversation isn't to your liking.

Some threads go on forever. Shocking, I know.

I feel sure the last word will come at some point, though have no idea what relevance that has to the topic. It will probably be some random account that only has 12 posts in 3 years who writes some shit one-liner.

Ivanwah said:

There is also an issue of Witcher 3 getting much bigger amount of votes than the next game and I don't know if that (or any other game on the list) would be considered an outlier and discarded.

Click to expand...

Let's find out, simply remove the undeniably most loved game of the decade and see what it does to the line.

lackblogger said:

I have no idea why I feel the need to quote these lines from three separate posts of yours, but I feel I must.

Perhaps I find them indicative of the quality of your argument and posting style generally. And therefore somehow relevant to the discussion.

Click to expand...

They are all relevant at the time. The first one I was done with your circle arguments.

Then more people chimed in, and since there was more than a wall to talk to, I felt compelled to keep the debate going. But you lost your temper and began insulting me instead of addressing the topic, so I lost interest in you.

I should have stopped there, but then you became all docile and nice, trying to backpedal in a hail-mary attempt to save face, so I made a closing statement. I am truly done educating you here.

I am a polite person, that was talking to another polite person that has nothing to do with you or educating you to commend them on the good work they did with their graph. Then you asked a question, and because I am a polite person, I answered. I am still done educating you, but that does not impede me from being polite when addressed.

The fact that you need to resort to such childish exercise as to quote that to gain any foothold about anything in this thread proves how desperate you are, and how far you've fallen, but don't fret. There is always one level more to go.

And please, do not take that as a challenge.

Nereida said:

They are all relevant at the time. The first one I was done with your circle arguments.

Then more people chimed in, and since there was more than a wall to talk to, I felt compelled to keep the debate going. But you lost your temper and began insulting me instead of addressing the topic, so I lost interest in you.

I should have stopped there, but then you became all docile and nice, trying to backpedal in a desperate attempt to save face, so I made a closing statement. I am truly done educating you here.

I am a polite person, that was talking to another polite person to commend them on the good work they did with their graph. Then you asked a question, and because I am a polite person, I answered. I am still done educating you, but that does not impede me from being polite when addressed.

The fact that you need to resort to such childish exercise as to quote that to gain any foothold about anything in this thread proves how desperate you are, and how far you've fallen, but don't fret. There is always one level more to go. And that is not a challenge.

Click to expand...

Well I'm glad you've got all that posturing off your chest, now we can get back to looking at that the stats, huh.

lackblogger said:

Let's find out, simply remove the undeniably most loved game of the decade and see what it does to the line.

Click to expand...

The upward trend seems to remain but the slope is a bit flatter this time.

Ivanwah said:

The upward trend seems to remain but the slope is a bit flatter this time.

Click to expand...

Yes indeed, you can detect the fact that score certainly has a small effect on likely popularity.

Looking at those dots though, on both sides of the graph, they really do look incredibly random don't they.

If you were a betting man, and you now have the power of stats to use to help you make a bet with your mate, could you use those stats to make the bet that Game A, which released to high praise and good initial reviews, say 88%, is defiantly going to be more fondly remembered than the other game released that month to middling praise and slightly more rocky reviews, say 79%?

Could you, with your newly found knowledge, put your money where your mouth was? Or would you, like me, refrain from betting but be happy to offer an educated prediction that includes life experience?

I say this because any other answer than "yes I would lump $100 on the 88% without hesitation" directly contradicts Nereida's entire argument.

Again, this is a small data-set and it's hard to make predictions, but if that is all I have to work with then sure, why not, put $100 on the 88% game.

But instead of lamenting over the small data-set, I remembered this gem. It's not most up-to-date list out there but it comes with the added bonus of a ready made spreadsheet with the scores. I just need to add the metacritic scores and make the charts. It's still not perfect, but the list has over 200 games and more than 350 people voted. It is also an "all-time" list so newer games (at the time of the voting) being on the mind of voters will have a lesser impact.

Let's see what I can do with this. I'll start right now, but I' won't finish it all tonight since I have a work in the morning (it's 23:38 here at the time of writing). Expect the thing to be ready sometime tomorrow.

Ivanwah said:

Again, this is a small data-set and it's hard to make predictions, but if that is all I have to work with then sure, why not, put $100 on the 88% game.

Click to expand...

:lol::lol::lol::lol:

Yeah, ok bro.

Now how about $10,000?

(And yes, I'll keep going until I reach a figure that isn't what you consider loose change)

lackblogger said:

Now how about $10,000?

(And yes, I'll keep going until I reach a figure that isn't what you consider loose change)

Click to expand...

By doing that you are testing how much of a betting man I am and that is not relevant. I can tell you that I personally wouldn't bet a dime on it because I don't like betting at all. But if I were a betting man who goes by the data the amount is irrelevant, the data shows which one is more likely and I would go by that.

But let's see what we get from the Codex data, I might change my mind.

Ivanwah said:

By doing that you are testing how much of a betting man I am and that is not relevant. I can tell you that I personally wouldn't bet a dime on it because I don't like betting at all. But if I were a betting man who goes by the data the amount is irrelevant, the data shows which one is more likely and I would go by that.

Click to expand...

When you say 'more likely' you make it sound like it's a favourite though, when it's not at all, it's effectively evens. Ok, marginally off-evens, the 88% has a 55% chance and the 79% a 45% chance.

That's not what 'sensible' people 'bet' on. You're likely imagining yourself at a roulette table with an instruction to bet. Well, hey might as well go red or black, heh. It's not a rational decision, it's a coin flip.

You have no idea if it will be red or black, but your imagining yourself as a betting man at a roulette table making a tourist bet aren't you.

You're "fuck it, probably" shows this.

Professional betters bet on sure things. sure, they may bluff here and there, but I think it's frowned upon to bet your estates wealth on a near 50/50.

Now think again. Take yourself away from the fact that I said "If you don't make the bet then you've proven Nereida incorrect", forget you're heavy bias to disprove me for a moment. Would you bet your house on the 88% game? Given the stats readily available for all to see. is 55/45 a 'sane' situation in which you wish to stake your house? You also do not need to bet, no-one's forcing you.

lackblogger said:

When you say 'more likely' you make it sound like it's a favourite though, when it's not at all, it's effectively evens. Ok, marginally off-evens, the 88% has a 55% chance and the 79% a 45% chance.

That's not what 'sensible' people 'bet' on. You're likely imagining yourself at a roulette table with an instruction to bet. Well, hey might as well go red or black, heh. It's not a rational decision, it's a coin flip.

You have no idea if it will be red or black, but your imagining yourself as a betting man at a roulette table making a tourist bet aren't you.

You're "fuck it, probably" shows this.

Professional betters bet on sure things. sure, they may bluff here and there, but I think it's frowned upon to bet your estates wealth on a near 50/50.

Now think again. Take yourself away from the fact that I said "If you don't make the bet then you've proven Nereida incorrect", forget you're heavy bias to disprove me for a moment. Would you bet your house on the 88% game? Given the stats readily available for all to see. is 55/45 a 'sane' situation in which you wish to stake your house? You also do not need to bet, no-one's forcing you.

Click to expand...

Irrelevant post. How about we wait I do the Codex analysis and see what comes from that instead imagining stupid scenarios?

Ivanwah said:

Irrelevant post. How about we wait I do the Codex analysis and see what comes from that instead imagining stupid scenarios?

Click to expand...

It's really not. It's everything.

The value of the bet is everything - this simulates Nereida's definite position.

Only one bet can be made because Nereida's position is so definite that no room is made available for contrary thoughts. There is no hedging in Nereida's position.

If Nereida's position does not motivate you to 'bet it all', Nereida's position falls apart.

Your reaction is very telling. And sadly disappointing.

lackblogger said:

It's really not. It's everything.

The value of the bet is everything - this simulates Nereida's definite position.

Only one bet can be made because Nereida's position is so definite that no room is made available for contrary thoughts. There is no hedging in Nereida's position.

If Nereida's position does not motivate you to 'bet it all', Nereida's position falls apart.

Your reaction is very telling. And sadly disappointing.

Click to expand...

I don't recall her offering any betting advice. Can you point me to where she does that?

@Ivanwah; I find it commendable that you are putting a lot of time in what you find to be honest research on the topic, but I hope you realise that he is trying to get you to compare a sample size of [everyone who reviews games in the world] to a sample size of [20-50 people who voted in a very niche oldschool RPG site], and average them in a 1:1 proportion, as if they were of equal value, while one of them is not even representative of a 0.1% of the other, and what is more, that 0.1% is already included in the massive amount of data gathered by the other, so it's pointless to average it further, it's already factored in.

It is a manipulation strategy commonly used by conmen and preachers in the internet, that you are familiar with when you are in the business of debunking fallacies.

Just saying, because I think you have good intentions and because of it you're being taken advantage of and pushed into a corner.

Nereida said:

but I hope you realise that he is trying to get you to compare a sample size of [everyone who reviews games in the world] to a sample size of [20-50 people

Click to expand...

Wrong again. You have zero idea how many people voted, zero knowledge at all and make a definitive statement. The irony is astounding.

Further, so population size IS now important for you all of a sudden? Well, talk about cherry picking

Nereida said:

that 0.1% is already included in the massive amount of data gathered by the other, so it's pointless to average it further, it's already factored in.

Click to expand...

Incorrect again. I know for a fact I've never left a review on Steam and I know for a fact that lots of other users haven't as GoG is still a preferred venue for game purchases by some here.

Nereida said:

It is a manipulation strategy commonly used by conmen and preachers in the internet, that you are familiar with when you are in the business of debunking fallacies.

Click to expand...

Yes, it is indeed obvious how YOU operate

I will just finish the analysis I started (I'm half way there, expect it later today). The data I have, while relatively small, is very detailed. I have the info on how many people voted, on how many games voted, and even who voted for which game. I'm not interested in arguing. I find the question interesting, actually, and I'm a bit of a data nerd so it's a fun exercise. I'll provide the analysis and you can decide what it means to you and if it means anything to you.

Alright, just needed to make you aware that if you take the, say, 50,000 people who left a review on steam and then the 20 people who voted in a poll about Neoverse in a Neoverse fan site, and averaged them on a 1:1 proportion, sure as hell Neoverse will come up as the best RPG of all time, so the data has no scientific value.

Even the voting rules are different in the different media that you are comparing, and they don't cross or translate in the direct way you are doing it. The only way it could have any scientific validity is if the RPGWatch/Polls were assigned an actual positive/negative value for each individual voter, and each vote was individually added to the existing poll of 50,000 votes from all other RPG players in the world that left a review, and then averaged out.

It would still be inconsistent, as some people could vote three times, one in RPGWatch, one in Codex, and one in Steam, and make those who have presence in the three sites worth 3, making their niche tastes appear more valuable than what it really is, and no legal system would validate anything of the sort. It would still get a somewhat more accurate picture of the reality, as 20 votes duplicated would still affect the result by a tiny margin, emphasizing on how meaningless is what the active voters of a site like RPGWatch/Codex think of the general quality of RPGs.

That said, it's still a beautiful exercise, and I can tell you are enjoying it, so for what is worth, I'm happy to see what you come up with, if not as evidence for anything, just to see the contrast between the general taste of the collective of RPG players, and the biased taste of a selected few.