Steam - User Reviews Revisited

Henriquejr spotted the following Steam news:

User Reviews Revisited

Some time ago we made some changes to how we presented the User Reviews for games, and their resulting Review Score. We talked about those changes in this blog post. As we describe in that post, we want to ensure that players who've played a game can voice their opinions about why other people should or shouldn't buy the game, and that our summary of those opinions into a single Review Score should represent the likelihood that a future purchaser will be happy with their purchase.

Since that post, we've continued to listen to feedback from both players and developers. It's clear to us that players value reviews highly, and want us to ensure they're accurate and trustworthy. Developers understand that they're valuable to players, but want to feel like they're being treated fairly. We've also spent a bunch of time building analysis tools to help us better understand what's happening in the reviews across all titles on Steam. With that feedback and data in hand, we think we're ready to make another change.

That change can be described easily: we're going to identify off-topic review bombs, and remove them from the Review Score.

But while easy to say, it raises a bunch of questions, so let's dig into the details. First, what do we mean by an off-topic review bomb? As we defined back in our original post, a review bomb is where players post a large number of reviews in a short period of time, aimed at lowering the Review Score of a game. We define an off-topic review bomb as one where the focus of those reviews is on a topic that we consider unrelated to the likelihood that future purchasers will be happy if they buy the game, and hence not something that should be added to the Review Score.

Obviously, there's a grey area here, because there's a wide range of things that players care about. So how will we identify these off-topic review bombs? The first step is a tool we've built that identifies any anomalous review activity on all games on Steam in as close to real-time as possible. It doesn't know why a given game is receiving anomalous review activity, and it doesn't even try to figure that out. Instead, it notifies a team of people at Valve, who'll then go and investigate. We've already run our tool across the entire history of reviews on Steam, identifying many reasons why games have seen periods of anomalous review activity, and off-topic review bombs appear to only be a small number of them.

Once our team has identified that the anomalous activity is an off-topic review bomb, we'll mark the time period it encompasses and notify the developer. The reviews within that time period will then be removed from the Review Score calculation. As before, the reviews themselves are left untouched - if you want to dig into them to see if they're relevant to you, you'll still be able to do so. To help you do that, we've made it clear when you're looking at a store page where we've removed some reviews by default, and we've further improved the UI around anomalous review periods.

[…]

More information.

Wow, horrible! Somebody better tell TheQuartering about this.

There's a slippery slope, Steam.

Originally Posted by Capt. Huggy Face
There's a slippery slope, Steam.

Thought the same thing. Also wondering if Steam will do the same with off-topic positive reviews.

Yeah, I'm sticking with consulting with this site or friends before I make my purchases. Doesn't everyone do that, though? I mean maybe not this specific site but ask friends or review products elsewhere? It just occurs to me from the outset that there could be a huge conflict of interest involved were you to evaluate your purchases at a site that's hosting the store.

how about introducing a neutral review option

With Steam reviews, I don't pay much attention to the numbers, I just scroll through the reviews to find a couple that appear to have been written by functioning humans.

It's a tricky thing. On one hand, I see why this could be useful. Lately, when reading reviews of certain games on steam, I do see some "review bombing" going on. These are reviews such as complaining about a DRM often, or even just about a particularly nasty bug that the game might be temporarily experiencing. Those reviews are most often not very helpful. Bugs can and are fixed, and DRM is something that is routine on most games.

On the other hand, there are definitely pitfalls about defining what a "review bomb" may consist of, and if steam will decide wrongly if a valid complaint being echoed by many players is a review bomb, which could artificially inflate game scores in general across all of steam.

Luckily I note that you can opt out of this according to steam's press release, and continue to see review scores as normal. Let's hope they don't remove this later down the line.

Am I understanding it wrong or Is this a way of stopping a game from being downvoted for asinine reasons taht don't affect the gameplay such as "it has Denuvo", "The CEO made a snarky comment in his twitter" or "it was made by EA"?

Originally Posted by Nereida
Am I understanding it wrong or Is this a way of stopping a game from being downvoted for asinine reasons taht don't affect the gameplay such as "it has Denuvo", "The CEO made a snarky comment in his twitter" or "it was made by EA"?

That's the idea, but I think the concern is about who gets to decide what's asinine.

Count me as another who thinks this a grey slippery slope also. I get why they are doing this as some games are review bombed over various reasons like the Metro games.

At least you can still read them as the text above confirms.

As before, the reviews themselves are left untouched - if you want to dig into them to see if they're relevant to you, you'll still be able to do so. To help you do that, we've made it clear when you're looking at a store page where we've removed some reviews by default, and we've further improved the UI around anomalous review periods.

Originally Posted by Arkadia7
Luckily I note that you can opt out of this according to steam's press release, and continue to see review scores as normal. Let's hope they don't remove this later down the line.

Steam should just have a Community Concerns section for outstanding issues/feedback from the community. It won't effect review scores and could be prominently displayed so that it can't be ignored by the developer. But the option to ignore it should be given to Steam gamers.

As before, the reviews themselves are left untouched - if you want to dig into them to see if they're relevant to you, you'll still be able to do so.
….
Finally, we've also enabled you to opt out of this entirely, if that's your preference - there's now a checkbox in your Steam Store options where you can choose to have off-topic review bombs still included in all the Review Scores you see.

What to say? Ok then.

Not that I don't see the manipulation, most users will never bother change the default review setup, so even optional it will be effective for 99.99% of users.

Ok but the choice is here to ignore or not the review censorship effect on reviews rate.

For me it's a much bigger concern that keys not bought on steam aren't count, but clearly it will not change, nor it will ever make me buy a game on Steam because of that.

If Steam actually had people do this, it would probably be a good idea, but let’s be realistic, it’s just going to be a bunch of bots doing the work, so I’m expecting it to blow up in their face.

regulation against poor review is great. especially in IT where u can get an AI to post a shit ton of them.
i think its a step forward. the only risk i see is saving some titles that are decent games but the culture has solid reason for rejecting .. see captain marvel. from being shunned as they should.
art is about context as well. and the culture matters. this was a way of protest.. now there will those people who wont join in the conflict when/where it starts. causing damage inadvertent to the cause

I find that the overall score can be misleading so I adopt a search pattern which involves looking at the reviews with poor scores as well as those with high scores.

Recently I bought Victor Vran on GOG. Overall it rates at 3.9 out of 5 so a decent enough score. Digging a bit deeper it seems that a lot of low scores are about some DRM thing when you go online. I never play online. So I don't care. Bought the game and despite the inability to save when I want to it is an enjoyable game.

With regards to Steam misusing this new facility. Bots are used to find them, people are used to decide what stays what does not. Yes - it could be open to abuse but so much in life is.

I think, as with anything involving money, buyer beware.

yea a lot of games run into this problem in that the online services use the services of the publisher/developer. People have to accept that as the nature of online play. I suppose the exception might be those games that allow lan online.

Originally Posted by SleepingDog
I find that the overall score can be misleading so I adopt a search pattern which involves looking at the reviews with poor scores as well as those with high scores.

Recently I bought Victor Vran on GOG. Overall it rates at 3.9 out of 5 so a decent enough score. Digging a bit deeper it seems that a lot of low scores are about some DRM thing when you go online. I never play online. So I don't care. Bought the game and despite the inability to save when I want to it is an enjoyable game.

With regards to Steam misusing this new facility. Bots are used to find them, people are used to decide what stays what does not. Yes - it could be open to abuse but so much in life is.

I think, as with anything involving money, buyer beware.

Seems logical.

https://www.youtube.com/watch?v=vqy-OafTE1E