Steam Reviews Study - Its the Game Design not the Bugs

Gamasutra reports on a study into Steam reviews which examined negative and positive reviews.

Conducted by researchers at Queen's University in Canada, the academic study delves into several factors that come into play when a review is written on Steam, such as examining hours logged before the review was posted, whether the game was indie or not, and complaints of performance issues.

A particularly interesting conclusion the researchers came to was that players complained more about game design as opposed to bugs. As the table below shows, only 8 percent of all the reviews examined mentioned bugs in games, while 34 percent of the reviews mentioned the cons related to game design.

[…]

More information.

I usually don't think poorly of a game due to its bugs, unless the bugs are frequent and/or very impactful such as crash to desktop during an extended section of the game when saving is not allowed.

Often, bugs get fixed in the weeks or months after release anyway, and I usually just wait for that such as I did with Assassin's Creed Unity (which ended up being a mediocre game) and Batman Arkham Knight (which ended up being a great game), both of which were apparently extremely buggy at launch. But even if some bugs remain I don't usually mind much.

Gamers handed publishers and developers a “bug” hall pass decades ago. It’s frustrating for individuals to have bugs affect their gameplay with a game they would otherwise enjoy or still do enjoy despite the bugs. But “team gamer” still buys by the billions bug infested software every year despite the quiet desperation of those vocal few - hence the hall pass. Personally if I catch wind a game is suffering from a ridiculous level of bugs l keep away no matter how interested I might otherwise be for such game. Put off Skyrim for almost two years. Have been holding off with kingdom come deliverance. Skyrim to this day still has tons of bugs but by the time I bought it the big game breaking ones had been fixed and for the most part I never noticed any bugs in my hundreds of hours with the game.

Originally Posted by TheMadGamer
Gamers handed publishers and developers a “bug” hall pass decades ago. It’s frustrating for individuals to have bugs affect their gameplay with a game they would otherwise enjoy or still do enjoy despite the bugs. But “team gamer” still buys by the billions bug infested software every year despite the quiet desperation of those vocal few - hence the hall pass.

Agree as most games are patched like crazy, and don't forget the huge day one patch. I used to help a site that made game trainers, and nowadays it's just to much work.

It's worse when the reviews are based on politics, or even pricing like "This game is good but doesn't deserve a $50 price tag", so a few months later when the game is on sale in Steam for $10, the review is still negative.

I think the reason for this is bugs can be fixed, but bad game design normally can't. Gamers will buy a game that is buggy if it looks like it has potential to be a very enjoyable game, which in turn finances bug fixing. Normally they won't buy something with bad game design, either because the developers are in denial about their vision being bad game design, or its impossible or unfeasible to repair.

But what is bad game design?
For all I know every product has it's audience and fans that adore certain design.
Me? I usually call out outdated designs. But even those generate tons of followers for some reason.

What about UI and controls. Seems that gamasutra didn't care about that or I missed something. Search through PDF has no results with "control", "UI" or "interface".
And IMO that's more important than subjective impressions on design - while I hate platformers and racing, others adore them, although with different taste we can still agree if PC controls are horrible, good or superb.

Originally Posted by joxer
But what is bad game design?
For all I know every product has it's audience and fans that adore certain design.
Me? I usually call out outdated designs. But even those generate tons of followers for some reason.

What about UI and controls. Seems that gamasutra didn't care about that or I missed something. Search through PDF has no results with "control", "UI" or "interface".
And IMO that's more important than subjective impressions on design - while I hate platformers and racing, others adore them, although with different taste we can still agree if PC controls are horrible, good or superb.

Not really. You yourself say that if a game doesn't open journal with 'J' it's unplayable, while for me that's the nitpickiest of the nitpicks

Aren't there key remapping software you can use to force games to behave? I know quite a few JRPGs are quite bad with the control schemes and some have no remapping function built-in.

Originally Posted by TheMadGamer
Gamers handed publishers and developers a “bug” hall pass decades ago.

This was made worse by the crowdfunding movement as releasing buggy products happens on purpose.
Buggy products are better as they allow feedback and participation as wished by the crowdfunding support.

Originally Posted by joxer
But what is bad game design?

Anything players do not like as RTwP instead UgoIgo.

Design underlies purpose and players will reject any purpose they can not bear.

Originally Posted by wolfing
Not really. You yourself say that if a game doesn't open journal with 'J' it's unplayable, while for me that's the nitpickiest of the nitpicks

I said if a game emulates mushrooms with ijkl where PC "standard" is i for inventory and j for journal, then it's the case of get off my property. Go emulate mushrooms with arrow keys or something else for all I care, just don't touch generally accepted keymappings.
In fact scratch all of this - mushrooms being emulated with any keys idiocy needs to die, we have the bloody mouse.

Originally Posted by ChienAboyeur
Design underlies purpose and players will reject any purpose they can not bear.

Have to disagree. It's the purpose that underlies design. Streaming though undermines them both. Players reject streaming.

Originally Posted by forgottenlor
I think the reason for this is bugs can be fixed…

What you wrote is sort of the "buy in" that gamers made decades ago that has normalized what is acceptable bugginess these days. I have a pretty huge collection of video game cartridges dating all the way back to the original Magnavox Odyssey. All those decades worth of game cartridges I can't remember a single one that had a game breaking bug or even a minor bug… until around the time of PS2/Xbox 360, well into the years the hall pass was given.

I'm not one to expect reasonably bug-free games anymore since I've accepted that ship sailed long ago. So I just hold off with my purchases and take a wait and see approach. I would love to play Kingdom Come Deliverance… that game is sort of my thing having read reviews and watching some videos. But that game is still in no condition for me personally to put up with… so maybe in a year or two if the developer keeps fixing it I'll buy it.

Originally Posted by joxer
But what is bad game design?
For all I know every product has it's audience and fans that adore certain design.
Me? I usually call out outdated designs. But even those generate tons of followers for some reason.

What about UI and controls. Seems that gamasutra didn't care about that or I missed something. Search through PDF has no results with "control", "UI" or "interface".
And IMO that's more important than subjective impressions on design - while I hate platformers and racing, others adore them, although with different taste we can still agree if PC controls are horrible, good or superb.

Features can be broken up into a variety of categories I would think.
Lets take combat, for example.
It can take too long.
It can be overly repetitive.
It can offer no sense of danger.
It can be too arbitrary
It can be too simple.

Any of these can be to a certain degree subjective. For example, randomness doesn't necessarily bother me within a certain extent, while for others any randomness is a killer. Still something like the earlier Final Fantasy or Wizardry has short but deadly combat. Dragging out combat is almost always disadvantageous, while killing an enemy ASAP is usually a good idea, though you do have to see you don't burn through limited resources unless its necessary. In these games combat feels nicely paced. In many other RPGs I've played that's not the case and combat quickly becomes boring. Its hard to necessarily put a finger on, but there is a reason many gamers come to a consensus that a game is good or not.

Originally Posted by TheMadGamer
What you wrote is sort of the "buy in" that gamers made decades ago that has normalized what is acceptable bugginess these days. I have a pretty huge collection of video game cartridges dating all the way back to the original Magnavox Odyssey. All those decades worth of game cartridges I can't remember a single one that had a game breaking bug or even a minor bug… until around the time of PS2/Xbox 360, well into the years the hall pass was given.

I'm not one to expect reasonably bug-free games anymore since I've accepted that ship sailed long ago. So I just hold off with my purchases and take a wait and see approach. I would love to play Kingdom Come Deliverance… that game is sort of my thing having read reviews and watching some videos. But that game is still in no condition for me personally to put up with… so maybe in a year or two if the developer keeps fixing it I'll buy it.

I completely agree with you. I've been playing computer game for over 30 years, starting in the mid 80s on my Commodore 64. The first game I encountered with a bug was Joe Montana Football. I wrote the publisher a complaint letter and three weeks later got a patch disc in the mail. However, today bug fixing can occur much more quickly, as communication and patching are almost instant. Like you, I almost never play a program when it is released. I did play Siege of Dragonspear on the release day, since I promised Myrthos I'd write a review. I also played Avadon:Warborn before release, since I got a key from the developer, and did not encounter a bug in either. But those are exceptions. I prefer to wait anywhere between 3-9 months after release, depending on the reported bugginess of the product.

Originally Posted by joxer
But what is bad game design?
For all I know every product has it's audience and fans that adore certain design.
Me? I usually call out outdated designs. But even those generate tons of followers for some reason.

What about UI and controls. Seems that gamasutra didn't care about that or I missed something. Search through PDF has no results with "control", "UI" or "interface".
And IMO that's more important than subjective impressions on design - while I hate platformers and racing, others adore them, although with different taste we can still agree if PC controls are horrible, good or superb.

I think about this a lot, too. "Good" game design can be labeled as "good" by the people who already have an established affection for whatever game they're playing. If you've been playing cRPGs for many years, you'll likely consider party based cRPGs good game design. But someone who mostly plays shooters might not see that as good game design.

I agree with the idea that there are outdated designs to a point. In the case of Square Enix, their latest Final Fantasy entries have moved away from turn based and more toward action/real time battle systems. And yet, they question why their games don't sell as well as they used to. They've built themselves on an established formula, but consider that formula outdated. This doesn't make sense to me. There's clearly an audience for it - their main audience.

In the case of video games, I don't relate "outdated" with "old". But I do relate "outdated" with "obsolete".

To me, outdated designs are formulas that are superseded by later iterations which are more intuitive. If game controllers used to have start buttons on the backside and were replaced with controllers that had the start button on the front where it's clearly visible, that's simply better design because everyone can see it. Anyone who plays games (or will be playing games) uses their eyes. I would say the original controllers were obsolete.

If controllers had the start button in the middle, and newer controllers moved that start button to the upper right corner, I wouldn't say the old one is outdated. Some people have big hands or are left handed, that doesn't make the older model obsolete, just old.

Originally Posted by Silver
Aren't there key remapping software you can use to force games to behave? I know quite a few JRPGs are quite bad with the control schemes and some have no remapping function built-in.

This is a big reason why I use the Steam Controller with almost all games now. You can easily re-map it to do exactly what you want it to do, regardless of the game (some really ancient games may need an extra step or two to get the controller functioning properly, but I haven't found a game yet I couldn't use it with.) It's a really nice feature.

For the topic, I'm actually glad people are more interested in the game design than bugs personally. I usually never have a problem with bugs, maybe I'm lucky, but usually the minor stuff is just that, minor, and doesn't bother me. A gamebreaking bug or crashing bug is a big deal, but even those are scarce when I game nowadays. Even Skyrim, which pioneered the Bug Memes, I had only relatively minor graphical glitches here or there, and nothing that bothered me much (worst thing was the duplicating guy outside of Whiterun. I didn't even get to see a backwards flying dragon. )

So bugginess in games for me has largely been a non-issue. But I do not like when I hear a certain stat, important status effect or other RPG feature doesn't work in the game.

Originally Posted by forgottenlor
I think the reason for this is bugs can be fixed, but bad game design normally can't. Gamers will buy a game that is buggy if it looks like it has potential to be a very enjoyable game, which in turn finances bug fixing. Normally they won't buy something with bad game design, either because the developers are in denial about their vision being bad game design, or its impossible or unfeasible to repair.

So much this. Some developers just can't seem to grasp that. If you look at civilization 6 you would regard it as having some fundamental flaws with its AI but the developers Firaxis are building on top of those flaws new features which is deepening the problem with the game design by exposing the AI more.