Data-driven analysis of review scores against sales

[Ivo]

This editorial may be of interest, in particular to those interested in psychology e.g. JT and maybe Luke)

http://penny-arcade.com/report/editoria ... -we-think1

Ivo.

[Erik_Twice]

I'm no statistician but the analysis seems to stand on very shaky grounds specially when bias is obviously systematic, you don't need to bribe every single journalist when you can simply bribe IGN. Whoever thins that corruption isn't systematic is simply wrong, and even small releases of old games (Simpsons arcade, Sonic CD, Radiargy, etc) are panned just to avoid incongruences. (Because if you say a retro game is good despite the graphics, perhaps they aren't as important...mhm...)

The fact that 70/100 is considered a bad score tells more about the systematic bias of the game industry than anything else. Let's nor forget that people look for reviews of games they are already interested in.*


*You might have noticed that practically everything I have written is a reaction against those two things.

[Ivo]

What I like in the article is just the data, the interpretation is up to each person.
The data is just "better scores have correlated with better sales". That is it. That does not involve the quality of the games, and already automatically incorporates any systematic bias of the reviews, it is just a cross-check of two well-defined numbers without concern whether one of those numbers correlates more with the quality of the game or with its marketing budget or whatever.

In terms of opinion though, I think I do agree with Kuchera - there are limits to what you can do with stuff when you are talking about aggregate scores.

They do discuss bias to some extent, when they go about the anchoring effect (the aspect of looking for reviews to confirm your own opinion could be a bit similar to that I think).

There is a follow-up where they discuss the Wii games in the first year having sales all over regardless of review score, which they attribute to the "new gamers" not using reviews at all. That is relatively convincing as far as I'm concerned that I think they know what they are doing with the data.

Disclaimer: I think the most recent game I bought is something from one of the Humble Indie Bundles. I still think companies are making good games, some of which I would like to play in the not-so-near future.

I'm no statistician but the analysis seems to stand on very shaky grounds specially when bias is obviously systematic, you don't need to bribe every single journalist when you can simply bribe IGN. Whoever thins that corruption isn't systematic is simply wrong (...)

[Zing]

Anyone interested in this sort of thing should read the book Thinking Fast and Slow. It talks about the anchoring effect as well as the multitude of other ways our decision-making is affected without us even knowing.