General concensus: "the average mark"

[Lewis Denby]

I'm currently writing a bit of a journalism-journalism article about people's gravitation towards "average" marks. Now, of course, sites like Metacritic, GameRankings and N4G are feeding this something rotten, but the specific problem I'm looking at seems to be exclusive to games, and something we press-folk are feeding equally. Why, when film and music critics assign a wide variety of scores to an individual product based on greatly conflicting opinions, do scores (and, of course, the text) tend to sit neatly around a specific scoring band? If a film's Metacritic average is 90, it almost exclusively also has a few dissenters who absolutely detested it for a perfectly valid reason. With games, you can pretty much guarantee that most publications awarded it around about that score. Why on earth is this?

The result, of course, is that whenever a magazine does stray away from the expected mark, there's an abundance of conspiracy theories flying around concerning the reason. Unusually high marks garner theories of bribes; low scores develop into accusations of bias towards a different console manufacturer. Rarely does anyone stop to think about the fact that, maybe, that was the writer's honest opinion of a game.

Why are we all so often in agreement? Surely that's the main problem. We deviate from the pack so rarely that, when we do, it's confusing people.

Is it because there's something quantifiable about games that doesn't exist in film, music or literature? If a game "performs to a reasonable standard," it usually garners a reasonable mark for that alone. Is that adequate reason? Is our assumption that games are split into identifiable components, and the game's quality is no more than the sum of its parts, the reason for this general concensus? And is that a fair assumption to make, or not?

Very interested to hear people's thoughts, and hopefully nick a few quotes from the article if possible.

[Jonathan Gronli]

The reason there is usually an average score around there is two-fold. One fold is if the game is genuinely good. If it is genuinely good, the people who didn't like it are usually grading based off of what worked for their favorite games (regardless of genre or developer). After all entertainment at large hasn't been exactly original since the times of the Greeks, and even then there were artists of various kinds stealing ideas or concepts from other artists. The audience or critic sees something they liked in another form of entertainment that they feel was misapplied and then look for other problems like that. The other fold is if it's a genuinely bad game. If it's a bad game that gets high ratings, there are quite a few reasons for it. One of which is the conspiracy theory of bribes that you brought up. Ultimately though it boils down to little more than "There's no accounting for taste."

[Jonathan Gronli]

Forgot one thing that I just remembered, since you asked if there's something about this average of games that doesn't apply to other mediums of entertainment. No, the same principles apply to the other forms of entertainment whether it be art, film, literature, etc. The only reason that the averages seem more erratic is that some people tend to specialize in a certain type whereas gamers usually are more willing to wander outside of their areas of expertise. Film, if you love horror, you're more likely to rate a horror film well than a non-horror film. Yes this does mean that you run the risk of giving a good non-horror film a lower grade than a bad horror film.

Another thing that goes into it is the fact that it is easier to teach what's good in terms of film, literature or art but without specialized training in programming you can only guess if things that go wrong were purposefully programmed that way, like Call of Cthulhu: Dark Corners of the Earth or Eternal Darkness: Sanity's Requiem. Sometimes mistakes make it through and sometimes mistakes aren't even mistakes so how can you accurately judge. The average still applies though but since more people know what goes into a good book, movie or art exhibit than they do what goes into a good game the average tends to be range from 75-80 for some of the better work.

[Andrew Groen]

A) In order to keep themselves out of hot water with publishers, as well as to keep them listed with Metacritic, most publications require that a game is treated objectively as a piece of software. Game reviewers are generally only too happy to oblige because this is by far and away, the easiest way to judge a game (Graphics good. Sound mediocre but does the job without being annoying. Not as good as GTAIV.) The debate is out on whether or not this is an appropriate way to judge a game. If we were judging art or even entertainment products you'd start to see more of the varying opinions. If you want an example of this go look up the metacritic rankings for games like Okami and Braid. You'll find your dissenters.

B) People are scared. They don't trust their own opinions enough, and they're afraid that if they stray too far from the precedent set by GameSpot then they'll look stupid or they'll look as if they didn't actually play the product. Unfortunately, when you're a smaller outlet a rapid way to destroy your credibility is to deviate from the review range in the eyes of your readers. The problem isn't just us or the publications, its the readers who are weaned on this system and who now assume that every game should have a consensus. If 100 outlets are the same and then 1 small place on the outskirts says "no, you're all wrong" they just end up looking ignorant.

You're right though. I think it's positively embarrassing that the enthusiast press isn't the place where we can go for great, thoughtful reviews. Instead we have to let the New York Times continuously out pace us in our understanding of gaming.

[Jared Newman]

I'm not sure I buy the idea that movie reviews are all over the map while game reviews are almost completely homogeneous. I've seen movies get uniformly positive reviews (pick an Oscar winner) and games get mixed reviews (No More Heroes being a good example), and there's always dissenting opinion around.

Obviously, it's not an ideal situation. There are definitely publications that strictly adhere to consumer reports, as Andrew mentions, but I've seen others that write critiques. And there are definitely fanboys that lose it when their game of choice doesn't get the score they were hoping for, but that has less to do with the journalist than with the reader. When was the last time you saw a writer soften a score because he or she was afraid of how the readers react (we all know it's the publisher's reaction that matters, har har, wink wink).

This is a tangent, but I make a point not to read other reviews or scores of a game that I'm playing. It's impossible to avoid the pre-release buzz, but its the best way to keep an open mind, no matter how tempting it is to peek at Metacritic.

[Brendon Lindsey]

Having worked in both game and movie journalism, I can tell you right now the disparity in reviews is probably equal in both. The difference? Movie journalism has, outside of Ebert, DOZENS of big names and publications that are "authorities." 14 of them like it and 3 of them hate it? No big deal, it happens.

Gaming on the other hand only has a handful of sites the average reader would consider an authority. If 4 of them give a game a 90, and 1 gives it a 70 (which we KNOW isn't bad, but that's an entirely different debate all together) that 1 going against the flow is going to stand out and be bashed because it's more obviously a delinquent score.

Plus you could argue (and I often do) that in general, professional movie reviews are much better written than gaming reviews. Movie journalism has had decades to finetune what a movie review is, and what it should constitute. Gaming is still trying to figure out if they should review based on what they like, how fun it is, if it's worth the $60, compared to other games of dissimilar genres, etc.

[Jared Newman]

eek, when i say "game that I'm playing" i meant "game that I'm reviewing."