Quote:

 

http://www.bit-tech.net/gaming/2012/02/17/review-scores/1

"The first rule laid down to any new writer looking to publish a piece of work online should be this: do not take some of the comments to heart. It's a briefing that everyone who writes regularly for the Internet has to go through.

That's no slight against commenters, rather the odd one or two who, er, take things to a more extreme level. Most comments, particularly related to video games, are of the harmless variety. Some are even revelatory; containing unknown embellishments that spark off meaningful debate among the readership. This is a very good thing.

Yet, for as wonderful and varied as article comments are, it's the negative ones which catch the eye. One or two aggressive disagreements can quickly turn a peaceful feed into a raging flame war and in the latter half of 2011, one special flavour of murderous rage began to draw particular attention.

'You have officially lost the plot this site is a joke.'

'This site is awful, its (sic) like you choose to score games low for the controversy.'

These pleasant missives appeared at the bottom of Simon Parkin's review of Uncharted 3. The Eurogamer scribe had dared to give Naughty Dog's public darling 8/10, provoking a belligerent minority into a bubbling torrent of rage. Comments emerged decrying everything from Simon's personal integrity to the inconsistency of previous reviews.

The vast majority of these expulsions appeared before the game had even gone on general release. People were angry on principle. Unwilling to believe that a sequel to a game they loved so dearly could be worse than its predecessor (those with far-reaching minds might recall the maiden issue of Gamesmaster magazine attracting similar anger for its less-than-enthusiastic review of Sonic 2).

During the high season, this phenomenon could be spotted all across the web. Any time a score skewed slightly below perfect, the floodgates opened. Reviews for Arkham City, Skyrim, and Skyward Sword were all under scrutiny from enraged onlookers.

Back in the distant haze of my childhood, perspectives were different. Personally, I can still remember my delirious excitement at learning that legendary Japanese magazine Famitsu had given upcoming Dreamcast title Shenmue 35/40. A whole five points away from perfection, but a high score from such a vaunted institution surely meant great things.

Famitsu's judging process is famously stringent. Four separate critics review each game, giving their own figure out of ten, which is then collated into an overall score out of forty. The magazine was first published in 1986, but it wasn't until 1998 that it awarded its first 40/40, to The Legend of Zelda: Ocarina of Time. In the next ten years, there were only five more perfect scores.

Then suddenly, in 2008, the wind seems to change. From 2008 onwards there have been 12 more 40/40s. A fairly small number by some standards, but a huge increase for a magazine as harsh as Famitsu.

Review aggregator Metacritic shows an even sharper change. Year on year, the number of games scoring 90 or higher was roughly the same up until to the end of the last decade. Then things start to change. In 2009, a total of 24 90+ games were released. That's an increase of six over the previous year.

A drop in the water compared to what comes next. The collected reviewers of print and web declared that a whopping 40 games were good enough to earn a score of 90 or more in 2010. That's more than double the number for every year prior to 2009. Gawp at this graph, made with my own fists, for incontrovertible evidence.

When you consider this data, along with the sheer volume of Internet bile poured at the feet of 8/10 reviews, a pretty convincing case for changing standards begins to form.

Destructoid's Jim Sterling certainly thinks so. The arch nerd believes that, 'we've dished out so many tens, that number means nothing any more.' He cites Yahoo Games' score of 6/5 for Arkham City as a testament to the saturation of big figures.

It's a neat argument, before you take a closer look at those Metacritic figures. The PlayStation 2 was released in 2000 and the Xbox in 2001. 2001, though, is actually the highest scoring year months before 2008, even though it featured the first run of PS2 releases and the initial two months of Xbox titles.

These Metacritic numbers also take into account PC games, which don't suffer from the radical jumps in technology that affect their console brethren. Dedicated PC development may not be at its peak, but there's no reason to suggest such a massive increase in scores has been matched by a sudden improvement in game quality.

Newshound Pat Garratt believes that Metacritic itself is partially to blame for inflated review scores. He argues that a "need for very high Metacritic marks has led to a culture where games that carry sub-9 scores are no longer seen as true hits.' It's a badly kept secret that big development studios reward their staff with bonuses for high ratings on Metacritic. This ethos filters down to PR representatives, who increase the pressure on reviewers to overstate their scores.

I'm inclined to agree with Pat. Most gaming outlets operate in a symbiotic relationship with publishers. Journalists rely on them for access to preview content and review discs, while publishers depend on the reliable marketing push a positive review will garner. This back-and-forth has lead to a culture in which it is considered de rigueur to award good games a nine or ten.

Journalists have not lost their consistency, but the paradigm shift in what review scores mean has devalued the scale. Pitching your figures upwards is workable when your ceiling is very high, but reviewers now have no room to manoeuvre.

There's no easy way out of this situation. Critics will have to loosen their ties with publishers. To strive to provide only coverage which truly interests their readership. Brave reviewers perhaps need to abandon numbers altogether, forcing readers to engage with the nuance of a written review.

Some, like Sterling, believe that you need to change too. "Reviews are an emotional crutch for people who clearly have no idea how to operate in the real world," he says. But his error is to imagine that the belligerent minority are more than just that. Take a longer look at Simon Parkin's Uncharted 3 review and you'll spot as many people rushing to the defence of intelligent criticism as there are lining up to attack him.

It's, ultimately, readers like your good selves that can make a difference. Most writers struggle to resist reading the comments on their article or taking them (at least a little bit) to heart. But if people genuinely want a change, it will happen, and it's constructive comments that help in making that happen. As, perhaps, will a lesser all-round reliance on review scores.

The irony of that may be that, if the number at the end of an article holds less interest, we may yet return to an era where 8/10 signifies near greatness, not near failure."

# of 90s on meta by year:
2000: 10
2001: 19
2002: 16
2003: 18
2004: 16
2005: 14
2006: 12
2007: 18
2008: 18
2009: 24
2010: 40
2011: 40

BlizShadow

Feb 17 2012, 03:28 PM

I give this article a 7/10.

Lenne

Feb 17 2012, 03:35 PM

BlizShadow

Feb 17 2012, 03:28 PM

I give this article a 7/10.

This just made my day.

BlizShadow

Feb 18 2012, 10:52 PM

Most review scores tend to come with actual written reviews and summations of the experience anyway, sooo.... xP