Everyone’s a Critic
Or at least I wish they were.
Today’s rumblings are inspired by a post made by Gav Thorpe on his blog about criticism. He’s specifically talking about criticism of his work as a writer and how he reacts to that but a lot of what he says is applicable to other fields and especially the field of community management.
In case you don’t know, Gav is a former Games Workshop games developer who is now a freelance author. While he was at GW he wrote Codex: Chaos Space Marines (an army supplement for one of the popular Warhammer 40,000 factions) which launched to mixed reactions amongst the notoriously passionate fans of Warhammer 40k. Nowadays he earns a crust by writing fiction for GW’s publishing imprint Black Library as well as for more mainstream publishers. His post on criticism is clearly a result of the huge amount of feedback readers of his blog decided to give him about the Chaos Space Marines.
So, what does this all have to do with computer games?
Well, firstly criticism is criticism. The kind of things that are useful for an author to hear about his work are also useful to a games designer. Collecting and analysing criticism is also a large part of the job of a community manager (a hat I wore for several years). Generally people are pretty bad at providing criticism for a variety of reasons, many people are also bad at receiving it for entirely different reasons. We’ll address those people later.
Giving criticism is something that a lot of people are not comfortable with. While they may have deeply held opinions, it can be hard to express those opinions without sounding hostile or rude, thus many people prefer to stay silent and keep what would otherwise be useful feedback to themselves. Not all opinions are negative of course, but the ones you hear almost always will be. This is because things that meet your expectations tend not to incite you to write about them. If things are simply ‘ok’ then we smile and move on, things have to be significantly outside of our expectation zone before we are moved to comment on them. This is usually manifested in gaming circles as a rule where, for every person posting in a 200 page threadnaught on your game forums, there are several hundred people playing the game quite happily oblivious to this apparently all consuming issue.
Another problem with criticism is that people are always right when they say what they do or don’t like but are usually almost always wrong when they try to describe it. This is because it’s easy to get hung up on symptoms without thinking through the issues to identify the actual problem causing them. A large part of being a successful community manager is listening to problems that are described by the players and trying to determine what it is that they are actually complaining about rather than what it is that they are saying.
Taking feedback can be difficult for other reasons. Gav mentions confirmation bias and that’s certainly a problem that needs to be confronted. It’s not always so much of a problem in games where a team is responsible rather than an individual but it certainly still exists. A bigger problem is enabling useful feedback at all. Most games companies run forums for fans to discuss the product, most have a community team to filter the useful nuggets from the vast seas of noise and most have some kind of feedback form or CS ticketing system for more direct contact. All of that by itself doesn’t make people want to tell you the things you need them to be saying though. Companies should be training their customers to give feedback effectively, the tools to do so should be seamless and it should be regularly solicited. If spamming customers sounds bad then incentivise it instead, reward those who tell you what they think and encourage quality over quantity. Ask people to think about your product and give you those thoughts, help them to frame them and give them the tools to do so easily.
In all the projects I’ve worked on, getting quality commentary has always been the hardest part of my job. I wish people would express their opinions more.
November 26th, 2009 at 6:40 pm
A very good post (and thanks for the links!).
The usefulness of feedback is directly connected to one’s ability to act on it. Some of it is so minor or so specific to an individual’s personal tastes that it can’t be helped. The only genuine reponse is, “Yeah, it sucks to be you, sorry about that,” although probably in a more polite way…
In asking for feedback, it should always be clear that it is not an open promise to fix everything those providing the feedback don’t like. It’s hard to say “Well, that’s just your opinion but others disagree” when openly inviting people to comment, and it can create a false sense of entitlement if someone’s opinion has not only been expressed but actively sought.
With a book, feedback is pretty much something to be filed away for the next project – anything more serious that a typo anyway. Computer games have the ability to patch, and that opens up a whole new scope of response, some of it positive, some of it negative.
On the plus side, things can be changed. On the down side, just because something can be changed doesn’t mean it should be. Between beta tests and patches, there’s often little distinction in the reviewers’ minds between an error that needs fixing and an aspect of the game they simply don’t like but is otherwise mechanically sound.
The other pitfall can come from a constant feedback loop (heh heh, see what I did there?). Changes implemented from one set of feedback can prompt further corrections required by the impact of those changes, which in turn leads to a whole different group of people dissatisfied and wanting further changes, until the whole process just descends into a constant state of flux to the point where nobody is happy with the situation. At some point, designers have to make a decision to stop tinkering and get on with their lives. Fans too have to accept that the feedback process must eventually come to an end and, like it or lump it, this is the game as it’s going to be.
Any system for providing feedback should include a caveat to keep the reviewers’ expectation of action at an appropriate level.
November 26th, 2009 at 9:34 pm
What you say is true and this really is part of what I meant when I said that companies have to take some responsibility for training their customers to give quality feedback.
It’s impossible in any case to fix absolutely everything that’s wrong because so much of it will be purely subjective. If you asked 10 people how they’d change a given thing you’ll generally get 10 contradictory and mutually exclusive answers. At this point we plot the best line possible and tell the outliers that it ‘sucks to be them’ as you put it.
It remains the case however that in general companies are bad at getting the feedback they need because they are bad at explaining to their beta testers and fans how to criticise.