Wrong is often right

Playtesting games is very much a science. The process of designing tests, conducting them and analysing the results is much closer to running a psychology experiment than simply playing games.

But just as important as choosing the right methods is selecting the right players. So who are they, and is it possible to select the wrong players?

The wrong players

Well, yes it is. A bad choice of players would include other developers, not just those working on the game, but also working elsewhere in the studio. Other bad choices would extend to family and friends of the developers.

Such players would be a potential waste of time as they have too much attachment to the developer. They may also be biased from being privy to information that 'real' users wouldn't. It's also possible that the developer may influence the player's experience unconsciously, unaware that they are biasing the outcome by giving hints or saying something they shouldn't.

We talked to one developer who said that they don't conduct playtests as such, but instead they watch people play their game at trade shows. By only watching those who go to such events, you're only ever going to see how a very small set of niche gamers are experiencing your game.

If you absolutely must test with people that you know, then focus on observing their behaviour. This will give you as objective a viewpoint as possible, but it does not capture the more interesting data of how they're feeling or why they played the game a particular way. However, asking people you know what they think of your game is almost certainly going to lead to less reliable results than asking a stranger.

The right players

So who should we be testing? Well, this is a bit more difficult to answer as it'll depend on the game being playtested. But a reasonable starting point would be to select players who would potentially buy the game. After all, although there could be value is selecting absolutely anyone, the best use of your resources is to better understand the kind of player the game is aimed at.

This is where things become tricky. Who is a potential buyer? It's most likely that it's those who have bought similar titles, or those who are fans of the brand, but of course in reality it's not quite as straightforward as that. It's also not just a matter of selecting those who are good at games - it's entirely possible that significant numbers of people who buy a game are actually quite bad at it.

And it's here that an interesting playtesting phenomena occurs. Sometimes, it's not that the wrong players have been selected for playtests, but some developers have the wrong perception of the wrong player.

Game and watch

I can sympathise. It must be nerve-wracking for developers to watch playtests. After spending a significant period of time developing a game, it finally placed in the hands of its potential buying public.

There are many ways playtests can turn out, but let's discuss the two main paths. Path one is where the game is enjoyed by the player just as the designer intended, so everyone is happy. The second path is where the player doesn't experience the game as intended. The question is, why? One reason, obviously, could be that the game has issues which need to be addressed, but another could be that the player is not an ideal match for the game.

And it's at this point where a pause needs to be taken. It's always going to be tempting for a developer to blame the player rather than the game, but let's consider any assumptions that may have been made. If you expect the player to know as much about games as you, well they probably won't. Even self-described core players who spend many hours a week playing games may still not know the names of quite basic gaming terminology.

For example, even though some players have owned a PlayStation 3 for several years, they may not know that the user interface is called the XMB, or XrossMediaBar. Also, many iOS users are unaware that they can quit or switch between running applications by double tapping the home button. Such lack of core knowledge in playtesters can lead some developers to think that the wrong players have been chosen, but the fact is that many players just don't care about such things.

Then there's the perceived lack of skill of the playtester. If they're not able to play the game as expected, then they can be considered 'not good enough'. But remember that these players have usually volunteered to playtest this very genre of game, and have bought games similar to it in the past. Considering the broad market, they are probably better at games than most. The abilities of the average gamer may shock some developers - and the way they play will, too.

After all, does it matter that some players lack gaming knowledge and skill, or does it matter more that they are experiencing enjoyment in their own way and have the motivation and intention to purchase?

Developers want the best possible information during playtests to help inform and craft great games, but they sometimes perceive the player they need to watch as the wrong one.