BLOG: The one about waves of play-testers

I have this provocative statement that designer should never listen to playtesters and that playtesters have no idea what they are talkin’ about. Every designer panel I run, I always say: ‘Ignore them. Just don’t listen. They have no clue.’

It’s provocation statement of course, but there is a seed of truth in it. Today I’d like to show you the real life example when I totally ignored my playtesters.

***

Last months I spent developing Alien Artifacts. The rules were changing every day. It was quite a ride. Those of you who watch Portal Games vlog saw it every week – a new version of the game, new series of test, changes over and over again. That was real work, real search for better mechanisms, for better flow, and smoother experience.

At some point we had it. The game worked nice. Playtesters liked it a lot. I remember Asia (one of our playtesters) asking me for a p’n’p files because she wanted to play it at home. That was a good sign. That was a sign I was waiting for for months. It looked like we finally had a winner. We had a version we could start balancing.

And then, few weeks later once again I changed the game.

OMG. You should hear my play-testers. I killed the game. I broke it. I made it a disaster. I made a mistake. This new version is a bullshit. They were whining like babies.

I didn’t listen. I asked them to play. They were complaining. Trying to prove me their point. I didn’t listen. I just watched. The game was smooth. The game was fluent. It was just a noise of disappointed play-testers that made the whole experience less fun that I’d wish for. But I knew what I wanted. We kept balancing the game. Ignoring the noise of complaining playtesters was my daily routine.

‘We are balancing the game. Shut up and play.’

‘But the new version is much worse. These changes are a huge mistake, Ignacy.’ they said.

‘We are balancing the game. Shut up and play.”

***

I ignored the feedback. After a long weeks of that ridiculous playtesting sessions, I packed the game and went to The Dice Tower con. I run about 20 games of Alien Artifacts per day, five days in a row. New players. New feedback. And guess what.

No single person complained, no single person even mentioned these two rules. Almost everybody praised the game, people loved it, people were coming back with their friends to play it again. The game was smooth and fluent.

I was right. My playtesters were wrong.

So what exactly happened?

***

We like the things we know. We get used to things. We hate changes. We can listen to Smoke on the water thousands of times in our life and we are happy. We can watch Die Hard over and over again and we are happy. We do really feel home sweet home even though we spent a week in the best resort ever.

We like the things we know.

It is crucial for us, designers to have different teams of play-testers. I’d call them waves. At some point, when one of the groups loose fresh perspective, when they got too familiar and too attached to the version, you must thank them for their work and move on to the new group. The old group already likes the game as it is. They will resist every change. They will send you the wrong message. They are – I am sorry to say that – no longer value for you.

You must move to the other group.

With Alien Artifacts I lacked enough groups. I got stuck with not enough playtesters. I got stuck with a group that was resisting changes.

Luckily I knew when to stop to listen to them.

***

Original text here: https://boardgamegeek.com/blogpost/67524/one-about-waves-play-testers