Games, Design & Game Design

Playtesting & The Designer as Expert

In Commentary, Game Design, My Thoughts On on February 18, 2011 at 11:32 am

Ah! A Kerfluffle! Ben Lehman, who I count as a friend and respect mightily as a game designer, has posted a bit of a screed against playtesting over at anyway. I’m not going to go through his argument, you should read his post if you haven’t already (you being “independent amateur game designer” cuz that’s who he’s talking to and for.) I am interested in some issues related to what he brings up, though.

I’m not sure where this first came up (and I may have mashed it up from a number of sources), but there’s a phrase that I keep in mind going through the  design process that goes like this:

Talk to playtesters to find out problems, but ignore their solutions. Talk to other designers to work out solutions, but ignore their problems.

Playtesting is really good at revealing problems but terrible at solving them. If playtesters could solve the problem themselves, they wouldn’t need you to design the game in the first place. That’s because you, Designer, are the expert in this situation – you have specialized knowledge, training and experience in this field that gives you superior insight into the situation. It’s like user testing in product design – if your widget doesn’t work, real-world users will reveal that really quickly. But if you ask them how to fix it, you’ll get as many responses as there are users, because everyone has opinions about things and specific contexts for use and their own preferences, and your job as a designer is to take into account their experience without just averaging their feedback and calling it good.

Other designers are, of course, fellow experts. They have a critical framework within which to place your design and evaluate it. They are problem-solvers in your field, and know how to judge your goal against the expressions of that idea in an evaluative fashion. That’s why they are really helpful to solving the problems you run into (whether stemming from a playtest or any other part of your process). However, because they are creative individuals who think about this kind of stuff a lot, and have a million ideas that haven’t had a chance to see the light yet, they will find opportunities in your design for their ideas if you let them. I see this all the time in group critiques at the early stages of design work – if you say “here’s my basic idea, what do you think,” every designer there will take your idea off in a different direction, dress it in different clothes and make it do a different dance. This is great for brainstorming, but really unhelpful once you have a set goal and you’re trying to solve a particular problem.

Now, playtesting with a group of fellow designers can be the best of both worlds or the worst, depending on your relationship with them, their experience with your creative process and a host of other factors. I’ve had it go both ways; I suspect, though, that a lot of the Ben-you-talking-crazy responses are from people who have healthy playtest groups made of other designers.

Playtesting is critical for a game because games are all about the human element. In short, your game can be as mathematically sound and rigorously written as you like, but only playtesting will tell you if it’s any fun. But that’s also pretty much the only thing that it can tell you better than any other method, and uncritically accepting all your playtesting feedback is unhelpful at best and actively detrimental to your game at worst.

  1. “uncritically accepting all your playtesting feedback is unhelpful at best and actively detrimental to your game at worst.”

    Sure. Yes. Was *anybody* arguing *for* uncritical acceptance of all playtesting feedback, though?

    I certainly don’t think Mr. Lehman’s argument or attitude in that post is helping him make this otherwise unremarkable opinion—bad playtesting is bad—more accessible to people who might most need to hear it.

  2. Will, I’m not sure what to make of your response. Are you reducing Nathan’s post to one sentence, or responding to Ben’s original post (and the follow-up arguments)?

    I think the real crux of what Nathan is getting at has to do with how playtesting can lead to muddled design goals, game texts, and mechanics. Many designers who playtest (especially with other designers) lose touch with their core design goals (I know because I’ve done this myself only to have to go back and start over). Playtesting is therefore dangerous because if you don’t have a strong sense of what you want out of your own game, the game can lose focus and cohesion. In these cases, the designer might be tempted to say something like, “I’ve playtested so much, it must be good!”, when in fact all of the undirected playtesting has only led the designer astray.

    Playtesting is much more difficult to do correctly as a designer than many people seem to think. And done incorrectly (i.e., without a strong sense of direction), playtesting is often counterproductive.

  3. […] This post was mentioned on Twitter by Nathan D. Paoletta, Tim Koppang. Tim Koppang said: RT @ndpaoletta: Playtesting & The Designer as Expert […]

  4. I’m not reducing the post to its closing argument, I’m commenting on that closing argument. If I have to weigh in on everything to weigh on something, I’ll also say that I think you make a better case, Nathan and Tim, for the dangers of playtesting without strong authorial vision than Mr. Lehman does. I have issue with what looks like a strawman in the summary passage at the end.

    If we’re talking about the dangers of playtesting, though, I’ll add this: ignoring playtester’s solutions is equally dangerous. Good solutions come from many places and ignoring a good idea because it came from a playtester is hubris. Yes, a great deal of playtester suggestions are not the winning solutions to revealed design problem, but a critical eye means recognizing a good idea wherever it comes from. A fix from outside the designer’s own head or gut may be less satisfying for the designer, it may even be humbling, but an idea isn’t valueless because it came from outside.

    Yes, absolutely, strong designer vision should not be a slave to the averaged value of the testing audience. Yes, it can be frightening, as a designer, to feel like you’re not essential to the process when you see a winning idea come out of a tester or fellow designer. Still, I have to disagree with this: “If playtesters could solve the problem themselves, they wouldn’t need you to design the game in the first place.” Leading a design—sparking a design—and fixing a design problem are different things.

    Take a good idea wherever it comes from—do so with a critical eye, with your vision clear and the actual design goals for the project in plain sight—but take the good idea. Learning how to take an editorial or reader note is vital.

    I say this not as an amateur game designer talking to amateurs, but as a developer and tester who has shipped a bunch of playtested products, talking to designers with published games.

  5. And another thing: There’s no excuse for being a dick like I was. I apologize. I don’t know what’s wrong with me.

  6. I’d say I qualify as an independent amateur game designer. Where would you say “specialized knowledge, training and experience” comes from or what form does it take?

  7. Will: Well, everything in moderation, right? I thought about further qualifying my post by talking about the 80% of everything is crap rule, but I decided I had enough going on already. Of COURSE sometimes you get a brilliant solution from a playtester, and of COURSE sometimes your designer friends are idiots and don’t help you at all. But I think you and Tim have sufficiently condensed/paraphrased the takeaway points in your comments here.

    As for the “amateur” thing – I don’t make a living off of my RPG design. Never have. That’s the single distinction for “amateur” status that I’m making, and I’ll go ahead and put words in Ben’s mouth and say that was what he meant as well.

    Hi Atminn! I would say that it takes writing and self-publishing one full stand-alone roleplaying game to get the crash course, and then writing a couple more and publishing at least one of them to get to where you can start seriously separating wheat from chaff in the playtest-and-feedback process. That’s roughly how I’d say my craft has developed, and I feel like it’s a pattern I’ve observed in a number of other folks, but I am just speaking for myself here! Playing a ton of games is good training, playing them and discussing them critically is also really good. For me, theory discussion was/is helpful for analysis, but less so for actual design. I don’t know if that answers your question – if not, maybe it’s worth a whole ‘nother post to discuss? You tell me.

  8. […] a counterpoint that has a terrific quote I want to focus on afterwards. First of all the article. Playtesting & The Designer as Expert. And the […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: