I was incredibly excited to come across this presentation on Valve’s approach to play-testing. For those who don’t want to page through the whole PDF, they go over the advantages and limitations of many different methods of gathering data: observation, talk-alouds, surveys, in-game data collection, physiological measures and more. Each method is good at getting at certain kinds of data, and you have to think about the game design problem you’re trying to solve when you choose which one to use.
It doesn’t surprise me that Valve does a great job with play-testing; their games are super-polished! What did surprise me, though, was how similar their analysis was to what I learned in my research methods classes. There’s a lot of talk about how far apart academia and industry are, but this particular area seems very closely aligned.
I was, of course, also reading with an eye to the data collection we’re doing for Lit. As we start play-testing our first digital prototypes, there’s that moment of “Oh, man! We’re a bunch of academics. What the hell are we doing?” The answer, it seems, is learning from the best that’s out there. Rock.