While scanning through the Tabula Rasa closed beta test forums today, I kinda noticed that approx. 97% of player feedback is absolutely useless.
Beta test means: the key elements are implemented in the game and the testers are now asked to provide feedback about those elements, find bugs, and come up with suggestions and ideas of how those key elements should be combined to make the game fun and appealing. This process is iterative, meaning the developers continuously add more features like more races, more classes, secondary features, and improved key features. However, the key features are not and just cannot be subject to change since they define the core gameplay aspects of the game. Or, to say it frankly: devs will probably ignore (maybe even cry about) suggestions that would cause shipment delay of several months to implement. Really, they are just not interested in things that would require implementation or even extinction of the key elements that are already in the game because all those key elements are connected in many different ways. Well, I think you got the point. The question is: how to get qualitative and valuable tester feedback?
There's even one more challenge to keep in mind: only a very very small part of the beta testers participates in official testing-related feedback mechanisms (e.g. beta forums) at all.
Many games seem to use basic empiric and mandatory evaluations to collect feedback, e.g. whenever you complete a quest or level up, you have to answer a few questions like „did you like that quest?“ or „do you think your current equipment if ok for your level?“. This method may provide general gameplay happiness feedback, but won't really help in collecting broad qualitative feedback about the different key elements of the game.
Preselection of beta testers? Nigh impossible if you have 50k+ beta signups. And the applicants might've just copied well-written texts from somewhere else. Game exterior feedback collection? Only a small percentage of beta testers will even use it. So basically you need ingame-collected qualitative feedback. How do you get that? Well, I'd say by making your QA employees secretly interview the beta testers ingame. Tell them to found guilds and continuously form groups and then chat / talk to them about the game – within the game / current build without ever revealing their actual position within the company. [This of course would only work for multiplayer games.]
Beta test means: the key elements are implemented in the game and the testers are now asked to provide feedback about those elements, find bugs, and come up with suggestions and ideas of how those key elements should be combined to make the game fun and appealing. This process is iterative, meaning the developers continuously add more features like more races, more classes, secondary features, and improved key features. However, the key features are not and just cannot be subject to change since they define the core gameplay aspects of the game. Or, to say it frankly: devs will probably ignore (maybe even cry about) suggestions that would cause shipment delay of several months to implement. Really, they are just not interested in things that would require implementation or even extinction of the key elements that are already in the game because all those key elements are connected in many different ways. Well, I think you got the point. The question is: how to get qualitative and valuable tester feedback?
There's even one more challenge to keep in mind: only a very very small part of the beta testers participates in official testing-related feedback mechanisms (e.g. beta forums) at all.
Many games seem to use basic empiric and mandatory evaluations to collect feedback, e.g. whenever you complete a quest or level up, you have to answer a few questions like „did you like that quest?“ or „do you think your current equipment if ok for your level?“. This method may provide general gameplay happiness feedback, but won't really help in collecting broad qualitative feedback about the different key elements of the game.
Preselection of beta testers? Nigh impossible if you have 50k+ beta signups. And the applicants might've just copied well-written texts from somewhere else. Game exterior feedback collection? Only a small percentage of beta testers will even use it. So basically you need ingame-collected qualitative feedback. How do you get that? Well, I'd say by making your QA employees secretly interview the beta testers ingame. Tell them to found guilds and continuously form groups and then chat / talk to them about the game – within the game / current build without ever revealing their actual position within the company. [This of course would only work for multiplayer games.]
No comments:
Post a Comment