Wednesday, September 20, 2006

Game Testing and NDAs

I try to stay fairly plugged into industry news and communities, and thus I run across a myriad of opinions about game testing from the denizens of the dank rocks under which I hide. One of my pet peeves is the prevalent attitude that game testing, specifically public or private alpha and beta phases, are for the benefit of the test participant rather than for the company or the game title.

Here are some things that game testing is NOT:
  • Reward: Game testing is NOT an entitlement program for your good citizenship, your connections in the industry or your 1337 skilz at pwnz0r teh nubs in X game!
  • Free Gaming: Game testing is NOT an opportunity for you to try out a new game before you have to pay for it. It is NOT there for your entertainment, gratis.
  • Skill Development: Game testing is NOT an opportunity to hone your skills at the next big MMO title, so that you can get a jump on everyone else or join the uber guild.

Here are some things that game testing IS:

  • Stress Testing: Game testing IS an opportunity for developers to see how their game features and servers perform under 'normal' conditions.
  • Bug Discovery: Game testing IS an opportunity for developers to find and correct game bugs, unintended mechanics, and close loopholes and exploits.
  • Focus Group: Game testing IS an opportunity for game developers to get a pulse on what the player think about the game and the various features of said game.

So what impact does game testing have on the success of a given title? The answer varies, but with the right testers and the proper action from the development team, game testing can be the difference between a disastrous flop of the highest magnitude, and a well polished game shipped when it is ready.

When you join any phase of game testing you agree to certain rules that prohibit the sharing of information, outside of the private forums and sanctioned events. These Non-Disclosure Agreements (NDAs) are critical for the integrity of the testing phase, for a number of reasons. An example of this, is World of Warcraft vs. the current beta leaks coming out of Vanguard: Saga of Heros. During my time in the World of Warcraft testing, my initial impressions were terrible. Game play was choppy, bugs were prevalent, content was disastrously minimal and the servers were unstable as all hell. However, after months of restarts, rollbacks, patches, feedback and fixes, World of Warcraft shipped as one of the most polished MMOs in industry history. Sure there were bad news leaks out of the WOW alpha and betas, but they were minimal and did little to quell the hype surrounding the game, and the result is history.

On to Vanguard... Everywhere I turn, non-public information about the current state of the game is available. Videos, screens, commentary and frankly, very little of it has been positive. Is Vanguard a bad game then? I would wager money that it is not a bad game at all, however the people who violate the NDA are painting that picture with very vocal brushes. This is unfair to the company, the title, the fans eager for this game, and the industry as a whole. When you join a testing phase, you are agreeing to represent that company and until said NDA is lifted, it is imperative that the information and opinions that you develop during your time there are secret.

So what do I do when I am invited to test a game?

  • I log in, and play hard.
  • I try to break the game, exploit every way I can think of, use every feature and push the limits of the world that the development team has created.
  • I take detailed notes. This servers two purposes, as I can both formulate my opinions for public discourse after the NDA is lifted and I can make informative posts for the development team.
  • I report my findings often. Find a bug? Repeat it and record the facts. Write a synopsis and submit it with detailed concise information and a clear header [BUG] Moss Snake Kicks.
  • I respect the NDA. I may think this game sucks and will flop, but it is not my place to push it over the precipice. There will be plenty of time for that after the NDA is lifted.

I hope everyone who is invited to join in the game testing process, can make a similar set of mantras for themselves, and help the industry grow and develop.

No comments: