As I read through a lot of level reviews, I see again and again how this bug our that one should have been caught by a beta tester. I could write this entire column about how important it is to test a level, but you probably know everything I would say, and if you are like me, by the time a level is finished (at least in your opinion), you just want to get it out the door.
Rather than going on and on about how important beta testing is, I'll sum it up real quick and move on. Beta testing is NOT important and you shouldn't waste your time doing it. At least it isn't important if you don't care about the quality of the levels you put out. Its unimportant if you want people to delete the level off of their hard drive after a few minutes of playing it and it is definitely unimportant if you don't want to make the best level you can make. After all, you probably just spent a few weeks (quite a few, actually) making a level and so what if a few bugs makes all that time wasted. Does it matter if nobody plays your levels? Well, guess what? It matters to me.
Lets suppose you ignore my advice and decide you want to beta test your level. What is beta testing? When do you start? Who should do it and how should they do it? These are just a few of the questions you might be asking yourself. There are basically two kinds of testing, alpha testing and beta testing. Alpha testing is done before the level is finished. You probably do most of this yourself, adding a few rooms or cogs and then loading JK and giving the level a test. Usually, only the level's author will do alpha testing, although it is not a bad idea to have one or two other people alpha test a level to help give insight and ideas. The problem with doing outside testing early is that the author usually knows a lot of the bugs that are in the level and it becomes redundant to report bugs that the author knows about.
One thing that I have found alpha testing really useful for is ideas and showing the level to a texture artist to get ideas for new textures and tweaking other ones. Having an alpha tester also gives a sounding board to try ideas against. A lot of good ideas for WarZone came out of comments from alpha testers (for example, the storm trooper/ReeYees battle at the beginning of level 2 was proposed by the texture artist, Lou Graziani).
If you are working with a team of editors on multiple levels, you should also have the person that wrote the design document for your project on the alpha testing list to make sure the level is on track (you do have a design document, don't you?).
As great of ideas may come from your alpha testers, I do recommend keeping the number of alpha testers down. Once you have played through a level (even its alpha version), you are much more likely to pass things over on your second or third time through. For this reason, I hate to use a lot of alpha testers and prefer to save them for beta testings. This way, the level will be fresh to them and they are more likely to take a good look at everything the way I intended it to be.
So, your level is almost done; is it ready for beta test? Nope, not in my opinion. Don't beta test until you believe the level is completely finished (a mistake that I made in WarZone). Let your beta testers see the level in its final form. This makes sure that all your final tweaks get tested and also finds any problems you might have introduced into the level at the last second. Do everything you can to find/fix errors and have the level ready for release before you send it to beta testers.
Who makes a good beta tester? I use two different kinds of people for testing. Probably the most useful one I have found is someone who I can watch, in person, play through the level. This is especially useful when it comes to puzzles and problem solving. I've also found that I tend to notice little things that don't look quite right when I'm watching rather than playing. The hardest thing with this method of testing, though, is to not give the player any help or make any comments, even when they turn to you and ask a question. Remember, the player that downloads the level over the Internet will not have that luxury (of course, inform your tester that you are not going to help them and tell them why, so they don't get upset).
The second type of person I like to use for a beta tester is another level editor,.someone who knows what type of bugs to look for. The general knowledge of a level editor is invaluable when going through a level looking for problems.
The rest of this article I'm going to aim more at the beta tester (although it applies to anyone testing a level, including the author). Here is a thought for you: pencil and paper. Whenever testing a level, have them both handy. Any problem you see, write it down, be it texture alignments, HOM errors, non-solid floors and what-ever. If you don't like the way something looks, write it down. If you have a suggestion, write it down.
I always like to do multiple passes through a level when I beta test. The first time through, I play it for fun (but if I see a problem, I take time out to write it down immediately). I may spend a little longer than I might otherwise checking things out in a little more detail, but in general, try and enjoy the level the way a player would. This gives you a good feel for the level and the ambience that the author is trying to achieve. I typically do a first pass on medium difficulty.
The second time through a level, I look for all the problems I can find, including the texture stitching errors, walk-able surfaces and what not. I try to do the unexpected, to mess up puzzles or anything else I can think of that might not work. I also recommend listening. Does the level have ambient sound? Is it appropriate to the area it is in? Do I have any other general comments I want to make? Keep in mind that the level author may throw out anything you suggest and don't be offended by it. I like to do a second pass on easy difficulty.
I usually like to do a third pass on a level, after having taking a break from it (such as the next day or something). One last chance to get a fresh perspective on it. Typically, I do this third pass in non-3D accelerated mode. Things can look very different with the 3D accelerator turned off. The third pass I try the hard level of difficulty.
Ok, who am I kidding? How many beta testers are going to go through a level multiple times? Not too many. When you distribute your level, you might ask specific beta testers to try it on specific difficulty settings. Try and match testers with their skill levels.
I am including a list of some things to look for. This is in no way a complete list, but includes some common problems I have found.
Architecture: Lack of detail, to much detail (low frame rate), Hall of Mirrors (HOM), stairs that you can't run up, stairs that you can't walk up (sometimes you can run up stairs, but can't walk up them), switches that aren't inset into the wall, floors you sink into and objects that disappear when looked at from certain angles.
Lighting and colors (note that this needs to be checked in accelerated and non-accelerated modes): Color pallet problems, areas without light calculations run, poor lighting/shadowing
Cogs: All cogs need to work correctly. Probably the most common cog error is when a door opens and you get the HOM effect, but they come in all shapes and sizes.
Single player: Difficulty settings, enemy placement, weapon/item placement.
Multi-player: Frame rate is more important in MP levels, weapon placement, dead end corridors (usually bad).
On a final note to the beta testers, is also a good idea to note things that really give a 'wow' type feeling. If there is some especially nice architecture in an area, or a great cog trick, let them know. Tell the author the kind of feeling the level gave. They will appreciate it.
As I've already stated, this is not the end-all, be-all of testing, but it is a start. Each new level will bring new bugs with it, especially as we start to see more new 3DO's and animations. If you think of any common bugs that I have left out (I'm sure there are a ton of them), let me know. Good luck and lets see if we can get a few more bugs out of these levels before release.