Adding a main() method
One simple way to test the functionality of your Applet is to make it into an application by adding a static main() method.

The simplest kind of main does all the testing itself, as shown in Listing 1.

This kind of test is simple to run and interpret and simple to repeat, but is at best somewhat inconvenient to write. It requires enumerating the test cases and expected results in the test driver, which may be a lot of code to add to your Applet. It also requires the main to create an instance of the tested class, which may be difficult.

The example shows a very simple kind of check on the result of a method: comparing it to a tabulated result. This is not the only kind of check that can be done. Other, less direct, checks on the result of a method are possible, or required, in many cases.

double r = cubeRoot(f);
// somewhat arbitrary margin for floating-point error
if (Math.abs(r*r*r-f) > 1.0e-20) {
error();
}

Component c = this;
c.enable();
if ( ! c.isEnabled()) {
error();
}

The main() added for testing purposes can be more general than just a list of test cases. It can be, for example, a simple command processor that drives the methods of the application and prints the results; this can be used to exercise your Applet from the command line.

Use a Test Generator
Rather than writing a main() to drive the methods of your class, you can use a test program generator to create the code. An example of such a test program generator, the only one I am aware of for Java programs, is JavaSpec, from my company, SunTest (http://www.suntest.com/).

A test program generator works by taking a specification of the required behavior of the program and of the data to use for testing, and using those specifications to generate a test program. JavaSpec, in particular, uses fragments of Java code as specifications. Essentially, JavaSpec automates the bookkeeping required to implement the test driver and the test scorecard, leaving to you the interesting work of creating the test data and the checking code. In addition, JavaSpec provides a framework for the structure of the test, helping you to create comprehensive test data sets and helping you with the structure of the checking code.

GUI Testing
The simplest way to test an applet is to click the mouse at it. This can be done by hand, or with a GUI testing tool. In either case, the hard part is not clicking, but checking.

Recall that the purpose of testing is to show that a program has certain properties. The utility of GUI testing depends strongly on what properties you are trying to show. If you are trying to show that a program obeys buttons pushes and displays data, then GUI testing is indispensable. If you are trying to show that a program updates a database or calculates a square root correctly, then GUI testing may not be the most direct method.

GUI testing is generally best for two kinds of testing:

Safety checking is done by driving the Applet through a sequence of actions, trying to exercise all of the states of the system.

GUI Testing Tools
Using a tool to automate the GUI test has several advantages: It is much easier to repeat a test exactly, and the test can be much more extensive. In addition, the GUI testing tool may provide convenient ways to check the display, making it much easier to check the results of the test actions.

If you do decide to use a GUI testing tool, you must choose between a platform-specific GUI testing tool and a Java GUI testing tool. A platform-specific GUI test tool, like XRunner or WinRunner from Mercury Interactive, QA Partner from Segue, QC/Replay from Centerline, SQA Robot from SQA, or CAPBAK/X from Software Research, tests at the level of native code. A Java GUI testing tool, like JavaSTAR from SunTest, tests at the level of the Java AWT.

The advantage of a platform-specific tool is that it can test the complete operation of your applet in a specific browser on a specific platform, including the interaction of your applet, the browser's Java implementation and the native window system. The disadvantage is that the tool and the test are tied to the specific platform and applet viewer or browser used.

A Java GUI testing tool has the advantage of producing portable, cross-platform tests and of itself being portable, so that the test can be created on any platform. Its disadvantage is that it relies on the correctness of the Java implementation.

SunTest's GUI testing tool, JavaSTAR, has the additional feature, which we think an advantage, that the scripting language is Java; the test scripts are Java programs that can be edited as required. Generally, the platform-specific tools have a proprietary scripting language for describing tests, which must be learned for effective use of the tool.

Self-checking Code
One special technique that may make it simpler to write a test is to write a self-checking version of the class you are testing by adding assertions to your code using a method like this:

private void assert(boolean cond, String msg) {
if ( ! cond) {
System.err.println("assert failed: " + msg);
errorCount++;
}
}

If you have instrumented your code like this, then every program run becomes a test run; however, it is hard to do comprehensive checking from within a class and you must still plan and create the test data.

Portability Testing
Once you are satisfied that your Applet does what you want it to do on one platform, you need to check that it does what you want it do on all Java platforms. This is the province of portability testing.

Portability has various aspects. Fundamentally, portability testing is a check that you have stayed within the parts of the language that will work the same in every environment, or that you have worked around the differences.

An Applet may run on a WebTV, on a multiprocessor Solaris workstation with a 24-bit framebuffer, on a PC with no sound card, or on a JavaStation running JavaOS. The goal of portability testing is to check that your Applet will perform its function in all these environments.

The most important resource for improving the portability of your Applet is Javasoft's "100% Pure Java" program [http://www.javasoft.com/100percent], which provides information, training, testing and certification aimed at improving the portability of Java programs.

A good portability test requires a sanity check, static checking and dynamic testing.

Sanity Check
Try your applet in various browsers on various platforms. Does it display? Try resizing the window; leaving the page and coming back; pushing a few buttons; typing a few characters. This is preliminary to a full test; it's like checking that a car has four fully-inflated tires before going for a test drive. You could call it the "tire-kicking" step.

Its purpose is to catch errors so fundamental that they would prevent the more detailed test from running, and also to catch errors so unexpected that they are not checked in the test plan.

Static Checking
Does your Applet use any unportable methods? Use the Java Purity Checker, the "100% Pure Java" static checker, to point out possible problem areas. The Java Purity Checker will also flag classes that may not run under a SecurityManager. You can obtain the Java Purity Checker from SunTest [http://www.suntest.com].

Dynamic Testing
Firewalls
If possible, set up your Applet on a server behind a firewall and see if it still works. Common problems include loading the Applet class, loading other required classes and loading other resources like sounds and images; in general, anything that requires a new network connection back to the Applet's server may run afoul of the firewall.

. If your Applet does any of these things, you should make sure that your firewall testing covers those parts of the program. For example, if your Applet plays a sound when a button is pushed, then make sure that your GUI test script includes a push of that button.

Specific Problem Areas Using the test planning and creation techniques described above, check that your applet doesn't run into these known problem areas.

Layout If your applet uses a layout manager, try resizing it; if it changes the contents of a Container, exercise that part of the code in various browsers.

Threads If your applet creates a thread or synchronizes, it may behave differently on different Java implementations, or even at different times of day. This class of problem is, unfortunately, almost impossible to check by testing; there are just too many program states to check, and too little control over what state the program is in. The best cure is prevention: think hard about multithread behavior.

For example, if your program uses synchronization locks, define a global lock order and obey the rule that locks must be acquired in order (no thread may acquire a lock unless it holds the preceding lock). This rule prevents deadlock.

Test Automation
No applet test plan is complete without a human in the loop, for sanity checking and to evaluate reported errors. Nonetheless, automated test tools may significantly enhance your applet testing. The benefits of static checking tools like Java Purity Checker has already been discussed; the advantages of dynamic testing tools include:

Repeatability
Automated tests are repeatable, which makes it possible to run the same tests on different platforms. It also makes it possible to repeat the test in a regression run after fixing bugs.

Portability
Testing tools that use and are written in Java are portable to any Java environment, and produce tests that are portable to any Java environment. This is a key advantage for portability testing.

Completeness
An automated test can range over more test data, and perform more result checks, than a manual test. Hence, an automated test can give a more thorough test than a manual test.

Conclusion
The strategy and techniques presented should give you a good start at developing a test strategy for your Applets. The most important point is to have a test strategy; know what you are looking for, know how you will measure it and know how you will know when you have measured it. That ensures that your test efforts are not wasted.