WSJ Management

Much has happened to the World Wide Web since its start, with continuing and dramatic improvements that have created one of the most powerful information sharing and communications tools worldwide. During the past few years, Web applications and services have burst onto the scene, expanding on the Web's ability to deliver on its original promise of rich functionality, features, and integration. Today's most successful deployments integrate robust networking and application designs along with stringent testing in order to ensure solid, ongoing performance and reliability.

Web application infrastructures, like any significant network deployment, can suffer from myriad issues and vulnerabilities. At best, these issues result in performance and scalability problems that annoy end users. But the ongoing drive to reap the benefits of a Web applications- enabled enterprise have driven the integration of Web infrastructure into core business functions, with performance implications and security vulnerabilities that, if left unresolved, can result in millions of dollars in lost productivity and serious liabilities from stolen personal and financial data along with confidential corporate information.

What must enterprises solve in order to deliver on the promise of Web applications? Web applications arguably share many of the same issues as any system deployment. At a minimum, the list usually includes:

• Reliability: The ability of the system to function properly under various conditions.
• Availability: The ability of the system to provide nonstop services over a particular service period, which for the Web is often "forever."
• Scalability: How effectively and how easily a system can adequately respond to changes in demand.
• Security: While security has always been a consideration for most system deployments, Web applications and services must often contend with a lack of built-in security while running in an environment that often leaves them continually exposed to security threats.
• Performance: The ability of the system to serve responses rapidly.
• Interoperability: The ability of the system to integrate and work with other connected systems and enterprises. This is indeed one of the advantages of Web applications and services.

Your environment may have different pressures from the list above, but will almost inevitably include cost and time - the bane, and indeed driving force, of most deployments. Given all these pressures, requirements, and goals, what can help ensure success? In a word, testing. Testing provides valuable insight to understand how things are progressing during design and deployment and how well things will go once deployed. As with any network and application deployment, testing benefits from knowledge and expertise. This article provides a foundation towards robust testing and ultimately, a reliable, high-performing Web application deployment. As you read the tips presented here, remember to adapt and apply the ideas to your specific requirements and environment rather than adopting them blindly.

A doctor receives a call from his patient, who says, "A month has passed since I saw you and I'm still feeling sick."

The doctor replies, "Did you follow the directions on your bottle?"

"Yes, I did", said the patient, "It says 'Keep tightly closed'.''

Critical Success Factors
All too often we sacrifice quality and proper deployment testing to meet development schedules based on time-to-market pressures. The following are the key ingredients toward avoiding that trap.

Have a Plan, Man!
You have an idea that might "have some legs," ask several colleagues to have a working meeting with you, and the idea develops into a proposal for a Web application.

Rigorous planning can make a dramatic difference in the success of a Web application deployment. By outlining the entire process from start to finish, participants can better understand their role in the process and propose improvements early. A plan provides clarity and a sense of direction to what can seem like a daunting process. Start by documenting the goals, answering important questions such as the features that the deployment will support and its intended users. Next, consider the success factors, including cost and deadlines along with specifics such as availability (e.g., 99.5% or 99.99%, 24X7 or weekdays from 8 am to 6 pm) and performance (e.g., sub 2-second response time). Incorporate the important project steps and phases into the plan, such as design, testing, development, deployment, final assessment, and maintenance. Finally, obtain agreement and sign-off on the plan from involved parties, including marketing, management, end users, engineering, and operations/information technology.

Test Early and Often
During early design of the Web application network, you consider two approaches for data exchange that could work. You prototype the ideas that afternoon and test them, discovering that your first idea not only performs better, but is simpler and easier to build.

Getting an early start to testing, which may seem to delay a project, will actually end up saving both time and money. Projects have particular costs associated with them; the simplified graph in Figure 1 depicts the cost life cycle for development and testing/ updates. As a project progresses from the design phase to deployment, development costs increase. After deployment, the project moves into the maintenance phase and development slows down. For testing and updates, the costs start very low - most testing during the design phase will involve comparing prototypes and testing ideas, with changes easily made. As the project proceeds, the complexity increases, requiring increased effort to test and implement changes spurred from testing. Early testing will help find issues and design flaws earlier, resolve performance problems, locate bottlenecks and failures, and ultimately deliver a final system that is more reliable and has fewer unexpected issues.

'Think Test' from the Onset
Everyone on your project team agrees performance is an important goal so that all meet early on to discuss ways to ensure excellent performance in the final web application.

Incorporate testing into your Web application right from the start by having its developers and architects integrate visibility and testability into their designs. Providing visibility into a Web application ensures testers can obtain details on its status during testing, while also enabling operations to monitor status in production. For example, if performance is a critical goal, the application could provide timings for certain operations (database insert took 350 ms, record search took 1249 ms) so that bottlenecks can be quickly identified. If reliability is a goal, the application could provide status codes to highlight important internal events. Of course, do not overlook the value of documentation for testability, which could provide important details such as cookie format, dynamic session IDs, state diagrams, etc.

Having the developers, architects, and testers work together from the start not only fosters improved communications throughout the test, but also promotes a mindset of high quality early on.

Web Services Performance - Surviving to Tell the Tale
Part of your new system includes a Web services component to allow partners to access important data quickly and easily. Initial performance tests of this feature reveal performance issues as the number of simultaneous accesses increase. You work with IT to add another server, and meet performance targets.

Many times, enterprises choose to migrate a legacy, proprietary service over to Web services in order to gain some valuable benefits. Often, this migration comes with an unexpected cost in performance when the new architecture fails to perform anywhere near the legacy application. Because Web services work with XML formatted data in clear text (typically less efficient than binary data), it ultimately increases time and resource utilization on both the client and server to parse and marshal the data. Incorporating SSL and WS-Security also reduces performance while increasing the amount of data that needs to be transmitted. Web services deployments should include both functional and performance testing to ensure that the delivered service meets expectations.

'Get Real' When Testing, a.k.a. 'Garbage In, Garbage Out'
You are now ready to move testing to the next stage, incorporating realism into the test by simulating several populations of users that use different browsers, connect at different link speeds, and exercise your Web application in different ways.

While testing provides dramatic benefits, its value stems from proper test design and usage. Having a highly advanced and expensive testbed means little if used improperly, and can lead to incorrect conclusions that end up with disastrous results in production. Incorporate realism into your testing, which for Web applications means considering two aspects: user behaviors and network realism.

Quantifying users involves capturing their particular behaviors, which can include their Web application usage patterns, browser versions, page think times, and user abandonment. Networks themselves also have certain issues that can dramatically affect Web application performance - this becomes more and more likely with larger and more complex networks, with the Internet as a prime example. Issues faced by networks include end-to-end latencies, link speeds, packet loss, IP fragmentation, jitter, and bursty traffic patterns, all of which can adversely affect Web application performance and stability.

How much realism is needed? This question is not easily answered, but ultimately ends up being a judgment call on the tester's part, balancing the effort and expense of incorporating increased realism with the costs and criticality of failure in the Web application. However, prioritization will help, choosing the user and network behaviors that have the most dramatic effect on your Web application.

System Failures Are Fine, as Long as They're During Testing
During one early morning test run, the Web application fails, causing concern in one of the senior developers, who comes to talk to you. You point out that this is actually good news.

Try to "break" your Web applications during testing. This can mean tactics such as sending incorrect data, overrunning it with double its expected traffic (or more!), sending a large denial of service attack, trying to access the system with cookies turned off, etc. Every failure discovered during testing not only prevents it from appearing in production, but also helps answer many important questions:

• Was the failure expected? What was the performance near failure?
• What systems/functions failed? Which ones first? These answers point to candidates for current or future upgrades.
• What were the signs and system behaviors leading up to failure? This information can be included in system monitoring, helping to predict failures before they occur.
• Is recovery from failure quick? Easy? Reliable? Automatic?
• Did backup systems take over during failure? How did they do in performance and function?

Security Must Be Tested
You arrive on Monday to learn that a distributed denial of service attack effectively brought down your company's Web site that weekend and quickly realize that your pre-deployment Web application is currently just as vulnerable.

Network attacks continue to increase in potency as their creators learn to exploit the prevailing weaknesses of network devices and software. No single strategy can effectively mitigate security issues, but security testing plays an important role in validating and ensuring the overall security of a security infrastructure. Beyond helping to discover security and privacy issues in a Web application, security testing determines the performance impact of the security deployment - increasing security usually comes at the cost of reduced performance.

When testing for security, test both the inside and outside of the network perimeter - network intrusions and attacks can just as easily come from internal people or hackers that have successfully infiltrated other parts of the network. Also, closely examine network security while under network load - networks that seem secure without traffic can, under load, overload and send traffic to backup systems that do not have the same network protections.

Test Holistically
During testing, you systematically test each network component individually, starting with the Web server. You discover a serious performance issue and post these issues on a forum. A few hours later, the forum contains several suggestions; the second suggestion triples the performance, and the other suggestions provide another 20% increase.

A modern Web application infrastructure consists of many systems interconnected in a large network that simultaneously attempts to address performance, security, availability, reliability, and scalability. Holistic testing starts from the early deployment stages, dividing each component of the infrastructure to test each one individually. This means firewalls, intrusion detection and prevention systems, caches, server load balancers, Web servers, application servers, databases, and file servers should all be candidates for testing. This helps to focus attention on each component, ensuring that each is tuned to properly address the needs of the overall Web application while also allowing poorly performing components to be replaced early.

After individual testing, connect the individual components and test again. This test locates overall system bottlenecks, discovers interoperability issues, and validates proper network functionality and performance.

At the Finish Line, Get a Baseline
You and your team have put in many hours to create a system that everyone is proud of. Before turning the Web application live onto the Internet, you conduct a final performance test and locate a performance issue tied to an application that was still running in debug mode.

Conduct a final test suite to get a baseline of performance of the new system. This baseline provides valuable information:

• Validates expected performance.
• Reduces the chance of surprises in production.
• Provides visibility into performance changes: System performance often changes with use. For example, a database with 500 records will perform better than when it grows to 500 million records.

To increase the effectiveness of baselining, consider "bracketing" your baselines just like photographers do when capturing that important shot. Because of film's limited exposure latitude, photographers take photos at the exposure that they expect will produce the best result and then take extra shots below and above the expected exposure, just in case. Baselining can incorporate this same idea, testing at various settings and levels to obtain performance and behavior under various conditions.

Finally, use baselines as the starting point for future upgrades to the Web application, comparing upgrades to the baselines' performance to ensure that an improvement has actually been realized.

Your Work Is Never Done
With your Web application in production, you find that your assumptions on how it would be used were off-target on a few items, and tweak your tests to incorporate these differences.

Testing is a continual learning cycle. The successful deployment of a Web application is certainly one of the more important milestones in this cycle. Once in production, the Web application moves into maintenance while becoming one of the most valuable sources of information for future deployments and testing. It provides information that can often only be "guesstimated" beforehand, such as usage patterns and peak loads. The effectiveness of testing can be determined and adjustments made to improve future testing.

Monitoring usually plays a critical role in production, continually recording important system parameters, performance, and availability. Effective monitoring can often be the first indication of system failures, sometimes even being able to point out failures before they happen. Testing points out candidates for monitoring, but the live deployment will ultimately determine which monitoring candidates matter while adding new ones.

Summary
Today's Web applications contribute billions of dollars to our world economy, and there is no reason to believe this growth and innovation will not continue. The most successful deployments will deliver rich functionality and compelling features along with users that will expect top performance and reliability. Competitive pressures and concerns with cost only serve to augment the necessity of delivering superior applications and networks. Indeed, the demands are many, and there is no magic bullet, no easy way out. However, rigorous testing provides the foundation to answer these demands effectively, saving costs, improving quality, increasing security, and delivering high performance.