General Java

When Java was first released, it was immediately attractive due to its ease-of-use and the promise of WORA (write once, run anywhere). As it evolved, the value of the JRE abstraction has manifested itself in many ways not immediately apparent from the days of animated applets. For example, the widespread adoption of Java on the server helped to drive the development of several performance profiling and application monitoring tools and techniques. These tools and techniques bring great value to the Java platform but some also have significant limitations and drawbacks. This article surveys current tools and techniques and looks at new initiatives to evolve the JRE into a truly manageable runtime.

Managed Runtime Environments
One of the original key goals of Java was to provide a layer of abstraction that allowed a programmer to write code that was portable to many different hardware/OS platforms. However, the Java Virtual Machine (JVM) doesn't just offer programmers a machine-independent way to run their code, it dynamically manages executing code. The JRE was the first mainstream Managed Runtime Environment (MRTE) with characteristic properties such as:

• Bytecode: A machine-independent executable representation of application code
• Automatic memory management: Object allocation, garbage collection, dynamic heap resizing, and compaction
• Program security and correctness: Bytecode verification, sandbox for applets, no pointers, type checking, array bounds checks, and null pointer checks
• Dynamic optimization: The VM can adapt to the characteristics of the running application and selectively optimize the code it generates
• Exception handlers: Deal with unusual situations and errors

These things are not unique to Java. Some are also characteristics of Smalltalk (developed in the early 1970s) and there are also close similarities to Microsoft's .NET CLR. Interestingly, at least one MRTE implementation supports both Java and Microsoft's Common Language Infrastructure (CLI), with only a relatively small amount of customization required to handle the different bytecode formats of CLI and Java.

MRTEs have emerged rather quietly. When Java was first released by Sun in 1995, most of us weren't really framing the benefits of Java in terms of managed code, rather we were struck by the promise of WORA and the benefit of a powerful JDK API, which made it pretty straightforward to start developing your own applications, even if you were new to the world of object orientation. Many of the intrinsic benefits of an MRTE were initially secondary to a new breed of programmer but proved key to Java's expansion to the server, where it became easier to write secure, robust, multithreaded, memory-intensive applications. It was soon also widely recognized that the abstraction required to handle an intermediate binary format need not come at the cost of performance, as MRTEs can use a dynamic code optimizer to react to the runtime characteristics of an application. Look at what Microsoft is doing with .NET, or the work Intel is doing to evolve its new range of 64-bit Itanium processors, and it's clear that the (inevitable) migration to MRTEs is widely recognized by both software and hardware vendors. In fact, the proliferation of MRTEs is seen by many as the biggest paradigm shift in software since the move from assembler to high-level programming languages.

For an MRTE to be as efficient, stable, and scalable as possible, it shouldn't be just a black box. We should be able to peer inside to make sure there are no obvious performance bottlenecks, monitor memory usage, etc. To be viable in the enterprise, a managed runtime should also be manageable.

Rather than just blindly executing bytecode, a JRE has the ability to add value by adapting its environment to the running application. A few examples of how this is possible are:

• Progressive levels of code optimization: Optimization requires processor time, so the JRE should focus its most aggressive optimization on the most frequently called methods.
• Dynamic expansion or contraction of the heap size, according to the needs of the application
• Adapting the garbage-collection algorithm to the requirements and behavior of the application: A typical trade-off here is acceptable maximum pause time versus overall throughput.

Not surprisingly, there are limitations to the JRE's ability to adapt to the behavior of a running application. Also, hardware resources are always limited, of course, and programmers often introduce performance bottlenecks, which are hard to remove. To get the best performance and scalability from your Java environment, you'll need to:

1. Optimize your application code
2. Tune the JVM for the application and underlying OS and hardware

These two things are obviously not entirely independent, so it may be an iterative process to fully optimize your runtime system. Since the early days of "almost-black-box" JVMs, various tools and techniques have evolved to help optimize your application during development and monitor your application in production. These tools generally use bytecode instrumentation or the Java Virtual Machine Profiler Interface (JVMPI). Several commercial tools are aimed at monitoring J2EE applications, e.g., Introscope (Wily Technologies), PerformaSure (Quest Software), and Cyanea/One (Cyanea).

Keep in mind that all of these tools introduce a performance overhead into your system - the more information you want, the higher the overhead - and they don't tell you much about the behavior of the JVM. To tune the JVM you'll probably benefit from running your application under load, looking closely at the performance and behavior of the application, and doing some trial-and-error adjustment of the available JVM parameters (which vary across different JVM implementations).

The BEA JRockit Management Console is a tool designed primarily to monitor the runtime characteristics of the JVM (as opposed to the behavior of the application). The console can connect to one or more remote Java processes and be used to monitor many runtime statistics and parameters of the running JVM. Two benefits here for a production system are: (1) the monitoring overhead is insignificant (measured to be less than 1% for several different workloads), and (2) the console has a configurable event notification framework that can be used to notify an operator of certain warning events, giving them the opportunity to react to a situation before something more dramatic happens (e.g., a trigger can be set to identify a low available free memory condition, perhaps allowing an operator an opportunity to take corrective action and avoid the infamous OutOfMemory Exception).

The JRockit console shows important characteristics of the JVM, such as the heap and CPU usage. It also includes a method profiler and an exception counter. Figure 1 shows the memory panel of the management console. Most of this information is also available in the administration console of WebLogic Server (which uses a standard JMX interface to expose JRockit runtime statistics).

Future Directions
Many of the current profiling tools and techniques can be invaluable in tuning your application or JVM, debugging problems, and capacity planning. However, many of these approaches also have some significant drawbacks:

• Applying any of the profiling techniques causes a perturbation to the running system. You can't measure the system without having some impact on it (the inescapable Heisenberg Uncertainty Principle).
• Licensing costs.
• They work only with specific JVMs (e.g., JFluid or the JRockit class preprocessor).
• The tools lack an active feedback loop, i.e., these approaches are monitoring rather than true management solutions. For example, if an operator sees the JVM is running out of heap space in the JRockit Management Console, it's not currently possible to dynamically resize the heap.

Most of these tools also miss an essential part of the equation, as they focus on application performance and give you little indication of whether performance problems are caused by the JVM (or maybe the underlying OS). BEA has recently made public their JRockit Runtime Analyzer (JRA). This tool collects a wide variety of JVM runtime data for a specified sampling period. A graphical tool is then used to display this data, providing deep insight into the behavior of both the application and the JVM. One of the major benefits of this tool is that the profiling capability is built directly into the JVM, which allows for a much lower overhead (measured to be <1%) compared to the other common techniques of bytecode instrumentation, or JVMPI.

Another great benefit of JRA is that it collects a lot of data that's not possible with a standard JVMPI-based tool, e.g., JRA shows you which methods have been optimized and provides detailed information about all the GC cycles that have occurred during the profiling phase. The JRA is a very versatile tool that can add value in a variety of different situations:

• Developers can use the JRA to identify bottlenecks in their application, or provide data to help tune their heap and GC parameters.
• The BEA Customer Support team can analyze JRA profiles sent by customers who suspect a JVM problem.
• JRA data is used internally within BEA, in advanced development of the JRockit JVM.

More details on this tool can be found at http://edocs.bea.com/wljrockit/docs81/jra/jra.html.

In addition to vendor-specific solutions, various JSRs have emerged to evolve the JVM into a manageable environment and address some of the limitations of earlier implementations of the experimental JVMPI. J2SE 1.5 will include a new Java Virtual Machine Tool Interface (JVMTI), and an API to monitor the state and control the execution of programs running inside the JVM. JVMTI is designed to provide an interface for a wide variety of tools that need access to the VM state, e.g., profiling, debugging, monitoring, thread analysis, and code coverage tools. This work is being implemented through JSR 163 (and is closely related to JSR 174). JVMTI supercedes the Java Virtual Machine Debugger Interface (JVMDI) and prerequisites the new Java Platform Profiling Architecture (JPPA), which will supercede and improve upon the experimental JVMPI.

JVMTI agents run in the same process as the JVM being examined, communicating through a native interface that's designed to allow maximum control while introducing minimal overhead on the running system. Agents should be as lightweight as possible, and are controlled by a separate process that implements the bulk of a tool's function without interfering with the target application's normal execution. JVMTI is a two-way interface whereby agents can both query and control applications through functions, and be notified of interesting events. JVMTI functions may be categorized into different groups. Some examples of available functions are given in Table 1.

Agents can respond to many different events that occur while running a Java application. Some example events that may be generated:

• Breakpoint: When the running application hits a predefined breakpoint
• Field Modification: When the application accesses a designated field
• Method Entry/Exit: When the application enters/exits a particular Java programming language or native method
• Class Load/Unload: Generated when a particular class is loaded/ unloaded
• Garbage Collection Start/Finish: Generated when a full-cycle GC begins/ends

JVMTI includes an extension mechanism whereby a JVMTI implementation can provide functions and events beyond those defined in the specification. This mechanism can be used to provide JVM implementation-specific information, which may be invaluable for monitoring or debugging situations.

Also included in JSR 163 are APIs for JVM monitoring and management (java.lang.management) and in-process instrumentation (java.lang.instrument). The former API, described in more detail in JSR 174, provides a monitoring and management interface to the JVM and the operating system. It is designed to be suited to production environments and can be used to gather data on the threading, class loading, and memory subsystems, for example. The java.lang.instrument API, known as Java Programming Language Instrumentation Services (JPLIS), allows agents written in Java to instrument applications running on the JVM. The agent provides an implementation of the ClassFileTransformer interface, which is used to modify the bytecode of a class before it's actually defined in the JVM (i.e., providing a standard way to do class preprocessing). Details of the proposed "management" and "instrument" APIs may be found by downloading the latest version of the JPPA.

Profiling Your Java Code
Here's a quick survey of some of the tools and techniques available.

Bytecode Instrumentation (and Class Preprocessing)
Java bytecodes can be instrumented statically (i.e., by the generation of an instrumented .class file) or dynamically at runtime during the class-loading process. Many profiling tools use the latter approach, utilizing a custom class loader that modifies the byte stream read from the .class file during the class-loading process. Usually only certain classes will be instrumented, to minimize the performance overhead of the profiler.

The Byte Code Engineering Library (BCEL) provides freely available tools that can be used to instrument existing classes, or create them from scratch (http://jakarta.apache.org/bcel/). The BCEL API may be used to instrument classes both statically or on the fly. It offers a convenient abstraction that avoids the need to directly modify the bytecode yourself.

Another freely available tool is AspectWerkz (http://aspectwerkz.codehaus.org), an open source project sponsored by BEA's JRockit development team. AspectWerkz is a dynamic, lightweight, and high-performance AOP/AOSD framework for Java. It utilizes runtime bytecode modification to weave classes at runtime. It hooks in and weaves classes loaded by any class loader except the bootstrap class loader, with special support for runtime weaving using JRockit, allowing the addition, removal, and restructuring of advices as well as swapping the implementation of introductions at runtime.

Bytecode instrumentation is also used by many commercial tools such as Introscope from Wily Technology (www.wilytech.com). This tool provides a real-time performance monitoring solution for application servers. On-the-fly bytecode instrumentation is used to provide a lot of performance data that is not accessible through application server vendor–specific tools.

JFluid is an experimental dynamic bytecode profiler from Sun (http://research.sun.com/projects/jfluid/). It can be used to profile an arbitrary subset of your Java program and be activated/deactivated while the program is running. Unlike the other tools above, it requires the use of a special version of the HotSpot VM that supports dynamic bytecode instrumentation.

The JRockit JVM provides an interface to make it easy for you to plug in your own class preprocessor. In other words, rather than requiring a special class loader, you register your own implementation of the com.bea.jvm.ClassPreProcessor interface, which has only one method:

public byte[] preProcess(java.lang.ClassLoader classLoader,
java.lang.String className, byte[] classBytes)

where the method parameters are the class loader instance used to load the class in question, the name of the class, and the byte array that represents the bytecodes of the class. A class preprocessor may be registered in the JVM instance using:

JVMFactory.getJVM().getClassLibrary().setClassPreProcessor
(preProcessor);

A simple example of how to use a preprocessor can be found at http://dev2dev.bea.com/products/wljrockit81/resources.jsp.

The Java Virtual Machine Profiler Interface
Many of today's Java profiling tools use the experimental Java Virtual Machine Profiler Interface. JVMPI defines a bidirectional interface between a JVM and an in-process profiler agent. This agent sends profiling data to the profiler front end, which displays information on CPU usage, memory allocation, object references, and monitor contention. JVMPI is used by both Borland's Optimizeit Profiler (includes a memory and CPU profiler) and JProbe from Quest Software.

Hprof (http://java.sun.com/j2se/1.4.2/docs/guide/jvmpi/jvmpi.html#hprof) is a JVMPI profiler agent shipped with many JDKs. It uses the JVMPI to gather information about a running JVM and writes out profiling information either to a file or to a socket.

HPjmeter from Hewlett-Packard is a useful free tool that allows you to read (and compare) the output files from hprof (www.hp.com/products1/unix/java/hpjmeter/index.html). Another useful tool from HP (not based on JVMPI) is HPjtune, which can be used to read the GC metrics in the log files produced by the Java VM (output of -Xverbosegc).

Summary
We've described some of the benefits of Java in the context of the JRE as a Managed Runtime Environment. We surveyed several tools and techniques available to profile Java code and monitor the runtime characteristics of a JRE. In the future, the manageability of JRE implementations will be enhanced through the standard JVMTI interface as well as through vendor-specific tools. Driven by the needs of server-side, business-critical applications, the Java managed runtime will evolve into a truly manageable runtime.

References

Intel Technology Journal, Vol. 7, issue 1: www.intel.co.jp/technology/itj/2003/volume07issue01/art01_orp/p03_mre.htm

JSR 163: Java Platform Profiling Architecture: www.jcp.org/en/jsr/detail?id=163

"Using the Monitoring and Management APIs": http://edocs.bea.com/wljrockit/docs81/jmapi/index.html

JSR 174: Monitoring and Management Specification for the Java Virtual Machine: www.jcp.org/en/jsr/detail?id=174