XML Protocols

The single most significant factor in the performance and scalability of XML may be the runtime engine that processes it. Parsers are the key component of that runtime engine and present a unique set of issues. By looking at the current capabilities and limitations of parsers, we can assess the performance and scalability constraints of a given XML application.

The processing of an XML document by an application must begin with the parsing of the document's various elements and attributes. This processing communicates both data and metadata to the calling application. With XML, the DTD represents the metadata that describes what data elements are contained in the XML document and how they relate to each other. To access the data contained in an XML document, parsers read the DTD and validate the data against it.

Because of XML's flexibility, multiple APIs have evolved to process information in an XML document. This processing starts with parsing, which can occur two ways:

1. Tree-based: Loads the document into an object, which allows the application to traverse the nodes of the tree
2. Event-based: Allows the application to have a request/response conversation with the XML document

Each approach has a preferred implementation method. For the tree-based approach, the DOM is the preferred method since it's currently recommended by the W3C. For the event-based approach we can assume the preferred method of implementation is the Simple API for XML (SAX), based on the number of parsers that support this API. Most commercial parsers support both DOM and SAX parsing. While each approach offers processing capabilities consistent with the independence and flexibility of XML, they also create significant overhead that can dramatically impact the scalability and performance of an application.

Regardless of the implementation method used, for any system to act on an XML document it must parse the document in order to understand the information as the author intended it. In this final article on performance and scalability we'll examine the impact of parsers, since they drive most XML-aware applications. We'll also discuss the relative strengths and weaknesses of the two parsing approaches and their applicability in typical implementations.

Note: XML parsers are constantly changing to keep up with the evolving DOM and SAX specifications. The research for this article utilized the Java version of the Apache Xerces Parser version 1.1.3.

The Role of the Parser
To assess the performance and scalability constraints of parsers, it's necessary to discuss the functions they perform. The parser performs one or more of the following checks on the XML document:

• Well-formed: Ensures that the structure of the document has the appropriate "begin" and "end" element pairs and that the nesting of these pairs is appropriate
• Validity: Compares the structure of an XML document to the DTD referenced by the document, and ensures the contents adhere to the element and attribute constraints

For an implementation to take full advantage of XML's flexibility and extensibility, the system must rely on the parser to communicate the necessary information from the XML document. After conducting the aforementioned checks, the parser can process the XML document using the tree- or event-based approach.

Let's consider what happens when a parser processes an XML document into a DOM. In Figure 1 the chart illustrates the size differences of the file for various storage methods. (Note: Although this increase is linear for smaller XML messages, these results were consistent across multiple documents in the 3-4KB range.) This transaction increased in memory by 10-15 times when represented in a DOM. Although the specific multiple varies from document to document, we consistently see an order of magnitude increase when going from XML to DOM. This increase must be a consideration when processing documents via the DOM. For large XML documents or simultaneous processing of multiple XML documents, the increase in required memory can become a major factor.

In the implementation used for this article, additional constraints included a maximum file size that the parser could support (16MB), a limit on the size of any particular tag, and limits on the total number of nested tags (260) that the parser could handle successfully. Although a majority of the constraints were probably outside the realm of most real-world XML applications, they're still worth noting for completeness and for trapping errors so overflows can be properly dealt with.

DOM Approach
Parsers that implement a tree-based approach must process the entire XML document in order to generate a DOM representation of the document in memory prior to enabling interaction from the calling application. This hierarchical tree structure provides easy manipulation of the document while maintaining the integrity of the structure. New nodes can be inserted or modified within the structure as required by the application.

The W3C has created multiple levels of the DOM to provide additional methods and to support broader functionality. Building on the functionality of the basic browser, the additional levels specified by the W3C (and under various states of definition) include:

• Level 1: Concentrates on the actual core document models, contains functionality for document navigation and manipulation, and applies to both XML and HTML
• Level 2: Includes a stylesheet object model and document filters, defines an event model, provides support for XML namespaces, and defines functionality for manipulating the style information attached to a document
• Level 3: Addresses document loading and saving, reading other content models (including schemas, document validation support, key events, and groups), and document views and formatting

When considering performance and scalability, the first concern with the DOM approach is the effect it has on system resources. Since DOM parsers load the entire XML document into memory, the implementation must have the physical memory available for this task. This requires the application to manage overflows as there's no real recourse for a document that's too large. Perhaps more important is how this limitation impacts the support for the number of documents that can be opened in parallel by the calling application. Since the DOM specification doesn't enable an implementation to process the document in sections, this request can add significant overhead to the processing of multiple documents. Furthermore, current parser implementations are not reentrant; that is, they can't allow multiple data sets to be mapped against a single DOM held in memory.

The upfront resource costs of using the DOM approach are more than justified if the core function of the application is to substantially or repeatedly modify the content or the structure of the document. In this case working with the DOM allows efficient and reusable application calls that can interface directly with the XML document. Procedurally, if the calling application requires this level of access to the entire XML document or the ability to process different sections of the document, the use of the DOM API may be warranted.

SAX Approach
While a DOM parser reads the entire document into a tree structure before allowing any processing, an event-based parser allows the application to have a conversation with the XML document by communicating events in response to invoked methods by the application's implemented handler. Because of this conversational approach, the memory and initial processing requirements are lower for an application processing a document using SAX. This resource efficiency, though, requires the application to manage the various conversation handlers that are necessary to fully communicate with an XML document. Managing these handlers becomes the responsibility of the calling application.

The SAX API has evolved via open source with two levels of the API:

• Level 1: Concentrates on the core handlers and exceptions for the specification
• Level 2: Incorporates support for namespaces, filter chains, and querying and setting features and properties in the parser

This approach introduces additional constraints. Since each conversation must occur sequentially through the XML document, the application has no concept of the ancestors or children of any given node. If an application needs to access different sections of the document, it has to start at the beginning and process the entire document rather than jump directly to another node. Finally, the SAX approach doesn't allow you to manipulate the structure and write out the revised XML document through a single call.

However, the advantages to the system resources of this conversational approach to parsing make these constraints acceptable. After all, the demand for this API was a result of the memory constraints of the DOM approach. The wide adoption of the SAX API attests to the efficiencies of the approach.

Common Constraints
Regardless of the interface used for parsing, there are constraints that require parser capabilities to evolve further. Even though there are several instances where parsing text documents creates inefficiencies in the process, it can be argued that the benefits gained by maintaining the flexibility and extensibility inherent in XML as a format far outweigh the cost incurred by these inefficiencies. However, additional constraints can't be defended in the same way. In this section we'll discuss two such examples.

Let's consider the inability to address DTD caching. Conceptually, a cache provides a place to store information that's required multiple times. The cache provides an easily accessible location to store program information, memory addresses, or data, which enables systems to improve overall efficiency. This basic concept must be applied to parsers so complex DTDs don't need to be reloaded every time a new XML document is to be parsed. Many DTD dialects leverage XML's flexibility by nesting (referencing multiple DTDs within a single XML document). This nesting creates a significant processing burden for any parser. Since the calling application can't determine which of the nested DTDs are needed to process the sections of the document, the parser must load all the DTDs. Given the inefficiency this creates, the ability to manage and cache DTDs referenced externally would be a natural extension to the current parser specifications.

Next, consider that parsers don't support object pooling. Object pooling allows the parser's configuration to have instances that are available in a pool, ready to be used by the calling application. This allows the application to configure and monitor the pool for the parser and tune elements such as size, creation, and timeouts. By enabling object-pooling parsers to be reused in this manner, significant performance and scaling benefits can be achieved.

Conclusion
For any application, interacting with a parser introduces a bottleneck. To effectively receive numerous XML documents - most of them representing extremely complex business transactions (such as those defined by standards bodies such as RosettaNet and OAGI) - an application must open each document to extract data from different fields. For each field it's necessary to perform tests on the specific data elements to determine its type, sender, and recipient; this must be done for every XML document.

By fully understanding the limitations of current parsers, it's possible to manage their constraints within calling applications. For example, you have an application that reads the XML document and communicates changes in state back to the calling application. In this scenario a SAX implementation may allow greater performance and scalability. However, if your primary requirement is to manipulate the content or structure of the XML document, incurring the upfront cost of the DOM approach may be more efficient. Choices become more difficult when a mixed mode operation is required. An application that needs to read only data elements may appear to be a SAX candidate, but not if the structure of the document includes nested DTDs and the relationship of the data elements aren't known in advance. In this latter case, the DOM may prove more appropriate since it can be easily traversed for multiple reads.

Beyond these current limitations, parsers and the APIs for accessing XML documents are evolving at a pace consistent with the widespread adoption of XML. This is good news and means the demands for scalability and performance will create multiple solutions with varying cost/benefit trade-offs. In the near future, hardware will be optimized to process XML transactions and the next generation of parsers will provide a hybrid approach that builds the DOM by numerous SAX calls. Either of these approaches may provide further control in tuning the performance and minimizing the bottlenecks in your solution.

This concludes a four-part series that has focused on performance and scalability issues associated with XML, but it shouldn't be viewed as an indictment of the core technology. Rather, it should reinforce the key benefits that make dealing with these issues worthwhile. XML provides the flexibility and extensibility that's core to solving today's business challenges. Its self-describing nature allows us to build systems and solutions that can adapt to rapidly changing business requirements. I also believe that raising these issues is key to ensuring they get addressed through the evolution of specifications and the cooperation of the standards bodies entrusted with them. A better understanding of these business issues enables us to push for further innovation and evolution in the technology, tools, and applications.

If you have any comments regarding this article, please to send me a note.

References

1. Source For SAX Parser API: www.megginson.com/SAX/index.html
2. Source for DOM Parser API: www. w3c.org/dom/
3. Open Source SAX Parser: www. apache.org
4. Implementation examples: www.textuality.com/