If the content of a SOAP message is not understood or the recipient of a message does not know what to do with it when they get it, then using Web services for business, even with extensions for reliable delivery and security, will just not work.

To solve this problem you not only need to define how to "deliver the message" (see WSJ, Volume 3, Issue 11), you also need to define what the contents of a message mean as well as define what the recipient should do with a message when it is received. In other words, you need to define the message semantics.

This brings me to the purpose of this article, which is to describe the different semantic definitions you need to create so that Web services can be used for business. It then describes the leading standards that are being developed to support the definition of semantics and suggests an architectural approach to solving the problem of handling multiple different semantic definitions.

Why Standard Semantic Definitions Are Needed
Standard semantic definitions are needed for three main reasons:
1.  Shared understanding: Without a precise definition of what things mean that is shared and agreed to by the developers and implementers of the systems being built and connected using Web services, there will be confusion and no interoperability.
2.  More choice: If the semantics of services, messages, business documents, and so on, are all defined with the same definition language, then it should matter less what technology was used to develop them, enabling more customer choice in the tools they use.
3.  Reduced information loss: If a buyer and a supplier each create their own versions of the order documents they create or accept, but they are different, then the buyer might create data that the supplier can't use and the supplier might be missing some information that they need.

For example, what does a customer ID in an order mean? Is it the identifier the supplier uses to identify the customer, or is it the one used by the customer to identify itself? Similarly, if an invoice is sent to a business it could mean "please pay the amount specified in the invoice for services and products provided" but it could just as easily mean "please calculate the tax and customs duties due on this invoice, then return the invoice so that I can send it to the customer". Without a clear statement of what the Web service that receives a message should do with the message, interoperability will not occur.

The Role of Context
Semantic definitions also vary depending on the context. For example, a date is always a date and identifies a particular day in a year. However, the meaning of a date can vary depending on where the item is being used. For example, a "shipment date" is the date on which a shipment is made and a "sell by date" is the date by which a product should be sold.

Even this can be further qualified depending on the purpose for which the data is being used. For example, in a planning document a shipment date could be a date that identifies when the buyer is requesting the goods be shipped. Later, in an Advanced Shipment Notice, it would be the date set by the supplier that identifies when the supplier plans to deliver the goods; later still. In a Delivery Note, it would be the actual date the goods were shipped.

This means that semantic definitions must almost always include information about the context in which the definitions should be used.

What Semantic Definitions Are Needed?
Creating semantic definitions requires two different types of standards: Definition Languages, which describe "how" you can describe the semantics and structure; and Semantic Definitions, which define the "actual" structure and semantics (see Figure 1). For example, XML is a "definition language" that can be used to define business documents such as an order or invoice. However, to get interoperability you also need to create a "Semantic Definition" for an actual order explaining what each field means as well as creating XML definitions that define what the order looks like on the wire.

 

In general, two groups of semantic definitions need to be created using these standards: data-related, including languages to define business documents, messages, and transformations or maps; and process-related, including languages to define business processes, choreographies, and service interfaces (see Figure 2).

 

Note: the remainder of this article references several standards initiatives. Brief descriptions and links to these initiatives, together with a glossary of the terms used are available from the Web Services Journal Web site at www.sys-con.com/webservices/sourcec.cfm. (Further analysis of Web services developments is also available on the author's weblog at www.davidburdett.com)

The Semantic Web
The Semantic Web is a W3C effort that describes how to use RDF to define additional information about resources on the Web so that they can be more easily understood by machines. Although the Semantic Web will make searching for resources on the Web more accurate, it isn't very helpful when using Web services for business as developers (i.e. people), rather than automated software, will, for the foreseeable future need to understand the semantics so that they can work out how to use Web services to connect businesses together.

Data-Related Semantics
Data-related semantics describe the business documents and messages that are exchanged between Web services. As the sender and receiver of the message may use different message and document formats, transformations or maps are also needed to translate between them. Each of these areas is reviewed in turn.

Business Document Definitions
Unsurprisingly, XML is "the" definition language for defining business documents. Because of its popularity and ease of use, many organizations - RosettaNet, PIDX, ACORD, and hundreds of others - have created their own business documents and semantic definitions using XML.

This variety of business documents, often with slightly varying semantics and different XML representations, means that at least in the short term transformations have to be used by either the sender or the receiver. However, transformations require technology to make them work. This means that some businesses - especially smaller businesses that work in different industries - won't be able to support all the different formats they would like because of the costs involved.

The initiative that promises to address this problem most directly is Universal Business Language (UBL) from OASIS, which is developing a set of simple supply-chain business documents that are universally applicable. It's gaining support from many vertical industry groups who plan to use it as the basis for versions in their industries. Other popular business document formats include OAGI and xCBL.

Message Definitions
A message consists of one or more business documents wrapped inside a SOAP message - either with or without attachments - with additional SOAP headers for security, reliable delivery, etc. (see November's article).

WSDL allows messages to be defined in the context of a particular service. However, messages can be reused by many services, by different businesses, and in many different contexts (see earlier). It would be better if messages were defined separately from the service definition so that a service definition could just reference the message definition. However, there are no standards efforts to develop a separate definition language for messages.

Lower implementation costs would also result if semantic definitions that specified the structure and semantics of a specific combination of business documents, the version of SOAP, and associated SOAP Headers for actual messages were standardized. However, no standardization efforts are attempting this mainly because of the wide variety of business document standards; the need to finalize standards for security, reliable delivery; and the absence of a stand-alone standard for defining messages.

That said, a number of vertical industry initiatives, such as RosettaNet, PIDX, CIDX and others, have created their own message definitions that are either non-SOAP or that use SOAP-based specifications such as ebXML Messaging.

The result is that there is still a long way to go before standardized message definitions will be available.

Transformation (Maps)
The lack of standardized business documents and messages means that implementers will often need to transform the data they handle, requiring technology to do the actual transformation and "maps" that define what the transformation should do.

The only existing "standard" for transformations is the powerful and flexible XSLT. However, this power makes it less than ideal for use in business document transformations as most current XSLT implementations store the complete XML document in memory. This limits XSLT usefulness since business documents can sometimes be large - at least several megabytes.

There are also few transformation maps publicly available that describe how to map between the popular document formats as most maps are built into proprietary mapping software.

Until standard business document formats become widely accepted, a business can minimize costs by adopting an "internal" business document standard, e.g. xCBL or UBL, to which business documents received from other systems and partners are mapped. This means that instead of having to create a transformation map for each combination of business document and message format you use, you only need to create maps to and from your standard formats - significantly reducing the number of maps required.

Process-Related Semantics
Process-related semantics are the next area where standard definitions are needed. These definitions include (see Figure 2): internal business processes definitions that typically execute on either a single system or within a single business; choreography definitions that describe how two or more independent businesses or processes cooperate in terms of the sequence of the messages they exchange; and service interface definitions that describe the interfaces to a service that is either executed by a business process or is a target of one of the messages in a choreography definition.

The distinction between a business process definition and a choreography definition is important. A business process definition describes how internal, private business processes work - for example the Sales Order Management process where a business uses its sales management system, stock management system, and fulfillment system to satisfy orders that the business receives. In this case, the business handling those orders is in complete control of how those internal and external systems are integrated and combined with existing manual processes.

Choreography definitions, on the other hand, define how one "independent" business or process interacts with another by defining the sequence and conditions in which messages are exchanged between them. In this latter case no single business or process is in control so each has to agree with the other on how to cooperate. For example, if a buyer sends a supplier an order, the supplier needs to know how to respond. Should they: a) return an order response indicating the extent to which they can meet the order, b) just ship the goods and send an invoice, or c) do something different. No single business can unilaterally decide what do without informing, and getting the agreement of, the other businesses involved.

Let's look at these in more detail.

Business Process Definitions
As I said earlier, business processes by their very nature tend to focus on operations within a single business or organization. Good, effective business processes are also one of the ways in which businesses realize competitive advantage. One business is unlikely, therefore, to want to or need to share its internal business process definitions with anyone else.

Historically, specifications such as XLANG, WSFL, BPML, and others have provided ways of defining a business process by linking together Web services. However, the Business Process Execution Language for Web Services (BPEL) is now replacing them as the leading standard in this area. All these languages enable new business processes to be defined by combining Web services that have been defined using WSDL.Once a new business process is defined, access to the functions it provides may itself be exposed as a Web service by providing a WSDL definition for it.

Given that business process definitions won't often be shared outside of an individual business and therefore are not critical to interoperability, why will languages such as BPEL be used? There are probably two reasons: consultants and others with expertise in a particular business topic or vertical industry may develop process definitions that represent "best business practice," which they then sell to their customers as the basis for their client's solutions; and a business may develop a business process definition that represents their own "best practice," which they can then replicate in several divisions of their business.

Although porting business process definitions between platforms should be possible, it is likely to face the same problems as porting between Java implementations - it's not too hard as long as you haven't used any platformspecific extensions.

Choreography Definitions
Choreography definitions, on the other hand, are critical to interoperability as they define how two or more businesses or processes cooperate by specifying the sequence and conditions in which messages are exchanged together with explanations of what sending a particular message means.

This is important, at least in the short term, as a Web service will often be used as a "frontend interface" for some existing "legacy" application with hard-to-change behavior that will process the data. If the semantics of the choreography is not both well defined and properly understood by each role involved (e.g., buyer and seller), then misunderstandings will often lead to incorrect processing of the message.

At run time, choreography definitions, as long as you know the role you are taking, can be used to verify that the messages are exchanged in the sequence required. Also, since a single definition is shared between all the roles, it is easy to independently check prior to "going live" that each implementation behaves correctly. Checking the compatibility of separate, independently designed business process definitions that businesses do not want to share is harder, especially if several roles are involved.

Major choreography definition language standards include BPSS, which focuses primarily on B2Bi, is closely tied to other ebXML specifications, and not easily used with other Web services specifications; and WS-Choreography, which is developing a choreography definition language that can be used to define how businesses or Web services in general can cooperate.

Several vertical industry groups, such as RosettaNet, have defined their own choreography definitions. However there are no cross-industry initiatives working on standard choreography definitions that would be more widely applicable.

Web Service Definitions
WSDL version 1.1 is mature in the way that it defines an interface to a Web service in terms of the input and output messages, and the destinations (called ports) to which messages are sent. The WSDL1.2 specification is also being developed and is making good progress, although there are many differences with version 1.1, making it unlikely that it will be backwards compatible with WSDL 1.1.

However, more work is needed on both specifications so that they can record additional information such as the choreography definitions the service supports as well as additional SOAP features such as security and reliable delivery.

Although none are being developed, standardized service definitions with clearly defined semantics for sending/receiving a message would help as there would then be a standard way of connecting businesses, for example to place an order, where only the URL of the destination and security information needed to be changed.

What Should You Do Now?
Clearly there are multiple standards, and in particular, multiple different semantic definitions for business documents, service interfaces, and choreographies. This means that, in the short term, a business will have to support multiple different ways of connecting to their existing systems and partners. However, waiting for standards to mature so that connectivity is easier will mean missing the real benefits of better, lower-cost solutions that using Web services for business can bring.

Adopting the Right Architectural Approach
The solution to this problem is adopting the "right" architectural approach (see Figure 3) for the solutions you build or buy. This approach, which is an extension of the approach discussed in last month's article, solves the problem by expanding the "interoperability bridge" to transform business documents as well as handle different Web services standards. It also includes a "process management engine" that dynamically selects the business process definitions to use that support the choreography definition required.

 

It is also a good idea to adopt a single "internal" business document standard (described earlier) to simplify transformations of business documents as well as expose internal applications and the external systems you connect to as small, discrete Web services that provide a single function such as accepting an order. As a result it is much easier for the process management engine to link those Web services together in different sequences so that you can more easily and rapidly support the wide variety of different choreography definitions that many businesses will be asked to support.

This approach separates the business application from the changing environment in which it operates. As a result you can, relatively independently, change an application, the business process, the business documents, and the Web service standards being used. It also means that when changes occur the interdependencies can be handled in one place - in the rules - at lower cost and more consistently.