ZapThink's integration cost curve, which we published back in 2002, continues to stir discussion amongst our Licensed ZapThink Architects. In brief, our argument is that while traditional middleware-based integration leads to unpredictable spikes in cost when business requirements change, taking a Service-Oriented Architecture (SOA) approach to solving integration challenges leads to a flattened cost of change. Implementing SOA means building for change, so the argument goes, so while there will continue to be some ongoing costs, a fundamentally agile architecture will smooth out the ups and downs of IT integration expense.

Cut to the end of 2008. Today, in spite of ZapThink's repeated warnings about taking an ESB-first approach to SOA, many organizations have bowed to vendor pressure and have undertaken an ESB-first, "SOA platform" approach to implementing SOA. As we stated in that ZapFlash, it's possible to implement SOA with an ESB, and many organizations are doing just that -- but the essential best practice is to leverage the ESB as a Service intermediary, rather than as integration middleware. Placing this best practice in the context of our integration cost curve, leveraging an ESB as integration middleware will lead to the spikes in cost when requirements evolve, eliminating the flattened cost of change benefit that SOA promises. Only by taking an architecture-first approach to SOA will oganizations be able to achieve this benefit.

Middleware for your Middleware?
We're now seeing evidence of this trend in the marketplace. As organizations attempt to scale their SOA platform-centric SOA implementations, they soon run into problems. Because of the size of today's enterprises, no single platform addresses the SOA infrastructure needs of the entire organization. As a result, they must implement different platforms for different parts of the business -- what some analysts are calling "SOA domains" (not to be confused with the business-centric notion of Service domains that ZapThink has been discussing since 2004). Instead, a SOA domain centers on a SOA platform and the Services that are running on that platform. As a result, scaling the SOA initiative requires interactions between SOA domains, which leads to the challenges of interoperating and federating across SOA domains. And how do you best accomplish such interoperation and federation? By purchasing more middleware, of course!

Herein lies the most dangerous pitfall of the SOA platform-centric approach to SOA: because it depends upon integration middleware, it succumbs to the issues with middleware that SOA was meant to address, namely the lack of agility and the increasing costs of integration over time. Basically, you'll eventually need more middleware to get your various ESBs running in various SOA domains to interoperate or federate with each other. Now, it's possible to implement SOA in this manner, but if you're unable to achieve either the business agility or cost savings benefits of the new architecture, then why bother?

Interestingly enough, it was Mike Gilpin at Forrester Research who clued us into the depth of this problem. While Gilpin's report is impeccably researched and internally coherent, his conclusion is incorrect. Basically, his argument is in part that since enterprises have a SOA platform strategy, as they scale their SOA initiatives, they will likely need to interoperate or federate between SOA domains, which will require more middleware. Gilpin, however, fails to take this argument to its natural conclusion, which is a reductio ad absurdum: if you assume that implementing SOA depends on a SOA platform strategy, then eventually you'll need middleware for your middleware, which will eliminate the benefits of SOA that led you to the architecture in the first place. Therefore, SOA should not depend on a SOA platform strategy.

Taking the Plunge into No-Platform SOA
This ZapFlash need not delve into the details of how we recommend approaching SOA from an architectural, rather than an integration-centric perspective, as we've covered this topic in great depth in previous ZapFlashes. We advise against caving into software vendor pressure to take a SOA platform approach. We also explain architecture-driven infrastructure, discuss the intermediary pattern, explain its importance to loose coupling, and then focus on the difficult task of building and maintaining the Business Service abstraction. In fact, we warned against middleware for your middleware back in 2005.

Interestingly enough, the core principle for the No-Platform SOA approach appeared in our seminal 2002 Service-Oriented Process report, where we explained how integration should be a byproduct of Service composition. More than six years later, we're still fighting this battle: integration should be something the business does with the Services, not something IT does to connect one bit of infrastructure to another. Instead, the core technical challenge of SOA is building and maintaining the Business Service abstraction, so that the business can build flexible processes by composing those Services. In other words, SOA requires us to move away from a "connecting things" approach to distributed computing to a "composing Services" approach.

After all, the integration-centric approach to business process, what we might call traditional Business Process Management (BPM), has long suffered from the limitations of the technology. In traditional BPM, the process engine is either part of the platform or tightly coupled to it. As a result, even if you're implementing processes by composing Services, those Services must run on the platform, or the engine is unable to adequately control and manage them. On the other hand, if you take a Service-oriented approach to business process, then we're abstracting out the underlying runtime environments of the Services, which can now run anywhere we like -- locally or remotely, on or off the bus, in Java, .NET, or even mainframe environments. Such processes maintain process instance state via the messages the Services exchange, instead of relying upon the engine to spawn threads or instantiate other objects, which are essentially non-Service-oriented techniques.

The point is this: the SOA platform approach to implementing business processes has more limitations than advantages. It impedes cross-platform processes in addition to requiring additional middleware to scale. Furthermore, the larger the implementation becomes, the less agile it is. On the other hand, the no-platform approach is more difficult, as it requires that the organization get the architecture right, including proper governance, leveraging the intermediary pattern, and all the other aspects of building and maintaining the Business Service abstraction. Most difficult of all is that the no-platform approach to SOA requires a different way of thinking about distributed computing that for people comfortable with traditional integration-centric environments feels much like jumping from an airplane without a parachute. That sense of risk, however, is in large part illusory: a combination of unfamiliarity and vendor pressure. It's time to cut the cords and take the plunge into architecture-driven, no-platform SOA.

The ZapThink Take
If you've been through our LZA course, or even if you've simply been following our ZapFlashes for a while, the no-platform approach to SOA is clearly appealing to you -- but unfortunately, the software-first alternative has always seemed to be lower risk. After all, if you dive into the no-platform approach and fail, it's your head that's on the block, but if you buy some big package from your favorite vendor and have difficulties, then your job is unlikely to be on the line. And if the big analyst firms (many of whom are in the employ of the vendors, by the way) say that a SOA platform approach is reasonable, then why rock the boat?

Now, however, something has changed: the economy. How well do you think the "middleware for your middleware" story for scaling your SOA implementation will sell in a down economy? It doesn't matter now if the boss golfs with the vendor's VP of Sales, or if you consider yourself a vendor "shop." Today it's time to economize, and thrift is the word of the day. ZapThink has been saying for years that you probably won't have to buy much software to implement SOA, and now maybe it's time to get the boss to listen.

It's still too early to tell just how bad this economy is going to get, but rest assured only some organizations will survive. While SOA promises costs savings and greater agility, two essential elements of surviving a steep downturn, simply having a SOA initiative doesn't guarantee success. After all, you have to get it right. If you implement SOA and fail to achieve its desired benefits, or if you attempt to implement it and fail along the way, that doesn't mean that there's something wrong with SOA. What it means is that you didn't do it properly. When we see enterprises who've taken a SOA platform approach consider purchasing middleware for their middleware to scale their SOA initiatives, oblivious to the fact that following that path will prevent them from achieving the goals of SOA, all we can say is that we'll be placing our bets on their competition -- the ones who are taking an architecture-first approach to SOA.

Learn more at one of our upcoming Licensed ZapThink Architect Bootcamps or SOA & Cloud Governance courses.