Industry Commentary

ILM Was Cloud Computing Buzzword de Jour A Few Years Ago

Thu, 26 Nov 2009 10:45:00 EST

Disclaimer, warning, be advised, heads up, disclosure, this post is partially for fun so take it that way. Remember ILM, that is, Information Lifecycle Management among other meanings. It was a popular buzzword de jour a few years ago similar to how cloud is being tossed around lately, or in the recent past, virtualization, clusters, grids and SOA among others. One of the challenges with ILM besides its overuse and thus confusion was what it meant, after all was or is it a product, process, paradigm or something else?

What Happened To SOA?

Sun, 04 Oct 2009 17:00:00 EDT

Is SOA dead? This question keeps popping up every now and then within the IT community and though we all seem to agree that SOA is far from dead something has indeed radically changed: our expectations. Just a few years ago, perhaps months, we considered SOA to be the solution to all of our (IT) problems, promising to unleash the power of our business, and that just by thinking 'services'!

US Government & Federal Agencies Keen on Enterprise Architecture & SOA

Mon, 17 Aug 2009 09:15:00 EDT

Over the past two years, governments around the world have caught the enterprise architecture and SOA fever. This is especially the case in the US Federal Government, where a number of regulations, EA frameworks, and major spending initiatives are all pointing towards continued and lasting expenditure and investment in the area of EA. One can also notice this trend based on the number of EA and SOA events now focused on the US Federal Government, or at least taking place in the Washington, DC area. For example, ZapThink’s Practical SOA event in DC on October 2, 2009 is focused on cost-effective legacy enablement and data management, while its 4-day licensed ZapThink Architect (LZA) boot camp running from September 21-24, 2009 provides SOA certification and credentialing to government and other DC-area firms. ZapThink’s SOA & Cloud governance certification and training class in DC sold out earlier this month (August) and was so popular the company is running it again, in DC, September 30-October 1.

Understanding the Value of Reference Architectures

Mon, 10 Aug 2009 12:30:00 EDT

There’s nothing more that architects love to do than argue about definitions. If you ever find yourself with idle time in a room of architects, try asking for a definition of “Service” or “architecture” and see what sort of creative melee you can start. That being said, definitions are indeed very important so that we can have a common language to communicate the intent and benefit of the very things we are trying to convince business to invest in. From that perspective, a number of concepts have emerged in the past decade or so that have become top of mind for self-styled enterprise architects: architecture frameworks and reference architectures. In previous ZapFlashes, we discussed architecture frameworks, which leaves the topic of reference architectures left untouched by ZapThink. Since we can’t leave a good argument behind, we’re going to use this ZapFlash to explore what reference architectures are all about and what value they have to add to the Service-Oriented Architecture (SOA) story.

Demand for Lean, Green Business Technology

Wed, 15 Jul 2009 09:45:00 EDT

According to a recent market study by Datamonitor, the current global economic recession may also prove to be a significant driver for Green Computing. Their market assessment raises lots of questions -- including, is it better for the world, and overall business profitability, if executives cut-back on their IT investments? "The global economic recession has spurred a paradigm shift in the way organizations evaluate, budget for and deploy green IT," says Rhonda Ascierto, senior analyst at Datamonitor. "The downturn has also resulted in green IT trends for datacenters, client devices and asset lifecycle management, as well as re-shaped return on investment (ROI) models."

SOA: Integration vs. Business Process?

Mon, 13 Jul 2009 11:15:00 EDT

Ever since ZapThink coined the term "Service Oriented Integration" (SOI) back in 2002, there's been unceasing confusion on just how Service-Oriented Architecture (SOA) and integration relate to one another. Several recent blog posts have refocused attention on this confusion, including ones from Anne Manes, Todd Biske and Loraine Lawson. Perhaps some of the confusion is over the definition of the SOI term, and what distinguishes SOI from Web Services-based integration, if anything. But the bigger controversy is over the larger question as to SOA's fundamental purpose: is SOA's purpose to solve integration problems, or is it more of a business transformation approach centered on implementing agile business processes?

Scalability Only One Half the Reliability Equation

Fri, 10 Jul 2009 09:00:00 EDT

Without availability scalability is irrelevant

I really enjoyed Jeff Atwood’s recent blog on Scaling Up vs Scaling Out, which includes a fairly detailed comparison of the costs associated with each approach to scalability. I enjoyed it because not only did it take into consideration the cost of hardware, but also remembered to include the cost of software licensing. And of course there’s the fact that Jeff’s site is focused on development and coding, and this discussion broadened the discussion into the realm of application networking – a demesne with which I am of course particularly fond.

Now the problem is – and you knew there had to be one, this is me after all – that the discussion is primarily about scalability. Scalability is good, of course, but it’s not the whole story. The discussion stemmed from an inability of a real server to meet capacity which resulted in the dread d-word: downtime.

Scalability is of course a Very Good Thing. There’s no disputing the fact that it’s absolutely critical to ensure that applications scale – one way or another. But simply ensuring scalability should not be confused with providing reliability because even a scalable application can end up “unavailable” despite having access to vast amounts of resources.

WHAT AVAILABILITY REALLY MEANS

The other half of the reliability equation is availability. At first glance it may appear that scalability automatically ensures availability, but that’s simply not true. There are several reasons why these two are not synonyms and should not be treated as interchangeable. First, in a business environment availability is not just accessibility – which is ensured through scalability - but timely accessibility. An application that does not meet its agreed upon service level agreements is not considered available. Abandonment rates increase dramatically as performance degrades and productivity plummets when applications are slow to respond.

Second, availability can be affected by other mitigating factors such as security. The recent (and ongoing) attacks on U.S. government websites, for example, are primarily denial of service attacks. Such attacks are focused on consumption of resources, and not just server resources but network resources as well. After all, it doesn’t matter how fast an application responds to a request if the response is going to be “stuck in traffic” as it were. A successful attack on the application targeting some known vulnerability, too, can result in the dread d-word by causing crashes or otherwise corrupting the application’s ability to service requests.

Thirdly, availability can be affected by applications that perform correctly at nominal load levels but which become unstable at high load levels. The simplest of examples of such a situation involves database connectivity. At high load levels it is often the case that an application server’s connection pool is simply not robust enough to keep up with demand. No amount of scaling up will solve this problem, because it isn’t a hardware resource issue, it’s a software resource issue that isn’t even under the control of the developers because the connectivity is provided by a third-party – the web or application server vendor. While the application server may be responding even under heavy lead, an application without data is really quite useless and, like failure to meet service level agreements, by business standards it is considered unavailable.

ACHIEVING RELIABILITY

In order to reach reliability nirvana, i.e. five 9s with consistent adherence to service level agreements, you cannot rely on scalability alone. It just doesn’t address all the factors that are necessary to ensure the availability required for a reliable application or site. You need to address three potential points of failure in the application architecture to avoid potential downtime:

Performance
Adherence to service level agreements are, admittedly, a pain in the proverbial rear-end because it seems like so many factors are often out of the control of the developer because they are. And yet developers are the ones often blamed for poor application performance. The application infrastructure needs to provide tools and mechanisms that can counter the natural tendency of application performance to degrade as capacity increases.
Security
Security needs must be addressed and not just the typical application vulnerabilities. Security needs to be aware of the potential for resource consuming attacks for which the purpose is to take a site or application “offline” as opposed to exploitation for other purposes. The security architecture must address both network and application security as a means to prevent an application from becoming unavailable.
Functional correctness
Applications must respond – and respond as expected – under myriad levels of load. In order to ensure this is true the application must be monitored in such a way as to report not just network or transport layer failures, but application response errors. This requires intelligent monitoring – the ability to not only check connectivity, but validate content for correctness.

None of these can really be addressed by the application itself, because the measurements and monitoring required must occur outside the application. That means some other solution must be responsible for providing availability of applications.

The best solution is a unified application delivery solution, because it brings together – as the term implies – the disparate external functions necessary to availability assurance in a single solution. Application and network security, bandwidth management, acceleration and optimization are all a part of application delivery platforms, and it is that platform that will provide the best chance for realizing true reliability of applications and web sites through the assurance of availability. A solution such as a unified application delivery platform is external to the application and, if you’re scaling out, you’ll need its core functionality of a load balancer anyway. This means it can monitor for functional correctness, provide network and application security, and offers a variety of acceleration and optimization features that can manage performance needs.

Technorati Tags: MacVittie,f5,application delivery,security,availability,reliability,scalability,monitoring,unified application delivery and data services,failure,web,internet,blog,coding horror,Jeff Atwood,infrastructure,architecture,development

Related blogs & articles:

Categories: Development and General , Performance , Monitoring/Management , Security

Google Jumps into the Cloud Wave

Thu, 28 May 2009 12:55:00 EDT

Busy day for cloud interoperability related news. Google just announced a new service called Google Wave, described as an open communication and collaboration platform & protocol based on hosted XML documents (called waves) supporting concurrent modifications and low-latency updates. In simple terms Google Wave can be thought of like an ajax spreadsheet over XMPP.

Does XML Have a Future on the Web?

Thu, 10 Jan 2008 08:00:00 EST

A more interesting question is 'Is XML on the web trending up or trending down?' Clearly, it is trending down. For data transfer applications, XML is losing ground to JSON because JSON is simply a better data transfer format. And XHTML has failed to displace HTML in the marketplace. The benefit of clientside validation has proven to not be a benefit. I think you can argue, and in fact I did argue, that because of W3C's adventures with XML, the web itself may not have a future. The browser has a lot of problems, the worst of which are the security problems that came with Netscape Navigator 2. That was 12 years ago, and there has been no progress since that time in fixing the fundamental problems. There have been lots of patches on top of patches. Nothing more.

AJAX: "XML for Client-Side Computing"

Mon, 25 Sep 2006 08:45:00 EDT

XML is a simple, flexible text format initially designed for large-scale electronic publishing. It is flexible, open, and human-readable, and can be learned easily. XML can also be generated, parsed, analyzed, and transformed easily. It's no wonder that XML has been widely used for server-side computing: J2EE, .NET, and Web services.

Out of Step - NIEM and N-DEx

Wed, 25 Jan 2006 20:15:00 EST

Since the horrific events of September 11, 2001, the federal government has intensified its efforts to improve communications, collaboration, and information sharing between government and private sector agencies at all levels. The task of creating a seamless system of data and communication between disparate agencies has faced both technological and political obstacles.

Open Source Database Special Feature: An Introduction to Berkeley DB XML

Tue, 20 Dec 2005 11:45:00 EST

In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

Flexible Identity Federation XML Gateways to The Rescue

Fri, 16 Dec 2005 19:45:00 EST

Imagine a fresh business relationship between ACME Corporation and Partner. As a result of this relationship, ACME wants to grant Partner limited access to one of its core internal applications. They do this, naturally, by exposing a Web service.

Untangling the Semantic Web

Fri, 18 Nov 2005 10:00:00 EST

The Semantic Web is a hot topic in information circles today, and its adoption will largely depend on stakeholders understanding its potential benefits and tools vendors providing an easy entry for developers to learn and work with its related technologies.

Overview and Analysis of the Department of Defense Discovery Metadata Specification (DDMS)

Wed, 01 Jun 2005 10:00:00 EDT

The Department of Defense (DoD) Discovery Metadata Specification (DDMS) describes the DOD's preferred approach for decorating data assets with metadata. By providing a common convention for metadata, the DoD is building a common system for asset discovery, search, description, consumption, and security. This article provides a summary of the DDMS's purpose, structure, and capability. Upon completion the reader should have a basic understanding of the DDMS and should know where to go to get more detail and related materials. All questions regarding this article should be directed to Michael Sick at mike@serenesoftware.com.

Top Five Traps in a "Content Supply Chain"

Wed, 02 Feb 2005 00:00:00 EST

Before the information age, car manufacturers only made cars, libraries only stored books, and newspapers only printed the news. Now, however, companies from all industries are realizing that in addition to what they do, they are also publishers, and there is a learning curve.

WSJ Exclusive: Bright Future for J2EE Web Services Development

Thu, 04 Nov 2004 00:00:00 EST

Web services will continue to play a vital role within enterprises, as companies strive to create cost-effective solutions that can be integrated into existing infrastructures. J2EE and Microsoft's .NET are the two primary platforms used in Web services. And while these two platforms continue to be actively developed, they are still in their infancy. How these platforms are developed is critical for the continued viability of Web services.

XML and Web Services: Connecting Information Islands

Fri, 20 Feb 2004 00:00:00 EST

The Web has become the world's greatest repository of information on anything and everything. It's extremely useful - as long as you can find what you're looking for. Despite the arrival of Google and other powerful search tools, information seekers can't always connect with the most relevant information.

Why We Need XML Query Standards

Wed, 07 Jan 2004 00:00:00 EST

Despite a shortage of sophisticated XML query tools, Internet demands have forced companies to present their data in various formats. In one sense little has changed, as SQL queries have long been used to combine data for different purposes and audiences.

Monitoring Air Pollution in Real Time Using XML - A reliable and cost-effective open source solution

Wed, 07 Jan 2004 00:00:00 EST

A few years ago there was no indication that XML could play an important role in computer-based process control. However, fast development and the spread of XML to different fields, and emerging trends in the field of automation, have changed the situation tremendously.

The Future of XML

Wed, 03 Dec 2003 09:51:51 EST

The end of the year is here again, a time when we traditionally take a long look at the progress we've already made and then turn our eyes toward the future, attempting to forecast the year to come.

The Key to Success with Web Services

Wed, 03 Dec 2003 09:49:53 EST

Web services provide a way to allow efficient communication between disparate services. For years, enterprises have struggled to find reliable, cost-effective ways to integrate and automate critical processes between different application packages.

Designing an Open, Standards-Based Reporting System - XML meets the challenges and design goals of a business reporting system

Wed, 03 Dec 2003 00:00:00 EST

As XML has grown more prevalent as a data delivery mechanism, so too has the need to use it for presentation in a wide variety of reporting formats. XML is useful for more than just the delivery of information, however.

Rise of the Standards-Based Integration Machines

Wed, 29 Oct 2003 00:00:00 EST

It occurs to me that my choice of title for this guest editorial may be at least partially influenced by the recall-induced elections in California (can you see the Arnie connection?). But this column is not about politics; it's about a new, industry-standard ecosystem built around XML to address today's business integration and process automation challenges.

Sarbanes-Oxley Will Change Your Life

Fri, 03 Oct 2003 11:05:06 EDT

This column may require a little patience on your part, but I think it will be worth it in the end. Let's start with a simple premise: within a year, nearly everyone reading these words will be deeply impacted by Sarbanes-Oxley, yet many have never heard of it.

XML in the Real Real World

Wed, 06 Aug 2003 11:51:00 EDT

When we at Antarctica start talking to a potential customer or partner, they say, 'That's Tim Bray's company? So this is XML-based data visualization?' And we have to say, 'No, it's ordinary database visualization. But because it's modern software, there's a lot of XML in the plumbing.' And in this there's a lesson.

Send Me Your Stylesheet

Tue, 01 Jul 2003 00:00:00 EDT

In these cynical, post-bubble times, most chief information officers are rightfully dismissive about new technologies that promise to boost efficiency or customer service...but once in a while the claims are very true. Stylesheets can make application development cheaper and faster while increasing customer satisfaction.

Leveraging XML Open Standards to Integrate Your Business

Thu, 22 May 2003 00:00:00 EDT

I've been focused on defining product strategy for business integration software for the past seven years. During that time I've watched XML go from being a fledgling document standard with lots of potential to a core technology that is critical for business integration. In this article, I'm going to discuss some of the reasons behind XML's meteoric rise in the business integration space and some of the ways we at IBM are leveraging XML in our integration products.

The New Role of XML

Tue, 29 Apr 2003 00:00:00 EDT

In the history of XML to date, its role in application development has been mostly on the edge - it has been used primarily as the format for applications to communicate with each other, as a way to serialize data or configuration information, or for some other use at the 'front door' of the application. The internal data model and processing that made applications run were entirely driven by objects (Java, C#, or what have you), relational database schema, and the like. Developers used the same approach to data modeling they always had and leveraged XML on the outside of their applications.

Who Owns Your Data?

Tue, 29 Apr 2003 00:00:00 EDT

The incompatibility of today's proprietary file formats goes well beyond the inconvenience of, say, unreadable e-mail attachments. It raises the larger issue of ownership - and cost of ownership. The data in your spreadsheet, the content in your business presentation, the words in your word processor - all of these belong to you. You created them.

WS-I and JCP: Creating Value for Enterprises

Tue, 29 Apr 2003 00:00:00 EDT

I'm frequently asked about the difference between portability and interoperability, and am often surprised at how many people refer to one when they mean the other. On the surface, the terms are pretty understandable: interoperability means that different systems will work together.

Moving Toward Convergence

Fri, 28 Mar 2003 00:00:00 EST

A few years ago at an early XML conference, an attendee made the point that XML was such a useful technology for data portability that it would eventually become ubiquitous - part of every tool, server, and application. He went on to predict that XML would become so commonplace that the idea of attending an XML conference would eventually seem silly. 'In a few years, a conference on XML will seem as ridiculous as a conference on ASCII would today,' he quipped.

Simplifying the Development of Transactional Web Apps

Fri, 28 Mar 2003 00:00:00 EST

The hardest part of writing transactional Web applications is finding a way to produce dynamic pages. The main underlying component of these pages, HTML forms, was added to what was originally a static, document-based standard, to allow the simple exchange of data between the user and the Web site. The more complex the information and the more sophisticated the interaction, the harder it's been to create these pages.

No Man Is an Island in the World of Pervasive Computing

Fri, 28 Mar 2003 00:00:00 EST

Do you want to understand our industry? Forget the big-name industry pundits and think-tanks. Look to the great poets like Donne and Shakespeare. You can't go wrong. The great poets can provide a long-term, human perspective on how we think, dream, and scheme. That insight is useful even in the new world of Web services and pervasive computing.

Integration Matters

Thu, 27 Feb 2003 00:00:00 EST

The challenge of integrating software and systems will always be with us. In the brief but turbulent history of information technology, creation and destruction go hand in hand. Old technologies and approaches give way to new ones, sometimes quietly and sometimes with a fight. Yet, in this maelstrom of activity one thing remains unchanged. Our desire to solve bigger and more important business problems breeds increasing complexity. To battle this complexity we divide and conquer. We don't want to reinvent the wheel.

Can Software Cross the Standards Divide?

Thu, 27 Feb 2003 00:00:00 EST

No standards movement in the history of the software industry has garnered as much attention or support as Web services. After the previous decade's failed attempts to reach unity, the industry has lauded the promise of more flexible, open, and interoperable software as a revolution and a breath of fresh air.

XML and Integration

Thu, 27 Feb 2003 00:00:00 EST

Better known in the i-technology world as enterprise application integration (EAI), B2B integration, or middleware, integration involves connecting internal systems with external business partners, customers, and suppliers. Integrating systems running on heterogeneous platforms, typically developed in different programming environments and managed by different groups (or different companies), is quite a complex task.

Behind the Firewall

Fri, 31 Jan 2003 00:00:00 EST

Questions about Web services and their uptake in financial services create a very black/white answer set. Some claim there's no usage; others say critical mass has been reached. The answer is somewhere in the middle.

Web Service Abstractions

Fri, 03 Jan 2003 00:00:00 EST

Joel Spolsky, a long-time blogger who frequently publishes interesting software development observations, recently wrote a seminal piece entitled 'The Law of Leaky Abstractions'(www.joelonsoftware.com/articles/ LeakyAbstractions.html). I encourage you to take a few minutes to read it, especially if you're involved with XML Web services. It summarizes one of the growing and troublesome trends surrounding Web services development and tools.

Industry Commentary

ILM Was Cloud Computing Buzzword de Jour A Few Years Ago

What Happened To SOA?

US Government & Federal Agencies Keen on Enterprise Architecture & SOA

Understanding the Value of Reference Architectures

More Linked Data and RDF

Demand for Lean, Green Business Technology

SOA: Integration vs. Business Process?

Scalability Only One Half the Reliability Equation

Google Jumps into the Cloud Wave

Does XML Have a Future on the Web?

AJAX: "XML for Client-Side Computing"

Out of Step - NIEM and N-DEx

Open Source Database Special Feature: An Introduction to Berkeley DB XML

Flexible Identity Federation XML Gateways to The Rescue

Untangling the Semantic Web

Overview and Analysis of the Department of Defense Discovery Metadata Specification (DDMS)

Top Five Traps in a "Content Supply Chain"

WSJ Exclusive: Bright Future for J2EE Web Services Development

XML and Web Services: Connecting Information Islands

Why We Need XML Query Standards

Monitoring Air Pollution in Real Time Using XML - A reliable and cost-effective open source solution

The Future of XML

The Key to Success with Web Services

Designing an Open, Standards-Based Reporting System - XML meets the challenges and design goals of a business reporting system

Rise of the Standards-Based Integration Machines

Sarbanes-Oxley Will Change Your Life

XML in the Real Real World

Send Me Your Stylesheet

Leveraging XML Open Standards to Integrate Your Business

The New Role of XML

Who Owns Your Data?

WS-I and JCP: Creating Value for Enterprises

Moving Toward Convergence

Simplifying the Development of Transactional Web Apps

No Man Is an Island in the World of Pervasive Computing

Integration Matters

Can Software Cross the Standards Divide?

XML and Integration

Behind the Firewall

Web Service Abstractions