Industry Commentary

The New ROI: A Return on Innovation, and Using Data to Do It

Tue, 15 Nov 2011 07:00:00 EST

What does American Express, Foursquare, Etsy, and NYC OPEN Government all have in common? They all need to innovate through data...and they need to go outside to do it big. At first glance one cannot help take note of the almost impossible task of merging the words "American Express" and "hackathon" under one event umbrella, but Scott Roen of Amex OPEN Forum did just that. He suspected before the hackathon that innovation could in fact come from the outside, even if the "outsiders" (aka developers) break a company's long-held views of what innovation even looks like. CIOs and CMOs should both take notice. Innovation is possible once again, though admittedly it takes thick skin to run a hackathon at a Fortune 500. You might not like what developers and data partners say about your brand and relevance in the digital space.

Are You Your Own Worst Enemy?

Sat, 22 Oct 2011 10:00:00 EDT

In building, marketing and selling software, the biggest enemy isn’t the competition. Or “rivals” at work. It's YOU and the people you trust the most. Your biggest enemies are smart - sometimes blindingly smart. They're confident, and convincing. And incredibly dangerous - because you trust them, and you think they're helping you succeed. They may not be guiding your boat right into the rocks, but they're probably not taking you where you need to be. And the difference between survival, success, and phenomenal success can come down to very slight variances in navigation over time.

$3 Trillion Problem: Three Best Practices for Today's Dirty Data Pandemic

Sat, 10 Sep 2011 12:00:00 EDT

Incorrect, inconsistent, fraudulent and redundant data cost the U.S. economy over $3 Trillion a year - an astounding figure that is over twice the amount of the 2011 Federal Deficit. In survey after survey, about half of IT executives consistently agree that data quality and data consistency is one of the biggest roadblocks to them getting full value from their data. I've long been a proponent of healthy software – but healthy software can only function properly in the presence of healthy data. Does quality software even matter if the underlying data are defective? Agreed – that’s pushing the point to the extreme. How much does this dirty data pandemic cost us? What can be done about it?

How I Became a REST 'Convert'

Sun, 17 Jul 2011 15:00:00 EDT

Representational State Transfer, commonly known as REST, is a style of distributed software architecture that offers an alternative to the commonly accepted XML-based web services as a means for system-to-system interaction. Many of you know me as one half of the ZapThink team – an advisor, analyst, sometimes-trainer, and pundit that has been focused on XML, web services, service oriented architecture (SOA), and now cloud computing over the past decade or so. Some you may also know that immediately prior to starting ZapThink I was one of the original members of the UDDI Advisory Group back in 2000 when I was with ChannelWave, and I also sat on a number of standards bodies including RosettaNet, ebXML, and CPExchange initiatives. Furthermore, as part of the ZapThink team, I tracked the various WS-* standards from their inception to their current “mature” standing.

Opinion: Who Made Amazon the Judge of What's Legal on the Web?

Tue, 07 Dec 2010 07:45:00 EST

[This post originally appeared on Joseph Galarneau's blog and is republished here in Cloud Computing Journal by kind permission of the author.] When you visit Newsweek.com, the words you read started their journey milliseconds earlier from an Amazon.com datacenter somewhere in northern Virginia. And if you visited Wikileaks.org earlier this week, the bytes comprising leaked U.S. embassy cables would have traveled a similar path from Amazon servers based in northern California or Ireland.

China Outsourcing Offers Cheaper Alternative to India

Sun, 14 Nov 2010 14:57:00 EST

For the last ten to twenty years, outsourcing to India has long dominated the outsourcing landscape. But that appears to be changing with recent emerging players raising their heads to take advantage of the outsourcing game. Although there are some good options for outsourcing to Latin America, Eastern Europe and Russia, Mexico and other Asian counterparts like Vietnam and the Philippines, China is touted take the lead in the next few years as the leading outsourcing country. Despite worries that the Philippines might become the biggest business process outsourcer in a decade or so, China is closing in on the gap within the industry. The clear advantage that China brings to the table is the six billion people or more that populate the mainland. With more and more engineers graduating from Chinese universities, there is scope for more work to come their way.

$55 Per Hour Software Developers

Mon, 24 May 2010 09:45:00 EDT

Last week, a US based Flex/Flash developer who IMO belongs to the top 20% Flash developers twitted that he was contacted by a recruiter offering a contract paying $55 per hour. While $55 per hour may sound a lot in some of the developing countries, the cost of living in the USA makes this rate a joke. And we are not talking about some rookie who learn Flex last week in a classroom. Pretty sad, isn’t it?

Next day, a manager of a large corporation asked me if our company can provide them with a senior Flex/Java consultant working onsite in a greater New York area. The rates were $60-$80 per hour plus this software developer would have to go through another consulting company that was on a preferred vendors list of this large corporation. This middleman would also need to get a cut from this rate. Let’s do some math together assuming that the corporate client is willing to pay $80 p/h for this developer.

The preferred vendor will shave off, say, eight bucks. Our company has bills to pay too, and let’s say we’ll take $12. This means that we can pay $60 per hour to a senior Flex/Java developer living in the USA. I can’t find a plumber in our geographical area who’d be willing to clean my toilet for $60 per hour. Last week I had to bring my car to a repair shop to replace worn out brake pads. They charged me $90 per hour for labor.

I deeply regret that I can’t outsource cleaning toilets and changing pads to one of the less expensive countries.
So what does it mean for Flex/Java development in the USA? These are the choices that the enterprise development managers will face:

1.   Hire anyone who knows how to spell Binding, AMF, and Servlet and keep he fingers crossed that these guys won’t bring the project to a full stop.

2.   Outsource the software development to another country where Flex senior developers started to lay eggs and breed.

3.   Stop fooling around and offer reasonable rates in the USA to bring local talent to the project.
In the past, the Wall Street companies were known for selecting the last option. During the last two years their habits changed and they go number two.

Other industries do a number one. When a development manager smells troubles, s/he hits the Panic button and tries to switch gears to select number three, which may not be available cause the good spellers ate almost all the budget already.

Dear corporate hiring manager! Don’t fool yourself. There’s no free lunch. I know, the rules in your company changed, and the HR rats wrote the instructions that tie your hands down. Still, fight with them to make an exception that would allow you to bring a real talent on board. Your career is at stake here.

Is This the End of Enterprise Software?

Mon, 15 Mar 2010 10:45:00 EDT

The future of enterprise software is the platform. No software company has all the smart people in the world on its payroll. Apple recognized this and created the App Store. Google are now doing the same. These new cloud platforms will unleash unprecedented innovation and creativity throughout all areas of business and the web, with thousands of new apps being created that will revolutionize IT.

Enterprise Cloud Computing Requires Service-Level Discipline

Fri, 11 Dec 2009 13:00:00 EST

The Enterprise Cloud Requires a real time infrastructure and a management discipline that understands and can enforce service level discipline. Organizations have become increasingly dependent on technical infrastructure to enable customer interactions. As such, the business has a vested interest in making sure its technology partners understand what constitutes good customer experience so that it’s prepared for projected volumes and rapidly knows how to resolve any impediments.

ILM Was Cloud Computing Buzzword de Jour A Few Years Ago

Thu, 26 Nov 2009 10:45:00 EST

Disclaimer, warning, be advised, heads up, disclosure, this post is partially for fun so take it that way. Remember ILM, that is, Information Lifecycle Management among other meanings. It was a popular buzzword de jour a few years ago similar to how cloud is being tossed around lately, or in the recent past, virtualization, clusters, grids and SOA among others. One of the challenges with ILM besides its overuse and thus confusion was what it meant, after all was or is it a product, process, paradigm or something else?

What Happened To SOA?

Sun, 04 Oct 2009 17:00:00 EDT

Is SOA dead? This question keeps popping up every now and then within the IT community and though we all seem to agree that SOA is far from dead something has indeed radically changed: our expectations. Just a few years ago, perhaps months, we considered SOA to be the solution to all of our (IT) problems, promising to unleash the power of our business, and that just by thinking 'services'!

US Government & Federal Agencies Keen on Enterprise Architecture & SOA

Mon, 17 Aug 2009 09:15:00 EDT

Over the past two years, governments around the world have caught the enterprise architecture and SOA fever. This is especially the case in the US Federal Government, where a number of regulations, EA frameworks, and major spending initiatives are all pointing towards continued and lasting expenditure and investment in the area of EA. One can also notice this trend based on the number of EA and SOA events now focused on the US Federal Government, or at least taking place in the Washington, DC area. For example, ZapThink’s Practical SOA event in DC on October 2, 2009 is focused on cost-effective legacy enablement and data management, while its 4-day licensed ZapThink Architect (LZA) boot camp running from September 21-24, 2009 provides SOA certification and credentialing to government and other DC-area firms. ZapThink’s SOA & Cloud governance certification and training class in DC sold out earlier this month (August) and was so popular the company is running it again, in DC, September 30-October 1.

Understanding the Value of Reference Architectures

Mon, 10 Aug 2009 12:30:00 EDT

There’s nothing more that architects love to do than argue about definitions. If you ever find yourself with idle time in a room of architects, try asking for a definition of “Service” or “architecture” and see what sort of creative melee you can start. That being said, definitions are indeed very important so that we can have a common language to communicate the intent and benefit of the very things we are trying to convince business to invest in. From that perspective, a number of concepts have emerged in the past decade or so that have become top of mind for self-styled enterprise architects: architecture frameworks and reference architectures. In previous ZapFlashes, we discussed architecture frameworks, which leaves the topic of reference architectures left untouched by ZapThink. Since we can’t leave a good argument behind, we’re going to use this ZapFlash to explore what reference architectures are all about and what value they have to add to the Service-Oriented Architecture (SOA) story.

Demand for Lean, Green Business Technology

Wed, 15 Jul 2009 09:45:00 EDT

According to a recent market study by Datamonitor, the current global economic recession may also prove to be a significant driver for Green Computing. Their market assessment raises lots of questions -- including, is it better for the world, and overall business profitability, if executives cut-back on their IT investments? "The global economic recession has spurred a paradigm shift in the way organizations evaluate, budget for and deploy green IT," says Rhonda Ascierto, senior analyst at Datamonitor. "The downturn has also resulted in green IT trends for datacenters, client devices and asset lifecycle management, as well as re-shaped return on investment (ROI) models."

SOA: Integration vs. Business Process?

Mon, 13 Jul 2009 11:15:00 EDT

Ever since ZapThink coined the term "Service Oriented Integration" (SOI) back in 2002, there's been unceasing confusion on just how Service-Oriented Architecture (SOA) and integration relate to one another. Several recent blog posts have refocused attention on this confusion, including ones from Anne Manes, Todd Biske and Loraine Lawson. Perhaps some of the confusion is over the definition of the SOI term, and what distinguishes SOI from Web Services-based integration, if anything. But the bigger controversy is over the larger question as to SOA's fundamental purpose: is SOA's purpose to solve integration problems, or is it more of a business transformation approach centered on implementing agile business processes?

Scalability Only One Half the Reliability Equation

Fri, 10 Jul 2009 09:00:00 EDT

Without availability scalability is irrelevant

I really enjoyed Jeff Atwood’s recent blog on Scaling Up vs Scaling Out, which includes a fairly detailed comparison of the costs associated with each approach to scalability. I enjoyed it because not only did it take into consideration the cost of hardware, but also remembered to include the cost of software licensing. And of course there’s the fact that Jeff’s site is focused on development and coding, and this discussion broadened the discussion into the realm of application networking – a demesne with which I am of course particularly fond.

Now the problem is – and you knew there had to be one, this is me after all – that the discussion is primarily about scalability. Scalability is good, of course, but it’s not the whole story. The discussion stemmed from an inability of a real server to meet capacity which resulted in the dread d-word: downtime.

Scalability is of course a Very Good Thing. There’s no disputing the fact that it’s absolutely critical to ensure that applications scale – one way or another. But simply ensuring scalability should not be confused with providing reliability because even a scalable application can end up “unavailable” despite having access to vast amounts of resources.

WHAT AVAILABILITY REALLY MEANS

The other half of the reliability equation is availability. At first glance it may appear that scalability automatically ensures availability, but that’s simply not true. There are several reasons why these two are not synonyms and should not be treated as interchangeable. First, in a business environment availability is not just accessibility – which is ensured through scalability - but timely accessibility. An application that does not meet its agreed upon service level agreements is not considered available. Abandonment rates increase dramatically as performance degrades and productivity plummets when applications are slow to respond.

Second, availability can be affected by other mitigating factors such as security. The recent (and ongoing) attacks on U.S. government websites, for example, are primarily denial of service attacks. Such attacks are focused on consumption of resources, and not just server resources but network resources as well. After all, it doesn’t matter how fast an application responds to a request if the response is going to be “stuck in traffic” as it were. A successful attack on the application targeting some known vulnerability, too, can result in the dread d-word by causing crashes or otherwise corrupting the application’s ability to service requests.

Thirdly, availability can be affected by applications that perform correctly at nominal load levels but which become unstable at high load levels. The simplest of examples of such a situation involves database connectivity. At high load levels it is often the case that an application server’s connection pool is simply not robust enough to keep up with demand. No amount of scaling up will solve this problem, because it isn’t a hardware resource issue, it’s a software resource issue that isn’t even under the control of the developers because the connectivity is provided by a third-party – the web or application server vendor. While the application server may be responding even under heavy lead, an application without data is really quite useless and, like failure to meet service level agreements, by business standards it is considered unavailable.

ACHIEVING RELIABILITY

In order to reach reliability nirvana, i.e. five 9s with consistent adherence to service level agreements, you cannot rely on scalability alone. It just doesn’t address all the factors that are necessary to ensure the availability required for a reliable application or site. You need to address three potential points of failure in the application architecture to avoid potential downtime:

Performance
Adherence to service level agreements are, admittedly, a pain in the proverbial rear-end because it seems like so many factors are often out of the control of the developer because they are. And yet developers are the ones often blamed for poor application performance. The application infrastructure needs to provide tools and mechanisms that can counter the natural tendency of application performance to degrade as capacity increases.
Security
Security needs must be addressed and not just the typical application vulnerabilities. Security needs to be aware of the potential for resource consuming attacks for which the purpose is to take a site or application “offline” as opposed to exploitation for other purposes. The security architecture must address both network and application security as a means to prevent an application from becoming unavailable.
Functional correctness
Applications must respond – and respond as expected – under myriad levels of load. In order to ensure this is true the application must be monitored in such a way as to report not just network or transport layer failures, but application response errors. This requires intelligent monitoring – the ability to not only check connectivity, but validate content for correctness.

None of these can really be addressed by the application itself, because the measurements and monitoring required must occur outside the application. That means some other solution must be responsible for providing availability of applications.

The best solution is a unified application delivery solution, because it brings together – as the term implies – the disparate external functions necessary to availability assurance in a single solution. Application and network security, bandwidth management, acceleration and optimization are all a part of application delivery platforms, and it is that platform that will provide the best chance for realizing true reliability of applications and web sites through the assurance of availability. A solution such as a unified application delivery platform is external to the application and, if you’re scaling out, you’ll need its core functionality of a load balancer anyway. This means it can monitor for functional correctness, provide network and application security, and offers a variety of acceleration and optimization features that can manage performance needs.

Technorati Tags: MacVittie,f5,application delivery,security,availability,reliability,scalability,monitoring,unified application delivery and data services,failure,web,internet,blog,coding horror,Jeff Atwood,infrastructure,architecture,development

Related blogs & articles:

Categories: Development and General , Performance , Monitoring/Management , Security

Google Jumps into the Cloud Wave

Thu, 28 May 2009 12:55:00 EDT

Busy day for cloud interoperability related news. Google just announced a new service called Google Wave, described as an open communication and collaboration platform & protocol based on hosted XML documents (called waves) supporting concurrent modifications and low-latency updates. In simple terms Google Wave can be thought of like an ajax spreadsheet over XMPP.

Does XML Have a Future on the Web?

Thu, 10 Jan 2008 08:00:00 EST

A more interesting question is 'Is XML on the web trending up or trending down?' Clearly, it is trending down. For data transfer applications, XML is losing ground to JSON because JSON is simply a better data transfer format. And XHTML has failed to displace HTML in the marketplace. The benefit of clientside validation has proven to not be a benefit. I think you can argue, and in fact I did argue, that because of W3C's adventures with XML, the web itself may not have a future. The browser has a lot of problems, the worst of which are the security problems that came with Netscape Navigator 2. That was 12 years ago, and there has been no progress since that time in fixing the fundamental problems. There have been lots of patches on top of patches. Nothing more.

AJAX: "XML for Client-Side Computing"

Mon, 25 Sep 2006 08:45:00 EDT

XML is a simple, flexible text format initially designed for large-scale electronic publishing. It is flexible, open, and human-readable, and can be learned easily. XML can also be generated, parsed, analyzed, and transformed easily. It's no wonder that XML has been widely used for server-side computing: J2EE, .NET, and Web services.

Out of Step - NIEM and N-DEx

Wed, 25 Jan 2006 20:15:00 EST

Since the horrific events of September 11, 2001, the federal government has intensified its efforts to improve communications, collaboration, and information sharing between government and private sector agencies at all levels. The task of creating a seamless system of data and communication between disparate agencies has faced both technological and political obstacles.

Open Source Database Special Feature: An Introduction to Berkeley DB XML

Tue, 20 Dec 2005 11:45:00 EST

In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

Flexible Identity Federation XML Gateways to The Rescue

Fri, 16 Dec 2005 19:45:00 EST

Imagine a fresh business relationship between ACME Corporation and Partner. As a result of this relationship, ACME wants to grant Partner limited access to one of its core internal applications. They do this, naturally, by exposing a Web service.

Untangling the Semantic Web

Fri, 18 Nov 2005 10:00:00 EST

The Semantic Web is a hot topic in information circles today, and its adoption will largely depend on stakeholders understanding its potential benefits and tools vendors providing an easy entry for developers to learn and work with its related technologies.

Overview and Analysis of the Department of Defense Discovery Metadata Specification (DDMS)

Wed, 01 Jun 2005 10:00:00 EDT

The Department of Defense (DoD) Discovery Metadata Specification (DDMS) describes the DOD's preferred approach for decorating data assets with metadata. By providing a common convention for metadata, the DoD is building a common system for asset discovery, search, description, consumption, and security. This article provides a summary of the DDMS's purpose, structure, and capability. Upon completion the reader should have a basic understanding of the DDMS and should know where to go to get more detail and related materials. All questions regarding this article should be directed to Michael Sick at mike@serenesoftware.com.

Top Five Traps in a "Content Supply Chain"

Wed, 02 Feb 2005 00:00:00 EST

Before the information age, car manufacturers only made cars, libraries only stored books, and newspapers only printed the news. Now, however, companies from all industries are realizing that in addition to what they do, they are also publishers, and there is a learning curve.

WSJ Exclusive: Bright Future for J2EE Web Services Development

Thu, 04 Nov 2004 00:00:00 EST

Web services will continue to play a vital role within enterprises, as companies strive to create cost-effective solutions that can be integrated into existing infrastructures. J2EE and Microsoft's .NET are the two primary platforms used in Web services. And while these two platforms continue to be actively developed, they are still in their infancy. How these platforms are developed is critical for the continued viability of Web services.

XML and Web Services: Connecting Information Islands

Fri, 20 Feb 2004 00:00:00 EST

The Web has become the world's greatest repository of information on anything and everything. It's extremely useful - as long as you can find what you're looking for. Despite the arrival of Google and other powerful search tools, information seekers can't always connect with the most relevant information.

Why We Need XML Query Standards

Wed, 07 Jan 2004 00:00:00 EST

Despite a shortage of sophisticated XML query tools, Internet demands have forced companies to present their data in various formats. In one sense little has changed, as SQL queries have long been used to combine data for different purposes and audiences.

Monitoring Air Pollution in Real Time Using XML - A reliable and cost-effective open source solution

Wed, 07 Jan 2004 00:00:00 EST

A few years ago there was no indication that XML could play an important role in computer-based process control. However, fast development and the spread of XML to different fields, and emerging trends in the field of automation, have changed the situation tremendously.

The Future of XML

Wed, 03 Dec 2003 09:51:51 EST

The end of the year is here again, a time when we traditionally take a long look at the progress we've already made and then turn our eyes toward the future, attempting to forecast the year to come.

The Key to Success with Web Services

Wed, 03 Dec 2003 09:49:53 EST

Web services provide a way to allow efficient communication between disparate services. For years, enterprises have struggled to find reliable, cost-effective ways to integrate and automate critical processes between different application packages.

Designing an Open, Standards-Based Reporting System - XML meets the challenges and design goals of a business reporting system

Wed, 03 Dec 2003 00:00:00 EST

As XML has grown more prevalent as a data delivery mechanism, so too has the need to use it for presentation in a wide variety of reporting formats. XML is useful for more than just the delivery of information, however.

Rise of the Standards-Based Integration Machines

Wed, 29 Oct 2003 00:00:00 EST

It occurs to me that my choice of title for this guest editorial may be at least partially influenced by the recall-induced elections in California (can you see the Arnie connection?). But this column is not about politics; it's about a new, industry-standard ecosystem built around XML to address today's business integration and process automation challenges.

Sarbanes-Oxley Will Change Your Life

Fri, 03 Oct 2003 11:05:06 EDT

This column may require a little patience on your part, but I think it will be worth it in the end. Let's start with a simple premise: within a year, nearly everyone reading these words will be deeply impacted by Sarbanes-Oxley, yet many have never heard of it.

XML in the Real Real World

Wed, 06 Aug 2003 11:51:00 EDT

When we at Antarctica start talking to a potential customer or partner, they say, 'That's Tim Bray's company? So this is XML-based data visualization?' And we have to say, 'No, it's ordinary database visualization. But because it's modern software, there's a lot of XML in the plumbing.' And in this there's a lesson.

Send Me Your Stylesheet

Tue, 01 Jul 2003 00:00:00 EDT

In these cynical, post-bubble times, most chief information officers are rightfully dismissive about new technologies that promise to boost efficiency or customer service...but once in a while the claims are very true. Stylesheets can make application development cheaper and faster while increasing customer satisfaction.

Leveraging XML Open Standards to Integrate Your Business

Thu, 22 May 2003 00:00:00 EDT

I've been focused on defining product strategy for business integration software for the past seven years. During that time I've watched XML go from being a fledgling document standard with lots of potential to a core technology that is critical for business integration. In this article, I'm going to discuss some of the reasons behind XML's meteoric rise in the business integration space and some of the ways we at IBM are leveraging XML in our integration products.

The New Role of XML

Tue, 29 Apr 2003 00:00:00 EDT

In the history of XML to date, its role in application development has been mostly on the edge - it has been used primarily as the format for applications to communicate with each other, as a way to serialize data or configuration information, or for some other use at the 'front door' of the application. The internal data model and processing that made applications run were entirely driven by objects (Java, C#, or what have you), relational database schema, and the like. Developers used the same approach to data modeling they always had and leveraged XML on the outside of their applications.

Who Owns Your Data?