Virtualization Journal: FCoE’s Role in Data Center Convergence
Or why a range of multi-protocol fabrics will be around for a while
Apr. 22, 2009 06:00 AM
It's a long-standing goal in enterprise networking to cost-effectively converge all local and storage area network (LAN and SAN) traffic on a single powerful infrastructure via one flexible, reliable, high-performance and low-latency protocol. And the future data center will, indeed, one day rely on some form of converged fabric with server and storage virtualization.
Only two of today's technologies stand to keep pace with Moore's Law and potentially serve as that sought-after unifying fabric: InfiniBand and Ethernet. This makes emergent Fibre Channel over Ethernet (FCoE) an especially promising interconnect candidate. Proponents of the protocol feel FCoE one day could shoulder all LAN and SAN traffic on the same, commonly managed, low-cost infrastructure - while delivering performance as fast, reliable and low latency as offered via highly sophisticated protocols such as InfiniBand.
There is a compelling value proposition for the adoption of FCoE on some types of midrange and rack-mounted servers. But, for many years to come, the real-world enterprise data center based on high-end servers will continue to support a range of existing multiprotocol fabrics. As always, the challenge faced by data center managers will be smartly matching technologies with applications to most affordably satisfy technical requirements and business objectives. Protocol-agnostic Wavelength Division Multiplexing (WDM) will continue to serve as the convergence cloud for an array of valuable protocols linking together multiple enterprise-class data centers as well as transporting converged fabrics where required.
For managers of enterprise data centers, FCoE's promise for cost-effectively converging Fibre Channel and Ethernet - the two dominant enterprise networking protocols - is too great to be overlooked. At the same time, its hype as today's single unifying fabric for all enterprise LAN and SAN traffic must be closely scrutinized.
A Precedent Has Been Set
In the excitement surrounding the rollout of a number of technologies - Fibre Channel, Asynchronous Transfer Mode (ATM) and InfiniBand, among them - there has acclaim for the arrival of the unifying interconnect fabric for all LAN and SAN services run across enterprises. This is compelling language to data center managers, who would gladly welcome the simplicities and cost efficiencies delivered by convergence given the dazzling pace of innovation in Internet, mobile, wireless, storage and other business capabilities that the data center must somehow accommodate.
Each of these protocols have served - and, in many cases, continue to serve - critical roles in enterprise networking. But the truth is that they have failed to deliver the complete transformation of convergence. In some instances, the various specific requirements of competing protocols have not been satisfied, or proprietary features have been too heavily emphasized. Or, in some cases, cost points have not been met because production volumes were insufficient.
Total LAN and SAN unification - managers of data centers have come to learn - is not achieved lightly.
Sorting the Hype from Reality
It must be noted that FCoE does have a lot going for it. Conventional Ethernet is being enhanced in areas such as flow control, Quality of Service (QoS), and packet loss, and developers are leveraging the lessons learned from previous convergence efforts in working toward an industry-standard FCoE protocol. The promise of this technology is considerable.
The open version of this new standard is called Converged Enhanced Ethernet, though some vendors use proprietary versions with different names. Thus, the preferred implementation of this standard would be Fibre Channel over Converged Enhanced Ethernet (FCoCEE), but for simplicity we will refer to this market as simply FCoE for now.
At the same time, it must be noted that the FCoE standards are just being finalized this year, which is an important factor for data center managers who prefer to base their infrastructure on open, international standards rather than proprietary implementations. As the industry prepares to roll out the first FCoE installations, there are a number of obstacles which must be overcome.
First, data center managers must select their converged fabric switch provider with care. For at least some switches, convergence of the SAN and LAN onto FCoE will require an upgrade of the existing Ethernet infrastructure with new low-latency Ethernet switches. This could amount to a major, disruptive and costly overhaul of the enterprise network core. Migration will not be a simple story of taking the existing corporate SAN, shifting it to the LAN and deploying the FCoE protocol. Data center managers will need to carefully re-examine their infrastructure partners, including several new companies just entering the market, to obtain the best value with minimal disruption. Companies would be well advised to partner with a server provider who understands when FCoE is appropriate, rather than those who tout this technology as one size fits all.
Beyond that, we already know that some of the proposed non-standard FCoE implementations on the table fail to sufficiently address real world issues such as latency, synchronous recovery and continuous availability - key requirements for some of today's most important enterprise data center applications. Once again, partnering with a server company that understands the end-to-end data center application will be important in the design of FCoE enabled data centers.
Server and Storage Virtualization Via FCoE?
Server virtualization allows enterprises to consolidate applications onto less hardware in order to lower costs. Prior to server virtualization, it was difficult to convert applications between different operating systems. By virtualizing the server it could, in principle, run any operating system (OS), enabling applications to be consolidated without converting them. This is one reason server virtualization has taken off. Storage virtualization simplifies today's increasingly complex storage environment, allowing organizations to more easily manage their infrastructures and consolidate storage systems from different vendors into one pool of storage.
Storage virtualization usually requires a SAN to go between the storage and servers, and has proven to cause the least disruption to these systems. This approach of placing the virtualization in the SAN between the storage and the server has been around for years yet somehow has languished while server virtualization has become more widely accepted. The problem again lies in the support of so many heterogenous storage and management systems that the "virtualized SAN" must support in a scalable fashion. Thus, the preferred location for virtualization services remains unclear, and will likely require a combination of server and network features to fully deliver the promise of converged networking.
Storage virtualization via FCoE is an intriguing concept, which some large data centers will soon be attempting to demonstrate across a spectrum of customer installations where Fibre Channel is currently the dominant protocol. The practical implications are far-reaching and hard to foresee. For instance, sharing one physical device among several logical partitions brings up a number of technical issues across links, servers, storage adapters, switches and points across the network - congestion, prioritization and queuing among them.
These technical problems are certainly not unsolvable, and the industry is working on an extension of networking standards and new features in server hardware and software to achieve such goals. Achieving these benefits will require data center managers who understand the full range of problems, and implement phased solutions to evolve the data center from its current design into something more energy and cost efficient. Initial deployments of FCoE will likely limit the use of FCoE to a chassis, such as within a rack or within a Blade Server. This usage model can be applied across multiple racks to obtain significant capital and operational cost savings. In the long term, as FCoE standards, hardware, software and management evolve, FCoE may one day enable full data center fabric convergence with associated economies of scale and resource savings. In the meantime, 8G Fibre Channel is an option on some of the latest FCoE switches that provides customers the flexibility to move to FCoE at their own pace.
Understanding Where FCoE Fits Today
The traditional data center compute model, especially in the case of x86 servers, has consisted of lightly utilized servers running a bare metal operating system or a hypervisor with a small number of virtual machines. In this environment, lower bandwidth links were sufficient for attachment to server resources. We see the industry moving to a dynamic infrastructure model that has highly utilized servers running many virtual machines per server, and using higher bandwidth links to communicate with virtual storage. This compute model offers several short-term business advantages to customers, including lower capital expenses through higher utilization of midrange servers, storage, and converged network fabrics, as well as lower operational expenses that are achieved through automated and integrated management that optimizes the end-to-end data center infrastructure. By contrast, enterprise servers that already offer high link utilization and make extensive use of virtual machine partitions will realize other long-term benefits.
There are scenarios where a data center manager might see FCoE as a wise choice today. For example, in cases where an enterprise needs to move a lot of raw data over a network for storage backup, then FCoE might be acceptable. Greenfield opportunities where FCoE's performance, reliability and latency levels would be acceptable might be a good place to experiment with the new protocol, by initially deploying FCoCEE at the chassis level as described earlier. Or, large, Internet-scale data centers might take a look at FCoE as a way to consolidate the LAN and SAN and reduce servers, adapters and cables. Finally, environments where low-cost connectivity is a higher priority than reliability - social networking sites and search engines, for instance - also might make a good fit for FCoE, where high-end "cloud computing" could represent a new market opportunity by utilizing existing data center assets. These sites always operate on a low-cost commodity model. FCoE might well be their answer to a reliable transport without ever investing in a separate Fibre Channel SAN network. FCoE is also expected to play well in the analytics and high-performance computing market. As FCoE matures and meets the performance, reliability and quality requirements of larger enterprise customers, we expect this technology will come to play well in these areas; this will also require addressing related technical issues, such as ensuring the enterprise server I/O subsystem design provides sufficient start rates and takes advantage of opportunities to increase link utilization.
One of Many Valuable Protocols
InfiniBand is a perfect example of a mission-critical high-end enterprise protocol of unique benefits that FCoE is not likely to render obsolete any time soon. Managers of most enterprise data centers will need to strategically and cost-effectively balance a host of factors - technology capabilities, distances to be linked, access options and cost issues, among them - and continue to run multiple protocols to most cost-effectively satisfy business objectives.
WDM, then, effectively becomes the convergence foundation for a data center's FCoE, InfiniBand and other legacy protocols when multiple data centers need to be interconnected over any significant distance. Flexibly and transparently multiplexing applications of any standard or non-standard protocol in "virtual channels" across optical fiber networks, WDM delivers the reliable, native-speed performance for multiprotocol traffic that real-world data centers will require moving forward.
For a number of critical reasons, it's not quite that easy. Some companies may release pre-standard (CEE and FCoCEE) products in early 2009. Clients who are evaluating deployment of such products should ask themselves several questions. Does the product support input from the CEE Authors group and the T11 FC-BB-5 standard? If not, will the vendor provide a free upgrade to the de-facto standard version in the future? Does the product support intermediary switches (i.e., a CEE Cloud), and if not, will the vendor provide a free upgrade to enable a CEE cloud in the future ?
Finally, there are real issues of management tools, behavior and organization that simply cannot be ignored in the convergence conversation. The management tools need to be vetted in real-world 24x7 environments. Today, the SAN and LAN are managed by two different organizations. Convergence implies that the server and storage groups within an enterprise need to entrust their GDPS STP, Enterprise System Connection (ESCON), Fiber Connection (FICON) and other well-established, mission-critical channels to the network group. The not-inconsequential political ramifications of such a transformation across the enterprise must be managed carefully, gracefully and gradually. Managers of large data centers should work with an IT company that has the service skills necessary to undergo this organizational transformation in careful steps.
For the foreseeable future, large data centers will continue to manage a variety of protocols when it comes to interconnecting over an extended distance. And that means WDM will continue to play its integral role in enterprise networking - as the flexible, protocol-agnostic cloud for converging all of the multi-protocol traffic, interconnecting critical data centers, and delivering necessary levels of performance, bandwidth and protection.
Reader Feedback: Page 1 of 1
Latest Cloud Developer Stories
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
SYS-CON Featured Whitepapers
Most Read This Week