In January, as thousands of Linux OS enthusiasts streamed onto the show floor and into conference sessions at the 2004 LinuxWorld Conference & Expo, they caught up with an old friend who many barely recognized. In a span of only 12 months, Linux has evolved from an emerging technology to an operating system on the forefront of production IT deployments.

The breakneck evolution of Linux into a viable environment for enterprise, research and development, and high-performance computing has converted skeptics who once dismissed the open source environment as a low-cost - and not particularly scalable - alternative to more robust and proven operating environments.

To many commercial users, the earliest implementations of Linux didn't seem suited to drive anything more mission-critical than an edge server, some desktop systems, or a few specialized applications. But much has changed since those early days. Committed and enthusiastic developers have worked hard to push Linux beyond its old limitations, and the results have touched the industry. Sensing a good thing and seeing the opportunity to finally have a unified common operating system across applications and hardware, users and vendors are endorsing Linux as the "one-Unix" once sought by POSIX standards. The increased attention has resulted in added momentum behind Linux and increased expertise in the community that has allowed a multiplying effect of innovation to occur. At LinuxWorld 2003 in New York, attendees saw the unveiling of new systems designed to scale well past the operating system's storied eight-processor ceiling. Silicon Graphics, for instance, debuted a new Linux server and supercluster family - dubbed SGI Altix 3000 - that scales efficiently to 64 processors under a single Linux kernel. This product introduction has been accompanied by hardware vendors who continued to increase their support for Linux by introducing a variety of platforms (both 32- and 64-bit), in addition to growing momentum among ISVs. Support from the Linux community, hardware vendors, and developers made Linux ready for the most demanding customer environments. With Linux now poised for success at the highest levels, corporate bureaucracy and stagnant IT organizations constitute the last rapidly crumbling barrier.

In the past year, developers and users have seen that Linux, with a 2.4 kernel and enhanced software and utilities, has what it takes to drive massively scalable systems to solve HPC-class problems in commercial settings as well as in campus labs. Today, in fact, Linux habitually resets industry expectations of its scalability and applicability. As Linux on HPC platforms like the SGI Altix continues to shatter records on the highest end of computing, a rapidly growing number of end users are embracing Linux as a production environment for running everything from databases to computational fluid dynamics analysis.

Converging Trends
The rise of Linux has resulted from a confluence of crucial industry trends and the dedicated efforts of the open source community, with help from hardware and software solution providers.

64-bit Processor Technology
Cost-efficient processor technology helped prompt the Linux community's initial success in HPC by enabling hobbyists to join the global collaborative movement, and by adding to and improving the operating system's functionality. The relatively recent introduction of 64-bit technologies capable of unseating RISC processors has been vital to the development of Linux as a mature, robust, and performance-oriented operating system as an alternative to proprietary Unix implementations. The wide availability of 64-bit processors has spawned a new generation of powerful servers and supercomputers based on the Intel Itanium 2 family, AMD Opteron, and IBM POWER 4. These systems comprise a growing landscape of options for users, establishing a competitive marketplace for Linux servers that simply did not exist before. This competitive Linux server market is forcing innovation...and that's good.

HPC System Architectures
While standard "white-box" solutions inevitably find their way into any server market segment, they do little to push the boundaries of the Linux operating environment. Because price/performance is always a consideration for Linux users, the challenge for hardware vendors is to build sufficiently differentiated solutions from industry standard components.

In the HPC space, hardware vendors must meet the dramatically increasing expectation for systems to accommodate ever-growing data sets and computational needs of technical and enterprise users - needs not adequately addressed by standard solutions for those who solve complex problems in oil and gas exploration, manufacturing, aerospace design, earth and planetary sciences research, and life sciences. These applications tend to generate terabyte data sets, which in turn create I/O bottlenecks and demand enormous memory availability. As a result, they require system architectures designed to leverage each component to its fullest. SGI's built-in memory channel interconnect fabric with a 3.2GB/sec aggregate bandwidth, for instance, allows the system to deliver data across nodes five to six times faster than conventional clustering interconnects. Beyond this, Altix class systems uniquely provide up to 8 terabytes of total addressable memory. With system architectures affording so much throughput and memory availability, more complex data sets and complete workflows can be driven entirely out of memory, enabling productivity breakthroughs that traditional Linux can't tackle.

It was clear early on that the ability to scale Linux is imperative to meeting the needs of cutting-edge users. System architectures that support reliable scalability - whether scaling up from 4 to 16 processors, or from 256 to 512 - are also vital to Linux evolving as a serious multidiscipline enterprise computing environment. NASA Ames Research Center recently installed an SGI Altix 3000 system that demonstrated 512 processors in a single system image - the largest Linux supercomputer of its kind ever created. By scaling in near-linear fashion to more processors than even some Linux enthusiasts thought possible, the NASA system was the first computer in history to break the 1 terabyte/second barrier on the STREAM Triad benchmark, an industry-standard measurement of memory bandwidth. By the standards of most users, NASA's needs are extreme. But its efforts to throttle Linux into performance overdrive offer a telling example of what Linux can do on the right platform. What's more, it credibly illustrates the Linux community's ability to outpace proprietary solutions, and provides a proof-point of how far Linux can penetrate industry and science in particular.

64-bit Applications
With the initial success of these systems has come growing support from commercial software suppliers, whose customers hungrily demand ever-heightening levels of system scalability. Customer adoption has fueled aggressive Linux porting and development among ISVs - some of the world's most powerful and sophisticated technical applications now run on Linux: BLAST, GAMESS, NX.Nastran, STAR-CD, and Gaussian. Meanwhile, leading database applications from Oracle, Sybase, and MySQL provide even more options for high-performance production Linux environments.

Linux has also proven itself outside of its old perceived boundaries. By expanding its reach to end users in a vast array of markets, it's increasingly clear that Linux can serve as a unifying operating environment, rather than the affordable substitute applicable only to specific systems or application areas.

Ready for Prime Time
As the attendees of this year's LinuxWorld discovered, Linux is on an evolution curve unlike any other operating system in history. The achievements of just the past year have helped shape this OS into a production-quality environment. Customers around the world depend on it for mission-critical applications, including Tata Motors, NASA, the University of Cambridge COSMOS Project, SARA (Dutch National HPC and Networking Center), Marathon Oil Co., and the Pacific Northwest National Laboratory.

Some are first-time Linux users; others, long-time devotees. But whether investigating the origins of the universe, predicting the effects of global warming, or hunting for oil beneath the North Sea, all of these users reap the benefits of each other's efforts. They are part of a global Linux community that, every day, astounds the world with its achievements.

By all accounts, 2004 will be another groundbreaking year for everyone developing, supporting, and deploying Linux. Just imagine what surprises are in store for LinuxWorld 2005.

64-bit Linux: A Community Project
Within the open source community, SGI has been committed to taking a leadership role in the ongoing development and improvement of the performance and reliability of the Linux operating system. With its focus on high-productivity computing environments, SGI has contributed to Linux scalability, scheduling, memory usage, I/O, and other efforts critical to high-demand application performance.

Linux Kernel Work

• CpuMemSets (processor and memory placement)
• kdb (Linux kernel debugger)
• Kernprof (kernel profiling)
• Lockmeter (Linux kernel lock-metering)
• DISCONTIG (original DISCONTIG memory support for NUMA)

• PAGG (process aggregates)

• XFS (high-performance journaling filesystem)
• fam & imon (file alteration monitor and inode monitor)

• GLX (OpenGL extensions to X)
• OpenGL Sample Implementation (standard cross-platform 3D and 2D graphics API)
• Open Inventor (object-oriented toolkit for interactive 3D graphics)

• Audio File Library

• Linux on Large Systems Foundry
• Linux Scalability Effort Project

• Performance Co-Pilot (system performance monitoring and management framework)
• LKCD (Linux kernel crash dumps)
• ob1 (sample implementation of a trusted operating system)
• LTP (Linux test project)
• Rhino (infrastructure for system administration applications)