(IDG News Service) — A security vulnerability in one of the most commonly used e-mail server software packages could have a wide ranging impact, akin to the Microsoft Corp. SQL Server vulnerability that spawned the recent Slammer worm, according to an advisory published Monday by Internet Security Systems Inc. (ISS).

The buffer overflow vulnerability was found in a number of versions of the open source Sendmail Mail Transfer Agent (MTA), ranging from the most recent release of that software to versions that first appeared in the late 1980s. The vulnerability could allow a remote attacker to gain "root" (superuser) access to a Sendmail server, according to ISS.

Sendmail is the most popular Unix-based implementation of the Simple Mail Transfer Protocol (SMTP), which is used to transmit e-mail messages. Predating the modern Internet itself, Sendmail is used to process incoming e-mail messages.

A vulnerability exists in the software code that is used to evaluate whether addresses in the header field of an e-mail message are valid.

Attackers who understand the vulnerability could compromise a server by sending an e-mail message with an improperly formatted message header, causing a buffer overflow that would enable the attacker to place and execute their own malicious code on the server.

What makes the new vulnerability particularly pernicious is that attackers would need to know little about the server they were attacking other than its Internet address, according to Dan Ingevaldson, team leader of X-Force research and development at ISS.

"It's quite a dangerous vulnerability because an exploit could be contained in the e-mail message itself. The attacker doesn't need to set up an elaborate system to launch the attack. They could just send an e-mail message to a server, and if the server is vulnerable the attack would be launched," Ingevaldson said.

While the vulnerability requires sophisticated knowledge of the Sendmail program to understand and exploit, it could still be quickly leveraged by hackers in the form of a Slammer-like worm, according to Ingevaldson.

Part of the reason for that is that, as an open source product, the Sendmail source code and the new patch code are visible to hackers as well as e-mail server administrators. The recently released patch will immediately flag vulnerable areas of the Sendmail code.

Once the vulnerability is understood and an exploit is developed, it would be easy work to join that exploit to an engine that scans for messaging servers, creating a fast-spreading and dangerous new worm, Ingevaldson said.

"There are only a handful of services that firewalls are powerless to block. Things like DNS (domain name system), e-mail and Web have to be exposed -- just hanging out on the open Internet and taking anything that comes to them," Ingevaldson said.

ISS discovered the vulnerability in late 2002, according to Ingevaldson. The company "sat on it for a little bit," eventually contacting the nonprofit group that manages the Sendmail code base in addition to commercial Sendmail vendors and the National Infrastructure Protection Center (NIPC), which is now part of the U.S. Department of Homeland Security.

As a patch was developed in the months that followed, the NIPC coordinated with government entities and critical infrastructure owners to make sure that those organizations got out in front of the problem.

Ingevaldson offered praise for the NIPC's efforts.

"It was a great experience. I think it's the first time something of this scale was managed in this way, and I thought it was a big success," Ingevaldson said.

Patches for the vulnerability were available for both open source and commercial Sendmail distributions as of Monday afternoon [see: http://www.sendmail.org].

The combination of freely visible source code, a severe and remotely exploitable vulnerability, and an enormous installed base of vulnerable servers make the new Sendmail vulnerability an extremely high-value target for the hacking community, according to Ingevaldson.

That means that it is critical for affected organizations to patch their servers, Ingevaldson said.

"Once an exploit is published, all bets are off," Ingevaldson said. "The window of vulnerability has decreased. We've seen very robust powerful exploits released within a few months of the exploit being published, so if (patching) was not a big deal before, it is now."