From the very first steps this column took, a journey of discovery was promised which, hopefully, has not disappointed. Using Visual Cafe as our development platform, we have ventured into areas of Java that others have feared to tread. From building complete applets with database connectivity to developing classes for sending and receiving e-mail, this column has so far tackled a wide variety of different problems. On the face of it, many of the topics appear difficult or far too complicated to have in an ordinary applet, application or servlet. But I think you'll agree that once the initial wall has been scaled, the rest of the problem becomes somewhat academic.

My previous two columns looked at interfacing to two of the more popular TCP services; POP and SMTP. This month, we will round off the TCP services with an implementation for a generic class to interface to the Usenet newsgroup network.

Usenet
The Usenet, or newsgroups, is a place where users exchange information in topics of conversation. These range from computing right through to knitting. In order to view or posts news items, a Usenet viewer program is required. This is simply a program that interfaces to a Usenet server and retrieves and post news articles. One of the more popular Usenet clients for the PC platform is Free Agent. Free Agent gives an interface to the newsgroups, downloading and uploading appropriate articles. It is this functionality we will build into a class that will allow any Java program to have access to the Usenet.

Connecting to a Usenet server is a simple matter of opening up a socket connection to the server which usually sits and listens for client connections on port 119. The procedure for performing this can be seen in the complete listing of the class. Like the POP and SMTP servers, communication with the Usenet server is through command strings passed as carriage-returned lines.

The available command set to communicate with the server is surprisingly small, with only a handful of commands required to get the most out of a server. But before we go into the commands, let's take a quick look at how articles are arranged on the server.

Articles
Each article that exists on the Usenet system is identified with a global message id. This is normally created using a mixture of time/date/posters domain, to ensure complete uniqueness across all the newsgroup servers. When a new article is posted to a server, the server then distributes the article throughout the network. This procedure propagates the article throughout the whole network.

Associated with each article is a header. This header, shown in Listing 1, describes the article, which includes the e-mail of the poster, the subject of the article, the newsgroup the article has been posted in and the date it was posted (to name some of the more useful fields). Retrieving the article from the server is a simple matter of issuing the commands HEAD
BODY

where is the ID of the article, as stated in the Message-ID field of the article header. If the article exists, it will be sent back to the client. The header is sent first, then the body. With this, we can write a simple method to pull a particular article back from the server. Listing 2 illustrates this method.

Listing 2 shows the method, getArticle(...) which downloads the article from the server. First, it issues the HEAD command which asks the server to download the header of the given article. As each line is received, it is parsed so we may extract the e-mail address of the poster, the date it was posted and the subject of the article. If for some reason, the article is no longer available, then the server will return an error message line which contains a status code at the start of the line. If this code is either 423 or 430, we can assume the article is no longer available. If this is the case, the body is not downloaded and the method returns a null.

The body operates on somewhat the same principle, copying the lines of the article into a Vector for later use. Note that in this version we are actively filtering out all binary attachments as this can make the article body very large indeed. You may have noticed that the method returns a class of type cArticle. This is a simple wrapper class that represents a single Usenet article, with the body held in a Vector class.

Posting Articles
Posting an article to the Usenet server is much easier than downloading one. Using the command POST, the server will expect the head and then the body of the article, terminated with a single period (.) on a line of its own. The head is separated from the body by a single blank line.

Listing 3 shows the implementation of the postArticle(...) method. This method takes in the newsgroup the article is to be posted to, the e-mail of the poster, the subject and finally the actual body of the article formatted in a Vector, where each element is a single line.

The most important part of posting to the server is correctly formatting the header. If any of the syntax is wrong, then the article will be rejected. One of the features of the Usenet system is the ability to cross-post. This is where the same article is posted to multiple newsgroups at the same time. This is particularly handy as it saves on both bandwidth and time. To do this, simply list the newsgroups as a comma separated list. The server will then do the rest without any further intervention from the client.

Article Listings
So far we have seen how to post an article to the newsgroups, and how to request a particular article based on its message identification. However, this begs the question, how does one determine the message ids of the articles, so the client optionally may download them?

Fortunately, this is a fairly easy operation to perform. The server can be asked to return all the message ids for a particular newsgroup, since a certain date. For example, to request all the messages since the 9th of December 1997, midnight, for the newsgroup comp.lang.java.programmer, the following command would be issued

NEWNEWS comp.lang.java.programmer 971209 000000

The server will then return a list of message ids, each in their own line, with the list terminated using a period (.). This message id can then be used in the method described earlier to retrieve that article. Listing 4 demonstrates the communication with the server to determine the list of message ids.

The method returns a Vector of message ids back to the caller, or null if an error occurred. Again, as with the posting of articles, the formatting of the date is most important; otherwise, the server will recheck the command.

Summary
This article looked at the NNTP (News Network Transport Protocol) protocol, which is used to communicate with the Usenet servers. By building a simple class, the majority of the functionality required to send and receive articles is a simple method call away. Although only a subset of the available commands was implemented, other commands exist that allow further interrogation of the server. For example, a command exists that determines the available newsgroups.

Communication with the more popular TCP servers is performed using simple ASCII commands, which allow for very straightforward clients to be implemented. This is one of the real powers the Internet has to offer.

In the next column we will go further down the road of Java enlightenment and tackle another 'looks-far-too-complicated' topic area.