20.4. Case Study: The Streamer Service

You've seen some of the beauty of CIP at the architectural level. It's time to focus on one of its middleware services—the streamer service—as a case study, and examine some of the nails that allowed us to meet the mission's strict functional, reliability, and robustness requirements. You'll see that the nails were not particularly fancy; the beauty was in knowing just where to pound them in.

20.4.1. Functionality

One of the MER mission's data management needs is to allow users to download data and image files from the mission file servers located at JPL to their personal workstations and laptops. As described earlier, CIP data-tier utilities generate metadata that allow users to find the files they want based on various search criteria. Users also need to upload files that contain their analysis reports to the servers.

CIP's streamer service performs file downloading and uploading. We gave the service that name because it streams the file data securely across the Internet between the mission file servers at JPL and users' local computers. It uses the web services protocol, so client applications can be written in any language that supports the protocol, and these applications are free to devise whatever GUI they deem suitable.

20.4.2. Service Architecture

Like each of the other middleware services, the streamer service uses web services to receive client requests and to return responses. Each request is first fielded by the streamer service provider, which is implemented by a stateless session bean. The service provider creates a file reader, implemented by a stateful session bean, to do the actual work of downloading the requested file contents to the client. Conversely, the service provider creates a file writer, also implemented by a stateful session bean, to upload file contents (see Figure 20-3).

Figure 20-3. The streamer service architecture

At any given moment, multiple users can be downloading or uploading files, and any single user can also have several download or upload operations going at once. So, there can be numerous file reader and file writer beans active in the middleware. A single stateless Streamer Service Provider bean handles all the requests, unless the load becomes heavy, at which time the application server can create more provider beans.

Why does each file reader and file writer have to be a stateful session bean? Unless the file is small, the streamer service transfers the file contents one block at a time in response to "Read Data Block" or "Write Data Block" requests from the client. (The download block size is configurable on the middleware server. The client application chooses the upload block size.) From one request to the next, the stateful bean keeps track of the open source or destination file on the mission file servers and the position within the file of the next block to be read or written.

This is a very simple architecture, but it very effectively handles multiple downloads simultaneously from multiple users. Figure 20-4 shows the sequence of events for downloading a file from the mission file servers to a user's local machine.

Figure 20-4. How the two-layer service handles a file read

Note that the Streamer Service Provider bean does not maintain any state between service requests. It functions as a rapid service dispatcher that parcels work out to the stateful File Reader beans. Because it doesn't need to track requests or maintain state, it can handle requests intermingled from several client applications. Each File Reader bean maintains state information (where to get the next block of data) for a single client application as the application makes multiple "Read Data Block" requests to download a complete file. This architecture enables the streamer service to download multiple files to multiple clients simultaneously while providing acceptable throughput for all.

The sequence of events for a file upload from a user's local machine to the mission file servers is just as straightforward. It's shown in Figure 20-5.

Figure 20-5. How the two-layer service handles a file write

These tables don't show it, but besides a file token, each client request also includes a user token. A client application first obtains a user token when it makes a successful login request (with a user name and password) to the middleware's user management service, thus authenticating the user. A user token contains information that identifies a particular user session, including the user's role. It enables the streamer service to verify that a request is coming from a legitimate user. It checks the user's role to ensure that she has the right to download a particular file. For example, the MER mission disallowed users from foreign (non-U.S.) countries from accessing certain files, and CIP respected all such security restrictions.