Section 26.4. Implementing Concurrent Logging Servers

26.4. Implementing Concurrent Logging Servers

To overcome the scalability limitations of the iterative and reactive servers shown in the previous sections, the logging servers in this section use OS concurrency mechanisms: processes and threads. Using the APIs provided by operating systems to spawn threads or processes, however, can be a daunting task due to accidental complexities in their design. These complexities stem from semantic and syntactic differences that exist not only between different operating systems, but also different versions of the same operating system. Our solution to these complexities is again to apply wrapper facades that provide a consistent interface across platforms and integrate these wrapper facades into our OO Logging_Server framework.

26.4.1. A Thread-per-Connection Logging Server

Our thread-per-connection logging server (TPC_Logging_Server) runs a main thread that waits for and accepts new connections from clients. After accepting a new connection, a new worker thread is spawned to handle incoming log records from that connection. Figure 26-10 shows the steps in this process.

Figure 26-10. Steps in the thread-per-connection logging server

The main loop for this particular logging server differs from the steps depicted in Figure 26-3 because the call to handle_data( ) is not necessary, as the worker threads are responsible for that call. There are two ways to handle this situation:

We could note that the base run( ) method calls handle_data( ) with the default argument of a NULL pointer, and simply have our implementation exit immediately for that input.
We could simply override the run( ) method with our own implementation that omits this call.

The second solution may at first appear advantageous because it avoids a virtual method call to handle_data( ). The first solution is better in this case, however, because the performance hit of that virtual call is not a limiting factor, and overriding the run( ) template method would prevent this class from benefiting from changes to the base class implementation, potentially causing it to fail in subtle and pernicious ways.

The main challenge here is implementing the concurrency strategy itself. As with the Iterative_Server in the earlier section "An Iterative Logging Server," the wait_for_ multiple_events( ) method is superfluous because our main loop simply waits for new connections, so it is sufficient for handle_connections( ) to block on accept( ) and subsequently spawn worker threads to handle connected clients. Our TPC_Logging_Server class must therefore provide a method to serve as an entry point for the thread. In C and C++, a class method may serve as an entry point to a thread only if the class is defined as static, so we define the TPC_Logging_Server::svc( ) static class method.

At this point, we have an important design decision to make: what exactly does the thread entry point do? It is tempting to simply have the svc( ) method itself perform all of the work necessary to receive log records from its associated connection. This design is less than ideal, however, because static methods cannot be virtual, as that would cause problems if we later derive a new logging server from this implementation to change the way it handles data events. Application developers would then be forced to provide an implementation of handle_connections( ) that is textually identical to this class to call the proper static method.

Moreover, to leverage our existing design and code, it is preferable to have the log record processing logic inside the handle_data( ) method and to define a Thread_Args helper object that holds the peer returned from accept( ) and a pointer to the Logging_Server object itself. Our class interface will therefore look like the diagram in Figure 26-11.

Figure 26-11. Thread-per-connection server interface

The remainder of TPC_Logging_Server is straightforward to implement, requiring only that our thread entry point delegate processing to the virtual method handle_data( ) using the server_pointer contained within the Thread_Args helper object passed to the svc( ) method, as shown in Figure 26-12.

Figure 26-12. Thread-per-connection thread behavior

The following code implements a TPC_Logging_Server main program that uses the secure socket API and the readers/writer lock:

	int main (int argc, char *argv[]) {
	  TPC_Logging_Server<SSL_Acceptor, RW_Lock> server (argc, argv);
	  server.run ();
	  return 0;
	}

This main( ) function instantiates a TPC_Logging_Server that communicates using SSL connections, and uses an RW_Lock to synchronize the count_connections( ) function in the Logging_Server base class. Except for the name of the class we are instantiating, this main( ) function is identical to the one that was written earlier in this chapter for the Reactive_Logging_Server. This commonality is another beautiful aspect of our design: regardless of the particular combination of concurrency, IPC, and synchronization mechanisms we choose to use, the instantiation and invocation of our server remains the same.

The thread-per-connection logging service addresses the scalability limitations with the sequential implementations described earlier in the section "Evaluating the Sequential Logging Server Solutions." The design of our OO framework makes it straightforward to integrate this concurrency model with minimal changes to the existing code. In particular, TPC_Logging_Server inherits implementations of open( ), count_request( ), and most importantly run( ), allowing this class to leverage bug fixes and improvements to our main event loop transparently. Moreover, adding the necessary synchronization around the request_count_ is simply a matter of parameterizing the TPC_Logging_Server with the RW_LOCK class.

26.4.2. A Process-per-Connection Logging Server

The process-per-connection logging server described next is similar to the thread-per-connection design shown in Figure 26-10 , except that instead of spawning a thread, we spawn a new process to handle incoming log records from each client. The choice of processes over threads for concurrency forces us to make design choices to accommodate the variations in process-creation semantics between platforms. There are two key semantic differences between the process APIs on Linux and Windows that our server design must encapsulate:

In Linux (and other POSIX systems) the primary vehicle for creating new processes is the fork( ) system function, which generates an exact duplicate of the calling program image, including open I/O handles. The processes differ only in their return value from fork( ). At this point, child processes can choose to proceed from that point, or load a different program image using the exec*( ) family of system calls.
Windows, however, uses the CreateProcess( ) API call, which is functionally equivalent to a POSIX fork( ), followed immediately by a call to one of the exec*( ) system functions. The impact of this difference is that in Windows you have an entirely new process that by default does not have access to I/O handles open in the parent. To use a connection accepted by the parent process, therefore, the handle must be explicitly duplicated and passed to the child on the command line.

We therefore define a set of wrapper facades that not only hide the syntactic differences between platforms, but also provide a way to hide the semantic differences as well. These wrappers consist of the three cooperating classes shown in Figure 26-13. The Process class represents a single process and is used to create and synchronize processes. The Process_Options class provides a way to set both platform-independent process options (such as command-line options and environment variables) and platform-specific process options (such as avoiding zombie processes). Finally, the Process_Manager class portably manages the life cycle of groups of processes. We won't cover all the uses of these wrapper facades in this chapter, though they are based on the wrapper facades in ACE. ^[||||||]It is sufficient to know that not only can processes be created portably on Linux and Windows, but also that I/O handles can be duplicated and passed portably and automatically to the new process.

^[||||||] C++ Network Programming, Vol. 1: Mastering Complexity with ACE and Patterns, Douglas C. Schmidt and Stephen D. Huston, Addison-Wesley, 2001.

The design challenge is therefore to accommodate the fact that processes spawned after new connections are accepted will start at the beginning of our program. We certainly don't want child processes to attempt to open a new acceptor and listen for connections of their own; instead, they should listen for data events only on their assigned handle. A naïve solution to this problem would rely on applications to detect this condition and call a special entry point defined in the interface to our process-based Logging_Server class.

Figure 26-13. Portable process wrapper facades

This simple solution, however, is less than ideal. It would require us not only to change the public interface of our process-based Logging_Server, but to expose intimate implementation details to applications, violating encapsulation. A better solution is to override the run( ) template method inherited from the Logging_Server base class, which is passed a copy of the command-line argument by users, to determine whether it has been passed any I/O handles. If not, the process assumes it is a parent and delegates to the base class run( ) method. Otherwise, the process assumes it's a child, so it decodes the handle and calls handle_data( ), as shown in Figure 26-14.

Figure 26-14. Process-per-connection run( ) template method

The remainder of this server implementation is straightforward. As shown in Figure 26-15 , the process wrapper facade makes the procedure for spawning our worker processes fairly simple. The implementation for handle_data( ) should be textually identical to that shown in Figure 26-12.

Figure 26-15. Connection handling for the process-per-connection server

Our reimplementation of the run( ) method from the Logging_Server base class allows us to maintain the beautifully simple, straightforward, and uniform invocation used by our other logging servers:

	int main (int argc, char *argv[]) {
	  PPC_Logging_Server<SSL_Acceptor, Null_Mutex> server (argc, argv);
	  server.run ( );
	  return 0;
	}

This main( ) program differs from the thread-per-connection server only in the name of the class that is instantiated and the choice of a Null_Mutex for synchronization. The dispatch of either a parent or a child process is handled transparently by the run( ) method, driven by the command-line arguments passed to the PPC_Logging_Server constructor.

26.4.3. Evaluating the Concurrent Logging Server Solutions

Both concurrent logging servers described in this section significantly enhance the Reactive_Logging_Server and Iterative_Logging_Server in their ability to scale as the number of clients increases by taking leveraging hardware and OS support for multiple threads of execution. It is hard, however, to develop thread-per-connection and process-per-connection concurrency strategies in a platform-agnostic manner. We accomplished this task by using wrapper facades to hide platform differences. Our framework-based server design also provided a common external interface to the Logging_Server class, shielding the bulk of the logging server from the configured concurrency strategy. Moreover, our design leveraged the run( ) template method inherited from the Logging_Server base class, allowing our implementations to integrate bug fixes or other enhancements to the main server event loop.