IT-SC book: Advanced CORBA® Programming with C++

Using multithreaded programming techniques to implement server applications provides benefits such as the following.

Simplified program design

Multiple server tasks can proceed independently, and no artificial task-switching boundaries need be maintained in the application.

Improved throughput

On multiprocessor hardware, the operating system assigns multiple threads to different CPUs, thus achieving true concurrency.

Improved response time

Clients need not worry about their requests being starved for attention or denied because of long-running requests from other clients.

In the next section we explain how multithreaded programming techniques provide these and other benefits.

21.4 Fundamentals of Multithreaded Servers

The problems with single-threaded applications described in the preceding section indicate that distributed applications do not perform or scale well if they are designed and written to use a single thread of control. As we explain in this section, using preemptive multithreading instead provides a much more elegant, and potentially more efficient, means of supporting scalable server applications.

With preemptive multithreading, the underlying operating system kernel or a special threading library controls the scheduling of threads to allow them to execute their tasks. A slice of CPU time is given to each thread. When the thread either uses up its time slice or makes a blocking call such as reading from a socket, the scheduler preempts the thread and allows another one to run. This arrangement relieves programs of the added complexity of ensuring that all necessary tasks get the CPU time they need to complete. It also allows orthogonal parts of the application to remain wholly separate, permitting you to implement and maintain them without fear of compromising the correctness of the application because of task starvation.

Most portions of a server application are affected by the use of multithreading, including the ORB and POAs, servant implementations, and third-party and system libraries. When we say that a portion of an application is "affected," we do not mean to imply that it becomes full of invocations of arcane multithreading functions. Instead, we mean that you must keep the following two points in mind.

You must take the presence of multithreading into account in all areas of your application that you are responsible for writing.

For third-party and system libraries, the ORB, and POAs, you must understand the implications of invoking their functions in a multithreaded environment.

Applications must be designed explicitly to support and use multithreading. A program designed to execute with only a single thread of control will almost certainly fail to work

831

IT-SC book: Advanced CORBA® Programming with C++

correctly in a multithreaded environment. Such a program usually requires significant redesign and rework to make it handle multiple threads effectively.

In the following sections we explain the effects of using multiple threads on the various portions of server applications.

21.4.1 ORB Infrastructure Multithreading Issues

Multithreaded ORB implementations have available a wide variety of options for handling requests. As we discuss in Section 21.3, single-threaded ORBs must perform non-blocking I/O, request queuing, and explicit task switching, thus limiting their flexibility and configurability. Multithreaded ORBs, on the other hand, can support different strategies [38] for request dispatching, even simultaneously. The use of multiple threads allows the ORB to separate concerns.

For example, one implementation of a multithreaded ORB core might have a single thread, called a listener thread, that listens for requests. When a request arrives, the thread reads the entire network message containing the request and all its arguments and places it in a request queue. The other end of the queue might be monitored by a pool of threads that wait for requests to appear in the queue. When a request is put into the queue by the listener thread, a thread from the pool removes it from the queue and takes charge of dispatching it to the right POA and eventually to the right servant.

Another ORB core implementation might choose instead to use multiple listener threads, with each thread listening to a single network port. Still another might choose to create a new thread to handle each incoming request. Other variations are also possible, such as mixing support for multiple strategies into a single ORB core.

As you might imagine, each solution for applying threads to the processing of requests has its own benefits and drawbacks.

A thread-per-request solution, in which a new thread is spawned for each incoming request, works well for servers that receive a low volume of long-running requests. Because each request executes in its own thread, it will not block other requests from being processed no matter how long it takes to complete. However, if too many requests are in progress simultaneously, the server application might use excessive resources because of the presence of too many threads.

A thread-per-connection approach, in which a different thread is used for each separate client connection, works well for applications in which clients invoke numerous requests on the same server over a lengthy period. This technique avoids the cost of creating a new thread for each request as in the thread-per-request approach. However, if the server has a lot of clients, it could result in many threads being created to handle them. Also, if client connections are short-lived, this solution approaches the thread-creation overhead of the thread-per-request model.

832

IT-SC book: Advanced CORBA® Programming with C++

A thread pool solution involves spawning a number of threads at server startup and then assigning incoming requests to non-busy threads as they arrive. If all threads in the pool are already busy handling requests, either the request can be queued until a thread becomes available to handle it, or new threads can be created and added to the pool. This model works well for servers that want to bound their request-handling resources, because all the necessary threads and queues can be allocated at program start-up. One drawback to this approach is that switching requests from one thread to another via a queue can result in excessive thread context switching overhead. Also, if the server allocates insufficient resources to handle the volume of requests it receives, queues could become filled, and incoming requests might have to be temporarily rejected.

Detailed analyses of other threading models and variations on the models described here can be found in [38].

21.4.2 POA Multithreading Issues

After the ORB core dispatches a request to the POA where the target object is located, the threading policy of the POA must be taken into account. As explained in Section

11.4.7, a POA can have either the SINGLE_THREAD_MODEL value or the ORB_CTRL_MODEL value for its ThreadPolicy. How the POA completes the request dispatch to the appropriate servant depends entirely on its ThreadPolicy value.

When a POA has the SINGLE_THREAD_MODEL policy value, it guarantees that all servant invocations will be serialized. Even if the underlying ORB and other POAs in the same server use multiple threads, a POA created with the SINGLE_THREAD_MODEL policy value never performs request dispatching on multiple servants simultaneously. If the underlying ORB is multithreaded, a SINGLE_THREAD_MODEL POA must be able to switch incoming requests onto the single thread it uses for all servant dispatching. Application designers should beware that switching requests from one thread to another results in thread context switching overhead. Also, on some platforms SINGLE_THREAD_MODEL POAs must perform their dispatching using the main thread (the one in which the program's main function was invoked); otherwise, calling code that is unaware of multithreading (perhaps because it was not compiled with the proper options) will not work correctly. In this case, it is the responsibility of the application to invoke ORB::perform_work or ORB::run to ensure that the ORB allows the POAs to get access to the main thread.

When you perform collocated requests from within the same application on objects in SINGLE_THREAD_MODEL POAs, be sure that your vendor's POA implementation properly conforms to the specification. Even requests made locally must be dispatched on the POA's single thread rather than dispatched directly using the caller's thread. If your ORB bypasses the dispatching mechanisms for collocated requests when calling objects in SINGLE_THREAD_MODEL POAs, perhaps by invoking virtual functions directly on the servant, it does not conform to the specification.

833

IT-SC book: Advanced CORBA® Programming with C++

POAs created with the ORB_CTRL_MODEL threading policy value are far less constrained with respect to request dispatching than their single-threaded counterparts. The ORB_CTRL_MODEL policy value only implies that the POA is allowed to dispatch multiple requests concurrently; it does not prescribe how requests are assigned to threads. This means that an ORB_CTRL_MODEL POA might be implemented to use its own threading policy independent of that of the underlying ORB, or it might be implemented to fit seamlessly with the model used by the ORB.

The POA might use the thread pool model. In that case, it has its own pool of threads and its own queue for holding requests when all its threads are busy. This approach works well except for the price of switching requests from one thread to another. Also, there is always the possibility that the volume of requests that the POA receives will far outstrip the dispatching capacity of its thread pool, thus requiring requests to be temporarily rejected.

A POA might use a thread-per-servant approach, creating a new thread for each servant added to its Active Object Map. In this approach, the POA dispatches all requests for a given servant on that servant's thread. This technique performs well if the set of servants in a POA is relatively fixed and small. Otherwise, a POA that has either many registered servants or many servants that are only briefly registered and then destroyed may incur too much thread creation overhead, or it may try to create too many threads at once.

Using the thread-per-request model means that the POA creates a new thread for each incoming request. This approach works well only if the POA receives a relatively low volume of long-running requests. Otherwise, the overhead of creating many threads or having too many threads active simultaneously becomes too great.

A POA might simply continue the request dispatch on the same thread that the ORB used to dispatch the request from the network port it was received on. This approach avoids the cost of switching the request processing from one thread to another, but it ties up an ORB thread for the duration of the request and prevents it from being used to do other ORB work.

Each POA in a group of POAs sharing a single POAManager might rely on it to supply a request-dispatching thread. This technique merely pushes the multithreading issues from the POA to the POAManager, and it might also mean increased contention for threading resources if the POAManager is controlling request flow for multiple POAs.

Other request-dispatching strategies are also possible for ORB_CTRL_MODEL POAs. Depending on their policies, different POAs in the same server application might even employ different strategies.

Because the POA specification does not require any particular threading model for ORB_CTRL_MODEL POAs, POA implementers are permitted to use any of these approaches, or any other approach that they deem useful. Although this arrangement provides maximum flexibility for POA implementers, it makes it impossible to write portable applications that make assumptions about the underlying POA threading model.

834

IT-SC book: Advanced CORBA® Programming with C++

This implementation freedom benefits application developers by enabling ORB vendors to compete with one another. At the same time, however, it makes it difficult for application developers to make important implementation decisions, such as how many POAs their applications should have, how many CORBA objects to create under each POA, whether to use a single servant per object or to make a single servant incarnate multiple objects, and how in general to distribute their objects across multiple servers. Being able to control, or at least being able to know, the multithreading strategies of the underlying ORB and POA enables developers to make more informed architectural, design, implementation, and deployment decisions, thus improving the overall scalability and performance of their distributed systems.

Fortunately, it is possible for the OMG to extend the set of ThreadPolicy values to address this shortcoming by adding new policy values that identify specific threading models, such as THREAD_PER_SERVANT or THREAD_POOL. Such policy values would provide applications with explicit control over the assignment of requests to threads. As of this writing, the POA specification is still very new, however, so new policy values such as these are not likely to be added until the CORBA community gains additional practical experience with how to best use the POA. For the time being, you must ask your ORB vendor if you want to find out how your ORB implements the

ORB_CTRL_MODEL policy.

21.4.3 Servant Multithreading Issues

Because a POA having the ORB_CTRL_MODEL value is permitted to dispatch multiple requests concurrently to the same servant, the presence of multithreading has a strong influence on how you must design and implement your servants. To perform requests, almost all servants access either their own data members or state variables shared between objects. Access to all such state must be carefully serialized and synchronized among threads so as to avoid corrupted data that causes method implementations to return garbage results.

A serious portability problem exists for the development of multithreaded applications: threading primitives are not portable between platforms. Although standard programming interfaces for multithreading exist—notably the POSIX 1003.4a pthreads API [2]—not all of them are supported across all platforms. Some systems provide this interface or slight variations of it, but others, such as Windows, do not support it at all. Fortunately, ORB vendors typically ship threads portability libraries with C++ interfaces as part of their products. Freely available software, such as the ADAPTIVE Communication Environment (ACE) Toolkit [33], also provides portable C++ wrappers for platformspecific multithreading primitives.

One of the most difficult issues is how to safely remove servants. When multiple threads allow multiple requests to be present simultaneously within a single servant, it is a complex task to make sure that no other threads are accessing the servant when it is time to destroy it. Fortunately, the POA provides several guarantees that help with this

835

IT-SC book: Advanced CORBA® Programming with C++

problem. We investigate this issue along with multithreaded servant development in more detail in Section 21.6.

21.4.4 Issues with Third-Party Libraries

CORBA is primarily an integration technology, so it is no surprise that many applications use third-party and system libraries. Unfortunately, you may find yourself in the position of having to use a third-party library or component that is not thread-safe. You can address this situation in several ways.

Work with the library supplier to see whether a thread-safe version of the library is available. Assuming that the vendor has implemented it to be truly thread-safe, this is by far your best option.

Implement your own thread-safe wrappers for the library, and make sure that you call the library only through the wrappers and never directly. Although this can be tedious, the work can be worthwhile if the library is reused for other projects. Such wrappers can be hard to maintain, however, if the underlying library changes frequently.

Isolate all invocations of the library in a single thread. One straightforward way to do this is to develop an IDL interface for the library and provide access to it via a CORBA object. By registering the object's servant with a SINGLE_THREAD_MODEL POA, you can easily guarantee that all invocations of the library are serialized. With this approach, you must take care never to advertise the object reference outside your server process; you probably do not want your single-threaded object wrapper to be invoked from code outside your process.

If the library is to be used heavily, wrapping it with an object may not be practical depending on the dispatching overhead of your ORB. In that case, you may have to resort to using lower-level threading primitives and queues to transfer work items from other threads that require library invocations.

Sometimes libraries that were not intended for use in multithreaded environments cannot be linked with your threaded applications. Depending on the platform, compiler, and linker, single-threaded and multithreaded libraries may be unable to coexist because of different compile-time or link-time options. They may also fail to work together if each one depends on other libraries that themselves are mutually exclusive. Usually, the only way to fix this problem is to use a different library altogether.

If the library you are trying to use with your ORB has its own event loop, as many GUI libraries do, you must integrate its event loop with the ORB's event loop. In the next section we discuss ways of accomplishing this.

21.4.5 ORB Event Handling Multithreading Issues

836