High-level object-oriented concurrency for C++

PETER A. BUHR AND

RICHARD C. BILSON

Concurrency is the most complex form of programming, with certain kinds of real-time programming being the most complex forms of concurrency.

Often the mechanisms for writing concurrent programs only exacerbate the complexity because they are too low level and/or independent from the language. Currently, concurrency is being thrust upon all programmers indirectly through the push for higher performance in hardware. To maintain Moore’s Law, it is becoming necessary to add parallelism at a number of hardware levels — instruction pipeline, multithreading, multicore processors, shared-memory multiprocessors, and distributed clusters. Some successful attempts have been made to implicitly discover concurrency in a sequential program; for instance, by parallelizing loops and access to data structures. While this approach is appealing because of the simple sequential programming model and ability to parallelize legacy code, there is a limit to how much parallelism can be found, and current techniques only work on certain kinds of programs. Therefore,

Peter is an associate professor in the School of Computer Science at the University of Waterloo. He can be contacted at pabuhr@ uwaterloo.ca. Richard is a research assistant in the School of Computer Science at the University of Waterloo, working in the Programming Languages Group. He can be contacted at rcbilson@ plg.uwaterloo.ca.

explicit concurrent mechanisms are necessary to achieve maximum concurrency potential. Luckily, approaches to this are complementary, and can appear together in a single programming language.

C++ Concurrency

Given the need for explicit concurrency, modern programming languages such as Beta, Ada, Modula-3, Java, and C#, among others, provide some direct support for concurrency. Surprisingly, however, C++ has no concurrency support. During C++’s 20-year history, many different concurrency approaches for C++ have been suggested and implemented with only varying degrees of adoption. As a result, there is no de facto standard dominating concurrent programming in C++. (In C, there are two dominant, but incompatible, concurrency libraries —Win32 and pthreads.) In this article, we argue that C++’s lack of concurrency is significantly limiting the language’s future. This deficiency has also been recognized by the C++ Standards committee, which is currently examining concurrency extensions. We also outline how high-level object-oriented concurrency can be added to C++ through a freely available concurrent dialect of C++ called “µC++” (http://plg.uwaterloo.ca/ usystem/uC++.html).

Concurrent Design Principles

There are a number of major design principles for adding concurrency to objectoriented languages, such as:

•Object-oriented design is built on the notion of the class. Hence, concurrency should be built on the class notion, allowing it to leverage other class-based language features.

•All concurrent systems must provide three fundamental properties: thread, a mechanism to sequentially execute statements, independently of (and possibly concurrently with) other threads; execution context, the state needed to permit independent execution, including a

separate stack; and mutual exclusion/synchronization (MES), mechanisms to exclusively access a resource and provide necessary timing relationships among threads. These properties cannot be expressed in an architectureindependent way through existing language constructs. (Even algorithms for MES, such as Dekker’s algorithm, do not

“µC++ was designed to provide high-level, integrated, lightweight, object-oriented concurrency

for C++”

always work without a sufficient memory model.) Therefore, any concurrency system must provide abstractions to implement these properties.

•Because MES causes the most errors for programmers and the greatest difficulty for safe code optimizations, it should be implicit through concurrent language constructs.

•If the routine call is the basis for normal object communication, it should also be used for concurrency. Mixing mechanisms, such as routine call with messagepassing/channels, is confusing and error prone, and may lose important capabilities such as static type checking.

Joining the fundamental concurrency properties with the class model is best done by associating thread and execu- tion-context with the class, and MES with member routines. This coupling and the

interactions among the concurrency properties generate the programming abstractions in Table 1.

•Case 1 in Table 1 is a standard C++ object. Its member routines do not provide MES, and the caller’s thread and stack are used to perform execution.

•Case 2 has all the properties of case 1 but only one thread at a time can be executing among the member routines with the MES property, called a “mutex member.” Within a mutex member, synchronization with other tasks can be performed. This abstraction is a monitor, which is well understood and appears in many concurrent languages (Java, for instance).

•Case 3 is an object that has its own execution context but no MES or thread; the execution context is associated with a distinguished member in the object. This abstraction is a coroutine, which goes back to the roots of C++ in Simula.

•Case 4 is like case 3 but deals with concurrent access by adding MES. This abstraction is a coroutine monitor.

•Cases 5 and 6 are a thread without a stack, which is meaningless because a thread must have a stack to execute.

•Case 7 is an object that has its own thread and execution context but no MES. This case is questionable because explicit locking is now required to handle calls from other threads, which violates design principle 3.

•Case 8 is like case 7 but deals with concurrent access by adding MES. This abstraction is a task, which is an active object and appears in many concurrent languages (Ada). Note, the abstractions are derived from fundamental properties and not ad hoc decisions by a language designer, and each has a particular set of problems it can solve well. Simulating one abstraction with the others often results in awkward solutions that are inefficient; therefore, each has a place in a programming language.

µC++: Concurrency in C++

µC++ was designed using these concurrency design principles and engineered to provide high-level, integrated, lightweight, object-oriented concurrency for C++. By being high level, you can code in a race-free style, which eliminates the need for a complex memory model. By being integrated into the C++ language, the compiler can understand precisely when it can safely perform optimizations. Currently, µC++ is a translator that converts to C++, but its design ultimately assumes it is part of C++.

Figure 1 shows the syntax for adding the programming abstractions in Table 1 to C++. There are two new type constructors _Coroutine and _Task, extensions

in important domains, direct support of the coroutine is crucial, independent of concurrency.

Monitor

A monitor is a concurrency abstraction that encapsulates a shared resource with implicit mutual exclusion and provides for complex synchronization among tasks using the resource; see Listing Three.

Any member routine can be qualified with the MES qualifiers, _Mutex/_Nomutex, indicating the presence or absence of MES, respectively. Only one thread at a time can be executing among the mutex routines of a monitor object; other threads calling mutex routines of the same object implicitly block. Recursive entry is allowed for the thread currently using the monitor; that is, it may call other mutex members. The MES qualifiers can also qualify a class, which defines the default qualification for the public member routines. Hence, the presence of a single mutex member in a class makes it a monitor. Member variables cannot be MES qualified. The destructor of a monitor is always _Mutex, because the thread terminating a monitor object must wait if another thread is executing in it.

The mutex property ensures exclusive access to the monitor’s data by multiple threads. For simple cases, such as an atomic counter, exclusive access is sufficient and the order of access is unimportant. For complex cases, the order of access can be crucial for correctness; for example, one task may need to communicate information to another task and wait for a reply, or a resource may have strict ordering rules with respect to thread usage. Ordering is controlled by threads synchronizing among

(a)calling

shared data

calling shared data

urgent exit

Figure 2: Monitor synchronization: (a) condition variable; (b) accept statement.

themselves within a monitor using condition variables and operations wait( ), signal( ), signalBlock( ), or an _Accept statement on mutex members.

A condition variable is a place where a task within the monitor can wait for an event to occur by another task using the monitor. Figure 2(a) illustrates the internal parts of a monitor object for synchronization with condition variables. Calling threads wait until no mutex member is active. A condition variable (for example, c) is a queue of waiting threads. The thread active in the monitor object waits on queue c by executing c.wait( ) (dotted line), which either implicitly restarts the last signalled thread, or if no signalled threads, releases the monitor lock so a new thread may enter. The active thread may execute c.signal( ) (dashed line) to restart a waiting thread at the front of a condition queue; the signalled thread can only restart after the signaler thread blocks or exits due to the mutual exclusion property, which is accomplished by having the signalled thread wait temporarily on the hidden urgent condition. Alternatively, the active thread may execute c.signalBlock( ) (solid line), which makes the active thread wait on the urgent queue and immediately starts the signalled thread at the front of the queue. Using these mechanisms, order of access within the monitor can be precisely controlled.

Tasks within the monitor can wait for an event to occur by a calling task using an accept statement. Figure 2(b) illustrates the internal parts of a monitor object for synchronization with an _Accept statement. _Accept selects which mutex member call to execute next (like Ada’s select). _Accept(m1) unblocks the thread on the front of the mutex queue after the accepter is implicitly blocked (like signalBlock). If there is no calling task, the accepter waits (on the hidden urgent queue) until a call to the specified member occurs. When the member call ends, the accepter implicitly restarts after the _Accept statement. _Accept can appear at any level of routine nesting in the monitor. The _Accept statement can check multiple mutex members for calls:

_Accept( m1, m2,…);

The call on the first nonempty mutex queue is selected (so the order of member names is important); if no calls are pending, the accepter waits until a call occurs. Finally, each selected member can be separated and supplied with a guard:

_When( conditional-expression ) _Accept( m1 ) statement

else _When( conditional-expression ) _Accept( m2 ) statement

…

else

statement

The guard must be true before a mutex queue is considered; if there is a terminating else, the accepter does not block, rather it polls for callers. The statement after an _Accept is executed by the accepter after the mutex call, allowing it to perform different actions depending on which call occurred.

Monitor Examples

Listing Four(a) shows the classic datingservice problem implemented with condition variables, where two kinds of tasks exchange information based on some criteria. In this case, there are girl and boy tasks exchanging phone numbers if they have matching compatibility codes (values 0 –19). A girl task checks if a boy with the same code is waiting. If not, she waits; otherwise, she copies her phone number to the shared variable GirlPhoneNo, and does a signalBlock to immediately restart the waiting boy while she waits on the urgent queue. The waiting boy restarts, copies his phone number to the shared variable BoyPhoneNo, and returns with the girl’s phone number. The waiting girl is then implicitly restarted from the urgent queue after the boy returns, and she now returns with the boy’s phone number. Listing Four(b) shows the classic read/write problem implemented with _Accept, where multiple reader tasks can simultaneously read a resource, but writer tasks must be serialized to write the resource. Tasks access the resource like this:

The variables rcnt and wcnt count the number of simultaneous reader or writer tasks using the resource. EndRead/EndWrite decrement the appropriate counter when a task finishes using the resource. StartRead checks if a writer is using the resource, and if so, accepts EndWrite, causing the reader task to wait on the urgent queue and preventing calls to any other mutex member. When the current writer finishes writing, it calls EndWrite; then the waiting reader implicitly restarts in

StartRead and increments rcnt. StartWrite begins with the same check for a writer and the same actions as for a reader if a writer is using the resource. Alternatively, if there are rcnt readers using the resource, the writer loops and performs rcnt accepts of EndRead, one for each of the completing reader tasks. After the last reader finishes reading and completes its call to EndRead, the waiting writer implicitly restarts and increments wcnt. Because the accept statement strictly controls entry into

the monitor, new (calling) tasks may not enter out of order.

Coroutine Monitor

The properties of a coroutine and monitor can be combined to generate a concurrency abstraction for resource sharing and synchronization along with retaining data and execution state; see Listing Five.

A coroutine monitor is ideal for an FSM used by multiple threads, such as a shared formatter printer. The printer is the shared resource called by multiple threads to print each thread’s data, and the printer can be a complex FSM organizing the data into rows and columns with appropriate markings and headings. Combining these fundamental properties into a single construct simplifies the job of developing the solution for a complex problem.

Task

The properties of a coroutine monitor can be combined with a thread to generate a concurrency abstraction for an active object that dynamically manages a resource; see Listing Six, for example.

Active objects appear in many concurrent languages. The use of both wait/signal on condition variables and accepting mutex members occurs frequently in a task (less so in a monitor). Finally, because the destructor is a mutex member, it can be accepted to determine when to terminate a task or monitor.

Listing Seven(a) shows a basic worker task, Adder, generating the subtotals for each row of a global matrix by summing the elements of a particular row. (Global variables are used to simplify the example.) In µC++, the member routine uMain::main serves as the program’s main (starting) routine. This routine reads the matrix and starts a block that creates an array of Adder tasks, one for each row of the matrix. Each task’s main starts implicitly after its constructor completes — no explicit start is needed. Similarly, no explicit join is needed because the block containing the array of tasks cannot end until all the tasks in the array terminate, otherwise, the storage for the tasks could be deallocated while threads are executing. After all tasks in the block terminate, the block allocating the array of tasks ends, and the subtotals generated by each worker task can be safely summed to obtain the total. The constructor for each Adder task selects a specific row to sum by incrementing a shared variable; no mu-

tual exclusion is necessary for the selection as each task of the array is created serially. The main member of each Adder task adds the matrix elements for its particular row in its corresponding subtotal location.

Listing Seven(b) shows a classic administrator server, where clients call different members for service. The server may provide multiple interface members for different kinds of clients and client requests. A client’s call may be serviced immediately or delayed using condition variables. The server’s main loops accept client calls using the _Accept statement; an accepted call may complete immediately or require subsequent servicing and signalling of the delayed client. Finally, the server’s destructor is also being accepted to know when the server is being deleted by another thread.

Miscellaneous µC++ Features

µC++ has a number of other features that integrate concurrency with C++:

•Both termination and resumption exception handling are supported, as well as the ability to raise exceptions among coroutines and tasks; see Listing Eight(a). For resumption, the stack is not unwound, and control returns after the _Resume when the handler completes. The _At clause provides nonlocal delivery of an exception to another coroutine or task. Nonlocal delivery of exceptions is controlled by _Enable and _Disable statements; see Listing Eight(b). Specifying no exceptions enables/disables all exceptions.

•The execution environment can be structured into multiple clusters of tasks and processors. Each cluster has a scheduler to control selection of its tasks to run on its processors; tasks and processors can migrate among clusters.

•C++ streams and UNIX files/sockets are augmented to be thread safe, ob- ject-oriented, and nonblocking; for example, safe stream I/O is performed like this:

isacquire( cin ) >> . . .; osacquire( cout ) << ...<< endl;

The declaration at the start of the I/O expression provides necessary locking of the specified stream for the duration of the expression. There are three classes for accessing sockets: uSocketServer, uSocketAccept, and uSocketClient, which hide most of the socket complexity and

support connectionless and connected protocols with timeout capabilities.

•Basic real-time programming is available through three extensions of the task:

_RealTimeTask R {…}; _PeriodicTask P {…}; _SporadicTask S {…};

Fixed and dynamic priority schedulers are provided for use with clusters, including a transitive priority-inheritance protocol. The _Accept statement is extended to handle timeouts:

_Accept( M1, M2 ) {…} else _Accept ( M3 ) {…}

else _Timeout( 1 ) {…} // restart after // 1 second if // no call

•There is a debug mode for testing with many assertions and runtime checks, and µC++ also generates reasonable error messages. µC++ compiles on GCC 3.2 or greater and Intel icc 8.1/9.0; for Linux Intel x86/Itanium and AMD 32/64; Solaris 8/9/10 SPARC; and IRIX 6.x MIPS.

Conclusion

Providing concurrency via low-level libraries such as pthreads makes no sense for C++. This approach is error prone and does not integrate with existing C++ mechanisms. Medium-level approaches that attempt to leverage existing language features with a concurrency library also fall short, as programmers still struggle with multiple coding conventions and limitations of use, and some primitive concurrency properties are still hidden from the compiler. To truly help you and the compiler, concurrent programming requires high-level concurrency models and constructs. The three fundamental properties in concurrency— thread, execution context, and mutual-exclusion/ synchronization— can be integrated directly into C++’s core programming notion— the class — and subsequently work with other C++ mechanisms. This approach retains the language’s object-oriented programming model and provides multiple concurrency approaches and models, while requiring only a few new keywords and mechanisms in C++. µC++ is a full implementation of these ideas, providing a system that lets you tackle complex concurrent projects.

DDJ

Listing One

_Coroutine C {

void main() { // distinguished member / executes on coroutine's stack

...suspend()... // restart last resume

}

public:

void m1(...) {... resume();...} // restart last suspend void m2(...) {... resume();...} // restart last suspend

};

(continued on page 40)

(continued from page 39)

Listing Two

_Coroutine Phone { public:

enum status { MORE, GOOD, BAD }; private:

char ch; status stat; void main() {

int i;

stat = MORE; // continue passing characters if ( ch == '(') { // optional area code ?

for ( i = 0; i < 3; i += 1 ) { suspend();

if ( ! isdigit(ch) ) { stat = BAD; return; }

}

suspend();

if ( ch != ')') { stat = BAD; return; } suspend();

}

for ( i = 0; i < 3; i += 1 ) { // region code ? if ( ! isdigit(ch) ) { stat = BAD; return; }

suspend();

}

if ( ch != '-') { stat = BAD; return; } // separator ? for ( i = 0; i < 4; i += 1 ) { // local code ?

suspend();

if ( ! isdigit(ch) ) { stat = BAD; return; }

}

stat = GOOD;

}

public:

status next( char c ) { // pass one character at a time to FSM ch = c;

resume(); // activate coroutine return stat;

}

};

Listing Three

_Mutex class M { // default MES for public member routines // SHARED DATA ACCESSED BY MULTIPLE THREADS

uCondition c1, c2[10], *c3 = new uCondition; //different condition variables // default for private/protected is no MES

void m1(...) {... /* MES statements */...} // no MES _Mutex void m2(...) {... /* MES statements */...}; // MES

public:

void m3(...) {.../* MES statements */...} // MES _Nomutex void m4(...) {...} // no MES

... // destructor is ALWAYS mutex

};

Listing Four

(a)

_Mutex class DatingService { uCondition Girls[20], Boys[20]; int GirlPhoneNo, BoyPhoneNo;

public:

int Girl( int PhoneNo, int code ) { if ( Boys[code].empty() ) {

Girls[code].wait(); GirlPhoneNo = PhoneNo;

} else {

GirlPhoneNo = PhoneNo; Boys[code].signalBlock();

}

return BoyPhoneNo;

}

int Boy( int PhoneNo, int code ) { if ( Girls[code].empty() ) { Boys[code].wait();

BoyPhoneNo = PhoneNo; } else {

BoyPhoneNo = PhoneNo; Girls[code].signalBlock();

}

return GirlPhoneNo;

}

};

(b)

_Mutex class ReadersWriter { int rcnt, wcnt;

public:

void ReadersWriter() { rcnt = wcnt = 0;

}

void EndRead() { rcnt -= 1;

}

void EndWrite() { wcnt -= 1;

}

void StartRead() {

if ( wcnt == 1 ) _Accept( EndWrite ); rcnt += 1;

}

void StartWrite() {

if ( wcnt == 1 ) _Accept( EndWrite ); else while ( rcnt > 0 )

_Accept( EndRead ); wcnt += 1;

}

};

Listing Five

_Mutex _Coroutine CM { // default MES for public member routines

uCondition c1,c2[10],*c3 = new uCondition; // different condition variables void m1(...) {.../* MES statements */ ...} // no MES

_Mutex void m2(...) { .../* MES statements */ ...}; // MES void main() {...} // distinguished member / has its own stack

public:

void m3(...) { ...resume()/* MES statements */ ...} // MES _Nomutex void m4(...) { ...resume(); ...} // no MES

...// destructor is ALWAYS mutex

};

Listing Six

_Task T { // default MES for public member routines

uCondition c1,c2[10],*c3 = new uCondition; // different condition variables void m1( ... ) {.../* MES statements */ ... } // no MES

_Mutex void m2(...) {.../* MES statements */...}; // MES

void main() {...} // distinguished member/has own stack/thread starts here public:

void m3(...) {.../* MES statements */ ...} // MES _Nomutex void m4(...) {...} // no MES

...// destructor is ALWAYS mutex

};

Listing Seven

(a)

const int rows = 10, cols = 10; int M[rows][cols], ST[rows];

_Task Adder { // add specific row

static int row; // sequential access int myrow, c;

void main() {

ST[myrow] = 0; // subtotal location for ( c = 0; c < cols; c += 1 )

ST[myrow] += M[myrow][c];

}

public:

Adder() { myrow = row++; } // choose row

};

int Adder::row = 0; void uMain::main() {

// read matrix

{

Adder adders[rows]; // create threads } // wait for threads to terminate

int total = 0; // sum subtotals

for ( int r = 0; r < rows; r += 1 ) total += ST[r];

cout << total << endl;

}

(b)

_Task Server { uCondition delay; void main() {

for ( ;; ) { // for each client request _Accept( ~Server ) { // terminate ?

break;

// service each kind of client request

}else _Accept( workReq1 ) {

...

delay.signalBlock(); // restart client

...

}else _Accept( workReq2 ) {

...

}

}

// shut down

}

public:

void workReq1( Req1_t req ) {

...delay.wait();...// service not immediate? // otherwise service request

}

void workReq2( Req2_t req ) { ... }

...

};

Listing Eight

(a)

_Throw [ throwable-exception [ _At coroutine/task-id ] ] ; // termination _Resume [ resumable-exception [ _At coroutine/task-id ] ] ; // resumption

(b)

_Enable <E1><E2>... {

// exceptions E1, E2, ... delivered

}

_Disable <E1><E2>... {

// exceptions E1, E2, ... not delivered

}

DDJ

Conquering the shortcomings of string-based APIs

WILLIAM R. COOK

AND CARL ROSENBERGER

While today’s object databases and object-relational mappers do a great job in making object persistence feel native to developers,

queries still look foreign in object-oriented programs because they are expressed using either simple strings or object graphs with strings interspersed. Let’s take a look at how existing systems would express a query such as “find all Student objects where the student is younger than 20.” This query (and other examples in this article) assume the Student class defined in Example 1. Different data access APIs express the query quite differently, as illustrated in Example 2. However, they all share a common set of problems:

•Modern IDEs do not check embedded strings for syntactic and semantic errors. In Example 2, both the field age and the value 20 are expected to be numeric, but no IDE or compiler checks that this is actually correct. If you mistyped the query code — changing the name or type of the field age, for example — all of the queries in Example 2 would break at runtime, without a single notice at compile time.

•Because modern IDEs will not automatically refactor field names that appear in strings, refactorings cause class model and query strings to get out of

William is an assistant professor of computer science at the University of Texas in Austin. Carl is chief software architect at db4objects. They can be contacted at wcook@cs.utexas.edu and carl@db4o.com, respectively.

sync. Suppose the field name age in the class Student is changed to _age because of a corporate decision on standard coding conventions. Now all existing queries for age would be broken, and would have to be fixed by hand.

•Modern agile development techniques encourage constant refactoring to maintain a clean and up-to-date class model that accurately represents an evolving domain model. If query code is difficult to maintain, it delays decisions to refactor and inevitably leads to lowquality source code.

•All the queries in Example 2 operate against the private implementation of the Student class student.age instead of using its public interface student.getAge( )/student.Age (in Java/C#, respectively). Consequently, they break objectoriented encapsulation rules, disobeying the principle that interface and implementation should be decoupled.

•You are constantly required to switch contexts between implementation language and query language. Queries cannot use code that already exists in the implementation language.

•There is no explicit support for creating reusable query components. A complex query can be built by concatenating query strings, but none of the reusability features of the programming language (method calls, polymorphism, overriding) are available to make this process manageable. Passing a parameter to a string-based query is also awkward and error prone.

•Embedded strings can be subject to injection attacks.

Design Goals

Our goal is to propose a new approach that solves many of these problems. This article is an overview of the approach, not a complete specification. What if you could simply express the same query in plain Java or C#, as in Example 3? You could write queries without having to think about a custom query language or API. The IDE could actively help to reduce typos. Queries would be fully typesafe and accessible to the refactoring features of the IDE. Queries could also be prototyped,

tested, and run against plain collections in memory without a database back end.

At first, this approach seems unsuitable as a database query mechanism. Naively executing Java/C# code against the complete extent of all stored objects of a class would incur a huge performance penalty

“Queries should be expressed in the implementation language, and they should obey language semantics”

because all candidate objects would have to be instantiated from the database. A solution to this problem was presented in “Safe Query Objects” by William Cook and Siddhartha Rai [3].

The source code or bytecode of the Java/C# query expression can be analyzed and optimized by translating it to the underlying persistence system’s query language or API (SQL [6], OQL [1,8], JDOQL [7], EJBQL [1], SODA [10], and so on), and thereby take advantage of indexes and other optimizations of a database engine. Here, we refine the original idea of safe query objects to provide a more concise and natural definition of native queries. We also examine integrating queries into Java and .NET by leveraging recent features of those language environments, including anonymous classes and delegates.

Therefore, our goals for native queries include:

•100-percent native. Queries should be expressed in the implementation language (Java or C#), and they should obey language semantics.

•100-percent object oriented. Queries should be runnable in the language

derivation of the API by providing a full example. Specifically, we want to show what a query against a database would look like, so we can compare it against the string-based examples given in the introduction. Example 9 completes the core idea. We have refined Cook/Rai’s concept of safe queries by leveraging anonymous classes in Java and delegates in .NET. The result is a more concise and straightforward description of queries.

Adding all required elements of the API in a step-by-step fashion lets us find the most natural and efficient way of expressing queries in Java and C#. Additional features, such as parameterized and dynamic queries, can be included in native queries using a similar approach [4]. We have overcome the shortcomings of existing string-based query languages and provided an approach that promises improved productivity, robustness, and maintainability without loss of performance.

Specification Details

A final and thorough specification of native queries is only possible after practical experience. Therefore, this section is speculative. We would like to point out where we see choices and issues with the native query approach and how they might be resolved.

Regarding the API alone, native queries are not new. Without optimizations, we have merely provided “the simplest concept possible to run all instances of a class against a method that returns a Boolean value.” Such interfaces are well known: Smalltalk-80 [2, 5], for instance, includes methods to select items from a collection based on a predicate.

Optimization is the key new component of native queries. Users should be able to write native query expressions and the database should execute them with performance on par with the string-based queries that we described earlier.

Although the core concept of native queries is simple, the work needed to provide a solution is not trivial. Code written in a query expression must be analyzed and converted to an equivalent database query format. It is not necessary for all code in a native query to be translated. If the optimizer cannot handle some or all code in a query expression, there is always the fallback to instantiate the actual

(a)

// Java student.getAge() < 20

(b)

// C# student.Age < 20

Example 3: (a) Java query; (b) C# query.

objects and to run the query expression code, or part of it, with real objects after the query has returned intermediate values. Because this may be slow, it is helpful to provide developers with feedback at development time. This feedback might include how the optimizer “understands” query expressions, and some description of the underlying optimization plan created for the expressions. This will help developers adjust their development style to the syntax that is optimized best and will enable developers to provide feedback about desirable improved optimizations.

How will optimization actually work? At compile or load time, an enhancer (a separate application or a plug-in to the compiler or loader) inspects all native query expressions in source code or bytecode, and will generate additional code in the most efficient format the database engine supplies. At runtime, this substituted code will be executed instead of the original Java/C# methods. This mechanism will be transparent to developers after they add the optimizer to their compilation or build process (or both).

Our peers have expressed doubts that satisfactory optimization is possible. Because both the native query format and the native database format are well defined, and because the development of an optimizer can be an ongoing task, we are very optimistic that excellent results are achievable. The first results that Cook/Rai produced with a mapping to JDO implementations are very encouraging. db4objects (http://www.db4o.com/) already shows a first preview of db4o with unoptimized native queries today and plans to ship a

//pseudoJava (Student student){

return student.getAge() < 20

&&student.getName().contains("f");

}

//pseudoC#

(Student student){

return student.Age < 20

&& student.Name.Contains("f");

}

Example 5: PseudoJava and pseudoC#.

(a)

// Java

new Predicate(){

public boolean match(Student student){ return student.getAge() < 20

&& student.getName().contains("f");

}

}

(b)

// C#

delegate(Student student){ return student.Age < 20

&& student.Name.Contains("f");

}

Example 6: (a) Java; (b) C#.

SQL dialects may be achievable for native queries: Because the standard is well defined by programming-language specifications, native queries can provide 100-percent compatibility across different database implementations.

•Efficiency. Native queries can be automatically compiled to traditional query languages or APIs to leverage existing high-performance database engines.

•Simplicity. As shown, the API for native queries is only one class with one method. Hence, native queries are easy to learn, and a standardization body will find them easy to define. They could be submitted as a JSR to the Java Community Process.

Acknowledgments

Thanks to Johan Strandler for his posting to a thread at TheServerSide that brought the two authors together, Patrick Roomer for getting us started with first drafts of this paper, Rodrigo B. de Oliveira for contributing the delegate syntax for .NET, Klaus Wuestefeld for suggesting the term “native queries,” Roberto Zicari, Rick Grehan, and Dave Orme for proofreading drafts of this article, and to all of the above for always being great peers to review ideas.

References

[1]Cattell, R.G.G., D.K. Barry, M. Berler, J. Eastman, D. Jordan, C. Russell, O. Schadow, T. Stanienda, and F. Velez, editors. The Object Data Standard ODMG 3.0. Morgan Kaufmann, 2000.

[2]Cook, W.R. “Interfaces and Specifications for the Smalltalk Collection Classes.” OOPSLA, 1992.

[3]Cook, W.R. and S. Rai. “Safe Query Objects: Statically Typed Objects as Remotely Executable Queries.” G.C. Roman, W.G. Griswold, and B. Nuseibeh, editors. Proceedings of the 27th International Conference on Software Engineering (ICSE), ACM, 2005.

[4]db4objects (http://www.db4o.com/).

[5]Goldberg, A. and D. Robson. Small- talk-80: The Language and Its Implementation. Addison-Wesley, 1983.

[6]ISO/IEC. Information technology — database languages — SQL — Part 3: Call-level interface (SQL/CLI). Technical Report 9075–3:2003, ISO/IEC, 2003.

[7]JDO (http://java.sun.com/products/ jdo/).

[8]ODMG (http://www.odmg.org/).

[9]Russell, C. Java Data Objects (JDO) Spec-

ification JSR-12. Sun Microsystems, 2003. [10]Simple Object Database Access (SODA) (http://sourceforge.net/projects/

sodaquery/).

[11]Sun Microsystems. Enterprise JavaBeans Specification, Version 2.1. 2002 (http://java.sun.com/j2ee/docs.html).

DDJ

A new way to profile Java applications

IAN FORMANEK

AND GREGG SPORAR

Profiling Java applications is often considered a black art. There are tools available to help you track down performance bottlenecks and memory-allocation problems, but they are not always easy to use, particularly when profiling large applications. Because large applications tend to be those that most need profiling, this presents a significant problem. It is therefore not surprising that Java applications have a reputation for running slowly — in many cases, this is solely because no performancerelated analysis and

tuning has been done.

Dynamic bytecode instrumentation is an innovative solution to these problems. It lets you control precisely which parts of an application are profiled. As a result, only relevant information is reported, and the impact on application performance is reduced so that even large applications can be profiled easily.

Gregg is a technology evangelist for Sun Microsystems and Ian is project lead and architect of the NetBeans Profiler. They can be contacted at gregg.sporar@sun

.com and ian.formanek@sun.com, respectively.

Obstacles to Profiling

The two biggest obstacles to profiling have been runtime overhead and interpretation of the results.

The runtime overhead imposed by a Java profiler can be a showstopper. The instrumentation added by the profiler can cause the application to run differently — which may change the performance problem symptoms, making it harder to find the cause of the problem. At the very least, if the application is running more slowly, it will take longer for it to get to the point where problems occur. In a worst-case scenario, the application might not even run correctly at all — unexpected timeouts caused by slower performance could result in an application crash.

After wrestling with those issues, you then have to interpret the results produced by the profiler. This can be overwhelming, even when working on a small application. For large applications, it can be a serious impediment to tracking down the cause of the problem. The larger the application, the higher the likelihood it has code in it that you did not write and therefore might not understand. This is particularly true for web and enterprise applications with several layers of abstraction that are specifically designed to be hidden from developers. Additionally, larger applications tend to have larger heaps and more threads. Profilers will deliver information on all of these things, which is usually more information than you can interpret efficiently. Filters are provided, but frequently are not precise enough and can therefore end up excluding useful information.

Traditional Profiler Technologies

There are two Java APIs available for profiling — the Java Virtual Machine Profiling Interface (JVMPI) and Java Virtual Machine Tool Interface (JVMTI). Most Java profiling tools use one of these APIs for doing instrumentation of an application and for notification of Virtual Machine (VM) events.

“Dynamic bytecode instrumentation lets you control precisely which parts of an application are profiled”

The most widely used profiling technique is bytecode instrumentation. The profiler inserts bytecode into each class. For CPU performance profiling, these bytecodes are typically methodEntry( ) and methodExit( ) calls. For memory profiling, the bytecodes are inserted after each new or after each constructor. All of this insertion of bytecodes is done either by a postcompiler or a custom class loader.

The key limitation of this technique is that once a class has been modified, it does not change. That lack of flexibility causes problems. If you choose to profile the entire application, the overhead of all