156 Java Concurrency In Practice

Listing 12.3. Testing Blocking and Responsiveness to Interruption.

void testTakeBlocksWhenEmpty() {

final BoundedBuffer<Integer> bb = new BoundedBuffer<Integer>(10); Thread taker = new Thread() {

public void run() { try {

int unused = bb.take();

fail(); // if we get here, it's an error } catch (InterruptedException success) { }

}}; try {

taker.start(); Thread.sleep(LOCKUP_DETECT_TIMEOUT); taker.interrupt(); taker.join(LOCKUP_DETECT_TIMEOUT); assertFalse(taker.isAlive());

} catch (Exception unexpected) { fail();

}

}

12.1.3. Testing Safety

The tests in Listings 12.2 and 12.3 test important properties of the bounded buffer, but are unlikely to disclose errors stemming from data races. To test that a concurrent class performs correctly under unpredictable concurrent access, we need to set up multiple threads performing put and take operations over some amount of time and then somehow test that nothing went wrong.

Constructing tests to disclose safety errors in concurrent classes is a chicken and egg problem: the test programs themselves are concurrent programs. Developing good concurrent tests can be more difficult than developing the classes they test.

The challenge to constructing effective safety tests for concurrent classes is identifying easily checked properties that will, with high probability, fail if something goes wrong, while at the same time not letting the failure auditing code limit concurrency artificially. It is best if checking the test property does not require any synchronization.

One approach that works well with classes used in producer consumer designs (like BoundedBuffer) is to check that everything put into a queue or buffer comes out of it, and that nothing else does. A naive implementation of this approach would insert the element into a "shadow" list when it is put on the queue, remove it from the list when it is removed from the queue, and assert that the shadow list is empty when the test has finished. But this approach would distort the scheduling of the test threads because modifying the shadow list would require synchronization and possibly blocking.

A better approach is to compute checksums of the elements that are enqueued and dequeued using an order sensitive checksum function, and compare them. If they match, the test passes. This approach works best when there is a single producer putting elements into the buffer and a single consumer taking them out, because it can test not only that the right elements (probably) came out but that they came out in the right order.

Extending this approach to a multiple producer, multiple consumer situation requires using a checksum function that is insensitive to the order in which the elements are combined, so that multiple checksums can be combined after the test. Otherwise, synchronizing access to a shared checksum field could become a concurrency bottleneck or distort the timing of the test. (Any commutative operation, such as addition or XOR, meets these requirements.)

To ensure that your test actually tests what you think it does, it is important that the checksums themselves not be guessable by the compiler. It would be a bad idea to use consecutive integers as your test data because then the result would always be the same, and a smart compiler could conceivably just precompute it.

To avoid this problem, test data should be generated randomly, but many otherwise effective tests are compromised by a poor choice of random number generator (RNG). Random number generation can create couplings between classes and timing artifacts because most random number generator classes are threadsafe and therefore introduce additional synchronization.[4] Giving each thread its own RNG allows a non thread safe RNG to be used.

[4] Many benchmarks are, unbeknownst to their developers or users, simply tests of how great a concurrency bottleneck the RNG is.

Rather than using a general purpose RNG, it is better to use simple pseudorandom functions. You don't need high quality randomness; all you need is enough randomness to ensure the numbers change from run to run. The xor-Shift function in Listing 12.4 (Marsaglia, 2003) is among the cheapest medium quality random number functions. Starting it

6BPart III: Liveness, Performance, and Testing 24BChapter 12. Testing Concurrent Programs 157

off with values based on hashCode and nanoTime makes the sums both unguessable and almost always different for each run.

Listing 12.4. MediumǦquality Random Number Generator Suitable for Testing.

static int xorShift(int y) { y ^= (y << 6);

y ^= (y >>> 21); y ^= (y << 7); return y;

}

PutTakeTest in Listings 12.5 and 12.6 starts N producer threads that generate elements and enqueue them, and N consumer threads that dequeue them. Each thread updates the checksum of the elements as they go in or out, using a per thread checksum that is combined at the end of the test run so as to add no more synchronization or contention than required to test the buffer.

Depending on your platform, creating and starting a thread can be a moderately heavyweight operation. If your thread is short running and you start a number of threads in a loop, the threads run sequentially rather than concurrently in the worst case. Even in the not quite worst case, the fact that the first thread has a head start on the others means that you may get fewer interleavings than expected: the first thread runs by itself for some amount of time, and then the first two threads run concurrently for some amount of time, and only eventually are all the threads running concurrently. (The same thing happens at the end of the run: the threads that got a head start also finish early.)

We presented a technique for mitigating this problem in Section 5.5.1, using a CountDownLatch as a starting gate and another as a finish gate. Another way to get the same effect is to use a CyclicBarrier, initialized with the number of worker threads plus one, and have the worker threads and the test driver wait at the barrier at the beginning and end of their run. This ensures that all threads are up and running before any start working. PutTakeTest uses this technique to coordinate starting and stopping the worker threads, creating more potential concurrent interleavings. We still can't guarantee that the scheduler won't run each thread to completion sequentially, but making the runs long enough reduces the extent to which scheduling distorts our results.

The final trick employed by PutTakeTest is to use a deterministic termination criterion so that no additional inter thread coordination is needed to figure out when the test is finished. The test method starts exactly as many producers as consumers and each of them puts or takes the same number of elements, so the total number of items added and removed is the same.

Tests like PutTakeTest tend to be good at finding safety violations. For example, a common error in implementing semaphore controlled buffers is to forget that the code actually doing the insertion and extraction requires mutual exclusion (using synchronized or ReentrantLock). A sample run of PutTakeTest with a version of BoundedBuffer that omits making doInsert and doExtract synchronized fails fairly quickly. Running PutTakeTest with a few dozen threads iterating a few million times on buffers of various capacity on various systems increases our confidence about the lack of data corruption in put and take.

Tests should be run on multiprocessor systems to increase the diversity of potential interleavings. However, having more than a few CPUs does not necessarily make tests more effective. To maximize the chance of detecting timing sensitive data races, there should be more active threads than CPUs, so that at any given time some threads are running and some are switched out, thus reducing the predictability of interactions between threads.