Chapter 11. Test Quality
See how fast the tests run. If you find it unacceptably long, or notice some pauses during the execution of the tests, then look further, and answer the following questions:
•Are these unit tests, or rather integration tests? Typically, setting up an application context or database connection takes time.
•Are the setup methods used properly? Although this happens rarely with unit tests, do look to see whether some objects might, for example, be created before the class rather than just before each method (see Section 3.8.4).
•Are there any deliberate Thread.sleep() calls? (See the next section for some discussion).
Also, look at the messages printed during the execution of the tests (usually on the console). If you cannot keep up with them, it means you have a Loudmouth issue (see Section 8.5).
Make sure the tests are repeatable. Usually they are, but in cases of multithreading tests (see Section 6.10), or tests with random values (see Section 7.2), this might not be the case.
Date Testing
Experience tells me that the testing of time-dependent business logic is rarely done correctly (see Section 6.11). Common issues you should look for are the following:
•Look for any Thread.sleep() code constructs which make unit tests run longer than is really required.
•A common mistake is to test only the current date, which means the bug will be discovered exactly on the day the code fails in production. In general you need to make sure all test cases are covered.
11.5.3. Conclusions
Of the many approaches to ensuring the quality of your test code, code reviews are to be recommended the most strongly. They help to find bugs, help your team to converge with respect to how they all go about coding, and serve to disseminate knowledge amongst team members about parts of the software being developed. Moreover, by examining the test code, a lot can be discovered about production code. Frankly, what more could one ask for? The only downside is that to perform a thorough examination of test code, a lot of time is required.
I strongly recommend making test code reviews a part of your team’s working routine. Code reviews should belong to your Definition of Done, and they should embrace test code in the same manner that they embrace production code.
11.6. Refactor Your Tests
God grant me serenity to accept the code I cannot change, courage to change the code I can, and wisdom to know the difference.
— Erik Naggum
Now that we know in what ways we can measure test quality, it is time to actually fix the deficiencies uncovered by code coverage (see Section 11.3), mutation testing (see Section 11.4) and code reviews
254
Chapter 11. Test Quality
(see Section 11.5). In this section we discuss the process of refactoring, which is a common way of altering code. We have already discussed a refactoring in Chapter 4, Test Driven Development, as one of the phases of the TDD rhythm. In this section we shall discuss the refactorings that are common when working with tests. Some of them are identical to the ones we perform on production code, but some are specific to test code.
Numerous examples of making improvements to test code by introducing changes of various sorts have been given already in other parts of this book – or, indeed, are easily deducible from what has been discussed earlier. For example, we already know about the advantages of using assertEquals() over assertTrue() for comparing objects, and we know what the advantages of matchers are (see Section 6.6): both can be the source of multiple refactorings, which should be used to improve the readability of assertions. Similarly, making use of data providers (instead of for loops), or having specialized setUp() methods, can also serve as a refactoring hints. This section supplements what has already been discussed with new examples and approaches that have yet to be examined.
Before we see some code, let us think about the reasons for test code refactorings. The first and most significant one is that the test code is very important and should be cleaned so that it is easy to understand and maintain. Refactoring helps you achieve this. By introducing various, sometimes minor, changes to test code, you can make your tests convey better exactly what you are trying to test. If a wellwritten test fails, it is also easier to find out why.
There is also a question about how to refactor test code. The situation is different from that with production code, because we do not have tests for our tests. However, we can deal with this issue by following these simple pieces of advice:
•You should perform refactorings in small steps, and rerun tests often along the way.
•Additional tools - code coverage and mutation testing - may help to find out if your safety net of tests is not getting loosened up.
Some people say that before starting to refactor tests, you should first change the SUT’s implementation, so the test fails21. After refactoring tests and rerunning them, you should still see them failing - which is a sign that the assertions are still working. Now, when you revert the changes in production code, the tests should pass again. I have to admit, I have never used the last technique during any serious refactoring. Moving in small steps worked well enough for me.
Another thing we should discuss is when to refactor a particular test. An obvious moment for this is the refactoring phase of the TDD rhythm, but this is not the only one. In fact, I would encourage you to do it every time you do not feel comfortable when browsing the test code. It does not matter if you have written that code yourself or not. Remember the Boy Scout Rule ("Leave the campground cleaner than you found it!") and dare to make the small changes which, added together, still make a big difference!
Before we see some examples, let me inform you that all of them involve real code. Some names of methods and classes have been changed to "protect the innocent". :)
The examples are only big enough to demonstrate what is to be done, and how the code changes after refactoring. Each of the refactorings shown addresses some real issues within the test code. Some of them might not seem very dangerous when looking at ten lines of test code examples, but their importance
21Please note that this approach is exactly the opposite of what you do when refactoring production code, which should be performed only if all tests are green.
255
Chapter 11. Test Quality
will grow along with the size of your test codebase. Likewise, the benefits of refactorings are much more clearly visible when they are introduced for real-sized test classes. Please bear this in mind when contemplating the examples in this section.
11.6.1. Use Meaningful Names - Everywhere
We have already discussed the importance of good naming. We have discussed various patterns for test class names and test method names, and pondered over the naming schema for test doubles (see Section 9.2). Throughout the book I have been encouraging you to rename things if you do not feel comfortable with the existing names. Good, meaningful names are invaluable!
This section presents another facet of the same issue. It shows how the simple refactoring of renaming variables can make it easier to understand the logic of test code.
Imagine a test of a class from a security module, which is supposed to allow or deny access to users based on their permissions. First the test registered users (identified by string id) and assigned them some permissions. Then it verified that users had only those permissions that had been assigned to them. Listing 11.6 shows a small part of this test.
Listing 11.6. User_1, user_2, user_3 - who are you?
public static Object[] userHasPermissions() { return $(
$("user_1", Permission.READ),
$("user_1", Permission.WRITE),
$("user_1", Permission.REMOVE),
$("user_2", Permission.WRITE),
$("user_2", Permission.READ),
$("user_3", Permission.READ)
);
}
@Parameters(method = "userHasPermissions") @Test
public void shouldReturnTrueIfUserHasPermission(String username, Permission permission) {
assertTrue(sut.hasPermission(username, permission));
}
The problem here is that this test is not obvious. For someone who looks at the test for the first time, it is not obvious whether user_2 should or should not have been granted the READ permission. Who the heck is user_2? Well, this must be checked by analyzing the data of previously registered users (probably in a setUp() method somewhere).
A simple change can achieve wonders. Look at the updated version shown in Listing 11.7.
256
Chapter 11. Test Quality
Listing 11.7. Admin, logged on user, guest - I know who you are!
public static Object[] usersPermissions() { return $(
$(ADMIN, Permission.READ),
$(ADMIN, Permission.WRITE),
$(ADMIN, Permission.REMOVE),
$(LOGGED, Permission.WRITE),
$(LOGGED, Permission.READ), $(GUEST, Permission.READ)
);
}
@Parameters(method = "usersPermissions") @Test
public void shouldReturnTrueIfUserHasPermission(String username, Permission permission) {
assertTrue(sut.hasPermission(username, permission));
}
Now this is clear! Admin should have all possible permissions. A "normal" logged on user should be able to read and write, but not to remove things. A guest user can normally only see what others have uploaded, but cannot change the content himself or herself. There is no need to consult any documentation: the code speaks for itself.
11.6.2. Make It Understandable at a Glance
The readability of tests can be improved in lots of ways. Take a look at the following snippet of test code, which creates an object of the MockServer class:
Listing 11.8. Not easy to understand what type of server is created
server = new MockServer(responseMap, true,
new URL(SERVER_ROOT).getPort(), false);
What properties does a server variable have? What kind of server is created? If you do not remember the API of MockServer, then you need to ask your IDE for help, so it explains the meaning of true and false flags being passed to the MockServer constructor. Would it be possible to change this code so it is easier to understand? Yes, by introducing some values whose names tell a story:
Listing 11.9. Self-explanatory values passed to the MockServer constructor
private static final boolean NO_SSL = false;
private static final boolean RESPONSE_IS_A_FILE = true;
server = new MockServer(responseMap, RESPONSE_IS_A_FILE, new URL(SERVER_ROOT).getPort(), NO_SSL);
Now this makes more sense - this server responds with a file, and does not use SSL.
Another way in which this code could be made more readable is by using the Test Data Builder pattern (see Section 9.6.2).
257