Chapter 7. Points of Controversy

Listing 7.6. Creation of objects in every test method

public class EachMethodTest {

@Test

public void testA() {

Collaborator collaborator = mock(Collaborator.class); OtherCollaborator otherCollaborator = mock(OtherCollaborator.class); SUT sut = new SUT(collaborator, otherCollaborator);

sut.someMethod();

// assertions

}

@Test

public void testB() {

Collaborator collaborator = mock(Collaborator.class); OtherCollaborator otherCollaborator = mock(OtherCollaborator.class); SUT sut = new SUT(collaborator, otherCollaborator);

sut.otherMethod();

// assertions

}

}

An obvious advantage of this approach is that every method can be understood without the need to look into other parts of the code.

On the other hand, creating the same SUT objects and collaborators in multiple test methods also has its downsides. Test classes grow in length, and the DRY principle is violated. One could also question the readability of test methods: they are often quite long and contain a lot of set-up logic. It is possible that the essence of the test case is lost in all this set-up code.

And what is the correct answer to this riddle? I would say there are few factors in play here:

The DRY principle. If the DRY principle is very dear to you, you will probably use the set-up methods approach. After you have got used to the fact that some of the test logic is also included within set-up methods, you will have no problems reading your tests.

The readability of tests. On the other hand, if you are into BDD then you will, instead, appreciate your tests’ being self-contained.

Consistency. If you like your codebase to be written in the same way throughout, you will probably want to follow one approach in all your tests. If this is not the most important factor, then you will probably do what I did when writing tests for this book: use the set-up approach for test classes with multiple test methods (and multiple collaborators), but put the test-fixture creation within test methods in cases of shorter tests.

Complexity. Sometimes the complexity of the test case will make the decision for you. For more complex tests you will end up with a mixture of the above techniques, so be prepared for that!

7.4. How Many Assertions per Test Method?

Chuck Norris can unit test an entire application with a single assert.

152

Chapter 7. Points of Controversy

— Wisdom of the Internet ;)

The idea of having only One Assertion Per Test Method (I will use the acronym OAPTM for this from here on) is quite popular among developers. Some of its popularity may be attributed to the misuse of multi-assertion test methods, which tend to grow into many lines of code (in the hands of inexperienced or incompetent developers). It has also been discussed on popular forums and promoted by a number of prominent TDD and testing proponents, including Dave Astel and Roy Osherove.

All of the code examples presented so far in this book make it quite obvious that I myself do not subscribe to this idea and do not follow it. And even though I understand and share some of the motivations behind it, I will try to persuade you not to subscribe to it either – at least, not in its dogmatic version.

7.4.1. Code Example

Let us take a look at an example, to make sure we understand the problem in hand. I will use an example taken from the "One Assertion per Test" post by Dave Astels6. It relates to the problem of parsing some information passed as a String argument and creating a reasonable domain class out of it. In Listing 7.7 I use original test method names.

Listing 7.7. Address parsing – one assert per test method

public class AddressParsingOneAssertTest {

private Address anAddress

= new Address("ADDR1$CITY IL 60563$COUNTRY");

@Test

public void testAddr1() { assertEquals("ADDR1", anAddress.getAddr1());

}

@Test

public void testCsp() {

assertEquals("CITY IL 60563", anAddress.getCsp());

}

@Test

public void testCountry() { assertEquals("COUNTRY", anAddress.getCountry());

}

}

setting the test fixture for every test method, each test method contains only one assert.

As you can see, OAPTM in us having a lot of very focused, tiny, often even just one-line, test methods. To get a broader understanding of "what address parsing looks like", you need to scan at least a few of them. What you get in return is that if a constructor of the Address class does not work properly, you will know all about it - i.e. failing test cases will tell you exactly what works and what doesn’t.

A counterpart test – i.e. a test which uses several assertions per test method – is shown in Listing 7.8.

6See http://www.artima.com/weblogs/viewpost.jsp?thread=35578

153

Chapter 7. Points of Controversy

Listing 7.8. Address parsing – several assertions per test method

public class AddressParsingManyAsserts {

@Test

public void testAddressParsing() {

Address anAddress = new Address("ADDR1$CITY IL 60563$COUNTRY");

assertEquals("ADDR1", anAddress.getAddr1()); assertEquals("CITY IL 60563", anAddress.getCsp()); assertEquals("COUNTRY", anAddress.getCountry());

}

}

Only one test method - with three assertions.

No need for setUp method.

This time the number of test methods is significantly smaller, but they themselves are slightly larger (though still reasonably short). It is very easy to understand what "address parsing" means, since all parsing assertions are grouped together. On the downside, though, it might happen that any bugs introduced into the Address class constructor only reveal themselves gradually – one by one, in consecutive runs of the AddressParsingManyAssertsTest test class.

One important thing to notice is that even though in the second version of the test there is more than one assert per test method, each test method tests only one thing - the ability of the Address constructor to parse addresses properly.

7.4.2. Pros and Cons

Allow me now to take up the commonly cited reasons for following the OAPTM rule (as put forward by its proponents) and comment on them as they relate to the code present in the Listing 7.7 and Listing 7.8.

Table 7.2. Arguments for using only one assert per test

154

Chapter 7. Points of Controversy

7.4.3. Conclusions

Even as a firm believer in small, focused, easy-to-understand methods, I think that OAPTM goes too far, and does more harm than good. It results in too many lines of code being scattered amongst a multitude of tiny methods – something which is much harder to understand and maintain. It saves me from one problem (having too many assertions per test method), only to inflict on me another (having too many test methods).

No. Sorry, OAPTM! I can judge on my own when a test method is getting too large, without needing to follow such dogmatic rules. And as long as I stick to the TDD approach, I will rarely, if ever, be stricken by the downside of the multiple asserts approach. My test methods are very concise and readable, and I can easily find intention-revealing names for them. My IDE will guide me straight to the fallen assertion every time a test fails.

My advice would be not to take the "one assert per test" recommendation too literally. Get clear about the reasons behind it, but follow another, similar rule, which says that each test method should concentrate on a single feature of the SUT. Sometimes numerous assertions will be required to test a certain feature, sometimes one will suffice. But in my opinion, the number of assertions should not be your main concern. What really matters is the scope of the SUT features covered by each test method.

The only thing that would get me writing so many tiny test methods would be if I was being paid by LOCs. :)

155