Chapter 1. On Tests and Tools

1.2.5. Conclusions

All of the types of test presented in the preceding sections are important. From the point of view of a development team, each of them will have its own value. Unit tests help to ensure high-quality code, integration tests verify that different modules are cooperating effectively, while end-to-end tests put the system through its paces in ways that reflect the standpoint of users. Depending on the type of application you are implementing, some of them may be more suitable than others.

Another way to think about the various types of test is to place them on an scale. At one end of this scale are unit tests, whose role is just to check whether we are implementing a given system correctly. At the other are end-to-end tests, whose main purpose is to verify that we are implementing the right system. Integration tests lie somewhere between.

This book concentrates on unit tests, only to a very limited extent touching on other kinds of test. However, it is very important to be aware of their existence, and not to rely solely on unit tests. Unit tests are the foundation of developers’ tests, but rarely are they sufficient in themselves. Please bear this in mind as you learn about unit tests.

So which tests should you write for your application? Alas, there is no easy answer to this question. No golden rule exists, which would describe the right proportion of tests of different kinds. It depends to a very high degree on the type of application you are writing.

1.3. Verification and Design

The continuum of testing approaches falls between two opposing beliefs. I will introduce both extremities to make the distinction clear.

Some people (I will call them verifiers for convenience) want to check that their code works. That is their goal – to make sure it does what it should do. In the case of code that is hard to test, they will resort to any available techniques to be able to test it. They will sacrifice some OO rules if they believe that is what they need to do to achieve their Holy Grail of testability. They will modify method visibility using reflection or use classloading hacks to deal with final classes. In this way they are able to test just about anything, including tons of nightmarish legacy6 code. When accused of using "dirty hacks", they shrug their shoulders, and reply that they "don’t feel dirty if they are already swimming in mud".

The other group – let us call them designers – believe that following OO rules is the most important thing, and that it leads to easily testable code. They treat tests as an indicator of code health. Easy-to- write tests denote sound code. Difficulties encountered during test-writing indicate problems in the code, and are treated as a clear sign that the code should be reworked. They tend to write tests using the same techniques as they use for production code, and renounce the use of reflection or classloading hacks. Designers particularly like the TDD approach, which guarantees a certain level of code quality. In the case of legacy code they will tend to refactor (or rather rewrite) it in order to make it more testable.

6By legacy code I mean any code without tests (i.e. unit tests).

7

Chapter 1. On Tests and Tools

As you can see, the conflict between these two approaches could never be resolved. The proponents hold different views, have different needs and value different things. Both also have some good examples to "prove" their superiority. The following paraphrase of a discussion on StackOverflow7 shows the difference between these two worlds:

-Reflection is the best way to test private methods.

-Yes, you should reflect on your design!

—Stack Overflow discussion (paraphrased)

This distinction is also visible if you examine the features offered by different testing tools that are available. Some of them (e.g. JMockit and Powermock) are there to test the untestable, by giving you the power to mock static classes, final classes and constructors, or to call private methods. Others avoid using any such hacks. For example JUnit has never introduced any feature that would make testing of private methods easier, even though many have requested such a thing since JUnit’s early days.

The terms designer and verificator have been introduced to stress a significant difference in how one may approach testing. However, I know no one who would be 100% a designer or 100% a verificator. We all fall somewhere in between.

I’m inclined to position myself closer to designers – I share their concern for good design. This has an obvious impact on the tools and testing techniques I use.

1.4. But Should Developers Test Their

Own Code?!

Probably you have heard, many times over, that you (a developer) should not test your own code. Many reasons are given in support of this claim, but two of them seem to stand out as being the strongest and most commonly used:

•developers lack testing skills,

•you should not be your own judge.

Let us be clear about this. Both of them are well-reasoned and, without a doubt, both emerged on the basis of the real – and no doubt sad – experiences of many development teams. They should not be dismissed too easily. Yet I am convinced that this "common knowledge" about testing reflects a kind of misunderstanding – or maybe, rather, a general failure to appreciate the multifariousness of the characteristics and purposes of testing.

If we are talking about final tests before shipping software to customers, then I believe that in general such tests should be executed by professional testers. I would agree that no developer can click through the GUI and be as aggressive and inquisitive as an experienced tester. But those are not the only tests out there! There are many valuable tests that can, and should, be performed by developers themselves.

What is more, some software is not easily testable by anyone other than developers themselves! Think about all those back-end solutions. No GUI, no decent entry points, a (sometimes) strong relation to architectural (hardware) aspects of the environment, etc.

7http://stackoverflow.com

8

Chapter 1. On Tests and Tools

Checking software from the customer’s point of view is crucial, but this is only a single piece of a larger puzzle. Before you can do that, a development team must provide you with software. And if they do not perform their own tests – developers’ tests – they will probably furnish you with something of low quality. Developers’ tests increase the quality of the product delivered to the customer, but also that of the codebase, which means a great deal for any development team. This is not something to be disdained. The more trust a development team has (and the more pride it takes!) in its code, the better the results it will achieve. Developers’ tests help a team to gain confidence and proceed further without being hampered by too much self-doubt.

Also, catching bugs early on (greatly) reduces cost, and shortens the time of repair. The more bugs you find early on, the less they will cost you. This well-known time-to-cost ratio is shown in Figure 1.5.

Figure 1.5. The cost of bug fixing

Developers’ tests are the first line of defense against bugs. They kill them as soon as they appear. Of course, for the reasons mentioned at the beginning of this section, some bugs will probably make it through. Well, yes, it might just happen! That is why other lines of defense have to be in place, too: i.e. highly skilled, specialized testers. Hopefully they will hunt down all the remaining bugs8.

In fact, many companies rely (almost) solely on developers’ tests. Big names – like Facebook or WordPress – adopt a continuous deployment approach, which can be summarized as "if it has passed the automatic tests it goes into production". No human testing involved! So it is possible after all, isn’t it?

So, should developers tests their own code? Oh yes, they should!

…and if you disagree, please stop reading now.

1.5. Tools Introduction

Use the right tool for the job.

— Andrew Hunt and David Thomas The Pragmatic Programmer: From Journeyman to Master (1999)

This section introduces the tools that I will be using for the writing and execution of tests. It relates to our recent discussion about the different approaches to testing (see Section 1.3). These different approaches are responsible for the existence of many tools covering (almost) identical problem areas. Take, for

8We will discuss this topic some more in Section A.1.

9

Chapter 1. On Tests and Tools

example, mocking frameworks. Do we really need so many of them? The answer is "yes", and the reason is that each mocking framework is slightly different, and facilitates a slightly different approach to writing test doubles. This is also true for other groups of tools – from test frameworks to IDEs.

In general, tools for testing are very simple to use. That is good news, isn’t it? But be warned – there is a catch! This deceptive ease of use leads many developers to assume that they know how to test, just because they can use testing tools – i.e. they are able to write a few lines of JUnit code. This is plainly wrong. Tools can be used mindlessly, or they can be used by a skilled hand. They can dominate you or they can become your obedient servants. The point is to grasp the ‘why’ and the ‘what for’, so that you know when to use (or not use) them.

Throughout this book I will be stressing the importance of the ideas embodied in certain tools. If you get a good grasp of those ideas, you will be able to follow them with almost any tool. If you concentrate on tools, you will soon need to relearn all that was dear to you. Ideas are everlasting9, but tools are there only for a short period of time.

Let me introduce my friends now. There are a few of them, but we will mostly use just two: JUnit and Mockito. The remainder will only play a secondary role.

And what if your choices are different, and you use different tools? It is not a problem. Nowadays tools are very similar. They have evolved along different paths, but have also influenced one another, and have often ended up in the proximity of their competitors, with similar sets of features. Using any modern tools you can achieve similar results and still utilize the techniques presented in this book. The ones I have selected are my personal favourites, and have been chosen with great deliberation. I suspect that some of the techniques presented in the book may be easier to master using them than they would be using any other tools. The only risk for you is that you, too, may end up convinced of their superiority, so that you then add some new toys to your toolbox. This doesn’t sound so bad, does it?

Testing Framework: JUnit

JUnit (http://junit.org) is an open-source testing framework for Java. It was created by Kent Beck around 1997, and since that time has been, de facto, the standard testing tool for Java developers. It is supported by all IDEs (Eclipse, IntelliJ IDEA), build tools (Ant, Maven, Gradle) and by popular frameworks (e.g. Spring). JUnit has a wide community of users, and is supplemented by a range of interesting extension projects. It was built especially for unit testing, but is also widely used for other kinds of test.

I used version 4.11 of JUnit when writing this book.

Mock Library: Mockito

Mockito (http://mockito.org) is a relatively new mocking framework (or rather test-spy framework). It was born in Q4 2007 (Szczepan Faber being the proud father) and has quickly matured into a top-quality product. It offers complete control over the mocking process, and "lets you write beautiful tests with clean & simple API". Originally, Mockito was derived from Easymock, but has evolved substantially, and now differs in many respects from its predecessor. It is very easy to use Mockito in combination with JUnit.

I used version 1.9.5 of Mockito when writing this book.

9Or rather, they live as long as the paradigm they belong to does.

10

Chapter 1. On Tests and Tools

Other Tools

It would be simplistic to say that everything other than the testing framework and mock library plays a secondary role. If you master writing unit tests, you will find good uses for many more tools. Here are my choices.

Matcher Libraries: FEST And Hamcrest

In order to make tests more readable and easier to maintain, in some parts of the book I have used the following matcher libraries: FEST Fluent Assertions (http://code.google.com/p/fest/) and Hamcrest (http://code.google.com/p/hamcrest/).

I used version 1.4 of FEST and version 1.3 of Hamcrest when writing this book.

JUnitParams

Even though JUnit is the most popular testing framework, it does not mean it is also the best in every respect. In fact it has some serious weak points, and support for parameterized tests is one of these. Fortunately, over the course of years many additional libraries have been created in order to provide a decent level of parameterized test support for JUnit users. I have decided to use the JUnitParams library (http://code.google.com/p/junitparams/).

I used version 1.0.0 of JUnitParams when writing this book.

EasyTest

The project website announces EasyTest as a "Data Driven Testing Framework which is based on JUnit framework". EasyTest provides some nice enhancements to the default parameterized tests of JUnit. In the book we will be using its Excel-files handling capabilities.

I used version 0.6.3 of EasyTest when writing this book.

Code Coverage: Cobertura

Among code coverage tools there are a few interesting ones, my personal choice being Cobertura (http:// cobertura.sourceforge.net). It works well with all build tools, IDEs, and continuous integration servers.

I used version 1.9.4.1 of Cobertura when writing this book.

Mock Libraries: PowerMock and EasyMock

Even though Mockito is the mocking framework used within the book, in some situations it might be worthwhile to take a look at the other options. Powermock (http://code.google.com/p/powermock/) offers some powerful features (e.g. mocking of final classes and static methods), which we will use once, just in order to be able to demonstrate and discuss an alternative approach to testing. EasyMock (http://easymock.org) will be used to demonstrate the difference between mock-based testing and spybased testing.

I used version 1.5 of PowerMock, and version 3.1 of EasyMock, when writing this book.

11

Chapter 1. On Tests and Tools

Mutation Testing: PIT

PIT Mutation Testing (http://pitest.org) is "a fast bytecode based mutation testing system for Java that makes it possible to test the effectiveness of your unit tests." It works with Java 5 and JUnit 4.6 (and above).

I used version 0.29 of PIT when writing this book.

Utilities: Catch-Exception, Tempus-Fugit, and Unitils

The Catch-Exception library (http://code.google.com/p/catch-exception/) helps to write tests that verify whether appropriate exceptions have been thrown by production code. It requires Java 1.6.

Unitils (http://www.unitils.org) is an "open source library aimed at making unit and integration testing easy and maintainable". We will use only a small subset of its features, namely the "Reflection asserts" module.

Tempus-fugit (http://tempusfugitlibrary.org) is a "Java micro-library for writing & testing concurrent code". Among other things it enhances JUnit with some capabilities which are important for testing code meant to be run by many threads in parallel.

I used version 1.0.4 of Catch-Exception, version 3.3 of Unitils, and version 1.1 of Tempus-Fugit when writing this book.

Build Tools: Gradle and Maven

Unit tests are usually included within the build process, which means they are run by a build tool. In this book I present how to run tests using Maven (http://maven.org) and Gradle (http://gradle.org).

I used version 1.3 of Gradle, and version 3.04 of Maven, when writing this book.

IDEs: IntelliJ IDEA and Eclipse

Even though IDE is THE tool in your toolbox, we will not devote a lot of time to it. All we need to know is how to use an IDE to execute unit tests. I decided to discuss it with reference to two very popular IDEs: Eclipse (http://eclipse.org) and IntelliJ IDEA (http://www.jetbrains.com/idea).

I used version Juno (4.2) of Eclipse, and version 12 (Community Edition) of IntelliJ IDEA, when writing this book.

12