Chapter 8. Getting Feedback

This report, by its very length, defends itself against the risk of being read.

— Winston Churchill

Writing tests, especially in a test-first manner, is all about getting rapid and precise feedback. We have already tackled this subject. We have discussed the green/red colors which indicate the success or failure of tests, and mentioned the importance of error messages. In this section we will learn what (detailed) information about the tests is available, and how to customize it to our liking.

When reading about the ways of getting feedback from test tools (usually in the form of various reports), please bear in mind that there are two common uses for them. First of all, they are here to help a developer learn about the issues with his tests. Secondly, they help the whole team (or even some stakeholders) to get a good understanding of the situation. While your IDE can help you with the former, your colleagues and superiors may require something more. This section covers both requirements

Is Customization Worth the Trouble? This section gives a lot of information about the customization of existing test reports and about writing custom ones. The question is, whether such activities are worth the trouble. Well, I cannot answer this question for you. It is up to you to decide. If, like me, you like tools to behave exactly as you want them to, if you set your own keyboard shortcuts in MS Word, Thunderbird or your IDE, if you believe that a one-off effort can bring everyday benefits for a long time to come, then you should probably read this carefully and think about how you would like your reports to be. But if you are usually happy with default settings (even to the extent of never touching the gear-system on your bike "cos it’s good the way it is right now") then simply read about the default reports, and do not bother with customization.

When talking about feedback information we must mention Javadocs. In contrast to what we have discussed so far, they do not give us information about the execution of tests, but help us instead to understand their purpose and business value. Some hints about writing Javadocs have already been given in Section 4.2.3.

8.1. IDE Feedback

If you usually run your tests via an IDE (see Appendix B, Running Unit Tests for some hints) then it is crucial that you know how to make the best of it. This section explains what information you can acquire from IntelliJ IDEA and Eclipse about the tests being executed. As both IDEs are subject to ongoing development, you should also check their original documentation. It is possible that some new helpful features will have been added after this book has been written.

8.1.1. Eclipse Test Reports

No matter if you use JUnit or TestNG, Eclipse prints information about the executed tests in a unified manner.

In the event that all your tests have passed, all you will usually want to know is which tests were run. Eclipse provides this information as shown below:

177

Chapter 8. Getting Feedback

Figure 8.1. Eclipse: passed tests

The green bar on top makes it clear there are no failed tests. The pane on the left provides the list of executed tests (in the form of a tree, so you can see which class and package they belong to). Information on execution time is also provided, even though this is usually not so important with unit tests. As Figure 8.1 shows, in cases of parameterized tests, detailed information on arguments is printed as well. Moreover, there is a console pane, which shows the output of each test and any other messages printed by the testing framework. (Even so, there are not many occasions when we would have a reason to want to check its content, if all of the tests have passed).

In the event of failure, Eclipse will warn you about the problem by changing the bar color from green to red. As Figure 8.2 shows, Eclipse also provides the following information:

•which test method failed and, in the case of parameterized tests, what the arguments were (in our case -7 and BLA),

•a failure exception message which is "clickable", so you can easily move to the test code and start fixing things.

Figure 8.2. Eclipse: failed tests

Eclipse also makes it possible to rerun all recently executed tests, or to run only the failed ones (these two options are marked with red rectangles in the picture above).

8.1.2. IntelliJ IDEA Test Reports

IntelliJ IDEA supports both JUnit and TestNG frameworks. It offers common commands and a common user interface (called Test Runner Tab) for running tests for each of them. In this section we shall be discussing what sort of information you can gain from IntelliJ IDEA after running your tests.

178

Chapter 8. Getting Feedback

In the event that all of your tests have passed, you will in general be interested in two things: what tests were run and how long it took (this is especially important if you run integration or end-to-end tests). Both kinds of information are provided by IntelliJ IDEA, but you will need to configure it a little bit to get to see them. By default, when all tests pass, you see only the basic status information - a green bar and a printed summary message. This is shown in Figure 8.3.

Figure 8.3. IntelliJ IDEA: passed tests

If you want to know more you need to customize the view. IntelliJ IDEA will use the latest settings of the Test Runner Tab, so configuring it will be a one-off action. In order to see what tests have been run, switch off the Hide Passed button and select the Expand All button. In addition, to get the test execution statistics (i.e. execution time) select the Show Statistics option from the cog button menu. The resulting view is shown in Figure 8.4 (red rectangles mark the aforementioned options).

You can adjust the size of each pane to your liking (when preparing illustrations for this book, I was obliged to shrink this slightly, so unfortunately some parts may not be fully visible).

Figure 8.4. IntelliJ IDEA: passed tests - customized view

This time much more information is available. The left pane lists all the test methods executed during the last run in the form of a tree (with each package and class as a node, and each test method as a tree leaf). The pane in the middle shows the console output of each of the tests (in this case the tests have not printed any messages, so it is almost empty). Clicking on test names (in the left panel) results in the middle pane showing the test output for a selected test case.

The pane on the right shows the test execution statistics. They are not very useful for unit tests, because the execution time of such tests is negligible, so you may as well switch statistics off.

In the event of failure, IntelliJ IDEA will warn you about the problem by changing the bar color from green to red. Obviously, when some test fails, you will want to fix the bug right away, so your expectations of support from IDE are going to be much higher than if all of the tests passed. First of

179