- •Practical Unit Testing with JUnit and Mockito
- •Table of Contents
- •About the Author
- •Acknowledgments
- •Preface
- •Preface - JUnit
- •Part I. Developers' Tests
- •Chapter 1. On Tests and Tools
- •1.1. An Object-Oriented System
- •1.2. Types of Developers' Tests
- •1.2.1. Unit Tests
- •1.2.2. Integration Tests
- •1.2.3. End-to-End Tests
- •1.2.4. Examples
- •1.2.5. Conclusions
- •1.3. Verification and Design
- •1.5. Tools Introduction
- •Chapter 2. Unit Tests
- •2.1. What is a Unit Test?
- •2.2. Interactions in Unit Tests
- •2.2.1. State vs. Interaction Testing
- •2.2.2. Why Worry about Indirect Interactions?
- •Part II. Writing Unit Tests
- •3.2. Class To Test
- •3.3. Your First JUnit Test
- •3.3.1. Test Results
- •3.4. JUnit Assertions
- •3.5. Failing Test
- •3.6. Parameterized Tests
- •3.6.1. The Problem
- •3.6.2. The Solution
- •3.6.3. Conclusions
- •3.7. Checking Expected Exceptions
- •3.8. Test Fixture Setting
- •3.8.1. Test Fixture Examples
- •3.8.2. Test Fixture in Every Test Method
- •3.8.3. JUnit Execution Model
- •3.8.4. Annotations for Test Fixture Creation
- •3.9. Phases of a Unit Test
- •3.10. Conclusions
- •3.11. Exercises
- •3.11.1. JUnit Run
- •3.11.2. String Reverse
- •3.11.3. HashMap
- •3.11.4. Fahrenheits to Celcius with Parameterized Tests
- •3.11.5. Master Your IDE
- •Templates
- •Quick Navigation
- •Chapter 4. Test Driven Development
- •4.1. When to Write Tests?
- •4.1.1. Test Last (AKA Code First) Development
- •4.1.2. Test First Development
- •4.1.3. Always after a Bug is Found
- •4.2. TDD Rhythm
- •4.2.1. RED - Write a Test that Fails
- •How To Choose the Next Test To Write
- •Readable Assertion Message
- •4.2.2. GREEN - Write the Simplest Thing that Works
- •4.2.3. REFACTOR - Improve the Code
- •Refactoring the Tests
- •Adding Javadocs
- •4.2.4. Here We Go Again
- •4.3. Benefits
- •4.4. TDD is Not Only about Unit Tests
- •4.5. Test First Example
- •4.5.1. The Problem
- •4.5.2. RED - Write a Failing Test
- •4.5.3. GREEN - Fix the Code
- •4.5.4. REFACTOR - Even If Only a Little Bit
- •4.5.5. First Cycle Finished
- •‘The Simplest Thing that Works’ Revisited
- •4.5.6. More Test Cases
- •But is It Comparable?
- •Comparison Tests
- •4.6. Conclusions and Comments
- •4.7. How to Start Coding TDD
- •4.8. When not To Use Test-First?
- •4.9. Should I Follow It Blindly?
- •4.9.1. Write Good Assertion Messages from the Beginning
- •4.9.2. If the Test Passes "By Default"
- •4.10. Exercises
- •4.10.1. Password Validator
- •4.10.2. Regex
- •4.10.3. Booking System
- •Chapter 5. Mocks, Stubs, Test Spies
- •5.1. Introducing Mockito
- •5.1.1. Creating Test Doubles
- •5.1.2. Expectations
- •5.1.3. Verification
- •5.1.4. Conclusions
- •5.2. Types of Test Double
- •5.2.1. Code To Be Tested with Test Doubles
- •5.2.2. The Dummy Object
- •5.2.3. Test Stub
- •5.2.4. Test Spy
- •5.2.5. Mock
- •5.3. Putting it All Together
- •5.4. Example: TDD with Test Doubles
- •5.4.2. The Second Test: Send a Message to Multiple Subscribers
- •Refactoring
- •5.4.3. The Third Test: Send Messages to Subscribers Only
- •5.4.4. The Fourth Test: Subscribe More Than Once
- •Mockito: How Many Times?
- •5.4.5. The Fifth Test: Remove a Subscriber
- •5.4.6. TDD and Test Doubles - Conclusions
- •More Test Code than Production Code
- •The Interface is What Really Matters
- •Interactions Can Be Tested
- •Some Test Doubles are More Useful than Others
- •5.5. Always Use Test Doubles… or Maybe Not?
- •5.5.1. No Test Doubles
- •5.5.2. Using Test Doubles
- •No Winner So Far
- •5.5.3. A More Complicated Example
- •5.5.4. Use Test Doubles or Not? - Conclusion
- •5.6. Conclusions (with a Warning)
- •5.7. Exercises
- •5.7.1. User Service Tested
- •5.7.2. Race Results Enhanced
- •5.7.3. Booking System Revisited
- •5.7.4. Read, Read, Read!
- •Part III. Hints and Discussions
- •Chapter 6. Things You Should Know
- •6.1. What Values To Check?
- •6.1.1. Expected Values
- •6.1.2. Boundary Values
- •6.1.3. Strange Values
- •6.1.4. Should You Always Care?
- •6.1.5. Not Only Input Parameters
- •6.2. How to Fail a Test?
- •6.3. How to Ignore a Test?
- •6.4. More about Expected Exceptions
- •6.4.1. The Expected Exception Message
- •6.4.2. Catch-Exception Library
- •6.4.3. Testing Exceptions And Interactions
- •6.4.4. Conclusions
- •6.5. Stubbing Void Methods
- •6.6. Matchers
- •6.6.1. JUnit Support for Matcher Libraries
- •6.6.2. Comparing Matcher with "Standard" Assertions
- •6.6.3. Custom Matchers
- •6.6.4. Advantages of Matchers
- •6.7. Mockito Matchers
- •6.7.1. Hamcrest Matchers Integration
- •6.7.2. Matchers Warning
- •6.8. Rules
- •6.8.1. Using Rules
- •6.8.2. Writing Custom Rules
- •6.9. Unit Testing Asynchronous Code
- •6.9.1. Waiting for the Asynchronous Task to Finish
- •6.9.2. Making Asynchronous Synchronous
- •6.9.3. Conclusions
- •6.10. Testing Thread Safe
- •6.10.1. ID Generator: Requirements
- •6.10.2. ID Generator: First Implementation
- •6.10.3. ID Generator: Second Implementation
- •6.10.4. Conclusions
- •6.11. Time is not on Your Side
- •6.11.1. Test Every Date (Within Reason)
- •6.11.2. Conclusions
- •6.12. Testing Collections
- •6.12.1. The TDD Approach - Step by Step
- •6.12.2. Using External Assertions
- •Unitils
- •Testing Collections Using Matchers
- •6.12.3. Custom Solution
- •6.12.4. Conclusions
- •6.13. Reading Test Data From Files
- •6.13.1. CSV Files
- •6.13.2. Excel Files
- •6.14. Conclusions
- •6.15. Exercises
- •6.15.1. Design Test Cases: State Testing
- •6.15.2. Design Test Cases: Interactions Testing
- •6.15.3. Test Collections
- •6.15.4. Time Testing
- •6.15.5. Redesign of the TimeProvider class
- •6.15.6. Write a Custom Matcher
- •6.15.7. Preserve System Properties During Tests
- •6.15.8. Enhance the RetryTestRule
- •6.15.9. Make an ID Generator Bulletproof
- •Chapter 7. Points of Controversy
- •7.1. Access Modifiers
- •7.2. Random Values in Tests
- •7.2.1. Random Object Properties
- •7.2.2. Generating Multiple Test Cases
- •7.2.3. Conclusions
- •7.3. Is Set-up the Right Thing for You?
- •7.4. How Many Assertions per Test Method?
- •7.4.1. Code Example
- •7.4.2. Pros and Cons
- •7.4.3. Conclusions
- •7.5. Private Methods Testing
- •7.5.1. Verification vs. Design - Revisited
- •7.5.2. Options We Have
- •7.5.3. Private Methods Testing - Techniques
- •Reflection
- •Access Modifiers
- •7.5.4. Conclusions
- •7.6. New Operator
- •7.6.1. PowerMock to the Rescue
- •7.6.2. Redesign and Inject
- •7.6.3. Refactor and Subclass
- •7.6.4. Partial Mocking
- •7.6.5. Conclusions
- •7.7. Capturing Arguments to Collaborators
- •7.8. Conclusions
- •7.9. Exercises
- •7.9.1. Testing Legacy Code
- •Part IV. Listen and Organize
- •Chapter 8. Getting Feedback
- •8.1. IDE Feedback
- •8.1.1. Eclipse Test Reports
- •8.1.2. IntelliJ IDEA Test Reports
- •8.1.3. Conclusion
- •8.2. JUnit Default Reports
- •8.3. Writing Custom Listeners
- •8.4. Readable Assertion Messages
- •8.4.1. Add a Custom Assertion Message
- •8.4.2. Implement the toString() Method
- •8.4.3. Use the Right Assertion Method
- •8.5. Logging in Tests
- •8.6. Debugging Tests
- •8.7. Notifying The Team
- •8.8. Conclusions
- •8.9. Exercises
- •8.9.1. Study Test Output
- •8.9.2. Enhance the Custom Rule
- •8.9.3. Custom Test Listener
- •8.9.4. Debugging Session
- •Chapter 9. Organization Of Tests
- •9.1. Package for Test Classes
- •9.2. Name Your Tests Consistently
- •9.2.1. Test Class Names
- •Splitting Up Long Test Classes
- •Test Class Per Feature
- •9.2.2. Test Method Names
- •9.2.3. Naming of Test-Double Variables
- •9.3. Comments in Tests
- •9.4. BDD: ‘Given’, ‘When’, ‘Then’
- •9.4.1. Testing BDD-Style
- •9.4.2. Mockito BDD-Style
- •9.5. Reducing Boilerplate Code
- •9.5.1. One-Liner Stubs
- •9.5.2. Mockito Annotations
- •9.6. Creating Complex Objects
- •9.6.1. Mummy Knows Best
- •9.6.2. Test Data Builder
- •9.6.3. Conclusions
- •9.7. Conclusions
- •9.8. Exercises
- •9.8.1. Test Fixture Setting
- •9.8.2. Test Data Builder
- •Part V. Make Them Better
- •Chapter 10. Maintainable Tests
- •10.1. Test Behaviour, not Methods
- •10.2. Complexity Leads to Bugs
- •10.3. Follow the Rules or Suffer
- •10.3.1. Real Life is Object-Oriented
- •10.3.2. The Non-Object-Oriented Approach
- •Do We Need Mocks?
- •10.3.3. The Object-Oriented Approach
- •10.3.4. How To Deal with Procedural Code?
- •10.3.5. Conclusions
- •10.4. Rewriting Tests when the Code Changes
- •10.4.1. Avoid Overspecified Tests
- •10.4.2. Are You Really Coding Test-First?
- •10.4.3. Conclusions
- •10.5. Things Too Simple To Break
- •10.6. Conclusions
- •10.7. Exercises
- •10.7.1. A Car is a Sports Car if …
- •10.7.2. Stack Test
- •Chapter 11. Test Quality
- •11.1. An Overview
- •11.2. Static Analysis Tools
- •11.3. Code Coverage
- •11.3.1. Line and Branch Coverage
- •11.3.2. Code Coverage Reports
- •11.3.3. The Devil is in the Details
- •11.3.4. How Much Code Coverage is Good Enough?
- •11.3.5. Conclusion
- •11.4. Mutation Testing
- •11.4.1. How does it Work?
- •11.4.2. Working with PIT
- •11.4.3. Conclusions
- •11.5. Code Reviews
- •11.5.1. A Three-Minute Test Code Review
- •Size Heuristics
- •But do They Run?
- •Check Code Coverage
- •Conclusions
- •11.5.2. Things to Look For
- •Easy to Understand
- •Documented
- •Are All the Important Scenarios Verified?
- •Run Them
- •Date Testing
- •11.5.3. Conclusions
- •11.6. Refactor Your Tests
- •11.6.1. Use Meaningful Names - Everywhere
- •11.6.2. Make It Understandable at a Glance
- •11.6.3. Make Irrelevant Data Clearly Visible
- •11.6.4. Do not Test Many Things at Once
- •11.6.5. Change Order of Methods
- •11.7. Conclusions
- •11.8. Exercises
- •11.8.1. Clean this Mess
- •Appendix A. Automated Tests
- •A.1. Wasting Your Time by not Writing Tests
- •A.1.1. And what about Human Testers?
- •A.1.2. One More Benefit: A Documentation that is Always Up-To-Date
- •A.2. When and Where Should Tests Run?
- •Appendix B. Running Unit Tests
- •B.1. Running Tests with Eclipse
- •B.1.1. Debugging Tests with Eclipse
- •B.2. Running Tests with IntelliJ IDEA
- •B.2.1. Debugging Tests with IntelliJ IDEA
- •B.3. Running Tests with Gradle
- •B.3.1. Using JUnit Listeners with Gradle
- •B.3.2. Adding JARs to Gradle’s Tests Classpath
- •B.4. Running Tests with Maven
- •B.4.1. Using JUnit Listeners and Reporters with Maven
- •B.4.2. Adding JARs to Maven’s Tests Classpath
- •Appendix C. Test Spy vs. Mock
- •C.1. Different Flow - and Who Asserts?
- •C.2. Stop with the First Error
- •C.3. Stubbing
- •C.4. Forgiveness
- •C.5. Different Threads or Containers
- •C.6. Conclusions
- •Appendix D. Where Should I Go Now?
- •Bibliography
- •Glossary
- •Index
- •Thank You!
Chapter 10. Maintainable Tests
10.3.5. Conclusions
As the code examples within this section have demonstrated, bad code makes it hard to write tests. Allow me to back up this claim with two quotes, illustrating the most important points connected with what we have just been discussing.
Consistency. It is only a virtue, if you are not a screwup.
— Wisdom of the Internet ;)
The misery begins with a single, innocent-seeming line such as "ask object x for the value of y (x.getY()) and make some decisions based on the value of y". If you encounter code which breaches the "Tell,
Don’t Ask!" principle, then do not copy and paste it into your code. What you should do, instead, is clean it, usually by adding methods in places where they ought to be6. Then proceed - writing clean, well-designed code.
Do not copy other sloppy work! Do not become one of the blind led by the blind! An abyss awaits you if you do. (Wow, that has really got you scared, hasn’t it?)
Every time a mock returns a mock, a fairy dies.
— Twitter @damianguy 2009 Oct 19
When writing a test requires you to have a test double which returns another test double, then you know you are about to do something very bad indeed. Such a situation indicates that the code you are working with contravenes "Law of Demeter", which is really most regrettable. Repair the code, and only then get down to testing it. After all…you do not want fairies to die, do you?
10.4. Rewriting Tests when the Code Changes
A change in the requirements occurs. Developers analyze it and implement the required changes. Then tests are run and some of them fail. You can see the disappointment written all over the developers’ faces when they sit down to "fix these *(&(#$ failed tests!".
Have you ever witnessed such a scenario? Have you ever had the feeling that your tests are a major nuisance, and that their existence makes the process of introducing changes a good deal longer and harder than it would be without them? Well, I have certainly seen this many times, and have personally become angry at the fact that after having performed some updates of production code I also had to take care of the tests (instead of moving on to another task).
There are two explanations of why this situation is so common. The first relates to the quality of your tests, the second to the code-first approach.
Let us agree on something, before we begin. If you rewrite part of your implementation, then it is normal that some of your tests will start to fail. In fact, in the majority of cases this is even desirable: if no tests fail, then it means your tests were not good enough!7 The real problems arise if:
6If you need more information about this, please read about the "Feature Envy" code smell. 7This is exactly the behaviour that mutation testing takes advantage of; see Section 11.4.
225
Chapter 10. Maintainable Tests
•the change which made the tests fail is really a refactoring - it does not influence the observable external behaviour of the SUT,
•the failed tests do not seem to have anything to do with the functionality that has changed,
•a single change results in many tests failing.
The last of the above highlights the fact of there being some duplication in respect of tests – with the result that multiple tests are verifying the same functionality. This is rather simple to spot and fix. The other two issues are more interesting, and will be discussed below.
10.4.1. Avoid Overspecified Tests
The most important rule of thumb we follow to keep our tests flexible is: Specify exactly what you want to happen and no more.
— JMock tutorial
What is an overspecified test? There is no consensus about this, and many examples that can be found describe very different features of tests. For the sake of this discussion, let us accept a very simple "definition": a test is overspecified if it verifies some aspects which are irrelevant to the scenario being tested.
Now, which parts of the tests are relevant and which are not? How can we distinguish them just by looking at the test code?
Well, good test method names are certainly very helpful in this respect. For example, if we analyze the test in the listing below, we find that it is a little bit overspecified.
Listing 10.9. Overspecified test - superfluous verification
@Test
public void itemsAvailableIfTheyAreInStore() { when(store.itemsLeft(ITEM_NAME)).thenReturn(2);
assertTrue(shop.isAvailable(ITEM_NAME));
verify(store).itemsLeft(ITEM_NAME);
}
stubbing of a DOC,
asserting on the SUT’s functionality, verifying the DOC’s behaviour.
If this test truly sets out to verify that "items are available if they are in store" (as the test method name claims), then what is the last verification doing? Does it really help to achieve the goal of the test? Not really. If this cooperation with the store collaborator is really a valuable feature of the SUT (is it?), then maybe it would be more appropriate to have a second test to verify it:
226
Chapter 10. Maintainable Tests
Listing 10.10. Two better-focused tests
@Test
public void itemsAvailableIfTheyAreInStore() { when(store.itemsLeft(ITEM_NAME)).thenReturn(2);
assertTrue(shop.isAvailable(ITEM_NAME));
}
@Test
public void shouldCheckStoreForItems() { shop.isAvailable(ITEM_NAME);
verify(store).itemsLeft(ITEM_NAME);
}
Each of the tests in Listing 10.10 has only one reason to fail, while the previous version (in Listing 10.9) has two. The tests are no longer overspecified. If we refactor the SUT’s implementation, it may turn out that only one fails, thus making it clear which functionality was broken.
This example shows the importance of good naming. It is very hard to decide which part of the testShop() method is not relevant to the test’s principal goal.
Another test-double based example is the use of specific parameter values ("my item", 7 or new Date(x,y,z)) when something more generic would suffice (anyString(), anyInt(), anyDate())8. Again, the question we should ask is whether these specific values are really important for the test case in hand. If not, let us use more relaxed values.
Also, you might be tempted to test very defensively, to verify that some interactions have not happened. Sometimes this makes sense. For example in Section 5.4.3 we verified whether no messages had been sent to some collaborators. And such a test was fine - it made sure that the unsubscribe feature worked fine. However, do not put such verifications in when they are not necessary. You could guard each and every one of the SUT’s collaborators with verifications that none of their methods have been called9, but do not do so, unless they are important relative to the given scenario. Likewise, checking whether certain calls to collaborators happened in the order requested (using Mockito’s inOrder() method) will usually just amount to overkill.
We can find numerous examples of overspecified tests outside of the interactions testing domain, as well. A common case is to expect a certain exact form of text, where what is in fact important is only that it should contain several statements. Like with the example discussed above, it is usually possible to divide such tests into two smaller, more focused ones. For example, the first test could check whether a message that has been created contains the user’s name and address, while the second one might perform a full text-matching. This is also an example of when test dependencies make sense: there is no point in bothering with an exact message comparison (which is what the second test verifies), if you know that it does not contain any vital information (verified by the first test).
Based on what we have learned so far, we can say that a good rule of thumb for writing decent, focused tests is as follows: test only the minimally necessary set of features using each test method.
8See Section 6.7 for discussion and more examples.
9Mockito provides some interesting functions for this - verifyZeroInteractions() and verifyNoMoreInteractions().
227
Chapter 10. Maintainable Tests
As is by no means unusual where problems connected with tests are concerned, the real culprit may be the production code. If your test really needs to repeat the petty details of the SUT’s implementation (which will certainly lead to it being overspecified), then maybe the problem lies with how the SUT works with its collaborators. Does the SUT respect the
"Tell-Don’t-Ask!" principle?
10.4.2. Are You Really Coding Test-First?
So the change request came. A developer updated the production code, and then also worked on the failed tests which stopped working because of the implemented change. Wait! What? By implementing changes in production code first, we have just reverted to code-first development, with all its issues! The price that we pay is that now we will have to rewrite some tests looking at the code we wrote a few minutes ago. But this is boring: such tests will probably not find any bugs, and they themselves will most probably be very closely linked to the implementation of the production code (as was already discussed in Chapter 4, Test Driven Development).
Much better results (and less frustration for developers) can be achieved by trying to mimic the TDD approach, following the order of actions given below:
•requirements change,
•developers analyze which tests should be updated to reflect the new requirements,
•tests are updated (and fail because code does not meet the new requirements),
•developers analyze what changes should be introduced into the production code,
•code is updated and tests pass.
This is somewhat different from the TDD approach as we have observed it so far. If we write a new functionality, then we ensure that each individual test that fails is dealt with at once. However, when the requirements of an existing functionality change, we may find ourselves forced to rewrite several tests at once, and then have to deal with all of them failing.
We may sum things up here by saying that in order to avoid having to fix tests after code changes (which is pretty annoying, let’s face it), you should:
•write good tests (i.e. loosely coupled to implementation), minimizing the number of failed tests,
•use test-first in all phases of the development process - both when working on new features and when introducing changes to the existing codebase.
10.4.3. Conclusions
The mime: Developing the code first and then repeating what the code does with expectations mocks. This makes the code drive the tests rather than the other way around. Usually leads to excessive setup and poorly named tests that are hard to see what they do.
— James Carr
As with many other things related to quality, how you start makes a difference. If you start with production code, then your tests will (inevitably, as experience proves) contain too many implementation
228