- •Contents
- •Preface
- •Acknowledgments
- •About the author
- •About the cover illustration
- •Higher product quality
- •Less rework
- •Better work alignment
- •Remember
- •Deriving scope from goals
- •Specifying collaboratively
- •Illustrating using examples
- •Validating frequently
- •Evolving a documentation system
- •A practical example
- •Business goal
- •An example of a good business goal
- •Scope
- •User stories for a basic loyalty system
- •Key Examples
- •Key examples: Free delivery
- •Free delivery
- •Examples
- •Living documentation
- •Remember
- •Tests can be good documentation
- •Remember
- •How to begin changing the process
- •Focus on improving quality
- •Start with functional test automation
- •When: Testers own test automation
- •Use test-driven development as a stepping stone
- •When: Developers have a good understanding of TDD
- •How to begin changing the team culture
- •Avoid “agile” terminology
- •When: Working in an environment that’s resistant to change
- •Ensure you have management support
- •Don’t make test automation the end goal
- •Don’t focus on a tool
- •Keep one person on legacy scripts during migration
- •When: Introducing functional automation to legacy systems
- •Track who is running—and not running—automated checks
- •When: Developers are reluctant to participate
- •Global talent management team at ultimate software
- •Sky Network services
- •Dealing with sign-off and traceability
- •Get sign-off on exported living documentation
- •When: Signing off iteration by iteration
- •When: Signing off longer milestones
- •Get sign-off on “slimmed down use cases”
- •When: Regulatory sign-off requires details
- •Introduce use case realizations
- •When: All details are required for sign-off
- •Warning signs
- •Watch out for tests that change frequently
- •Watch out for boomerangs
- •Watch out for organizational misalignment
- •Watch out for just-in-case code
- •Watch out for shotgun surgery
- •Remember
- •Building the right scope
- •Understand the “why” and “who”
- •Understand where the value is coming from
- •Understand what outputs the business users expect
- •Have developers provide the “I want” part of user stories
- •When: Business users trust the development team
- •Collaborating on scope without high-level control
- •Ask how something would be useful
- •Ask for an alternative solution
- •Make sure teams deliver complete features
- •When: Large multisite projects
- •Further information
- •Remember
- •Why do we need to collaborate on specifications?
- •The most popular collaborative models
- •Try big, all-team workshops
- •Try smaller workshops (“Three Amigos”)
- •Pair-writing
- •When: Mature products
- •Have developers frequently review tests before an iteration
- •When: Analysts writing tests
- •Try informal conversations
- •When: Business stakeholders are readily available
- •Preparing for collaboration
- •Hold introductory meetings
- •When: Project has many stakeholders
- •Involve stakeholders
- •Undertake detailed preparation and review up front
- •When: Remote Stakeholders
- •Prepare only initial examples
- •Don’t hinder discussion by overpreparing
- •Choosing a collaboration model
- •Remember
- •Illustrating using examples: an example
- •Examples should be precise
- •Don’t have yes/no answers in your examples
- •Avoid using abstract classes of equivalence
- •Ask for an alternative way to check the functionality
- •When: Complex/legacy infrastructures
- •Examples should be realistic
- •Avoid making up your own data
- •When: Data-driven projects
- •Get basic examples directly from customers
- •When: Working with enterprise customers
- •Examples should be easy to understand
- •Avoid the temptation to explore every combinatorial possibility
- •Look for implied concepts
- •Illustrating nonfunctional requirements
- •Get precise performance requirements
- •When: Performance is a key feature
- •Try the QUPER model
- •When: Sliding scale requirements
- •Use a checklist for discussions
- •When: Cross-cutting concerns
- •Build a reference example
- •When: Requirements are impossible to quantify
- •Remember
- •Free delivery
- •Examples should be precise and testable
- •When: Working on a legacy system
- •Don’t get trapped in user interface details
- •When: Web projects
- •Use a descriptive title and explain the goal using a short paragraph
- •Show and keep quiet
- •Don’t overspecify examples
- •Start with basic examples; then expand through exploring
- •When: Describing rules with many parameter combinations
- •In order to: Make the test easier to understand
- •When: Dealing with complex dependencies/referential integrity
- •Apply defaults in the automation layer
- •Don’t always rely on defaults
- •When: Working with objects with many attributes
- •Remember
- •Is automation required at all?
- •Starting with automation
- •When: Working on a legacy system
- •Plan for automation upfront
- •Don’t postpone or delegate automation
- •Avoid automating existing manual test scripts
- •Gain trust with user interface tests
- •Don’t treat automation code as second-grade code
- •Describe validation processes in the automation layer
- •Don’t replicate business logic in the test automation layer
- •Automate along system boundaries
- •When: Complex integrations
- •Don’t check business logic through the user interface
- •Automate below the skin of the application
- •Automating user interfaces
- •Specify user interface functionality at a higher level of abstraction
- •When: User interface contains complex logic
- •Avoid recorded UI tests
- •Set up context in a database
- •Test data management
- •Avoid using prepopulated data
- •When: Specifying logic that’s not data driven
- •Try using prepopulated reference data
- •When: Data-driven systems
- •Pull prototypes from the database
- •When: Legacy data-driven systems
- •Remember
- •Reducing unreliability
- •When: Working on a system with bad automated test support
- •Identify unstable tests using CI test history
- •Set up a dedicated continuous validation environment
- •Employ fully automated deployment
- •Create simpler test doubles for external systems
- •When: Working with external reference data sources
- •Selectively isolate external systems
- •When: External systems participate in work
- •Try multistage validation
- •When: Large/multisite groups
- •Execute tests in transactions
- •Run quick checks for reference data
- •When: Data-driven systems
- •Wait for events, not for elapsed time
- •Make asynchronous processing optional
- •Getting feedback faster
- •Introduce business time
- •When: Working with temporal constraints
- •Break long test packs into smaller modules
- •Avoid using in-memory databases for testing
- •When: Data-driven systems
- •Separate quick and slow tests
- •When: A small number of tests take most of the time to execute
- •Keep overnight packs stable
- •When: Slow tests run only overnight
- •Create a current iteration pack
- •Parallelize test runs
- •When: You can get more than one test Environment
- •Try disabling less risky tests
- •When: Test feedback is very slow
- •Managing failing tests
- •Create a known regression failures pack
- •Automatically check which tests are turned off
- •When: Failing tests are disabled, not moved to a separate pack
- •Remember
- •Living documentation should be easy to understand
- •Look for higher-level concepts
- •Avoid using technical automation concepts in tests
- •When: Stakeholders aren’t technical
- •Living documentation should be consistent
- •When: Web projects
- •Document your building blocks
- •Living documentation should be organized for easy access
- •Organize current work by stories
- •Reorganize stories by functional areas
- •Organize along UI navigation routes
- •When: Documenting user interfaces
- •Organize along business processes
- •When: End-to-end use case traceability required
- •Listen to your living documentation
- •Remember
- •Starting to change the process
- •Optimizing the process
- •The current process
- •The result
- •Key lessons
- •Changing the process
- •The current process
- •Key lessons
- •Changing the process
- •Optimizing the process
- •Living documentation as competitive advantage
- •Key lessons
- •Changing the process
- •Improving collaboration
- •The result
- •Key lessons
- •Changing the process
- •Living documentation
- •Current process
- •Key lessons
- •Changing the process
- •Current process
- •Key lessons
- •Collaboration requires preparation
- •There are many different ways to collaborate
- •Looking at the end goal as business process documentation is a useful model
- •Long-term value comes from living documentation
- •Index
Chapter 4 Initiating the changes |
59 |
Introduce use case realizations
When: All details are required for sign-off
Matthew Steer worked on several projects based on a structured process (Rational Uniied Process). The process required full sign-off on all the details, and speciications were captured as use cases. In addition to use cases, Steer and his team introduced use case realizations that effectively illustrated use cases with examples. This allowed them to use Speciication by Example in a structured process. Steer says:
The requirements were captured as use cases and supplementary speciications for nonfunctional requirements—quite traditional, by-the- book capture. With use cases we produced use case realizations, examples and scenarios that use cases would fulill. We created tables with lots of parameters and hooked up data from there and then had lows to show how the use case would be realized. Use case realization was a working version of what that meant to the business, using real scenarios.
Adding details such |
as |
use |
case |
realizations |
is |
a |
good |
idea |
to |
get |
Speciication |
|||||
by |
Example |
into a |
formal |
process—under the |
radar |
of the methodology police. |
||||||||||
It |
can |
also |
help |
to |
implement |
the |
ideas |
of |
Speciication |
by |
Example |
when |
||||
commercial contracts |
require |
sign-of on |
requirements |
but |
still |
allow |
for |
later |
||||||||
variability |
in |
detail. |
|
|
|
|
|
|
|
|
|
|
|
|
|
Steer’s team, as well as others mentioned in the previous section, used examples—even if disguised as use-case realizations—instead of only using use cases or tests made against more generic requirements. This made their delivery process more effective.
Technically, a living documentation system instantly provides traceability in requirements changes, because teams use version control systems to store their executable speciications. Iterative development is generally at odds with up-front sign-offs, but you can use the tips from this section to deal with that while the process changes and the delivery team gains trust from the business users. The visibility that a living documentation system provides, along with collaboration on speciications, should help to eliminate the need for sign-offs.
Warning signs
You can track your progress to check if you’re implementing Speciication by Example properly. As with any metrics, make sure that metrics themselves don’t become a goal; otherwise, you’ll locally optimize a process to get the numbers right but hurt the longterm results. Use these as measurements to check if the process needs adjustment.
60 Speciication by Example
Watch out for tests that change frequently
At XPDay 2009, Mark Striebeck talked about what Google is doing to advance its testing practices.8 One of the ideas that impressed me was how they measure whether a (unit) test is good or not. When a test breaks, they track changes in the source code until the test starts passing again. If the underlying code was changed, they consider the test to be a good one. If the test changed and the code did not change, they consider it to be bad. By collecting these statistics, they hope to analyze unit test patterns and identify what makes a test good or bad.
I believe that the same criteria can be applied to executable speciications. If a validation fails and you change the code, that means you found and ixed a problem. If a validation fails and you have to change the speciication, that means it wasn’t written properly.
Business rules should be much more stable than the technology that implements them. Watch out for executable speciications that change frequently. Look for ways to write them better.
You can also measure the time your team spends on refactoring speciications and related automation code in order to keep it under control. If you spend a signiicant portion of your iteration doing this, look for better ways to automate tests (see chapter 9 for some good tips).
Watch out for boomerangs
Another good metric to check if you’re doing something wrong is checking for the presence of boomerangs. A boomerang is a story or a product backlog item that comes back into the process less than a month after it was released. The team thought it was done, but it needs rework. But it’s not a boomerang when the business later extends existing requirements to incorporate innovation as the product evolves.
Once you implement Speciication by Example, the number of boomerangs should be reduced signiicantly until they’re a rare occurrence. Collaboration on speciications and better alignment of testing and development should eliminate wasteful rework caused by misunderstanding. Reviewing your boomerang trends over several months will show you how much you’ve improved. If the rate doesn’t drop, it means that there’s something wrong with the way you implemented the process.
Tracking boomerangs doesn’t take a lot of time, usually a few minutes every iteration, but it can help a lot when the time comes to challenge or prove that Speciication by Example is working. In larger companies, it can also provide compelling evidence that it’s worth doing with other teams. For more complex statistics, you can also track
8 http://gojko.net/2009/12/07/improving-testing-practices-at-google
Chapter 4 Initiating the changes |
61 |
the time spent on boomerangs, because this igure directly translates into wasted development/testing time and money. If people complain about the time spent on automating executable speciications as unnecessary overhead, compare that to the time they spent working on boomerangs several months earlier. This should be more than enough to build a business case for Speciication by Example.
Once the number of boomerangs goes down and they occur relatively rarely, you can stop tracking them. If a boomerang occurs, try to understand where it’s coming from. One of my clients had many boomerangs coming from their inancial department. This pointed to a communication problem with that particular part of the company; as a result, they looked for better ways to engage the department.
Tracking boomerangs is also a good way to build a business case for introducing Speciication by Example. It can help a team pinpoint the waste caused by vague requirements and functional gaps in speciications.
Watch out for organizational misalignment
Many teams started implementing Speciication by Example as a way to better align their activities with iterations. After you become familiar with executable speciications and the automation code becomes stable, you should be able to implement a story and completely inish testing it (including manual exploratory testing) inside the same iteration. If your testers are lagging behind development, you’re doing something wrong. A similar warning sign is misaligned analysis. Some teams start analysis ahead of the relevant iteration, but they still have regular intervals and low. Analyzing too much up front, analyzing things that won’t be implemented immediately, or being late with analysis when details are needed are signs that the process is wrong.
Watch out for just-in-case code
In Lean Software Development9 Mary and Tom Poppendieck wrote that the biggest source of waste in software development is just-in-case code—software that was written without being needed. I’m not sure whether it’s the biggest source of waste, but I’ve certainly seen lots of money, time, and effort wasted on things that nobody needed or asked for. Speciication by Example signiicantly reduces this problem because it helps us build a shared understanding of what we need to deliver. Jodie Parker says that the conversations and collaboration on speciications helped her team achieve just that:
9Mary Poppendieck and Tom Poppendieck,Lean Software Development: An Agile Toolkit
(Addison-Wesley Professional, 2003).