- •Contents
- •Preface
- •Acknowledgments
- •About the author
- •About the cover illustration
- •Higher product quality
- •Less rework
- •Better work alignment
- •Remember
- •Deriving scope from goals
- •Specifying collaboratively
- •Illustrating using examples
- •Validating frequently
- •Evolving a documentation system
- •A practical example
- •Business goal
- •An example of a good business goal
- •Scope
- •User stories for a basic loyalty system
- •Key Examples
- •Key examples: Free delivery
- •Free delivery
- •Examples
- •Living documentation
- •Remember
- •Tests can be good documentation
- •Remember
- •How to begin changing the process
- •Focus on improving quality
- •Start with functional test automation
- •When: Testers own test automation
- •Use test-driven development as a stepping stone
- •When: Developers have a good understanding of TDD
- •How to begin changing the team culture
- •Avoid “agile” terminology
- •When: Working in an environment that’s resistant to change
- •Ensure you have management support
- •Don’t make test automation the end goal
- •Don’t focus on a tool
- •Keep one person on legacy scripts during migration
- •When: Introducing functional automation to legacy systems
- •Track who is running—and not running—automated checks
- •When: Developers are reluctant to participate
- •Global talent management team at ultimate software
- •Sky Network services
- •Dealing with sign-off and traceability
- •Get sign-off on exported living documentation
- •When: Signing off iteration by iteration
- •When: Signing off longer milestones
- •Get sign-off on “slimmed down use cases”
- •When: Regulatory sign-off requires details
- •Introduce use case realizations
- •When: All details are required for sign-off
- •Warning signs
- •Watch out for tests that change frequently
- •Watch out for boomerangs
- •Watch out for organizational misalignment
- •Watch out for just-in-case code
- •Watch out for shotgun surgery
- •Remember
- •Building the right scope
- •Understand the “why” and “who”
- •Understand where the value is coming from
- •Understand what outputs the business users expect
- •Have developers provide the “I want” part of user stories
- •When: Business users trust the development team
- •Collaborating on scope without high-level control
- •Ask how something would be useful
- •Ask for an alternative solution
- •Make sure teams deliver complete features
- •When: Large multisite projects
- •Further information
- •Remember
- •Why do we need to collaborate on specifications?
- •The most popular collaborative models
- •Try big, all-team workshops
- •Try smaller workshops (“Three Amigos”)
- •Pair-writing
- •When: Mature products
- •Have developers frequently review tests before an iteration
- •When: Analysts writing tests
- •Try informal conversations
- •When: Business stakeholders are readily available
- •Preparing for collaboration
- •Hold introductory meetings
- •When: Project has many stakeholders
- •Involve stakeholders
- •Undertake detailed preparation and review up front
- •When: Remote Stakeholders
- •Prepare only initial examples
- •Don’t hinder discussion by overpreparing
- •Choosing a collaboration model
- •Remember
- •Illustrating using examples: an example
- •Examples should be precise
- •Don’t have yes/no answers in your examples
- •Avoid using abstract classes of equivalence
- •Ask for an alternative way to check the functionality
- •When: Complex/legacy infrastructures
- •Examples should be realistic
- •Avoid making up your own data
- •When: Data-driven projects
- •Get basic examples directly from customers
- •When: Working with enterprise customers
- •Examples should be easy to understand
- •Avoid the temptation to explore every combinatorial possibility
- •Look for implied concepts
- •Illustrating nonfunctional requirements
- •Get precise performance requirements
- •When: Performance is a key feature
- •Try the QUPER model
- •When: Sliding scale requirements
- •Use a checklist for discussions
- •When: Cross-cutting concerns
- •Build a reference example
- •When: Requirements are impossible to quantify
- •Remember
- •Free delivery
- •Examples should be precise and testable
- •When: Working on a legacy system
- •Don’t get trapped in user interface details
- •When: Web projects
- •Use a descriptive title and explain the goal using a short paragraph
- •Show and keep quiet
- •Don’t overspecify examples
- •Start with basic examples; then expand through exploring
- •When: Describing rules with many parameter combinations
- •In order to: Make the test easier to understand
- •When: Dealing with complex dependencies/referential integrity
- •Apply defaults in the automation layer
- •Don’t always rely on defaults
- •When: Working with objects with many attributes
- •Remember
- •Is automation required at all?
- •Starting with automation
- •When: Working on a legacy system
- •Plan for automation upfront
- •Don’t postpone or delegate automation
- •Avoid automating existing manual test scripts
- •Gain trust with user interface tests
- •Don’t treat automation code as second-grade code
- •Describe validation processes in the automation layer
- •Don’t replicate business logic in the test automation layer
- •Automate along system boundaries
- •When: Complex integrations
- •Don’t check business logic through the user interface
- •Automate below the skin of the application
- •Automating user interfaces
- •Specify user interface functionality at a higher level of abstraction
- •When: User interface contains complex logic
- •Avoid recorded UI tests
- •Set up context in a database
- •Test data management
- •Avoid using prepopulated data
- •When: Specifying logic that’s not data driven
- •Try using prepopulated reference data
- •When: Data-driven systems
- •Pull prototypes from the database
- •When: Legacy data-driven systems
- •Remember
- •Reducing unreliability
- •When: Working on a system with bad automated test support
- •Identify unstable tests using CI test history
- •Set up a dedicated continuous validation environment
- •Employ fully automated deployment
- •Create simpler test doubles for external systems
- •When: Working with external reference data sources
- •Selectively isolate external systems
- •When: External systems participate in work
- •Try multistage validation
- •When: Large/multisite groups
- •Execute tests in transactions
- •Run quick checks for reference data
- •When: Data-driven systems
- •Wait for events, not for elapsed time
- •Make asynchronous processing optional
- •Getting feedback faster
- •Introduce business time
- •When: Working with temporal constraints
- •Break long test packs into smaller modules
- •Avoid using in-memory databases for testing
- •When: Data-driven systems
- •Separate quick and slow tests
- •When: A small number of tests take most of the time to execute
- •Keep overnight packs stable
- •When: Slow tests run only overnight
- •Create a current iteration pack
- •Parallelize test runs
- •When: You can get more than one test Environment
- •Try disabling less risky tests
- •When: Test feedback is very slow
- •Managing failing tests
- •Create a known regression failures pack
- •Automatically check which tests are turned off
- •When: Failing tests are disabled, not moved to a separate pack
- •Remember
- •Living documentation should be easy to understand
- •Look for higher-level concepts
- •Avoid using technical automation concepts in tests
- •When: Stakeholders aren’t technical
- •Living documentation should be consistent
- •When: Web projects
- •Document your building blocks
- •Living documentation should be organized for easy access
- •Organize current work by stories
- •Reorganize stories by functional areas
- •Organize along UI navigation routes
- •When: Documenting user interfaces
- •Organize along business processes
- •When: End-to-end use case traceability required
- •Listen to your living documentation
- •Remember
- •Starting to change the process
- •Optimizing the process
- •The current process
- •The result
- •Key lessons
- •Changing the process
- •The current process
- •Key lessons
- •Changing the process
- •Optimizing the process
- •Living documentation as competitive advantage
- •Key lessons
- •Changing the process
- •Improving collaboration
- •The result
- •Key lessons
- •Changing the process
- •Living documentation
- •Current process
- •Key lessons
- •Changing the process
- •Current process
- •Key lessons
- •Collaboration requires preparation
- •There are many different ways to collaborate
- •Looking at the end goal as business process documentation is a useful model
- •Long-term value comes from living documentation
- •Index
222 Speciication by Example
Introducing the business framework and making sure that the executable speciications actually get validated frequently and stay relevant enabled them to create a useful living documentation system. They had a relevant source of information on what the system does, which anyone could access.
Living documentation as competitive advantage
With such a good living documentation system, they were able to handle very big changes eficiently. Three months before the end of the project, the business model of the company suddenly had to change. They normally fund loans through a bond sale. Because of the credit crisis in 2008, the bond sale failed. The business is technology driven, so this business model change had to be relected in their software. Andersen says that the living documentation system helped them understand what was required to support this business change:
Typically, we use bond proceeds to fund private student loans. However, we changed our business model and made all of the funding portion of the system conigurable so that we could use lenders to provide funds and continue to provide loans to the students. It was a dramatic overhaul of a core piece of the system. Before this new funding requirement, our system didn’t even have the concept of a lender because we were able to assume Iowa Student Loan was the lender.
We were able to use our existing acceptance tests and repurpose them to say, “OK, here’s our funding requirement.” For all of the tests we had, we discussed the impact and provided funding so they would still work. We had some interesting discussions based on scenarios where there is no more funding available, or funding is available but not for this school or this lender, so we had some edge cases for these requirements, but it was really making the new funding model more lexible and conigurable.
Once they understood the impacts of this new business model on the software, they were able to implement the solution eficiently. According to Andersen, such change would be impossible to implement quickly without a living documentation:
Because we had good acceptance tests, we were able to implement a solution within a month. Any other system that didn’t have the tests would halt the development and it would have been a rewrite.
Chapter 14 Iowa Student Loan |
223 |
This is when the investment in the living documentation system paid off. It supported them in analysis, implementation, and testing of the impact of a business model change, at the same time enabling them to quickly verify that the rest of the system is unaffected.
Key lessons
They started out by focusing on a tool and quickly realized that it doesn’t help them achieve the goal of bringing business users into the process. So they started approaching the speciications from the perspective of a user. This enabled them to communicate with their business users better and reduce the costs of maintenance for tests. When the tool prevented them from collaborating effectively, they modiied it. This is another argument for using open source tools.
Implementing Speciication by Example at Iowa Student Loan was driven not by the need to improve quality or automate tests but by the need to build a relevant documentation system in order to be more effective and engage business users better. They invested heavily into building a good living documentation system, which paid off well. It helped them implement a business model change, which was very powerful.
15
Sabre Airline Solutions
Sabre Airline Solutions offers software and services to help airlines plan, operate, and sell their products. They are an early adopter of Extreme Programming and Speciication by Example and an interesting case study because they applied
SBE on a massive project, with a relatively large distributed team.
The project was Sabre AirCentre Movement Manager, a software system that monitors airline operations and alerts the relevant teams when it inds issues, allowing them to adjust schedules to minimize the impact to customers and the airline. According to Wes Williams, an agile coach at Sabre, two previous projects to build similar systems failed because of the domain complexity and quality issues. Speciication by Example enabled them to complete this project successfully.
Changing the process
Because of the complexity of the domain, the teams at Sabre were looking for a collaborative way to specify and automate acceptance testing soon after implementing Extreme Programming. Williams said that they initially tried to do it with a technical unit-testing tool. That approach didn’t help with collaboration and it wasn’t reusable, so they abandoned it.
They started looking for a tool to drive collaboration. In 2003, Williams found FIT, the irst widely available tool for automating executable speciications. His team started implementing acceptance testing with FIT, but they focused on the tool, not the practices. Williams says that it didn’t give them the improvement in collaboration they expected:
224
Chapter 15 Sabre Airline Solutions |
225 |
We liked the idea that a customer could deine the test and drive the value you deliver in an application. In reality, we never got a client to write FIT tests in HTML. The tests got written most of the time by a developer. We had a hard time getting the customers to do it. Testers were using QTP. That never drove collaboration, and developers never ran QTP tests or got involved in writing them.
The developers were the only ones who wrote executable speciications, and they understood that didn’t give them the beneits they expected. To improve the communication and collaboration, everyone had to be involved. A single group of people was unable to do that on their own.
A senior vice president of product development, inluenced by the development team, brought in consultants from ObjectMentor to train everyone. They made the wider group aware of the goals of Speciication by Example and the beneits they could get out of it. Although they didn’t get everyone on board immediately, the training helped them get more people enthusiastic about the practices. Williams says:
Not everyone adopted it. Still, there was a core group of people who believed in it, and they learned a lot. Those who didn’t continued to ight against it.
That core group of people started with a relatively simple web project—an internal system for aggregating software build information. They wanted to try out the practices and get their heads around the tools, which were a lot worse in 2004 than they are now. The team selected FitNesse to manage executable speciications collaboratively. They wrote executable speciications either just before the development or roughly at the same time. The business stakeholder for the project was an internal manager, who got involved in reviewing the tests. The team initially looked at the automation layer as second-grade test code and cared little about making it clean, which caused numerous maintenance problems. They also ended up with a lot of duplication in test speciications. Williams says:
We learned that we should try to keep ixtures as simple as possible and that duplication is bad. It [automation layer] is code, like any other code.
Developers didn’t care much about making the automation layer or executable speciications maintainable because they just associated them with testing. By the end of the project, they realized that this approach led to huge maintenance problems. Similar to