Shared Responsibility
Managing Technical Debt
Testing in Context
Summary
Chapter 7. Technology-Facing Tests that Support the Team
An Agile Testing Foundation
The Purpose of Quadrant 1 Tests
Supporting Infrastructure
Why Write and Execute These Tests?
Lets Us Go Faster and Do More
Making Testers’ Jobs Easier
Designing with Testing in Mind
Timely Feedback
Where Do Technology-Facing Tests Stop?
What If the Team Doesn’t Do These Tests?
What Can Testers Do?
What Can Managers Do?
It’s a Team Problem
Toolkit
Source Code Control
IDEs
Build Tools
Build Automation Tools
Unit Test Tools
Summary
Chapter 8. Business-Facing Tests that Support the Team
Driving Development with Business-Facing Tests
The Requirements Quandary
Common Language
Eliciting Requirements
Advance Clarity
Conditions of Satisfaction
Ripple Effects
Thin Slices, Small Chunks
How Do We Know We’re Done?
Tests Mitigate Risk
Testability and Automation
Summary
Chapter 9. Toolkit for Business-Facing Tests that Support the Team
Business-Facing Test Tool Strategy
Tools to Elicit Examples and Requirements
Checklists
Mind Maps
Spreadsheets
Mock-Ups
Flow Diagrams
Software-Based Tools
Tools for Automating Tests Based on Examples
Tools to Test below the GUI and API Level
Tools for Testing through the GUI
Strategies for Writing Tests
Build Tests Incrementally
Keep the Tests Passing
Use Appropriate Test Design Patterns
Keyword and Data-Driven Tests
Testability
Code Design and Test Design
Automated vs. Manual Quadrant 2 Tests
Test Management
Summary
Chapter 10. Business-Facing Tests that Critique the Product
Introduction to Quadrant 3
Demonstrations
Scenario Testing
Exploratory Testing
Session-Based Testing
Automation and Exploratory Testing
An Exploratory Tester
Usability Testing
User Needs and Persona Testing
Navigation
Check Out the Competition
Behind the GUI
API Testing
Web Services
Testing Documents and Documentation
User Documentation
Reports
Tools to Assist with Exploratory Testing
Test Setup
Test Data Generation
Monitoring Tools
Simulators
Emulators
Summary
Chapter 11. Critiquing the Product Using Technology-Facing Tests
Introduction to Quadrant 4
Who Does It?
When Do You Do It?
“ility” Testing
Security
Maintainability
Interoperability
Compatibility
Reliability
Installability
“ility” Summary
Performance, Load, Stress, and Scalability Testing
Scalability
Performance and Load Testing
Performance and Load-Testing Tools
Baseline
Test Environments
Memory Management
Summary
Chapter 12. Summary of Testing Quadrants
Review of the Testing Quadrants
A System Test Example
The Application
The Team and the Process
Tests Driving Development
Unit Tests
Acceptance Tests
Automation
The Automated Functional Test Structure
Web Services
Embedded Testing
Critiquing the Product with Business-Facing Tests
Exploratory Testing
Testing Data Feeds
The End-to-End Tests
User Acceptance Testing
Reliability
Documentation
Documenting the Test Code
Reporting the Test Results
Using the Agile Testing Quadrants
Summary
Part IV. Automation
Chapter 13. Why We Want to Automate Tests and What Holds Us Back
Why Automate?
Manual Testing Takes Too Long
Manual Processes Are Error Prone
Automation Frees People to Do Their Best Work
Automated Regression Tests Provide a Safety Net
Automated Tests Give Feedback, Early and Often
Tests and Examples that Drive Coding Can Do More
Tests Are Great Documentation
ROI and Payback
Barriers to Automation—Things that Get in the Way
Bret’s List
Our List
Programmers’ Attitude—“Why Automate?”
The “Hump of Pain” (The Learning Curve)
Initial Investment
Code that’s Always in Flux
Legacy Code
Fear
Old Habits
Can We Overcome These Barriers?
Summary
Chapter 14. An Agile Test Automation Strategy
An Agile Approach to Test Automation
Automation Test Categories
Test Automation Pyramid
What Can We Automate?
Continuous Integration, Builds, and Deploys
Unit and Component Tests
API or Web Services Testing
Testing behind the GUI
Testing the GUI
Load Tests
Comparisons
Repetitive Tasks
Data Creation or Setup
What Shouldn’t We Automate?
Usability Testing
Exploratory Testing
Tests that Will Never Fail
One-Off Tests
What Might Be Hard to Automate?
Developing an Automation Strategy—Where Do We Start?
Where Does It Hurt the Most?
Multi-Layered Approach
Think about Test Design and Maintenance
Choosing the Right Tools
Applying Agile Principles to Test Automation
Keep It Simple
Iterative Feedback
Whole-Team Approach
Taking the Time to Do It Right
Learn by Doing
Apply Agile Coding Practices to Tests
Supplying Data for Tests
Data Generation Tools
Avoid Database Access
When Database Access Is Unavoidable or Even Desirable
Understand Your Needs
Evaluating Automation Tools
Identifying Requirements for Your Automation Tool
One Tool at a Time
Choosing Tools
Agile-Friendly Tools
Implementing Automation
Managing Automated Tests
Organizing Tests
Organizing Test Results
Go Get Started
Summary
Part V. An Iteration in the Life of a Tester
Chapter 15. Tester Activities in Release or Theme Planning
The Purpose of Release Planning
Sizing
How to Size Stories
The Tester’s Role in Sizing Stories
An Example of Sizing Stories
Prioritizing
Why We Prioritize Stories
Testing Considerations While Prioritizing
What’s in Scope?
Deadlines and Timelines
Focus on Value
System-Wide Impact
Third-Party Involvement
Test Planning
Where to Start
Why Write a Test Plan?
Types of Testing
Infrastructure
Test Environments
Test Data
Test Results
Test Plan Alternatives
Lightweight Test Plans
Using a Test Matrix
Test Spreadsheet
A Whiteboard
Automated Test List
Preparing for Visibility
Tracking Test Tasks and Status
Communicating Test Results
Release Metrics
Summary
Chapter 16. Hit the Ground Running
Be Proactive
Benefits
Do You Really Need This?
Potential Downsides to Advance Preparation
Advance Clarity
Customers Speak with One Voice
Story Size
Geographically Dispersed Teams
Examples
Test Strategies
Prioritize Defects
Resources
Summary
Chapter 17. Iteration Kickoff
Iteration Planning
Learning the Details