The document discusses designing software for testability in production. It recommends removing staging environments and running automated tests and a subset of code in production to gain confidence in software quality. Specific techniques include using sandbox accounts to test functionality without affecting real users, running integration tests that create and delete sandbox environments, and monitoring databases and logs in real-time to detect issues. Designing systems with isolation, well-defined interfaces, and the ability to restrict access and trace effects can help make testing in production safer and more effective. Tools like API documentation generators and mocks can also help improve quality.
9. Software Quality is Hard
Think end to end for entire lifecycle of code
Verification in Production is a powerful technique
to help us build quality software
17. Example: Know That Checkout Works
Bot places an order every few minutes
Identify test orders and immediately cancel
18. Example: Support “Sandbox” Accounts
”SaaS” – even for internal accounts
Mark individual accounts as sandbox
One API Key for all sandbox accounts
"every service is a third party"
19. Example: End to End Integration Tests
Create Sandbox Org
Run tests
Delete Sandbox Org
“Safe and Repeatable”
32. Key Takeaways – Design Production to be Testable
Trust your tests, run subset in production
Invest in continuous delivery
Sandbox accounts are powerful
High quality, trustworthy mocks
Real-time feedback from production