Software Testing

Below are collections of article against heavy unit testing.

A1. Write tests. Not too many. Mostly integration

• Write tests
  • Static typing and linting tools
  • Write tests
• Not too many
  • No need 100% coverage because of maintainence cost when refactoring
  • Still need 100% coverage for libraries
• Mostly integration

vs

A2. Lean Testing

• End-to-end tests provide the greatest confidence, with highest time cost (implementation & execution)
• Integration tests provide the best balance of cost, speed and confidence
• Unit tests provides less cost to write and faster to run, but they test only a small part that might not even be critical

Summary:

• Use a typed language.
• Focus on integration and end-to-end tests.
• Use unit tests only where they make sense (e.g. pure algorithmic code with complex corner cases)

A3. Giving up on test-first development

Unit test:

• You become conservative to avoid breaking lots of tests
  • More reluctant to make large-scale changes
• Makes you focus on detail rather than structure

A4. Unit Test Fetish

• Return on investment of unit tests is an order(s) of magnitude lower than that of end-to-end tests.
• End-to-end tests test the critical path. Unit test do not.
• Unit tests ossify the internal architecture.
• There are things that can’t be unit-tested.
• Some stuff has no rigorous acceptance criteria.
• Unit tests are easy to measure and you should fear that.

Below are collections of article support unit testing.

B1. The Failures of “Intro to TDD”

• Failure #1: Encouraging Large Units
  • Solution: refactoring after every test passed (Red-Green-Refactor)
• Failure #2: Encouraging Costly Extract Refactors
• Failure #3: Characterization Tests of Greenfield Code
• Failure #4: Redundant test coverage
• Failure #5: Eliminating Redundancy Sacrifices Regression Value
• Failure #6: Making a Mess with Mocks

A Successful Approach to TDD

See the detail in post.

Basically:

• Top-down approach
• Fake it until you make it

My take away

• Unit test has quick feedback cycle
• Refactoring internal code of unit (keep same public interface):
  • Unit test is great
  • Quickly test if refactored unit is working
• Refactoring public interface:
  • Unit test is bad: requires more maintainance cost
  • Should prevent changes in public interface. How? -> Top-down approach
• E2E test:
  • Quickly spot behavior bug
  • Test critical paths
  • Bad at: hard to cover all cases/edge

Conclusion: Unit test and e2e test are complementary

• Unit Test:
  • Quick feedback
  • Useful for clear-defined requirement/responsibility of unit
  • Cover edge cases
  • Support internal refactoring/optimization
• E2E:
  • Useful for overall behavior, user scenario
  • Cover large areas, critical paths