By replacing integration tests with unit tests, we're losing...

By replacing integration tests with unit tests, we’re losing alignment guarantees. Consider the failure modes of integration tests: slow tests, lots of setup, flaky results, poor error reporting. These generally make the tests more frustrating to use, but they still serve the goal of higher quality, more correct software. The failure modes of unit testing, on the other hand, are things like overreliance on mocks and designing your code to be more unit testable. These are failures modes that make unit testing a poorer indicator of code quality and correctness.5 They are also failure modes that make unit tests easier to write, speeding up the feedback loop. That’s a good example of Goodhart’s Law, where the measure takes priority over the actual goal.

Comments
www.joshbeckman.org/notes/563106559