How mutation testing showed me my test suite was lying

A project with no tests, picked up again after years. The obvious choice: add a suite and use every tool at hand, AI agents included, to get there faster. A few weeks later: hundreds of green tests, high coverage, CI in place. The feeling of having paid off a debt. Then a bug in production. Covered by the tests, passed by every check, sitting there quietly for three weeks. The code had little to do with it. The tests were formally correct, well written, flawless in style: they simply could not find that bug. That is when I ran mutation testing for the first time. It told me what coverage could not. Coverage asks whether this code was executed; mutation testing asks whether the tests would notice if this code were wrong. This talk tells that story. We look at where the gap between coverage and mutation score comes from, how to read a mutation testing report to get better tests out of it without multiplying them, and how to bring it into CI without it turning into noise.

Gallery