Verbose Logging

software development with some really amazing hair

T + G I F R

manbearpig: Mutation Testing for Go

· · Posted in Programming
Tagged with

Mutation testing isn't about testing your code, but about improving your tests.

What happens is the testing tool looks through your source code for instances of some known thing. This usually something easily tweaked, like ==. For each instance, it changes it to some opposite value that makes sense. In the case of ==, we can change it to !=. For each single change it makes, it runs the tests. The idea is that the tests should fail. If they don't, you might want to consider writing more tests to cover the cases the mutation exposed. After a test run, the mutation is reset, so as to not taint subsequent mutations.

For my book Go, The Standard Library I wrote an example to show off how you can alter a Go AST. I've reworked the example and packaged up an an application called manbearpig.

manbearpig

manbearpig as seen in South Park

Get start by installing manbearpig

go get github.com/darkhelmet/manbearpig

Now you can run it on a standard library to see it in action.

manbearpig -import crypto/sha256 -mutation ==

You can see that it switches == to !=, and that the tests break, as they should.

Run manbearpig -list to see the list of available mutations.

Fin

Run this on your own package to see where your tests are lacking.