There are many things that make Go a great programming language. One of those things is the first-class support for testing. After installing Go, you already have everything you need to get started with testing your code:
- The built-in testing package provides a simple but powerful API to write tests.
go testcommand runs the tests and prints a summary of the test results.
That, however, only scratches the surface of what Go can do. Since Go version 1.2, the toolchain has been able to compute and display test coverage statistics. Test coverage helps you find untested parts of your codebase and, when used properly, can be an important indicator of software quality. I was pleased to see this feature being added to Go’s already excellent tooling.
Everybody interested in test coverage and Go should first read Rob Pike’s superb post, The cover story, which is a great introduction to the topic. Be sure to come back, though, as I will show you how to get more out of Go’s test coverage support.
Coverage of multiple packages
To get test coverage statistics, you first have to tell
go test to generate a coverage profile (a plaintext file with collected coverage results). Such a profile can then be fed into other programs like
go tool cover for further analysis/processing. Here is a quick example demonstrating how to create a beautiful HTML coverage report for a single Go package:
$ cd chef-runner/ $ go test -coverprofile=cover.out ./util ok github.com/mlafeldt/chef-runner/util 0.017s coverage: 78.1% of statements $ go tool cover -html=cover.out -o coverage.html
Unfortunately, there is a catch. As of Go 1.3.3,
go test can’t compute coverage statistics for multiple packages at once. Trying to do so results in an error:
$ go test -coverprofile=cover.out ./... cannot use test profile flag with multiple packages
This is a known issue which will probably be resolved in the foreseeable future. For chef-runner, one of my pet projects, I still wanted a way to get coverage statistics for all of its packages. I wanted to see at a glance whether I missed testing some key functionality. And I wanted it now.
Fortunately, there is a simple workaround which involves the following steps:
- Create a coverage profile for each individual package using
- Concatenate all coverage profiles into a single one.
- Pass the final profile to
go tool coveras usual.
Since I didn’t want to repeat these three steps over and over again, I wrote a shell script doing it for me. I store this script as
script/coverage in every Go project I need it. By default,
script/coverage outputs test coverage information for each package and, after that, for each function of a project’s source code. This is what it looks like for chef-runner:
$ cd chef-runner/ $ ./script/coverage ok github.com/mlafeldt/chef-runner 0.012s coverage: 4.5% of statements ok github.com/mlafeldt/chef-runner/bundler 0.013s coverage: 100.0% of statements ok github.com/mlafeldt/chef-runner/chef/cookbook 0.014s coverage: 89.7% of statements ok github.com/mlafeldt/chef-runner/chef/cookbook/metadata 0.012s coverage: 90.5% of statements ok github.com/mlafeldt/chef-runner/chef/omnibus 0.019s coverage: 77.3% of statements ok github.com/mlafeldt/chef-runner/chef/runlist 0.012s coverage: 100.0% of statements ok github.com/mlafeldt/chef-runner/cli 0.016s coverage: 95.2% of statements ok github.com/mlafeldt/chef-runner/driver 0.006s coverage: 0.0% of statements ok github.com/mlafeldt/chef-runner/driver/kitchen 0.013s coverage: 72.2% of statements ok github.com/mlafeldt/chef-runner/driver/ssh 0.012s coverage: 72.7% of statements ... github.com/mlafeldt/chef-runner/rsync/rsync.go:47: Command 100.0% github.com/mlafeldt/chef-runner/rsync/rsync.go:92: Copy 0.0% github.com/mlafeldt/chef-runner/util/util.go:15: FileExist 100.0% github.com/mlafeldt/chef-runner/util/util.go:22: BaseName 100.0% github.com/mlafeldt/chef-runner/util/util.go:31: TempDir 100.0% github.com/mlafeldt/chef-runner/util/util.go:36: InDir 62.5% github.com/mlafeldt/chef-runner/util/util.go:55: InTestDir 80.0% github.com/mlafeldt/chef-runner/util/util.go:65: DownloadFile 75.0% github.com/mlafeldt/chef-runner/version.go:15: VersionString 100.0% github.com/mlafeldt/chef-runner/version.go:23: TargetString 0.0% total: (statements) 74.2%
If the option
--html is given,
script/coverage will additionally create a HTML report and open it in a web browser. The report contains annotated source code with different colors highlighting what code is being tested a lot (green) and what code doesn’t have any tests yet (red). Here is a sample HTML report for chef-runner. Isn’t it fantastic? It’s actually my favorite piece of the tooling.
Integration with Coveralls
Shortly after learning how to generate proper coverage statistics, I stumbled upon Coveralls – a web service that provides test coverage history and statistics for projects hosted on GitHub. Coveralls also happens to support Go. Integrating it with Travis CI, another service I’m using to test my open source projects, turned out to be straightforward. Here is a basic Travis CI configuration:
# .travis.yml language: go install: - go get -t ./... - go get code.google.com/p/go.tools/cmd/cover - go get github.com/mattn/goveralls script: - PATH="$HOME/gopath/bin:$PATH" - script/coverage --coveralls
As you can see, I just added another option (
script/coverage that will cause it to push coverage statistics to Coveralls.
Last but not least, this is what Coveralls does with chef-runner’s coverage data. While I find the provided statistics interesting, I’m fully aware of the fact that they say nothing about how good the tests are. Please keep this in mind.
If you, fellow Gopher, find any of this useful, feel free to use my script to add test coverage to your own personal projects as well. Even if you don’t plan to use external services like Coveralls, having coverage statistics at your fingertips can still help you identify bits of untested code.
Tagged under: Golang, Coverage, Testing