The one thing I missed when moving away from full-framework and Visual Studio to VSCode and dotnet core, was simple code coverage.
Given the easy tooling
dotnet provides, with
dotnet test and
dotnet publish, I looked for something that integrated nicely with these commands without adding to much complexity to the code project itself. After som googling, I stumbled over Scott Hanselman's blogpost about a cool little project called Coverlet. Coverlet was just what I was looking for:
Coverlet is a cross platform code coverage library for .NET Core, with support for line, branch and method coverage.
coverlet can be installed as a
dotnet tool with
dotnet tool install --global coverlet.console
to make it globally available, providing it's own CLI tool running directly at the test assemblies.
The strategy I have settled on is using the
coverlet.msbuild package that can be added to your test projects with
dotnet add package coverlet.msbuild
When using the
coverlet.msbuild package, no extra setup is needed, and
coverlet integrates directly with
dotnet test with some extra parameters,
dotnet test /p:CollectCoverage=true /p:Threshold=80 /p:ThresholdType=line /p:CoverletOutputFormat=opencover
The clue here is
/p:CollectCoverage=true, the parameter that enables collection of code coverage. if no other option is specified, the coverage will be reported to the console when the tests are finished running:
+-----------------+--------+--------+--------+ | Module | Line | Branch | Method | +-----------------+--------+--------+--------+ | BikeshareClient | 93.2% | 94.6% | 85.7% | +-----------------+--------+--------+--------+
Now the other parameters specified in the example is
/p:ThresholdType=line. So if the code coverage drops below 80%, the build breaks here, while
/p:CoverletOutputFormat=opencover writes a report in the opencover format.
For most new projects, I have found myself using a simple
Dockerfile along with some CI/CD tool like Travis, AppVeyor or Azure Pipelines. This approach helps keeping the builds simple, as large
Dockerfiles are harder to work with. The sole purpose of
Docker is to keep things reproducible no mather the environment it builds and runs images in, so migrating from one CI provider to another is hardly any work. Building locally will always match the result on the CI system.
But, let's say build using multi-stage Dockerfiles. In a multi-stage build, we separate the SDK, build and test tools in one image, while copying the resulting artifacts to another image, more suitable for production runtimes. The rule is, have a small production image containing just what is needed for running your artifacts. Just one problem: How do we take care of that
coverage.opencover.xml file? We don't what to transfer that file to the production image to grab hold of it, code coverage results don't belong in a production image.
Docker stores layers that can be brought up after building the image.
Here is our example multi-stage
In short, we build, test and publish the app with the
microsoft/dotnet:2.2-sdk base image, before copying over the binaries to the
coverlet and extract code coverage, this line does the trick:
RUN dotnet test /p:CollectCoverage=true /p:Include="[BikeDashboard*]*" /p:CoverletOutputFormat=opencover
label on line 3:
With the label, it is possible to look up the id of the
docker build layer containing the code coverage file, create a container from that image layer and use
docker copy to grab hold of the coverage XML. Take a look:
export id=$(docker images --filter "label=test=true" -q | head -1) docker create --name testcontainer $id docker cp testcontainer:/app/TestBikedashboard/coverage.opencover.xml .
Wrapping it up with Travis and codecov.io
So now we have a simple build chain with a multi-stage
Dockerfile and code coverage generation. As a last feature, the coverage report can be used by code coverage analyzers like codecov.io. codecov.io integrates with Github, and can automatically analyze incoming pull-request and break a build if coverage drops by merging the PR. Quite nifty.
Integrating codecov.io with CI systems like Travis is done with a one-liner, thanks to the provided upload-script. When using Travis, not even a token is required.
.travis example file: