Setting up TeamCity coverage report for node.js test with mocha and jscoverage

Hi!

The mentioned tech stack has been popular for quite a while, leading to many people using it (especially in startups, for quick and robust API implementations, etc.). The question of testing is solved quite well (there is a number of testing, mocking, assert frameworks out there which I find astonishing).

This post is about quite a regular task on how to setup coverage metrics automatically on the build server (TeamCity is my favourite).

Disclaimer: I have a strong belief that coverage metrics should never be given to the business people w/out tech control. When they see _any_ metrics, they become excited with it and we have that overused. I believe, that coverage should be used by developers themselves to see what code paths have not been tested, and what tests can they add.


So, let's start. Prerequisites:

1. You have mocha tests
2. You have a working installation of TeamCity, which is already pulling the sources from repo, building app, running these tests on post-commit basis (and surely sends notifications to the people in charge).
3. You want to add coverage metrics to the party to see what's the progress and what areas need more help. 

Given that your project structure knows nothing of jscoverage, we'll go least-intrusive, scripting way. 

Jscoverage

https://npmjs.org/package/jscoverage

You need a specific jsCoverage build, which knows about node. You can have that installed, as usual, with npm:

 

npm install -g jscoverage

Verify the install, by issuing jscoverage on the commandline. The output is not very helpful, but gives the most interesting parameters you'll need. 

The way all coverage tools work is that either instrument the product of your code (or code itself, for scripting tools), OR plug into the interpreter, and when you apply some interaction to this code (tests, users coming in and out, etc), collect how many times each of the lines was called. After these numbers are obtained, static analysis may plug in to verify how many functions/branches/conditions were covered. 

So, we need to obtain such an 'instrumented' code first from our target code. That's what jscoverage doing:

jscoverage --exclude tests,node_modules,.git,target,.idea  CURRENT_FOLDER OUTPUT_FOLDER

After that you can examine the results in the output folder. All your files should be copied there (except the ones included), and they should have some weird code inside (that's the recording part!), with pieces and bits of your code scattered across the file. 

mocha

As you are already probably using mocha for testing, nothing is really tough here. I assume that you are loading your modules-under-test with the require('../../lib/something'), rather than via absolute paths. 

As we already have a specific folder, which contains jscoverage-processed main app code, w/out all the unnecessary folders, we could now just copy out the tests themselves to that folder, and run mocha. No results yet, we just verify that mocha tests run with our processed code. 

Next, we are good to go to add some html-cov reporting. This reporter is already bundled with mocha, so no need to reference it in package.json. what it does, is that after the mocha tests are ran, it gets the coverage statistics from coverage-augmented files under test, and flushes that (in form of HTML) to the stdout. So, to see what's in the coverage, we do (whilst in the OUTPUT_FOLDER): 

../.././node_modules/mocha/bin/mocha -t 20000 --noinject --reporter html-cov --recursive ./tests/ --coverage

--noinject is required for me, as I am using rewire library for testing, and that somehow clashes with the jscoverage stuff.

You could divert that output to file, or just pipe to w3m.

TeamCity

Quite an easy part now; since we have all working locally on the server, we just need to automate it. I have finally used a html-file-cov mocha reporter, it prints out the tests results as dots to the stdout, and writes the coverage report to coverage.html within working folder; quite an easy and handy tool.  https://npmjs.org/package/html-file-cov

In teamcity, we need to add another build step to the post-commit configuration. I did the easy and ugly script way instead of built-in report processing, since I was quite happy with the html format. 

So, as we have added that build step (of type Custom script), we could now fill it with the actual logic: 

#!/bin/bash

rm -rf ../target-cov/*

rm -rf ./target/target-cov

rm -rf ./target/coverage.html

mkdir -p ../target-cov

jscoverage --exclude tests,node_modules,.git,target,.idea . ../target-cov

mv ../target-cov ./target

cp -r ./tests ./target/target-cov

echo "JS files instrumented correctly"

cd target/target-cov

echo "Running mocha tests with 'dot' reporter"

NODE_ENV=dev ../.././node_modules/mocha/bin/mocha -t 20000 --noinject --reporter html-file-cov --recursive ./tests/ --coverage

mv ./coverage.html ..

echo "Done. Configuration stored to `pwd`/../coverage.html"

As a result of this manipulation (making a dir one level up, storing there the jscoverage-processed files, copying it to the local 'target' folder, copying over the tests non-processed, running the tests, moving the coverage.html to the target dir), we have the artifact - coverage.html - which contains all the stuff we need. 

We just need to publish it - in TeamCity project settings, on the very first page, there's an 'Artifacts' box, which takes paths. Just add +:target/coverage.html, and after each build you'll have a nice drop-down 'Artifacts' with a link to your html with coverage. 

Voila!