Unit Tests¶
Unit testing is a fundamental tool in every engineer's toolbox. Unit tests not only help us test our code, they encourage good design practices, reduce the chances of bugs reaching production, and can even serve as examples or documentation on how code functions. Properly written unit tests can also improve engineer efficiency.
It is no secret that writing unit tests is hard, and even harder to write well. Writing unit tests also increases the development time for every feature. So why should we write them? Unit tests should:
- reduce costs by catching bugs earlier and preventing regressions
- increase engineer confidence in changes
- speed up the engineer inner loop
- act as documentation as code
Unit tests should also be very predictable (i.e. any failures should indicate broken code) and very fast (unit testing a )
Integration tests¶
Integration testing is a software testing methodology used to determine how well individually developed components, or modules of a system communicate with each other. This method of testing confirms that an aggregate of a system, or sub-system, works together correctly or otherwise exposes erroneous behaviour between two or more units of code.
Code Coverage¶
The easiest way of quantitatively measuring the quality of automated tests is by examining code coverage. Whilst a high code coverage percentage does not necessarily mean that the tests are of a high quality, it is certainly true that a low code coverage is indicative of there not being enough automated tests!
Note that we are more interested in branch coverage than line coverage.
There are two ways that our repositories measure and report on code coverage:
- By generating an HTML report
- By checking that committed code meets a minimum coverage in GitHub Actions
Note: if the repository does not contain the tools to measure code coverage, follow this guide for .NET repositories to add it.
The HTML report can be very simply generated by selecting the Run Code Coverage item from the Tools menu in Visual Studio. This will take a few moments runs the tests and generates the report and displays it in the Visual Studio window. The report gives a detailed breakdown of test coverage by class. Clicking through to a class will then break this down by method and individual line.
The report can then be used to:
- Give an indication of parts of the code that are not tested (and thus need new tests or test cases writing)
- A methods having a high number of branches (or high cyclometric complexity) is potentially and indication that the code in the method should be broken down or split out into other classes
- Provide the current code coverage
The .github/workflows/tests.yml workflow definition file contains the step to validate code coverage, and sets the minimum coverage level for the repository.
Code Coverage Requirements¶
As part of our Quality Standards, we require that around 80% of our code is covered by automated testing. When working on a repository, an engineer should ensure that any new code is also covered by new automated tests to maintain or increase this coverage.
Existing repositories may not hit this 80% coverage threshold, so expecting to meet it when making a change to an existing repository may not be realistic.
When committing code, an engineer should consult the latest coverage report to get the latest branch coverage, and update the BRANCH_THRESHOLD variable in the tests.yml.