Incremental testing: save time and money on CI for monorepo

To use monorepo or not is an eternal debate. Each has its pros and cons. Let’s say you decide to go with monorepo, one major issue you will face over time is slow testing. Imagine a monorepo, consisting of an Android app, an iOS app, some backend code, some web frontend code. In only very few occasions will someone modify more than one of those simultaneously.

Further, add to the fact that most of these projects confined to their directories would be using different build systems as well, for example, gradle for Android, yarn/npm for Javascript, go/rust/java/npm for the backend. The total build time, as well as test time, will only grow over time. It annoys developers making small modifications to their part of the codebase. And it slows down the development velocity drastically.

At my previous job, I ended up writing an elementary version of the incremental testing code. The thesis was that we had a set of directories, most unrelated to each and some dependent on each other. As long as we come up with a good directory-dependency tree, then a script would check if any files in the pull request modified any of those dependencies; if yes, the test continues, if not, it is skipped. We can build incremental testing and cut down a lot of extraneous builds. The goal was not to eliminate all extraneous builds but to minimize the biggest offenders. And that alone made the testing much more manageable. We saved money by reducing the number of Circle CI instances we reserved, and we increased our developer velocity (in a monorepo like ours, you can expect 5-10x faster testing time).

The code is open-source, so I can easily reference it here. We used Circle CI, but the approach is generic enough and can be used on any other build system as well.

Consider, for example, protocol-test

The wrapper script

And the full script which checked which found out the patch side of this pull request over the base branch can be seen here. The most crucial method being,

 

One Reply to “Incremental testing: save time and money on CI for monorepo”

  1. […] make it hard to find code and one single repository makes it harder to do simple things like testing, bisecting (to find buggy commit), deciding repository […]

Leave a Reply

Your email address will not be published. Required fields are marked *