Make your tests run in less than a second

What I did

For the open-source project docxtemplater I work on, my whole test suite runs in less than 500ms (280ms on my machine, 345ms on Travis). It contains more than 1000 assertions in total, about 50 describe statement. This permits to get very fast feedback about what you change, and is extremely pleasant to work with. On my computer, I have created a command that watches every file save made from vim and triggers the tests automatically, turning a usb-light green or red depending on the result of the tests.

I have written a small bash script called swatch with which you can do swatch npm test which means « watch vim file saves and run the command specified on the right every time that event happens ». Source code of swatch

Having a very fast test suite allows to integrate your test checks inside your workflow, without having to run your tests manually once you think you have finished a feature. When your code is complex, it is a great way to refactor without having to wait and see if your changes broke something.

How to speed up your test suite

You can do a few things to improve the speed of your test suites :

First things first, if your tests need a lot of time, maybe your code is just slow : a HTTP request (including data-transfer and business logic) should take less than 500ms. If it is slower, you should be more concerned by the speed of your application.

If you test only the business logic, your code should be taking less than 50ms to run with normal data.

Here is a small list of what you should avoid in a test suite :

  • Reads/Writes on the filesystem
  • Interactions with a database
  • HTTP requests

I’m not a huge fan of mocking because maintaining them wastes a lot of developer time. Try to minimize the number of mocks you decide to write, by first analysing where the biggest bottlenecks are (this will depend on the system under test) and mock only those.

I’m not saying that you should use mocks in every program. For example, in my docxtemplater project I don’t mock anything and still do a few reads on the filesystem. If I decided to mock the calls to the filesystem, I know it could even more improve the speed of my test suite. However I think it isn’t worth the effort because the speed is already very satisfying.

Don’t test your whole test suite everytime

Many testing frameworks have an option to filter tests, so that you can choose which tests you want to run. If you are working only on a part of an application, you can narrow the tests you run to get faster feedback.

For example, with mocha, you can write your test by using the .only property:

describe.only("This is the only test that will run", function(){
    // TESTCODE
})

I’d be happy to help you improve the speed of your tests

If you maintain an open-source project and would like to speed up your test suite, I’d like to help you for free. (just contact me at contact{AT}{this-domain-name})

Laisser une réponse

Votre adresse de messagerie ne sera pas publiée.