The Case Against Unit Tests

We’ve all heard of Unit Tests as the de facto way of testing our applications and libraries, but in this post, I’ll cover the best way to apply Unit Tests, why they’re overrated and possibly overused, and the better alternative most of the time.

What are unit tests?

I assume you may already know unit tests, but I’ll provide a quick summary here nonetheless.

Unit tests are one kind of tests that attempt to test a single unit of code, with the least external behavior possible. By unit, this is often meant as a function or a class. These tests often include the usage of mocks, where you change the dependencies you’re relying on (service, libraries) to avoid the external behavior from impacting your test.

These tests are fast to run and easy to understand, but, I think everyone can agree, boring to write.

Why were unit tests so glorified in the past? (my opinion)

Nowadays, we have libraries for nearly everything, we spend more time chaining different library calls than writing custom code or complex algorithms to perform something. Our business logic now refers only to business, and as much as algorithmic logic as in the past.

Plus our compiler or virtual machine handles the memory management for us (contrary to C++). In the past, unit tests were critical to avoid a change that could end up in a buffer overflow or other memory management issues.

This is why I believe the unit tests were really relevant in the past; but today, not so much.

When are unit tests mostly useless?

Throughout this post, we’ll see 2 examples from a project I’m developing. (shameless plug)

Quick summary, this projects queries pieces from an agenda API and sends to a social chat.

The first example is this method: send_event, it takes the event and the channel to send it to. Unit testing fans would immediately jump into writing a unit test for this. How? By mocking the client struct field and ChannelId parameter, and verifying the message_builder given to send_message was what we would expect.

But how do we benefit from this? One could argue that we could ensure the fields had the name we’d expect, the author was the correct one, and the color of the message was right as well. However, this alone seems like a rather weak test, we’re only testing fields being passed around, while developing this, we do either manual or automated testing (more on this later) and can see it works as we want. (and this is a visual thing)

And would this help prevent future changes from corrupting the code? It could, although less likely with code review. We’d be better off ensuring it works, and not simply we do it as we wish to do it.

Don’t take me wrong, this is great to ensure, but a unit test may not be the best way to do so.

A better alternative – Diamond testing

We saw unit tests don’t suit our needs, so what does?

This leads us to integration tests and the concept of Diamond testing.

Integration Tests

In integration tests, we want to test our code (functions/classes) integrated with the libraries. That is, without any mocks, and seeing it work effectively.

In my project, I perform requests to the agenda API and then the social chat, so true integration tests would at least mock the APIs themselves, with a mock http server (which responds to the API calls with the responses we’d expect). However, this would be really expensive to do, for no real benefit.

Instead, I write what some would call an automated test, where I effectively call that send_event, unmocked, and sending the event to a testing channel on the social chat. (code)

I simply call the function and, in case there is a failure sending, the function throws a panic/exception, making the test fail.

I could have written asserts for validating that the message that lands on the chat is as I expected it to be, but that would be more of an end-to-end test, and add code to the test, which would make the test more fragile to failure on the test itself.

In integration tests, we don’t test the libraries, that is, I expect calling the send_message function of this library, will work when no panics/exceptions are thrown.

The Diamond testing

This way of testing, a few unit tests, many integration tests, plus a few more end-to-end tests, is called Diamond testing.

Image source

The source of this image also covers the other testing strategies, along with the benefits of each.

The drawback of this strategy vs the traditional pyramid (many unit, some integration, a few end-to-end) is that the testing takes longer to run. Honestly, for a better development experience and trust in my tests, I’m willing to wait those extra 1/2 mins to run all tests. (while developing, I only run the relevant tests, so these don’t impact me much)

Cases where unit tests are useful

The Diamond testing also includes unit tests and for a good reason. Unit tests are particularly useful for functions with business logic or algorithms.

In my project, I have unit tests for that like this one, which validates the date is extracted as I want it to, or this other one, which validates the description is extracted from an HTML page properly. These lean more towards the integration tests since I’m not mocking the libraries, but in these cases mocking would be unnecessary. (and these tests are still fast to run)

A hypothetical example of a true unit test would be validating a pay(creditCard, amount) method that would debit the amount from the creditCard.

Should your test coverage include Integration Tests?

As I said in the previous chapter, we want most of our tests as Integration tests. Since our tests should cover at least 70% of the code (test coverage) – to ensure we are testing most of our code – yes, we should include integration tests.

What about Test-driven development? (TDD)

TDD is all about writing tests before writing the actual code to pass the tests.

One complaint I received is that unit tests are crucial for TDD. However, in my opinion, and experience, integration tests can (and should) be used on TDD (commonly called Behavior-driven development). In fact, on the project I mentioned earlier, I encountered some bugs and immediately wrote an integration test that confirmed them, later fixing the bug in the code to confirm I fixed it. On that project, I rarely ran it to test it, solely tested via integration tests.

Again, you can still have unit tests on TDD – it’s not one against the other, it’s just a matter of seeing which one better fits the use case.

Conclusion

There are multiple strategies for testing your application, and the traditional pyramid seems outdated for the reasons given.

Integration and unit tests should be a staple of every project, and they should be balanced in the right way.