[Tech Blog] Test-Pyramid

Hi, I’m Minh Khoi, a front-end developer at AnyMind Group. In this article, I’m going to introduce to you what is a test pyramid in a software product, and to be more specific: a front-end project.

The problem in software testing

In my experience, automation tests provide the facility to record/run a series of interactions with the application and check the expected result or just simply input and output of a function. Those tests work well initially. It’s easy to write, execute but costly to maintain especially in an early stage or in a fast-growing project.

Some argue that testing is costly, redundant in a(n) early birth/fast-growing project. Because an enhancement to the system or a requirement change can easily end up breaking lots of such tests and it will slow down development progress, and you should focus more on manual testing, hire a good QA team instead of writing a bunch of automation tests.

And even in a released application, testing is costly when it comes to refactoring, optimizing the old code. Each change could lead to series of fail tests.

So I agree with that, testing is costly but it’s not redundant and in my humble opinion, a good developer should able to write a testable code and should test their code before send it to the QA. This is why the Pyramid in testing is good to know when we develop an application.

The Pyramid in testing

The test pyramid is a way of thinking about how different kinds of (automated) tests should be used to create a balanced portfolio. Its essential point is that you should have many more low-level UnitTests than high level BroadStackTests running through a GUI.

The test pyramid consists of three layers that your test suite should consist of (bottom to top):

1. Unit tests
2. Service/Integration tests.
3. User Interface tests.

• Unit tests are low-level testing, the term ‘low-level’ means they approach the lowest scope of your code (e.g: a condition, loop, function, class). This is the foundation of your test system because unit tests make sure that a certain unit of your codebase works as intended. And the number of unit tests will and it should be largely outnumber any other type of test.

• Your application will likely interact with some other parts (databases, filesystems, http calls). The interact part needs be tested and Integration Tests are there to help. You could mock and stub all collaborators with your application or allow taking real connection to test it. The goal of an integration test is to ensure that your integrated system functions as expected. So it is nice to have some tests that taking real connection of all components in your system altogether. But if you just want to test the connection of some specific part of your system, you could try to take a real connection of that part and mock the rest for a faster test.

• UI tests or E2E tests is the closest with the real user interaction on your web application. User input should trigger the right actions, data should be presented to the user, the UI state should change as expected. This is the highest level of automation testing. By using a tool or framework, E2E test tell the browser what to do, click, type and then expect the behavior of application base on the action. It is good to have full coverage with unit and integration tests. However, users can face unexpected behaviors when they run an application in the browser — E2E tests are the perfect solution for that.

Note: Modern web applications using single page application frameworks (React, Vue, Angular) often come with their own tools and helpers that allow you to perform the interaction between components in your application and with the collaborators in a pretty low-level manner. Therefore the line between unit test and integration test is kind of blur nowadays. Even with vanilla javascript, you can use tools like Jasmine, Mocha, or my favorite of all – Jest to test your application both unit tests and integration tests.

Base on the pyramid, you could easily spot some important things from it.

• 1: The more high-level you get, the fewer tests you should have.

  • Because of the higher level you get, the cost to write, run in the pipeline, and maintain is more expensive.
  • Low-level tests tend to run faster because they dont require a full DOM tree to run and easy to maintain because these test is small.

• 2: Write tests with different levels of detail.

  • You should focus on unit tests, write lots of small and fast unit test to cover all branches of the condition statement, input and output of function, etc.
  • Write some more integration tests to make sure your application behave correctly.
  • Write few high-level tests that test your application from end-to-end.
  • Watch out that you don’t end up with a massive test suite on integration and E2E that will be a nightmare to maintain and takes way too long to run.

• 3: Even we could bring up all the applications and connections from end-to-end to test. But we can not test all the styles or whether UI is matched with the design. That is why the high-level test should focus on the important scenarios.

Note: In my experience, if you get a failure in a high-level test, this likely means you have a bug in your functional code, which also means you have a missing or incorrect unit test. Therefore, after fixing that bug, you should replicate the bug with a unit test and a unit test will ensure the bug will be found if it ever happens again.

Wrapping things up

That’s it! I hope that there’s something useful in this article. Now go ahead and prioritize testing in your application’s test cycle. This is will pay off some days.

And remember: Eliminate bugs will save your company’s reputation.