Inverting the Test Pyramid, by Joel Masset

Great Article from Joel Masset, Global Head of Product Assurance and Chief Quality Officer in the Financial Services Industry

Inverting the Test Pyramid
by Joel  Masset
https://www.linkedin.com/pulse/inverting-test-pyramid-joel-masset

Although most organizations focus on what it means for development teams to be agile, what that changes for them, and how to make it happen, testing is still very important in an agile delivery model. It is even more critical than in a traditional model.

I want here to highlight the big change that agility represents from a testing point of view. This is highly inspired from Mike Cohn’s well known test pyramid. However, I thought it was worth sharing this once more, as it is so close to what many major software delivery organizations need to go through nowadays.

In a traditional software delivery approach (waterfall, V cycle, iterative V cycle, washing machine…), all activities are serialized. Business requirements, design, development, test, debugging, […], release. The focus is put on finding bugs.

This leads to very little test effort to happen at the beginning of the cycle, and massive focus at the end, after developers have integrated their components into a product.

Roles are very clear and distinct. Developers’ role is to code and debug, testers’ is to find bugs, and check they are fixed.

Automated tests developed after the integration has happened will be essentially based on the UI. Their development cost is very high and since they are very fragile, maintenance costs are very high too. Any update to the code is likely to break the tests, which will require automation code to be updated prior to being ran. Since most of the testing focus is happening so close to the expected delivery date, there is a lot of pressure on the team to give its test campaign results. In most cases, it results into massive manual testing. In the best case, automated test development will happen in parallel, so that these can run post the release, but very regularly, the team will just give up on automating.

As a result, automation is extremely painful, costly, and inefficient, with a very low Return On Investment. The manual testing in this model is very costly, and happening under stress.

Don’t get me wrong, I am not saying this model cannot work, I have myself been using it for years. But oh my god, is it stressful…
Agile testing takes the opposite approach, by inverting the test pyramid.

The focus is now put on preventing bugs from existing. This means that most of the test effort will happen at the beginning, at the code and UI level. In this context, automation will be extremely efficient. Unit tests will be written alongside development, or even before the actual code is written or updated (this is the Test Driven Development model).

Acceptance tests will be created at the API layer level during development and integrated in the continuous build and integration tools and processes.

At the time the developer checks the code in, it has already been successfully tested. Post-build and integration tests will automatically be ran and results checked.

Since all those tests happen during the sprint, a failed test will immediately lead to a code change fixing the issue. Not only the cost of fixing a problem is much smaller at this stage, but the risk associated to re-opening code that was already pushed, days or weeks ago is gone. The team then rarely has to switch context from the current version of the code to an older one. This is a much more reliable process.

Some testing will still be required after the integration steps. Some will still be automated at the UI level. This represents very boring and repetitive tests like links checks, or basic user workflow steps. The essential part of testing at that stage is exploratory testing. Based on end user workflows, these are tests that only a human person can do. But the effort will be limited here.

In this new approach, you end up with developers and testers both testing and coding together. A much higher synergy, and very little stress, because the delivery process is made much more reliable.

Results are impressive on the Quality of the software, the predictability of the releases, and as well on the motivation of the teams.

This change requires a big cultural change obviously, hence very high senior leadership, this will certainly be the focus of a future article…

Bill Fulbright
770-880-0959
Bill@2100solutions.com