Improving Testing through Testing Debt Quadrants
I was lucky to attend Euro Testing Conference 2020, in Amsterdam, last month; I learned so much. One of the workshops that I attended was Rob Meaney's "Exploring Testability" which inspired me to try something within my team. At the conference, we did a hands-on exercise related to testing debt. Testing what?! Debt! This article will explain the testing debt concept, how to make it visible and how to address it.
What is testing debt?
We all heard about the technical debt concept before - the development decisions made to provide benefits in the short-term that will, however, inhibit benefits in the future; in other words, imagine doing a quick "hack" to provide a feature instead of properly choosing the right architecture to support it. Testing debt is similar.
Testing debt occurs when a team intentionally or unintentionally chooses an option that yields benefit in the short term but results in accrued testing cost in terms of time, effort or risk in the longer term. -- Rob Meaney
Peter Varhol, puts it slightly differently:
The effort needed to find and fix issues that remain in the code when an application is released. -- Peter Varhol
In simple words: testing debt is all about the testing that you should have done but for some reason, you didn't.
Testing Debt Quadrants
During Rob's workshop, we mapped our debt using Testing Debt Quadrants.
The testing debt quadrants provide a way to visualize debt, including:
- untested things
- testing that is slow or impractical and that can lead it to be forgotten/not performed
- items/components that are complex to test
- items you cannot rely on
Therefore, we splitted the cardboard into 4 sections:
Note: instead of seeing all of the above as problems, they are in fact opportunities.
The process starts by adding post-its, identifying a "piece of debt" (i.e. something that we are not testing thoroughly or the way we should). As a team, we identify the ones that are most relevant (i.e. the ones that contribute the most to the testing debt). For each one of these, we discuss possible ways of addressing them and add them to new post-its. And now we have concrete actions that we can take to reduce our debt and improve the overall testing.
From theory to practice: how we did it?
After the conference, I wanted to try this exercise internally and see if it was valuable or not. I also wanted to make visible some pain-points on our testing process. We did this with a small group, including myself, part of our testers and also the team lead. Even though my role was essentially a facilitator, as I've been the product owner of the product we are testing, I do have particular insights about it which I cannot neglect :)
We started by doing a retrospective on our current development & testing process. By drawing it, we could easily see who was doing what and when. It also allowed us to depict some improvement opportunities right away. I see this kind of diagrams as starting points for discussion; some care should be taken though, or else this can lead to long discussions.
We then moved to the Testing Debt Quadrants and starting writing and adding post-its to the 4 quadrants. Each one did this alone and in the end, we could see some overlap (which I'd say it is "normal" to happen). Where we position each post-it in the cardboard is somehow subjective; however, as we reviewed the whole cardboard, we move some of them and agreed on it. Note that, for example, some items can be slow and complex at the same time. Where the post-its end up doesn't need to be scientific; use "positioning" as a means for further discussion/insights. We saw that we had some unfinished work that could tackle some of the identified problems. We just need to turn that into reality.
From discovery to actions
Having discovered and discussed part of the items that contribute to our testing debt, it's important to have actionable, prioritized items. Thus, we decided to create a very simple Trello board to track their implementation and/or discuss anything around them.
It's interesting that as per our initial discussion during the "visual mapping" of our development/testing process, we could depict some opportunities to improve testing considerably. Thus, the board contained actions not only found during the Testing Debt Quadrants but also from the initial sketch we did about our overall process.
Things to have in mind
Whenever looking at its full extent, it's easy to get lost and start discussing a ton of different topics.
Some recommendations for us and for anyone trying it out:
- timebox it, so we don't extend it endlessly
- focus (no interruptions)
- involve other roles
- think on improving gradually, a step at a time (this will avoid finding the "perfect solution")
- revisit the cardboard regularly
I would also recommend having the conversations in such a way where there is no pinpointing. This should be a team exercise, open but not individual-focused.
And now what?
I think mapping your testing issues, which are also opportunities to improve your overall testing process, using Testing Debt Quadrants is a simple and valuable exercise. All the conversations you have are important to have a joint overview of how testing is being addressed and how it can be improved. Just make sure that these conversations aren't pointless and, instead, lead to actions that you can track and revisit together somehow.