When is a defect not a defect - the different flavors of errors in development

When working with IT projects you can not help but to run into defects and issues in different environments that require fixes. Strangely enough this is often poorly defined and I have seen some creative ways to manage the many different things that is being called defects. So what is a defect really and if something that is not working is not a defect, what is it and how do we handle those?

Defect. Bug. Incident. You have probably heard all of them mentioned, probably in the same development project even. Maybe you are one of those unfortunate ones that even have something as weird as a "defect type" to define what type of defect you are dealing with? Well, you are not alone. This is a common issue in many organizations and in this article I will describe my point of view on this subject.

What is a defect, for real?

Quote

"A defect is when the solution does not match the agreed requirements"

- Jimi Wikman

This is the ONLY definition of a defect.

I know a lot of people will object to this because defects can be found that is not described in a functional, or non-functional, requirement. This is true, but you forget that we always have implicit requirements such as that the system should work as intended.  This means that all defects can be tied to a specific requirement, or to an implicit requirement.

The only issue with this is that for some tests it can be hard to actually connect a defect to a specific requirement. This happens especially with exploratory testing and in large, complex solutions. That is perfectly ok and you should not spend time finding the exact requirement if it does not have a positive impact on the work process itself. We want to improve the quality of the system after all, not build the perfect documentation.

The different types of defects

As a Jira expert who design workflows for many organizations I sometimes get asked to add a custom field for labeling defects in different ways. The answer is always no. You can not have different types of defects because it is either something that is not working according to a requirement or it is not. It is as simple as that. Adding labels to defects is just a band-aid for a work processes that does not work.

Common labels people want to use in these situations are:

• Not a Defect - This one is hilarious to me. If it is not a defect, then just close it and move on. No need to have it open with a label saying it is not a defect. That is just a sign of uncertainty from the team that you do not have the courage to actually close things.
• System issue - This is a label that again is just stupid. It is still a defect, you are just pointing towards someone else to blame for the mess. Don't add a label, just hand it over to another team.
• New Requirement - You can not have a defect that is also a new requirement for obvious reasons. If this was timetravel you would just be responsible for creating an endless loop. Close the defect and create a new requirement, or change the defect to a new requirement.
• Changed Requirement - This is just petty and a bit cowardly because if you are dealing with unclear requirements then you bring that up as a problem and fix it. If you have a proper requirement process, then this will never occur, so just close the defect if someone change the requirement after development starts and add a new development task for it. Don't feed the trolls that are responsible for poor requirement management, squash them and get things working the way they should.
• Design bug - Design is part of the requirement process. If you have a defect because the design was updated, then see my comment on changed requirement. If you did not follow the design specification, then it's just a defect.
• Can Not Reproduce - Well, that happens and what you do is that you make sure you have steps to reproduce and that you talk to the reporter. If you can not reproduce despite that, then you just close the defect.
• Wrong Team - Well, then just reassign the defect to the right team then. This is just lazy...

In most cases labels like these are invented because the team do not feel comfortable bringing up that the requirement and test process have issues. Rather than adding labels to passively stick it in the face of the people in charge, I suggest you instead add other types of data to be added to every defect:

• Steps to reproduce - This is a must and it should be detailed so anyone can verify. I usually say that the reporter should ask the janitor or receptionist to verify the steps to reproduce before submitting the defect. If this is not clear, then close the defect with can not reproduce. Eventually people will learn how to write this properly out of sheer frustration. Better yet is to help educate the reporters so they get this right.
• Environment - Where was this found. All development should be done in 4 environments: Development, Test, Pre-prod and production. All defects should be found in test or pre-prod. Defects found in production are incidents and defects found in development just means that the development is not done yet.
• Affected Version - Not always applicable, but if you do not have continuous deploys several times a day all the way to production, then you should have a controlled versioning system. Anything deployed to test and pre-prod have a version, even if it is just a tag. So make sure you ask that this is added as it will help you understand what code base the defect is present in. It also make the reporters more inclined to keep track of the code deploys.
• Connected Requirement - This can help reducing the problem with "nagging" or scope creep. That is when you get defects that have no basis in the requirements, but are usually new requirements. If this is a big issue, then make this mandatory or close defects without this field until the reporters learn what a defect actually is.
• Component, Module or System - Useful to make sure the right team get the defect if you are in a multi team setup with multiple components or systems.

Stop working with poorly defined defects and take control over the process for reporting defects. Crappy defects exist for a reason and while it may be very difficult to speak up in some cases you should always try. Bad defect processes will impact morale and system stability.

Remember: A defect is when the solution does not match the agreed requirements.

Nothing more, nothing less.