Testing

2 articles in this category

When is a defect not a defect - the different flavors of errors in development defect management
When working with IT projects you can not help but to run into defects and issues in different environments that require fixes. Strangely enough this is often poorly defined and I have seen some creative ways to manage the many different things that is being called defects. So what is a defect really and if something that is not working is not a defect, what is it and how do we handle those?
Defect. Bug. Incident. You have probably heard all of them mentioned, probably in the same development project even. Maybe you are one of those unfortunate ones that even have something as weird as a "defect type" to define what type of defect you are dealing with? Well, you are not alone. This is a common issue in many organizations and in this article I will describe my point of view on this subject.
What is a defect, for real?
This is the ONLY definition of a defect.
I know a lot of people will object to this because defects can be found that is not described in a functional, or non-functional, requirement. This is true, but you forget that we always have implicit requirements such as that the system should work as intended.  This means that all defects can be tied to a specific requirement, or to an implicit requirement.
The only issue with this is that for some tests it can be hard to actually connect a defect to a specific requirement. This happens especially with exploratory testing and in large, complex solutions. That is perfectly ok and you should not spend time finding the exact requirement if it does not have a positive impact on the work process itself. We want to improve the quality of the system after all, not build the perfect documentation.
The different types of defects
As a Jira expert who design workflows for many organizations I sometimes get asked to add a custom field for labeling defects in different ways. The answer is always no. You can not have different types of defects because it is either something that is not working according to a requirement or it is not. It is as simple as that. Adding labels to defects is just a band-aid for a work processes that does not work.
Common labels people want to use in these situations are:
Not a Defect - This one is hilarious to me. If it is not a defect, then just close it and move on. No need to have it open with a label saying it is not a defect. That is just a sign of uncertainty from the team that you do not have the courage to actually close things. System issue - This is a label that again is just stupid. It is still a defect, you are just pointing towards someone else to blame for the mess. Don't add a label, just hand it over to another team. New Requirement - You can not have a defect that is also a new requirement for obvious reasons. If this was timetravel you would just be responsible for creating an endless loop. Close the defect and create a new requirement, or change the defect to a new requirement. Changed Requirement - This is just petty and a bit cowardly because if you are dealing with unclear requirements then you bring that up as a problem and fix it. If you have a proper requirement process, then this will never occur, so just close the defect if someone change the requirement after development starts and add a new development task for it. Don't feed the trolls that are responsible for poor requirement management, squash them and get things working the way they should. Design bug - Design is part of the requirement process. If you have a defect because the design was updated, then see my comment on changed requirement. If you did not follow the design specification, then it's just a defect. Can Not Reproduce - Well, that happens and what you do is that you make sure you have steps to reproduce and that you talk to the reporter. If you can not reproduce despite that, then you just close the defect. Wrong Team - Well, then just reassign the defect to the right team then. This is just lazy... In most cases labels like these are invented because the team do not feel comfortable bringing up that the requirement and test process have issues. Rather than adding labels to passively stick it in the face of the people in charge, I suggest you instead add other types of data to be added to every defect:
Steps to reproduce - This is a must and it should be detailed so anyone can verify. I usually say that the reporter should ask the janitor or receptionist to verify the steps to reproduce before submitting the defect. If this is not clear, then close the defect with can not reproduce. Eventually people will learn how to write this properly out of sheer frustration. Better yet is to help educate the reporters so they get this right. Environment - Where was this found. All development should be done in 4 environments: Development, Test, Pre-prod and production. All defects should be found in test or pre-prod. Defects found in production are incidents and defects found in development just means that the development is not done yet. Affected Version - Not always applicable, but if you do not have continuous deploys several times a day all the way to production, then you should have a controlled versioning system. Anything deployed to test and pre-prod have a version, even if it is just a tag. So make sure you ask that this is added as it will help you understand what code base the defect is present in. It also make the reporters more inclined to keep track of the code deploys. Connected Requirement - This can help reducing the problem with "nagging" or scope creep. That is when you get defects that have no basis in the requirements, but are usually new requirements. If this is a big issue, then make this mandatory or close defects without this field until the reporters learn what a defect actually is. Component, Module or System - Useful to make sure the right team get the defect if you are in a multi team setup with multiple components or systems. Stop working with poorly defined defects and take control over the process for reporting defects. Crappy defects exist for a reason and while it may be very difficult to speak up in some cases you should always try. Bad defect processes will impact morale and system stability.
Remember: A defect is when the solution does not match the agreed requirements.
Nothing more, nothing less.

Written by: Jimi Wikman
Category: Testing
April 3, 2020
0 comments
639 views
• 

Defect versus Incident - what is the difference and why does it matter? defect management
Some people say Defect and some say Incident. Both are correct, but what is the difference and why does it matter? In this article I will explain how I see the difference, why it matters and perhaps more importantly when it matters. 
When it comes to defects, then we have already defined that a defect is when the solution does not fit the requirement. We also defined what a defect is not, so we should be fine with just the term defect. Right? Well, there are actually a few reasons why we should actually have have two different definitions for a defect.
The Legal Aspect
One of the least understood aspect of defects is the fact that there is a point in the development process that change the way we manage defects. That point is when a solution is accepted, which occur during the acceptance phase before code is deployed and released in production. From a legal perspective this is when the standard agreement between the client and the provider is fulfilled.
There might be additional services added beside the standard agreement that add another layer on top of the standard one, like a post go live support period or extended defect management after release.
This point in the development is also where responsibilities sometimes shift from the development team to a maintenance team for example.
The Complexity Aspect
Another important aspect is the complexity aspect that comes into play as the deploy is released into a production system. Even if the pre-prod environment is identical to the production environment it is still withing the development teams responsibility. Once put into production this usually changes and the complexity with the full solution as well as the full network make it a much more costly affair to manage problems.
When issues occur in the development teams environments it rarely involve other teams. Problem are managed within the team with the occasional support from surrounding system teams or network teams. Once you go to production however problems often become the affair of many teams.
The Workflow Aspect
How the team work with a defect is not the same for something found in a production environment, for obvious reasons. In a normal development workflow you can choose what code go to production, but for a defect found in production you do not have that choice. This is why you bypass the normal workflow and use hotfix, which is a fast track to correct problems in production environments.
Two ways of working, two types of defects
Based on the fact that we do have different processes for a defect found in production and for a defect found in the lower environments we should separate the issues themselves. This is not just to make it easier to identify in for example Jira, but also since there can be differences in who will manage the defect and the process to manage the defect itself. It also allow us to define the defects differently.
Defects are problems that prevent acceptance.
If there is a defect, then acceptance should not be given. It is a problem that only affect the legal aspects of an agreement. It can cause delays, which in turn can damage the company financially, but it will not affect the end user as it has never been released. By definition a defect is considered an internal problem and solving it involve only the development team within the standard way of workflow. It is simply development not yet complete.
The severity of a defect determine when in time it should be fixed based on dependencies withing the project as well as surrounding systems.
Defects are considered free since they are part of the development agreement.
In Jira a defect is placed inside a development to block it from going into production.
Incidents are problems that can damage the company.
An incident found in production is available to end user and as such it can cause damage to the company. It can cause disruption of service, it can damage the brand perception or even cause direct impact on sales. Solving this problem often involve multiple teams and it require a separate workflow that disrupt the standard workflow.  By definition this is an external problem and it can cause legal issues as it is found after acceptance has been given.
The severity of the incident determine when in time it should be fixed based on the damage it causes.
Incidents always comes with a cost, which usually is some form of support or maintenance agreement.
In Jira an incident is a standalone issue that is handled as development task.

 
This is my way of defining the differences between a defect and an incident. I have found that by making this distinction I can design better workflows in Jira that make incidents more visible. I can also use this in Jira Service Desk and other systems and by separating the two types I am not limiting the different workflows and I do not have to bloat the workflow by combining the two into one.
Do you agree with my definition?

Written by: Jimi Wikman
Category: Testing
April 4, 2020
0 comments
5,370 views
•