Defect Management Today

In the defect management world, the best defect is the one that never happens. Prevention is better than a cure. But until we reach a state of perfection in our product development teams, tools, and, processes, we should consider how we can manage defects for easier, faster new product introductions (NPI) and to continuously improve products.

The problem is that defect management is often considered a small piece of the development process, only belonging to one or two teams in the company with no need for executive attention, improvement, or investment.

Let’s consider what defect management looks like today. Below is a composite generalization of what we see most often in the world of complex products (those which have mechanical, electrical, software and/or firmware components).

PEOPLE (Who?)
Teams	Managing defects often happens solely within individual engineering teams—software teams manage their bugs, hardware teams manage their defects, firmware teams manage their issues—most likely using separate tools, prioritization, terminology, and communication (or lack of) methods.
METHOD (How?)
Disconnected Tools	When defect management is not made a priority, the tools used can be those already available (e.g., MS Office, Google Docs). Being thrifty is good for budgets—if the solution provides the functionality needed. Many defect management tools used today in teams have issues—difficult to access, maintain, and keep current. The tool may not provide easy enforcement of the process, which means supporting data unclear or missing, assignments undocumented or out of date, and/or defects not properly prioritized, reviewed, or resolved.
PROCESS (What?)
Limited Ways to Improve	Defects may not be categorized properly or traceable across product history (revisions) and product lines. Lack of connection to the product record system means a gap in the product history for anyone outside those “in the know” on the product.

Make Defect Management a Priority

Can you risk a product recall? Samsung’s Galaxy Note 7 may prove to be one of the most insightful case studies in overall brand management impact, total costs of the recall, and global customer market reactions for our crazy, connected world. Whether the flaming phones are the result of supply chain oversight issues or internal lab testing challenges remains to be seen, but defect management practices most likely will be at the core of the initial issue, as well as subsequent steps to respond that also misfired.

Poorly handled defect management can result in that kind of nightmare. Recall can cause stock share downturns, loss of revenue, customers, and potential customers, damaged brand reputations, and more. The best result of poorly done defect management is increased costs in a creeping, invisible way that is hard to calculate and a longer new product development (NPD) phase.

Everyone fears product recalls, missed time to market, customer disappointment, and the other possible repercussions of product done poorly. Often, however, we are so busy designing, testing, and producing products to ship that we cannot get ahead to put in place proactive defect management processes and tools to reduce the risks upfront.

2020 FDA Recalls by Class

Consider that 256 product recalls occurred in 2020. These recalls are only consumer-focused products. Consumer Product Safety Commission

When we look at other industries and regulatory bodies, we find no industry is immune to recalls. For example, in 2020, the FDA issued a total of 736 medical device recalls. FDA recalls are categorized into three different classes (i.e., Class I, II, or III) based on the risk for serious health issues or death.

– Class I recall: Use or exposure to device will most likely cause adverse health issues or death

– Class II recall: Use or exposure to device may cause temporary adverse health issues

– Class III recall: Use or exposure to device is not likely to cause adverse health issues

There were 52 Class I, 664 Class II, and 20 Class III recalls. Device design was identified as the root cause for 59% of the Class II recalls.

Defect Management Steps and Strategies

Here we provide four steps to assess where you are today with defect management, identify any gaps, and move closer to the goal of zero defects.

Step 1: Defect Visibility

Do we have visibility into defects discovered across our products and teams (software, hardware, firmware, electrical)?

• Yes, definitely.
• Maybe, I would need to ask people.
• No.

If you cannot answer yes emphatically, you should start here. We cannot manage what we cannot see. NPD success comes from managing the fluctuating variables of cost, quality, features, and time. Without visibility into defects across the product, it is difficult for everyone on the team to know what decisions are being made and how these decisions impact all the other efforts to finish the product.

• Visibility begins with an understood and accepted process for defect management and a standard tool or system for all defects across the products and teams. The solution could be as simple as a shared spreadsheets for smaller companies to an enterprise relational database built with defect manahhttps://www.arenasolutions.com/resources/articles/excel-bill-of-materials/ttps://www.arenasolutions.com/resources/articles/excel-bill-of-materials/gement best practices in mind.
• For Step 1 Visibility, the key is a standard, accessible, single system, or place all teams can access and will use.

Step 2: Defect Prioritization

Do we have a way to do risk analysis and prioritization of defects?

• Absolutely. Everyone knows the process.
• Mostly, but ad hoc and varies from team to team.
• Not really … we treat each decision as its own.

Once we have the first step of visibility, we need to make sure we are tracking the right defect data – the information that matters and can lead to action. Data for data’s sake is noise. Quality of data collected matters for team participation. Collecting too much data or data that does not show value reduces your team’s willingness to complete the defect process, remain engaged, and be accurate.

• To prioritize defects, you need to capture the right information to take action today and prevent defects in the future. Most companies find the fields of information needed include some or all of the following from obvious to not so obvious: correct/complete description, of course, risk level, cost level, phase defect is found, who uncovered defects (team/people) for further learning, classification (what type of defect is it?) to help with causal links, and which revision(s)/release state(s) have the defect.
• For Step 2 Prioritization, the key is capturing the right information at the right time — consider quality, accuracy, ease of update, and purpose of the data.

Step 3: Defect Resolution

Do we have a timely way of acting on defect information?

• Yes, we have a process in place to make fast, informed decisions and we capture those decisions in a system.
• We don’t have one standard process, but we get the work done and product out, even if a bit painful.
• The teams spend too much back and forth, chasing, or missing details; we struggle way more than we should.

With visibility (a process and system) and the right, high-quality data, you have opportunities. Timely, educated decisions can be made faster and earlier in the process. Studies (and common sense) show the costs of fixing later rather than earlier in the process are substantially more—conservatively estimated at 3-5x or more (Soni, citing NIST and IBM research).

• Action requires team collaboration, assignment tracking, and closure. Preferably, your tool (if a software system) from Step 2 ensures the resolution becomes part of the quality data captured, not happening only in email threads and conversations. With a more modern system, you also increase participation with friendly features, including social chat type interfaces, email notifications, and easy filtering to pinpoint the defects that matter.
• For Step 3 Resolution, your process and system should provide fast, team collaborative steps to resolve defects with the capture of discussion and resolutions (you will need the history for Step 4).

Step 4: Defect Analysis

Do we have a defect prevention methodology and culture in place to now drive defects closer to zero?

• Yes, we are Six Sigma ready.
• Work in progress. We sometimes do defect analysis, but we want to make it a priority.
• No, we aren’t here yet — too busy getting individual defect management working well.

Once you have visibility, prioritization, and resolution of individual defects working well, you can consider your defect analysis and review process — a process that considers both inputs about individual defects as well as looks across defects by classifications, product lines, revision history, involved teams, and more. Defect analysis moves teams closer to that nirvana goal of zero defects.

• Defect prevention requires collecting all that critical defect data and corrective action implementation (Steps 1-3). Once collected, you want lessons learned shared within a framework that includes root cause analysis after the resolution, consideration of future possible actions, and modifications in teams, processes, and products to prevent future defects. Reviews – self and peer – can be powerful learning tools and motivators. Having a system that supports the capture of the defect analysis process for sharing and historical traceability is a requirement in driving to zero defects.
• For Step 4 Analysis, you make defect analysis a priority for future product development success.

While it is sometimes hard to make the time to do post-resolution review, it is a discipline found in almost all highly successful, growing companies, so we emphasize it here. Elon Musk has said, “I think it’s very important to have a feedback loop, where you’re constantly thinking about what you’ve done and how you could be doing it better.” You may love or hate Musk of Tesla and SpaceX fame, but difficult to argue with the successes achieved so far in two challenging and still undeveloped areas — alternative energy and space travel. Analysis isn’t the only key to Musk’s success, but it is one of the keys. Don’t let the teams move on to the next project without performing this critical step and capturing the results.

Getting Ahead with Defect Management

Engineering teams divert and reassign idle resources frequently (and not always with the results desired; see Thomke and Reinertsen on myths of resource utilization and queuing theory for more discussions on managing teams). Use some of your team’s idle time to assess your defect management process and drive to that zero defects goal.

Can you risk a product recall? A better question might be: What are the risks of driving defect management as a priority for the company? What do you have to lose in making it a priority? Not much. But you could gain so much — faster NPD cycles, brand integrity, better cost controls, reduced risks, and happier customers.

References

Soni, Mukesh. “Defect Prevention: Reducing Costs and Enhancing Quality.” iSixSigma, March 2015.
Thomke, Stefan and Reinertsen, Donald. “Six Myths of Product Development.” Harvard Business Review, May 2012.