Missing Data in DMAIC: Diagnose the Mechanism Before You Impute or Delete

Missing Data in DMAIC: Diagnose the Mechanism Before You Impute or Delete

You open the data set for your Measure phase and find blank torque readings, missing cycle times, and three shifts without operator IDs. It is tempting to fill the gaps quickly and move on to Analyze.

That shortcut can distort the analysis. This article walks through how to investigate why values are absent, how to read a missingness pattern, what MCAR, MAR, and MNAR actually mean for your conclusions, and how to compare deletion against imputation with a sensitivity check that keeps uncertainty visible instead of hidden.

Key Takeaways

  • Map gaps by variable, time, and process condition.
  • Investigate collection failures and recover source records.
  • State the MCAR, MAR, or MNAR assumption.
  • Choose methods for the question and missingness pattern.
  • Test whether plausible gaps change the decision.

Start with the question that most project teams skip.

How Should You Handle Missing Data in a DMAIC Study?

How Should You Handle Missing Data in a DMAIC Study?

Investigate the missingness mechanism and the decision context before choosing deletion, imputation, or recovery. No single rule covers every gap, because the reason a value is absent changes what the remaining records can tell you. A missing pressure reading caused by a sensor fault behaves very differently from one caused by an operator skipping entries during high-volume runs.

So the first move is diagnostic, not computational. You map the pattern, trace the operational cause, state your assumption openly, then test how fragile your answer is.

Here is the sequence this guide follows:

  • Build a missingness map across variables and time.
  • Trace operational causes back to the measurement system.
  • Frame the MCAR, MAR, or MNAR assumption and accept its limits.
  • Compare analysis choices against the decision at stake.
  • Run a sensitivity check and report the spread.

One caution worth repeating. A statistical test will not prove which mechanism applies. The evidence is partly observable, partly judgment, and the judgment part belongs in your project documentation.

Before any assumption gets named, look at where the holes sit.

Reading the Missingness Pattern With a Missingness Map

A missingness map is a simple visual of present and absent values across rows and columns. Each column is a variable, each row an observation, and shaded cells mark gaps. Patterns jump out immediately in a way that a count of nulls never shows.

Look for these structures:

  • Monotone patterns. Once a unit drops out, every later measurement is absent, which often signals scrap, attrition, or an aborted run.
  • Block patterns. Entire date ranges or shifts go blank, pointing to a logging outage or a changed form.
  • Paired gaps. Two variables disappear together because one calculation feeds the other.
  • Scattered gaps. Isolated cells without an obvious pattern; this does not establish MCAR.
  • Boundary clustering. Gaps concentrated at extreme values of a related variable, a reason to investigate how values are recorded.

For example, if discharge timestamps are absent mainly on weekend overnight shifts, investigate staffing and recording procedures before analyzing wait times.

There is no universal missing percentage threshold for choosing an analysis method. A small gap concentrated in a critical subgroup may matter more than a larger gap spread across the sample.

Maps show where. The next step asks why.

Tracing Operational Causes Behind Missing Data in DMAIC Projects

Tracing Operational Causes Behind Missing Data in DMAIC Projects

Gaps may reveal a problem in the measurement process. Check logs and source records before treating absent fields as lost; the Measure phase data cleaning guide provides a related workflow.

1. Instrument and Sensor Faults

Calibration drift, power interruptions, and saturated sensors drop readings. Check maintenance logs for the affected period and ask whether the instrument retained any readings.

2. Manual Entry Workload

Operators may skip fields when throughput spikes. The absence then correlates with production rate, which is exactly the kind of dependency that breaks naive analysis.

3. Detection Limits

Values below an instrument floor get recorded as blank rather than as censored. This is a different problem from true missingness and deserves censored-data methods.

4. System Migrations and Form Changes

A field renamed during an upgrade creates a clean block of nulls. Check archived tables and exports for the original records.

5. Definitional Mismatch

Two plants record the same metric differently, so merged data shows gaps that never existed locally. Review the Measure phase operational definitions and reconcile the source fields before merging.

6. Deliberate Non-Reporting

Rework or out-of-spec events sometimes go unlogged. This is the hardest cause to confirm and the most damaging to leave unexamined.

Recover original measurements when feasible. Ask data owners whether instrument memory, logs, or earlier exports contain the values, then document which records were recovered and how.

Once causes are traced, name the assumption you are relying on.

MCAR MAR MNAR: Naming the Assumption and Its Limits

These three labels describe the relationship between missingness and the data values themselves. They drive which methods stay valid. Getting them wrong quietly biases everything downstream.

  • MCAR, missing completely at random. Missingness is unrelated to observed and unobserved values. Complete case analysis can be unbiased for many estimates, but loses precision.
  • MAR, missing at random. After conditioning on observed variables such as shift or machine, missingness no longer depends on the unobserved value. A suitable imputation or likelihood model may support valid inference.
  • MNAR, missing not at random. Missingness still depends on the unobserved value after accounting for observed information. If out-of-spec readings are less likely to be logged, an analysis assuming MAR may be biased.

Observed data alone generally cannot establish MAR over MNAR. Research on when multiple imputation helps shows that complete case analysis can sometimes be valid beyond MCAR, depending on the target estimate and missingness process; a missing data statistical primer explains the assumptions behind common methods.

Comparing observed characteristics between complete and incomplete records helps assess plausibility, but it cannot prove the mechanism. State the assumption, support it with process knowledge, and document it.

Teams that want structured practice in this reasoning often work through it in our Lean Six Sigma training tracks, where data assumptions get examined alongside the tools that depend on them.

With the assumption named, weigh your options against the decision.

Comparing Analysis Choices for Missing Data in DMAIC Work

Comparing Analysis Choices for Missing Data in DMAIC Work

Each approach trades something. Match the trade to what the decision can tolerate.

Approach Assumption Required Main Risk Reasonable Use
Recover original records None Time and access effort Always attempt first
Complete case analysis Depends on target estimate Lost precision; possible selection bias When validity is justified for the analysis
Mean or median substitution Rarely defensible Understates variability and uncertainty Not advised for inference
Regression imputation Model dependent Understates uncertainty if treated as observed Exploratory displays, with limitations stated
Multiple imputation MAR Model misspecification Structured gaps, adequate predictors
Likelihood or mixed models MAR Modeling complexity Repeated measures over time
Pattern-mixture sensitivity MNAR scenarios Requires explicit guesses Suspected non-random absence

Filling missing measurements with the sample mean can compress estimated spread and understate uncertainty. Because process capability indices use variation, a calculation on such filled data may look better without process improvement.

A missing data reporting framework recommends explaining how missingness was handled and reporting sensitivity analyses where assumptions matter.

No method removes doubt. So show the doubt instead.

A Sensitivity Check That Makes Missing Data Uncertainty Visible

Sensitivity analysis answers one question. Does the conclusion hold across plausible assumptions about the missing values?

  1. Fix the primary analysis. State your working assumption and produce the main estimate under it.
  2. Run complete case analysis as a contrast. Note how the estimate and interval shift when incomplete records drop out.
  3. Apply multiple imputation under MAR. Generate several completed data sets so the uncertainty from imputation appears in the standard errors.
  4. Add adverse MNAR scenarios. Shift imputed values by a realistic offset, such as assuming missing readings ran worse than observed ones.
  5. Find the tipping point. Identify how large that offset must be before your decision reverses.
  6. Report the range in the tollgate. Present estimates side by side rather than a single number with false precision.

If the result holds across plausible scenarios, report that robustness alongside its limitations. If it flips under a modest shift, the honest answer is to collect better data before committing capital. Show leadership where additional collection would reduce uncertainty.

AirAcad Resources for Making Defensible Missing-Data Decisions

AirAcad Resources for Making Defensible Missing-Data Decisions

Missing-data judgment calls sit at the intersection of process knowledge and analytical method, so the right training depends on the question at hand. These four options from Air Academy Associates connect directly to the diagnostic sequence described above.

Historical Data Analysis Short Course

DMAIC studies may use records originally collected for operational purposes. This short course covers working with existing data and understanding what those records can and cannot support. Apply it when you need to judge whether an archived data set has enough coverage to answer your question, or whether the evidence limitations push you toward fresh data collection instead of heavier statistical patching.

Big Data and Predictive Analytics Short Course

Incomplete inputs can change the data used for models and dashboards. This course introduces analytics methods applied to larger data sets. Use it to think through how missing fields affect model behavior, which variables carry predictive weight, and why a gap in one upstream field can distort outputs that leaders rely on for resource decisions.

Hypothesis Testing Short Course

Comparison tests depend on the quality and suitability of their input data. This short course covers the logic of inferential testing and the conditions attached to common tests. Apply the skill when deciding whether a before-and-after comparison remains valid after records drop out, and when explaining to stakeholders why a p-value inherits the weaknesses of the data feeding it.

One-to-One Training and Coaching

Some missingness problems resist general rules and need a second opinion on a live project. Coaching pairs you with an experienced instructor for project-specific discussion. Bring your missingness map, your proposed approach, and your sensitivity scenarios, then work through whether the analysis choice holds up before you present findings at a tollgate review.

You can also review the available online short courses or the current training schedule if a broader program fits better.

Closing Thought

Gaps may reveal how the data were collected. Diagnose the cause, state the working assumption, and report how conclusions change under plausible alternatives. For project-specific support, discuss the findings through our professional services team or contact us about training.

Air Academy Associates offers Lean Six Sigma training and coaching for teams working through measurement and analysis decisions. Explore the courses or contact the team to discuss your project.

FAQs

How Do MCAR, MAR, and MNAR Differ?

MCAR means missingness is unrelated to observed and unobserved values. Under MAR, it is independent of the missing value after conditioning on observed information; under MNAR, dependence remains after that conditioning.

Stratify gaps by time, shift, process step, and relevant CTQs, then use process knowledge to assess the assumption. Observed records alone generally cannot prove MAR rather than MNAR.

When Can Deleting Incomplete Records Bias DMAIC Findings?

Deleting incomplete records can bias a DMAIC estimate when selection into the complete cases depends on variables relevant to that estimate. MCAR is sufficient for many analyses, but complete case validity has exceptions even when data are not MCAR.

Check whether gaps cluster by shift, line, supplier lot, or outcome before excluding records. Explain the effect on baseline and root-cause findings.

What Sensitivity Checks Should Accompany Missing-Data Treatment?

Compare the primary estimate with complete case and suitable model-based analyses, documenting each assumption. Check gaps across time, shift, site, product, and operator; for plausible MNAR concerns, vary missing values within defensible scenarios.

Report when the decision changes, including the shift needed to reverse it. The DMAIC tollgate evidence checklist can help structure that review.

Related Articles

Posted by
Air Academy Associates
Air Academy Associates is a leader in Six Sigma training and certification. Since the beginning of Six Sigma, we’ve played a role and trained the first Black Belts from Motorola. Our proven and powerful curriculum uses a “Keep It Simple Statistically” (KISS) approach. KISS means more power, not less. We develop Lean Six Sigma methodology practitioners who can use the tools and techniques to drive improvement and rapidly deliver business results.

How can we help you?

Name

— or Call us at —

1-800-748-1277

contact us for group pricing