Testing root cause hypotheses and verifying the result

Why it matters

If the overall defect rate falls after a change to a setting, it is tempting to say the setting caused it. In the same period easier products may have been run, new material may have arrived, or the share of the shift with experienced workers may have grown. A before–after comparison shows the change over time; it does not by itself separate the cause of that change. When a root cause hypothesis is tested, it is written clearly which condition was changed and which other conditions could affect the result. The scope of the supported explanation is also stated. The difference between two measurements can show a relationship; for a decision on cause, a comparison arrangement that addresses alternative explanations is needed.

Stratification by product and shift is the separate examination of similar conditions. The numerator and the denominator of each product–shift group are calculated together. The overall rate is the weighted combination of these groups according to their production shares. If the shares change, the total can change even when the results in the groups stay the same. Indeed, the total can worsen while every group improves; the total can improve while every group worsens. This does not have to be an arithmetic error. The direction in which the mix changed must be examined. The total describes the result of actual production; a calculation with a common mix helps compare different periods more fairly. The two indicators answer different questions.

Stratification does not remove all uncertainty. Within the same product and shift, the material, the equipment condition, the measurement definition or worker support may have changed. For this reason the expected effect for the hypothesis is written beforehand and a safe and practicable comparison is chosen. Where possible, the change conditions are balanced across the groups and the order effect is reduced. An old condition that would compromise safety is not returned to merely for a comparison. If there is an effect of improvement over time, learning or maintenance, these are recorded. Whether the change was actually implemented is also verified. If the planned new arrangement was not used, the result questions not only the hypothesis but also the implementation arrangement.

Verification is not seeing a good result once and closing the file. The baseline, the first trial and the later follow-up are tracked with the same measurement definition. When defects decrease, it is also checked whether other losses have increased. A missing denominator prevents the rate calculation; writing zero defects does not remove this deficiency. Seeing no defects at all in a small sample does not mean the true rate is exactly zero. The decision record should show which finding supports the hypothesis, what is still open and which new evidence would change the decision. Evidence-based work also requires being able to state the limits of the result clearly. The figure and the short quiz scenarios in this lesson also fall within the synthetic data of the examples. This dataset was generated synthetically for training; it does not represent a real business, employee or customer study.

Formula