Measurement Resolution, Bias and Linearity

Why it matters

A measurement result does not carry only the property of the part. The instrument display, the way of contact, the clamping arrangement, the environment and the method of the person doing the measuring also affect it. Measurement system analysis separates these effects and tests the basis of the decision. The first question here is how many digits the instrument shows; the second is where it places a known reference on average. The answers to these questions cannot replace each other. Even if the display can show small changes, it may shift all results in the same direction. This distinction must be made before reading a small difference between two production measurements as an improvement. Otherwise an adjustment may be made for a change that does not exist in the process, or a real change may go unnoticed.

Resolution concerns the smallest increment the display can distinguish. Repeatability is the scatter seen when the same object is measured again under the same conditions. Bias is the signed difference between the measurement mean and the accepted reference value. A positive bias shows that the system reads high, a negative bias that it reads low. The reference itself is not a perfect number with zero uncertainty. The conditions under which the reference value is valid and the size of its uncertainty must be known. The calculations in this lesson take the given reference value as the basis; when the uncertainty components are missing, no claim is made that a full measurement uncertainty calculation has been done. The aim is to make the difference visible and to choose the next investigation correctly.

A linearity study looks at the bias at different points of the measurement range. The question here is not whether the measured values increase; it is whether the bias stays the same while the true size increases. A good result in the middle of the range does not confirm the lower and upper ends. Conversely, a similar bias at all levels may suggest a common correction; but repeats, references and conditions of use must be verified before applying it. Subtracting the difference at one point from all measurements treats unknown behavior as if it were known. Especially in measurements that use different tips, different clamping forces or different temperatures, the same instrument name may not mean the same measurement system. The study covers these conditions as well.

Diagram: [Display step] → [signed biases from the references] → [change of bias across the range] → [verification with new measurements]. Each arrow shows a separate evidence requirement; the number of digits does not take the place of the last box. The decision order of this lesson is resolution → difference from the reference → behavior across the range → limit of use. Read the arrows as a decision diagram: moving on to the next box does not remove a lack of evidence in the previous one. More repeats can improve the estimate of the mean difference; they do not correct an insufficient display increment by themselves. Better resolution does not remove unstable clamping either. For this reason instrument selection, calibration information and the practice at the workplace are evaluated together. The result is not a general good-instrument label but a measurement setup that can be used for a defined size and defined conditions. A limit defined this way also forms the starting condition of the next crossed study. All numerical scenarios in this lesson's short quiz are also covered by the following notice: This dataset was generated synthetically for training; it does not represent a real business, employee or customer study.

Formula