1. Performance Characteristics: How to Read a Datasheet

1.1 Static characteristics

Before you can compare instruments, you need to read their datasheet honestly. The vocabulary breaks into two halves: static characteristics, describing the instrument's behavior on slowly varying inputs, and dynamic characteristics, describing how it handles fast change.

Accuracy is how close a reading is to the true value. Always quoted as a tolerance, never as a single number. A common DMM specification reads "$\pm(0.05\% \text{ of reading} + 0.02\% \text{ of range})$". The first term is proportional to the reading, the second is a fixed floor that becomes dominant when you measure something near zero on a wide range.

Archery analogy. Picture a target. Accuracy is how close, on average, your arrows land to the bullseye. Precision is how tightly your arrows cluster, regardless of where the cluster sits. Hit dead-center every time: accurate and precise. Cluster tightly in the upper-left of the target: precise but inaccurate (you have a systematic offset). Scatter your arrows in a wide ring around the bullseye: accurate (mean is correct) but imprecise (variance is large). Scatter them randomly across the field: neither.

Precision, sometimes called repeatability, is the spread of readings when you measure the same input over and over. You can be precisely wrong: a DMM with a stuck offset will produce repeatable but biased numbers. Statistical bias is removable by calibration; precision is a fundamental noise floor.

Resolution is the smallest input change that produces a detectable output change. For a digital instrument it is one count of the least-significant digit. A "$6\frac{1}{2}$-digit" DMM displays values like 1.999999, so its resolution on the 2 V range is 1 microvolt. For an analog galvanometer, resolution is set by needle thickness and parallax: maybe 1% of full scale.

Sensitivity is the change in output per unit change in input. For a galvanometer voltmeter it is sometimes given in "ohms per volt" because the deflection coil current is fixed at full-scale, so doubling the voltage range doubles the series resistor. A 20 k$\Omega$/V meter on the 10 V range presents 200 k$\Omega$ to the circuit under test. Compare that to a modern DMM at 10 M$\Omega$ on every range and you understand why old VOMs had to be applied with care: they loaded the circuit they were measuring.

Range is the span of inputs the instrument handles. Multi-range instruments switch attenuators or shunts. Linearity is how well a straight line fits the input-output relationship; non-linearity is usually expressed as a percentage of full scale.

Error percent of full scale vs percent of reading. This trips people up. A 0.5% of full-scale meter on its 10 V range can be off by 50 mV anywhere on the range. Reading 1 V? You could be off by 5%. Reading 9 V? Off by 0.55%. Always pick a range where your reading is in the upper third of the scale. Modern DMMs give a hybrid spec: $\pm$(0.05% of reading + 0.02% of range), which is more honest because it explicitly separates the gain error from the offset/quantization error.

1.2 Dynamic characteristics

When the input changes faster than the instrument can follow, three new specs matter.

Speed of response is how quickly the instrument settles to its final reading. A bench DMM in slow integration mode might take 200 ms per reading; a fast DMM in fast mode might do 10,000 readings per second at lower resolution. Speed and resolution trade off: more averaging gives more precision but slower response.

Fidelity is how well the instrument reproduces the shape of the input. A scope with insufficient bandwidth will round off square-wave edges; the instrument is unfaithful to the input.

Lag is the delay between an input change and the instrument's response. A thermocouple has thermal lag (its bead has to physically heat up), so even though its electrical bandwidth might be MHz, its temperature bandwidth is fractions of a hertz.

Dynamic error is the error introduced by these limitations. For a first-order instrument with time constant $\tau$, a step input of magnitude $V$ produces a reading $V(1-e^{-t/\tau})$, so the dynamic error at time $t$ is $V e^{-t/\tau}$. For sinusoidal inputs at frequency $\omega$, the magnitude error is $1/\sqrt{1+\omega^2\tau^2}$ and the phase lag is $\arctan(\omega\tau)$. These shapes will reappear when we discuss the rise time of an oscilloscope and the settling time of a sample-and-hold.

1.3 Errors in measurement: the big three

Every measurement carries error. Identifying the type of error tells you how to fight it.

Gross errors. Human mistakes. Reading 12.5 V as 1.25 V because you misread the range switch. Recording a value into the wrong column. These are not statistical, they are blunders. Defenses: double-check, automate logging, and use units (a lab notebook entry of "V_out = 5" is dangerous; "V_out = 5.012 V $\pm$ 1 mV at 25 °C" is auditable).

Systematic errors. Consistent bias. The DMM is reading 1% high because its internal voltage reference drifted. The thermocouple cold junction is at 22 °C but your software assumed 0 °C. The current shunt has self-heating, so its resistance grows with the current you measure. These errors do not average out; if you measure 1000 times, you get 1000 wrong numbers in the same direction. Defenses: calibration against a traceable reference, periodic verification, and physical understanding (knowing that shunts self-heat, thermocouples need cold-junction compensation, and op-amp inputs have offset voltage).

Random errors. Unpredictable fluctuations. Thermal noise in resistors, shot noise in semiconductors, EMI from a fluorescent light, variations in line voltage. These do average out: $N$ averages reduce random error by $\sqrt{N}$. This is why side-channel attacks like differential power analysis (Chapter 24) average tens of thousands of traces: they are using $\sqrt{N}$ averaging to dig a tiny correlation signal out of a sea of random noise.

1.4 Statistics: mean, median, standard deviation, standard error

Suppose you take $N$ readings $x_1, x_2, \ldots, x_N$. Five numbers summarize the dataset.

Mean: $\bar{x} = \frac{1}{N}\sum x_i$. The arithmetic average. Sensitive to outliers (one bogus reading drags it).

Median: the middle value after sorting. Robust to outliers.

Standard deviation (sample): $\sigma = \sqrt{\frac{1}{N-1}\sum (x_i - \bar{x})^2}$. The "typical" distance from the mean. The $N-1$ in the denominator (Bessel's correction) accounts for the fact that you used the data itself to estimate the mean, so you have one less degree of freedom.

Variance: $\sigma^2$. The standard deviation squared. Adds linearly when uncorrelated noise sources combine, which is why we work with it as much as with $\sigma$.

Standard error of the mean: $\sigma_{\bar{x}} = \sigma/\sqrt{N}$. The standard deviation of the estimate of the mean. As you take more readings, your estimate of the true mean gets tighter as $1/\sqrt{N}$. This is the reason averaging helps but only up to a point: to halve your uncertainty, you must quadruple your reading count.

1.5 Type A and Type B uncertainty (ISO GUM)

Modern metrology distinguishes two flavors of uncertainty, codified by the ISO Guide to the Expression of Uncertainty in Measurement (GUM).

Type A uncertainty is evaluated from the data. You took $N$ readings and computed the standard error of the mean. Statistical, in the strict sense.

Type B uncertainty is evaluated from other information: the manufacturer's accuracy spec, prior calibration data, physical limits, expert judgment. A DMM datasheet might say "$\pm 0.05\%$ of reading"; you can convert that to a standard uncertainty by dividing by $\sqrt{3}$ if you assume a uniform distribution within the spec, or by 2 if you assume a 95%-confidence Gaussian.

Both types are then combined via root-sum-square (assuming independence): $u_c = \sqrt{u_A^2 + u_B^2}$. Multiply by a coverage factor $k$ (typically 2 for ~95% confidence) to report a final expanded uncertainty.

This formalism matters because traceable metrology, calibration certificates, and any audit-quality measurement uses it. When you claim "this voltage was 5.000 V $\pm$ 1 mV", you should be able to defend the 1 mV by listing the contributors: scope offset, ADC quantization, reference drift, temperature coefficient, lead resistance.