3. Sampling: From Analog to Digital // Signals and Systems // bhaswanth

This is the bridge between the continuous-time world (where audio, voltages, and physical phenomena live) and the discrete-time world (where computers and DSP algorithms operate). Almost every signal you'll process in the rest of this curriculum is discrete, having gone through this bridge.

3.1 The mechanics of sampling

To digitize a continuous-time signal $x(t)$ , we sample it at regular intervals $T_s$ :

$x[n] = x(n T_s)$

for integer $n$ . The sampling rate is $f_s = 1/T_s$ Hz; the corresponding sampling angular frequency is $\omega_s = 2\pi/T_s$ .

Modeling sampling mathematically: multiply $x(t)$ by a train of impulses spaced $T_s$ apart:

$x_s(t) = x(t) \cdot \sum_n \delta(t - nT_s)$

The sampled signal is now a sequence of weighted impulses at the sampling instants. The impulses' weights are exactly the values $x(nT_s)$ .

What does the spectrum of $x_s(t)$ look like? The spectrum of the impulse train (the train alone, no $x$ ) is itself an impulse train in frequency, spaced $\omega_s$ apart. So when we multiply $x$ by the time-domain impulse train, we convolve the spectrum of $x$ with the spectrum of the impulse train (multiplication in time becomes convolution in frequency, from Section 2.6). Convolving a spectrum $X(j\omega)$ with a frequency-domain impulse train replicates $X$ at every multiple of $\omega_s$ :

plaintext

   X(jω)   (original spectrum, before sampling, bandlimited to ±W)
    ╱╲
___╱  ╲____________________________________________ ω
  -W  W
 
   Xs(jω)  (after sampling, spectrum is replicated at multiples of ωs)
    ╱╲    ╱╲    ╱╲    ╱╲    ╱╲    ╱╲
___╱  ╲__╱  ╲__╱  ╲__╱  ╲__╱  ╲__╱  ╲__ ω
  -W  W
       ↑    ↑
      ωs  2ωs   ...

This single picture is the entire reason the sampling theorem works. Each replica is a copy of the original spectrum, shifted by some multiple of the sampling rate. As long as the replicas don't overlap, the original spectrum can be picked out by a low-pass filter and the original signal recovered.

The replicas don't overlap when the original signal's bandwidth $W$ is less than half the sampling rate. In other words:

$\omega_s > 2W \quad\Leftrightarrow\quad f_s > 2 f_{\max}$

That's the Nyquist criterion, derived from a picture rather than memorized.

3.2 The Nyquist-Shannon sampling theorem

A bandlimited signal containing no frequencies above $f_{\max}$ can be perfectly reconstructed from samples taken at any rate $f_s > 2 f_{\max}$ .

The minimum required rate $2 f_{\max}$ is called the Nyquist rate. Sample slower than this and information is lost, irrecoverably.

Picnic-table analogy. You are trying to capture the shape of an ocean wave by photographing it at intervals. If the wave moves slowly compared to your photo rate, you capture the shape easily: sample, sample, sample, you see the wave. If the wave moves fast (too fast for your photo rate) different photos catch the wave at different points in different cycles, and you cannot tell what the original wave was doing. Many waves of different frequencies could produce the same set of snapshots. Sample at least twice per cycle (once near the crest and once near the trough) and you can tell unambiguously what the wave was. That is Nyquist.

3.3 Aliasing: what happens if you sample too slowly

If you sample at a rate below $2 f_{\max}$ , frequencies above $f_s/2$ "fold back" and appear as lower frequencies in the sampled signal. This is aliasing: high-frequency content is misidentified as low-frequency content, and there is no way to tell the difference after the fact.

In the picture from 3.1, the replicas overlap. The piece of the high-frequency replica that overlaps with the original now appears to be a low-frequency component of the same signal. They've become indistinguishable.

Visual examples of aliasing in everyday life:

Wagon wheels appearing to spin backwards in old movies. The wheel's spokes pass too quickly for the camera's frame rate, so each frame catches each spoke at a position slightly back from where the eye expects. The wheel looks like it's spinning backwards.
Helicopter rotor blades freezing in place on phone video. The rotor's frequency happens to be a multiple of the camera's frame rate, and each frame catches a blade in the same position.
Moiré patterns on patterned shirts on TV. Fine stripes alias against the camera's pixel grid, producing weird wavy patterns.
CD audio's 22 kHz cutoff. CDs sample at 44.1 kHz to avoid aliasing of audio above 22 kHz. Ultrasound is filtered out before sampling (anti-aliasing filter), because once sampled, you cannot distinguish 25 kHz audio from 19 kHz audio.

A short Python demo makes this visceral:

python

import numpy as np
import matplotlib.pyplot as plt
 
# A 9 Hz sinusoid sampled at 10 Hz (way below Nyquist)
t_dense = np.linspace(0, 1, 1000)
x_dense = np.sin(2*np.pi*9*t_dense)
t_sampled = np.arange(0, 1, 1/10)
x_sampled = np.sin(2*np.pi*9*t_sampled)
 
plt.plot(t_dense, x_dense, label='true 9 Hz')
plt.stem(t_sampled, x_sampled, label='samples at 10 Hz')
plt.legend()
plt.show()

Connect the samples and you see a slow 1 Hz wave. The 9 Hz signal aliased to a 1 Hz signal, and after sampling, you have no way to recover the 9 Hz original.

3.4 The anti-aliasing filter: required, no exceptions

Always filter out frequencies above $f_s/2$ before sampling. The filter is called an anti-aliasing filter. Without it, every bit of out-of-band noise (and there is always some) gets aliased into your signal band, where you cannot remove it later.

In practice, every audio ADC has an anti-aliasing low-pass filter built in. When you record at 44.1 kHz with your phone, an analog filter (or, in modern sigma-delta ADCs, a digital filter operating at much higher rate before downsampling) blocks anything above ~20 kHz before quantization.

For hardware-security power-trace acquisition, anti-aliasing is just as important. If you sample at 200 MS/s but the chip has activity at 1 GHz from clock harmonics, those harmonics alias into your 0–100 MHz band as random-looking noise unless filtered out first. A bench-top low-pass filter ahead of the digitizer is a routine addition to the side-channel rig.

3.5 Reconstruction: the dual problem

Going the other way (DAC, digital to analog) reverses the process. Each digital sample $x[n]$ becomes an analog impulse at time $nT_s$ with weight $x[n]$ . The result is a series of impulses at the sample rate, then low-pass filtered to remove the high-frequency images and recover the original analog signal.

The mathematically ideal reconstruction filter is a sinc:

$x(t) = \sum_n x[n]\,\text{sinc}\!\left(\frac{t - n T_s}{T_s}\right)$

The sinc is the perfect rect-shaped low-pass filter (it cleanly keeps the original spectrum and rejects all the replicas). But sinc has infinite length, so practical DACs use simpler "zero-order hold" reconstruction (each sample is held flat until the next one arrives, producing a stair-step output), followed by an analog low-pass filter to smooth the steps. Modern DACs often oversample, internally interpolating to a much higher rate before the analog reconstruction, which simplifies the analog filter requirements and pushes residual images far above audible range.

3.6 Practical sampling rates

Speech: 8 kHz is the telephony standard, capturing voice up to 3.4 kHz (with Nyquist margin). Sounds tinny but intelligible.
CD audio: 44.1 kHz, capturing up to 22 kHz (the limit of human hearing).
Studio audio: 48 kHz, 96 kHz, or 192 kHz, capturing higher frequencies than humans can hear, with the rationale that subsequent processing (filtering, time-stretching) is cleaner with extra headroom.
Video: typically 30 or 60 frames per second; not a frequency-domain audio sampling but a spatial and temporal sampling.
Oscilloscope sampling rates: 1 GS/s, 100 GS/s, or 1 TS/s in cutting-edge instruments, capturing nanosecond glitches.
Software-defined radio (SDR): typically 2 MS/s to 60 MS/s of complex (I + Q) baseband, for handling modern radio bandwidths.
Side-channel power capture: typically 200 MS/s to 5 GS/s for chip-level work; you want at least a few samples per chip clock period to see individual cycle activity.