2. Fourier: The Language of Frequency

Possibly the single most important idea in signal processing: any reasonable periodic signal can be decomposed into a sum of sinusoids at integer multiples of the fundamental frequency. Joseph Fourier proved this in 1807, working on heat conduction. It changed everything. We'll build to it slowly.

2.1 Why we even want to decompose into sinusoids

Pause for a moment. Why bother breaking a signal into sinusoids at all? Sinusoids seem arbitrary. Why not square waves, or triangles, or some other building block?

The reason is deep, and it is the single fact you must believe in your bones if you want signals-and-systems to feel inevitable rather than arbitrary:

Sinusoids are the eigenfunctions of every LTI system. That is, if you push a sinusoid into any linear time-invariant system (any RC filter, any amplifier in its small-signal regime, any room responding to sound), the output is the same sinusoid, possibly scaled and phase-shifted, but still a sinusoid at the same frequency. No new frequencies are created. Push a 1 kHz sine into an RC low-pass and you get a 1 kHz sine out. Smaller, maybe phase-shifted, but still 1 kHz.

This is not true of square waves, ramps, or any other shape. A square wave through an RC filter comes out as something curvy that is no longer a square wave. A sinusoid through an RC filter comes out as a sinusoid.

So if we can write a complicated signal as a sum of sinusoids, we can analyze each one separately through the system, then sum the results. The math works out beautifully because LTI systems preserve the sinusoidal "atoms."

Smoothie analogy. You have a fruit smoothie. It is hard to predict, looking at the smoothie, what individual flavors are in it. But if you could decompose the smoothie into its component fruits (banana, strawberry, mango), then analyzing the smoothie's flavor becomes "what fraction of each fruit is in there?" Fourier does this for signals: he decomposes them into sinusoids ("fruit"), and the recipe is the Fourier series or Fourier transform. The bonus is that LTI systems, the focus of this chapter, treat each "fruit" independently, so the analysis becomes one-fruit-at-a-time instead of all-mixed-up.

2.2 Orthogonality, the trick that makes Fourier coefficients computable

To find the recipe, we need a way to say: "given a signal, how much of $\cos(n\omega_0 t)$ is in it?" The trick is orthogonality.

Two functions $f$ and $g$ are orthogonal on the interval $[a, b]$ if their inner product (in the integral sense) is zero:

$\int_a^b f(t)\, g(t)\, dt = 0$

The astonishing fact: sinusoids at different integer multiples of a common frequency are orthogonal over one period. Specifically, for $m \neq n$:

$\int_0^{T_0} \cos(m\omega_0 t)\,\cos(n\omega_0 t)\, dt = 0$

(and similarly for sin-sin, sin-cos, etc.). And for $m = n$:

$\int_0^{T_0} \cos^2(n\omega_0 t)\, dt = T_0/2$

This is just trigonometry plus integration; expand $\cos A \cos B = \frac{1}{2}[\cos(A-B) + \cos(A+B)]$ and integrate. The first term gives $\cos((m-n)\omega_0 t)$, which integrates to zero over a period if $m \neq n$. The second term gives $\cos((m+n)\omega_0 t)$, which also integrates to zero. So the integral is zero whenever $m \neq n$.

Why does this matter? Suppose we want to know how much of $\cos(n\omega_0 t)$ is in $x(t)$. If we believe $x(t) = \sum_k a_k \cos(k\omega_0 t)$, multiply both sides by $\cos(n\omega_0 t)$ and integrate over a period. By orthogonality, all terms except $k = n$ vanish, leaving:

$\int_0^{T_0} x(t)\cos(n\omega_0 t)\, dt = a_n \cdot T_0/2$

So $a_n = (2/T_0) \int_0^{T_0} x(t)\cos(n\omega_0 t)\, dt$. The Fourier coefficient is computed by projecting the signal onto the corresponding basis sinusoid, just like extracting a vector's $x$-component by taking its dot product with $\hat{x}$.

The whole formal apparatus of Fourier is built on this single fact: sinusoids at different harmonic frequencies are orthogonal. Once you see that, the formulas for the coefficients stop looking arbitrary.

2.3 Fourier series for periodic signals

A periodic signal $x(t)$ with period $T_0$ (fundamental frequency $\omega_0 = 2\pi/T_0$, fundamental frequency in Hz $f_0 = 1/T_0$) can be written:

Trigonometric form:

$x(t) = a_0 + \sum_{n=1}^{\infty}\left[a_n \cos(n\omega_0 t) + b_n \sin(n\omega_0 t)\right]$

where the $a_n$ and $b_n$ are the Fourier coefficients, computed (as we just derived) by integrating $x$ against the corresponding sinusoid:

$a_0 = \frac{1}{T_0}\int_{T_0} x(t)\, dt \quad\text{(the DC component, the time average of }x\text{)}$ $a_n = \frac{2}{T_0}\int_{T_0} x(t)\cos(n\omega_0 t)\, dt$ $b_n = \frac{2}{T_0}\int_{T_0} x(t)\sin(n\omega_0 t)\, dt$

(The integration range $T_0$ means "any one full period.")

Exponential form (what engineers actually use, more compact, derived using $\cos\theta = (e^{j\theta} + e^{-j\theta})/2$):

$x(t) = \sum_{n=-\infty}^{\infty} c_n\, e^{jn\omega_0 t}, \qquad c_n = \frac{1}{T_0}\int_{T_0} x(t)\, e^{-jn\omega_0 t}\, dt$

The $c_n$ are complex numbers encoding both amplitude and phase of each frequency component. Plotting $|c_n|$ vs $n\omega_0$ gives the magnitude spectrum; plotting $\angle c_n$ gives the phase spectrum. This is exactly what spectrum analyzers display.

Worked example: square wave. A symmetric square wave with period $T_0$, swinging between $+1$ and $-1$, has the Fourier series $x(t) = \frac{4}{\pi}\left[\sin(\omega_0 t) + \frac{1}{3}\sin(3\omega_0 t) + \frac{1}{5}\sin(5\omega_0 t) + \cdots\right]$ All odd harmonics, with amplitudes that fall as $1/n$. There's no DC component because the wave is symmetric around zero, no even harmonics because of the wave's symmetry, and no cosine terms because the wave is odd around $t = 0$ (when chosen that way). This is the archetypal series; you should recognize it on sight.

If you sum just the first term (the fundamental), you get a sinusoid that vaguely follows the square wave. Add the third harmonic at $1/3$ amplitude, you get a stair-stepped wiggly thing that is slightly more square-shaped. By the 50th harmonic, you have a sharp-cornered square wave with tiny wiggles near the discontinuities (the Gibbs phenomenon, an artifact of trying to represent a discontinuity with a finite sum of smooth sinusoids).

2.4 The Fourier transform: extending to non-periodic signals

What if your signal is not periodic? A handwritten Morse-code "S," a single voice utterance, a single radar pulse: none of these repeat. We extend the Fourier idea by letting $T_0 \to \infty$ (so the period is so long the signal effectively never repeats). The discrete sum of harmonics becomes a continuous integral, and we get the Fourier transform:

$X(j\omega) = \int_{-\infty}^{\infty} x(t)\, e^{-j\omega t}\, dt$

The inverse transform recovers the time signal:

$x(t) = \frac{1}{2\pi}\int_{-\infty}^{\infty} X(j\omega)\, e^{j\omega t}\, d\omega$

$X(j\omega)$ is the frequency-domain representation of $x(t)$. It tells you "how much of frequency $\omega$ is contained in $x(t)$." Magnitude and phase: $|X(j\omega)|$ is the magnitude spectrum, $\angle X(j\omega)$ is the phase spectrum.

The two formulas (time→frequency and frequency→time) are striking in their symmetry. The forward transform integrates against $e^{-j\omega t}$; the inverse against $e^{+j\omega t}$, with a factor of $2\pi$. Two essentially identical operations. Time and frequency are duals.

2.5 The transforms of standard signals

Look at the first row. The Fourier transform of an impulse is a flat spectrum, every frequency equally present. This is not a coincidence; an impulse has zero duration, so by the time-frequency duality (which we'll formalize in a moment) it occupies infinite bandwidth. Hit a chime once: you hear all the chime's resonant frequencies because the impulse contains all frequencies and the chime selects the ones it likes.

The last two rows are spectacular. The rectangular pulse in time has a sinc in frequency. The sinc in time has a rect in frequency. This duality is one of the central facts of signal processing. It means: any signal that is bounded in time has tails extending to infinity in frequency, and vice versa. You cannot have a signal that is finite in both time and frequency simultaneously (except the zero signal). That is the uncertainty principle for signals, mathematically the same uncertainty that limits how well we can know a particle's position and momentum simultaneously.

Real-world manifestation. If you transmit a 1 microsecond pulse, its frequency content extends out to roughly $1/(1\,\mu\text{s}) = 1$ MHz and beyond. So pulse radar systems and digital communications occupy bandwidth proportional to inverse pulse width, a fundamental physics-imposed limit. Your high-speed USB cable carries 1 ns rise-time edges, which is why it must support GHz of bandwidth; you can't have a fast edge in a narrow band.

2.6 Properties of the Fourier transform

These come up so often you should memorize them, but more importantly you should understand why each one is true.

• Linearity: $a x_1 + b x_2 \leftrightarrow a X_1 + b X_2$. The transform is a linear operation; integration is linear.
• Time shift: $x(t - t_0) \leftrightarrow X(j\omega)\, e^{-j\omega t_0}$. A delay in time becomes a linear phase rotation in frequency. Magnitudes don't change at all (you can't tell from the magnitude spectrum whether a signal has been delayed). Only the phase tells you.
• Frequency shift / modulation: $x(t)\,e^{j\omega_c t} \leftrightarrow X(j(\omega - \omega_c))$. Multiplying by a complex exponential shifts the spectrum. This is how AM/FM/SSB radio works: multiplying a baseband signal by $\cos(\omega_c t)$ shifts its spectrum to lie around $\pm\omega_c$.
• Time scaling: $x(at) \leftrightarrow (1/|a|)\, X(j\omega/a)$. Compress in time, stretch in frequency. Stretch in time, compress in frequency. This is the time-frequency duality made quantitative.
• Convolution becomes multiplication: $x_1(t) * x_2(t) \leftrightarrow X_1(j\omega) \cdot X_2(j\omega)$. The most useful property in all of signal processing. It is the entire reason we work in the frequency domain at all.
• Multiplication becomes convolution: $x_1(t) \cdot x_2(t) \leftrightarrow (1/2\pi)\, X_1(j\omega) * X_2(j\omega)$. Multiplication in time is convolution in frequency, the dual of the previous property.
• Parseval's theorem: $\int |x(t)|^2\, dt = (1/2\pi)\int |X(j\omega)|^2\, d\omega$. Energy in time equals energy in frequency. Whatever total energy your signal carries, the time and frequency views agree on it.

The convolution property deserves a moment. We'll see in Section 4 that LTI systems output a convolution of input with impulse response. So if we Fourier-transform everything, that ugly time-domain convolution becomes a simple multiplication in frequency. That's the whole reason we go to the frequency domain: the operation we cared about (convolution) becomes the operation we know how to do quickly (multiplication, frequency by frequency).

2.7 What the Fourier transform looks like for a real signal

Plot a piece of an audio recording (say, a recorded note from a piano) and you see a complicated waveform that looks more or less random. Take its Fourier transform and you see a comb: a peak at the fundamental frequency of the note (say 440 Hz for A above middle C), then peaks at 880 Hz, 1320 Hz, 1760 Hz, ... the harmonics. The relative heights of these peaks (the "timbre") are what makes a piano sound different from a violin playing the same note.

A clarinet, by contrast, produces mainly odd harmonics (440, 1320, 2200 Hz), because of how the air column inside resonates (it's a tube closed at one end, which suppresses even modes). Different timbre, same note. The Fourier transform makes timbre quantitative.

Try this in a few lines of Python:

import numpy as np
import matplotlib.pyplot as plt
 
fs = 44100                  # 44.1 kHz sample rate (CD quality)
T = 1.0                     # 1 second
t = np.arange(int(fs*T)) / fs
# Sum of fundamental + 3 harmonics with falling amplitudes
x = (np.sin(2*np.pi*440*t)
     + 0.5*np.sin(2*np.pi*880*t)
     + 0.25*np.sin(2*np.pi*1320*t)
     + 0.125*np.sin(2*np.pi*1760*t))
 
X = np.fft.rfft(x)          # real-input FFT
freqs = np.fft.rfftfreq(len(x), 1/fs)
plt.plot(freqs[:5000], np.abs(X[:5000]))
plt.xlabel("Frequency (Hz)")
plt.ylabel("Magnitude")
plt.show()

You'll see four sharp peaks at 440, 880, 1320, 1760 Hz, with heights in the ratios you put in. Change the amplitudes; the peaks change. Add a fifth harmonic; another peak appears. This is the easiest way to feel the Fourier transform.

2.8 The Hilbert transform: a sibling

Closely related to the Fourier transform: the Hilbert transform shifts every frequency component by exactly $-90°$ (negative for positive frequencies, positive for negative ones). It is what you use to construct an analytic signal, a complex-valued signal whose real part is the original and whose imaginary part is its Hilbert transform. Useful for:

• Single-sideband (SSB) modulation (Chapter 7): the phasing method exploits Hilbert.
• Envelope detection of complex signals: the magnitude of the analytic signal is the instantaneous envelope. Used in AM detectors and demodulators.
• Instantaneous frequency detection in FM demodulators.

It's a deep idea but rarely the first transform you reach for; mention it now, return when needed.