4. Linear Time-Invariant Systems

A system transforms an input signal into an output. We write $y = T\{x\}$. The most important class of systems we will study are LTI (linear and time-invariant). Almost every passive electronic circuit, every small-signal amplifier, every digital filter is LTI to a very good approximation.

4.1 What "linear" and "time-invariant" mean

Linear: if you double the input, the output doubles. If you add two inputs, the outputs add. Mathematically:

$T\{a x_1(t) + b x_2(t)\} = a\, T\{x_1(t)\} + b\, T\{x_2(t)\}$

for any constants $a, b$ and any inputs $x_1, x_2$. This is the superposition principle (Chapter 2).

Time-invariant: if you delay the input, the output gets delayed by the same amount, with no other change. $T\{x(t - t_0)\} = y(t - t_0)$. The system has no clock, no time-varying parameters; whether you push your input at noon or at midnight, the response is the same shape, just delayed.

A system that is both linear and time-invariant is LTI. Most of the circuits we have studied so far are LTI to a good approximation. Resistors, capacitors, inductors are LTI. Transistors are not (the diode equation has an exponential, the MOSFET has a square law). But around a Q-point, with small signals, transistor circuits behave linearly, which is exactly what the small-signal models from Chapter 1 captured.

4.2 The impulse response, and the derivation of convolution

For an LTI system, the impulse response $h(t)$ is the output when the input is a unit impulse $\delta(t)$. The impulse response completely characterizes the system: knowing $h(t)$ tells you the output for any input.

Why is that true? Here's the reasoning chain. Pay attention; this is one of those derivations that, once you see it, makes the whole machinery feel inevitable.

Step 1. Start with the sifting property: any signal $x(t)$ can be written as a sum (integral) of weighted, shifted impulses:

$x(t) = \int_{-\infty}^{\infty} x(\tau)\, \delta(t - \tau)\, d\tau$

This is just the sifting property running in reverse: $x(t)$ is the sum, over all $\tau$, of impulses at time $\tau$ with weight $x(\tau)$.

Step 2. Push this through the LTI system. By linearity, the output is the sum of the outputs to each weighted, shifted impulse:

$y(t) = T\{x(t)\} = T\!\left\{\int x(\tau)\,\delta(t - \tau)\, d\tau\right\} = \int x(\tau)\, T\{\delta(t - \tau)\}\, d\tau$

(We pulled $x(\tau)$ out of $T$ because it's a constant from the system's point of view, and we pulled the integral out because $T$ is linear.)

Step 3. By time-invariance, $T\{\delta(t - \tau)\} = h(t - \tau)$: the response to a delayed impulse is the impulse response delayed by the same amount.

Step 4. Putting it together:

$y(t) = \int_{-\infty}^{\infty} x(\tau)\, h(t - \tau)\, d\tau = (x * h)(t)$

This integral is the convolution of $x$ with $h$. We've derived it from first principles: linearity decomposes the input into impulses, and time-invariance makes the response to each impulse just a shifted copy of $h$. The output is the integral of all those weighted, shifted copies.

Drum-strike analogy. You walk into an empty hall and slap your hands once: a sharp impulse of pressure. The hall's reverb is its impulse response. Different halls have different impulse responses (long reverb tail in a cathedral, short and dry in a closet). Now play music in the hall, any music. The sound you hear is the music convolved with the hall's impulse response. The music is the input; the hall is the LTI system; the heard sound is the output. By measuring the impulse response of the hall (slap your hands once and record), you can predict how any input will sound: just convolve. Audio plug-ins called "convolution reverbs" do exactly this with sampled impulse responses of famous halls.

4.3 Visualizing convolution

The convolution integral can feel abstract until you've drawn one out. Here's the recipe.

For each output time $t$:

1. Take $h(\tau)$ and flip it to get $h(-\tau)$.
2. Shift it by $t$ to get $h(t - \tau)$, a copy of the impulse response, flipped, sliding to the right as $t$ increases.
3. Multiply this sliding flipped-shifted $h$ against $x(\tau)$.
4. Integrate over all $\tau$. That integral is $y(t)$.

It's "flip, shift, multiply, integrate" for every output time. The output at time $t$ measures how much of $h$ overlaps with $x$ when $h$ is positioned (flipped) at time $t$.

For discrete signals the formula is the same with sums instead of integrals:

$y[n] = \sum_k x[k]\, h[n - k]$

By the time you've worked through three convolutions by hand, the visualization sticks. Try $x[n] = [1, 2, 3]$ convolved with $h[n] = [1, 1, 1]$. The result is $[1, 3, 6, 5, 3]$ (you can verify by writing out the multiplications). The output is one longer than $x$ plus $h$ minus 1 because the last sample of $x$ overlaps the first sample of $h$ at output time 4.

4.4 The transfer function: frequency-domain view

Now Fourier-transform both sides of $y(t) = x(t) * h(t)$. By the convolution-to-multiplication property:

$Y(j\omega) = X(j\omega) \cdot H(j\omega)$

where $H(j\omega)$ is the Fourier transform of the impulse response, called the frequency response or transfer function of the system. Multiplying frequency-by-frequency is much easier than convolving in time.

The system's behavior to each frequency separately is governed by $H(j\omega)$:

• $|H(j\omega)|$ is the gain at frequency $\omega$, how much the system amplifies or attenuates a sinusoid at that frequency.
• $\angle H(j\omega)$ is the phase shift the system applies to a sinusoid at that frequency.

This is the frequency-domain way to think about every circuit, every filter, every amplifier. Ask "what does this system do?" and the answer is "it has this $H(j\omega)$, which boosts low frequencies, kills the rest, and adds this much phase delay." Once you know $H$, you know the system.

Recall from Chapter 2 that an RC low-pass filter has $H(j\omega) = 1/(1 + j\omega RC)$. Magnitude rolls off at 20 dB/decade above $\omega = 1/RC$. Phase swings from 0 at DC to $-90°$ at high frequencies. That little formula encodes everything the filter does to any input signal: just multiply.

4.5 Filter types

Most LTI systems we design fall into a handful of categories based on their frequency response:

An ideal filter has a perfectly flat passband, perfectly zero stopband, and an instantaneous transition. Real filters approximate this with various tradeoffs (passband ripple, stopband attenuation, transition steepness, group delay flatness). We will study filter design in detail in Chapter 17 (DSP) and revisit analog filters in Chapter 10 (linear ICs).

4.6 Distortionless transmission

An LTI system can preserve a signal's shape if and only if:

• $|H(j\omega)| = K$ (constant magnitude across all frequencies of interest).
• $\angle H(j\omega) = -\omega t_d$ (linear phase, constant delay $t_d$).

Output is just $K \cdot x(t - t_d)$, same shape, scaled and delayed. Any deviation from this (non-flat magnitude or non-linear phase) distorts the signal. The non-flat magnitude is intuitive: different frequency components arrive scaled differently, so the waveform shape changes. The non-linear-phase part is subtler: if different frequencies are delayed by different amounts, the components arrive out of sync, and the waveform smears.

Linear phase is a big deal in systems where waveform shape matters: digital communications (eye diagrams!), pulsed radar, audio (where phase distortion shows up as smearing on transients). Filters with linear phase (like FIR filters with symmetric impulse response, Chapter 17) are preferred for these.

4.7 Bandwidth and rise time

For a low-pass filter with bandwidth $B$, the step response has a rise time $t_r$ (10% to 90%) inversely proportional to the bandwidth:

$t_r \cdot B \approx 0.35$

This is fundamental: every interconnect, every amplifier, every system has a bandwidth, and the bandwidth limits how fast the system can switch. To get a 1 ns rise time you need 350 MHz of bandwidth. To get 100 ps you need 3.5 GHz. This is why high-speed digital systems are so demanding on PCB layout, connectors, transmission line discipline, and substrate quality.

4.8 Causality and stability

Causality: a system is causal if $h(t) = 0$ for $t < 0$. Real-world physical systems are causal; the system cannot respond before the input arrives. (There are some special exotic systems involving negative group delay, but for our purposes, real systems are causal.)

Causality has a famous consequence, the Paley-Wiener criterion, that says you cannot have an absolutely flat magnitude response with a sharp cutoff in a causal system. Some transition band is forced by the math.

Stability: the system is BIBO stable (bounded-input, bounded-output) iff $\int |h(t)|\, dt < \infty$. Equivalently, all poles of the Laplace transform of $h$ are in the left half plane (we'll meet poles in Section 5). Practically: a system is stable if its impulse response decays to zero over time.

4.9 Correlation: measuring similarity

Cross-correlation of two signals:

$R_{xy}(\tau) = \int x(t)\, y(t + \tau)\, dt$

It measures how similar $x$ and $y$ are when one is shifted by $\tau$. The peak of the cross-correlation tells you the time shift at which they are most aligned. Notice the resemblance to convolution: the only difference is that there's no flip. (In fact, $R_{xy}(\tau) = x(-\tau) * y(\tau)$, so cross-correlation is convolution with one signal time-reversed.)

Auto-correlation $R_{xx}(\tau)$, correlation of a signal with itself, tells you about periodicities and self-similarity.

The Wiener-Khinchin theorem: the Fourier transform of the auto-correlation function is the power spectral density (PSD), the right way to talk about the frequency content of random signals.

Real-world uses of cross-correlation:

• Radar. Cross-correlate the received echo with the transmitted pulse. The peak's location gives the round-trip delay (and hence the range); the peak's height gives target reflectivity.
• GPS. Each satellite transmits a unique pseudo-random code. The receiver cross-correlates the received signal with locally-generated copies of all the satellites' codes. Peaks indicate which satellites are visible and at what time-of-arrival, which then gives position by triangulation.
• Sonar. Same idea as radar but with sound waves underwater.
• DPA / CPA hardware-security attacks. The attacker correlates measured power traces with a predicted power consumption based on a hypothesis about a key byte. The hypothesis with the highest correlation is the right key. We'll see this in detail in Chapter 24.
• Pattern matching in images. Finding a template inside a bigger image is 2D cross-correlation.

The matched filter (Chapter 12) is just cross-correlation packaged as a filter: the optimal way to detect a known signal in additive Gaussian noise.