Phase 12 - Lesson 12.1

Stationarity, Autocorrelation, and Partial Autocorrelation

What makes a series analyzable, how the ACF and PACF fingerprint its memory, and why financial series so rarely cooperate.

⏱ 50 min● Intermediate🔗 Prereqs: 11.1, 7.x probability

↖ Phase 12 hub

Builds on: Estimation and covariance (Phase 7, 11.1) underpin the autocovariance function.
Leads to: ACF/PACF shapes identify AR, MA, and ARMA orders in 12.2-12.3.

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

Distinguish strict from weak (covariance) stationarity and state why weak stationarity suffices for linear models.
Define the autocovariance and autocorrelation functions and compute them for simple processes.
Interpret the partial autocorrelation function as conditional correlation controlling for intermediate lags.
Use ACF/PACF signatures to distinguish AR, MA, and non-stationary behavior.
Explain how non-stationarity and volatility clustering complicate financial time series.

Key Vocabulary

Strict stationarity: The full joint distribution is invariant to time shifts: \((X_{t_1},\dots,X_{t_k})\) and \((X_{t_1+h},\dots,X_{t_k+h})\) agree.

Weak stationarity: Constant mean, constant variance, and autocovariance depending only on the lag \(h\), not on \(t\).

Autocovariance: \(\gamma(h)=\Cov(X_t,X_{t+h})\), a function of lag \(h\) under stationarity.

Autocorrelation (ACF): \(\rho(h)=\gamma(h)/\gamma(0)\); correlation at lag \(h\), with \(\rho(0)=1\).

PACF: Partial autocorrelation at lag \(h\): the correlation of \(X_t\) and \(X_{t+h}\) after removing the linear effect of the intervening lags.

White noise: Uncorrelated, mean-zero, constant-variance sequence; \(\rho(h)=0\) for \(h\ne0\); the residual ideal.

Unit root: A root on the unit circle (e.g. a random walk); the series is non-stationary and must be differenced.

Intuition & Motivation

Intuition

Almost every time-series tool assumes the statistical rules of the game do not drift: the mean, the variance, and the way today relates to yesterday stay put over time. That is stationarity. If a series trends or its volatility explodes, correlations estimated on it are unstable and forecasts are built on sand. So the first act of any time-series analysis is to check - and if necessary transform (difference, log, de-trend) - toward stationarity.

Once stationary, a series is fingerprinted by two functions. The ACF \(\rho(h)\) measures how strongly today correlates with the value \(h\) steps away, memory including everything in between. The PACF strips out those intermediaries: it is the correlation with lag \(h\) after regressing away lags \(1,\dots,h-1\). Their contrasting shapes - which one cuts off sharply and which one decays - tell you whether you are looking at an AR, an MA, or a mixed process, which is the whole model-identification game of the next lessons.

Two notions of stationarity

Strict stationarity demands that the entire joint distribution be shift-invariant - a strong condition we can rarely verify. Weak (covariance) stationarity asks only for the first two moments to be stable:

\[\E[X_t]=\mu\ \ \forall t,\qquad \Var(X_t)=\gamma(0)\lt \infty,\qquad \Cov(X_t,X_{t+h})=\gamma(h)\ \text{ depends only on } h.\] (12.1)

For linear models driven by second moments, weak stationarity is exactly what we need. (For Gaussian processes the two notions coincide, since a Gaussian law is pinned down by its mean and covariance.)

Autocovariance and autocorrelation

The autocovariance function \(\gamma(h)\) is symmetric (\(\gamma(-h)=\gamma(h)\)) and positive semidefinite. Its normalized version is the ACF:

\[\rho(h)=\frac{\gamma(h)}{\gamma(0)}\in[-1,1],\qquad \rho(0)=1.\] (12.2)

The sample ACF estimates it by \(\hat\rho(h)=\sum_{t=1}^{n-h}(x_t-\bar x)(x_{t+h}-\bar x)\big/\sum_{t=1}^{n}(x_t-\bar x)^2\). Under white noise, \(\hat\rho(h)\) is approximately \(\Normal(0,1/n)\), giving the \(\pm1.96/\sqrt{n}\) significance bands you see on ACF plots.

Worked Example - ACF of an AR(1) process

Let \(X_t=\phi X_{t-1}+\varepsilon_t\) with \(|\phi|\lt 1\) and white-noise \(\varepsilon_t\) of variance \(\sigma^2\).

Stationary variance: \(\gamma(0)=\phi^2\gamma(0)+\sigma^2\Rightarrow\gamma(0)=\sigma^2/(1-\phi^2).\)

Multiply the recursion by \(X_{t-h}\) and take expectations: \(\gamma(h)=\phi\,\gamma(h-1)\) for \(h\ge1\).

Hence \(\rho(h)=\phi^{|h|}\): the ACF decays geometrically. If \(\phi\to1\) it barely decays - the near-unit-root look of a trending series.

Partial autocorrelation

The PACF at lag \(h\), written \(\alpha(h)\), is the last coefficient \(\phi_{hh}\) in the best linear predictor of \(X_t\) from its \(h\) most recent lags - equivalently the correlation between \(X_t\) and \(X_{t-h}\) once \(X_{t-1},\dots,X_{t-h+1}\) are partialled out. This is what makes ACF and PACF complementary diagnostics:

Process	ACF	PACF
AR(\(p\))	tails off (geometric/damped sine)	cuts off after lag \(p\)
MA(\(q\))	cuts off after lag \(q\)	tails off
ARMA(\(p,q\))	tails off	tails off
White noise	zero for \(h\ge1\)	zero for \(h\ge1\)
Random walk	near 1, decays extremely slowly	spike near 1 at lag 1

Key Idea

The AR fingerprint is a PACF that cuts off; the MA fingerprint is an ACF that cuts off. A slowly-decaying ACF is the tell-tale of non-stationarity - difference the series and re-examine before modeling.

Interactive: simulate, view the ACF, and difference

Then verify the geometric decay numerically:

Common Mistakes to Avoid

Modeling a trending or heteroscedastic series as if stationary; the estimated ACF and any forecast are then unreliable.
Reading a slowly-decaying ACF as ‘strong predictability’ when it actually signals a unit root - difference first.
Confusing ACF and PACF diagnostics: AR shows a PACF cutoff, MA shows an ACF cutoff, not the reverse.
Ignoring that zero autocorrelation is not independence - squared returns can be strongly autocorrelated (volatility clustering) while returns are not.
Treating \(\pm1.96/\sqrt{n}\) bands as exact under dependence; they assume white noise.

Quant Practitioner Tips

Always plot the series, ACF, and PACF together before choosing a model - the shapes are your identification map.
Test for a unit root (ADF/KPSS) rather than eyeballing; near-unit-root series masquerade as stationary in short samples.
For returns, check the ACF of squared or absolute returns - it exposes the volatility clustering that motivates GARCH (12.4).
Financial stationarity is at best local; re-check it on rolling windows and expect structural breaks.

Knowledge Check

Q1 Medium

Weak (covariance) stationarity requires:

The full joint distribution to be shift-invariant

Constant mean and variance, and autocovariance depending only on the lag

Independent observations

A zero mean

Q2 Medium

For a stationary AR(1) with parameter \(\phi\), the autocorrelation at lag \(h\) is:

\(\phi/h\)

\(\phi^{|h|}\)

\(1/\phi^h\)

0 for \(h\ge1\)

Q3 Medium

An AR(2) process is best identified by:

An ACF that cuts off after lag 2

A PACF that cuts off after lag 2

Both ACF and PACF cutting off after lag 2

Neither function showing structure

Practical Exercise

Consider the random walk \(X_t=X_{t-1}+\varepsilon_t\) with white-noise \(\varepsilon_t\), \(\Var(\varepsilon_t)=\sigma^2\), \(X_0=0\). (a) Compute \(\Var(X_t)\) and show it is not stationary. (b) What is \(\Corr(X_t,X_{t+h})\) for large \(t\)? (c) Show the first difference is stationary and identify it.

▶ Show full solution

(a) \(X_t=\sum_{s=1}^t\varepsilon_s\) so \(\Var(X_t)=t\sigma^2\), which grows with \(t\): the variance is time-dependent, violating weak stationarity (a unit-root process).

(b) \(\Cov(X_t,X_{t+h})=\Var(X_t)=t\sigma^2\) (the shared innovations), so \(\Corr(X_t,X_{t+h})=t\sigma^2/\sqrt{t\sigma^2\cdot(t+h)\sigma^2}=\sqrt{t/(t+h)}\to1\) as \(t\to\infty\). This is the glacially-decaying ACF that flags non-stationarity.

(c) \(\nabla X_t=X_t-X_{t-1}=\varepsilon_t\), which is white noise: mean 0, variance \(\sigma^2\), zero autocorrelation - stationary. Differencing once removes the unit root, i.e. the series is \(I(1)\).

After the reveal, answer for yourself: A near-1, slowly-decaying sample ACF should trigger ‘difference and re-check’, not ‘great, strong signal’.

Lesson Summary

A series is analyzable when it is (weakly) stationary: stable mean and variance and lag-only autocovariance. The ACF \(\rho(h)=\gamma(h)/\gamma(0)\) and the PACF fingerprint its memory - AR processes show a PACF cutoff, MA processes an ACF cutoff, and a slowly-decaying ACF betrays a unit root to be differenced away. Financial series are at best locally stationary and exhibit volatility clustering, so squared-return autocorrelation and rolling re-checks are essential.

Formula Sheet Additions

ACF

\[\rho(h)=\gamma(h)/\gamma(0),\quad \rho(0)=1\]

Normalized autocovariance; \(\pm1.96/\sqrt n\) white-noise bands.

AR(1) ACF

\[\rho(h)=\phi^{|h|}\]

Geometric decay; \(\phi\to1\) gives the non-stationary look.

Error Log Checklist

Did I check/transform toward stationarity before estimating an ACF?
Did I distinguish an MA (ACF cutoff) from an AR (PACF cutoff)?
Did I test for a unit root instead of eyeballing a slow ACF decay?
Did I inspect the ACF of squared returns for volatility clustering?

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers

Q: State the three conditions for weak stationarity.
A: Constant mean \(\mu\), finite constant variance \(\gamma(0)\), and autocovariance \(\gamma(h)\) depending only on the lag \(h\).

Q: How do ACF and PACF distinguish AR from MA?
A: AR(\(p\)): PACF cuts off after \(p\), ACF tails off. MA(\(q\)): ACF cuts off after \(q\), PACF tails off.

Q: What does a slowly decaying sample ACF near 1 indicate, and the remedy?
A: A unit root / non-stationarity (e.g. a random walk); difference the series and re-examine.

Flashcards

Click to flip. These feed the site-wide spaced-repetition queue.

Weak stationarity

Constant mean, constant variance, autocovariance \(\gamma(h)\) depending only on lag \(h\).

ACF vs PACF

AR: PACF cutoff at \(p\). MA: ACF cutoff at \(q\). ARMA: both tail off.

Unit root tell

ACF near 1 decaying very slowly; difference once to reach stationarity.

Completion Checklist

I can explain the core ideas in my own words
I worked the derivations/examples by hand
I completed the interactive workbench(es)
I passed the knowledge check

Confidence / mastery rating

Personal notes

Source References

This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.

Time Series Analysis (James Hamilton, 1994) foundational - Hamilton 3 - Ch. 3: stationary ARMA processes, autocovariance-generating functions, ACF/PACF.
Probability: Theory and Examples (Rick Durrett, 5th ed.) current - Durrett 6 - Stationary sequences and ergodicity underlying consistent autocovariance estimation.

← 11.7 Next: 12.2 →