Phase 7 - Lesson 7.6

The Central Limit Theorem and Modes of Convergence

Why standardized sums are universally Gaussian, proved through characteristic functions, and the hierarchy of convergence modes that frames every limit theorem.

⏱ 50 min● Advanced🔗 Prereqs: 7.4, 7.5

↖ Phase 7 hub

Builds on: 7.4 gave characteristic functions; 7.5 gave the LLN and the $1/\sqrt n$ scale.
Leads to: The CLT and Gaussian limits underpin Brownian motion (Phase 8) and inference (Phase 11).

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

State the classical CLT and identify the correct $\sqrt n$ standardization.
Sketch the characteristic-function proof via the second-order Taylor expansion.
Order the four modes of convergence and give the implications between them.
Apply the CLT to build confidence intervals and interpret its finite-sample limits.

Key Vocabulary

Central limit theorem: For i.i.d. with mean $\mu$, variance $\sigma^2$, $\sqrt n(\bar X_n-\mu)/\sigma\Rightarrow\Normal(0,1)$.

Convergence in distribution: $X_n\Rightarrow X$ if $F_{X_n}(x)\to F_X(x)$ at every continuity point of $F_X$.

Convergence in probability: $X_n\xrightarrow{p}X$ if $\Prob(|X_n-X|\gt \varepsilon)\to0$.

Almost sure convergence: $X_n\to X$ with probability 1.

$L^p$ convergence: $\E|X_n-X|^p\to0$; mean-square is $p=2$.

Slutsky’s theorem: If $X_n\Rightarrow X$ and $Y_n\xrightarrow{p}c$, then $X_n+Y_n\Rightarrow X+c$ and $X_nY_n\Rightarrow cX$.

Intuition & Motivation

Intuition

The central limit theorem is the reason the bell curve is everywhere: sum many small independent effects, standardize by $\sqrt n$, and the shape is Gaussian - regardless of the parts’ own distribution, as long as they have finite variance. The LLN says $\bar X_n\to\mu$; the CLT magnifies the residual by exactly $\sqrt n$ to reveal a non-degenerate Gaussian fluctuation. The proof is startlingly clean in characteristic functions: a quadratic Taylor term survives the limit and is precisely the Gaussian signature $e^{-t^2/2}$. Surrounding it all is a hierarchy of convergence modes you must keep straight, because ‘$X_n\to X$’ can mean four different things.

The classical central limit theorem

Theorem - Lindeberg-Levy CLT

Let $X_1,X_2,\dots$ be i.i.d. with mean $\mu$ and finite variance $\sigma^2\gt 0$. Then

\[Z_n:=\frac{\sqrt n\,(\bar X_n-\mu)}{\sigma}=\frac{\sum_{i=1}^n X_i-n\mu}{\sigma\sqrt n}\ \Longrightarrow\ \Normal(0,1).\] (7.9)

Proof

Center and scale so $Y_i=(X_i-\mu)/\sigma$ has mean 0, variance 1, with characteristic function $\phi_Y(t)=1-\tfrac{t^2}{2}+o(t^2)$. By independence $\phi_{Z_n}(t)=\phi_Y(t/\sqrt n)^n=\big(1-\tfrac{t^2}{2n}+o(1/n)\big)^n\to e^{-t^2/2}$, the characteristic function of $\Normal(0,1)$. Levy’s continuity theorem (7.4) converts this into convergence in distribution.

The theorem says nothing about the rate; the Berry-Esseen theorem bounds the CDF error by $C\rho/(\sigma^3\sqrt n)$ when a third moment $\rho$ exists. Convergence is fast near the center and slow in the tails, which is why the Gaussian approximation for rare events must be used with care.

The four modes of convergence

Keep these straight; the arrows show which implications hold in general:

\[\text{a.s.}\ \Rightarrow\ \text{in probability}\ \Rightarrow\ \text{in distribution};\qquad L^p\ \Rightarrow\ \text{in probability}.\] (7.10)

Mode	Meaning	Implies
Almost sure	$\Prob(X_n\to X)=1$	in probability
$L^p$ (e.g. $L^2$)	$\E\|X_n-X\|^p\to0$	in probability
In probability	$\Prob(\|X_n-X\|\gt \varepsilon)\to0$	in distribution
In distribution	$F_{X_n}\to F_X$ at continuity pts	(weakest)

None of the reverse implications holds in general. Convergence in distribution to a constant does upgrade to convergence in probability - the exception that makes Slutsky’s theorem work.

Worked Example - From a die roll to a bell curve

A single fair die has mean $\mu=3.5$ and variance $\sigma^2=35/12\approx2.917$ - a flat, discrete law.

Roll $n=100$ dice and sum. The sum has mean $350$ and variance $100\times2.917=291.7$, so standard deviation $\approx17.08$.

Standardize: $Z=(S-350)/17.08$. The CLT says $Z\approx\Normal(0,1)$ despite the parts being uniform on $\{1,\dots,6\}$.

Estimate $\Prob(S\gt 380)$: $\Prob(Z\gt (380-350)/17.08)=\Prob(Z\gt 1.756)\approx0.040.$

The exact answer (by convolution) is close to this; the small gap is the finite-$n$ correction the Berry-Esseen bound controls.

Interactive: the CLT in action

Sample from a decidedly non-normal law and watch the standardized sum become bell-shaped as $n$ grows - universality made visible.

Interactive: verify the CLT numerically

Common Mistakes to Avoid

Standardizing by $n$ instead of $\sqrt n$: the correct scale is $\sqrt n(\bar X_n-\mu)$; dividing by $n$ collapses to 0.
Applying the CLT to infinite-variance data - without $\sigma^2\lt \infty$ the limit is a stable law, not Gaussian.
Assuming the CLT makes the tails accurate; convergence is slowest exactly where rare-event probabilities live.
Reversing an implication between modes (e.g. inferring almost-sure from in-distribution) - only the listed arrows hold.

Quant Practitioner Tips

The 95% interval $\bar X_n\pm 1.96\,\sigma/\sqrt n$ is the CLT in everyday form - know it cold.
For heavy-tailed returns, prefer the actual empirical distribution or a stable-law model over a naive Gaussian CLT.
Slutsky lets you replace an unknown $\sigma$ by a consistent estimate $\hat\sigma$ without changing the limiting Gaussian.

Knowledge Check

Q1 Easy

The correct standardization in the classical CLT divides the centered sum by:

$n$

$\sqrt n$

$n^2$

$\log n$

Q2 Medium

Which implication between convergence modes is TRUE in general?

In distribution $\Rightarrow$ almost sure

In probability $\Rightarrow$ in distribution

In distribution $\Rightarrow$ in probability

$L^2\Rightarrow$ almost sure

Q3 Hard

The characteristic-function proof of the CLT works because $\phi_Y(t/\sqrt n)^n\to$:

$e^{-t^2/2}$

$e^{-|t|}$

$1$

$e^{it}$

Practical Exercise

Daily P&L of a strategy is i.i.d. with mean \$200 and standard deviation \$2{,}000. (a) Approximate the distribution of the 250-day total using the CLT. (b) Estimate the probability the yearly total is negative.

▶ Show full solution

(a) The sum $S_{250}=\sum_{i=1}^{250}X_i$ has mean $250\times200=\$50{,}000$ and variance $250\times2000^2$, so standard deviation $2000\sqrt{250}\approx\$31{,}623$. By the CLT, $S_{250}\approx\Normal(50000,\,31623^2)$.

(b) $\Prob(S_{250}\lt 0)=\Prob\!\big(Z\lt (0-50000)/31623\big)=\Prob(Z\lt -1.58)\approx0.057.$

So roughly a 5.7% chance of a losing year, despite a positive daily edge - the CLT turns a per-trade edge into an annual risk statement. Caveat: real P&L has fat tails and autocorrelation, so treat this as a first-order estimate.

After the reveal, answer for yourself: How does the negative-year probability change if daily returns are positively autocorrelated?

Lesson Summary

The central limit theorem states that standardized sums of finite-variance i.i.d. variables converge in distribution to the standard normal, proved cleanly through characteristic functions where a surviving quadratic term yields $e^{-t^2/2}$. It sits atop a hierarchy - almost sure and $L^p$ each imply in probability, which implies in distribution - and delivers the $\pm1.96\,\sigma/\sqrt n$ confidence intervals used throughout quantitative practice.

Formula Sheet Additions

CLT

\[\frac{\sum X_i-n\mu}{\sigma\sqrt n}\Rightarrow\Normal(0,1)\]

Universal Gaussian fluctuation at the $\sqrt n$ scale.

Convergence hierarchy

\[\text{a.s.},\,L^p\ \Rightarrow\ \xrightarrow{p}\ \Rightarrow\ \text{in dist.}\]

Only these arrows hold in general.

Error Log Checklist

Did I standardize by $\sqrt n$, not $n$?
Did I verify finite variance before invoking the classical CLT?
Did I avoid trusting the Gaussian tail for rare events?

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers

Q: State the classical CLT and its standardization.
A: For i.i.d. with mean $\mu$, variance $\sigma^2\lt \infty$, $(\sum X_i-n\mu)/(\sigma\sqrt n)\Rightarrow\Normal(0,1)$.

Q: Give the general implications among the four convergence modes.
A: Almost sure $\Rightarrow$ in probability $\Rightarrow$ in distribution; $L^p\Rightarrow$ in probability. Reverses fail in general.

Q: What survives in the characteristic-function proof to produce the Gaussian?
A: The quadratic Taylor term: $\phi(t/\sqrt n)^n\to e^{-t^2/2}$.

Flashcards

Click to flip. These feed the site-wide spaced-repetition queue.

CLT scale

Standardize centered sums by $\sigma\sqrt n$ to get $\Normal(0,1)$.

Modes of convergence

a.s. and $L^p$ each imply in probability, which implies in distribution.

Completion Checklist

I can explain the core ideas in my own words
I worked the derivations/examples by hand
I completed the interactive workbench(es)
I passed the knowledge check

Confidence / mastery rating

Personal notes

Source References

This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.

Probability: Theory and Examples (Rick Durrett, 5th ed.) current - Ch. 3 - Ch. 3: central limit theorems, modes of convergence, Berry-Esseen.
Brownian Motion and Stochastic Calculus (Karatzas & Shreve, 2nd ed., GTM 113) foundational - Ch. 2 - Weak convergence framework leading to Brownian motion.

← 7.5 Next: 7.7 →

Mode	Meaning	Implies
Almost sure	\(\Prob(X_n\to X)=1\)	in probability
\(L^p\) (e.g. \(L^2\))	\(\E\|X_n-X\|^p\to0\)	in probability
In probability	\(\Prob(\|X_n-X\|\gt \varepsilon)\to0\)	in distribution
In distribution	\(F_{X_n}\to F_X\) at continuity pts	(weakest)