Phase 6 - Lesson 6.5
Product Measures, Lᵖ Spaces, and the Bridge to Probability
Fubini for iterated integrals, the geometry of \(L^p\), and the punchline: probability IS measure theory.
Builds on: Integration (6.3–6.4) and measures (6.2) combine into product spaces and function spaces.
Leads to: Phase 7 opens by declaring \((\Omega,\mathcal{F},\mathbb{P})\) a measure space; this lesson builds the bridge.
Leads to: Phase 7 opens by declaring \((\Omega,\mathcal{F},\mathbb{P})\) a measure space; this lesson builds the bridge.
Learning Objectives
Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.
- State the product measure construction and Fubini–Tonelli for iterated integrals.
- Define the \(L^p\) norm and prove/interpret Hölder and Minkowski inequalities.
- Explain why \(L^p\) is complete (Riesz–Fischer) and why \(L^2\) is special.
- Translate every measure-theoretic object into its probability counterpart.
- Compute an \(L^p\) norm and verify expectation equals an integral against \(\mathbb{P}\).
Key Vocabulary
- Product measure
- On \(X\times Y\), the measure with \((\mu\times\nu)(A\times B)=\mu(A)\nu(B)\); underlies joint distributions and independence.
- Fubini–Tonelli
- Iterated integrals equal the double integral: for \(f\ge0\) (Tonelli) or \(f\in L^1\) (Fubini), \(\int\!\!\int f\,d\mu\,d\nu=\int\!\!\int f\,d\nu\,d\mu\).
- L^p space
- Measurable \(f\) with \(\lVert f\rVert_p=(\int|f|^p d\mu)^{1/p}\lt \infty\), functions identified when equal a.e.
- Hölder's inequality
- \(\int|fg|\le\lVert f\rVert_p\lVert g\rVert_q\) for \(1/p+1/q=1\); source of Cauchy–Schwarz at \(p=q=2\).
- Riesz–Fischer
- \(L^p\) is a complete normed space (Banach); \(L^2\) is a Hilbert space with inner product \(\langle f,g\rangle=\int fg\).
- Probability space
- A measure space \((\Omega,\mathcal{F},\mathbb{P})\) with \(\mathbb{P}(\Omega)=1\); expectation is the integral against \(\mathbb{P}\).
Intuition & Motivation
Intuition
Two payoffs in this final analysis lesson. First, product measures + Fubini let you build joint distributions and swap the order of iterated integration - the everyday move behind independence and covariance calculations. Second, and more profound: probability is not a separate subject. A probability space is just a measure space with total mass 1. A random variable is a measurable function. Its expectation \(\mathbb{E}[X]=\int_\Omega X\,d\mathbb{P}\) is a Lebesgue integral. The variance lives in \(L^2\), a Hilbert space where ‘uncorrelated’ means ‘orthogonal’. Everything you built in Phase 6 becomes the grammar of Phase 7.
Product measures and Fubini
Theorem - Fubini–Tonelli
On a product of \(\sigma\)-finite spaces, if \(f\ge0\) (Tonelli) then the two iterated integrals equal the double integral; if \(f\in L^1(\mu\times\nu)\) (Fubini) the same holds and both iterated integrals are finite. Hence one may integrate in either order.
In probability this is exactly why a joint density factorizes under independence and why \(\mathbb{E}[XY]\) can be computed by iterated integration.
The \(L^p\) spaces
Definition - L^p norm
For \(1\le p\lt \infty\), \(\lVert f\rVert_p=\big(\int|f|^p\,d\mu\big)^{1/p}\), and \(\lVert f\rVert_\infty\) is the essential supremum. Functions equal almost everywhere are identified so that \(\lVert f\rVert_p=0\iff f=0\) a.e.
Theorem - Hölder and Minkowski
For conjugate exponents \(1/p+1/q=1\), \(\lVert fg\rVert_1\le\lVert f\rVert_p\lVert g\rVert_q\) (Hölder). And \(\lVert f+g\rVert_p\le\lVert f\rVert_p+\lVert g\rVert_p\) (Minkowski), so \(\lVert\cdot\rVert_p\) is a genuine norm.
At \(p=q=2\), Hölder becomes Cauchy–Schwarz \(|\langle f,g\rangle|\le\lVert f\rVert_2\lVert g\rVert_2\). By Riesz–Fischer, each \(L^p\) is complete; \(L^2\) additionally carries the inner product \(\langle f,g\rangle=\int fg\,d\mu\), making it a Hilbert space - the setting for projections, conditional expectation, and least squares.
The bridge: probability is measure theory
| Measure theory | Probability | Symbol |
|---|---|---|
| Measure space \((X,\mathcal{F},\mu)\) | Probability space | \((\Omega,\mathcal{F},\mathbb{P}),\ \mathbb{P}(\Omega)=1\) |
| Measurable function | Random variable | \(X:\Omega\to\mathbb{R}\) |
| Integral \(\int X\,d\mu\) | Expectation | \(\mathbb{E}[X]=\int_\Omega X\,d\mathbb{P}\) |
| \(L^2\) inner product | Covariance (centered) | \(\Cov(X,Y)=\langle X-\mathbb{E}X,\,Y-\mathbb{E}Y\rangle\) |
| Almost everywhere | Almost surely | a.e. \(\to\) a.s. |
| Product measure | Independence / joint law | \(\mathbb{P}_X\times\mathbb{P}_Y\) |
Worked Example - Expectation is an integral against \(\mathbb{P}\)
1
Let \(X\) take values \(x_i\) with probabilities \(p_i\) on a discrete \(\Omega=\{\omega_i\}\), where \(\mathbb{P}(\{\omega_i\})=p_i\).
2
As a simple function, \(X=\sum_i x_i\mathbf{1}_{\{\omega_i\}}\), so the Lebesgue integral is \(\int_\Omega X\,d\mathbb{P}=\sum_i x_i\,\mathbb{P}(\{\omega_i\})=\sum_i x_i p_i\).
3
That sum is exactly the elementary formula \(\mathbb{E}[X]=\sum_i x_i p_i\) - so expectation is a special case of the integral of 6.3.
4
For continuous \(X\) with density \(f\), the same integral becomes \(\mathbb{E}[X]=\int_{\mathbb{R}} x f(x)\,dx\). One definition, two familiar faces.
Interactive: \(L^p\) norm and expectation-as-integral
Compute an \(L^p\) norm, verify Cauchy–Schwarz, and confirm that discrete expectation equals the integral \(\sum x_i p_i\) against a probability measure.
Common Mistakes to Avoid
- Swapping iterated integrals without checking Tonelli (\(f\ge0\)) or Fubini (\(f\in L^1\)) - order can matter otherwise.
- Treating \(L^p\) elements as functions rather than a.e.-equivalence classes.
- Using Hölder with non-conjugate exponents (\(1/p+1/q\ne1\)).
- Forgetting that \(\mathbb{E}\) requires \(X\in L^1\); a heavy-tailed \(X\) may have no finite mean.
Quant Practitioner Tips
- \(L^2\) is the quant’s home: covariance is an inner product, so uncorrelated \(=\) orthogonal and regression \(=\) projection.
- Verify integrability before invoking Fubini; Tonelli on \(|f|\) is the standard pre-check.
- Read every probability statement as a measure statement - it removes the mystique and imports all of Phase 6’s theorems.
- Jensen, Markov, and Chebyshev inequalities are all measure-theoretic; you now have the tools to prove them.
Knowledge Check
Q1 Easy
The expectation \(\mathbb{E}[X]\) is, in measure-theoretic terms:
Q2 Medium
Hölder’s inequality at \(p=q=2\) specializes to:
Q3 Medium
Fubini’s theorem (as opposed to Tonelli) additionally requires:
Practical Exercise
Let \(X\) and \(Y\) be random variables in \(L^2(\Omega,\mathcal{F},\mathbb{P})\). Using Cauchy–Schwarz, prove \(|\Cov(X,Y)|\le\sigma_X\sigma_Y\), the correlation bound.
▶ Show full solution
Center: let \(\tilde X=X-\mathbb{E}X\), \(\tilde Y=Y-\mathbb{E}Y\). Then \(\Cov(X,Y)=\mathbb{E}[\tilde X\tilde Y]=\langle\tilde X,\tilde Y\rangle\) in \(L^2\).
Cauchy–Schwarz (Hölder at \(p=q=2\)) gives \(|\langle\tilde X,\tilde Y\rangle|\le\lVert\tilde X\rVert_2\lVert\tilde Y\rVert_2\).
But \(\lVert\tilde X\rVert_2=\sqrt{\mathbb{E}[\tilde X^2]}=\sigma_X\) and likewise \(\lVert\tilde Y\rVert_2=\sigma_Y\). Hence \(|\Cov(X,Y)|\le\sigma_X\sigma_Y\), so the correlation \(\rho=\Cov/(\sigma_X\sigma_Y)\in[-1,1]\).
After the reveal, answer for yourself: When is the bound tight, i.e. \(|\rho|=1\)? (When \(\tilde X,\tilde Y\) are linearly dependent - equality in Cauchy–Schwarz.)
Lesson Summary
Product measures and Fubini–Tonelli license iterated integration behind joint laws; the \(L^p\) spaces - complete by Riesz–Fischer, with \(L^2\) a Hilbert space - give the geometry of square-integrable randomness. The unifying message: a probability space is a mass-1 measure space, a random variable is a measurable function, and \(\mathbb{E}[X]=\int X\,d\mathbb{P}\) - probability IS measure theory, the gateway to Phase 7.
Retrieval Practice
Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.
▶ Show retrieval prompts & answers
Q: How does measure theory define expectation, and how does it reduce to \(\sum x_i p_i\)
A: \(\mathbb{E}[X]=\int_\Omega X\,d\mathbb{P}\); for a discrete \(X=\sum x_i\mathbf{1}_{\{\omega_i\}}\) this simple-function integral equals \(\sum_i x_i\mathbb{P}(\{\omega_i\})=\sum_i x_i p_i\).
A: \(\mathbb{E}[X]=\int_\Omega X\,d\mathbb{P}\); for a discrete \(X=\sum x_i\mathbf{1}_{\{\omega_i\}}\) this simple-function integral equals \(\sum_i x_i\mathbb{P}(\{\omega_i\})=\sum_i x_i p_i\).
Q: Why is \(L^2\) special among the \(L^p\)?
A: It carries an inner product \(\langle f,g\rangle=\int fg\), making it a Hilbert space; covariance is that inner product, so uncorrelated means orthogonal and regression is projection.
A: It carries an inner product \(\langle f,g\rangle=\int fg\), making it a Hilbert space; covariance is that inner product, so uncorrelated means orthogonal and regression is projection.
Completion Checklist
- I can explain the core ideas in my own words
- I worked the derivations/examples by hand
- I completed the interactive workbench(es)
- I passed the knowledge check
Confidence / mastery rating
Personal notes