Independence, Conditional Probability, and Conditional Expectation
From elementary conditioning and Bayes to conditional expectation as a projection onto a sigma-algebra - the object that becomes the martingale.
Leads to: Conditional expectation given a filtration defines martingales (7.7) and the whole of stochastic calculus.
Learning Objectives
Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.
- Define independence of events and random variables and test it on examples.
- Apply Bayes’ theorem and the law of total probability to update beliefs.
- Define conditional expectation \(\E[X\mid\mathcal{G}]\) and state its defining property.
- Use the tower property and interpret conditional expectation as an \(L^2\) projection.
Key Vocabulary
- Independent events
- \(\Prob(A\cap B)=\Prob(A)\Prob(B)\); knowing one gives no information about the other.
- Conditional probability
- \(\Prob(A\mid B)=\Prob(A\cap B)/\Prob(B)\) for \(\Prob(B)\gt 0\).
- Bayes’ theorem
- \(\Prob(A\mid B)=\Prob(B\mid A)\Prob(A)/\Prob(B)\); inverts the direction of conditioning.
- Conditional expectation
- \(\E[X\mid\mathcal{G}]\): the \(\mathcal{G}\)-measurable random variable matching \(X\)’s integral on every \(\mathcal{G}\)-event.
- Tower property
- \(\E[\E[X\mid\mathcal{G}]]=\E[X]\); more generally \(\E[\E[X\mid\mathcal{G}]\mid\mathcal{H}]=\E[X\mid\mathcal{H}]\) for \(\mathcal{H}\subseteq\mathcal{G}\).
- Sigma-algebra as information
- A sub-sigma-algebra \(\mathcal{G}\subseteq\F\) models ‘what is known’; measurability w.r.t. \(\mathcal{G}\) means observable.
Intuition & Motivation
Independence and Bayes
For independent \(X,Y\): \(\E[XY]=\E X\,\E Y\) and \(\Var(X+Y)=\Var X+\Var Y\). The converse is false - uncorrelated need not mean independent.
Conditional expectation given a sigma-algebra
Existence follows from the Radon-Nikodym theorem. Two special cases anchor intuition: \(\E[X\mid\{\emptyset,\Omega\}]=\E X\) (no information), and if \(X\) is \(\mathcal{G}\)-measurable then \(\E[X\mid\mathcal{G}]=X\) (full information).
Key properties
Conditional expectation is linear, order-preserving, and satisfies:
- Tower / iterated expectations: \(\mathcal{H}\subseteq\mathcal{G}\Rightarrow \E[\E[X\mid\mathcal{G}]\mid\mathcal{H}]=\E[X\mid\mathcal{H}]\); in particular \(\E[\E[X\mid\mathcal{G}]]=\E X\).
- Taking out what is known: if \(Z\) is \(\mathcal{G}\)-measurable and bounded, \(\E[ZX\mid\mathcal{G}]=Z\,\E[X\mid\mathcal{G}]\).
- Independence: if \(X\) is independent of \(\mathcal{G}\), then \(\E[X\mid\mathcal{G}]=\E X\).
- \(L^2\) projection: among all \(\mathcal{G}\)-measurable \(Y\), \(\E[X\mid\mathcal{G}]\) minimizes \(\E[(X-Y)^2]\) - it is the best mean-square forecast.
Interactive: Monte Carlo check of the tower property
- Confusing uncorrelated with independent: independence implies zero covariance, not conversely.
- Treating \(\E[X\mid\mathcal{G}]\) as a number - it is a random variable (a function of the information).
- Pulling a non-\(\mathcal{G}\)-measurable factor out of a conditional expectation.
- Swapping the order in the tower property; you must condition down to the coarser sigma-algebra.
- Whenever a sum has a random number of terms, condition on the count first - iterated expectations linearizes it.
- Read \(\E[X\mid\mathcal{G}]\) as ‘the best forecast of \(X\) given what \(\mathcal{G}\) knows’; it is a projection.
- The tower property is the single most-used identity in derivative pricing - today’s price is the conditional expectation of tomorrow’s.
Practice this in the Euler Lab
Computational problems that exercise exactly this technique. Each opens in the Euler Lab with a Python workbench, a progressive hint ladder, and answer checking. Tier A/B run at full scale in the browser.
Warm-up:
#59 XOR Decryption (4%, tier A) #79 Passcode Derivation (7%, tier A) #102 Triangle Containment (8%, tier A) #54 Poker Hands (9%, tier A)
Applied:
#938 Exhausting a Colour (16%, tier B) #816 Shortest Distance Among Points (17%, tier B) #323 Bitwise-OR Operations on Random In (18%, tier B)
Challenge:
#375 Minimum of Subsequences (41%, tier C) #666 Polymorphic Bacteria (41%, tier C)
124 Project Euler problems in total are mapped to this lesson. Open the Euler Lab to filter them all.
Knowledge Check
Practical Exercise
A test for a disease is 99% sensitive and 95% specific; the disease prevalence is 0.5%. A random person tests positive. (a) Use Bayes to find the probability they actually have the disease. (b) Interpret.
(a) Let \(D\) be disease, \(+\) a positive test. \(\Prob(+\mid D)=0.99\), \(\Prob(+\mid D^c)=0.05\), \(\Prob(D)=0.005\).
(b) Despite a ‘99% accurate’ test, only about 9% of positives are true - because the base rate is tiny, false positives from the healthy 99.5% dominate. This is the base-rate fallacy, and it recurs in signal detection and alpha screening.
Lesson Summary
Formula Sheet Additions
- Did I keep \(\E[X\mid\mathcal{G}]\) as a random variable, not a number?
- Did I check independence rather than mere zero correlation?
- Did I only pull out factors that are \(\mathcal{G}\)-measurable?
Retrieval Practice
Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.
A: It is the \(\mathcal{G}\)-measurable \(Y\) with \(\int_G Y\,d\Prob=\int_G X\,d\Prob\) for all \(G\in\mathcal{G}\); equivalently the \(L^2\) projection onto \(\mathcal{G}\)-measurable functions.
A: Given \(N\), the sum has a fixed number of i.i.d. terms so \(\E[S\mid N]=N\mu\); the tower property then gives \(\E[S]=\mu\E[N]\).
A: No. Independence implies zero covariance, but zero covariance can hold for dependent variables (e.g. \(X\) and \(X^2\) for symmetric \(X\)).
Flashcards
Click to flip. These feed the site-wide spaced-repetition queue.
Completion Checklist
- I can explain the core ideas in my own words
- I worked the derivations/examples by hand
- I completed the interactive workbench(es)
- I passed the knowledge check