Phase 7 - Lesson 7.1

Probability Spaces, Random Variables, and Distributions

The measure-theoretic scaffolding: sample spaces, sigma-algebras, probability measures, measurable maps, and the laws they induce.

⏱ 50 min● Advanced🔗 Prereqs: 6.x (measure and integration)

↖ Phase 7 hub

Builds on: Phase 6 built sigma-algebras, measures, and the Lebesgue integral.
Leads to: Every later probability and stochastic-calculus lesson lives on a probability space.

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

Define a probability space and verify the three axioms on a concrete example.
Explain why a sigma-algebra, not the full power set, is the right domain of events.
Define a random variable as a measurable map and compute the distribution (law) it induces.
Derive the relationship between a CDF, a PDF, and a discrete PMF and interpret each.

Key Vocabulary

Sample space: The set \(\Omega\) of all elementary outcomes of an experiment.

Sigma-algebra: A collection \(\F\) of subsets of \(\Omega\) closed under complement and countable union, containing \(\Omega\); the admissible events.

Probability measure: A set function \(\Prob:\F\to[0,1]\) with \(\Prob(\Omega)=1\) that is countably additive on disjoint events.

Random variable: A measurable map \(X:\Omega\to\R\): preimages of Borel sets are events.

Distribution (law): The pushforward measure \(\mu_X(B)=\Prob(X\in B)\) on \(\R\); all a random variable’s probabilistic content.

CDF: \(F_X(x)=\Prob(X\le x)\): non-decreasing, right-continuous, with limits 0 and 1.

Intuition & Motivation

Intuition

A probability space is a bookkeeping system with three parts: a set of possible worlds \(\Omega\), a list of questions you are allowed to ask about the world (the sigma-algebra \(\F\)), and a consistent way to assign each question a probability (the measure \(\Prob\)). A random variable is not random and not a variable: it is a deterministic readout \(X(\omega)\) that turns a world \(\omega\) into a number. Randomness lives only in which \(\omega\) nature draws; \(X\) just reports.

Why bother with sigma-algebras? Because for uncountable \(\Omega\) (like the reals) you cannot assign a consistent probability to every subset. Restricting to a sigma-algebra keeps the theory coherent while still containing every event you would ever want to measure.

The three ingredients

Definition - Probability space

A triple \((\Omega,\F,\Prob)\) where \(\Omega\) is a set, \(\F\) a sigma-algebra of subsets of \(\Omega\), and \(\Prob\) a measure with \(\Prob(\Omega)=1\). The Kolmogorov axioms are:

\[\Prob(A)\ge 0,\qquad \Prob(\Omega)=1,\qquad \Prob\Big(\bigcup_{i=1}^{\infty}A_i\Big)=\sum_{i=1}^{\infty}\Prob(A_i)\ \ (A_i\text{ disjoint}).\] (7.1)

From these follow \(\Prob(A^c)=1-\Prob(A)\), monotonicity, and continuity from below and above - the everyday rules of probability, now on a rigorous footing.

Random variables and their laws

Definition - Random variable

A function \(X:\Omega\to\R\) is a random variable if \(\{\omega:X(\omega)\le x\}\in\F\) for all \(x\). Equivalently \(X^{-1}(B)\in\F\) for every Borel set \(B\).

The law of \(X\) is the measure \(\mu_X\) on \(\R\) defined by pushing \(\Prob\) forward:

\[\mu_X(B)=\Prob\big(X^{-1}(B)\big)=\Prob(X\in B),\qquad B\in\mathcal{B}(\R).\] (7.2)

Two random variables on different spaces can share a law; probabilistic statements about \(X\) depend only on \(\mu_X\), summarized by the CDF \(F_X(x)=\Prob(X\le x)\).

Theorem - Characterization of CDFs

\(F\) is the CDF of some random variable iff it is non-decreasing, right-continuous, and satisfies \(\lim_{x\to-\infty}F(x)=0\), \(\lim_{x\to\infty}F(x)=1\). Jumps of \(F\) are atoms: \(\Prob(X=x)=F(x)-F(x^-)\).

Densities and mass functions

If \(F_X\) is absolutely continuous, \(X\) has a density \(f_X=F_X'\) with \(\Prob(a\le X\le b)=\int_a^b f_X\,dx\). If \(X\) takes countably many values, its PMF is \(p_X(x)=\Prob(X=x)\). Both are special cases of a density with respect to a reference measure (Lebesgue or counting).

Worked Example - Building a law from a space

Let \(\Omega=[0,1]\), \(\F\) the Borel sets, \(\Prob\) Lebesgue measure (the uniform experiment).

Define \(X(\omega)=-\tfrac{1}{\lambda}\ln(1-\omega)\) for \(\lambda\gt 0\). This is Borel-measurable, so a random variable.

Compute the CDF: for \(x\ge 0\), \(\{X\le x\}=\{\omega\le 1-e^{-\lambda x}\}\), so \(F_X(x)=1-e^{-\lambda x}\).

Differentiate: \(f_X(x)=\lambda e^{-\lambda x}\) - the exponential density. We built an exponential variable purely from a uniform draw (inverse-CDF sampling).

This is exactly how simulators generate non-uniform randomness from a single uniform generator.

Interactive: distributions and their shape

Manipulate a distribution and watch its CDF and density respond. Notice how the area under the density equals the CDF.

Common Mistakes to Avoid

Treating a random variable as ‘a number that changes randomly’ - it is a fixed measurable function; only the drawn \(\omega\) is random.
Assuming every subset of \(\Omega\) is an event. For uncountable \(\Omega\) only sets in \(\F\) are measurable.
Confusing the density value \(f_X(x)\) with a probability; \(f_X(x)\) can exceed 1. Only integrals of \(f_X\) are probabilities.
Forgetting right-continuity of the CDF: \(\Prob(X\le x)\) includes the atom at \(x\), while \(\Prob(X\lt x)\) does not.

Quant Practitioner Tips

When two models ‘feel’ different but share a CDF, they are probabilistically identical - work with the law, not the space.
Inverse-CDF sampling \(X=F^{-1}(U)\) with \(U\sim\)Uniform is the workhorse of Monte Carlo pricing.
Always check a proposed \(F\) against the four CDF properties before trusting a derivation built on it.

Practice this in the Euler Lab

Computational problems that exercise exactly this technique. Each opens in the Euler Lab with a Python workbench, a progressive hint ladder, and answer checking. Tier A/B run at full scale in the browser.

Warm-up:
#49 Prime Permutations (3%, tier A) #53 Combinatoric Selections (3%, tier A) #59 XOR Decryption (4%, tier A) #29 Distinct Powers (5%, tier A)

Applied:
#336 Maximix Arrangements (16%, tier B) #938 Exhausting a Colour (16%, tier B) #265 Binary Circles (17%, tier B)

Challenge:
#161 Triominoes (41%, tier C) #182 RSA Encryption (41%, tier C)

280 Project Euler problems in total are mapped to this lesson. Open the Euler Lab to filter them all.

Knowledge Check

Q1 Medium

Why do we restrict events to a sigma-algebra rather than all subsets of \(\Omega\)?

To save memory in computations

Because for uncountable \(\Omega\) no countably-additive probability can be consistently defined on every subset

Because sigma-algebras are easier to write down

Because the power set is not a set

Q2 Medium

The law (distribution) of a random variable \(X\) is:

The function \(X:\Omega\to\R\) itself

The sample space \(\Omega\)

The pushforward measure \(\mu_X(B)=\Prob(X\in B)\) on \(\R\)

The value \(X(\omega)\) for the realized \(\omega\)

Q3 Easy

Which statement about a density \(f_X\) is correct?

\(f_X(x)\) is the probability that \(X=x\)

\(f_X(x)\) must be at most 1

\(\int_a^b f_X\,dx=\Prob(a\le X\le b)\)

\(f_X\) equals the CDF

Practical Exercise

Let \(\Omega=\{1,2,3,4,5,6\}^2\) model two fair dice with \(\Prob\) uniform. Let \(S(\omega)=\) the sum. (a) Give the PMF of \(S\). (b) Compute \(F_S(4)\). (c) Explain why \(\{S=7\}\) is an event and \(\Prob(S=7)=1/6\).

▶ Show full solution

(a) By counting the 36 equally likely pairs, the PMF is:

\[p_S(k)=\frac{6-|k-7|}{36},\qquad k=2,\dots,12.\]

So \(p_S(2)=1/36,\ p_S(3)=2/36,\dots,p_S(7)=6/36,\dots,p_S(12)=1/36\).

(b) \(F_S(4)=\Prob(S\le 4)=p_S(2)+p_S(3)+p_S(4)=(1+2+3)/36=6/36=1/6.\)

(c) \(\Omega\) is finite, so \(\F\) is its power set and every subset - including \(\{S=7\}\) - is an event. There are 6 pairs summing to 7, hence \(\Prob(S=7)=6/36=1/6\).

After the reveal, answer for yourself: For finite spaces the sigma-algebra subtlety vanishes; it only bites for uncountable \(\Omega\).

Lesson Summary

A probability space \((\Omega,\F,\Prob)\) fixes outcomes, admissible events, and their probabilities. A random variable is a measurable readout whose pushforward law - encoded by a CDF, density, or PMF - carries all its probabilistic content. Working with the law rather than the underlying space is the central abstraction of the subject.

Formula Sheet Additions

Law / pushforward

\[\mu_X(B)=\Prob(X\in B),\quad F_X(x)=\Prob(X\le x)\]

The distribution is the measure X induces on the line.

Error Log Checklist

Did I verify measurability before calling a map a random variable?
Did I keep density (a rate) distinct from probability (an integral)?
Did I respect right-continuity when handling atoms of the CDF?

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers

Q: State the three Kolmogorov axioms.
A: Non-negativity \(\Prob(A)\ge0\); normalization \(\Prob(\Omega)=1\); countable additivity on disjoint events.

Q: What is the law of a random variable and why is it the object we care about?
A: The pushforward measure \(\mu_X(B)=\Prob(X\in B)\); it captures all probabilistic information, independent of the underlying space.

Q: How do a CDF, PDF, and PMF relate?
A: CDF \(F=\Prob(X\le x)\); PDF \(f=F'\) for continuous laws with \(\Prob(a\le X\le b)=\int_a^b f\); PMF \(p(x)=\Prob(X=x)\) for discrete laws (jumps of F).

Flashcards

Click to flip. These feed the site-wide spaced-repetition queue.

Random variable

A measurable map \(X:\Omega\to\R\); preimages of Borel sets are events.

CDF properties

Non-decreasing, right-continuous, limits 0 and 1; jumps are atoms.

Completion Checklist

I can explain the core ideas in my own words
I worked the derivations/examples by hand
I completed the interactive workbench(es)
I passed the knowledge check

Confidence / mastery rating

Personal notes

Source References

This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.

Probability: Theory and Examples (Rick Durrett, 5th ed.) current - Ch. 1 - Ch. 1: probability spaces, random variables, distribution functions.
Measure, Integration & Real Analysis (Sheldon Axler, GTM 282) current - Ch. 2 - Measure-theoretic foundations: sigma-algebras and measurable functions.

← 6.5 Next: 7.2 →