Phase 7 - Lesson 7.2

Expectation, Moments, and Key Inequalities

Expectation as the Lebesgue integral of a random variable, its moments, and the Markov, Chebyshev, Jensen, and Cauchy-Schwarz inequalities that power all of probability.

⏱ 50 min● Advanced🔗 Prereqs: 7.1

↖ Phase 7 hub

Builds on: 7.1 gave laws, densities, and CDFs.
Leads to: Chebyshev drives the weak law (7.5); Jensen underlies risk-averse pricing and the CLT scaling.

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

Define expectation as an integral and compute it from a density or PMF.
Derive variance and the moment structure, and interpret higher moments (skew, kurtosis).
Prove Markov and Chebyshev inequalities and use them to bound tail probabilities.
State and apply Jensen and Cauchy-Schwarz, distinguishing convex from concave cases.

Key Vocabulary

Expectation: \(\E[X]=\int_\Omega X\,d\Prob=\int_\R x\,d\mu_X\); the Lebesgue integral / probability-weighted average.

Variance: \(\Var(X)=\E[(X-\E X)^2]=\E[X^2]-(\E X)^2\); mean-squared spread.

Moment: \(\E[X^k]\) (raw) or \(\E[(X-\mu)^k]\) (central); the \(k\)-th shape descriptor.

Markov inequality: For \(X\ge0\), \(\Prob(X\ge a)\le \E[X]/a\).

Chebyshev inequality: \(\Prob(|X-\mu|\ge k\sigma)\le 1/k^2\); a distribution-free spread bound.

Jensen inequality: For convex \(\varphi\), \(\varphi(\E X)\le \E[\varphi(X)]\); reversed for concave \(\varphi\).

Intuition & Motivation

Intuition

Expectation is just the integral \(\int X\,d\Prob\): a weighted average where the weights are probabilities. The inequalities are the reason probability is useful without knowing the full distribution. Markov says a non-negative quantity rarely lands far above its mean; Chebyshev refines this using the variance; Jensen tells you how averaging interacts with curvature; Cauchy-Schwarz bounds correlation. These four crude-looking bounds are the load-bearing walls of the laws of large numbers, concentration, and risk analysis.

Expectation and moments

Definition - Expectation

For a random variable \(X\) on \((\Omega,\F,\Prob)\), \(\E[X]=\int_\Omega X\,d\Prob\), provided \(\E|X|\lt \infty\). By the change-of-variables (law) formula,

\[\E[g(X)]=\int_\R g(x)\,d\mu_X(x)=\int_\R g(x)f_X(x)\,dx \ \text{ or }\ \sum_x g(x)p_X(x).\] (7.3)

Expectation is linear: \(\E[aX+bY]=a\E X+b\E Y\), always (independence not required). The variance is the second central moment,

\[\Var(X)=\E[(X-\E X)^2]=\E[X^2]-(\E X)^2\ \ge 0.\] (7.4)

Higher central moments describe shape: the third gives skewness (asymmetry), the fourth kurtosis (tail heaviness) - both central to modeling fat-tailed asset returns.

The four inequalities

Theorem - Markov

If \(X\ge0\) and \(a\gt 0\), then \(\Prob(X\ge a)\le \E[X]/a\).

Proof

Since \(X\ge a\,\ind\{X\ge a\}\), take expectations: \(\E[X]\ge a\,\Prob(X\ge a)\). Divide by \(a\).

Theorem - Chebyshev

For any \(X\) with mean \(\mu\), variance \(\sigma^2\lt \infty\), and \(k\gt 0\): \(\Prob(|X-\mu|\ge k\sigma)\le 1/k^2\).

Proof

Apply Markov to the non-negative variable \((X-\mu)^2\) at level \(a=k^2\sigma^2\): \(\Prob((X-\mu)^2\ge k^2\sigma^2)\le \E[(X-\mu)^2]/(k^2\sigma^2)=1/k^2\).

Theorem - Jensen

If \(\varphi\) is convex and \(\E|X|,\E|\varphi(X)|\lt \infty\), then \(\varphi(\E X)\le \E[\varphi(X)]\). Concave \(\varphi\) reverses the inequality.

Theorem - Cauchy-Schwarz

\(\big(\E[XY]\big)^2\le \E[X^2]\,\E[Y^2]\), with equality iff \(X,Y\) are linearly dependent a.s. This bounds covariance: \(|\Cov(X,Y)|\le \sqrt{\Var X\,\Var Y}\), i.e. \(|\rho|\le1\).

Worked Example - Chebyshev bounds a tail with no distributional assumption

Let \(X\) have mean \(\mu=100\) and standard deviation \(\sigma=10\); the law is otherwise unknown.

Bound \(\Prob(|X-100|\ge 30)\). Here \(30=3\sigma\), so \(k=3\).

Chebyshev: \(\Prob(|X-100|\ge 30)\le 1/3^2=1/9\approx 0.111\).

For a normal law the true value is \(\approx 0.0027\): Chebyshev is loose but requires nothing beyond the variance - that generality is the point.

Its looseness is exactly why it still suffices to prove the weak law of large numbers in 7.5.

Interactive: convexity and Jensen

Plot a convex function and compare \(\varphi(\E X)\) with \(\E[\varphi(X)]\). The gap is the Jensen premium - the mathematical origin of why volatility has value.

Common Mistakes to Avoid

Assuming \(\E[g(X)]=g(\E X)\). This holds only for affine \(g\); Jensen shows the gap for convex/concave \(g\).
Using \(\Var(X+Y)=\Var X+\Var Y\) without independence - the cross term \(2\Cov(X,Y)\) is generally present.
Applying Markov to a variable that can be negative; it requires \(X\ge0\).
Reading Chebyshev as tight; it is a worst-case bound and is usually very conservative.

Quant Practitioner Tips

A concave utility plus Jensen is the whole content of risk aversion: \(\E[u(W)]\le u(\E W)\).
For option payoffs (convex), Jensen implies more dispersion in the underlying raises expected payoff - vega is positive.
When you know only a mean and variance, Chebyshev gives an honest, if loose, guarantee.

Practice this in the Euler Lab

Computational problems that exercise exactly this technique. Each opens in the Euler Lab with a Python workbench, a progressive hint ladder, and answer checking. Tier A/B run at full scale in the browser.

Warm-up:
#59 XOR Decryption (4%, tier A) #79 Passcode Derivation (7%, tier A) #102 Triangle Containment (8%, tier A) #54 Poker Hands (9%, tier A)

Applied:
#938 Exhausting a Colour (16%, tier B) #816 Shortest Distance Among Points (17%, tier B) #323 Bitwise-OR Operations on Random In (18%, tier B)

Challenge:
#375 Minimum of Subsequences (41%, tier C) #666 Polymorphic Bacteria (41%, tier C)

124 Project Euler problems in total are mapped to this lesson. Open the Euler Lab to filter them all.

Knowledge Check

Q1 Easy

Markov’s inequality \(\Prob(X\ge a)\le\E[X]/a\) requires which assumption?

\(X\) is normal

\(X\ge 0\)

\(X\) has finite variance

\(X\) is discrete

Q2 Medium

For a convex function \(\varphi\), Jensen’s inequality states:

\(\varphi(\E X)\ge\E[\varphi(X)]\)

\(\varphi(\E X)\le\E[\varphi(X)]\)

\(\varphi(\E X)=\E[\varphi(X)]\)

No general relation holds

Q3 Medium

A variable has mean 50 and standard deviation 5. The best distribution-free bound on \(\Prob(|X-50|\ge20)\) is:

\(\le 1/4\)

\(\le 1/16\)

\(\le 1/2\)

exactly 0.05

Practical Exercise

Let \(X\) be exponential with rate \(\lambda\), so \(f_X(x)=\lambda e^{-\lambda x}\) on \(x\ge0\). (a) Compute \(\E[X]\) and \(\Var(X)\). (b) Use Markov to bound \(\Prob(X\ge 3/\lambda)\) and compare to the exact value.

▶ Show full solution

(a) \(\E[X]=\int_0^\infty x\lambda e^{-\lambda x}dx=1/\lambda\). \(\E[X^2]=\int_0^\infty x^2\lambda e^{-\lambda x}dx=2/\lambda^2\), so \(\Var(X)=2/\lambda^2-1/\lambda^2=1/\lambda^2\).

(b) Markov with \(a=3/\lambda\): \(\Prob(X\ge3/\lambda)\le \E[X]/a=(1/\lambda)/(3/\lambda)=1/3\).

Exact: \(\Prob(X\ge3/\lambda)=e^{-\lambda\cdot3/\lambda}=e^{-3}\approx0.0498\). Markov’s \(1/3\) is valid but very loose - typical of a mean-only bound.

After the reveal, answer for yourself: How much does the bound tighten if you switch to Chebyshev using the variance you computed?

Lesson Summary

Expectation is the Lebesgue integral of a random variable and is always linear; variance and higher moments describe spread and shape. Markov and Chebyshev turn moment information into tail bounds without knowing the full law, while Jensen and Cauchy-Schwarz govern how expectation interacts with convexity and correlation. These four inequalities are the analytic engine behind the limit theorems.

Formula Sheet Additions

Chebyshev

\[\Prob(|X-\mu|\ge k\sigma)\le 1/k^2\]

Distribution-free spread bound from the variance.

Jensen

\[\varphi\ \text{convex}\Rightarrow \varphi(\E X)\le \E[\varphi(X)]\]

Curvature makes averaging inputs an underestimate.

Error Log Checklist

Did I confirm \(X\ge0\) before invoking Markov?
Did I add the covariance term when computing \(\Var(X+Y)\)?
Did I get the direction of Jensen right for convex vs concave?

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers

Q: Prove Chebyshev from Markov in one line.
A: Apply Markov to \((X-\mu)^2\ge0\) at level \(k^2\sigma^2\): \(\Prob((X-\mu)^2\ge k^2\sigma^2)\le \E[(X-\mu)^2]/(k^2\sigma^2)=1/k^2\).

Q: Why is \(\E[u(W)]\le u(\E W)\) for a concave utility?
A: Jensen for concave \(u\): the certainty of the mean is preferred to the gamble, which is exactly risk aversion.

Q: State Cauchy-Schwarz and its correlation consequence.
A: \((\E[XY])^2\le\E[X^2]\E[Y^2]\); hence \(|\Cov(X,Y)|\le\sqrt{\Var X\,\Var Y}\), i.e. \(|\rho|\le1\).

Flashcards

Click to flip. These feed the site-wide spaced-repetition queue.

Markov inequality

\(X\ge0\Rightarrow \Prob(X\ge a)\le \E[X]/a\).

Variance identity

\(\Var(X)=\E[X^2]-(\E X)^2\).

Completion Checklist

I can explain the core ideas in my own words
I worked the derivations/examples by hand
I completed the interactive workbench(es)
I passed the knowledge check

Confidence / mastery rating

Personal notes

← 7.1 Next: 7.3 →