Phase 7 - Lesson 7.4
Common Distributions and Transform Methods (MGF, Characteristic Functions)
The standard catalogue of laws and the transforms - moment generating and characteristic functions - that make sums, moments, and limits tractable.
Builds on: 7.2 gave moments; 7.3 gave independence, which makes transforms multiply.
Leads to: Characteristic functions prove the CLT (7.6); the Gaussian MGF recurs throughout finance.
Leads to: Characteristic functions prove the CLT (7.6); the Gaussian MGF recurs throughout finance.
Learning Objectives
Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.
- Recall the defining parameters, mean, and variance of the key discrete and continuous laws.
- Define the MGF and characteristic function and use them to generate moments.
- Prove that the transform of a sum of independent variables is the product of transforms.
- Identify a distribution from its characteristic function via the uniqueness theorem.
Key Vocabulary
- Moment generating function
- \(M_X(t)=\E[e^{tX}]\); when finite near 0, its derivatives at 0 give the moments.
- Characteristic function
- \(\phi_X(t)=\E[e^{itX}]\); always exists, uniquely determines the law.
- Gaussian law
- \(\Normal(\mu,\sigma^2)\) with density \(\tfrac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu)^2/2\sigma^2}\); \(\phi(t)=e^{i\mu t-\sigma^2 t^2/2}\).
- Poisson law
- Counts of rare events; \(\Prob(N=k)=e^{-\lambda}\lambda^k/k!\), mean = variance = \(\lambda\).
- Uniqueness theorem
- Equal characteristic functions imply equal laws; convergence of \(\phi\) gives convergence in distribution (Levy).
- Convolution
- The law of \(X+Y\) for independent \(X,Y\); its transform is the product \(\phi_X\phi_Y\).
Intuition & Motivation
Intuition
Adding independent random variables convolves their densities - painful directly. Transforms turn convolution into multiplication: \(\phi_{X+Y}=\phi_X\phi_Y\). That single fact makes sums, and therefore the laws of large numbers and the CLT, tractable. The MGF is the friendlier of the two (real, differentiate for moments) but can fail to exist for heavy tails; the characteristic function always exists and, by Levy’s theorem, its pointwise convergence is exactly convergence in distribution. Learn the standard catalogue so you recognize a law the moment you see its transform or its moments.
A working catalogue
| Law | Parameters | Mean | Variance |
|---|---|---|---|
| Bernoulli(p) | \(p\in[0,1]\) | \(p\) | \(p(1-p)\) |
| Binomial(n,p) | \(n,p\) | \(np\) | \(np(1-p)\) |
| Poisson(\(\lambda\)) | \(\lambda\gt 0\) | \(\lambda\) | \(\lambda\) |
| Geometric(p) | \(p\) | \(1/p\) | \((1-p)/p^2\) |
| Uniform(a,b) | \(a\lt b\) | \((a+b)/2\) | \((b-a)^2/12\) |
| Exponential(\(\lambda\)) | \(\lambda\gt 0\) | \(1/\lambda\) | \(1/\lambda^2\) |
| Normal(\(\mu,\sigma^2\)) | \(\mu,\sigma^2\) | \(\mu\) | \(\sigma^2\) |
| Gamma(\(\alpha,\beta\)) | shape,rate | \(\alpha/\beta\) | \(\alpha/\beta^2\) |
Transforms and moments
Definition - MGF and characteristic function
\(M_X(t)=\E[e^{tX}]\) (may be infinite) and \(\phi_X(t)=\E[e^{itX}]\) (always defined, \(|\phi_X|\le1\)). When \(M_X\) is finite in a neighbourhood of 0,
\[\E[X^k]=M_X^{(k)}(0),\qquad M_X(t)=\sum_{k=0}^\infty \frac{\E[X^k]}{k!}\,t^k.\] (7.7)
Theorem - Sums of independents multiply transforms
If \(X,Y\) are independent then \(M_{X+Y}(t)=M_X(t)M_Y(t)\) and \(\phi_{X+Y}(t)=\phi_X(t)\phi_Y(t)\).
Proof
By independence \(e^{t(X+Y)}=e^{tX}e^{tY}\) factorizes in expectation: \(\E[e^{tX}e^{tY}]=\E[e^{tX}]\E[e^{tY}]\). Same for \(e^{it\cdot}\).
Consequences drop out immediately: a sum of independent Poissons is Poisson (rates add); a sum of independent Gaussians is Gaussian (means and variances add). The Gaussian characteristic function \(\phi(t)=e^{i\mu t-\sigma^2t^2/2}\) times itself just adds parameters.
Theorem - Uniqueness and Levy continuity
If \(\phi_X=\phi_Y\) then \(X,Y\) have the same law. If \(\phi_{X_n}(t)\to\phi(t)\) pointwise with \(\phi\) continuous at 0, then \(X_n\Rightarrow X\) where \(\phi=\phi_X\).
Worked Example - MGF of the Gaussian, and moments
1
For \(Z\sim\Normal(0,1)\), complete the square: \(M_Z(t)=\int \tfrac{1}{\sqrt{2\pi}}e^{tz-z^2/2}dz=e^{t^2/2}\).
2
Then \(X=\mu+\sigma Z\sim\Normal(\mu,\sigma^2)\) has \(M_X(t)=e^{\mu t}M_Z(\sigma t)=e^{\mu t+\sigma^2 t^2/2}.\)
3
Read off moments of \(Z\): \(M_Z(t)=1+\tfrac{t^2}{2}+\tfrac{t^4}{8}+\cdots\), so \(\E[Z]=0,\ \E[Z^2]=1,\ \E[Z^4]=3\) (kurtosis 3).
4
Adding two independent Gaussians multiplies MGFs: \(e^{\mu_1 t+\sigma_1^2t^2/2}e^{\mu_2 t+\sigma_2^2t^2/2}=e^{(\mu_1+\mu_2)t+(\sigma_1^2+\sigma_2^2)t^2/2}\) - Gaussian, parameters added.
Interactive: shape of the standard laws
Common Mistakes to Avoid
- Using the MGF when it does not exist - heavy-tailed laws (e.g. Cauchy) have no MGF; use the characteristic function.
- Forgetting the factor \(i\): \(\phi_X(t)=\E[e^{itX}]\), so \(\E[X^k]=i^{-k}\phi_X^{(k)}(0)\).
- Multiplying transforms of dependent variables - the product rule needs independence.
- Confusing Poisson (variance = mean) with binomial; they agree only in the rare-event limit.
Quant Practitioner Tips
- Recognize a Gaussian instantly from \(\phi(t)=e^{i\mu t-\sigma^2t^2/2}\) - the quadratic exponent is the signature.
- To find the law of a sum, multiply transforms and match to a known form rather than convolving densities.
- The characteristic function is the right tool for the CLT because Levy’s theorem converts its limit into distributional convergence.
Knowledge Check
Q1 Medium
Why is the characteristic function preferred over the MGF for proving limit theorems?
Q2 Easy
For independent \(X\sim\)Poisson(2), \(Y\sim\)Poisson(3), the law of \(X+Y\) is:
Q3 Medium
The MGF satisfies \(M_X^{(k)}(0)=\) ?
Practical Exercise
Let \(X_1,\dots,X_n\) be independent Exponential(\(\lambda\)). (a) Write the MGF of one \(X_i\) (for \(t\lt \lambda\)). (b) Deduce the MGF of \(S_n=\sum X_i\) and identify its distribution.
▶ Show full solution
(a) \(M_{X}(t)=\int_0^\infty e^{tx}\lambda e^{-\lambda x}dx=\dfrac{\lambda}{\lambda-t}\) for \(t\lt \lambda\).
(b) By independence \(M_{S_n}(t)=\big(\tfrac{\lambda}{\lambda-t}\big)^n=\big(1-t/\lambda\big)^{-n}\).
This is the MGF of a Gamma(shape \(n\), rate \(\lambda\)) law - the Erlang distribution. So a sum of \(n\) i.i.d. exponentials is Gamma\((n,\lambda)\), which models the waiting time until the \(n\)-th event of a Poisson process.
After the reveal, answer for yourself: As \(n\) grows, what does the CLT predict about the shape of the standardized \(S_n\)?
Lesson Summary
A compact catalogue of discrete and continuous laws, plus two transforms, covers most modeling needs. The MGF generates moments but may not exist; the characteristic function always exists and uniquely determines the law. Their defining virtue is that transforms of independent sums multiply, turning convolution into algebra and setting up the limit theorems.
Formula Sheet Additions
Gaussian MGF
\[X\sim\Normal(\mu,\sigma^2)\Rightarrow M_X(t)=e^{\mu t+\sigma^2 t^2/2}\]
The quadratic exponent is the Gaussian signature.
Independent sums
\[\phi_{X+Y}=\phi_X\,\phi_Y\]
Convolution becomes multiplication.
Error Log Checklist
- Did I confirm the MGF exists before using it (finite near 0)?
- Did I require independence before multiplying transforms?
- Did I match the resulting transform to a known law?
Retrieval Practice
Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.
▶ Show retrieval prompts & answers
Q: Why do transforms turn sums of independents into products?
A: Because \(e^{t(X+Y)}=e^{tX}e^{tY}\) and independence factorizes the expectation, giving \(M_{X+Y}=M_XM_Y\) (same for \(\phi\)).
A: Because \(e^{t(X+Y)}=e^{tX}e^{tY}\) and independence factorizes the expectation, giving \(M_{X+Y}=M_XM_Y\) (same for \(\phi\)).
Q: Give the characteristic function of \(\Normal(\mu,\sigma^2)\).
A: \(\phi(t)=e^{i\mu t-\sigma^2 t^2/2}\).
A: \(\phi(t)=e^{i\mu t-\sigma^2 t^2/2}\).
Q: Mean and variance of Poisson(\(\lambda\))?
A: Both equal \(\lambda\).
A: Both equal \(\lambda\).
Flashcards
Click to flip. These feed the site-wide spaced-repetition queue.
MGF moments
\(M_X^{(k)}(0)=\E[X^k]\) when \(M_X\) is finite near 0.
Sum of Gaussians
Independent Gaussians add: means and variances sum; result is Gaussian.
Completion Checklist
- I can explain the core ideas in my own words
- I worked the derivations/examples by hand
- I completed the interactive workbench(es)
- I passed the knowledge check
Confidence / mastery rating
Personal notes
Source References
This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.
- Probability: Theory and Examples (Rick Durrett, 5th ed.) current - Ch. 2-3 - Ch. 2-3: distributions, characteristic functions, uniqueness and continuity.
- Measure, Integration & Real Analysis (Sheldon Axler, GTM 282) current - Ch. 3 - Integration machinery behind transform integrals.