Quant Academy Math · Probability · Quant Finance
Phase 17 - Lesson 17.2

Numerical Correctness and Testing

How quants prove their code is right: analytic checks, boundary cases, property-based invariants, and tolerances that respect floating point.

⏱ 55 min● Intermediate🔗 Prereqs: 17.1, 2.1
↖ Phase 17 hub
Builds on: 17.1 gave you vectorized code; now you make it trustworthy.
Leads to: Every backtest and pricer in Phases 13–18 must ship with the tests this lesson teaches.

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

Key Vocabulary

Unit test
A small automated check that a function returns the expected output on specified inputs, run on every change.
Oracle
A trusted reference answer - a closed-form value, a slow-but-correct implementation, or a mathematical identity.
Boundary case
An input at the edge of validity (zero volatility, expiry \(T=0\), empty array) where naive code often breaks.
Property-based test
A test asserting an invariant that must hold for many random inputs (monotonicity, parity, symmetry), not one fixed case.
Tolerance
The allowed gap atol/rtol in a floating comparison, chosen from the algorithm’s error, not arbitrarily.
Regression test
A test that pins a previously fixed bug so it can never silently return.

Intuition & Motivation

Intuition
The dangerous bug in quant code is not the one that crashes - that one announces itself. It is the one that returns a plausible number: an option price that is 3% too high, a volatility that is positive but wrong. No exception fires; it flows straight into a P&L or a risk report. Testing is how you catch the silent wrong answer before it costs money.

Because floating-point arithmetic is inexact, you almost never assert a == b on computed reals. You assert |a-b| ≤ \text{atol} + \text{rtol}\,|b|. And because you cannot enumerate all inputs, you test three things: known answers (oracles), the edges (boundaries), and properties that must hold for every input.

Never compare floats with ==

Real arithmetic in IEEE-754 rounds every operation. So \(0.1+0.2 \ne 0.3\) exactly. The correct assertion uses a tolerance, matching NumPy’s rule:

\[|a-b| \;\le\; \text{atol} + \text{rtol}\cdot|b|.\] (17.2)

Pick rtol from the algorithm’s accuracy (a Monte Carlo estimate with \(10^4\) paths has standard error \(\sim 1/\sqrt{10^4}=10^{-2}\), so a 1e-9 tolerance is nonsense). Pick atol for quantities near zero.

Three kinds of test, in order of value

  1. Oracle tests. Compare against a value you trust: a closed form (Black–Scholes at a known point), or a slow reference (the loop from 17.1).
  2. Boundary tests. Feed the edges: \(T=0\), \(\sigma=0\), a deep in/out-of-the-money strike, an empty or length-1 array. These expose 0/0, log(0), and off-by-one errors.
  3. Property tests. Assert invariants over random inputs: a call price is non-negative, increases in \(S\), and satisfies put–call parity \(C-P=S-Ke^{-rT}\).
Worked Example - A boundary bug and the test that catches it
1
A Black–Scholes call uses \(d_1=\frac{\log(S/K)+(r+\sigma^2/2)T}{\sigma\sqrt T}\). At \(\sigma\to 0\) or \(T\to0\) this divides by zero.
2
The correct limit is the discounted intrinsic value \(\max(S-Ke^{-rT},0)\) (deterministic payoff).
3
Naive code returns nan at \(T=0\); a downstream sum then poisons the whole portfolio value with nan.
4
Boundary test: assert bs_call(S=100,K=90,r=0.01,sigma=0.2,T=0) == max(100-90,0). It fails on the naive code, passes once you special-case \(T=0\).
5
Property test to back it up: for random inputs, abs((C-P) - (S-K*exp(-r*T))) < 1e-9 (put–call parity).

Why parity is a great test

Put–call parity is model-free: it follows from no-arbitrage, not from Black–Scholes. So it must hold for any correct European call/put pricer, at every input. A single parity assertion over random parameters catches a huge class of sign errors, discounting errors, and \(d_1/d_2\) mix-ups.

\[C(S,K,r,\sigma,T) - P(S,K,r,\sigma,T) = S - K e^{-rT}.\] (17.3)
Common Mistakes to Avoid
  • Asserting price == 12.34 on a Monte Carlo estimate; it is random - assert it is within a few standard errors.
  • Testing only the ‘happy path’ and never \(T=0\), \(\sigma=0\), empty arrays, or extreme strikes.
  • Choosing a tolerance tighter than the method’s own error (e.g. 1e-12 on a 1e-2-accurate MC price).
  • Deleting a test because it’s ‘annoying’ instead of fixing the code - that test was finding a real bug.
  • Letting a function return nan silently instead of raising or handling the boundary; nan spreads through sums.
  • No regression test after a fix, so the same bug returns in three months.
Quant Practitioner Tips
  • Write the failing test first (it reproduces the bug), then fix until green - that is the regression discipline.
  • Prefer invariants (parity, monotonicity, non-negativity) over hard-coded numbers; they generalize across inputs.
  • For Monte Carlo, test the estimator’s standard error, e.g. |estimate−truth| < 3×SE, not exact equality.
  • Keep one tiny deterministic dataset in the repo so tests are fast and reproducible.
  • Assert on shapes and dtypes too; a (T,1) vs (T,) mismatch is a classic silent broadcasting bug.

Interactive: write the test that catches the boundary bug

The starter contains a corrected pricer plus a parity check. Implement the boundary handling and the property test so all assertions pass.

Knowledge Check

Q1 Easy
Why do you assert abs(a-b)<tol instead of a==b for computed reals?
Because Python is slow
Because IEEE-754 rounds each operation, so mathematically-equal values differ in the last bits
Because == is deprecated
Because tolerances are faster to compute
Q2 Medium
Put–call parity is an especially strong test because it is:
Only true in Black–Scholes
Model-free (from no-arbitrage), so it must hold for any correct European pricer at every input
An approximation
True only at the money
Q3 Medium
A Monte Carlo price estimate from 10,000 paths should be tested by:
Exact equality to the closed form
|estimate−truth| within a few standard errors (~3×SE)
Checking it equals 0
A 1e-12 tolerance

Practical Exercise

You inherit a function impl_vol(price,S,K,r,T) that inverts Black–Scholes for implied volatility. Design a test suite: name one oracle test, two boundary tests, and one property test, each with the specific input and expected behavior.

▶ Show full solution

Oracle: price an option with a known \(\sigma=0.2\) via bs_call, feed that price back in, assert impl_vol(...) recovers 0.20 to a tolerance matching the root-finder (e.g. 1e-6).

Boundary 1: price below intrinsic value (arbitrage input) - assert it raises or returns NaN rather than a bogus vol.

Boundary 2: price equal to a deep-ITM intrinsic with tiny time value - assert vol is small and finite (no divide-by-zero or non-convergence).

Property: monotonicity - for two prices \(p_1<p_2\) on the same option, impl_vol(p1)<impl_vol(p2), since vega is positive (call price increases in \(\sigma\)). Test over random parameter draws.

After the reveal, answer for yourself: Which single test would you keep if you could keep only one, and why?

Lesson Summary

Correctness in quant code means catching the silent wrong answer. Replace float equality with tolerance-based comparison \(|a-b|\le\text{atol}+\text{rtol}|b|\); test against oracles (closed forms, slow references), the boundaries (\(T=0,\ \sigma=0\), empty arrays), and model-free properties such as put–call parity. Pin every fixed bug with a regression test.

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers
Q: State the tolerance rule NumPy uses for allclose and why float == is wrong.
A: \(|a-b|\le\text{atol}+\text{rtol}|b|\). Float == is wrong because IEEE-754 rounds each operation, so mathematically equal computations differ in the low bits.
Q: Give one boundary case and one property that should test a Black-Scholes call pricer.
A: Boundary: \(T=0\) (or \(\sigma=0\)) must return discounted intrinsic \(\max(S-Ke^{-rT},0)\), not NaN. Property: put-call parity \(C-P=S-Ke^{-rT}\) must hold for all inputs.

Completion Checklist

Confidence / mastery rating
Personal notes

Source References

This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.

← 17.1Next: 17.3 →