Performance: Profiling, Vectorization, and Introductory C++ Concepts
Measure before you optimize. Then vectorize the hot path - and understand why a compiled, statically-typed language like C++ is faster when you finally need it.
Leads to: Latency-sensitive execution and market-making (Phase 16, capstones 6–7) rely on the performance mindset here.
Learning Objectives
Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.
- Explain why you must profile before optimizing and identify the hot path from a profile.
- Compute the speedup bound implied by Amdahl’s law from the fraction of time in the hot path.
- Implement a vectorized replacement for a hot loop and measure the speedup.
- Explain why a compiled, statically-typed language (C++) executes numeric code faster than interpreted Python.
- Describe when to drop from Python to C/C++ (or Numba/Cython) and the correctness cost of doing so.
Key Vocabulary
- Profiling
- Measuring where a program actually spends time (per function/line) rather than guessing.
- Hot path
- The small fraction of code that consumes most of the runtime; the only place optimization pays.
- Amdahl’s law
- The bound on total speedup when you accelerate only part of a program: limited by the un-accelerated fraction.
- Static typing
- Types fixed and known at compile time (as in C++), enabling the compiler to emit specialized machine code.
- Compilation
- Translating source to machine code ahead of run time, so no interpreter overhead is paid per operation.
- Cache locality
- Data laid out contiguously so the CPU cache is used well; a major reason contiguous arrays beat pointer-chasing.
Intuition & Motivation
Once you know the hot path, the Python answer is usually vectorization (17.1): move the inner loop into compiled C. When even that isn’t enough - a genuinely serial, latency-critical inner loop - you drop to a compiled, statically-typed language. Understanding why C++ is faster (no interpreter, fixed types, cache-friendly layout) tells you when the jump is worth its considerable correctness and complexity cost.
Measure first: profiling and Amdahl’s law
A profiler attributes runtime to functions or lines. Before touching code, you profile, find the hot path, and estimate the payoff. If a section takes fraction \(p\) of total time and you speed it up by factor \(s\), the overall speedup is bounded by Amdahl’s law:
If the hot path is 90% of runtime (\(p=0.9\)) and you make it infinitely fast (\(s\to\infty\)), you can at most get \(1/(1-0.9)=10\times\). If it’s only 20% of runtime, no amount of optimization beats \(1.25\times\). This is why you optimize the hot path and ignore the rest.
compute_signal (a Python loop over dates), 9% in I/O, 9% in plotting.compute_signal; leave plotting alone. Re-profile after - the hot path often moves.Vectorization as the first-line optimization
For array math, a vectorized rewrite typically buys 50–200× on the hot loop for near-zero risk, because it is the same algorithm expressed as whole-array operations (17.1). Always try this before reaching for another language: it is faster to write, easier to test, and stays in one codebase.
Why C++ is faster: types, compilation, layout
Python is interpreted and dynamically typed. Every a + b asks at run time: what types are these? where are their methods? - then boxes the result as an object. C++ is compiled and statically typed: the compiler already knows a and b are doubles and emits a single machine instruction. Three concrete reasons a C++ inner loop outruns a Python one:
- No interpreter overhead. Machine code runs directly; there is no per-operation bytecode dispatch or type check.
- Static types. Knowing everything is a
doublelets the compiler specialize and use CPU vector (SIMD) instructions. - Cache-friendly memory. A
std::vector<double>is contiguous, like a NumPy array, so the CPU cache is used well instead of chasing pointers to boxed Python objects.
| Trait | Python (CPython) | C++ |
|---|---|---|
| Typing | dynamic, checked at run time | static, checked at compile time |
| Execution | interpreted bytecode | compiled machine code |
| Numeric loop | boxed objects, interpreter overhead | raw doubles, SIMD-able |
| Dev speed / safety | fast to write, memory-managed | slower, manual memory, easy to crash |
- Optimizing before profiling - you speed up the 9% and the program is still slow (Amdahl).
- Rewriting in C++ when a one-line vectorization would have given the same speedup with far less risk.
- Micro-optimizing readability away on the cold path, where it buys nothing and costs clarity.
- Assuming ‘C++ is always faster’ while calling it in tiny chunks - the Python↔C boundary crossing dominates.
- Trading correctness for speed: manual memory management and undefined behavior in C++ introduce bugs vectorized Python never had.
- Forgetting to re-profile after an optimization; the hot path moves and your next guess is again wrong.
- Profile, optimize the hot path, re-profile. Repeat until the profile is flat or fast enough.
- Use Amdahl’s law to decide if an optimization is even worth attempting before you start.
- Vectorize first (NumPy). Consider Numba/Cython next; reach for C++ only for serial latency-critical loops.
- When you do go to compiled code, keep the pure-Python version as an oracle and test them against each other (17.2).
- Measure wall-clock on realistic input sizes; asymptotic wins can lose to constant factors at small N.
Interactive: profile-motivated vectorization with a speedup check
The starter has a slow loop (the hot path) and asks for a vectorized version that returns identical numbers. The tests check equivalence; conceptually, this is the \(50\text{-}200\times\) win the profile predicts.
Knowledge Check
Practical Exercise
A pricing library spends, per a profile, 70% of its time in a Monte Carlo path loop, 20% in payoff evaluation, and 10% in reporting. (a) You can vectorize the path loop for a 100× speedup on that section. What is the overall speedup by Amdahl’s law? (b) After that change, re-profile in words: which section is now the new hot path? (c) When, if ever, would you rewrite the path loop in C++ instead of vectorizing?
(a) \(p=0.70,\ s=100\): overall \(=1/((1-0.70)+0.70/100)=1/(0.30+0.007)=1/0.307\approx3.26\times\).
(b) Originally path=70, payoff=20, report=10 (relative). After the 100× win the path costs \(0.70/100=0.007\), so relative shares become payoff \(0.20\), report \(0.10\), path \(0.007\) - total \(0.307\). The payoff evaluation (\(0.20/0.307\approx65\%\)) is now the hot path; re-profiling and vectorizing it is the next move.
(c) Rewrite the path loop in C++ (or Numba) only if it is genuinely serial and cannot be vectorized (e.g. path-dependent early exercise with per-step branching), it dominates runtime, and the vectorized/NumPy version is still too slow for the latency budget - and even then keep the Python version as a correctness oracle.
Lesson Summary
Retrieval Practice
Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.
A: \(\text{speedup}=1/((1-p)+p/s)\), bounded above by \(1/(1-p)\) as \(s\to\infty\), where \(p\) is the time fraction in the optimized section.
A: No per-operation interpreter overhead (machine code runs directly) and static types on unboxed values let the compiler specialize and use SIMD; contiguous memory also improves cache locality.
Completion Checklist
- I can explain the core ideas in my own words
- I worked the derivations/examples by hand
- I completed the interactive workbench(es)
- I passed the knowledge check