Quant Academy Math · Probability · Quant Finance
Phase 18 - Lesson 18.8

Capstone 8 - Final Quant Research Project

The full arc: a research question turned into a reproducible, bias-controlled, cost-aware, risk-analyzed study with an honest executive summary.

⏱ 8+ hrs● Research🔗 Prereqs: All prior phases; Capstones 1–7
↖ Phase 18 hub
Builds on: Integrates every phase: foundations (1–6), probability/stochastics (7–9), optimization/learning (10–13), finance (14–16), and engineering (17).
Leads to: This is the terminal deliverable of the academy - the artifact you would show to demonstrate quant competence.

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

Key Vocabulary

Research question
A precise, falsifiable statement of what you are testing and under what conditions.
Source map
An explicit mapping from each method used to the phase/book that justifies it.
Baseline
A simple benchmark the model must beat to be worth its complexity (e.g. buy-and-hold, zero-signal).
Bias controls
Procedures that remove survivorship, look-ahead, and multiple-testing bias from the study.
Risk analysis
Quantifying drawdown, tail risk, exposures, and scenario behavior of the result.
Executive summary
A concise, honest statement of question, method, result, uncertainty, and limitations.

Intuition & Motivation

Intuition
This capstone is not a new technique - it is the assembly of everything into one honest artifact. A quant research project is judged less by whether the answer is exciting than by whether it is trustworthy: could a skeptical peer reproduce it, are the biases controlled, are the costs real, is the risk understood, and does the summary claim exactly what the evidence supports - no more.

The single most valuable habit you can demonstrate is calibrated honesty. Most research questions, tested properly, yield a null or marginal result. Reporting that clearly - with reproducible code, out-of-sample net-of-cost numbers, and stated uncertainty - is worth more than a spectacular backtest that quietly leaked the future. Never promise profitability.

The brief: a complete research report

Choose a concrete, testable question (examples below) and produce a report plus reproducible code covering the fourteen required components. The report must let an informed reader judge the claim and a peer regenerate every number.

Required components (all must appear)

  1. Research question - precise and falsifiable.
  2. Mathematical rationale - why the question is well-posed and what theory motivates it.
  3. Source map - each method mapped to its phase/book (e.g. ‘walk-forward CV: Phase 12; ridge: Phase 11/ESL’).
  4. Data description - source, period, frequency, universe, and how it was snapshotted (point-in-time).
  5. Assumptions - stated explicitly (costs, execution, stationarity, no dividends, etc.).
  6. Reproducible code - seeded, version-controlled, one command regenerates results (17.3).
  7. Baseline - the simple benchmark the model must beat.
  8. Model - the method, its hyperparameters, and how they were chosen (on training folds only).
  9. Validation - out-of-sample, time-aware, with standard errors (Capstone 5).
  10. Bias controls - survivorship, look-ahead, and multiple-testing all addressed.
  11. Transaction costs - realistic, with a sensitivity curve (Capstones 5–6).
  12. Risk analysis - drawdown, tail behavior, exposures, scenario stress.
  13. Results - net, out-of-sample, with uncertainty; honest about significance.
  14. Limitations & executive summary - what the study does not show, then a concise action-oriented summary.

From question to rationale to source map

A good question is narrow and falsifiable: not ‘does momentum work?’ but ‘does a 12-1 month cross-sectional momentum signal, on the liquid US large-cap universe, 2005–2020, deliver positive net-of-cost out-of-sample returns after controlling for market exposure?’ The mathematical rationale states why you’d expect signal (e.g. autocorrelation structure, risk-based or behavioral hypotheses) and what would falsify it. The source map makes the intellectual lineage explicit and auditable.

ComponentDraws onWhat it delivers
Estimation / least squaresCapstone 1, Phase 4–5stable, validated fits
Probability & simulationCapstones 2–3, Phase 7–13null distributions, MC error bars
Validation & costsCapstone 5, Phase 11–12OOS, leakage-free, net-of-cost
Execution & microstructureCapstones 6–7, Phase 16realistic cost of trading the signal
EngineeringPhase 17reproducible, tested, modular code

The honesty scaffold: baseline, bias, cost, risk

Four checks separate a research project from a story:

Worked Example - A reference report skeleton and grading rubric
1
§1 Question & rationale: one paragraph, falsifiable; state the null.
2
§2 Data & assumptions: source, period, universe (point-in-time), costs, what’s excluded.
3
§3 Method & source map: baseline, model, hyperparameter selection on training folds; each choice cited to a phase.
4
§4 Validation & bias controls: walk-forward design (18.11), leakage audit, multiple-testing note.
5
§5 Results: OOS net returns with SE, cost-sensitivity curve, comparison to baseline.
6
§6 Risk: drawdown, CVaR, exposures, scenario stress.
7
§7 Limitations & executive summary: what it does not show; then the 5-sentence summary.
8
Rubric (weightings): reproducibility 20%, bias control 20%, cost realism 15%, validation rigor 15%, risk analysis 15%, clarity/honesty 15%. A brilliant result with leakage scores near zero.
▶ Executive-summary template (fill honestly)
Question: <the falsifiable question> Method: <baseline vs model, validation scheme, cost model> Result: <OOS net metric> +/- <SE> vs baseline <metric> Evidence: <is it distinguishable from the null? how many variants tried?> Risk: <max drawdown, tail, key exposure> Limits: <what this does NOT establish; regime/period caveats> Claim: <exactly what the evidence supports -- no profit promise>

If the honest result is ‘no reliable edge net of cost out-of-sample’, that is a complete and valuable answer. Fabricating a positive claim is the one unforgivable error.

Common Mistakes to Avoid
  • Picking a vague, unfalsifiable question so any result can be spun as success.
  • Building the universe from today’s survivors (survivorship bias) - inflates every historical return.
  • Leaking the future via full-sample scaling or test-set tuning (look-ahead bias).
  • Reporting gross, in-sample, or single-variant-best numbers as if they were out-of-sample evidence.
  • Omitting transaction costs or presenting a single optimistic cost point.
  • Reporting return without risk (drawdown, tail) or without a standard error.
  • Promising profitability - overclaiming beyond what leakage-free, net, out-of-sample evidence supports.
Quant Practitioner Tips
  • Write the executive summary and limitations first, in pencil; they force honesty about what you can actually show.
  • Make one command regenerate every figure from a fixed seed and a snapshotted dataset (17.3).
  • Include a leakage audit as an explicit section - a reviewer will look for it.
  • Always compare to a baseline and report OOS net-of-cost results with uncertainty.
  • Report the number of strategies/variants tried; discount the headline accordingly.
  • State the risk (drawdown, CVaR, exposures) alongside every return number.
  • If the answer is null, say so clearly - a well-executed null is a strong deliverable.

Interactive: assemble and validate the report’s integrity checks

Rather than a single model, this check verifies the discipline: a leakage-free split, a net-of-cost result vs. baseline, and a completeness check that all fourteen required components are present.

Knowledge Check

Q1 Medium
Which deliverable would a skeptical reviewer weight most heavily when judging a quant research project?
A high in-sample Sharpe
Reproducible, leakage-free, net-of-cost out-of-sample results with stated uncertainty
An elegant model
A large number of features
Q2 Hard
If your properly-validated study finds no reliable edge net of costs out-of-sample, the correct action is:
Keep tuning until it looks profitable
Report the null result clearly - it is a complete and valuable answer
Drop the transaction costs
Report the best of the 50 variants you tried
Q3 Medium
A baseline (e.g. buy-and-hold) is required in the report because:
It is traditional
A complex model must demonstrably beat a simple benchmark net of cost to justify its complexity
It guarantees profit
It removes the need for validation

Practical Exercise

Draft the full skeleton of your final project for a specific question of your choice (e.g. ‘Does a low-volatility factor earn positive net-of-cost out-of-sample returns on US large caps, 2005–2020?’). Provide one to two sentences for each of the fourteen required components, and make the executive summary honest about uncertainty. You may assume a null or marginal result.

▶ Show full solution

A defensible skeleton (illustrative; a null result is fully acceptable):

  1. Question: Does a low-volatility factor deliver positive net-of-cost OOS returns on liquid US large caps, 2005–2020, after controlling for market exposure?
  2. Rationale: The low-volatility anomaly is documented; hypothesized from leverage constraints/behavioral biases. Null: no excess return net of costs.
  3. Source map: factor construction (Phase 11/ESL), walk-forward validation (Phase 12/Capstone 5), costs (Capstone 6), risk (Phase 14).
  4. Data: daily prices, 2005–2020, point-in-time large-cap universe reconstructed monthly, snapshotted with a content hash.
  5. Assumptions: 10 bps round-trip cost, monthly rebalance, no leverage, dividends reinvested, no shorting constraints beyond borrow.
  6. Code: seeded, version-controlled; make report regenerates all figures.
  7. Baseline: cap-weighted buy-and-hold of the same universe.
  8. Model: rank stocks by trailing 1-yr volatility, long low-vol quintile, market-neutralized; no test-set tuning.
  9. Validation: walk-forward, expanding window; report OOS net returns with SE.
  10. Bias controls: survivorship-free universe, PIT features, leakage audit, note that 6 factor variants were examined.
  11. Transaction costs: net returns across 5–25 bps; sensitivity curve included.
  12. Risk: max drawdown, 5% CVaR, market beta, sector exposures, 2008/2020 stress.
  13. Results: e.g. OOS net excess return \(0.8\%\pm1.5\%\)/yr - not distinguishable from zero after costs.
  14. Limitations & executive summary: single market/period, cost model simplified, factor definition sensitive; Summary: ‘On this universe and period, the low-vol factor’s net-of-cost OOS excess return is statistically indistinguishable from zero; we do not find evidence of a tradeable edge after realistic costs, and make no profitability claim.’

Note how the summary states exactly what the evidence supports and no more - the hallmark of trustworthy quant research.

After the reveal, answer for yourself: Which two components, if omitted, would most damage a reviewer’s trust in your conclusion, and why?

Lesson Summary

The final capstone assembles the whole academy into one trustworthy artifact: a falsifiable research question with mathematical rationale and a source map, a point-in-time data description and explicit assumptions, reproducible code, a baseline, a model, time-aware validation, bias controls, realistic transaction costs, a risk analysis, honest results with uncertainty, and limitations plus an executive summary. Trust, not excitement, is the deliverable - and a well-executed null result is a complete answer. Never promise profitability.

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers
Q: List at least six of the fourteen required components of the final research project.
A: Any six of: research question, mathematical rationale, source map, data description, assumptions, reproducible code, baseline, model, validation, bias controls, transaction costs, risk analysis, results, limitations/executive summary.
Q: Why is a well-executed null result an acceptable, even valuable, deliverable?
A: Because most properly-tested research questions yield null or marginal results; honestly reporting a leakage-free, net-of-cost, out-of-sample null - with reproducible code and stated uncertainty - is trustworthy and informative, whereas a spectacular backtest that leaked the future is worthless. Calibrated honesty is the graded skill.

Completion Checklist

Confidence / mastery rating
Personal notes

Source References

This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.

← 18.7Next: 19.1 →