Cointegration, VAR, and Backtest Pitfalls
Modeling several series together, the long-run equilibrium of cointegrated pairs, and the research-integrity traps that turn great backtests into losing strategies.
Leads to: These pitfalls govern every strategy in Phases 14-18; this is the capstone of quant-research discipline.
Learning Objectives
Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.
- Write a vector autoregression (VAR) and state its stationarity condition.
- Define cointegration and the error-correction representation of a cointegrated system.
- Distinguish a genuine cointegrating relationship from a spurious regression.
- Explain look-ahead bias, backtest overfitting, and regime change with concrete mechanisms.
- Design a backtest that is robust to leakage, multiple testing, and structural breaks.
Key Vocabulary
- VAR(\(p\)
- Vector autoregression: \(x_t=c+\sum_{i=1}^p \Phi_i x_{t-i}+\varepsilon_t\) for a vector \(x_t\) of series.
- Cointegration
- Two or more \(I(1)\) series whose linear combination is \(I(0)\) (stationary): a shared stochastic trend / long-run equilibrium.
- Error-correction model
- A representation where changes respond to the lagged equilibrium error: \(\Delta y_t=\alpha(y_{t-1}-\beta x_{t-1})+\dots\).
- Spurious regression
- A high-\(R^2\), significant-looking regression between independent \(I(1)\) series - an artifact of common trends, not a real relationship.
- Look-ahead bias
- Using information not available at decision time; the most common way backtests cheat.
- Backtest overfitting
- Selecting a strategy from many trials so its performance is largely luck; apparent Sharpe inflated by the search.
- Regime change / structural break
- A shift in the data-generating process (policy, liquidity, crisis) that breaks a previously fitted relationship.
Intuition & Motivation
The other half of this lesson is a warning. Cointegration is easy to fake: two unrelated trending series will show a gorgeous, highly significant regression - a spurious regression - that means nothing. And even a real edge is trivially destroyed in a backtest by using tomorrow’s data today (look-ahead bias), by trying a thousand variants and reporting the winner (backtest overfitting), or by assuming the world does not change (regime shifts). Getting the statistics right is necessary; getting the research process right is what actually protects your capital.
Vector autoregression (VAR)
Stack \(d\) series into \(x_t\in\R^d\). A VAR(\(p\)) is \(x_t=c+\Phi_1 x_{t-1}+\dots+\Phi_p x_{t-p}+\varepsilon_t\) with \(\varepsilon_t\) vector white noise. It is stationary iff all roots of \(\det(I-\Phi_1 z-\dots-\Phi_p z^p)=0\) lie outside the unit circle. VARs are estimated equation-by-equation by OLS and used for multivariate forecasting, impulse-response analysis, and Granger-causality tests - but they need stationary inputs.
Cointegration and error correction
If each component of \(x_t\) is \(I(1)\) but there exists a vector \(\beta\) with \(\beta^\top x_t\) stationary (\(I(0)\)), the series are cointegrated: they share a common stochastic trend and cannot drift arbitrarily far apart. Granger's representation theorem says such a system has an error-correction form:
where \((y_{t-1}-\beta x_{t-1})\) is the lagged equilibrium error (the spread) and \(\alpha\lt 0\) is the speed of adjustment pulling the system back. Test for it with Engle-Granger (regress, then unit-root-test the residual spread) or Johansen (a systems test allowing several cointegrating vectors). This is the engine of statistical-arbitrage pairs trading: the spread is your tradable, mean-reverting signal.
Spurious regression: the trap that looks like a discovery
Regress one independent random walk on another. Because both trend, OLS reports a large \(R^2\) and a \(t\)-statistic that looks wildly significant - yet there is no relationship. The classical inference is invalid because the residuals are themselves \(I(1)\) (non-stationary), violating Gauss-Markov. The tell: a very high \(R^2\) with a Durbin-Watson near 0 (hugely autocorrelated residuals). Genuine cointegration requires the residual spread to be stationary; that is the test that separates a real tether from a coincidence of trends.
The backtest pitfalls, made explicit
Interactive: spurious regression between two random walks
- Running a VAR or regression on non-stationary levels and trusting the \(t\)-stats - spurious significance; difference or model in error-correction form.
- Estimating the cointegration vector \(\beta\) on the whole sample, then backtesting the in-sample spread - look-ahead bias.
- Reporting the best of many pairs/strategies with no trial-count adjustment - overfit Sharpe (eq. 11.17).
- Assuming a cointegrating relationship is permanent - pairs de-cohere after mergers, index changes, and regime shifts.
- Backtesting on a survivorship-biased universe or with same-bar execution - both silently manufacture returns (11.7).
- Confirm each leg is \(I(1)\) and the spread is \(I(0)\) before calling it cointegration; a stationary residual is the whole test.
- Estimate \(\beta\) and signals on a rolling window that ends strictly at the decision time; re-estimate as new data arrive.
- Report deflated Sharpe / trial count, keep an untouched out-of-time set, and prefer signals with an economic rationale.
- Stress-test across regimes and add break tests; size positions so a broken relationship is survivable, not fatal.
- Trade the spread, not the levels - and put a stop on the spread in case the cointegration itself breaks.
Knowledge Check
Practical Exercise
You build a stat-arb pairs strategy: pick the 5 best-cointegrated pairs from 500 stocks over 2005-2015, estimate each hedge ratio on the full window, trade the spreads, and report a backtest Sharpe of 3.0. (a) Identify every pitfall. (b) Explain each mechanism. (c) Redesign the study honestly and state what an honest Sharpe estimate would require.
(a) Look-ahead bias (full-window hedge ratios), backtest overfitting / multiple testing (5 best of \(\binom{500}{2}\approx125{,}000\) candidate pairs), survivorship bias (a fixed 2005-2015 universe of names that existed throughout), and regime risk (one decade, no out-of-regime test).
(b) Full-window \(\beta\) leaks future price relationships into early trades; screening ~125,000 pairs and keeping the top 5 is an extreme maximum-over-noise, so the reported Sharpe is dominated by luck (the \(\sqrt{2\log N}/\sqrt T\) inflation is huge here); a survivorship universe drops delisted/merged names, and cointegration often breaks because of the merger you excluded; a single regime hides fragility to structural breaks.
(c) Use a point-in-time, delisting-inclusive universe; select pairs and estimate hedge ratios on a rolling in-sample window and trade only the subsequent out-of-sample window (walk-forward, with purge/embargo); reserve a final untouched out-of-time period (e.g. 2016 onward) never used in selection; report the number of pairs screened and a deflated Sharpe ratio; and add break tests plus spread stop-losses. The honest number is whatever survives on the untouched out-of-time data after deflation - likely far below 3.0, and that lower figure is the one to trust or trade.
Lesson Summary
Formula Sheet Additions
- Did I confirm each leg is \(I(1)\) and the spread \(I(0)\) (real cointegration, not spurious)?
- Is every parameter estimated on a strictly backward-looking window (no look-ahead)?
- Did I count trials and report a deflated Sharpe, with an untouched out-of-time set?
- Did I test across regimes and add break tests / spread stops?
- Is the universe point-in-time and delisting-inclusive (no survivorship)?
Retrieval Practice
Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.
A: Non-stationary \(I(1)\) series with a stationary linear combination \(\beta^\top x_t\) (a common trend); the stationary spread mean-reverts and is the pairs-trade signal.
A: Spurious: high \(R^2\) but \(I(1)\) (non-stationary) residuals and near-zero Durbin-Watson. Cointegration: the residual spread is stationary \(I(0)\) (Engle-Granger/Johansen).
A: Backward-looking (no look-ahead) estimation; trial-count deflation of Sharpe (no overfitting); cross-regime out-of-time validation with re-estimation (no false permanence).
Flashcards
Click to flip. These feed the site-wide spaced-repetition queue.
Completion Checklist
- I can explain the core ideas in my own words
- I worked the derivations/examples by hand
- I completed the interactive workbench(es)
- I passed the knowledge check