Cointegration, VAR, and Backtest Pitfalls
Modeling several series together, the long-run equilibrium of cointegrated pairs, and the research-integrity traps that turn great backtests into losing strategies.
Leads to: These pitfalls govern every strategy in Phases 14-18; this is the capstone of quant-research discipline.
Learning Objectives
Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.
- Write a vector autoregression (VAR) and state its stationarity condition.
- Define cointegration and the error-correction representation of a cointegrated system.
- Distinguish a genuine cointegrating relationship from a spurious regression.
- Explain look-ahead bias, backtest overfitting, and regime change with concrete mechanisms.
- Design a backtest that is robust to leakage, multiple testing, and structural breaks.
Key Vocabulary
- VAR(\(p\)
- Vector autoregression: \(x_t=c+\sum_{i=1}^p \Phi_i x_{t-i}+\varepsilon_t\) for a vector \(x_t\) of series.
- Cointegration
- Two or more \(I(1)\) series whose linear combination is \(I(0)\) (stationary): a shared stochastic trend / long-run equilibrium.
- Error-correction model
- A representation where changes respond to the lagged equilibrium error: \(\Delta y_t=\alpha(y_{t-1}-\beta x_{t-1})+\dots\).
- Spurious regression
- A high-\(R^2\), significant-looking regression between independent \(I(1)\) series - an artifact of common trends, not a real relationship.
- Look-ahead bias
- Using information not available at decision time; the most common way backtests cheat.
- Backtest overfitting
- Selecting a strategy from many trials so its performance is largely luck; apparent Sharpe inflated by the search.
- Regime change / structural break
- A shift in the data-generating process (policy, liquidity, crisis) that breaks a previously fitted relationship.
Intuition & Motivation
The other half of this lesson is a warning. Cointegration is easy to fake: two unrelated trending series will show a gorgeous, highly significant regression - a spurious regression - that means nothing. And even a real edge is trivially destroyed in a backtest by using tomorrow’s data today (look-ahead bias), by trying a thousand variants and reporting the winner (backtest overfitting), or by assuming the world does not change (regime shifts). Getting the statistics right is necessary; getting the research process right is what actually protects your capital.
Vector autoregression (VAR)
Stack \(d\) series into \(x_t\in\R^d\). A VAR(\(p\)) is \(x_t=c+\Phi_1 x_{t-1}+\dots+\Phi_p x_{t-p}+\varepsilon_t\) with \(\varepsilon_t\) vector white noise. It is stationary iff all roots of \(\det(I-\Phi_1 z-\dots-\Phi_p z^p)=0\) lie outside the unit circle. VARs are estimated equation-by-equation by OLS and used for multivariate forecasting, impulse-response analysis, and Granger-causality tests - but they need stationary inputs.
Cointegration and error correction
If each component of \(x_t\) is \(I(1)\) but there exists a vector \(\beta\) with \(\beta^\top x_t\) stationary (\(I(0)\)), the series are cointegrated: they share a common stochastic trend and cannot drift arbitrarily far apart. Granger's representation theorem says such a system has an error-correction form:
where \((y_{t-1}-\beta x_{t-1})\) is the lagged equilibrium error (the spread) and \(\alpha\lt 0\) is the speed of adjustment pulling the system back. Test for it with Engle-Granger (regress, then unit-root-test the residual spread) or Johansen (a systems test allowing several cointegrating vectors). This is the engine of statistical-arbitrage pairs trading: the spread is your tradable, mean-reverting signal.
Spurious regression: the trap that looks like a discovery
Regress one independent random walk on another. Because both trend, OLS reports a large \(R^2\) and a \(t\)-statistic that looks wildly significant - yet there is no relationship. The classical inference is invalid because the residuals are themselves \(I(1)\) (non-stationary), violating Gauss-Markov. The tell: a very high \(R^2\) with a Durbin-Watson near 0 (hugely autocorrelated residuals). Genuine cointegration requires the residual spread to be stationary; that is the test that separates a real tether from a coincidence of trends.
The backtest pitfalls, made explicit
Interactive: spurious regression between two random walks
- Running a VAR or regression on non-stationary levels and trusting the \(t\)-stats - spurious significance; difference or model in error-correction form.
- Estimating the cointegration vector \(\beta\) on the whole sample, then backtesting the in-sample spread - look-ahead bias.
- Reporting the best of many pairs/strategies with no trial-count adjustment - overfit Sharpe (eq. 11.17).
- Assuming a cointegrating relationship is permanent - pairs de-cohere after mergers, index changes, and regime shifts.
- Backtesting on a survivorship-biased universe or with same-bar execution - both silently manufacture returns (11.7).
- Confirm each leg is \(I(1)\) and the spread is \(I(0)\) before calling it cointegration; a stationary residual is the whole test.
- Estimate \(\beta\) and signals on a rolling window that ends strictly at the decision time; re-estimate as new data arrive.
- Report deflated Sharpe / trial count, keep an untouched out-of-time set, and prefer signals with an economic rationale.
- Stress-test across regimes and add break tests; size positions so a broken relationship is survivable, not fatal.
- Trade the spread, not the levels - and put a stop on the spread in case the cointegration itself breaks.
Knowledge Check
Practical Exercise
You build a stat-arb pairs strategy: pick the 5 best-cointegrated pairs from 500 stocks over 2005-2015, estimate each hedge ratio on the full window, trade the spreads, and report a backtest Sharpe of 3.0. (a) Identify every pitfall. (b) Explain each mechanism. (c) Redesign the study honestly and state what an honest Sharpe estimate would require.
(a) Look-ahead bias (full-window hedge ratios), backtest overfitting / multiple testing (5 best of \(\binom{500}{2}\approx125{,}000\) candidate pairs), survivorship bias (a fixed 2005-2015 universe of names that existed throughout), and regime risk (one decade, no out-of-regime test).
(b) Full-window \(\beta\) leaks future price relationships into early trades; screening ~125,000 pairs and keeping the top 5 is an extreme maximum-over-noise, so the reported Sharpe is dominated by luck (the \(\sqrt{2\log N}/\sqrt T\) inflation is huge here); a survivorship universe drops delisted/merged names, and cointegration often breaks because of the merger you excluded; a single regime hides fragility to structural breaks.
(c) Use a point-in-time, delisting-inclusive universe; select pairs and estimate hedge ratios on a rolling in-sample window and trade only the subsequent out-of-sample window (walk-forward, with purge/embargo); reserve a final untouched out-of-time period (e.g. 2016 onward) never used in selection; report the number of pairs screened and a deflated Sharpe ratio; and add break tests plus spread stop-losses. The honest number is whatever survives on the untouched out-of-time data after deflation - likely far below 3.0, and that lower figure is the one to trust or trade.
Lesson Summary
Formula Sheet Additions
- Did I confirm each leg is \(I(1)\) and the spread \(I(0)\) (real cointegration, not spurious)?
- Is every parameter estimated on a strictly backward-looking window (no look-ahead)?
- Did I count trials and report a deflated Sharpe, with an untouched out-of-time set?
- Did I test across regimes and add break tests / spread stops?
- Is the universe point-in-time and delisting-inclusive (no survivorship)?
Retrieval Practice
Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.
A: Non-stationary \(I(1)\) series with a stationary linear combination \(\beta^\top x_t\) (a common trend); the stationary spread mean-reverts and is the pairs-trade signal.
A: Spurious: high \(R^2\) but \(I(1)\) (non-stationary) residuals and near-zero Durbin-Watson. Cointegration: the residual spread is stationary \(I(0)\) (Engle-Granger/Johansen).
A: Backward-looking (no look-ahead) estimation; trial-count deflation of Sharpe (no overfitting); cross-regime out-of-time validation with re-estimation (no false permanence).
Flashcards
Click to flip. These feed the site-wide spaced-repetition queue.
Completion Checklist
- I can explain the core ideas in my own words
- I worked the derivations/examples by hand
- I completed the interactive workbench(es)
- I passed the knowledge check
Source References
This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.
- Time Series Analysis (James Hamilton, 1994) foundational - Hamilton 11,18-19 - Ch. 11 (VAR), 18-19 (unit roots, cointegration, spurious regression, error correction).
- The Elements of Statistical Learning (Hastie, Tibshirani & Friedman, 2nd ed.) current - ESL 7 - Out-of-sample validation, multiple testing, and the optimism of selected models.