How to Tell If Your Trading Strategy Is Overfit: Seven Checks

An overfit strategy is one whose rules memorised the accidents of a particular history instead of capturing something that repeats. It looks brilliant on the data that built it and dies on data it has not seen. Overfitting does not feel like cheating from the inside; it feels like refinement, which is why it is the default failure mode of systematic trading. Seven checks that catch it.

1. Count the decisions, not just the parameters

Every choice fitted to the data is a parameter: indicator settings, but also which instrument you kept, which years you included, which sessions you excluded, which stop style survived. Ten such decisions on a few hundred trades is a memorisation engine. If listing your choices takes longer than stating your thesis, the backtest is describing your choices, not the market.

2. Check the sample per condition, not the total

A 500-trade backtest sliced by session, volatility and setup becomes dozens of small cells, and with enough small cells, luck produces perfection somewhere. Any condition with fewer than about 30 trades is an anecdote, and below 5 it is nothing. A result that only appears in thin slices is the signature of selection, not edge.

3. Ask where the money is concentrated

Remove the best single trade, then the best month. If the sign flips, the strategy is one event wearing a track record. Real edges are distributed; overfit ones are usually a handful of lucky prints plus noise.

4. Nudge the parameters

Shift each parameter a step in either direction and re-run. A robust edge degrades gently; an overfit one collapses, because the original values were chosen precisely for sitting on a spike in the noise. If 20 and 22 lose money while 21 prints, 21 is not a discovery, it is an address.

5. Honour a hard out-of-sample wall

Hold back the most recent segment, draw every conclusion from the rest, then evaluate once. The wall only works if information never flows backwards: test, tweak, retest is an in-sample fit with extra steps. The frozen-verdict version of this method, applied to 11,888 real trades with the failures published, is in our case study.

6. Account for how many things you tried

If you tested twenty ideas and one passed, the relevant question is how often noise passes your test, because that is what you measured. Selection across candidates overfits at the portfolio level even when each individual test looks clean.

7. Verify the arithmetic before any of it

Reconcile total P&L against an independent sum of exit minus entry times size, computed outside your engine. Every robustness check is downstream of the accounting, and a broken calculation passes all of them with confidence.

The pattern across all seven: overfitting is not detected by admiring results, it is detected by trying to break them. A strategy that survives deliberate breaking attempts has earned something. One that has only survived being admired has earned nothing yet.