Hard wall
Verdicts frozen on trades to 2025-05-16. Everything tested after is a market the classifier never saw.
01
Splitting the book by time, not by luck
The first 8,321 trades are the learning half; the last 3,567 are held back as a live test. On the learning half alone, the audit grouped every trade by the conditions it was taken in,
strategy | session | volatility | RSI | trend, and issued a verdict per group. Those verdicts were then locked.0
STRONG
proven edge
1
PROMISING
building a case
83
NO VERDICT YET
too few trades, or nothing either way
19
LOST MONEY
lost consistently
Read the first cell again. Across 8,321 trades and nearly three years, not one set of conditions cleared the bar for a proven edge at any point, on any slice of the data. The proven-edge column is empty and stays empty.
02
What the verdicts were worth in money
Three ways to trade the held-back window: take everything, take everything except the groups that lost money, or take only those groups. Identical trades and fills throughout. The only variable is whether you listened to a label written years earlier.
Cumulative P&L on held-back trades, 2025-05-16 to 2026-07-05
every point is a real trade, in sequence
Skip the losing groups −$4,965Everything −$19,393Only the losing groups −$14,428
03
Every way you could have traded it
The obvious follow-up: if the audit sorts conditions by quality, what if you had traded only the good ones? Here is every filter applied to the same held-back window. The answer to whether any group was strong is the one that matters, and it is not a number.
| Strategy in the test window | trades | net | $/trade | win% |
|---|---|---|---|---|
| Take every signalno filter | 3,567 | −$19,393 | -5.44 | 52.4% |
| Skip the losing groupsthe frozen losing conditions left out | 1,691 | −$4,965 | -2.94 | 54.9% |
| Strong onlytrade only proven-edge conditions | 0 | no trades | – | – |
| PROMISING onlyconditions still building a case | 3 | +$29 | +9.59 | 33.3% |
| No verdict yet onlyno verdict yet | 1,687 | −$4,990 | -2.96 | 55.0% |
| Losing groups onlywhat leaving them out removes | 1,876 | −$14,428 | -7.69 | 50.1% |
The strong row is empty because no condition was ever strong. Nothing in 11,888 trades ever cleared the bar for a proven edge, so that row has no trades to show. It is not bad luck in one window; it is the shape of the whole book. PROMISING fired 3 trades in thirteen months, too few to conclude anything from, and we are not going to pretend otherwise. Every filter that produced a meaningful number produced a negative one.
04
Every flagged condition, checked one by one
The 84% headline could still hide a single lucky group, so here is the full working: every losing condition with enough fresh trades, its average before the wall and after it. Sixteen kept bleeding. Three did not, and they are shown in place rather than dropped.
84%
16 of 19 conditions flagged as lost money stayed net-negative on trades that did not exist when the verdict was written.
| Condition flagged as lost money | learn $/t | test $/t | n | |
|---|---|---|---|---|
| ASIA|HIGH|30_70|FLAT | -12.94 | -18.38 | 218 → 97 | held |
| NEW_YORK_PM|HIGH|30_70|FLAT | -19.52 | -18.07 | 187 → 88 | held |
| NEW_YORK_AM|MEDIUM|30_70|FLAT | -25.91 | -16.72 | 266 → 100 | held |
| LONDON|HIGH|30_70|FLAT | -22.41 | -15.07 | 99 → 38 | held |
| NEW_YORK_PM|LOW|30_70|FLAT | -16.41 | -14.50 | 148 → 48 | held |
| NEW_YORK_AM|MEDIUM|30_70|DOWN | -19.28 | -11.53 | 97 → 63 | held |
| AFTER_HOURS|MEDIUM|30_70|FLAT | -10.84 | -11.16 | 408 → 190 | held |
| AFTER_HOURS|HIGH|30_70|FLAT | -12.45 | -10.38 | 177 → 70 | held |
| ASIA|LOW|30_70|UP | -5.02 | -9.61 | 153 → 38 | held |
| ASIA|LOW|30_70|FLAT | -4.79 | -8.47 | 353 → 143 | held |
| ASIA|MEDIUM|30_70|FLAT | -12.21 | -7.12 | 519 → 217 | held |
| NEW_YORK_PM|MEDIUM|30_70|DOWN | -7.79 | -5.26 | 141 → 74 | held |
| LONDON|LOW|30_70|FLAT | -5.17 | -4.81 | 494 → 190 | held |
| AFTER_HOURS|LOW|30_70|DOWN | -6.30 | -3.50 | 116 → 56 | held |
| LONDON|MEDIUM|30_70|FLAT | -6.14 | -2.66 | 486 → 215 | held |
| NEW_YORK_PM|MEDIUM|30_70|FLAT | -11.50 | -0.50 | 304 → 132 | held |
| ASIA|LOW|30_70|DOWN | -21.26 | +3.65 | 131 → 55 | flipped |
| NEW_YORK_PM|MEDIUM|30_70|UP | -12.92 | +7.52 | 167 → 49 | flipped |
| NEW_YORK_PM|LOW|30_70|UP | -17.35 | +11.14 | 50 → 13 | flipped |
05
The honest shape of it
Every group with enough data, plotted by its learning-half average against its test-half average. Red is lost money. The mass sits low and left, and the top-right quadrant, conditions that made money before and kept making it, is effectively empty. Three flagged groups did drift back above water, and they are shown here rather than dropped.
Every condition group: past vs future, $/trade
dot size = trades in the test window · shaded band = stayed negative
flagged as lost moneyeverything else
06
The autopsy: where the money actually went
One more question the raw fills can answer: was the strategy even wrong? Reconstruct every trade's gross price move, exit minus entry, times size, and compare it with what was left after modelled costs.
Gross price P&L
+$22,121
the strategy made money at the price level, 65.6% of trades moved in its favour
Total costs
−$95,104
modelled at $8.00 per trade flat, 0.04% per side on this book’s fixed $10k notional. Funding on held positions is not included, so real costs were higher
Net
−$72,983
after modelled costs
1,299 trades, 10.9% of the book, called the direction correctly and still lost money. Average loss on those: $3.58, fee-sized, not thesis-sized. Longs lost $36,213 and shorts lost $36,771, near-identical, because costs do not care which way you traded. And the equity curve declines almost perfectly linearly across four years: no blow-up, no regime break, just a constant per-trade drag.
Keep both win rates in view: 65.6% gross, 54.6% net. The eleven-point gap between them is the cost drag expressed as a rate, the 1,299 right-but-lost trades, seen from the other side.
So the diagnosis is sharper than “no edge.” This strategy has a real gross edge of about +$1.86 per trade, and pays a modelled $8.00 per trade to express it. The edge is not absent; it is 4.3× too small to pay its own costs. And the conclusion is robust to the cost assumption: even at half these costs the edge is still 2.2× too small. Break-even sits at $1.86 per trade, this book turns profitable at 77% lower execution costs, or 4.3× more gross edge, and no amount of condition-filtering changes that arithmetic.
Where this evidence runs out
The 84% above is a real out-of-sample result, and it is also a small one. Three things a careful reader should hold against it, stated here rather than left to be found.
Small n, and not independent
19 conditions is not many, and they are not 19 separate experiments. They partition one strategy on two instruments over one period, so they share the same underlying market and the same execution. Treat this as one strong observation, not 19 corroborating ones.
The misses share a shape
Of the three that reverted, two are NEW_YORK_PM in an UP trend. Of the three losing conditions in an UP trend, two flipped, while 12 of the 16 that held sit in a FLAT trend. The failures are not evenly scattered; the lost-money flag looks weakest in trending conditions.
One dimension did nothing
All 19 flagged conditions fall in the same RSI band, 30_70. On this strategy the RSI dimension separated nothing, so four of the five conditions carried the result. A different strategy would likely lean on them differently.
And the honest gap: this shows the audit cutting losses on a strategy that was already doomed. It does not yet show the reverse: that it confirms a real edge on a system that works. Until a profitable book is put through the same wall, a fair reader can ask whether the lost-money flag is detecting a broken strategy or simply flagging volatile conditions. That case study is the next one to run, and it should be published whichever way it lands.
What chance alone produces
Any set of rules finds something if you look at enough groups. So we ran the audit’s own rules on this book where there was nothing real to find, and counted what came out. Method: the 8,321 learning-half trades, the product’s own verdict rules, 1,000 random regroupings for each test.
Real results, conditions shuffled
We kept every trade’s real result and shuffled which conditions it was taken in, 1,000 times. A strong group appeared in 31 of the 1,000 shuffles. Random groups of these losing trades were flagged as lost money 21 to 29 times a run; the real conditions gave 19. On a book that loses almost everywhere, that count says little on its own. The better evidence for the flag is the forward test above: $7.69 lost a trade against $4.16.
A book with no edge at all
We shifted every trade so the book averaged exactly zero, then regrouped it at random 1,000 times. A group came out strong by chance 2.1% of the time at 20 to 59 trades, 1.6% at 60 to 119 and 1.1% at 120 or more. Across a whole book it adds up: with 103 groups, a book with no edge showed at least one strong group in 474 of 1,000 runs; with 20 groups, about one run in nine. The Edge page now shows that count next to your own.
A real edge, planted
We added an edge to one group of a book with none. At about a third of a standard deviation a trade ($27 here), a group of 118 trades was called strong in 90% of 400 runs; at 34 trades, in 38%. Strong can be reached by a real edge of a realistic size. The empty strong row above is a result, not a bar set too high to clear.
Why the answer was to stop
We could have kept going. Drop two more conditions. Tighten a threshold. Add a filter. Keep trimming until the curve finally clears zero and the whole thing looks like an edge.
That move has a name: overfitting. Every filter added to make a past look profitable is fitted to noise, the specific, never-repeating accidents of that exact history. It reads as a strategy. It dies the moment it meets a market it has not memorised. And it is dangerous precisely because it does not feel like a mistake. It feels like work.
The audit already told us what tinkering would have hidden. Cutting the losing groups removed $14,428 of real bleed, 74% of the damage, and 84% of those verdicts held on trades they had not seen. But zero conditions ever earned a proven-edge verdict, the gross edge is 4.3× smaller than the per-trade cost, and the best honest version still finishes red. There was no edge left after costs. There was a losing system in which some conditions lost faster than others.
The most valuable thing an audit can tell you is when to stop. This one said: do not chase it.
upload your fills · freeze the verdicts · find out if it is worth chasing