Gate 2: Matching SPY
Before any strategy's numbers are believed, the replay has to reproduce something the world already knows: SPY's total return over thirty-three years. It failed once, then passed. The tolerance did not move.
The gate, as the console printed it
One test validates the bar store, the dividend path, the fill simulator, the calendar and the day loop at once. If it is off by more than a small epsilon, something in the chain is wrong and we find out now rather than after a strategy's numbers have been believed.
The first attempt failed
Two defects, neither of them in the replay itself. FMP's dividend-adjusted endpoint silently truncates a response at 5,000 rows, so a request for 1993 onward came back starting in 2006 and the yardstick showed a multiple of ×8.07 against our ×31. The fix was to fetch the benchmark in five-year chunks. Separately, the trial number — which changes on every run by design — had been written into the tear sheet, which meant the tear sheet's hash changed on every run too, and the bit-identical rerun could never pass. It now lives only in the manifest.
Both were fixed. Neither tolerance moved.
Why an independent yardstick
The replay could have been checked against a number we computed ourselves from the same bars. That would test arithmetic, not honesty. FMP's dividend-adjusted series is built by a different party with a different method — adjusting the price history on the ex-date — from the same underlying dividend record. If our chain agrees with it across 8,455 days, then the bar store, the dividend record, the calendar, and the loop are all doing what they claim; if any one of them were wrong, thirty-three years of compounding would amplify the error far past ±3 %.
The residual that remains is explained, and now documented as the number to expect. We reinvest a dividend on its pay date, when the cash actually arrives; FMP adjusts on the ex-date, when the entitlement is fixed. Over thirty-three years that timing gap compounds to −0.73 % on the multiple and −0.02 pp on CAGR. Any future Plutus replay compared to a total-return index should show about that, and a larger gap is a defect.
Two more runs, with real frictions
coin-flip exists to prove the machine is honest. With haircuts and commissions on, it turned over its book twenty times a year and lost about two percentage points to SPY — roughly its costs and nothing more, which is exactly what a control should do.
dollar-mag7 is the more instructive result. A 30.95 % CAGR looks like a discovery. It is not. The dollar series it conditions on does not exist in the backtest host yet, so the strategy's macro input was absent every day and it held its default basket throughout: the seven biggest winners of the last thirty years, chosen with hindsight. Confident garbage — and the stamps say so. SURVIVOR-UNIVERSE and ABSENT:macro:DTWEXBGS are on the tear sheet, and any reader who takes the number without reading the stamps has been told.
A gate that fails is not renegotiated downward
If spy-hold cannot match SPY, the answer is to fix the chain, not to widen the tolerance. The ±0.10 pp and ±3 % were set before the first run and were the same numbers on the second. That is the whole value of a gate: it is a question the system has to answer, not a target the answer is allowed to move.
The FMP probe measured daily bars reaching 1993-01-29, SPY's first week of trading, with its complete dividend record and split-adjusted prices at no extra cost. Ten years would have excluded the 2000–2002 and 2007–2009 bear markets, which is most of what there is to learn. Longer was free, so a shorter test would have been a weaker test for no reason.
- The bar store and calendar walk 8,455 real days
- Dividends are found and reinvested
- The loop compounds without drift
- The same request reproduces the same bytes
- Not proven: that any strategy has an edge