TL;DR

  • The backtest was too generous about prices. When the strategy said “buy if the price reaches 446.2p”, the backtest always bought at exactly 446.2p. In real life, the price often jumps overnight - the market opens at 449.2p - and you buy at 449.2p.
  • That matters hugely here because the stops are tight. TFM’s stop-loss sits on average just 1.7% below the entry. On CBG the planned risk was 1.8p a share; paying 3p more to get in meant the trade started beyond its own stop.
  • It happened on almost half of all trades. 46% of entries and 34% of stop-losses were hit by a jump past the price. Each one quietly cost a slice of the trade’s risk - about 0.4R a trade in total.
  • The strategy’s whole edge was smaller than that slice. TFM made +0.2R a trade on paper. Take away 0.4R and it loses -0.24R a trade - before any commission.
  • The forward test caught it within a week. Four real paper trades lost an average of 3.5 times their planned risk. That was the warning; this post is the diagnosis.

How a backtest “fills” an order

A backtest walks through history candle by candle - here, 5-minute candles - and asks: did the price reach my order? For TFM’s buy-stop at 446.2p, the question is simply “did any candle’s high reach 446.2p?” If so, the backtest records a purchase at 446.2p.

That is exactly right when the price climbs smoothly through 446.2p - 445.8, 446.0, 446.2 - because the order is waiting there and triggers on the way up.

It is wrong when the price jumps. Shares don’t trade overnight, and news, results or a strong market can make a stock open well above where it closed. If yesterday closed at 444p and today opens at 449.2p, there was never a moment when 446.2p was available. The buy-stop triggers at the open and fills at 449.2p - the first price there is. The backtest still wrote down 446.2p.

The same happens on the way out. A stop-loss at 444.4p protects you only while prices move smoothly; if the stock opens at 440p, you sell at 440p.

Why TFM was especially exposed

Two features of the strategy made this much worse than it would be for most:

  1. The entry is a breakout of yesterday’s high. The buy-stop sits 0.1% above the previous day’s high - exactly where a strong morning open lands. Nearly half of all entries (45.6%) were triggered by the opening candle already being past the price.
  2. The stop is tight. The stop is worked back from the target at 3:1, so it averages only 1.7% from the entry. A 0.5% jump at the open - an ordinary move - is already a third of the trade’s whole risk.

Measured across all 2,834 trades of the CFD backtest (#259), the slippage the backtest was ignoring came to:

How often Average cost
Entry filled beyond its price (gap at the open) 44.5% of entries 0.26R a trade
Stop-loss filled beyond its price 33.5% of stop-outs 0.17R a trade

Against a strategy whose entire average profit was 0.20R a trade, that’s more than the edge.

What the paper account saw

Tip Planned entry Actual fill Planned risk Result
CBG 446.2p 449.2p 1.8p × 1,120 ≈ £20 −£89 (−4.4R)
OCDO 255.0p 254.8p 2.2p × 1,960 ≈ £43 −£78 (−1.8R)
PAY 641.6p 644.0p 2.0p × 779 ≈ £16 −£58 (−3.7R)
BMY 630.6p 632.0p 2.2p × 573 ≈ £12 −£53 (−4.2R)

A losing trade is supposed to cost about 1R. These cost 1.8R to 4.4R: fills a couple of pence worse than planned, on stops only about 2p away, plus commission - on positions where the planned risk was as little as £12. Four losses in a row on its own isn’t evidence (a 32% win rate gives that about one time in five), but losses this size are.

The re-run

Gecko’s backtester now fills orders the way a broker would: a stop entry fills at its price, or at the open when the market opens beyond it; a stop-loss the same; and a trade can now be stopped out inside the same 5-minute candle it entered. Same signals, same stocks, same ten years:

Run Fills Trades Win rate Avg R Gross Net
#259 (CFDs) perfect 2,834 27.3% +0.20 +£36,657 +£17,966
#261 (CFDs) realistic 2,834 26.5% −0.24 −£42,939 −£61,623

The win rate barely moves - the signals still pick direction about as well as before. What changes is how much each trade costs to get into and out of. Note the gross figure: the strategy now loses money before a penny of commission or financing.

Can it be rescued?

Six variants, all on the realistic model, all as CFDs:

Run Variant Trades Win rate Avg R Net
#261 as before (plain buy-stop) 2,834 26.5% −0.24 −£61,623
#262 stop-limit: never pay above the trigger 2,680 23.1% −0.09 −£34,399
#263 stop-limit: pay up to 0.1R above 2,715 23.7% −0.10 −£38,263
#264 stop-limit: pay up to 0.25R above 2,747 24.2% −0.12 −£42,268
#265 stop 1 × ATR (wider) 348 12.9% −0.06 −£4,378
#266 stop 1.5 × ATR 89 3.4% −0.16 −£1,874
#267 1 × ATR stop + stop-limit 0.1R 331 13.0% −0.01 −£2,908
  • A stop-limit order (buy when the price reaches X, but pay no more than Y) refuses to chase a gap - and halves the damage. But most of those orders still fill later in the day when the price falls back, and a breakout that falls back to where it started is usually a failing one: the win rate drops to 23%.
  • Wider, volatility-sized stops make a gap a much smaller fraction of the risk - but a target three times further away is rarely available, so only a few hundred trades qualify in ten years, and few of them reach it.

None of them makes money.

Entering at the next open instead

The simplest way not to chase a breakout is not to use a breakout order at all: when TFM confirms a signal, just buy (or sell short) at the next day’s open, with the same stop and target.

Run Variant Trades Win rate Avg R Net
#268 enter at next open 3,383 15.8% −0.11 −£47,951
#269 enter at next open, 1 × ATR stop 421 9.0% −0.05 −£4,800

Better than chasing the gap, but still a loss. Without the breakout to confirm it, many signals simply don’t follow through - the win rate falls to 16%. Every trade now enters, including those the buy-stop used to filter out by never triggering.

Verdict

Fail. TFM’s apparent edge - every positive TFM result on this blog before today - depended on fills that the real market doesn’t give. Those results are now marked (assumed perfect fills) wherever they appear, alongside the Stamp Duty marker.

The good news is how it was found. The backtests passed every test thrown at them, including a permutation test, because those tests all shared the same blind spot. Four real paper orders didn’t. That is precisely what a forward test is for, and why Gecko trades paper money before real money.

Automatic TFM tips are paused. The next idea for these signals needs an entry that doesn’t chase the opening jump - and it will be judged on the realistic model from the start.