What a backtest can and cannot tell you

A backtest answers one question: if these exact rules had been followed over this exact period, what would the portfolio have done? Nobody held that portfolio. The answer is arithmetic on history.

What it can tell you#

• Whether an idea was ever any good. A rule that lost to the index over fifteen years, across more than one kind of market, is unlikely to be a hidden gem.
• What the bad stretches looked like. The worst year and the deepest fall tell you what you would have had to sit through.
• How much the rule trades. High turnover means costs and taxes matter more than the headline return suggests.
• How it behaved in different markets. The year-by-year table shows whether the lead came from one lucky year.

What it cannot tell you#

• What happens next. The future is one path and the backtest is another.
• What you would have done. Rules do not panic at the bottom of a 40% fall. People do.
• What the trades would really have cost in a small stock or in size.
• Whether the rule was found by luck. Try enough rules and some will look good on any history.

Two records, kept apart#

The two are never blended into one number. A strategy's page shows the backtest and the live record side by side so you can see whether the live record is behaving like the backtest said it would.

Strategies with no backtest at all#

Some rules read company financials, such as return on capital or years of dividend growth. We only hold today's figures for those, not what was known at each past date. Testing them on the past would use numbers nobody had at the time, so we do not. Those strategies show a live record only.

The caveats travel with the numbers#

Every backtest we show carries its caveats on the same page: the period, the stock list used, how trades were timed, what costs were charged and how each figure is defined. They are never hidden behind a lock or a link.