How do I backtest a strategy honestly?

A backtest runs your rules over past prices, so it can only test rules a computer can apply the same way every time. Try many rules and keep the best, and the result is inflated by luck. Hold data back, nudge the inputs, and trust only the rule that survives.

What it is

A backtest replays your rules over past prices and reports what would have happened. It can only test rules a computer can apply identically every time; “looks strong” cannot be tested. That is why the strategy builder forces every rule into a testable form.

Why it matters

A single good equity curve is the easiest thing in trading to produce. The results page shows out-of-sample folds and a Monte Carlo shuffle of the trades because one curve, on its own, proves almost nothing.

What the research says

Two warnings run through the evidence-based literature. The first is data-mining bias: try enough rules and the best one looks great by chance, so the result you keep is inflated. The second is overfitting: a rule tuned until history fits perfectly tends to break first, and the robust rule is the one that still works when its parameters are nudged. Both point to the same habit of holding back data the rule never saw.