Blog / 30 Sep 2026
Overfitting in plain words: why your best settings are probably lying
Try enough combinations and one will look brilliant. How to tell a real effect from a lucky cell, with a sweep and an out-of-sample check on real prices.
Take any trading rule with two settings. Try every combination. One of them will have the best result. That is arithmetic, and it says nothing about whether the rule works. Mistaking the best cell for a discovery is called overfitting, and it is the most common way a backtest fools its owner.
What it looks like
We swept the example strategy, an SMA crossover on two years of daily Bitcoin, across 12 fast averages and 10 slow averages. The tool tuned on the first 70% of the history only. Most of the grid lost money. The best cell was a fast average of 37 and a slow average of 83, with +12.81% on that first part.
Look at what surrounds it. One step up the grid, the same fast average with a slow average of 75 lost 18.4%. One step to the left, a fast average of 35 lost 1.0%. A real effect usually changes gradually: if 37 / 83 works, 35 / 83 and 37 / 75 should work nearly as well. When a result swings from +12.8 to −18.4 on one small step, the likeliest explanation is that the winner happened to dodge a bad trade its neighbors took.
Why more searching makes it worse
Every combination you test is another draw from the same lottery. Test 120 combinations and the best of them will look good even if the rule is worthless, in the same way that the luckiest of 120 coin-flippers looks skilled. Add a third setting, or a stop-loss level, or a choice of five markets, and the number of draws multiplies. The best result improves and its meaning shrinks.
The defense: data it has never seen
Split the history. Choose settings while looking only at the first part (in-sample). Then run those settings, unchanged, on the rest (out-of-sample). The second number is the honest one.
For the sweep above, the tool does this for you: the 37 / 83 cell returned +11.14% on the unseen 30%, against +29.23% for buy & hold over the same days. Positive, and still well behind doing nothing.
The default 20 / 50 settings tell a different story: −9.82% on the first 510 candles and +18.91% on the last 219. StrategyLab’s verdict is the right one: “The two periods behaved differently; neither result alone says much.” Two halves that disagree are a sign that the market regime, not the rule, decided the outcome.
Habits that help
- Decide the split before you look. If you peek at the out-of-sample result and then adjust, it is no longer out-of-sample.
- Prefer regions to peaks. Pick settings from the middle of a broad patch of decent results, not the single best cell.
- Fewer settings. Every adjustable number is another way to fit the past.
- More trades. Nine trades can be arranged to look like anything. Three hundred are harder to fool yourself with.
- Use round, boring values. If 20 / 50 fails and 37 / 83 works, ask what is special about 37. Usually nothing.
Under every result and every sweep, StrategyLab prints the same sentence: a great backtest is not a promise. The more settings you try, the more likely the best one only fits the past by chance. The sweep and the split are there so you can find that out before the market tells you. Both are in the Robustness tab on Pro.
The figures in this article are simulated results from one example: real BTC-USD daily candles from Coinbase Exchange, 1 Oct 2024 to 29 Sep 2026, $10,000 starting cash. Hypothetical performance has limits and is not a guarantee of future returns. Educational use only, not investment advice. See the risk disclosure.