Try enough strategies and one of them will look brilliant
Test ten thousand rules with no edge whatsoever, over two years of daily data, and the best one comes back with an annualised Sharpe near 2.7. Nothing went wrong. That is just what the maximum of ten thousand random draws looks like.
Suppose every strategy you test is worthless. Not badly designed, not unlucky: worthless, with a true Sharpe ratio of exactly zero. Each backtest still returns a number, because a finite sample of a zero mean process almost never averages to zero. Some come back slightly negative, some slightly positive.
You then do the natural thing and keep the best one. And the expected value of the best of N draws is not zero. It grows with N, slowly but without limit.
Reading the curve
Ten attempts buy you a Sharpe of 1.11. A hundred buy you 1.79. A thousand buy you 2.30. Ten thousand buy you 2.73, which is a number most people would call an excellent strategy and some would call a career.
The curve rises with the square root of the logarithm of N, which is the part that makes this uncomfortable. It is slow enough that adding trials feels harmless and relentless enough that it never stops.
A Sharpe ratio is meaningless without the number of things you tried before it. We write down the trial count at the start of a study, before we know the answer, and the threshold moves with it.
Nobody thinks they ran ten thousand backtests
That is the trap. The count is not the number of times you clicked run. It is the number of distinct configurations the data has been asked about, and it multiplies faster than anyone expects.
Four entry rules, five exit rules, three holding periods, six instruments and two lookback windows is 720 combinations, and that is a small grid. Add the ones you tried, disliked and abandoned, and the ones a colleague tried last year on the same data, and the honest count is a great deal larger than the number in the notebook.
What helps
Recording the trial count before the search, not after. Holding out a period nobody has looked at and touching it once. Preferring a rule with three parameters over a rule with eleven. And treating any result that needed a grid search to find as weaker than one that came from a mechanism you could state in a sentence beforehand.
None of these are clever. They are just the parts that get skipped when a backtest is already looking good.