In a 1941 story by Jorge Luis Borges, the labyrinth turns out to be a novel in which every choice is taken. At each fork the story splits, and all the branches go on. Seventy years later the statisticians Andrew Gelman and Eric Loken borrowed the title for a quieter problem. A researcher rarely runs twenty tests and reports the best. More often they run one, and the data decide which one: which outliers to drop, which window to use, which subgroup looked interesting. Each choice is a fork, and the path actually taken was chosen by the noise.
The arithmetic is unforgiving. A rule with no edge at all passes a test at the 5% level one time in twenty. Try twenty such rules and the chance that at least one passes is about 64%. Try two hundred and it is a near certainty. The best result of a search is the luckiest one, and luck does not repeat.
The chart reader
A head and shoulders, an inverse one, a double top or bottom, a cup and handle, a rounding bottom: each comes with choices — how equal is equal, how deep is deep, when the neckline counts as broken. Candlesticks multiply them. The doji, the hammer, the engulfing pair, the morning and evening star are a few of dozens of named shapes, and a chart of any length contains some of them. When the patterns were written as code and tested on Dow Jones stocks, they added no value (Marshall, Young & Rose 2006). Writing the rule down before looking is what closes the forks.
The oscillator tinkerer
The stochastic oscillator, Williams %R, the CCI and the money flow index all ask much the same question — where is the price within its recent range? — each with a lookback and two thresholds to choose. Change 14 days to 9 and 80/20 to 70/30 until the backtest shines, and you have walked a dozen paths. Sullivan, Timmermann and White tested thousands of simple trading rules together and showed how much of the best rule’s performance disappears once the size of the search is taken into account.
The model tuner
Machine learning automates the garden. Optuna can try hundreds of hyperparameter settings in an afternoon, and the best validation score among them is flattered for the same reason the best trading rule is. The scikit-learn evaluation suite answers with nested cross-validation and a test set that is touched once; statsmodels answers with corrections for multiple testing, such as Benjamini and Hochberg’s. Public leaderboards are the same garden walked by a whole field: every team tunes against the same test set, which is why LLM evaluation keeps needing fresh benchmarks.
None of this means patterns are illusions. It means a result has to be judged together with the search that found it. A model is a map drawn from one territory, and a map redrawn until it fits every bump of the old ground is the one most likely to mislead on the new. That is the warning of The Map and the Territory and of Signal in the Noise.
Last reviewed October 2026 · How this site checks its content · Report a mistake
- Pattern EssaysSmall, Reversible BetsMake mistakes cheap to undo, and size the ones you cannot undo so that being wrong is survivable
- Pattern EssaysSignal in the NoiseFinding patterns is easy. The hard part is resisting them where there are none
- Timeseries EngineeringSARIMAARIMA with seasonal intelligence