A strong AI backtest can still contain tomorrow’s information

Data leakage, repeated strategy search and omitted costs can make historical results look more convincing than they are.

A historical trading result can look excellent because the test accidentally contains information from the future. It can also look excellent because a researcher tried many alternatives and reported the lucky winner. These are different problems. AI can accelerate strategy generation, which makes the second problem easier to create while doing nothing automatically to remove the first.

This article is educational information, not investment or trading advice. It does not endorse a cryptocurrency, token, platform, bot, provider or strategy, and it promises no profit. The examples are fictional research exercises, with no live trades or funds. A sounder backtest is still not proof of future returns.

The [backtest-overfitting research by Bailey and colleagues](https://www.davidhbailey.com/dhbpapers/backtest-prob.pdf) examines selection problems in historical testing. A [preprint submitted on 27 August 2026](https://arxiv.org/abs/2608.27734) studies leakage and repeated search in LLM-driven strategy discovery. The [Freqtrade documentation](https://www.freqtrade.io/en/stable/lookahead-analysis/) describes a tool for detecting certain lookahead problems in its own framework. The preprint is a primary author report, not established independent replication; the software documentation is a project description, not a profitability endorsement.

An original illustrative analyst asks an AI tool to propose a rule using yesterday’s market information. The dataset includes a column that was revised after the relevant day. The model’s apparent forecast may rely on that later revision. Even if the calculation is performed correctly, the information was unavailable when the hypothetical decision would have been made.

A point-in-time record needs more than a date on a row. It needs to establish when the observation became available, whether it was later revised and which assets existed then. A dataset built from today’s surviving assets can omit failed or delisted ones. That changes the research population and can make a historical result look better than the experience that was actually possible.

Repeated search creates a different illusion. Imagine a researcher asks an AI to generate a hundred variations, keeps the strongest chart and forgets the unsuccessful versions. The result reflects the search process as well as the proposed rule. A complete research log should retain the attempted variants, decision criteria and data examined. Otherwise readers cannot assess how much selection preceded the published winner.

A held-out period can help, but only if it stays held out. Repeatedly changing the rule after viewing the same later period makes that period part of development. The illustrative analyst can define the evaluation plan before examining the results and record departures. This is a methodological example, not a recommendation to trade the resulting rule.

Costs should match the hypothetical execution assumptions. A result based on frictionless prices may omit spreads, slippage, fees and periods in which an order would not fill. Crypto research can also involve venue differences, funding charges and unreliable market records. The appropriate accounting depends on the studied instrument and context. Report the assumptions instead of hiding them behind a single performance figure.

Automated checks are useful within their limits. A lookahead analyser can identify certain coding patterns or differences under changed test conditions. Passing that check does not prove that every input was available at the relevant time. It also does not correct an incomplete research log or guarantee that cost estimates reflect real execution.

AI-generated explanations deserve review too. A model may describe a successful historical pattern with a plausible economic story after seeing the outcome. That story is a hypothesis, not independent confirmation. Researchers should separate the proposed mechanism from the evidence supporting it and from the selection process that discovered the pattern.

The benefit of AI in this setting could be faster code review, clearer documentation and broader exploration of methodological weaknesses. The failure is faster production of persuasive but biased results. The 2026 preprint makes that concern timely, while its findings remain bounded to the author’s design and tested data.

A responsible research summary identifies the information available at each decision point, the number of alternatives tried, the cost model and the untouched evaluation evidence. It also states what remains unknown. Those details let readers examine a backtest honestly, without turning a statistical exercise into a promise about money.