How reliable is crypto backtesting?

Crypto backtesting is a useful but easily misleading tool — it's only as good as the assumptions you feed it, and crypto magnifies the usual backtesting pitfalls. Here's the honest picture:

Where backtesting genuinely helps

• It forces you to formalize a strategy (entry/exit rules, position sizing) instead of trading on gut feel.
• It lets you see how a rule performed across many historical regimes, not just the one you're living through now.
• It's good for stress-testing mechanics — e.g. "if I rebalance weekly vs. monthly, how many trades does that generate and what's the fee drag?"

The big reliability problems, especially in crypto

• Overfitting / curve-fitting. With a huge space of indicators and parameters, you can almost always find a set of rules that looks amazing on history — but those rules are describing noise, not a real edge. The more parameters you tune, the less trustworthy the result. Out-of-sample testing helps but doesn't fully cure it.

• Survivorship bias. Historical price data only includes coins that still exist. The coins that went to zero are missing from the dataset, so any backtested "buy and hold altcoins" strategy looks far better than reality.

• Regime non-stationarity. Crypto's character changes over time — 2017's retail ICO mania, 2020-21's DeFi era, the 2024-25 ETF-era moves. A pattern that "worked" in one regime often breaks in the next. Backtests can't tell you which regime you're in now.

• Execution assumptions. Most free backtests assume you get filled at the backtested price, no slippage, no spread, no fee, no funding cost (huge for leveraged perp strategies). On a volatile low-liquidity coin, the real fills can be dramatically worse. "Backtested +3% daily" can become "actually -5%" once you add realistic costs.

• Lookahead / data errors. Stale or repaired candles, split-adjusted data, missing delistings — all quietly inflate results. Garbage in, garbage out.

• Short history. Many coins only have a couple of years of data, so a "strategy" might be tested on one or two market cycles — statistically meaningless sample.

What makes a result more trustworthy

• Hold out a chunk of data you never touched during development.
• Simulate realistic fees, slippage, and funding rates.
• Test fewer parameters, not more.
• Treat an edge as real only if it survives across different market regimes (bear + bull), not one period.
• Walk-forward testing (rolling re-optimization) rather than one static fit.

Bottom line: backtesting is a screening tool, not proof. A clean backtest tells you a strategy is worth investigating live — it does not tell you it will make money going forward. In crypto specifically, costs and regime shifts are so large that backtests overstate performance more than in most other markets.

Want me to pull current data on anything concrete — say, a coin's live price and volatility to sanity-check a trading idea, or which assets are trending right now?