Research Methods

Validation and Ablation Methodology

A method note on frozen tests, baseline ladders, leakage control, and the evidence required before promoting a strategy candidate.

2026 · Method note

Research question

What evidence should a strategy candidate survive before it is treated as more than an attractive backtest?

Principle

The useful adversarial question is not only “does it work?” but “what change in data visibility, baseline, cost, or implementation would make the result disappear?”

Method

The validation ladder keeps simple comparators alive: no-news and no-LLM variants, transaction checks, stale or shuffled-input diagnostics where appropriate, and turnover-aware evaluation.

Promotion boundary

A candidate is promoted only when data visibility, implementation correctness, baseline comparisons, and failure modes remain intelligible.

Limitations

No checklist can eliminate specification search, regime dependence, or undetected implementation errors. The method creates evidence and traceability; it does not certify truth.

Next question

Which test would be most likely to falsify the current interpretation?