Input contract
Seven fields reconstruct the evidence boundary
Each row describes one match in one walk-forward fold. Timestamps must include a timezone because a naive local time cannot establish which fact existed first.
- match_id
- Stable match identity used to expose duplicate and cross-fold reuse
- fold
- Walk-forward evaluation fold or deployment replay window
- split
- Exactly train or test; same-fold membership in both is critical
- prediction_at
- Zoned time at which the forecast would have been produced
- kickoff_at
- Zoned match start; prediction_at must be strictly earlier
- outcome_known_at
- First time the settled training target was genuinely available
- feature_available_at
- Latest availability time among features consumed by that row
Download the synthetic leaky-fold example. Its findings demonstrate the contract and are not model-performance evidence.
Audit order
Fail the timeline before scoring the model
A Brier score or hit rate is uninterpretable until the fold manifest passes point-in-time integrity checks.
- 01
Parse time
Reject impossible dates and timestamps without Z or a numeric offset.
- 02
Check inference
Flag predictions at or after kickoff and features arriving after prediction.
- 03
Lock outcomes
Require every training outcome to be known strictly before the first test prediction.
- 04
Separate windows
Compare the latest train prediction with the earliest test prediction in each fold.
- 05
Trace identity
Expose exact duplicates, same-fold split crossings and repeated test matches.
- 06
Bind receipt
Store evidence rows, severity, formula version and the input SHA-256 together.
Severity semantics
Not every reused match is the same failure
- Critical
- Future feature, post-kickoff prediction, same-fold train/test crossing or unavailable training outcome.
- High
- Malformed timestamps, duplicate weighting or the same test match counted in multiple folds.
- Review
- Cross-fold reuse that may be valid for an expanding training window but needs a declared policy.
The verdict fails on any critical or high finding. A warning-only receipt remains “review” because the tool cannot infer the intended expanding- versus rolling-window protocol from timestamps alone.
Evidence boundary
A manifest audit is necessary, not sufficient
A CSV can declare a clean timeline while an undocumented feature store, manual join or training job used different data. The SHA-256 proves byte identity only; it does not prove historical existence, pipeline execution or independent certification. This tool has not received external peer review.
- Archive the exact feature snapshot.Keep immutable query parameters, extract time and source fingerprint.
- Store orchestration evidence.Bind code, model artifact, fold manifest and output probabilities in one run receipt.
- Repeat independently.Have a separate reviewer reconstruct at least one fold from primary data.
Continue the method
After time integrity, evaluate probability quality.
Pass the fold boundary first. Then report every out-of-sample row, uncertainty, calibration and proper scoring rule without deleting misses.
Primary and authoritative sources
Methods behind the boundary checks
- Hewamalage, Ackermann and Bergmeir: forecast-evaluation pitfalls and best practices
- scikit-learn: time-ordered model validation
- pandas: backward as-of joins for point-in-time features
Cite this tool as: Football Proof AI. “Walk-Forward Football Model Leakage Auditor.” Version football-model-leakage-audit/1.0.0, 13 July 2026, https://footballproofai.com/tools/football-model-leakage-auditor. Editorial technical note; not externally peer reviewed.