Football Proof AIProbabilities with a public recordEvidence library

Free paired audit · Runs on your device

Football prediction model comparison tool

Put model A and baseline B on the exact same matches. Compare probability quality, not two hit-rate headlines built from different fixtures.

Paired 1X2 evaluation · Formula v1

Compare the same matches, not two unrelated headlines.

Every accepted row contains both probability splits and one settled outcome. The tool makes no network request with your prediction history; parsing, scoring and SHA-256 fingerprinting stay on this device.

Required columns: match_id, published_at, kickoff_at, a_home, a_draw, a_away, b_home, b_draw, b_away, outcome. Limit: 2 MB and 20,000 data rows.

Ready for a paired test

Run the synthetic example, then replace it with two complete forecasts over exactly the same matches.

Paired input contract

Every comparison must share the same target

A row is accepted only when both complete home-draw-away splits, the settled result and a publication time before kick-off are valid. A missing model cannot be replaced with a convenient row from another match.

match_id
Stable, unique match identity used to prevent duplicate comparisons
published_at
Zoned ISO 8601 time when both compared forecasts were available
kickoff_at
Zoned ISO 8601 kick-off used by the pre-match integrity gate
a_home / draw / away
Complete candidate-model 1X2 distribution
b_home / draw / away
Complete baseline-model 1X2 distribution on the same match
outcome
One settled result: home, draw or away

Download the synthetic paired CSV. Its numbers demonstrate the format and must not be cited as model performance.

Three complementary questions

Did it pick better, price better or merely look more confident?

Hit rate
Which model's largest probability matched the result more often?
Brier
Which complete 1X2 distribution stayed closer to the one-hot result?
Log loss
Which model avoided assigning dangerously small probability to what happened?

A winner selected by only one metric is not a universal winner. Publish the complete table, sample and convention instead of choosing whichever score makes the candidate look strongest.

Uncertainty boundary

The interval is a screen, not a research-grade verdict

The lab calculates the mean of the per-match A − B loss differences and a normal 95% interval using their paired sample standard error. Pairing is stronger than comparing unrelated averages, but the interval still assumes independent rows.

  1. 01

    Time dependence

    Matches from the same season and team can produce correlated errors; block or cluster resampling may be required.

  2. 02

    Model selection

    Do not tune either model on this same comparison set and then describe the result as untouched out-of-sample evidence.

  3. 03

    Coverage

    A narrow league or confidence slice cannot establish performance over matches the model chose not to publish.

  4. 04

    Small samples

    Fewer than 30 accepted pairs trigger an explicit warning; much larger samples can still be unstable under drift.

Complete the evaluation

Compare here. Validate through time elsewhere.

This tool scores a locked paired sample. It does not replace a walk-forward protocol, calibration analysis, immutable publication record or independent timestamp evidence.

Open walk-forward protocolAudit one complete recordReproduce every metricAudit an AI prediction claim

Primary sources

Methods behind the comparison

Cite this tool as: Football Proof AI. “Football Prediction Model Comparison Tool.” Version football-model-comparison/1.0.0, 13 July 2026, https://footballproofai.com/tools/football-prediction-model-comparison.