One fixture set · Ten probability systems · No row switching
Which football prediction model performed best on the same EPL matches?
Direct answer: On the same 1,520 EPL matches, Elo to multinomial logistic regression had the lowest history-only class-averaged Brier score (0.1951354035). The later-information Shin closing-market reference scored 0.1900581029; that timing difference prevents a like-for-like algorithm-versus-market claim.
Every row below scored the same fixtures. Closing odds remain a later-information reference—not a fair real-time opponent and not evidence of betting profit.
Published by Football Proof AI · Published · epl-football-prediction-model-benchmark/1.0.0 · Editorial technical note; not externally peer reviewed
Held-out EPL fixtures scored by every published row.
Statistical, machine-learning and later-information market references.
Elo to multinomial logistic regression. Lower class-average Brier is better.
Shin de-vig closing market; later-information context, not a profit result.
Primary result · Same denominator · Lower is better
One leaderboard that cannot change the matches between rows.
Class-average Brier is the primary ordering metric. Log loss, normalized RPS, fractional-tie hit rate and two calibration-error summaries remain visible so the ranking cannot hide behind one convenient score.
Lowest Brier on this releaseShin de-vig closing market · 0.1901Shin de-vig closing market ranked first on the declared primary metric across all 1,520 common fixtures. The best history-only row was Elo to multinomial logistic regression at 0.1951.
- Evaluation window
- 2022-08-05 through 2026-05-24
- Fixture identity
be42108329815e8969c95eace1537d855bb8bcbce148800254a01b19836c7151- Primary score
- 1X2 class-average Brier; lower is better.
- Claim boundary
- Historical probability quality only; no live-accuracy or profit claim.
| Rank / model | N | Brier ↓ | Log loss ↓ | Normalized RPS ↓ | Hit rate ↑ | Top-label ECE ↓ | Macro class ECE ↓ |
|---|---|---|---|---|---|---|---|
| 1Shin de-vig closing market | 1,520 | 0.1901 | 0.9600 | 0.1947 | 55.13% | 2.88% | 1.74% |
| 2Proportional de-vig closing market | 1,520 | 0.1901 | 0.9603 | 0.1947 | 55.13% | 2.22% | 1.63% |
| 3Elo to multinomial logistic regression | 1,520 | 0.1951 | 0.9816 | 0.2015 | 53.62% | 2.92% | 1.63% |
| 414-feature multinomial logistic regression | 1,520 | 0.1960 | 0.9843 | 0.2022 | 53.16% | 3.34% | 2.17% |
| 5LightGBM plus training-only isotonic calibration | 1,520 | 0.2033 | 1.0306 | 0.2126 | 50.20% | 1.34% | 3.02% |
| 6Sequential training-only Dixon-Coles | 1,520 | 0.2051 | 1.0277 | 0.2169 | 49.41% | 3.23% | 2.65% |
| 7Independent ridge-Poisson | 1,520 | 0.2051 | 1.0273 | 0.2169 | 49.41% | 3.65% | 3.10% |
| 8Raw 14-feature LightGBM | 1,520 | 0.2129 | 1.0759 | 0.2195 | 47.83% | 11.79% | 8.56% |
| 9Availability-safe expanding EPL prior | 1,520 | 0.2153 | 1.0678 | 0.2314 | 44.47% | 0.57% | 0.88% |
| 10Uniform 1/3 baseline | 1,520 | 0.2222 | 1.0986 | 0.2376 | 33.33% | 0.00% | 7.43% |
The coral divider marks closing-market rows. Their probabilities use information available later than the history-only forecast point. Rank is descriptive for this frozen release and is not a universal model hierarchy.
Machine learning vs Poisson · Same matches
Does calibrated LightGBM beat a Poisson football model?
This page answers on one common EPL test set, not by stitching together attractive scores from unrelated studies. It also keeps raw LightGBM visible so calibration cannot disappear inside the label “AI”.
Independent ridge-Poisson and LightGBM plus training-only isotonic calibration
Difference, calibrated LightGBM minus Poisson: -0.00188. Negative favours calibrated LightGBM on this metric.
Raw 14-feature LightGBM and LightGBM plus training-only isotonic calibration
Isotonic minus raw Brier: -0.00962, paired 95% interval [-0.01336, -0.00593]. Log-loss difference: -0.04535, paired 95% interval [-0.07071, -0.01684]. Lower is better for both.
Uniform 1/3 baseline
Fixed probabilities (1/3, 1/3, 1/3) for every match.
Fit schedule: Never fitted. Inputs: none.
Availability-safe expanding EPL prior
Observed H/D/A frequencies from finals whose 24-hour result lag ended strictly before the 24-hour forecast timestamp.
Fit schedule: Updated before each fixture timestamp without same-time or unavailable results. Inputs: past EPL outcomes.
Elo to multinomial logistic regression
Three-class L2 multinomial logistic regression on the locked home-advantage Elo difference.
Fit schedule: Refitted once before each evaluation season using complete earlier seasons only. Inputs: elo_diff_home_adv.
14-feature multinomial logistic regression
Median imputation, standardization and L2 three-class multinomial logistic regression.
Fit schedule: Refitted once before each evaluation season using complete earlier seasons only. Inputs: home_points_sum_5, away_points_sum_5, home_goals_for_sum_5, home_goals_against_sum_5, away_goals_for_sum_5, away_goals_against_sum_5, home_home_points_sum_5, home_home_goal_diff_sum_5, away_away_points_sum_5, away_away_goal_diff_sum_5, elo_diff_home_adv, h2h_home_points_sum_5, home_rest_days, away_rest_days.
Independent ridge-Poisson
Independent home/away Poisson goal rates with home advantage and ridge attack/defence effects, converted to 1X2 probabilities.
Fit schedule: Refitted once before each evaluation season using complete earlier seasons only. Inputs: past EPL final scores, home team, away team.
Sequential training-only Dixon-Coles
Training-only Dixon-Coles low-score rho applied to the identical ridge-Poisson rates; not a joint Dixon-Coles fit.
Fit schedule: Poisson rates and rho are refitted once before each evaluation season from complete earlier seasons only. Inputs: past EPL final scores, home team, away team.
Raw 14-feature LightGBM
Locked deterministic multiclass LightGBM before probability calibration.
Fit schedule: Refitted once before each season on all complete earlier seasons; published as a calibration ablation. Inputs: home_points_sum_5, away_points_sum_5, home_goals_for_sum_5, home_goals_against_sum_5, away_goals_for_sum_5, away_goals_against_sum_5, home_home_points_sum_5, home_home_goal_diff_sum_5, away_away_points_sum_5, away_away_goal_diff_sum_5, elo_diff_home_adv, h2h_home_points_sum_5, home_rest_days, away_rest_days.
LightGBM plus training-only isotonic calibration
The same raw LightGBM probabilities passed through one-vs-rest isotonic calibrators and renormalized.
Fit schedule: The evaluation raw model uses all earlier seasons; calibrators use expanding raw predictions generated out of fold on earlier seasons only. Inputs: home_points_sum_5, away_points_sum_5, home_goals_for_sum_5, home_goals_against_sum_5, away_goals_for_sum_5, away_goals_against_sum_5, home_home_points_sum_5, home_home_goal_diff_sum_5, away_away_points_sum_5, away_away_goal_diff_sum_5, elo_diff_home_adv, h2h_home_points_sum_5, home_rest_days, away_rest_days.
Proportional de-vig closing market
Average closing 1X2 decimal odds converted to implied weights and normalized by booksum.
Fit schedule: Not a fitted AI model; closing prices are observed later than the 24-hour model timestamp. Inputs: AvgCH, AvgCD, AvgCA.
Shin de-vig closing market
Average closing 1X2 decimal odds de-vigged with deterministic Shin bisection.
Fit schedule: Not a fitted AI model; closing prices are observed later than the 24-hour model timestamp. Inputs: AvgCH, AvgCD, AvgCA.
Read the isolated Poisson versus Dixon–Coles benchmark, the LightGBM backtest release and the plain-language football prediction model guide for each family's assumptions.
Model vs bookmaker · Paired week-block uncertainty
Closing odds are a hard reference with later information.
Each difference below is model minus Shin de-vig closing market on the exact same fixtures. Negative favours the model. The interval is a pairedDeterministic paired percentile intervals from 5,000 ISO-week block-bootstrap resamples. Each sampled week retains both systems on identical fixtures. Model-minus-Shin differences below zero favour the model, but the closing reference has a later information set. across 143 ISO-week blocks with 5,000 deterministic resamples.
This is not a same-information-time contest.
History-only forecasts are fixed before the match. Closing prices can absorb later team news, market movement and other information. Their inclusion supplies demanding context; it does not establish that an earlier model had access to the same evidence.
95% interval [+0.00302, +0.00718]. The interval excludes zero in this design.
95% interval [+0.00998, +0.01638]. The interval excludes zero in this design.
95% interval [+0.01168, +0.01830]. The interval excludes zero in this design.
Recreate the de-vigging boundary in the football model vs bookmaker calculator and inspect the site's 1X2 margin-removal benchmark before comparing probabilities from quoted odds.
Calibration · Confidence should match frequency
Raw and calibrated LightGBM stay separate.
The ablation changes calibration while retaining the declared model family and common evaluation rows. Empty confidence bins remain empty rather than being silently merged into a better-looking chart.
11.79% top-label ECE
Macro classwise ECE 8.56% · Brier 0.2129 · log loss 1.0759.
1.34% top-label ECE
Macro classwise ECE 3.02% · Brier 0.2033 · log loss 1.0306.
| Confidence bin | Raw N | Raw mean confidence | Raw observed accuracy | Calibrated N | Calibrated mean confidence | Calibrated observed accuracy |
|---|---|---|---|---|---|---|
| 0%–10% | 0 | — | — | 0 | — | — |
| 10%–20% | 0 | — | — | 0 | — | — |
| 20%–30% | 0 | — | — | 0 | — | — |
| 30%–40% | 91 | 37.31% | 23.08% | 177 | 38.75% | 40.68% |
| 40%–50% | 399 | 45.25% | 40.85% | 717 | 44.36% | 44.35% |
| 50%–60% | 358 | 54.85% | 44.13% | 434 | 54.50% | 55.76% |
| 60%–70% | 270 | 64.71% | 49.63% | 185 | 63.29% | 68.65% |
| 70%–80% | 219 | 74.58% | 55.25% | 6 | 74.10% | 50.00% |
| 80%–90% | 139 | 84.03% | 68.35% | 1 | 86.96% | 100.00% |
| 90%–100% | 44 | 92.24% | 79.55% | 0 | — | — |
Learn why probability calibration can matter more than a headline hit rate in the 1X2 calibration benchmark and test your own rows in the calibration lab.
Season stability · Publish every declared slice
An overall rank is not a promise of yearly dominance.
The table keeps four interpretable reference rows visible in every evaluation season: Poisson, calibrated LightGBM, the best overall history-only row and the later-information Shin closing market.
| Season | Matches | Independent ridge-Poisson | LightGBM plus training-only isotonic calibration | Elo to multinomial logistic regression | Shin de-vig closing market |
|---|---|---|---|---|---|
| 2022/23 | 380 | 0.2042 | 0.2042 | 0.1931 | 0.1904 |
| 2023/24 | 380 | 0.1859 | 0.1921 | 0.1840 | 0.1753 |
| 2024/25 | 380 | 0.2134 | 0.2044 | 0.1974 | 0.1918 |
| 2025/26 | 380 | 0.2171 | 0.2125 | 0.2061 | 0.2028 |
Season-expanding folds · Forecast time before result time
Fit complete earlier seasons, then score the next season.
The same-match fixture hash prevents row switching. History-only models use complete earlier seasons across four expanding evaluation folds; every fixture sharing a kick-off is predicted from one pre-batch state. ISO weeks are bootstrap blocks, not model-refit boundaries. Closing odds are labelled as a separate later-information set.
- 01 · Identity
Lock one fixture key set
Every leaderboard row must report the same N and fixture-key SHA-256 or the comparison fails.
- 02 · Folds
Fit complete earlier seasons
Each expanding fold trains and calibrates before the next held-out EPL season is evaluated.
- 03 · Batches
Freeze same-kickoff state
Simultaneous fixtures are forecast before any result from that time batch updates ratings or form.
- 04 · Calibration
Fit without test outcomes
The raw LightGBM and isotonic rows remain distinct, with calibration learned before each held-out fold.
- 05 · Market
Label later information
De-vigged closing probabilities are contextual references and are never relabelled as pre-match history-only features.
- 06 · Uncertainty
Resample paired weeks
Deterministic ISO-week blocks keep every method paired on the same resampled fixtures.
Inspect the full walk-forward football model validation protocol and run the temporal leakage auditor before trusting a backtest.
Immutable evidence · JSON · CSV · Manifest · Source
Download the exact benchmark behind every number.
The aggregate files, release manifest and generator each expose a fixed digest. Raw third-party match rows remain with their publisher.
- JSON SHA-256
60eedfeca8f0e8b0bad76f65734fbd89797f5246910aea4a2e86553857d1cde1- CSV SHA-256
ecdbdf708c649791de3f118798c110e0ca9c2cbc5c026760e2634d5d4e0e746a- Manifest SHA-256
8037b509f57984b5c076ace79917a5c6868a3136e82d6737fbdbb1a6635a0e9b- Generator SHA-256
ae61549bf96803665660b153a20a973d29faa18d17e982e344a9c99ff9dd06a3- Source rights
- unknown/not asserted · checked 2026-07-18
- Aggregate rights scope
- CC BY 4.0 covers only Football Proof AI's original aggregate benchmark outputs and release metadata. It does not license or redistribute Football-Data.co.uk source rows or grant rights in the publisher's source files.
Canonical publication record
Abstract
An aggregate-only comparison of ten football 1X2 forecast systems on the same 1,520 held-out Premier League matches. Four season-expanding folds compare uniform and past-only baselines, Elo-logit, fourteen-feature logistic regression, ridge-Poisson, sequential Dixon-Coles, raw and isotonic LightGBM, plus proportional and Shin de-vigged closing-market references with proper scores, calibration and paired ISO-week uncertainty.
- Author and publisher
- Football Proof AI
- Publication version
epl-football-prediction-model-benchmark/1.0.0- Published
- Last modified
- Release status
- Current release
- Review status
- Editorial technical note; not externally peer reviewed
- Production method
- Editorial dataset maintained from the declared method, linked sources and any listed reproducible artifacts.
- Software and AI assistance
- Software, including generative AI, may assist drafting, transformation or quality checks. It is not treated as a source, author or independent reviewer; factual and quantitative claims must remain bound to cited sources, declared methods or reproducible artifacts.
- Why this exists
- Enable readers to inspect assumptions, reproduce calculations and reject claims that exceed the published evidence.
- Commercial boundary
- No bookmaker, prediction provider or paid placement controls the evidence rules, calculations or conclusions on this page.
- Version history
epl-football-prediction-model-benchmark/1.0.0: Initial public release.
- Immutable artifacts
- 1.0.0.json
sha256:60eedfeca8f0e8b0bad76f65734fbd89797f5246910aea4a2e86553857d1cde1 - epl-football-prediction-model-benchmark-1.0.0.csv
sha256:ecdbdf708c649791de3f118798c110e0ca9c2cbc5c026760e2634d5d4e0e746a - 1.0.0-manifest.json
sha256:8037b509f57984b5c076ace79917a5c6868a3136e82d6737fbdbb1a6635a0e9b - generate-epl-football-prediction-model-benchmark-1.0.0.py
sha256:ae61549bf96803665660b153a20a973d29faa18d17e982e344a9c99ff9dd06a3
- 1.0.0.json
- Release resources
- Public benchmark source repository
- Immutable repository release v1.0.1 — licence-metadata patch; benchmark data v1.0.0 unchanged
- Release citation metadata
- Aggregate release licence (CC BY 4.0)
- Generator code licence (MIT)
- External release JSON
- External release CSV
- External release manifest
- External release Python generator
- External release SHA-256 checksums
- Football-Data results and odds archive
- Football-Data field definitions
- References
- https://www.football-data.co.uk/data.php
- https://www.football-data.co.uk/notes.txt
- https://handbook.fide.com/chapter/B022024
- https://doi.org/10.1111/j.1467-9574.1982.tb00782.x
- https://doi.org/10.1111/1467-9876.00065
- https://proceedings.neurips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html
- https://icml.cc/Conferences/2005/proceedings/papers/079_GoodProbabilities_NiculescuMizilCaruana.pdf
- https://doi.org/10.1175/1520-0493(1950)078%3C0001:VOFEIT%3E2.0.CO;2
- https://doi.org/10.1198/016214506000001437
- https://doi.org/10.2307/2234526
Interpretation boundary · Read before quoting
This benchmark measures forecasts. It does not manufacture certainty.
The release is an aggregate historical comparison on one league, one declared protocol and one frozen source snapshot. Reuse should preserve its sample, timing and rights boundaries.
Event-time safe; source-revision replay incomplete.
The generator enforces event-time and declared 24-hour availability cutoffs, but the hash-pinned latest-state CSVs do not expose when later source corrections became observable. This research release therefore cannot activate a production model.
Production activation eligible: no.
- BoundaryThis aggregate benchmark covers one league and four evaluation seasons; it does not establish universal model superiority.
- BoundaryClosing-market rows use later information than every 24-hour history-only model and are contextual references only.
- BoundaryComplete-season fitting keeps one coherent leakage-safe comparison but does not update fitted coefficients or team effects within an evaluation season.
- BoundaryDynamic 14-feature and Elo snapshots may incorporate availability-safe earlier results during an evaluation season while their fitted classifier mapping remains frozen at the season boundary.
- BoundaryThe sequential Dixon-Coles row changes rho on shared ridge-Poisson rates and is not a full joint Dixon-Coles maximum-likelihood fit.
- BoundaryLatest-state source CSVs lack correction observed-at history; hash pins freeze release bytes but cannot prove revision-time replay.
- BoundaryWeekly block-bootstrap intervals preserve paired fixture weeks but do not prove independence, causality, profitability or a universal edge.
- BoundaryNo raw source rows, fixture probabilities, fitted estimators or calibrators are redistributed.
Citation-ready research record
Cite the frozen release, not a copied score.
Football Proof AI. (2026). Football Prediction Model Benchmark: Same EPL Matches. Version epl-football-prediction-model-benchmark/1.0.0. https://footballproofai.com/research/epl-football-prediction-model-benchmark- Identifier
fpai:dataset:epl-football-prediction-model-benchmark:1.0.0- Release status
- Current · Editorial technical note; not externally peer reviewed
- Public archive
- GitHub repository release v1.0.1 · licence-metadata patch; benchmark data v1.0.0 unchanged
- Version
epl-football-prediction-model-benchmark/1.0.0
Football prediction model benchmark FAQ
Six direct answers tied to the released evidence.
Which football prediction model was most accurate in this benchmark?
Elo to multinomial logistic regression was the highest-ranked history-only model on class-average Brier score across the same 1,520 held-out Premier League matches, with 0.1951. The overall lowest score was Shin de-vig closing market at 0.1901; closing-market rows are later-information references. This is one frozen historical design, not a universal ranking.
Was LightGBM better than Poisson for football prediction?
On this exact same-match sample, calibrated LightGBM recorded a 0.2033 class-average Brier score and ridge Poisson recorded 0.2051. The comparison holds the fixtures constant; it does not turn either method into a universal winner.
Did the football models beat bookmaker odds?
The paired table compares each model with Shin de-vig closing market. Those closing odds contain information observed later than the history-only model forecast point, so they are a contextual reference rather than a same-information-time opponent. The confidence intervals show the uncertainty, and none of these score differences proves obtainable profit.
Why compare raw and calibrated LightGBM probabilities?
The raw and isotonic rows use the same held-out fixture set. Calibration changes probability quality without changing the underlying model family. Here their class-average Brier scores were 0.2129 and 0.2033 respectively; the full table also reports log loss, RPS and calibration error.
Does the highest hit rate identify the best betting model?
No. Hit rate discards how much probability a model assigned to every outcome, does not include an obtainable price and does not calculate return. This benchmark ranks probability forecasts primarily by class-average Brier score and publishes several secondary metrics to prevent one flattering number from carrying the claim.
Can I download and reproduce this football model comparison?
Yes. Versioned JSON, CSV, manifest and Python generator downloads are linked on this page with SHA-256 digests. The aggregate release is reproducible from the declared hash-matched source files; third-party raw match rows are not republished here.