Model Calibration
Prediction Accuracy and Calibration
Perfect calibration means a 70% confidence pick wins 70% of the time. See how we measure up across 115 settled predictions.
115
Settled Predictions
0.2408
Brier Score (lower is better)
5
Confidence Buckets
Predicted vs Actual Win Rate
Calibration by Confidence Band
| Confidence Band | Predicted % | Actual % | Delta | Sample Size |
|---|---|---|---|---|
| 50-54% | 52.0% | 70.0% | +18.0% | 10 |
| 55-59% | 57.0% | 56.8% | -0.2% | 81 |
| 60-64% | 62.0% | 55.6% | -6.4% | 18 |
| 65-69% | 67.0% | 75.0% | +8.0% | 4 |
| 70-74% | 72.0% | 100.0% | +28.0% | 2 |
What is calibration? A well-calibrated model's stated confidence matches reality. If we say 80% confidence, those picks should win about 80% of the time.
What is the Brier Score? It measures the accuracy of probabilistic predictions. A score of 0 is perfect; 0.25 is no better than a coin flip. Lower is better.
The diagonal line on the chart represents perfect calibration. Points above the line mean we're underconfident (actual wins exceed predicted); points below mean we're overconfident.