Summary
cfb_season_odds builds its team set from the schedule, which contains every opponent an FBS team played. For 2023 that is 704 teams against 133 in cfb_ratings. make_ratings_compute_results documents that "teams absent from it are treated as league-average (0.0)", so the 571 unrated FCS/D2/D3/NAIA programs are simulated as median FBS teams.
Reproduction
from sportsdataverse.cfb import load_cfb_ratings, load_cfb_schedule
r = load_cfb_ratings([2023]); s = load_cfb_schedule([2023])
rated = set(r["team_id"].cast(int).to_list())
sched = set(s["home_id"].cast(int).to_list()) | set(s["away_id"].cast(int).to_list())
print(len(rated), len(sched), len(sched - rated)) # 133 704 571
Championship board from a 2023 run (2000 sims):
Michigan 0.569 <- actual champion, correct
Washington 0.159 <- actual runner-up, correct
South Dakota State 0.045
Southern Oregon 0.041
Harding 0.040
Ave Maria 0.025
Colorado Mines 0.024
Arizona Christian 0.011 ...
~24% of championship probability goes to schools that cannot enter the playoff. Rows show exp_wins 1.000 with conf_title_prob 0.000 and playoff_prob 1.000 — incoherent on its face, and nothing raises.
The engine itself is fine; the population fed to it is wrong.
Why the default is the bug
"Missing → league average" is a sound default for a team with sparse data and a catastrophic one for a team that does not belong in the population. At the lookup both are indistinguishable — a failed join. The default has to be chosen by why the row is absent.
Suggested fix
Restrict the team/game set to teams carrying a real rating before simulating, and raise (rather than silently pass) if the filter would empty the frame — that would indicate an id-namespace mismatch rather than a working filter.
Changing the shipped team set is arguably breaking for anyone consuming current output, so this may want a flag with a deprecation path.
Workaround in use
cfb_higher_models.season.simulate_season(..., fbs_only=True) in cfbfastR-cfb-data filters post-hoc and renormalises championship probability over the FBS field.
Summary
cfb_season_oddsbuilds its team set from the schedule, which contains every opponent an FBS team played. For 2023 that is 704 teams against 133 incfb_ratings.make_ratings_compute_resultsdocuments that "teams absent from it are treated as league-average (0.0)", so the 571 unrated FCS/D2/D3/NAIA programs are simulated as median FBS teams.Reproduction
Championship board from a 2023 run (2000 sims):
~24% of championship probability goes to schools that cannot enter the playoff. Rows show
exp_wins 1.000withconf_title_prob 0.000andplayoff_prob 1.000— incoherent on its face, and nothing raises.The engine itself is fine; the population fed to it is wrong.
Why the default is the bug
"Missing → league average" is a sound default for a team with sparse data and a catastrophic one for a team that does not belong in the population. At the lookup both are indistinguishable — a failed join. The default has to be chosen by why the row is absent.
Suggested fix
Restrict the team/game set to teams carrying a real rating before simulating, and raise (rather than silently pass) if the filter would empty the frame — that would indicate an id-namespace mismatch rather than a working filter.
Changing the shipped team set is arguably breaking for anyone consuming current output, so this may want a flag with a deprecation path.
Workaround in use
cfb_higher_models.season.simulate_season(..., fbs_only=True)incfbfastR-cfb-datafilters post-hoc and renormalises championship probability over the FBS field.