The Audibles are a series of short-form research articles about thorny issues in college football analytics
Calculating Unbiased Game Control
The vague concept of Game Control (GC) entered the college football lexicon in 2014, the inaugural season of the four-team College Football Playoff. The ESPN Analytics group, a fledgling component of the company, created this statistic with seemingly admirable intentions: to replace the problematic margin of victory as a principal means of determining dominance in games and the “most deserving” teams for the new CFP:
https://www.espn.com/blog/statsinfo/post/_/id/96761/determining-the-most-deserving-teams
The crux of the idea is to use play-by-play (PBP) win probability throughout the game, rather than the final score, to assess a team’s dominance. The win probabilities are based on various game situations, such as score, down, distance, field position, game clock, etc., and should be agnostic about each team’s perceived power or chances of winning before the game started. With a good win probability model, the underlying notion of Game Control is sound, mitigating the problems associated with meaningless points being added in already-decided games.
However, ESPN Analytics made at least one fundamental mistake in developing this algorithm:
Their formula allows for the losing team to have greater Game Control than the winning team!
Suppose Team A scores twice in the first five minutes, taking an early 14–0 lead in the first quarter. Throughout the rest of the game, Team B outplays Team A. Team B ties the game late in the 3rd Quarter, 14–14, and eventually takes the lead with a TD with five minutes left to play, winning 21–14.
According to ESPN Analytics, Team A “controlled” this game more than Team B because Team A had a higher average win probability throughout the game. Why? Because Team A took an early lead that they maintained until late in the 3rd Quarter, and Team B only sustained a higher win probability for the final five minutes of the game. But if you consider the flow of the game, Team A really only won the first 5 minutes, while Team B won the remaining 55.
A Game Control metric that favors Team A in this situation (by a significant amount!) is clearly biased and contradicts an essential axiom of GC:
The victorious team should have a higher Game Control than their opponent.
It’s not about how many minutes a team held a lead. It’s about all the win probability changes throughout a game that yielded the final result, and how convincingly the winning team controlled those changes.
If the scoring in the example game above were reversed, everyone would agree that Team A should have a higher GC, since they worked their way to a 21-0 lead over the course of 55 minutes before allowing two late TDs that narrowed the final score to 21-14. Swapping the scoring of the first five minutes with the last five minutes shouldn’t result in a different team “winning” the Game Control metric.
Now, maybe some think it doesn’t matter, as it’s just a way of considering teams beyond the final score, but we heartily disagree. The College Football Playoff Committee apparently values GC a great deal, and while it may or may not be the ESPN version they consider, it is highly likely their version is calculated similarly.
No one is better at projecting and explaining the CFP Rankings than Adam McClintock (@cfb_professor on X/Twitter), who regularly predicts the committee’s rankings with a high degree of precision and, most importantly, explains why teams land where they do. What’s fascinating about McClintock’s explanations is that they demonstrate clearly that, contrary to many public narratives, the committee is quite objective in their determinations, predictably basing their decisions on a series of metrics. One of the most important metrics? You guessed it: Game Control, according to McClintock.
Bias problems with a statistic that is highly influential with the committee indicate a critical flaw in the process of selecting teams for the College Football Playoff.
It’s no less serious than that.
Improving the Formula
This flaw cannot be avoided if treating the average in-game win probability as the target metric.
As alluded to in the preceding section, the key to dominance is controlling the dynamics throughout the game, presuming an equal 50-50 chance of winning at kickoff. It is still necessary to compute win probabilities on a play-by-play basis, but instead of computing the average over the course of the game, compute the changes throughout the game. In this construction, the sum of all the positive PBP changes for the winning team will be always be a half-point higher than the corresponding sum for the losing team:
End-game win probability = 1.00
Kickoff win probability = 0.50
Sum of the PBP changes = 1.00 – 0.50 = 0.50 for the winning team
Let X = Sum of positive PBP changes for winning team
Then Y = X – 0.50 = Sum of positive PBP changes for losing team
And Z = X + Y = (X + X – 0.50) = total magnitude of PBP changes in the game
The Game Control percentage can then be calculated as:
GC1 = X / Z = X / (2X – 0.5) for the winning team
GC2 = Y / Z = (X – 0.5) / (2X – 0.5) = 1 – GC1 for the losing team
Thus, by construction, the winning team is guaranteed to have a higher GC than the losing team, and the dominance is determined by the total magnitude in the denominator. The larger the magnitude (i.e., the more competitive the game), the closer the GC value is to 50 percent.
Consider the 2024 Miami at Cal game, on October 6th, in which Cal stormed ahead and held a dominant 35–10 lead with eight minutes left in the 3rd Quarter. At that point, our win probability estimate for Cal was 98.9 percent. (ESPN’s was a comparable 98.7 percent, according to their GameCast.)
Then Miami began clawing their way back, ultimately winning 39–38 in one of the biggest comebacks in the recent years of college football.
Returning to ESPN’s GameCast version of this game, because Cal took such a massive lead and stayed ahead until late in the 4th Quarter, Miami’s average win probability was approximately 33 percent. In other words, their Game Control value essentially treated it as a rather convincing loss, not a dramatic comeback victory.
Comparatively, in the McIllece Sports version of GC (focused on win probability changes rather than the overall average), the calculation is:
X (Miami) = 2.91
Y (Cal) = 2.41
Z (Magnitude) = 2.91 + 2.41 = 5.32
GC1 = 2.91 / 5.32 = 0.547 -> 54.7%
GC2 = 2.41 / 5.32 = 0.453 -> 45.3%
The final result of the big Miami comeback was a GC metric of nearly 55 percent, indicating a highly competitive game that the Hurricanes ended up winning. Despite racing out to a huge lead, Cal squandered that advantage, so their GC appropriately has a lower value at 45 percent.
The ESPN formulation would tell you that, despite losing, Cal had considerably higher control of the game and deserve a bigger boost to their national GC ranking compared to the team that actually won the game…and that’s not how college football works. Winning matters, especially in the context of something as consequential as the College Football Playoff.
In real metrics terms, at the time of this post (Week 13 of the 2024 CFB season), consider a few of these ESPN resume rankings:
- Miami, with a 9-1 record against the #55 schedule, ranks #25 in ESPN’s schedule-adjusted Average Win Probability ranking
- Cal, with a 5-5 record against the #82 schedule, ranks #31 in the same set of rankings
Despite four more losses and a much weaker schedule, ESPN grades Miami and Cal as near equals when considering their average win probabilities over the course of the season. The reason is simple: Cal led some games they ended up losing, while Miami trailed some games they ended up winning, and defining Game Control relative to average win probability biases the results.
Should the CFP Committee believe those two teams have had comparable Game Control throughout the season?
Any college football fan could tell you: Absolutely not.