“analytics…had it as a solid go 4 or less, we were at 2 and I thought that was the play” – Whit
Whenever I hear football coaches talk analytics I cringe. I trust analytics when done properly, I’ve created and trained analytics teams with clients for years. I don’t trust football coaches using analytics for lots of reasons, but here are two.
First – most statistics being sold to athletic orgs are based on Bayesian models. Fully developed models include 3 parts (renamed by me for simplicity) – historical data, likelihood and summary. The top result of most of these models is overconfidence. Bayesian statistics is notoriously bad at handling real-world situations with complex (unpredictable) situations.
Second – the summary is highly influenced by historical data, for which there is basically none in this situation. Weber St and Arizona aren’t acceptable proxies for either current team and the sample size is too small. I assume the model contains data from previous seasons to add “credibility” to the analysis. If so, that’s criminal. Bottom line, there is no applicable data for that situation outside of the situational data – 4&2, down 3 points, late in 2nd quarter, playing on the road against amped up rival. That’s it.
The last major cringe is Whit’s follow up line: “just because it was a bad outcome doesn’t mean it was a bad decision.” That’s the marketing line for the analytics firms that sell these services. A decision based on a faulty data set is a faulty decision.
.
Loading ...
