The tournament is not a referendum

We’ve all seen them because they are prolific. Every year in March come the pronouncements about conference X being better than conference Y because they have more teams in the Sweet 16 or because more teams won their first round game. Certain coaches must be better because they have advanced, others are bad because they lost. Here is a sample tweet to familiarize yourself with the genre.

According to the National Library of Medicine’s paper on how sample size influences research outcomes, “very small samples undermine the internal and external validity of a study.” I am here to posit that a single game, especially one in which Utah State went stone cold, is a very small sample from which only the woefully uninformed would attempt to draw a conclusion.

The NCAA tournament is made up of these tiny samples. I can’t imagine anyone is actually arguing that Fairleigh Dickinson is better than Purdue right now, but that’s exactly the same logic. The reason this is called March Madness is because of the inherent randomness. Sometimes the best team wins, sometimes it doesn’t. Sure, a long set of seven game series would give us a more “deserving” winner, but that’s also very boring.

Quick: what is the best basketball conference in the nation? I’ve read multiple thoughts breaking this down today, but it’s still the Big 12, despite the fact they have only two teams in the Sweet 16 and the Big East and SEC have three. Likewise, the Ivy is not on par with the Big 10 because they have the same number of teams in the Sweet 16. Anyone making that claim would be laughed down, as should anyone making any proclamations based on who is still playing. These 16 teams beat the teams in front of them, that’s about all that can be gleaned from that.

Where data can be found is where the sample is larger. Matt Painter, coach of Purdue, just keeps losing in the tournament with high ranked teams. In 18 seasons he has one Elite Eight despite having multiple teams that have finished in the top 15 of KenPom. It’s reasonable to draw from 18 years of data to determine that winning one off games may not be his strong suit. Sean Miller, on the other hand, is now 8-0 in the second round. That sample may be large enough to at least consider starting a conclusion.

Maybe your conference is very good, maybe it’s the American. Maybe your coach is really good, maybe he’s Matt Painter. Ultimately, success or failure in a tiny sample of high pressure games is not a good way to draw a conclusion about much of anything. No one with a great deal of sense would even try. Sit back and enjoy the Madness. The only sure thing is that it will be just as crazy next year.