Analyzing the regular-season performance of college basketball teams has always been difficult. The vexing problem is how to determine whether a team performed below or above its abilities. It’s a question that seems to exist in a vacuum, because really, what standard could we hold a team to besides our preconceived notions from October, which may be fatally flawed in the first place?
This year, though, I think I may have accidentally stumbled upon an interesting answer. Last season, a group of fellow college basketball nerds and I were seeking an alternative to a really addictive stock market pool that involved making investments in teams and watching their value (and our portfolios) rise and fall based on results throughout the year. As you might imagine, this involved an insane amount of work for the person in charge, so we scrapped it for this season.
In our efforts to find a simpler substitute, we hit on the idea of using stat guru Ken Pomeroy’s projections for each team’s conference record (18 games, in most cases) before conference play began, and gambling on which teams would live up to those projections and which would fall short. By December 30, Pomeroy had two months of nonconference results to mine from each team: how they played, whether they won, and what they could be expected to do in conference play. We decided to track teams from eight conferences: ACC, SEC, Pac-12, Big Ten, Big 12, Big East, AAC, and the Mountain West.
The system wasn’t perfect; teams peak at different times, and the nonconference schedule is hugely unbalanced. But at least it was a start — for the first time, we had intelligent projections based on real analytics from the same year, rather than preseason conjecture.
A confession: The pool was only so-so, by pool standards. We picked 10 overachievers and underachievers, based on Pomeroy’s up-to-date conference stats, used special bailout and freeze dates to maximize our points, and mostly just watched the leader board as the season evolved. But what the pool lacked in drama, it made up for in the huge amounts of data left over when it ended.
That’s where my pal Taylor Doherty comes into play. He created and ran the master spreadsheet (which was incredibly complicated, by the way; certain cells were so stuffed with byzantine formulas that I’m pretty sure they contained the coded secrets of the human soul), and is responsible for all the graphics that follow.
Here’s what we found:
• It probably won’t surprise you to know that the Oklahoma State Cowboys, who were ranked eighth in the preseason and sixth on December 30, took the booby prize as the year’s biggest underachievers. KenPom had the Cowboys slated for a 14-4 conference mark, and their 8-10 finish gave them a minus-6 total, “tops” in Division I. Butler, in its first season without Brad Stevens, was close behind at minus-5 in the new Big East, and Virginia Tech’s disastrous season in the ACC was also good for minus-5.
• What may surprise you is that the Georgia Bulldogs were the biggest overachievers on the season, with a 12-6 record that inverted Pomeroy’s 6-12 projection. Michigan and Virginia, who both had rough early-season results before winning their respective conferences, were close behind with plus-5, and six teams (Washington, Nevada, Florida, Nebraska, Oklahoma, Houston, and Cincinnati) finished at plus-4.
Check out the full results in Taylor’s graphic:
Let’s also take a closer look at the AP top 12 as of December 30, and how each team fared in conference play:
For the breakdown of each conference in graphic form, check out the links below:
ACC
Big 12
Pac-12
Big Ten
SEC
Big East
Mountain West
AAC
What does this data actually mean? If a team underachieves, what does that signify? Putting aside the vagaries of how teams change over the course of the year for a moment, my first instinct was to think that the numbers told us more about the coaches than the players — how well they were able to shape their teams to reach maximum potential.
Viewed through that lens, it’s almost funny, in some cases, how well the data correlates to what we already believe about certain coaches. Michigan’s John Beilein is almost universally considered a great game coach. What he accomplished this season, after losing Mitch McGary, was remarkable. Winning a Big Ten championship after enduring a good deal of adversity in the early part of the season is praise enough, and Michigan’s plus-5 mark bears out the achievement.
The trend continues with Virginia (Tony Bennett led the Cavaliers to a 16-2 ACC record), Florida (Billy Donovan overachieved his projection by four games), and Nebraska (many thought Tim Miles had committed career suicide by leaving Colorado State for the Cornhuskers, but has coached his team to the NCAA bubble).
On the opposite side of the spectrum, Oklahoma State’s Travis Ford has long been thought of as a top-notch recruiter who struggles as a game coach, and putting up an 8-10 mark with Marcus Smart on his team only exacerbates that image. Others who fall into that same category: John Calipari at Kentucky and Homer Drew at Baylor. They both ended up negative on the season.
But some of the results are less stereotypical. Look, there’s Rick Barnes with a solid plus-3 for Texas! And Thad Matta with an odd minus-3 for Ohio State. Numbers like those make me long for multiple years of data, because surely neither one is representative.
A team like Ohio State managed to rack up a 13-0 record in November and December without really playing a good team. Even in Pomeroy’s weighted metrics, that’s going to create unrealistic expectations in comparison to teams that played in early-season tournaments against strong, ranked opponents. Ohio State’s big weakness, offensive production, was effectively hidden against weak competition. Pomeroy’s 13-5 projection for the Buckeyes in the Big Ten may never have been realistic, and in some ways Matta had nowhere to go but down.
When expectations are high, and teams are very good, there’s much less room for deviation from projections. Of the top 12 teams on December 30 shown in the graphic above, only three varied from their original projections by more than two games. Those were Ohio State, with its aforementioned schedule issues; Florida, who benefited from playing in a weak SEC; and Oklahoma State, a team that entered a free fall. The rest — Syracuse, Arizona, Wisconsin, Duke, Baylor, Oregon, Villanova — stayed mostly within the confines of their projections. This was the case even if, as with Wisconsin and especially Oregon, the teams took significant dips in the rankings.
These numbers, to some extent, help us evaluate the strength Ken Pomeroy’s prediction system. Here’s the percentage breakdown of how he fared in his analysis of 94 teams on December 30, rounded to the nearest percentage:
Correct projection: 16 teams, 17 percent
Projection +/- 1: 34 teams, 36 percent
Projection +/- 2: 15 teams, 16 percent
Projection +/- 3: 12 teams, 13 percent
Projection +/- 4: 11 teams, 12 percent
Projection +/- 5: 4 teams, 4 percent
Projection +/- 6: 2 teams, 2 percent
So, if KenPom says you’re going to finish with a certain record in conference, there’s a 69 percent chance that you won’t deviate from that by more than two games, and a 94 percent chance you’ll be within four games. The first number actually seems more impressive than the latter, and to me at least, it’s an endorsement of his system.
And there you have it. It’s probably best to be cautious with these numbers, and they’re dependent on so many factors, putting the weight squarely on the coaches is unfair. But they are useful at seeing just how a program succeeded, failed, or stayed level with expectations within a calendar year. With multiple years of data, it could prove even more valuable. For now, though, it’s safe to say that Georgia should at least give head coach Mark Fox a nice, appreciative pat on the back.