The NHL Dives Into the Advanced-Stats Pool
News broke last week that the Edmonton Oilers had hired Tyler Dellow, a well-known blogger and advanced stats pioneer. The move follows on the heels of the Devils hiring Sunny Mehta, the Maple Leafs hiring Kyle Dubas as assistant general manager, and a mystery NHL team bringing aboard SB Nation’s Eric Tulsky. These days, it’s rare to go a week without hearing about yet another team staking out its territory in the world of advanced stats, or revealing that it has been there all along.
And just like that, hockey’s great Fancy Stats vs. Old School battle is over. It’s done. Soon it will seem quaint to remember that we ever argued about this stuff in the first place. There won’t be a victory parade or a formal surrender, but the numbers are here to stay, and we’re already seeing the inevitable transition from “These stats don’t tell me anything worthwhile” to “These stats don’t tell me anything I didn’t already know.” Before long the only reasons to still fight the battle will be ignorance, butt-covering, or tired shtick, and the rest of us will just move on. We won’t even hear about “advanced stats” anymore. They’ll just be stats.
It wasn’t hard to see this coming, of course, since the whole process has already played out in virtually every other major team sport. While the feud made good fodder for plenty of hot takes along the way, the end result was inevitable all along.
In recognition of the coming end of hostilities, and in an attempt to help any remaining stragglers climb onboard, let’s take one last look at some of the most common objections that have been raised to hockey’s analytics movement over the years, along with the fairly simple responses.
Advanced stats are nice, but they don’t tell the whole story
This is inarguably true. As has been well-documented, modern hockey stats are nowhere near the level of complexity and usefulness that we’ve seen in sports like baseball. There have been some significant discoveries, but there’s still a lot of work left to be done, and anyone who thinks you can replace an NHL scouting department with a spreadsheet is an idiot.
Luckily for us, those people do not exist.
That still seems to surprise some people. I’ve had conversations with more analytics folks than I can count over the last year, including most of the biggest names. I’ve followed the stats blogs. I’ve read as many of the reports as I could get my hands on, then read them again until I could start to understand them. I’ve asked questions, many of them hopelessly dumb, in an attempt to get my head around this stuff.
And in all that time, I’ve literally never once encountered anyone who thinks that stats tell the whole story. If anything, these guys tend to bend over backward to make sure you’re aware of how much they still don’t know, and how much work is left to do.
As for the common second half of this objection — “You still have to watch the games!” — well, the top stats guys do. They watch more hockey than you watch. They watch more hockey than I watch, and watching hockey is my job. They watch constantly, obsessively, poring over video of long-forgotten moments, because right now that’s the only way to get the good stuff.
Stats do not replace observation. They supplement, and support, and provide a reality check for those moments when your eyes try to lie to you. There’s a balance to be struck, and as time goes on and the stats get better, that balance might shift. But it will never come close to replacing a trained set of eyes, and nobody is trying to argue otherwise.
The stat guy with all the answers and no time for old-fashioned observation is a straw man. He exists only in the realm of anti-stat fantasy. Let him go.
If advanced stats are so great, why does [advanced stat] say [star player] isn’t as good as [obviously inferior player]?
This one’s basically Mad Libs for stats haters. Let’s try a few, using last year’s numbers:
• If advanced stats are so great, why does Corsi say Jonathan Toews isn’t as good as Tyler Toffoli?
• If advanced stats are so great, why does Fenwick say Sidney Crosby isn’t as good as Andrei Loktionov?
• If advanced stats are so great, why does Corsi Relative say Drew Doughty isn’t as good as T.J. Brodie?
Those conclusions would obviously be silly, and seeing them spelled out like that could make you want to doubt whether the stats guys have any clue what they’re talking about. But none of those stats make any claim to measuring how “good” a player is. They just measure how he does in one statistic. Under the right circumstances, that information can be useful. But it doesn’t say anything definitive about who the best overall player is, and it isn’t meant to.
Russell Wilson is a better quarterback than Andy Dalton, but last year Dalton threw for more touchdowns. Does that mean touchdown passes are a broken stat? No. It just means you can’t look at them in isolation. The same is true for hockey stats, advanced or not. Goals scored is a crucial stat we’ve relied on for years, but it doesn’t tell us everything — unless you want to argue that Chris Kunitz is a better player than Patrice Bergeron. Corsi, Fenwick, and friends are no different.
To some extent, this objection comes from some understandable confusion over what advanced stats claim to do. After all, there are stats like WAR in baseball that really do try to provide an overall ranking of player value. Hockey has had some attempts at a similar approach with mixed success (GVT is one example), but so far there’s no equivalent to WAR. Given how complex and free-flowing the sport is, there may never be.
No one stat is meant to definitively tell you that one player is better than another, so cherry-picking individual oddities doesn’t prove anything.
If advanced stats are so great, why do they say this team that got shelled last night played well?
There are any number of reasons why a team could generate great advanced stats and still lose. Small sample size, score effects, and just plain bad luck could all play a role.
But there’s a bigger issue in play here, and it’s one that’s not especially well understood: Today’s advanced hockey stats don’t actually do a great job of telling us what happened last night, or last month, or even over the first half of the season. You’ll occasionally see someone point out that teams that are badly outshot — which analytics tells us is a bad thing — often end up winning anyway, and that’s true. The new stats are just OK at telling us who won or lost. Luckily for us, we have better ones (like goals for and against, which in one-game samples have a 100 percent track record of picking winners).
The real power of advanced stats comes in telling us what’s going to happen in the future. They’re good at predicting who’ll win a playoff series, for example, and they’re a useful indicator of which teams may be over- or underperforming their true talent levels. In extreme cases, they can be a big flashing warning sign of impending doom.
That can pretty useful information to have. So look to advanced stats to tell you who’s going to win in the future, and let the scoreboard tell you who won last night.
If advanced stats are so predictive, why did this prediction turn out wrong?
Stats have a pretty good track record of predicting success and failure, but they still miss sometimes. If you relied on analytics, you didn’t see the Avalanche resurgence coming and you were probably high on teams like the Senators and Devils, who went nowhere.
So no, stats-based predictions aren’t perfect. But no predictions ever are. Go back and read your favorite expert’s preseason predictions, and you’ll find a ton of stuff that was wrong. (This idiot thought the Red Wings would be Cup contenders.)
Recent history tells us that advanced stats do a better job of predicting the future than just about anything else we have, including those experts. That still leaves a lot of room for error, but their track record is probably much better than yours.
The Leafs/Oilers/Devils hired a stats guy, so why aren’t they in first place?
OK, this one won’t arrive until the season starts, but you can pretty much bank on it showing up the first time a team that made a high-profile analytics hire has a losing streak.
Hiring a stats guy doesn’t guarantee success. Hiring a whole army of stats guys won’t guarantee success. And that’s especially true if you were already a bad team, like these ones. Everyone understands this, but by mid-October some people will pretend they don’t.
Some of these stats folks can be jerks sometimes
Well … yes. It’s certainly true that a subset of the analytics community can behave like jerks under certain circumstances. A smaller subset behaves like jerks pretty much all the time. That’s led to the stats community in general (especially as represented on Twitter) building a reputation for being thin-skinned, eager to pick fights over any perceived slight, and almost pathetically easy to troll.
They’re not alone — lots of old-school types are whiny and petulant too. It’s almost like we’re dealing with large groups of human beings, many of whom display the sort of traits that many human beings have.
But yes, many stats guys really can be jerks sometimes, and you should be free to factor all of this into how you feel about the people making the arguments. Just understand that none of it has anything to do with whether the arguments are right or wrong. A jerk who has the right answer is still right.
I just don’t like this stuff
Of all the common objections to analytics, this one is the most reasonable. Numbers aren’t for everyone, and nobody should have to feel like they need a math degree to enjoy sports.
Some movie fans love to watch the behind-the-scenes “making of” feature and learn as much as they can about a favorite film; others just want to watch the movie and be done with it. Neither side is right or wrong; they just have a different idea of how to enjoy something.
Similarly, if you just don’t like hearing (or reading) about advanced stats, that’s your call. Assuming you’re not getting paid to know this stuff, nobody should feel like they’re being browbeaten into knowing the numbers. Lots of fans have found that getting their heads around analytics helps them understand the game better and makes the whole thing more fun, but some fans prefer not to get into that level of detail. That’s fine, and you should feel free to tag out when the topic comes up.
Just remember that, again, your personal preference doesn’t mean the numbers are wrong. They’re not. That’s why the battle is over, and the numbers guys won.