Buying a car is a hard decision. There are just so many variables to think about. We’ve got to inspect the interior and analyze the engine, and research the reliability of the brand. And then, once we’ve amassed all these facts, we’ve got to compare different models.
How do we sift through this excess of information? When consumers are debating car alternatives, studies show that they tend to focus on variables they can quantify, such as horsepower and fuel economy. (Psychologists refer to this as the “anchoring effect,” since we anchor our decision to a number.) We do this for predictable reasons. The amount of horsepower directly reflects the output of the engine, and the engine seems like something that should matter. (Nobody wants an underpowered car.) We also don’t want to spend all our money at the gas station, which is why we get obsessed with very slight differences in miles per gallon ratings.
Furthermore, these numerical attributes are easy to compare across cars: All we have to do is glance at the digits and see which model performs the best. And so a difficult choice becomes a simple math problem.
Unfortunately, this obsession with horsepower and fuel economy turns out to be a big mistake. The explanation is simple: The variables don’t matter nearly as much as we think.1 Just look at horsepower: When a team of economists analyzed the features that are closely related to lifetime car satisfaction, the power of the engine was near the bottom of the list. (Fuel economy was only slightly higher.) That’s because the typical driver rarely requires 300 horses or a turbocharged V-8. Although we like to imagine ourselves as Steve McQueen, accelerating into the curves, we actually spend most of our driving time stuck in traffic, idling at an intersection on the way to the supermarket. This is why, according to surveys of car owners, the factors that are most important turn out to be things like the soundness of the car frame, the comfort of the front seats and the aesthetics of the dashboard. These variables are harder to quantify, of course. But that doesn’t mean they don’t matter.
The horsepower effect is part of a larger category of cognitive errors in which our obsession with information — we always want to know more — undermines our decisions. In one classic experiment, done at MIT in the late 1980s, people given access to an unlimited amount of information about the stock market earned half as much money from their investments as people given virtually no information. Although the “high information” group was far more confident, their confidence was misplaced. They knew more, but understood less.
But this is not a column about cars. My worry is that sports teams are starting to suffer from a version of the horsepower mistake. Like a confused car shopper, they are seeking out the safety of math, trying to make extremely complicated personnel decisions by fixating on statistics. Instead of accepting the inherent mystery of athletic talent — or at least taking those intangibles into account — they are pretending that the numbers explain everything. And so we end up with teams that are like the worst kind of car. They look good on paper — so much horsepower! — but they fail to satisfy. The dashboard is ugly, the frame squeaks, and the front seats make our ass hurt.
This is largely the fault of sabermetrics. Although the tool was designed to deal with the independent interactions of pitchers and batters, it’s now being widely applied to team sports, such as football and basketball. The goal of these new equations is to parse the complexity of people playing together, finding ways to measure quarterbacks while disregarding the quality of their offensive line, or assessing a point guard while discounting the poor shooting of his teammates. The underlying assumption is that a team is just the sum of its players, and that the real world works a lot like a fantasy league.
In many respects, sabermetrics has dramatically improved personnel decisions. By relying on unusual measurements of performance, such as base runs and plus-minus ratings, teams have been able to identify neglected talent, whom they can sign on the cheap. Sabermetrics has also helped sports executives double-check their instincts. Instead of blindly trusting some errant whim — and thus making a terrible trade or picking the wrong free agent — they can consult the math. If the Giants had trusted the numbers, for instance, they wouldn’t be saddled with Aaron Rowand’s five-year, $60 million contract. (He batted .230 last year.) They would have realized that his
OBS and OPS is pretty mediocre, especially once his two outlier seasons are taken into account.
For a nerd like me, this quantification of sports has been tremendous fun. Thanks to obsessive websites, even the casual fan now has access to statistical tools that would have boggled the mind of a GM 10 years ago. Sabermetrics has also transformed the act of being a spectator, so that watching a game is no longer just about cheering for our hometown team. The numbers have given us a whole new way to think about sports, elevating the conversation beyond disappointed groans, ecstatic high-fives, and subjective opinions.
But sabermetrics comes with an important drawback. Because it translates sports into a list of statistics, the tool can also lead coaches and executives to neglect those variables that can’t be quantified. They become so obsessed with the power of base runs that they undervalue the importance of not being an asshole, or having playoff experience, or listening to the coach. Such variables are the sporting equivalent of a nice dashboard. They can’t be quantified, but they still count.
This is the moral of the Dallas Mavericks. By nearly every statistical measure, the Mavs were outmanned by most of their playoff opponents. (According to one statistical analysis, the Los Angeles Lakers had four of the top five players in the series. The Miami Heat had three of the top four.) And yet, the Mavs managed to do what the best teams always do: They became more than the sum of their parts. They beat the talent.
Consider the case of J.J. Barea. During the regular season, the backup point guard had perfectly ordinary statistics, averaging 9.5 ppg and shooting 44 percent from the field. His plus/minus rating was slightly negative. There was no reason to expect big things from such a little player in the playoffs.
And yet, by Game 4 of the NBA Finals, Barea was in the starting lineup. (This promotion came despite the fact that he began the Finals with a 5-for-23 shooting slump and a minus-14 rating.) What Dallas coach Rick Carlisle wisely realized is that Barea possessed something that couldn’t be captured in a scorecard, that his speed and energy were virtues even when he missed his layups (and he missed a lot of layups), and that when he made those driving floaters their value exceeded the point score. Because nothing messes with your head like seeing a guy that short score in the lane. Although Barea’s statistics still look pretty ordinary — his scoring average fell in the Finals despite the fact that he started — the Mavs have declared that re-signing him is a priority. Because it doesn’t matter what the numbers say. Barea won games.
I’m thinking here of a Philip Roth metaphor. When asked by David Remnick, in a 2000 New Yorker profile, how he felt about a cramped literary interpretation of one of his novels, Roth busted out a sports analogy. He imagined going to a baseball game with a little boy for the very first time. The kid doesn’t understand what’s happening on the field, and so his dad tells him to watch the scoreboard, to keep track of all the changing numbers. When the boy gets home someone asks him if he had fun at the game:
“It was great!” he says. “The scoreboard changed thirty-two times and Daddy said last game it changed only fourteen times and the home team last time changed more times than the other team. It was really great! We had hot dogs and we stood up at one point to stretch and we went home.”
If that little kid were around today, he’d be obsessed with sabermetrics. He’d almost certainly win his fantasy league, but he’d miss the point of the game. Sure, he wouldn’t have squandered center field on Rowand, but he also wouldn’t have started Barea or bet on the Mavs. His car would have way too much horsepower and shitty seats.
Here’s my problem with sabermetrics — it’s a useful tool that feels like the answer. If we were smarter creatures, of course, we wouldn’t get seduced by the numbers. We’d remember that not everything that matters can be measured, and that success in sports (not to mention car shopping) is shaped by a long list of intangibles. In fact, we’d use the successes of sabermetrics to focus even more on what can’t be quantified, since our new statistical tools take care of the stats for us. We are finally free to think about how those front seats feel.
But that’s not what happens. Instead, coaches and fans use the numbers as an excuse to ignore everything else, which is why our obsession with sabermetrics can lead to such shortsighted personnel decisions. After all, there is no way to quantify the fierce attitude of a team that feels slighted, or the way even the best players can be undone by the burden of expectations, or how Kendrick Perkins meant more to the Celtics than his rebounding stats might suggest.2
But Nenad Krstic looks so good on paper!
For reasons that remain mysterious, some teammates make each other much better and some backup point guards really piss off Ron Artest. These are the qualities that often determine wins and losses, and yet they can’t be found on the back of a trading card or translated into a short list of clever equations. This is the paradox of sports statistics: What the math ends up teaching us that is that sports are not a math problem.
Jonah Lehrer is a contributing editor at Wired and the author of How We Decide. This is his first story for Grantland.