Somewhere in your favorite sports franchise’s front office, a team of analysts is teasing the truth out of a mess of misleading statistics. Regardless of the sport or the data source — Corsi, SportVU, or Statcast — the analysts’ goals are the same: to capture contributions that standard statistics omit or misrepresent, and to find the positive indicators buried beneath superficial failures. The shot on goal that goes wide? In a sense, it’s a good sign, since it might mean more shots in the future, some of which will find the net. The line drive caught by a leaping outfielder playing out of position? A double would’ve been better, but even an almost-double tells us that the player who came close to extra bases has the skills to drive the baseball at a speed and trajectory that would typically lead to a hit. Not all outs are created equal.
Whether they know it or not — and nowadays, most of them don’t — all of these quants are re-proving the principle at the core of a product developed two decades ago by a company called AVM Systems, a small outfit founded by Ken Mauriello and Jack Armbruster, two businessmen based in the Chicago suburb of Wheaton, Illinois. AVM’s central insight sounds hackneyed now, but it was — to borrow a latter-day business buzzword — disruptive at the time: Process is important, because results are sometimes deceiving.
In the 1980s, Mauriello and Armbruster were coworkers at a large Chicago trading firm that studied the price of commodities in various financial markets, searching for smart buys and sells. “The guy who founded our company thought that the pricing was very inefficient and the markets were very inefficient to respond when volatility would change or when the markets would move quickly,” Armbruster says. “He generated systems that would create very high-tech and moment-by-moment information that traders could use on the floor to make trading decisions.”
Mauriello and Armbruster played complementary roles. “Jack was a trader,” Mauriello says. “I was an analyst, so I was upstairs feeding Jack the information and he was down there fighting, scratching, and clawing every day.” Armbruster adds, “The company did extremely well, not by taking big positions and hoping, but just by capturing the inefficiencies of the market that most people would look at and say, ‘They really don’t matter.’”
To Mauriello, now in his mid-fifties, and Armbruster, now in his early sixties, their day jobs’ sports applications were obvious. Mauriello noticed a number of inefficiencies lurking within baseball’s data bank — areas in which the most-cited stats were missing something or inspiring erroneous conclusions. Eventually, he approached Armbruster about building a business based on correcting the record.
The two fit the old school’s often off-base stereotype of statheads: Their playing experience is limited to softball, and while Mauriello grew up watching the Miracle Mets and Armbruster rooted for the Big Red Machine, neither describes himself as a die-hard fan. Armbruster admits that the pair’s second career arose “more [out] of a love for the analytics of the game than the game itself.” Both business partners believed that baseball’s nature as a series of discrete matchups made it the sport best suited to their style of analysis, so while they bemoaned the inaccuracy of quarterback ratings, they did so in their spare time. “In football, you’ve got 22 guys [moving] in a five-second span,” Mauriello says. “They’re all doing something, and then it’s over. Good luck.”
You might remember AVM (Advanced Value Matrix) from its cameo in Moneyball as the purveyors of then-Oakland assistant GM Paul DePodesta’s secret weapon, a system that helped the A’s determine (among other things) that the difference in defense between center fielders Terrence Long and Johnny Damon wasn’t large enough to justify the difference in salary. AVM did this, Michael Lewis wrote, by “collecting ten years of data from major league baseball games, of every ball that was put into play,” and then comparing the outcome of each individual play to the average outcome of all plays with similar characteristics.
Consider the case of a home run robbery, in which an outfielder perfectly times a jump and pulls back a ball from beyond the wall. Traditional stats would credit the outfielder with a putout, the pitcher with a batter retired, and the batter with an out made, making no distinction between the near-dinger and a lazy fly ball, even though the two types of plays tell us dramatically different things about the abilities of the players involved. AVM would chalk up most of a homer to the hitter, crediting the fielder and docking the pitcher by similar amounts. Home run robberies are rare, but by following a similar process for every play, AVM could arrive at a more complete accounting of players’ contributions on both sides of the ball.
As one would expect, the value of this exercise wasn’t always an easy sell to prospective clients. This was several years before the publication of the Baseball Prospectus study that eventually led to BABIP becoming a common fantasy tool, and the idea that luck made a meaningful difference in a player’s performance over the course of a 162-game season met with some resistance.
“They’d always say, ‘Well, it comes out in a wash,’” Armbruster says. “The hard liner that’s caught, but then a soft hit. We were showing them it usually does, but it doesn’t always come out in a wash. There’s always going to be that one player out of 20 who’s going to be pretty far off from what the numbers are showing. And there’s going to be one guy in the league who’s just off the charts. You need mathematics to understand that. To understand that if you flip a coin 20 times, it could come up heads 16 times. It doesn’t mean it’s a very talented coin. It’s the randomness of life.”
How AVM delivered this data in the mid-’90s, when its product was ready to pitch to teams, is still something of a mystery. This was long before the advent of automated play-by-play and pitch-by-pitch logs, let alone the ball- and player-tracking technology that makes outcome-independent metrics possible without having human eyes on every play. Lewis mentions that AVM reclassified balls originally recorded by Stats Inc., using a field diagram of its own design to mark locations and other methods1 to track trajectory and velocity. A 1996 Sports Business Daily report sheds some additional light, noting that “In conjunction with STATS, Inc., the pair had observers at each MLB game over the past four years equipped with grids to track the exact location of each ball put in play. They assigned numbers to each player reflecting their contribution on every play. The data is then fed into the AVM system, which assigns common numerical values for each player in three categories: offense, defense, baserunning. Pitchers get a fourth number.”
This is the same conceptual framework Baseball Info Solutions used to present similar work at the SABR Analytics Conference this spring, and teams are using or developing models designed to do the same (or better) based on HITf/x and Statcast. As one AL analyst told me, “Rereading the outcome-independent stuff in Moneyball, it holds up well with what we’re seeing today.”
AVM helped the Oakland A’s understand that Johnny Damon’s defense didn’t justify his salary. (Tom Hauck/Getty Images)
One would think that the march of time and technology would have rendered AVM’s mid-’90s efforts obsolete, but according to the company, which wouldn’t reveal its own process, that isn’t the case. “Some information simply isn’t as irrelevant or redundant as it may seem,” Armbruster says. AVM was so far ahead of its time that even 20 years later, the company won’t concede that the competition has completely caught up.
As advanced as AVM’s approach was, it took years for its product to migrate from Mauriello’s imagination to Moneyball. “We scratched on yellow pads from nine months to a year, then probably took another year or so to program,” Armbruster says. Eventually, the two retired from trading to devote themselves to AVM full time, soliciting investors with whom they created a limited partnership. AVM incorporated in 1994, and the founders spent 1995 marketing themselves to teams.
Being so far ahead of the rest of the baseball industry created an opportunity, but it also made it more difficult for the two to find open minds. “We talked to anybody we could talk to, and it was pretty frustrating in the beginning,” Mauriello says. “Everyone seems to have an uncle who knows somebody who knows the guy who cleans the stadium, who knows the manager. So we just followed every single lead we possibly could. Once in a while, we’d get on the phone with someone.”
Think about where baseball was in the mid-’90s. Moneyball was the better part of a decade away. Billy Beane was still an assistant GM; Theo Epstein was graduating with a degree in American Studies. Juan Gonzalez was winning an MVP award with a sub-4.0 WAR (not that anyone knew what WAR was) because he drove in tons of runs. Aside from a few Bill James fans — some of whom, after meeting on message boards, would soon band together to produce the first Baseball Prospectus annual — there was no concerted sabermetric movement. Some of the most prominent examples of statistics seeping into the game — Oakland manager Steve Boros’s computerized matchup stats, outspoken Red Sox stat guy Mike Gimbel — had made their teams into easy targets for the press. Front offices were full of former players, with few of the quants and Wall Street wonks who work for baseball teams today and who might’ve embraced AVM’s pitch. And into that inhospitable environment marched Mauriello and Armbruster, two men with no backgrounds in baseball who were trying to tell baseball lifers about things they were missing that math and computers could reveal.
Part of AVM’s strategy was keeping up the appearance of a successful consultancy, as well as mining the mere-exposure effect by being unavoidable. “We were always around them, or tried to be around them,” Armbruster says. “At winter meetings, GM meetings, wherever they were staying, we stayed there.” Mauriello adds, “We didn’t have a penny to our name, and we’re staying at $500-a-night Arizona Biltmore at the general managers meeting trying to rub shoulders with these guys, pretending like we belonged.” The startup wouldn’t have worked without the limited partners, as Mauriello and Armbruster had cut into their savings to stay afloat.
Gradually, AVM landed more and more meetings with major league teams. In each meeting, Mauriello and Armbruster would find a club’s brain trust arrayed around a table, their attitudes usually ranging from skeptical to resentful to fearful that computers were trying to take their jobs. AVM had to choose between tactics for turning a tough crowd. Crack a quick joke about the outcome of the meeting not mattering as long as the presentation was sound? Lead with a Gladwellian anecdote about a player who’s better than the traditional stats suggest? Make a PowerPoint?
Whatever numbers AVM chose to highlight for teams, they wouldn’t be presented on sleek-looking technology. The company needed more processing power than 20th-century portable computers could provide. “Back in the day we weren’t doing presentations with laptops,” Mauriello says. “We were carrying around two enormous boxes with an enormous monitor and an enormous tower. It was like Planes, Trains & Automobiles traveling around with that stuff. Watching a great big Gateway box with your monitor come tumbling out upside down, and you pick it up and it’s rattling. … So we’re in the hotel, saying, ‘Please lord, let this thing work.’”
The computers always worked, but the presentations didn’t. “Several times we would go into a meeting and some guys would say, ‘You know, I’m just not so sure about this computer stuff. What can a computer tell me that my scout of 42 years can’t tell me?'” Armbruster says. The intellectual inertia the pair often encountered was both freeing (because it meant the two presenters almost couldn’t screw up) and demoralizing (because it was usually bad for business). “It was either wow, this is amazing, or wow, this is B.S.,” Mauriello says. “Most of it wasn’t determined by the presentation, but by the previous position in the room.”
Typically, the two would start by discussing their backgrounds and success in securities and explaining why they thought their approach would work in baseball. Then they’d go over a few areas in which standard statistics were missing something: for instance, a power-armed right fielder who holds runners through reputation alone isn’t credited with any extra assists. Finally, they’d let the spectators loose. “We would actually go to the system and let them start looking at numbers, and of course they would all run to their own team to look at numbers on their own team,” Armbruster says. “They would start pointing and talking and whispering and pretty soon they’d have questions about guys on their own team, or maybe other guys, and we’d get into discussions that way.”
Although they rarely tailored their presentation to particular teams, they were always aware of their audience. “If there were players on their team that we knew were significantly off [from the consensus], especially if we were showing a player much weaker than he was looking statistically, we would certainly not make an issue of that because we knew it was going to become an issue anyway when they looked at it,” Armbruster says. “Sometimes we had to research in advance. We had to very gently explain, ‘We thought you may have some questions about that, so here’s a little research and here’s what we found. That even though he did this, he did this, and that too.’” Armbruster remembers having that conversation about the lower-than-expected baserunning value of Marlins center fielder Chuck Carr, who finished fourth in the 1993 NL Rookie of the Year voting after swiping 58 bags. Although Carr had led the league in steals, he’d also topped the NL in times caught stealing (22). The following season, he posted a better success rate, but he was also picked off several times.
The same problem popped up with whiff-prone sluggers whose power numbers weren’t as impressive as they appeared in the era’s inflated offensive environment — think Henry Rodriguez in 1996, or Sammy Sosa in 1997. “We had some guys who were fairly big hitters with home run numbers, but huge strikeouts and just very one-dimensional — didn’t move guys over, didn’t put the ball in play,” Armbruster says. “It was a home run or bust, pretty much, didn’t draw walks. Our system would not be that impressed. It would give them credit for the credit due, but if a guy hits 35 home runs in a year out of 550 at-bats, that’s 515 other at-bats to do something good or bad, and a lot of those were really bad.” AVM’s value lay in its capacity to put a number on the credits and debits that didn’t stand out immediately, but taking advantage of that value required executives who were willing to listen.
By staying humble and steering clear of potential stats vs. scouts disputes, AVM tried to ensure it wouldn’t alienate anyone. “I think that was one area that always was very offensive,” Mauriello says. “Like it’s either the numbers or the scouts — which one is right? And we would always go in there and say we do not feel like we are a replacement to the scout. We almost feel like we’re adding to the scout because these numbers in our system that may look interesting, now you can send the scout out and watch a couple of games and take a better look at them.”
In 2015, front offices are much more homogenous (and smart) than they were in the mid-1990s, when the contrast between teams was clear, and the inefficiencies easier to find. “Many organizations were very well run,” Armbruster says. “Some, the general manager would be in the meeting with three or four other guys around and nobody would say a peep unless the general manager addressed them, like he was in total control. And there were other meetings where the GM would have four or five guys around him and you could tell these guys were empowered and had some authority and they were asking hard questions and good questions, and the GM was just letting them operate and dig into our thinking. … It was just amazing, the difference in the infrastructure of the individual ballclubs.” Mauriello is more blunt. “With a few organizations, I just couldn’t believe the way they were making player decisions,” he said. “At that time, I didn’t see anybody else really doing what we were doing.”
Twenty years later, most memories of the meetings blend together into a montage of conference-room rejections. But AVM’s front-office computer parade, an analytical equivalent of the Great White Fleet, left both men with mental blooper reels, the lighter sides of sabermetrics’ troublesome birth. “We were in one meeting where it was just like being in a hornet’s nest,” Armbruster says. “Guys were all over us and looking at the system. One guy was talking to me on the side. One guy is talking to Kenny on the side. And some others guys are punching on the computer and stuff. Next thing you know, we hear the guys on the computer go, ‘Uh-oh,’ and Kenny and I go look. I don’t know what they had done, but they had gotten completely out of the system and done something and were looking at some of the code, behind-the-scenes spreadsheets and all this stuff. How they had gotten into that, I have no idea.”
Chuck Carr’s baserunning made plenty of eyes pop, but AVM’s system didn’t share the love. (Ronald C. Modra/Getty Images)
Although Armbruster and Mauriello acknowledge that many of the executives they dealt with were open-minded, they did come across some minds so tightly closed that no computer could pry them open. “Back then … they couldn’t wrap their brain around the fact that a guy could have a lot of errors and still be a real quality fielder,” Mauriello says. “We’d talk about that and say, ‘OK, the guy made 15 more errors than the average player at his position, but you know what, he got 30 more balls.’ Now, these are balls you don’t pay much attention to that kind of go through the infield like, ‘Oh, there’s a single.’ But he got to 30 more than the average player.”
A few front-office types also exhibited weak grasps of the degree to which a known deficiency could do damage. “We would show them, ‘Well, this catcher is really hurting you on the basepaths,’” Mauriello says. “And they would make a comment like, ‘Well, we don’t pay him to run.’ And I’m kind of just shaking my head, going, ‘Well the rules say that he has to run. You know this isn’t T-ball.’” Mauriello would never vocalize his incredulity, but he was “overall surprised, because in the financial industry, you have to know where the markets are,” he said. “You have to know everything. So I just assumed that when we’d go into a baseball front office, these guys would know everything that was going on. I was amazed that they’d look at a certain player and be surprised at why these were negatives or why these were positives. And then someone would open up a book and look at the statistics and go, ‘Oh look at that!’”
Despite the high hurdles, word of mouth started to spread. In 1996, AVM began to get some traction, and by ’97, the company was working with eight to 10 teams. “There were a lot of teams that were willing to nibble at it for a year or two and see,” Armbruster says.
Neither founder could recall a case where an owner would push for the product over a GM’s objections, since the front office was a firewall: If the GM wasn’t interested in AVM, the owner would never know about the company. Occasionally, though, it would work the other way. “We had a general manager who just loved it,” Armbruster recalls. “But he just knew, and his team knew, they would never get it through, even though we weren’t asking that much money at the time. … He just said, ‘I’ll tell you what, guys, if you load this up and get this in our office we will get you paid within the course of the year. I can’t sign a contract and I can’t pay you the monthly amount you’re asking on a regular basis, but we will get you paid.’ We just thought, ‘We’re just going to take his word for it.’ We worked with his team, set up the system with them, and had a relationship with them for a year or two. We would get a check for $3,412.88, and then two weeks later we’d get a check for $6,544.12. Then maybe a month and a half would go by, and then we’d get a check for $2,200, and then we’d get a check for whatever. It was just so funny, because you knew this guy was hiding it somewhere in their budget to pay us off because he liked it but he knew the owner would not.”
That ownership end around wasn’t an isolated event. “There was this sweeping division between player salary and this other stuff, these numbers, and that was a very big hurdle to get these guys to realize it,” Mauriello says. “We’d put a baseball cap on the computer and sit him in the dugout and call him the 25th man. We would do anything we could to try and get them to take that leap and say, ‘You know what, this thing really had value and has the type of value of maybe a man you’re sitting on the bench with.’” There wasn’t much precedent for paying large sums for stats, both because they hadn’t been available and because the analysis that third parties could do with data wasn’t sophisticated. To many executives at the time, an offer of information about one’s own players, or on opposing players they were already scouting, must have seemed like a scam. “[Teams] were used to getting a lot of data free, and even more extensive conventional data they might get for a $5,000 package or somewhere along there,” Mauriello says. “To pay significantly more for player information was hard for teams to swallow.”
Paradoxically, the more teams worked with AVM, the less valuable the company’s system was to any one of them, since its appeal depended on its capacity to make clients aware of areas other teams were ignoring. If AVM worked with everyone, any edge it conferred to individual users would evaporate. Before long, though, that potential problem took care of itself. “That became a little bit of a normal filter, because after a couple of years we started ramping up the pricing aggressively,” Armbruster says. “So it sort of separated the sheep from the goats, so to speak. We kind of knew who the teams were who were building trust. And our ultimate goal was not to work with a third or half the league. Our ultimate goal was to work with a small group of teams that really understood it and trusted it and got it.”
Once they were on the inside, AVM’s founders got glimpses of the way baseball worked when the cameras were off, as GMs balanced their occasionally conflicting internal/external roles as talent evaluators and spokesmen. “We had one meeting with a guy and we were waiting on the GM because he was in a press conference talking about the young team that he had, and saying, ‘We just love the talent we have, and we’re so encouraged and think this could be a breakthrough year,’ just very upbeat,” Armbruster recounts. “We got into the meeting and talked for a while and looked at his club and were telling him stuff. Then we went out on this deck overlooking the spring training facility and he just shook his head and said, ‘We have such a shit team.’ … It was just funny to see the dichotomy of what they have to represent sometimes versus what was really going on behind the scenes.”
They also found that while team personnel were sometimes hesitant to corroborate things the system said in front of fellow employees, they often secretly agreed and would reveal their true opinions at private moments. “Baseball front offices and scouts were, I think, very reluctant to be the guy who was the first one to say, ‘You know, I think this guy has lost a step,’ or ‘He’s not quite fielding the way he should,’” Armbruster says. “We would point that out sometimes and it would raise eyebrows, and then when we would get behind closed doors some guys would say, ‘You know what, I would’ve never said this publicly, but I really agree with that. I think he has lost a step.’”
The increasing adoption of AVM’s system was satisfying for Armbruster and Mauriello on an intellectual level, in the sense that their business model was working, but it also raised the psychological stakes. “We would just live and die when teams would make a major decision based on our information or heavily based on our information,” Armbruster says. “We would die a thousand deaths when the guy would start not panning out the way [the system suggested]. Or the guy we wouldn’t care for would suddenly have a hot streak. We were so, so glad when eventually those things ratified [our recommendation]. It’s not a perfect system. It’s not a crystal ball. But we have a really good track record, so it was always fun when that panned out.”
Conversely, having input on transactions put Mauriello and Armbruster in a position where they felt obligated to root against guys, which led to a lot of guilt. The only way to alleviate it was to develop the proper perspective. “There were some players that threw up some real red flags that teams ended up not signing them, and now I’m watching the guy feeling bad because I’m rooting against him,” Mauriello says. “He’s probably a nice guy and here I am hoping he fails … I should’ve known better, because in the financial industry you can’t cry because you had a losing day, because it’s about the long haul. And that’s what baseball is about, too.”
Billy Beane and the A’s have become the poster team for Moneyball, but little-known AVM helped fuel those breakthroughs. (Michael Zagaris/Getty Images)
AVM is a little like the Illuminati of analytics, or the sabermetric Second Foundation — a mostly unseen organization that’s exerted a subtle pull on the market since the dawn of the industry’s data-driven era. The company developed alongside the wider sabermetric movement, but its efforts almost never overlapped. Armbruster and Mauriello weren’t James disciples, and they felt that he and other sabermetric trailblazers, while vitally important as pioneers, had been held back by the quality of the data available to them. “At that time, they were still using what we felt were inefficient numbers and reworking them, but they were still using those numbers,” Mauriello says. “An example is the pop up that lands for a double — they were still using the double, but they were massaging it. … We felt right off the bat we can’t use these numbers. We’ve got to do something, because they’re inefficient. It’s a lot like building your house on sand, so for that reason we didn’t.” Even now that advanced stats are mainstream, AVM is only peripherally aware of the wider community’s efforts. “I peek at stuff every once in a while, but that’s about it,” Mauriello says. “We’re too busy doing our own thing to care about what other people are doing.”
So what is AVM’s “own thing” today? It’s difficult to say. The company is extremely tight-lipped not only about its early methodology, but also its post-Moneyball work and client list. DePodesta says it’s “probably been 10 years” since his last contact with the company. Both Stats Inc. founder John Dewan and longtime front-office exec (and current Reds scout) Bruce Manno, who served as an industry-approved brand ambassador for a fledgling AVM, declined to comment.2 Most MLB execs I surveyed either told me they hadn’t heard of AVM or weren’t aware that the company still existed.
According to one source with some exposure to AVM’s system, the company had two clients as of four to five years ago: one American League club and one National League club, each of which was paying AVM six or seven figures for league-exclusive access to its insights, including a win-value stat that the teams believed was far superior to the public alternatives. That sort of arrangement would explain why AVM doesn’t advertise or make its founders easy to find without extensive search-engine stalking. On the other hand, it’s surprising that teams would spend so much on consultants, given how much more access to information there is, and how much analysis most teams handle in-house. Moneyball mentions that the A’s were able to replicate AVM’s system after a year or two of use, and other teams have had 15 years since then to be copycats. One team analyst theorizes that AVM still commands such a sum because “Some teams may face the obstacle of not knowing who to hire and how to verify that whoever they hire does a good job at creating an AVM-type system.” In other words, despite the lofty price tag, AVM’s longevity and track record (“As mentioned in Moneyball!”) might be an easier sell than taking a risk on a more recent startup. On the other hand, AVM has had plenty of time to create killer apps, so the company’s new clients could be getting more out of their deals than DePodesta did.
The minds behind AVM know they entered the market at the ideal time, even though the late adopters weren’t ready to listen. “We were fortunate, in a sense,” Mauriello says. “It seemed like the technology was just there. For instance, we wouldn’t have been able to start in ’90, because I don’t think there was enough computer power. We basically had just enough computer power, so every time — the Pentium, the 386, the 486 — we were constantly needing more and more.”
Today’s computer power is almost unlimited: For all we know, AVM might be MLB’s mystery supercomputer caretakers. But the competition has increased almost as exponentially. “We’re certainly glad we had a 20-year head start on the world,” Armbruster says. “We were way ahead and sabermetrics was so brand-new, and people didn’t really understand, so that was nice. In that regard, the technology and the advancements and all the thinking and capabilities out there are certainly creating more competition, but the advancement in technology has also given us the ability to expand what we do in terms of the resources and the research and capabilities we have.” Mauriello allows that uncovering advantages is “probably a little bit harder” than it was when they had the sector to themselves. “But we’ve been doing this a while,” he says, “so we keep doing smart things ourselves.”
It’s taken the online sabermetric community two decades to approximate what AVM was working on before an online sabermetric community existed. So while inquiring minds would like to know what Mauriello and Armbruster are up to now, the answers might have to wait until 2035.
