Commentary

Where do sports stats end?

Experts, team execs debate limits of metrics at MIT conference

Updated: March 7, 2010, 3:58 PM ET
By Brian MacPherson | Special to ESPNBoston.com

BOSTON -- Don't count St. Louis Cardinals assistant general manager John Abbamondi among those excited about the new developments in baseball teams' ability to chart defense -- but it's not for the reason you might think.

The advent of Hit f/x and Field f/x technology has a chance to change the way teams evaluate defense. Batted balls can be measured by speed and direction. Player positioning and field factors can be taken into consideration. Aspects of baseball once considered beyond measurement might be on the verge of joining on-base percentage and slugging percentage as easily measured and easily turned into statistical comparison.

If Abbamondi had his way, that type of data wouldn't be quite as accessible, he said, referring to a pine tar problem that has nothing to do with Kenny Rogers.

"If you are the only club that has pine tar, you have a pretty big advantage," Abbamondi said Saturday at a sports analytics conference hosted by MIT's Sloan School of Management. "But because everyone has pine tar and it's become a cost of doing business, you're no different competitively if all of us have it or none of us have it.

"I worry that Field f/x will make defensive evaluation too easy. I'd rather have a harder problem to solve and solve it slightly less well than to have everybody solve it. There's a difference between the search for the truth and the search for a competitive advantage."

[+] EnlargeBill Polian
Brian Spurlock/US Presswire"Geekdom, if you will, provides us with wonderful tools to do that. I would add this caveat: Speak English, please," said Indianapolis Colts president Bill Polian.

A panel focused on baseball analytics was only one of more than a dozen topics covered at a conference that featured, among others, Dallas Mavericks owner Mark Cuban, "Moneyball" author Michael Lewis, Indianapolis Colts president Bill Polian and ESPN executive editor John Walsh. More than 1,000 people gathered for the keynote panel discussion, a devil's-advocate conversation moderated by Lewis entitled "What Geeks Don't Get: The Limits of Moneyball."

Words such as "covariance," "granular data" and "recency bias" were thrown around as if it were a high-level economics or statistics convention having nothing to do with sports. With sports growing into big business, though, an understanding of statistics is as important to building a team as an understanding of the pick-and-roll.

"We are constantly looking for the next undervalued asset," Polian said during the keynote discussion. "Geekdom, if you will, provides us with wonderful tools to do that. I would add this caveat: Speak English, please. I have a very hard time understanding all the mathematics of it. I have a lot of work to do and a lot of decisions to make, and if you can't make it understandable for me, I have no use for it."

Two of the most pertinent discussions -- particularly in Boston, where the Red Sox have unveiled a radically overhauled roster and the Patriots have started the process of revamping theirs -- involved the way data analysis has become commonplace both in baseball and in several other sports that don't easily lend themselves to the type of statistics upon which decisions can be based.

Context matters. Sample size matters. Much can be gleaned from watching a quarterback take 10 seasons' worth of snaps out of the same formation and throw 10 seasons' worth of passes to the same receivers, but that sort of data simply is impossible to compile. Adjustments have to be made to compensate.

"We don't get the continuity to work on our models," said Simon Wilson, the head of performance analysis for the Manchester City soccer team in England. "Players change. Managers change."

Red Sox outfielder Jacoby Ellsbury unwittingly became a flash point this winter for the validity of emerging defensive statistics. Ellsbury ranked among the worst defensive center fielders last season, something many find hard to believe given his sensational raw speed and the numerous highlight-reel catches he made in the outfield.

Red Sox general manager Theo Epstein recently disputed the validity of those statistics, asserting that the team's internal evaluations peg Ellsbury as an above-average defensive outfielder. Even the creators of some of those statistics have tried, often futilely, to make the point that it takes more than one season's worth of defensive plays to form an accurate conclusion about a player.

Defensive ability also requires a certain context: An outstanding center fielder can take some of the onus off a subpar right fielder, and a lousy shortstop can make an average third baseman look bad. Even then, it's hard to know how much is instinct and coaching and how much is actual athletic ability -- and Ellsbury isn't the only one who raises questions.

"With Chase Utley, we found he was so good because he was ultrapositioning," said panelist John Dewan, whose baseball Fielding Bible measures defensive players on a plus-minus basis. "Against left-handed hitters, he was moving way over toward first. Against right-handed hitters, he was moving way over toward second. That made a big difference for him."

[+] EnlargeChase Utley
AP Photo/Eric GayChase Utley's ultrapositioning for lefty and righty batters greatly accounted for his fielding success, according to a panelist at the stats seminar.

Football, of course, provides much more of a challenge when it comes to statistical analysis. Unlike baseball, a sport with one-on-one battles between pitchers and hitters not influenced much by outside factors, almost everything about football must be evaluated in context. A running back can look great with talented blockers in front of him, but blockers can look great with a talented running back behind them.

Vince Wilfork, Polian said, would hold an entirely different value in the Colts' system than in the Patriots' system because what the Patriots do is "antithetical to ours."

"Ninety to 95 percent of the action in the NFL is equivalent to what they're doing with defense in baseball," said panelist Aaron Schatz, the creator and president of FootballOutsiders.com. "You have to couch it in a team context. When we say that Peyton Manning was 40 percent better than the average quarterback last season, we mean in that system with those blockers. You've got to look at it in context."

In football -- as well as in sports such as basketball, hockey or soccer -- understanding context is a crucial step toward finding players who can succeed in that context.

"You have to respect the fit with the culture and with the city," said Wilson, who faces the added challenge in the Premier League of assimilating international players onto his team far more often than any of the major sports in the United States. "The challenge for the organization is to know what its fit is. If you can identify that, you're halfway there."

Although baseball has embraced the statistical revolution -- in part because it doesn't take hours of film breakdown to develop its statistics -- the NFL has lagged behind.

Schatz lamented the lack of publicly available information about defensive statistics beyond sacks and tackles -- hurries, for example -- or player participation logs that can affect bonus payouts. Even during games, the NFL's ban on technology in coaches' boxes lends something of a Stone Age flavor to strategy development.

"You have a guy writing down players with a pen and paper, what play was run, how many yards were gained," said Paraag Marathe, the executive vice president for football and business operations of the San Francisco 49ers. "At halftime, a coordinator will ask, 'How successful were we on sweeps to the right?' He has to add it all up: 'Well, we had 13 runs that averaged out to 2.2 -- no, wait, 2.6 yards."

Polian emphasized the point with a detailed breakdown of the much-discussed fourth-and-2 that Patriots coach Bill Belichick attempted against Indianapolis last season. Polian broke down all the different aspects of that play -- the Colts' hot offense, the Patriots' banged-up defense, the tendency of the Patriots to run quarterback sneaks in short-yardage situations rather than throwing the type of pass that was called -- and backed up the decision.

Here's the funny thing: Polian agreed with the decision not because he believed the statistical data that supported it, he said, but because it made sense in that specific moment with those specific players on the field.

"Was it the right call? In my opinion, it was 100 percent the right decision," Polian said. "All of the statistical analysis that's done over the course of the season means nothing. The situation on the field at the time dictates the decision."

Statisticians still can find common ground with Polian: All the data in the world work only if they are put into the hands of coaches and players who are able to apply it correctly in a given situation. A trade or a free-agent signing can work only if a player is motivated to keep putting in the same work he has before the trade or free-agent signing. The will to get better, the will to work and the will to win still have value, intangible as they are.

"Just because we have trouble measuring it doesn't mean it doesn't exist," said Shiraz Rehman, director of baseball operations for the Arizona Diamondbacks. "That might be the most important factor. Make, personality, things along those lines -- clubs are searching for those answers."

Brian MacPherson is a frequent contributor to ESPNBoston.com. His e-mail address is brianrmacpherson@gmail.com.

SPONSORED HEADLINES

EDITORS' PICKS

MORE BOSTON HEADLINES