The NFL is preparing to deliver results of its research on football helmets next month, but some observers are raising questions about who has been running the league's testing program, the type of tests it is using and the validity of its results.
Some independent experts and helmet makers believe the process is misleading and even potentially dangerous -- concerns that have prompted one manufacturer to withdraw from the ongoing program.
"We are no longer participating in the NFL's helmet testing process," says Vin Ferrara, CEO of helmet maker Xenith. "We have deliberately gone against the grain of their mechanical tests, we think to the benefit of players. And the very idea of saying we did well or didn't do well gives credibility to these tests that they don't deserve."
Last month, the NFL announced it was conducting research on football helmets as part of what it called "the latest in a series of steps over the past 15 years to address player safety with respect to concussions." The league's Helmet Concussion Assessment Program (HCAP) plans to present its conclusions to team equipment managers and the NFL Players Association Executive Committee in March. But the program is being run by men who say there has been essentially no serious brain injury in the NFL since 1996, and researchers have been subjecting helmets to tests not recognized as standard by the sports equipment industry.
"One problem is the people doing the testing are not at arm's length from the league," says Robert Cantu, a prominent sports neurosurgeon and vice president of the board of directors of the National Operating Committee on Standards for Athletic Equipment (NOCSAE). "It's also just not realistic to study concussions by looking at a small group where players hit other players like missiles, which is what the NFL is doing.
"Further, if you are a helmet maker trying to protect against these rare, maximal impacts, you're likely to make the foam or air cells in your helmets stiffer, because otherwise they would never pass these tests," Cantu adds. "But as you lose compressibility inside the helmet, lower-speed hits will have much greater impact on players than if there were softer material protecting their heads. You may get more concussions than you have now."
"The purpose [of the tests] is to provide more information about the performance of helmets to players and manufacturers," says NFL spokesman Greg Aiello. Aiello notes that the NFL Players Association has taken part in the helmet testing project, and that the league has hired two independent scientists -- David Meaney, chairman of bioengineering at the University of Pennsylvania, and Barry Myers, professor of biomedical engineering at Duke University -- who will review its research.
The NFL program tested a total of 11 helmets from five manufacturers between August and December, according to internal correspondence ESPN has obtained. Researchers have compared the force absorbed by each helmet to a baseline group of 12 older helmets, some used and some never worn, when hit by a "linear impactor," a machine that rams synthetic heads from the side, and that can be adjusted to strike at different angles.
"The linear impactor was simply a way to use compressed air to achieve higher collision dynamics than could be achieved by a pendulum," says Chris Withnall, a senior engineer at Biokinetics, an Ottawa engineering firm that is one of two organizations testing helmets for the NFL. "It simulates another player's head [and helmet] colliding with yours."
But NOCSAE has no official protocols for using linear impactors. Instead, it tests the shock absorbance of football helmets by dropping them onto a firm rubber pad from a height of 60 inches.
While such drop tests are not a perfect simulation of football collisions, they are the industry standard. Moreover, several helmet makers are skeptical about the linear impactor's ability to produce consistent test results.
"It's a home-ec project," says an executive of one helmet manufacturer who requested anonymity because the league has forbidden participants from discussing the testing program until it releases its results. "We have tried to recreate their research, and we get completely different numbers."
"We need tests that are standardized, leading to reproducible, predictable results," says Robert Erb, CEO of Schutt Sports, the leading maker of football helmets for youth football teams and the No. 2 supplier to the NFL, whose designs are among those the league is testing. "NOCSAE has that, and it has led to a reduction in injuries. The linear impactor, sadly, will take years to get there. And even then, we are going to have to watch to see whether players who are supposedly taking less impact to their heads actually get fewer concussions on the field."
Further, to determine how hard to strike helmets during testing, the NFL program is relying on widely criticized research conducted by its Committee on Mild Traumatic Brain Injury (MTBI). The group looked at injuries that took place from 1996 to 2001, but chose to study only those concussion-causing hits that researchers could re-enact from watching video. As a result, the committee analyzed 25 concussions out of a total of 787 that team doctors and athletic trainers reported over that period, and didn't find one case in which a player suffered a concussion while striking another player.
But this small database of mostly extreme, open-field collisions -- averaging 9.3 meters per second, or 21 miles per hour -- still forms the group that the league is using to study helmets today, leading to concerns such as those expressed by Cantu.
The NFL program also is delivering test scores to manufacturers without detailing how much force their helmets actually absorbed. Instead, its reports list only how individual designs performed when compared with the control group of older helmets, according to copies of test results ESPN.com has obtained.
For example, a helmet might experience 25 percent more gravitational force than the baseline group when struck directly on the side at 9.3 meters per second, but 15 percent less when struck on the jaw pad at 7.4 meters per second.
Even the researchers doing the tests find it difficult to explain how to interpret those results.
"A helmet with a 25 percent better score than baseline should not be thought of as 25 percent safer," says Dave Halstead, technical director of the Southern Impact Research Center in Rockford, Tenn., the other laboratory performing testing for the NFL. "I know that sounds confusing and contradictory and may make you think the data [is] of little value. I do not think that is the case. I think for the NFL player who is concerned about his choice in helmets, this will be helpful. It is not, however, the answer to concussions in the NFL, and it is not likely to be of value to other levels of play."
"'Twenty-five percent better' doesn't mean anything," says Blaine Hoshizaki, professor of human kinetics at the University of Ottawa, who has served as an engineering adviser to Xenith. "Every helmet on that list could be 25 percent better than a group of older helmets, or twice as good, without that translating into preventing concussions. They're not saying that because they can't say that."
Decades of research and evolving NOCSAE standards have led to modern helmets that are much better at preventing skull fractures and subdural hematoma than those created with older technology, but researchers are just beginning to understand the various factors that go into how helmets might affect concussion rates.
Nevertheless, with concern mounting inside and outside the NFL about concussions on the field whether inside or outside the NFL, the league decided last year it would move fast to get helmet information into the hands of equipment managers and players.
Last July 27, David Viano, then co-chairman of the league's MTBI committee, wrote in an e-mail: "At a recent meeting with the NFL equipment managers, there was a request for independent test data on the concussion performance of various helmets. We have received the same requests from players and have been unable to provide independent data. As a result, the NFL has decided to conduct concussion testing and provide the players and EMs with information in time for next year's helmet purchases and player fittings."
Since then, Viano has led the helmet testing program with substantial assistance from Elliot Pellman, the NFL's longtime medical adviser. Both men are steeped in the controversies involving the league's MTBI committee.
In December 2004, for example, Pellman, Viano and three co-authors wrote that there was "no evidence" in the committee's work of "widespread permanent or cumulative effects of single or multiple MTBIs [concussions] in professional football players."
In January 2005, five members of the committee, again including Pellman and Viano, wrote: "Return to play [of NFL players with concussions] does not involve a significant risk of a second injury either in the same game or during the season."
After criticism of his methods and behavior, detailed in a 2006 ESPN the Magazine investigation, Pellman stepped down as head of the concussions committee, and was replaced by Viano and Ira Casson, another member of the committee. Following further uproar about the committee's research -- and its denials of the long-term effects of concussions -- Viano and Casson also resigned as chairmen in November 2009.
But Viano didn't stop conducting research for the NFL, and Pellman never left the MTBI committee. That has surprised and dismayed some player health advocates.
"It's like putting Richard Nixon in charge of the Watergate investigation," says Brent Boyd, founder of Dignity After Football and a former NFL player who has battled the effects of concussions for decades.
Viano declined comment. Pellman also declined comment, except to say he is not running the helmet testing program.
Viano and Pellman first met with helmet makers on Aug. 17 at NFL headquarters in New York, according to e-mails exchanged among Viano and the attendees as well as a PowerPoint summary of the testing program, which ESPN has obtained. The researchers said there had been 787 concussions reported in the NFL from 1996 to 2001 and 758 from 2002 to 2007, but "essentially no serious brain injury." Then they outlined the program.
Several in the audience said they doubted that any NFL group could test helmets impartially, since the league has named Riddell its official helmet maker since 1989. (Under Riddell's deal with the league, NFL players can wear helmets made by companies other than Riddell, but helmets cannot display the names of those manufacturers.) Others were concerned about the program's proposed research methods. Pellman brushed off those questions at the August meeting, saying, "We need to do something quickly," according to two sources who attended the meeting.
Viano and Pellman met again with the helmet companies, along with a small number of team equipment managers, on Dec. 15 -- first as a group, then individually to examine initial test results. But on Jan. 21, Viano rejected the idea of meeting again as a group to discuss final test results.
As numbers have started to roll out, concussion researchers and helmet makers are wondering if the final measures will be more reliable or meaningful than the data revealed so far. Also at issue is whether the results will inevitably favor Riddell.
Even before testing started, the NFL program specifically aimed to "demonstrate improvements with 'modern' helmets over older models for concussion risk," according to the PowerPoint summary. And research results that generally favor newer designs are likely simply to shift players from Riddell's older VSR-4 model, worn by about 55 percent of NFL players during the 2008-09 season, to its more recent Revolution helmet, which was worn by about 25 percent of players.
"The NFL has selected its official helmet, and it has selected its helmet tests," says one helmet company executive. "It's a closed system."
"We are very comfortable with the testing," says Dan Arment, CEO of Riddell. "The NFL determined they were going to do this one way or another. But we are confident putting our helmets into any legitimate, reasonable test."
"Whatever comes out of this, we will take it as a data point," says Schutt's Erb. "But the NFL probably should have handed this project and a couple million dollars to a university. One with a bad football program."
Peter Keating is a senior writer for ESPN The Magazine.