NFL's concussions expert also sells equipment to league

The National Football League's director of neuropsychological testing is also the chairman of a company that sells testing software to NFL teams, a dual role which raises questions about conflicts of interest.

Mark Lovell, director of the Sports Concussion Program at the University of Pittsburgh Medical Center, oversees neuropsychological testing programs for the NFL. In that capacity, he has helped teams use neurocognitive tests -- which essentially grade subjects on their memory and reaction time -- to help team doctors make decisions about when injured athletes can return to play. This season, baseline neuropsychological tests will be mandatory for all NFL players for the first time.

In the late 1990s, Lovell and Joseph Maroon, clinical professor of neurological surgery at the University of Pittsburgh and team neurosurgeon for the Pittsburgh Steelers, developed their own computer-based battery of tests, calling it the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) system. Together with Michael Collins, assistant director of the Sports Concussion Program at Pittsburgh, they launched a company called ImPACT Applications to make their product commercially available. Today, Lovell is chairman and software developer at ImPACT Applications, Collins is chief clinical officer and Maroon is chief medical officer.

At the same time, Lovell and Maroon are members of the NFL's Committee on Mild Traumatic Brain Injury (MTBI), which conducts research projects designed to help the league better understand and manage concussions.

Lovell is also a consultant to the Steelers, and oversees neuropsychological testing programs for the Indianapolis Racing League (IRL) and CHAMP Car Racing. From 1997 to 2007, he co-directed the National Hockey League's neuropsychology program.

Lovell's overlapping roles and financial interest in ImPACT have drawn criticism from several doctors and athletic trainers working in the field of sports concussions. Their ire has intensified as Lovell sometimes has not identified himself as one of ImPACT's developers in his scientific research. On at least seven occasions since 2003, Lovell has authored or co-authored studies on neuropsychological testing, including papers directly evaluating ImPACT, without disclosing his roles in creating and marketing ImPACT, according to an ESPN.com review of recent medical literature. In one case, an examination of Lovell's connections prompted an academic journal to rewrite its disclosure guidelines for authors.

"It is a major conflict of interest, scientifically irresponsible," says Christopher Randolph, professor of neurology at Loyola University Medical Center in Chicago and former team neuropsychologist for the Chicago Bears. "We are trying to get to what the real risks are of sports-related concussion, and you have to wonder why they are promoting testing. Do they have an agenda to sell more ImPACTs? And if you're writing a paper and you have anything to do with a company involved, it's imperative that you disclose it."

Earlier this year, NFL commissioner Roger Goodell ordered all teams to implement baseline neuropsychological tests. ImPACT, which is one of a handful of computerized neuropsychological systems available (CogSport, the Concussion Resolution Index, and the Automated Neuropsychological Assessment Metrics are among the others) has since become the league's de facto standard testing system. Thirty of the NFL's 32 teams now use ImPACT, according to the company's Web site.

"I can see why the league would want one standardized test," says a leading neurosurgeon, who asked to remain anonymous because his patients include former NFL players. "I can't for the life of me understand why they would want that standard to be a test that is owned by two members of the [MTBI] committee."

Lovell declined to reply by e-mail or telephone to questions sent to him by ESPN.com. He also declined a request to be interviewed by "Outside the Lines."

"These are very important issues that are too complicated to address in an edited 10-second sound-bite," says Susan Manko, spokesperson for the University of Pittsburgh Medical Center.

In a statement to ESPN.com, NFL spokesman Greg Aiello said: "The commissioner is not expecting the committee to specifically recommend a single neuropsych[ological] testing protocol. Most of the clubs have already decided to use the ImPACT test on their own, or were already using it. There are many members of the committee who have had no role in developing ImPACT and no financial interest in it. And ImPACT has been the subject of a good deal of independent study."

Scientists are currently debating how useful ImPACT and other computerized systems are in diagnosing concussions. Citing multiple studies, Lovell and his colleagues have stated that "ImPACT has been shown to be an effective tool for concussion management." They have asserted repeatedly that ImPACT measures real effects, not just the ability of subjects to improve on tests with practice, and that it can discern even mild concussions.

Outside the Lines

On Sunday, "Outside the Lines" will examine the journeys of two men who have become unlikely partners in the common belief that brain damage from repeated concussions in football can lead to depression, dementia and suicide. One is a former pro athlete whose career was ended by concussions; the other is a doctor who worked on the autopsies of two former NFL players. In addition, "Outside the Lines" takes a close look at Mark Lovell's ImPACT test and its use by the NFL, and explores whether it is good science to have a member of the league's concussions committee analyzing data and helping set policy using a product in which he has a financial interest. Guests include Garrett Webster, the son of late former NFL center Mike Webster; and ESPN The Magazine's Peter Keating. "Outside the Lines" is hosted by Bob Ley. Tune in Sunday at 9:30 a.m, ET, on ESPN.

But almost all of the research supporting ImPACT has been written or co-written by its inventors. For example, Lovell and Collins are co-authors of all 19 of the publications listed in the "Reliability and Validity" section on the ImPACT Web site. "I think ImPACT is a good system, and we use it at West Virginia," says Julian Bailes, chairman of the neurosurgery department at West Virginia University and medical director for the Center for the Study of Retired Athletes. "Most of the studies on ImPACT, however, have been by the people who developed it. Some of that is inevitable, because we are still relatively early in the process of trying to validate it."

In 2005, the Journal of Athletic Training published a study co-written by Loyola's Randolph that surveyed the preceding 15 years of medical literature on neuropsychological testing. It found: "Only one peer-reviewed article involving a prospective controlled study with ImPACT has been published."

More recently, independent research has cast doubt on the overall value of computerized testing. In a study that will be published later this year in the Journal of Athletic Training, researchers gave computerized tests to a group of uninjured college students, then tested them again 45 days later. ImPACT incorrectly identified the subjects as having some aspect of a concussion in 38.4 percent of cases. "We tested three computerized systems and found all of them to be less than optimal," says Steven Broglio, lead author of the study and professor of kinesiology at the University of Illinois at Urbana-Champaign. Broglio declined to elaborate on the data because it has yet to be published, and did not supply it to ESPN.com. But he confirmed that it was presented to NFL team doctors and athletic trainers at the concussion summit the league held on June 19 in Chicago.

"Neurocognitive testing is only one tool for assessing injuries," Broglio says. "Athletes can also be evaluated according to their symptoms and their postural control, meaning how well they maintain their balance. Our published research has found that when you look at all three, computerized tests have about the same sensitivity to concussions as paper-and-pencil tests."

Traditional paper-and-pencil tests gauge the subject's memory by methods such as asking for recall of word lists, and the subject's processing speed by measures such as using a key to coordinate symbols with numbers in a series of fill-in boxes on a page.

The computerized systems such as ImPACT basically adapt the same sorts of testing procedures to a machine.

"I have a lot of concern with ImPACT, as I still have not seen a good study of basic psychometric properties that suggest ImPACT is as good or better than paper-and-pencil tests," says one veteran NFL team neuropsychologist, who did not want to be identified as publicly criticizing Lovell or the NFL. "Computer tests look cool on the screen and seem very sophisticated, but they are really just fancy stopwatches." Among neuropsychologists' worries: differences in computer architecture and hardware (such as whether the software is running on a desktop or a laptop), the possibility that other applications are open and how a subject uses his computer's mouse all could affect the accuracy of ImPACT's timing measurements.

Nevertheless, Lovell's company, ImPACT Applications, is aggressively and successfully marketing its software. ImPACT is a private company and does not disclose its annual revenue or profits. But the firm, which is based in Pittsburgh, has sales representatives as far away as Australia and South Africa. It sells desktop and online versions of ImPACT to organizations and schools in packages ranging in price from $500 to $1000 per year, plus extra charges for additional tests. It also holds training workshops, charging doctors and athletic trainers from $100 to $200 to attend daylong seminars in Pittsburgh. Sessions include "Development of ImPACT," "On-Field Management of Concussion" and "Question and Answer Forum on the ImPACT Test, Marketing Your Services and Using the Media to Promote Your Sports Concussion Practice," according to a sample agenda on the company's Web site. An Aug. 3 workshop was filled, with 24 participants; ImPACT is now accepting registrations for another seminar series on Sept. 21.

But readers of Lovell's academic work will find scarce mention of these business activities, even in research he has conducted on ImPACT itself.

In the March 2006 edition of Brain Injury, for example, Lovell and three co-authors described ImPACT as a "brief computer-administered neuropsychological test battery" without detailing its origins. The acknowledgments to that study contained the equivalent of an advertisement for the software: "Additional information on ImPACT is available at www.impacttest.com ."

After receiving a complaint about that paper and another co-authored by Lovell last year, the editorial board of Brain Injury decided to require researchers to state more fully any possible conflicts of interest. "We share your concern … Your letter has inspired us to develop a fair and appropriate policy which will be communicated to our authors," Jeffrey Kreutzer, editor of Brain Injury, wrote on April 4, 2006, to a reader who had expressed concerns about Lovell and ImPACT.

"Some of the people publishing on neuropsychological testing had a financial interest in it," Kreutzer, who is also a professor of physical medicine and rehabilitation, neurosurgery and psychiatry at the Virginia Commonwealth University Medical Center, says today. "So we spiffed up our disclosure policy."

In the January 2006 issue of the British Journal of Sports Medicine, a study by Lovell and three other researchers found that athletes with one or two prior concussions did not differ significantly from athletes with no prior concussions in their performance on ImPACT tests. The authors rejected the idea that the results could have stemmed from any flaws in ImPACT: "The failure to detect possible persisting problems from one or two previous concussions is probably not due to inadequate sensitivity of the computerised screening measure."

The conclusion of that paper listed "competing interests" of the authors. It read: "None declared."

"I was responsible for the Web-based electronic submission of this article," says Grant Iverson, professor of psychiatry at the University of British Columbia and a co-author of the paper. Iverson contacted ESPN.com at Lovell's request. "If mistakes were made therein, they were my fault," he says.

Potential conflicts of interest have cropped up even in research where Lovell has revealed his involvement with ImPACT. In the February 2006 issue of Neurosurgery, Lovell and three members of the MTBI Committee published a study that used ImPACT test scores to look at how NFL and high school athletes recovered from concussions. The acknowledgments to that paper stated: "Dr. Lovell has a financial interest in the ImPACT computer based neuropsychological test battery used by many NFL teams."

But in the peer reviews immediately following the paper, the first two commenters were Collins and Maroon, Lovell's fellow corporate officers at ImPACT Applications. They lauded ImPACT, writing: "We think that computerized neuropsychological testing as outlined in this study will become the standard of care to assist physicians with their clinical judgment in assessing and managing athletes with mild traumatic brain injury."

Their roles in developing and marketing ImPACT were not disclosed.

Peter Keating writes about sports business for ESPN The Magazine.

• Recommend0
• Tweet0
• Comments0
• Email
• Print
• Subscribe

Peter Keating

• Archive

Peter Keating is a senior writer at ESPN The Magazine, where he covers investigative and statistical subjects. He started writing "The Biz," a column looking at sports business from the fan's point of view, in 1999. He also coordinates the Magazine's annual "Ultimate Standings" project, which ranks all pro franchises according to how much they give back to fans. His work on concussions in football has earned awards from the Deadline Club, the New York Press Club and the Center for the Study of Sport in Society.