Posted by Jef Akst, "The Scientist.com"
26th January 2010
Last month, the California Institute for Regenerative Medicine (CIRM) pulled 3 of the 76 SEED grants it had given in one of its very first rounds of funding due to inadequate progress -- a 4% revocation rate. On the flip side, the National Institutes of Health (NIH), which funds nearly 10,000 research grants each year, says it hardly ever pulls basic research grants. What's the right balance of oversight between giving out free money and thwarting scientific creativity?
On the one hand, science can be unpredictable -- adjusting to the curveballs an experiment may throw at an investigator is "how most discoveries are made, [and] we wouldn't want to squelch that creativity by [excessively] overseeing grants," said Stuart Lipton of the Burnham Institute for Medical Research. "On the other hand, [the NIH] wants to make sure you do the work that's in the general purvey to what you proposed."
The NIH monitors grant recipients via a required annual progress report for each grant they award, called a "non-competing continuation progress report." In order to receive the next year's money, investigators must have them approved by their program directors, who each oversee anywhere from 50 to 300 grants at a given time.
Robert Finkelstein, associate director for extramural research at National Institute of Neurological Disorders and Stroke (NINDS), could not think of any research project grants (e.g., R01s) that had been terminated by his institute in the last several years. "It's really pretty incredibly rare that we do this," Finkelstein said.
Should the NIH have a stronger hand in the projects it funds? This question, not surprisingly, evoked mixed responses from the research community. The NIH's lax oversight does appear to open the door to researchers to use the funds for something other than what they proposed in their grant application, said Andrea Elberger of the University of Tennessee Health Science Center. "The majority of us do what we're supposed to do and do it by the rules, but there are people who game the system and for those people it is easy to get away with it."
Elberger recalled an incident she knew of in which, by the time the award was granted, the investigator had already determined that the proposed experiment wasn't going to work and decided to do something completely unrelated. To avoid questions, the investigator simply resubmitted preliminary data, "saying the experiment didn't work," Elberger said. "It was quite a successful ploy," she added. "They never got caught and they've become a very successful person in what they're doing. If oversight didn't catch that, it won't catch anything."
Other stories reveal a less extreme, but perhaps more common, game -- using a grant for one thing to generate preliminary data for something else, in order to obtain another round of funding. Because a successful NIH grant application requires preliminary data, "the money that you're getting for [a particular] grant, you're using to generate preliminary data of the next grant," said David Adams of Duke University School of Medicine. "The people that have gotten into this continuity [and] have this system going [are] always publishing papers."
This may be common, but would the NIH call it kosher? "While there is flexibility within the grant mechanism to explore new avenues of research within the general aims of the funded grant," project director Neil Buckholtz of the National Institute on Aging wrote in an email to The Scientist, "the investigators should not go into totally different areas of research using funds from the current grant."
Such stories, though, demonstrate a flexibility of the NIH system that many scientists still feel is important to productive science. "Deep down I believe that scientists should be allowed to do whatever they want [within reason]," Nejat D?zg?nes of the University of the Pacific said. "We're at a certain stage in science [that] we really cannot tell where the science can go and what experiments the investigator needs to be doing 2 years or 4 years or 5 years from now. This is the way science is. Scrutinizing grantees would not be very productive."
That's fine, some researchers say, but it's inconsistent with how the grants are awarded in the first place. "The actual review process absolutely doesn't work that way," Elberger said. To the contrary, "experienced scientists are asked to provide ridiculous amounts of detail" in their grant applications, regarding such things as methodology that are likely to change between the time the application is written and the time it is awarded. Thus, while the annual progress reports appear to be more of a bookkeeping policy -- only occasionally requiring follow-up and hardly ever resulting in grant termination -- the process of applying for the grant in the first place requires painstaking time and effort. "Why make us write all this information out and then let people get away with whatever they want?" said Elberger. "Either trust us or don't."
"If you are going to spend so much [time] scrutinizing [the initial application] -- [and] making life so miserable for every scientist -- then you might as well scrutinize the grant afterwards," D?zg?nes agreed. "[Otherwise] it doesn't make sense to be doing the first scrutiny."
Even if there was a consensus that the NIH should tighten up their oversight, "I think it would be very tough to do," said David Kaplan of Case Western Reserve University in Ohio. "The man power [it would require] would be extraordinary. We're talking about hours of analysis of what [the researchers] have done, what the literature is, what their results are, [and] what's reasonable. It must be an incredible amount of time." With nearly 10,000 research grants per year, the shear volume of grantees the NIH has at any given time prohibits any substantial level of oversight, Kaplan said. "I don't think there's any easy answer for them to do it differently."
26th January 2010
Last month, the California Institute for Regenerative Medicine (CIRM) pulled 3 of the 76 SEED grants it had given in one of its very first rounds of funding due to inadequate progress -- a 4% revocation rate. On the flip side, the National Institutes of Health (NIH), which funds nearly 10,000 research grants each year, says it hardly ever pulls basic research grants. What's the right balance of oversight between giving out free money and thwarting scientific creativity?
On the one hand, science can be unpredictable -- adjusting to the curveballs an experiment may throw at an investigator is "how most discoveries are made, [and] we wouldn't want to squelch that creativity by [excessively] overseeing grants," said Stuart Lipton of the Burnham Institute for Medical Research. "On the other hand, [the NIH] wants to make sure you do the work that's in the general purvey to what you proposed."
The NIH monitors grant recipients via a required annual progress report for each grant they award, called a "non-competing continuation progress report." In order to receive the next year's money, investigators must have them approved by their program directors, who each oversee anywhere from 50 to 300 grants at a given time.
Robert Finkelstein, associate director for extramural research at National Institute of Neurological Disorders and Stroke (NINDS), could not think of any research project grants (e.g., R01s) that had been terminated by his institute in the last several years. "It's really pretty incredibly rare that we do this," Finkelstein said.
Should the NIH have a stronger hand in the projects it funds? This question, not surprisingly, evoked mixed responses from the research community. The NIH's lax oversight does appear to open the door to researchers to use the funds for something other than what they proposed in their grant application, said Andrea Elberger of the University of Tennessee Health Science Center. "The majority of us do what we're supposed to do and do it by the rules, but there are people who game the system and for those people it is easy to get away with it."
Elberger recalled an incident she knew of in which, by the time the award was granted, the investigator had already determined that the proposed experiment wasn't going to work and decided to do something completely unrelated. To avoid questions, the investigator simply resubmitted preliminary data, "saying the experiment didn't work," Elberger said. "It was quite a successful ploy," she added. "They never got caught and they've become a very successful person in what they're doing. If oversight didn't catch that, it won't catch anything."
Other stories reveal a less extreme, but perhaps more common, game -- using a grant for one thing to generate preliminary data for something else, in order to obtain another round of funding. Because a successful NIH grant application requires preliminary data, "the money that you're getting for [a particular] grant, you're using to generate preliminary data of the next grant," said David Adams of Duke University School of Medicine. "The people that have gotten into this continuity [and] have this system going [are] always publishing papers."
This may be common, but would the NIH call it kosher? "While there is flexibility within the grant mechanism to explore new avenues of research within the general aims of the funded grant," project director Neil Buckholtz of the National Institute on Aging wrote in an email to The Scientist, "the investigators should not go into totally different areas of research using funds from the current grant."
Such stories, though, demonstrate a flexibility of the NIH system that many scientists still feel is important to productive science. "Deep down I believe that scientists should be allowed to do whatever they want [within reason]," Nejat D?zg?nes of the University of the Pacific said. "We're at a certain stage in science [that] we really cannot tell where the science can go and what experiments the investigator needs to be doing 2 years or 4 years or 5 years from now. This is the way science is. Scrutinizing grantees would not be very productive."
That's fine, some researchers say, but it's inconsistent with how the grants are awarded in the first place. "The actual review process absolutely doesn't work that way," Elberger said. To the contrary, "experienced scientists are asked to provide ridiculous amounts of detail" in their grant applications, regarding such things as methodology that are likely to change between the time the application is written and the time it is awarded. Thus, while the annual progress reports appear to be more of a bookkeeping policy -- only occasionally requiring follow-up and hardly ever resulting in grant termination -- the process of applying for the grant in the first place requires painstaking time and effort. "Why make us write all this information out and then let people get away with whatever they want?" said Elberger. "Either trust us or don't."
"If you are going to spend so much [time] scrutinizing [the initial application] -- [and] making life so miserable for every scientist -- then you might as well scrutinize the grant afterwards," D?zg?nes agreed. "[Otherwise] it doesn't make sense to be doing the first scrutiny."
Even if there was a consensus that the NIH should tighten up their oversight, "I think it would be very tough to do," said David Kaplan of Case Western Reserve University in Ohio. "The man power [it would require] would be extraordinary. We're talking about hours of analysis of what [the researchers] have done, what the literature is, what their results are, [and] what's reasonable. It must be an incredible amount of time." With nearly 10,000 research grants per year, the shear volume of grantees the NIH has at any given time prohibits any substantial level of oversight, Kaplan said. "I don't think there's any easy answer for them to do it differently."