peer review adds social relevancy
TRANSCRIPT
n e w s of the w e e k
PEER REVIEW ADDS SOCIAL RELEVANCY National Science Board okays new system for NSF
R eform of the peer review process at the National Science Foundation, under study since early 1995, is
now complete. The changes, which some call profound and others believe deserve no more than a yawn, take effect Oct. 1.
On March 28, the National Science Board (NSB)—NSF's governing body—approved changes in the criteria grantees are required to meet to obtain an NSF grant. They are the first modifications to the peer (more precisely called merit) review process at NSF since 1981, and probably will be confusing for a while.
The reforms—which critics say amount to no more than NSF's surrender to political correctness—are designed to simplify the pro- | cess both for researchers ap- § plying for grants and for | reviewers and program of- " fleers assessing proposals. o
NSF Director Neal F. I Lane, ever careful to assuage tender feelings, says the changes represent only slight modifications to the existing system. "There's nothing broke here that needs to be fixed," he says. "But NSF's program portfolio has changed since 1981. These criteria are easier to apply and better reflect the spirit and philosophy of NSF's strategic plan" than the old ones.
What they really seem to do is incorporate social and economic relevancy into the criteria used to judge a researcher's proposed work, issues dear to the government that's providing the research support.
After considerable consultation with the research community, the changes now being implemented were recommended by a special NSB-appointed task force headed by NSB member Warren M. Washington, chief of the climate change research program at the National Center for Atmospheric Research, Boulder, Colo.
As NSB chairman Richard N. Zare, a
chemistry professor at Stanford University, explains, "A lot of effort was put into selling the community on what we are trying to do. The community has to understand that when we take federal support, we have to explain what we are doing with it. We need to show responsible stewardship of these precious funds."
The reforms reduce from four to two the criteria that establish the acceptability of a grant application. And probably more important, they simplify the language that tells grantees what should be included in a proposal.
Under the old set of four, potential
Zare (left): responsible stewardship; Lane: reflecting the spirit of NSF's strategic plan
grantees had to justify to NSF reviewers their competence as researchers, the importance of their research, its utility or relevance, and its effect on the "infrastructure" of science and engineering. The two new ones are: What is the intellectual merit of the proposed activity; and what are the broader impacts of the proposed activity?
This compaction of four criteria into two is significant. Four, in NSB's thinking, tended by their number to dilute the importance of the wider implications of the research, which was almost always ignored by grantees and reviewers alike.
Intellectual merit and broader impact are not quite equivalent in weight, how
ever. That judgment will be left to program officers and reviewers because the social worth of many areas of research is, at best, cloudy. Lane and NSB say nothing can possibly get funded without demonstration of competence. Still, of equivalent proposals, the one that demonstrates that "broader impact" will be deemed more meritorious.
Questions a grantee must answer to fulfill the broader impact requirement include: How will the research promote teaching, training, and learning? How would the proposal broaden the participation of underrepresented groups? How will it enhance facilities, instrumentation, networks, and partnerships? How will the work be disseminated? And finally, how would society benefit from the work?
A fact sheet accompanying the release of the new criteria seeks to assure the research community that there really is nothing to worry about. NSF staff and review boards, it indicates, will be kind and wise stewards of the process. Lane tells C&EN that the NSF family of program officers and peer reviewers will be undergoing training to see that the process remains honest and true to the critical importance of quality.
The research community was given three months to comment on the reform. A section of NSF's World Wide Web page was reserved for comments on the new criteria, and 310 replies came in by e-mail, and another 15 in letters. Of those, 56% indicated that NSF was moving in the right direction, 44% said it wasn't.
"Thank you for this breath of fresh air," went one comment; "a step toward the destruction of nonapplied science in the U.S.," went another. As for chemists, Janet Osteryoung, who heads NSF's chemistry division says: "I haven't seen that the changes are causing any angst. They aren't really qualitatively different from those they replaced."
Right or wrong, the changes will, however, have a psychological impact on the research community, and this shouldn't be minimized. NSF is going through revolutionary times. It is facing ever increasing pressure to justify its programs to Congress in terms of social utility.
In 1995, Congress passed the Government Performance & Results Act that required every agency to submit by 1999 evidence of how their programs were benefiting the people—taxpayers—who were funding them. NSF's review changes fit squarely in that requirement.
Wil Lepkowski
APRIL 7, 1997 C&EN 9