peer review adds social relevancy

1
news of the week PEER REVIEW ADDS SOCIAL RELEVANCY National Science Board okays new system for NSF R eform of the peer review process at the National Science Foundation, under study since early 1995, is now complete. The changes, which some call profound and others believe deserve no more than a yawn, take effect Oct. 1. On March 28, the National Science Board (NSB)—NSF's governing body—ap- proved changes in the criteria grantees are required to meet to obtain an NSF grant. They are the first modifications to the peer (more precisely called merit) review pro- cess at NSF since 1981, and probably will be confusing for a while. The reforms—which critics say amount to no more than NSF's surrender to politi- cal correctness—are de- signed to simplify the pro- | cess both for researchers ap- § plying for grants and for | reviewers and program of- " fleers assessing proposals. o NSF Director Neal F. I Lane, ever careful to as- suage tender feelings, says the changes represent only slight modifications to the existing system. "There's nothing broke here that needs to be fixed," he says. "But NSF's program portfo- lio has changed since 1981. These criteria are easier to apply and better reflect the spirit and philosophy of NSF's strategic plan" than the old ones. What they really seem to do is incor- porate social and economic relevancy into the criteria used to judge a research- er's proposed work, issues dear to the government that's providing the re- search support. After considerable consultation with the research community, the changes now being implemented were recommended by a special NSB-appointed task force headed by NSB member Warren M. Wash- ington, chief of the climate change re- search program at the National Center for Atmospheric Research, Boulder, Colo. As NSB chairman Richard N. Zare, a chemistry professor at Stanford Universi- ty, explains, "A lot of effort was put into selling the community on what we are trying to do. The community has to un- derstand that when we take federal sup- port, we have to explain what we are doing with it. We need to show respon- sible stewardship of these precious funds." The reforms reduce from four to two the criteria that establish the acceptabili- ty of a grant application. And probably more important, they simplify the lan- guage that tells grantees what should be included in a proposal. Under the old set of four, potential Zare (left): responsible stewardship; Lane: reflecting the spirit of NSF's strategic plan grantees had to justify to NSF reviewers their competence as researchers, the im- portance of their research, its utility or relevance, and its effect on the "infra- structure" of science and engineering. The two new ones are: What is the intel- lectual merit of the proposed activity; and what are the broader impacts of the proposed activity? This compaction of four criteria into two is significant. Four, in NSB's think- ing, tended by their number to dilute the importance of the wider implications of the research, which was almost always ignored by grantees and reviewers alike. Intellectual merit and broader impact are not quite equivalent in weight, how- ever. That judgment will be left to pro- gram officers and reviewers because the social worth of many areas of research is, at best, cloudy. Lane and NSB say noth- ing can possibly get funded without demonstration of competence. Still, of equivalent proposals, the one that dem- onstrates that "broader impact" will be deemed more meritorious. Questions a grantee must answer to fulfill the broader impact requirement in- clude: How will the research promote teaching, training, and learning? How would the proposal broaden the partici- pation of underrepresented groups? How will it enhance facilities, instrumentation, networks, and partnerships? How will the work be disseminated? And finally, how would society benefit from the work? A fact sheet accompanying the release of the new criteria seeks to assure the re- search community that there really is nothing to worry about. NSF staff and re- view boards, it indicates, will be kind and wise stewards of the process. Lane tells C&EN that the NSF family of pro- gram officers and peer reviewers will be undergoing training to see that the pro- cess remains honest and true to the criti- cal importance of quality. The research community was given three months to comment on the re- form. A section of NSF's World Wide Web page was reserved for comments on the new criteria, and 310 replies came in by e- mail, and another 15 in letters. Of those, 56% indicated that NSF was moving in the right direction, 44% said it wasn't. "Thank you for this breath of fresh air," went one comment; "a step toward the destruction of nonapplied science in the U.S.," went another. As for chemists, Janet Osteryoung, who heads NSF's chemistry division says: "I haven't seen that the changes are causing any angst. They aren't really qualitatively different from those they replaced." Right or wrong, the changes will, how- ever, have a psychological impact on the research community, and this shouldn't be minimized. NSF is going through revolu- tionary times. It is facing ever increasing pressure to justify its programs to Con- gress in terms of social utility. In 1995, Congress passed the Govern- ment Performance & Results Act that re- quired every agency to submit by 1999 evidence of how their programs were benefiting the people—taxpayers—who were funding them. NSF's review chang- es fit squarely in that requirement. Wil Lepkowski APRIL 7, 1997 C&EN 9

Upload: buithuan

Post on 18-Feb-2017

212 views

Category:

Documents


0 download

TRANSCRIPT

n e w s of the w e e k

PEER REVIEW ADDS SOCIAL RELEVANCY National Science Board okays new system for NSF

R eform of the peer review process at the National Science Foundation, under study since early 1995, is

now complete. The changes, which some call profound and others believe deserve no more than a yawn, take effect Oct. 1.

On March 28, the National Science Board (NSB)—NSF's governing body—ap­proved changes in the criteria grantees are required to meet to obtain an NSF grant. They are the first modifications to the peer (more precisely called merit) review pro­cess at NSF since 1981, and probably will be confusing for a while.

The reforms—which critics say amount to no more than NSF's surrender to politi­cal correctness—are de­signed to simplify the pro- | cess both for researchers ap- § plying for grants and for | reviewers and program of- " fleers assessing proposals. o

NSF Director Neal F. I Lane, ever careful to as­suage tender feelings, says the changes represent only slight modifications to the existing system. "There's nothing broke here that needs to be fixed," he says. "But NSF's program portfo­lio has changed since 1981. These criteria are easier to apply and better reflect the spirit and philosophy of NSF's strategic plan" than the old ones.

What they really seem to do is incor­porate social and economic relevancy into the criteria used to judge a research­er's proposed work, issues dear to the government that's providing the re­search support.

After considerable consultation with the research community, the changes now being implemented were recommended by a special NSB-appointed task force headed by NSB member Warren M. Wash­ington, chief of the climate change re­search program at the National Center for Atmospheric Research, Boulder, Colo.

As NSB chairman Richard N. Zare, a

chemistry professor at Stanford Universi­ty, explains, "A lot of effort was put into selling the community on what we are trying to do. The community has to un­derstand that when we take federal sup­port, we have to explain what we are doing with it. We need to show respon­sible stewardship of these precious funds."

The reforms reduce from four to two the criteria that establish the acceptabili­ty of a grant application. And probably more important, they simplify the lan­guage that tells grantees what should be included in a proposal.

Under the old set of four, potential

Zare (left): responsible stewardship; Lane: reflecting the spirit of NSF's strategic plan

grantees had to justify to NSF reviewers their competence as researchers, the im­portance of their research, its utility or relevance, and its effect on the "infra­structure" of science and engineering. The two new ones are: What is the intel­lectual merit of the proposed activity; and what are the broader impacts of the proposed activity?

This compaction of four criteria into two is significant. Four, in NSB's think­ing, tended by their number to dilute the importance of the wider implications of the research, which was almost always ignored by grantees and reviewers alike.

Intellectual merit and broader impact are not quite equivalent in weight, how­

ever. That judgment will be left to pro­gram officers and reviewers because the social worth of many areas of research is, at best, cloudy. Lane and NSB say noth­ing can possibly get funded without demonstration of competence. Still, of equivalent proposals, the one that dem­onstrates that "broader impact" will be deemed more meritorious.

Questions a grantee must answer to fulfill the broader impact requirement in­clude: How will the research promote teaching, training, and learning? How would the proposal broaden the partici­pation of underrepresented groups? How will it enhance facilities, instrumentation, networks, and partnerships? How will the work be disseminated? And finally, how would society benefit from the work?

A fact sheet accompanying the release of the new criteria seeks to assure the re­search community that there really is nothing to worry about. NSF staff and re­view boards, it indicates, will be kind and wise stewards of the process. Lane tells C&EN that the NSF family of pro­gram officers and peer reviewers will be undergoing training to see that the pro­cess remains honest and true to the criti­cal importance of quality.

The research community was given three months to comment on the re­form. A section of NSF's World Wide Web page was reserved for comments on the new criteria, and 310 replies came in by e-mail, and another 15 in letters. Of those, 56% indicated that NSF was moving in the right direction, 44% said it wasn't.

"Thank you for this breath of fresh air," went one comment; "a step toward the destruction of nonapplied science in the U.S.," went another. As for chemists, Janet Osteryoung, who heads NSF's chemistry division says: "I haven't seen that the changes are causing any angst. They aren't really qualitatively different from those they replaced."

Right or wrong, the changes will, how­ever, have a psychological impact on the research community, and this shouldn't be minimized. NSF is going through revolu­tionary times. It is facing ever increasing pressure to justify its programs to Con­gress in terms of social utility.

In 1995, Congress passed the Govern­ment Performance & Results Act that re­quired every agency to submit by 1999 evidence of how their programs were benefiting the people—taxpayers—who were funding them. NSF's review chang­es fit squarely in that requirement.

Wil Lepkowski

APRIL 7, 1997 C&EN 9