external review of the cooperative institute for mesoscale meteorological studies (cimms) 15-16...

EXTERNAL REVIEWEXTERNAL REVIEWOF THEOF THE

COOPERATIVE INSTITUTE FOR MESOSCALE COOPERATIVE INSTITUTE FOR MESOSCALE METEOROLOGICAL STUDIES (CIMMS)METEOROLOGICAL STUDIES (CIMMS)

15-16 OCTOBER 200315-16 OCTOBER 2003UNIVERSITY OF OKLAHOMAUNIVERSITY OF OKLAHOMA

NORMAN, OKLAHOMANORMAN, OKLAHOMA

Presented toPresented to

NOAA Science Advisory BoardNOAA Science Advisory Board

Washington, DCWashington, DC

17 March 200417 March 2004

External Review TeamExternal Review Team

Denise Stephenson Hawk, Ph.D., Panel ChairDenise Stephenson Hawk, Ph.D., Panel ChairChairman, The Stephenson Group, LLC; NOAA SABChairman, The Stephenson Group, LLC; NOAA SAB

Gregory S. Forbes, Ph.D.Gregory S. Forbes, Ph.D.Severe Weather Expert, The Weather ChannelSevere Weather Expert, The Weather Channel

Eve Gruntfest, Ph.D.Eve Gruntfest, Ph.D.Geography and Environmental StudiesGeography and Environmental StudiesUniversity of ColoradoUniversity of Colorado

Jack Hayes, Ph.D.Jack Hayes, Ph.D.Director, Office of Science and Technology, NOAA NWSDirector, Office of Science and Technology, NOAA NWS

Robert A. Weller, Ph.D.Robert A. Weller, Ph.D. Director, Director, Cooperative Institute for Climate and Ocean ResearchCooperative Institute for Climate and Ocean Research Woods Hole Oceanographic InstitutionWoods Hole Oceanographic Institution

Derek Winstanley, Ph.D.Derek Winstanley, Ph.D.Chief, Illinois State Water SurveyChief, Illinois State Water Survey

OVERVIEWOVERVIEW

Guidelines for External ReviewGuidelines for External Review CIMMS - Background, Purpose and ScopeCIMMS - Background, Purpose and Scope Review and RecommendationsReview and Recommendations

– Science PlanScience Plan– Scientific PartnershipsScientific Partnerships– Science ReviewScience Review– Education and OutreachEducation and Outreach– Science Management PlanScience Management Plan

Summary and ConclusionsSummary and Conclusions

Guidelines for External ReviewsGuidelines for External Reviews

NOAA Office of Oceanic and Atmospheric Research NOAA Office of Oceanic and Atmospheric Research (OAR) provided review format and guidelines to (OAR) provided review format and guidelines to CIMMS and Review TeamCIMMS and Review Team

– Science PlanScience Plan– Science ReviewScience Review– Education/ OutreachEducation/ Outreach– Science Management PlanScience Management Plan

Review conducted under auspices of NOAA SABReview conducted under auspices of NOAA SAB

CIMMS…CIMMS… Cooperative program between NOAA and University of Cooperative program between NOAA and University of

Oklahoma (OU)Oklahoma (OU)

Established in 1978 to…Established in 1978 to…– Act as research interface between OU, OAR, NWS, NOAAAct as research interface between OU, OAR, NWS, NOAA– Assist in transitioning products of research into operational Assist in transitioning products of research into operational

procedures /techniquesprocedures /techniques

Governed by Memorandum of Agreement (MOA) - Last dated Governed by Memorandum of Agreement (MOA) - Last dated June 1995 with addenda December 2000 and August 2002June 1995 with addenda December 2000 and August 2002

Budget for FY 2003: $10,146,626 [60% NOAA; 16% OU; 24% Budget for FY 2003: $10,146,626 [60% NOAA; 16% OU; 24% other – NASA, NSF, DOE, ONR, WMO, U.S. EPA…]other – NASA, NSF, DOE, ONR, WMO, U.S. EPA…]

Director: Peter Lamb, Ph.D.Director: Peter Lamb, Ph.D.George Lynn Cross Research Professor of MeteorologyGeorge Lynn Cross Research Professor of Meteorology

CIMMS Research ThemesCIMMS Research Themes

Basic convective and mesoscale research (1982)Basic convective and mesoscale research (1982) Forecast Improvements (1990)Forecast Improvements (1990) Climatic Effects of and controls on mesoscale Climatic Effects of and controls on mesoscale

processes (1990)processes (1990) Socio-economic impacts of mesoscale weather Socio-economic impacts of mesoscale weather

systems and of regional scale variations (200?)systems and of regional scale variations (200?) Doppler weather radar research and development Doppler weather radar research and development

(1995)(1995) Climate change monitoring and detection (2001)Climate change monitoring and detection (2001)

CIMMS Cooperating PartnersCIMMS Cooperating Partners

National Oceanic and Atmospheric AdministrationNational Oceanic and Atmospheric Administration Office of Oceanic and Atmospheric ResearchOffice of Oceanic and Atmospheric Research National Weather Service (NWS)National Weather Service (NWS) University of OklahomaUniversity of Oklahoma Operational Support Facility for WSR-88D RadarOperational Support Facility for WSR-88D Radar National Environmental Satellite, Data, and Information National Environmental Satellite, Data, and Information

Service (NESDIS)Service (NESDIS) Southern Region Headquarters of NWSSouthern Region Headquarters of NWS

HIGHLIGHTS OF CIMMSHIGHLIGHTS OF CIMMS

Significant role in modernization of NWS, particularly Significant role in modernization of NWS, particularly through upgrading of radar hardware and software through upgrading of radar hardware and software systemssystems

Recent advances in accuracy and lead time of tornado Recent advances in accuracy and lead time of tornado warningswarnings

Extensive partnerships with the Norman, OK scientific Extensive partnerships with the Norman, OK scientific communitycommunity

Department of Commerce Silver Medal in 1999 for Department of Commerce Silver Medal in 1999 for Warning Decision Support System (WDSS)Warning Decision Support System (WDSS)

Department of Commerce Gold Medal in 2003 for Department of Commerce Gold Medal in 2003 for Weather Event Simulator (WES)Weather Event Simulator (WES)

Review and RecommendationsReview and Recommendations

Science PlanScience Plan

Science Plan Science Plan defineddefined – –

Clearly articulated set of strategies for planning and Clearly articulated set of strategies for planning and managing the science portfolio within the managing the science portfolio within the framework of the MOAframework of the MOA

Recommendation 1Recommendation 1: To effectively and efficiently : To effectively and efficiently serve the needs of NOAA, as defined within the serve the needs of NOAA, as defined within the NOAA and line office strategic plans, CIMMS must NOAA and line office strategic plans, CIMMS must define a well-articulated strategic science plan that define a well-articulated strategic science plan that aligns with its established MOA.aligns with its established MOA.



Recommendation 2Recommendation 2: CIMMS management should : CIMMS management should engage the expertise of the NOAA Strategic engage the expertise of the NOAA Strategic Planning Office to provide training regarding the Planning Office to provide training regarding the tenets and development of strategic science plans.tenets and development of strategic science plans.



What is the scientific (not programmatic) vision for What is the scientific (not programmatic) vision for the Institute?the Institute?

Recommendation 3Recommendation 3: The leadership of CIMMS must : The leadership of CIMMS must engage the OU and NOAA partners in the engage the OU and NOAA partners in the development of a vision for the Institute. The vision development of a vision for the Institute. The vision should establish the rationale behind the six should establish the rationale behind the six themes agreed upon by NOAA and OU.themes agreed upon by NOAA and OU.



How is the scientific vision related to the NOAA How is the scientific vision related to the NOAA Strategic Plan?Strategic Plan?

Recommendation 4Recommendation 4: The leadership of CIMMS must : The leadership of CIMMS must work with NOAA to develop a scientific vision that work with NOAA to develop a scientific vision that aligns with the NOAA Strategic Plan.aligns with the NOAA Strategic Plan.



What are the goals and objectives [CIMMS]?What are the goals and objectives [CIMMS]?

Recommendation 5Recommendation 5: OU and CIMMS must establish a : OU and CIMMS must establish a process to coordinate specific thematic research process to coordinate specific thematic research goals and objectives with OAR as part of the NOAA goals and objectives with OAR as part of the NOAA strategic planning and programming process and strategic planning and programming process and to use these higher level specific goals and to use these higher level specific goals and objectives to challenge individual scientists.objectives to challenge individual scientists.



What criteria are used to measure progress in What criteria are used to measure progress in accomplishing these goals and objectives?accomplishing these goals and objectives?

Recommendation 6Recommendation 6: OU/ CIMMS must define more : OU/ CIMMS must define more specific indicators of progress in each theme area specific indicators of progress in each theme area and for each project for purpose of improving the and for each project for purpose of improving the linkage between OU/ CIMMS research and the linkage between OU/ CIMMS research and the NOAA strategic goals and objectives.NOAA strategic goals and objectives.



What are the major scientific themes?What are the major scientific themes? How were they identified?How were they identified? Which themes/sub-themes are near completion?Which themes/sub-themes are near completion? What are the emerging thematic areas? Why?What are the emerging thematic areas? Why?

Recommendation 7Recommendation 7: OU/ CIMMS should evaluate its : OU/ CIMMS should evaluate its current research focus relative to the mission, current research focus relative to the mission, goals and objectives of NOAA and develop a goals and objectives of NOAA and develop a strategic plan to better focus its research in areas strategic plan to better focus its research in areas that will result in enhanced contributions to the that will result in enhanced contributions to the science of NOAA.science of NOAA.


Scientific PartnershipsScientific Partnerships

What is the relationship to OAR Laboratories and What is the relationship to OAR Laboratories and other NOAA entities?other NOAA entities?

Thirteen PartnershipsThirteen Partnerships OAR National Severe Storms LaboratoryOAR National Severe Storms Laboratory NWS Warning Decision Training BranchNWS Warning Decision Training Branch NWS Storm Prediction CenterNWS Storm Prediction Center Radar Operations Center, Forecast Office in Norman, OKRadar Operations Center, Forecast Office in Norman, OK NWS Southern Region HeadquartersNWS Southern Region Headquarters NESDIS National Climatic Data CenterNESDIS National Climatic Data Center NWS International Activities OfficeNWS International Activities Office OAR Office of Global ProgramsOAR Office of Global Programs OUOU Other NOAA and non-NOAA organizations within Norman, OKOther NOAA and non-NOAA organizations within Norman, OK



Proposed Proposed National Weather CenterNational Weather Center (NWC) (NWC)

– Will transform the current Oklahoma Weather Will transform the current Oklahoma Weather Center into a larger, more structured Center into a larger, more structured organization, which will be led, administered and organization, which will be led, administered and managed by OUmanaged by OU

– Responsibilities of NWCResponsibilities of NWC Lead and implement the University Radar Meteorology Lead and implement the University Radar Meteorology

InitiativeInitiative Establish and direct Center for Radar Meteorology and Establish and direct Center for Radar Meteorology and

Engineering; Radar Phenomenology Laboratory, Radar Engineering; Radar Phenomenology Laboratory, Radar Data Portal, and National Mobile Observing FacilityData Portal, and National Mobile Observing Facility

Direct and administer CIMMS Direct and administer CIMMS



What, if any, formal procedures exist for joint What, if any, formal procedures exist for joint planning?planning?

Recommendation 8Recommendation 8: OU should endeavor to create an : OU should endeavor to create an environment within the facility [National Weather Center] that environment within the facility [National Weather Center] that fosters and maximizes scientific interactions between OU, fosters and maximizes scientific interactions between OU, CIMMS, NOAA personnel and the broader meteorological CIMMS, NOAA personnel and the broader meteorological community.community.

Recommendation 9Recommendation 9: CIMMS must develop operational policies : CIMMS must develop operational policies and processes, which will ensure that the conditions of the and processes, which will ensure that the conditions of the existing NOAA/OU Memorandum of Agreement, particularly existing NOAA/OU Memorandum of Agreement, particularly the delineated research themes, are managed in a manner to the delineated research themes, are managed in a manner to avoid any apparent or perceived conflicts-of-interest with the avoid any apparent or perceived conflicts-of-interest with the mission and mandates of NOAA.mission and mandates of NOAA.


Science ReviewScience Review

What are the Institute’s most recent scientific What are the Institute’s most recent scientific highlights and accomplishments?highlights and accomplishments?

Recommendation 10Recommendation 10: CIMMS should consider assigning a : CIMMS should consider assigning a higher priority to certain aspects of its research program higher priority to certain aspects of its research program where results would more directly impact NOAA operations. where results would more directly impact NOAA operations.

Recommendation 11Recommendation 11: CIMMS should consider expanding its : CIMMS should consider expanding its research initiatives in the theme area: Socioeconomic research initiatives in the theme area: Socioeconomic Impacts of Mesoscale Weather Systems and Regional Scale Impacts of Mesoscale Weather Systems and Regional Scale Variations. CIMMS should consider including in these Variations. CIMMS should consider including in these research initiatives a strategy to quantify the value-added research initiatives a strategy to quantify the value-added benefits to the nation of weather forecasting and research.benefits to the nation of weather forecasting and research.


Education and OutreachEducation and Outreach

What types of educational activities/opportunities What types of educational activities/opportunities (K-12, undergraduate and graduate students) does (K-12, undergraduate and graduate students) does the Institute offer on an on-going basis?the Institute offer on an on-going basis?

Fall 2003Fall 2003: Twenty-seven graduate students : Twenty-seven graduate students supported by CIMMSsupported by CIMMS

Average of ten additional undergraduate students Average of ten additional undergraduate students supported by National Science Foundationsupported by National Science Foundation



Recommendation 12Recommendation 12: It is strongly recommended : It is strongly recommended that CIMMS develop strategic, tactical and that CIMMS develop strategic, tactical and operational plans to address education, public operational plans to address education, public affairs, and outreach, and that NOAA funds are affairs, and outreach, and that NOAA funds are appropriated for these activities.appropriated for these activities.

Recommendation 13Recommendation 13: It is strongly recommended : It is strongly recommended that CIMMS increase its leveraged funding levels in that CIMMS increase its leveraged funding levels in the areas of education, public affairs and outreach.the areas of education, public affairs and outreach.



Recommendation 14Recommendation 14: CIMMS should provide statistics : CIMMS should provide statistics regarding the number/diversity of participants regarding the number/diversity of participants educated through outreach programs in terms of educated through outreach programs in terms of gender, age, ethnicity and citizenship disaggregated gender, age, ethnicity and citizenship disaggregated by participation during the academic year and during by participation during the academic year and during the summer and promote diversity in the workplace.the summer and promote diversity in the workplace.

Recommendation 15Recommendation 15: It is strongly recommended that : It is strongly recommended that CIMMS quantify its education, public affairs and CIMMS quantify its education, public affairs and outreach initiatives, in terms of demographics, to outreach initiatives, in terms of demographics, to demonstrate the value-added contribution of these demonstrate the value-added contribution of these initiatives to meteorology, to NOAA and to the Nation.initiatives to meteorology, to NOAA and to the Nation.



What are the current and planned outreach efforts?What are the current and planned outreach efforts?

CIMMS …CIMMS … 10% of CIMMS Fellows are not affiliated with OU10% of CIMMS Fellows are not affiliated with OU 57% (158 of 278) of refereed publications from 1998-2003 had at 57% (158 of 278) of refereed publications from 1998-2003 had at

least one non-CIMMS affiliated memberleast one non-CIMMS affiliated member Matching support for Vietnam graduate studentsMatching support for Vietnam graduate students Multi-institution program for the Warning Decision Support Multi-institution program for the Warning Decision Support

System II – CIMMS, OU, NSSL, NWS, Australian Bureau of System II – CIMMS, OU, NSSL, NWS, Australian Bureau of Meteorology and the Georgia Tech Research InstituteMeteorology and the Georgia Tech Research Institute

Dissemination of software and training for the Weather Event Dissemination of software and training for the Weather Event SimulatorSimulator

Dissemination of current climate research to the meteorological Dissemination of current climate research to the meteorological and hydrological services of developing nationsand hydrological services of developing nations


Science Management PlanScience Management Plan

Science Management PlanScience Management Plan– Prepared by managers to articulate to staff, Prepared by managers to articulate to staff,

senior officials, and other stakeholders how the senior officials, and other stakeholders how the organization is managed for purpose of organization is managed for purpose of developing and implementing the science plandeveloping and implementing the science plan

Recommendation 16Recommendation 16: OU/CIMMS should develop a : OU/CIMMS should develop a science management plan to state to staff, senior science management plan to state to staff, senior officials and other stakeholders how the officials and other stakeholders how the organization is managed for purpose of developing organization is managed for purpose of developing and implementing its science plan. and implementing its science plan.



How does the Institute identify new intellectual How does the Institute identify new intellectual opportunities?opportunities?

What are some recent examples of intellectual What are some recent examples of intellectual opportunities?opportunities?

CIMMS has…CIMMS has… Large and well-accomplished scientific research staffLarge and well-accomplished scientific research staff Collaborated with NOAA and its partners to define Collaborated with NOAA and its partners to define

intellectual opportunitiesintellectual opportunities Collaborated with OU to project in excess of $200 million Collaborated with OU to project in excess of $200 million

in revenue from NWC in its initial decadein revenue from NWC in its initial decade Involvement in the OU Meteorological Radar Research Involvement in the OU Meteorological Radar Research

Initiative Initiative



What is the strategy for new starts (projects, What is the strategy for new starts (projects, techniques, campaigns, etc.)?techniques, campaigns, etc.)?

– Scientist-driven; proposals submittedScientist-driven; proposals submitted

How much of the Institute resources are reserved for How much of the Institute resources are reserved for new opportunities or bright ideas?new opportunities or bright ideas?

– None identifiedNone identified



What is the demographic structure of the Institute What is the demographic structure of the Institute employees?employees?

– Summer : Employees (144); Male (72%); Caucasian or Asian/Pacific Summer : Employees (144); Male (72%); Caucasian or Asian/Pacific Islanders (90%); African Americans (3%); Native Americans (<1%); Islanders (90%); African Americans (3%); Native Americans (<1%); Hispanic (6%); U.S. Citizens (73%)Hispanic (6%); U.S. Citizens (73%)

What is provided for human resources development? What is provided for human resources development? (Recruitment, rewards, training) (Recruitment, rewards, training)

Recommendation 17Recommendation 17: CIMMS should develop a human : CIMMS should develop a human resource management plan to ensure a diverse resource management plan to ensure a diverse workforce.workforce.



What is the state of the financial health of the Institute? (Provide a What is the state of the financial health of the Institute? (Provide a budget summary and identify imbalances or needed adjustments?budget summary and identify imbalances or needed adjustments?

How does the institute intend to work towards accomplishing its How does the institute intend to work towards accomplishing its financial goals?financial goals?

CIMMS is…CIMMS is… Well-positioned for continued success by virtue of its Well-positioned for continued success by virtue of its

physical and human capitalphysical and human capital

Recommendation 18Recommendation 18: NOAA should require the : NOAA should require the Joint/Cooperative Institutes to provide to Review Teams Joint/Cooperative Institutes to provide to Review Teams briefing materials and presentations that are in direct briefing materials and presentations that are in direct response to the questions that NOAA requires the response to the questions that NOAA requires the Review Teams to assess and evaluate.Review Teams to assess and evaluate.


Science Management PlanScience Management Plan Are there any issues in interacting with NOAA that require Are there any issues in interacting with NOAA that require

attention?attention? Are there any issues in interacting with the University that require Are there any issues in interacting with the University that require

attention?attention?

Recommendation 19Recommendation 19: NOAA should distribute : NOAA should distribute appropriated funds to the Joint/Cooperative Institutes appropriated funds to the Joint/Cooperative Institutes in a timely manner, thus preventing cash-flow in a timely manner, thus preventing cash-flow problems between the host institutions and the problems between the host institutions and the Joint/Cooperative Institutes.Joint/Cooperative Institutes.

Recommendation 20Recommendation 20: NOAA should more fully involve : NOAA should more fully involve Joint Institutes in its strategic/operational and budget Joint Institutes in its strategic/operational and budget planning sessions to ensure that the value-added planning sessions to ensure that the value-added contributions of the JIs are included in the final budget contributions of the JIs are included in the final budget preparations and submissions. preparations and submissions.


Science Management PlanScience Management Plan Are there any issues in interacting with NOAA that require Are there any issues in interacting with NOAA that require

attention?attention? Are there any issues in interacting with the University that require Are there any issues in interacting with the University that require

attention?attention?

Recommendation 21Recommendation 21: The Review Team recommends : The Review Team recommends that NOAA/OAR review its involvement in NOAA’s that NOAA/OAR review its involvement in NOAA’s strategic planning process with a goal of more strategic planning process with a goal of more meaningful involvement on the part of the JIs.meaningful involvement on the part of the JIs.

Recommendation 22Recommendation 22: NOAA/OAR should complete an in-: NOAA/OAR should complete an in-depth review with the assistance of appropriate legal depth review with the assistance of appropriate legal and contracts personnel, and disseminate clear, and contracts personnel, and disseminate clear, unambiguous guidance to all Joint Institutes dealing unambiguous guidance to all Joint Institutes dealing with the Department of Commerce legal concerns over with the Department of Commerce legal concerns over “direct benefit research.”“direct benefit research.”



Are there any issues in interacting with NOAA that Are there any issues in interacting with NOAA that require attention?require attention?

Are there any issues in interacting with the University Are there any issues in interacting with the University that require attention?that require attention?

Recommendation 23Recommendation 23: CIMMS should support and : CIMMS should support and encourage staff to interact with researchers at the encourage staff to interact with researchers at the NOAA laboratories and Joint Institutes across the NOAA laboratories and Joint Institutes across the country and to share their expertise with NOAA country and to share their expertise with NOAA program managers and planners outside the program managers and planners outside the immediate CIMMS/OU area.immediate CIMMS/OU area.

Summary and ConclusionsSummary and Conclusions

CIMMS CIMMS hashas……– Extended NOAA expertise for over 25 yearsExtended NOAA expertise for over 25 years– Increased its staff from 38 in FY1991 to over 140 in FY2004Increased its staff from 38 in FY1991 to over 140 in FY2004– Provided significant contributions to the evolution of NOAA Provided significant contributions to the evolution of NOAA

sciencescience– Received awards for its scienceReceived awards for its science– Generated over 270 referred publications in the past five yearsGenerated over 270 referred publications in the past five years– Grown its budget from $1.4million in FY1991 to over $10 Grown its budget from $1.4million in FY1991 to over $10

million in FY2003million in FY2003

CIMMS, with support from NOAA, CIMMS, with support from NOAA, cancan … …– More strongly position itself to benefit NOAA and the nation More strongly position itself to benefit NOAA and the nation

external review of the cooperative institute for mesoscale meteorological studies (cimms) 15-16...

Documents