1 © 2000 technology commercialization group llc. all rights reserved technology commercialization...
TRANSCRIPT
1© 2000 Technology Commercialization Group LLC. All rights reserved
TECHNOLOGYCOMMERCIALIZATION
GROUP, LLCPAUL G. WAUGAMAN
LOUIS G. TORNATZKY
Higher Education PracticeWILLLIAM S. KIRBY
APPLYING BENCHMARKING TO SPONSORED PROGRAMS ADMINISTRATION
SRA - NCURASouthern Section/Southeastern Region Meeting
April 16-17, 2000
TCG
2© 2000 Technology Commercialization Group LLC. All rights reserved
Points to Cover:
Benchmarking in research administration» Overview
The Higher Education Benchmarking Consortium» Overview
The KPMG - NACUBO - SRA Sponsored Programs Benchmarking Effort» Organization» Initial results» Next steps
3© 2000 Technology Commercialization Group LLC. All rights reserved
Benchmarking
Systematic comparison of elements of the performance of an organization against that of other organizations, with the aim of mutual improvement.
McNair and Leibfried: “Benchmarking.” 1992
4© 2000 Technology Commercialization Group LLC. All rights reserved
The Utility of Benchmarking
Provides a basis for comparison and self-analysis.
Accelerates the sharing of useful and novel approaches.
Energizes everybody to do better.
5© 2000 Technology Commercialization Group LLC. All rights reserved
Purposes and Uses of Measurement in Research Administration
Measure against “industry standards”
Measure against goals» Performance drives improvement interventions.
Identify problems» Diagnose and forecast existing problems.
Build the “business case” for change or new resources.
6© 2000 Technology Commercialization Group LLC. All rights reserved
The Problem:In Research administration .... There are no generally accepted criteria
for success. We rely on anecdotal data and informal
case study rather than systematic data collection and analysis.
Benchmarking and measurement are not part of the improvement culture
Competitive comparisons may be viewed as threats rather than improvement drivers.
7© 2000 Technology Commercialization Group LLC. All rights reserved
The Problem (cont):
Breadth and diversity of the domain make data collection and measurement difficult.
Successful outcomes are difficult to demonstrate objectively.
There is little data available.
Who is going to do it?
8© 2000 Technology Commercialization Group LLC. All rights reserved
The SRA Benchmarking Initiative
Benchmarking and Best Practices Task Force established in 1998.
Task Force recommends working with NACUBO and KPMG to facilitate performance data collection.
Collaboration with NACUBO and KPMG initiated in September 1998.
Data collection instrument refined with SRA Task Force input in Spring, 1999.
9© 2000 Technology Commercialization Group LLC. All rights reserved
The KPMG Higher Education Benchmarking Consortium
The purposes of the consortium are:» To serve as a comprehensive source of
institutional comparative performance, benchmarking, and “best practice” information for the higher education community; and
» To facilitate the use of comparative performance data to meet strategic, operational improvement, and change management needs.
10© 2000 Technology Commercialization Group LLC. All rights reserved
The KPMG Higher Education Benchmarking Consortium
Operating Principles
» Participant-driven» “E-nabled”» Focused» Leading to "best practices” » Leveraged, where possible
11© 2000 Technology Commercialization Group LLC. All rights reserved
The KPMG Higher Education Benchmarking Consortium
Major Segments» Financial Health and Operations» Student Affairs» Physical Infrastructure» Information Technology» Human Resources» Research:
Sponsored Programs Technology Transfer
12© 2000 Technology Commercialization Group LLC. All rights reserved
KPMG - NACUBO - SRA Sponsored Programs Survey
Purposes To provide a common set of sponsored
programs performance indicators that will allow meaningful comparisons with other organizations and with peer groups of institutions.
To facilitate the development of a benchmarking data base.
13© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
SRA-NACUBO-KPMG Roles
NACUBO and KPMG are financing and organizing data collection and analysis, and are promoting the program with their constituencies and clients.
SRA is participating in survey design, is promoting the program with their members, and is participating in dissemination of findings.
14© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Features of the Program
Low cost to participating institutions. Individualized reports for
participants. Expert analysis. All institution-specific data will be
“blinded.” Only general reports will be available
to non-participants.
15© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Critical Success Factors for Research Administration
Sustaining or Enhancing Sponsored Research Activity and Funding (competitiveness)
Containing the Costs and Increasing the Efficiency of Sponsored Research Administration
Improving Service to Faculty (in order to achieve Objective 1)
Maintaining and Improving Institutional and Sponsor accountability
16© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey Indicators:
Sustaining or Enhancing Sponsored Research Activity and Funding
Number of proposals submitted per faculty FTE. Percentage of faculty working as principal
investigators. Sponsored project dollars received per
faculty FTE. Sponsored projects funding growth rate. Number of new awards as a percent of new
proposals submitted.
17© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Indicators:Cost and Efficiency
Number of proposals per sponsored projects admin. FTE.
Sponsored projects admin. cost per proposal. Number of awards per Sponsored projects admin
FTE. Sponsored projects admin costs as percent of
dollars received. Number of awards per research accounting FTE. Research accounting costs per award Dollars received per research accounting FTE. Research accounting cost as percent of dollars
received.
18© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey Indicators:
Service to Faculty
Number of funded PI’s per sponsored projects FTE.
Number of active PI’s per sponsored projects FTE.
Number of funded PI’s per research accounting FTE.
19© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey Indicators:
Institutional and Sponsor Accountability
Percent of Sponsored Projects Accounts Receivable over 120 days.
20© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Data Needed to Develop Indicators
Fy 1997-98 Data: Number of faculty FTE. Number of central administrative FTE (sponsored
programs, research accounting). Proposal data (number, dollars, sponsor: federal,
industry, other). Award data (number, dollars, sponsor: federal,
industry, other). Sponsored program expenditure data Administrative costs allocated to research (central and
departmental costs - from indirect cost proposal).
21© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Benefits for Participants
Access to useful comparable data for program evaluation and benchmarking.
Low cost. Ease of preparation. Uses readily available data
assembled for internal reporting or other purposes.
22© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One Questionnaire Design (Dec 98-Jun 99)
» Survey design was peer-based» Intentions were to use available data» Initial survey requested FY 1998 data
Data Collection (Jul-Oct 99)» A high response rate was a major objective» Timing did not accommodate academic calendar» Correct POC identification was a challenge» Follow up efforts: phone calls, letters, e-mails
23© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One Data Collection Outcomes:
» 62 institutions participated
(40% of all US college/university research expenditures)
» Definition issues arose Examples: Who are “research faculty;” new vs. renewal
awards Caused confounding metrics
Data Processing» 34 institutions chose peers» 3 sub-sets of institutions
24© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One Reporting:
» Institution-specific reports: Customized, Institution-specific. Provided ranking
information on 16 metrics, and 4 sub-groups for each metric.
» Data feedback workshops Gave participants opportunities to discuss data, and
next steps
» Revised Reports will be prepared after participants have an opportunity to revise submissions
» Public Report in preparation
25© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year OnePercentage of faculty working as principal investigators. Number Median
Value
Entire Sample 62 39.1%
Public 51 39.0%
Independent 11 40.0%
Research I 33 40.9%
Research II 16 40.4%
Other CarnegieClasses
13 20.8%
Land-Grant 18 44.1%
Medical Schools 27 40.4%
26© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year OneSponsored Project dollars per Faculty FTE
Number Median Value
Entire Sample 62 63,453
Public 51 61,289
Independent 11 86,118
Research I 33 80,923
Research II 16 53,944
Other CarnegieClasses
13 23,320
Land-Grant 18 68,915
Medical Schools 27 80,923
27© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Sponsored Projects Funding Growth - Fy 93 - 98Number Median
Value
Entire Sample 62 6.3%
Public 51 6.3%
Independent 11 6.6%
Research I 33 6.3%
Research II 16 3.7%
Other CarnegieClasses
13 8.3%
Land-Grant 18 5.5%
Medical Schools 27 6.8%
28© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Number of proposals submitted per central sponsored projects administrative FTE
Number MedianValue
Entire Sample 61 70.7
Public 50 73.7
Independent 11 54.9
Research I 32 83.9
Research II 16 58.8
Other CarnegieClasses
13 45.1
Land-Grant 17 90.8
Medical Schools 27 86.3
29© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Central sponsored projects administrative cost as a percent of sponsored project dollars
Number MedianValue
Entire Sample 62 1.02%
Public 51 0.97%
Independent 11 1.24%
Research I 33 0.80%
Research II 16 1.27%
Other CarnegieClasses
13 2.11%
Land-Grant 18 0.81%
Medical Schools 27 0.82%
30© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Central post-award financial administrative cost per active project
Number MedianValue
Entire Sample 60 $389
Public 49 $369
Independent 11 $477
Research I 32 $310
Research II 15 $470
Other CarnegieClasses
13 $628
Land-Grant 18 $289
Medical Schools 25 $375
31© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Central post-award financial administrative cost as a percent of sponsored project dollars
Number MedianValue
Entire Sample 61 0.70%
Public 50 0.65%
Independent 11 0.71%
Research I 32 0.49%
Research II 16 0.88%
Other CarnegieClasses
13 0.95%
Land-Grant 18 0.56%
Medical Schools 27 0.57%
32© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Number of active Principal investigators per central sponsored projects admin. FTE
Number MedianValue
Entire Sample 60 33.3
Public 49 34.2
Independent 11 23.6
Research I 31 37.3
Research II 16 27.7
Other CarnegieClasses
13 23.5
Land-Grant 18 44.5
MedicalSchools
26 38.1
33© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Next Steps
Complete Fy 1998 round» Reopen data base for revisions and
additional enrollments (April-May, 2000)
» Complete a brief public report (May, 2000)
» Complete revised individual reports (June, 2000)
» Brief Best Practices study (June - August, 2000)
34© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Next Steps
Initiate Fy 2000 round» Target date to begin data collection: September-
November, 2000» Activate Web Site for data collection on-line
(September, 2000) and report preparation by participants (January, 2000)
» Reports: January, 2001
Web site (for informational purposes): www.us.kpmg.com/highered/benchmark/
35© 2000 Technology Commercialization Group LLC. All rights reserved
Contact us ...
TECHNOLOGY COMMERCIALIZATION GROUP, LLC2237 OXFORD HILLS DRIVE
RALEIGH, NORTH CAROLINA 27608-1672 USA
TEL: 919-833-2569
FAX: 919-833-3277
E-MAIL: [email protected]
PAUL G. WAUGAMAN, PRINCIPAL
WWW.T-C-GROUP.COM
TCG