1

Research and Development Evaluation in Japan & MEXT

Naoko OkamuraDirector, Office of R&D Evaluation

Science and Technology Policy Bureau

Ministry of Education, Sports, Culture,

Science and Technology (MEXT), Japan

2

Content

Ⅰ． Overview

1. Administrative Structure for S&T in Japan

2. National S&T Strategy

3. Structure of R&D Evaluation in Japan

4. History of R&D Evaluation in Japan

Ⅱ． R&D Evaluation in MEXT

5. Summary of the MEXT Guideline

6. Current System of R&D Evaluation

7. Evaluation System in Competitive funds

(Example with Special Coordination Funds for Promoting R&D

8. Evaluation System of RIKEN

9. Evaluation System of NIMS

3

Ⅰ. Overview

4

1. Administrative Structure for S&T in Japan

Prime Minister

NPAMPHPT MOF

MLITMHLWMAFF MOE

University

Public Corporation, Independent Administrative Institution, National Research Institute

MOFA

etc.

Cabinet Office Minister of State (S&T Policy)

Presenting a Basic Policy, Comprehensive Coordination

CSTP : Council for S&T PolicyAEC : Atomic Energy CommissionNSC : Nuclear Safety CommissionNPA : National Police AgencyMPHPT : Ministry of Public Management, Home Affairs, Posts and TelecommunicationsMOFA : Ministry of Foreign AffairsMOF : Ministry of FinanceMEXT : Ministry of Education, Culture, Sports, S&TMHLW : Ministry of Health, Labour and WelfareMAFF : Ministry of Agriculture, Forestry and FisheriesMETI : Ministry of Economy, Trade and IndustryMLIT : Ministry of Land, Infrastructure and TransportMOE : Ministry of the Environment

AEC & NSC○Designing, Planning,

and coordinating Policies

CSTP○Formulation of comprehensive

strategy○Policy for allocation of resources

such as budget and human resources etc.

○Formulation & promotion of concrete R&D plans○Coordination among relevant ministries - Special Coordination Funds for Promoting S&T - Coordination on S&T international cooperation

(Since Jan. 2001)

METIMEXT

4

5

2. National S&T Strategy

2-1 S&T Basic Plan for 2001-2005

Three Basic Characteristics - Creating and utilizing scientific knowledge to contribute to the world - Developing international competitiveness and ability of sustainable development - Securing safety and quality of life

Basic Policies - Strategic Priority Setting - S&T System Reforms

Doubling competitive research funds, introduce indirect costs to enhance competitive atmosphere

Improving mobility of human resources in S&T Increasing young researchers funds to exhibit their talent Enforcing effective evaluation

- Internationalization

6

3. Structure of R&D Evaluation in Japan

Program for Major Activities

RIKEN Guideline JST Guideline

MEXT Guideline

NEDO Guideline NIAIT Guideline

METI Guideline Other Ministries

General Guideline for R&D Evaluation

Decided by the Prime Minister

S&T Basic Plan

Cabinet decision

National Level

Ministry Level

Organization Level

Program for Major Activities

CSTP is promoting R&D evaluation as an integral part of S&T policy Ministries and R&D Organizations carry out the major part ot evaluation activities

7

4. History of R&D Evaluation in Japan

1997: First National Guideline - Introduced R&D evaluation - Encouraged external evaluation - Aimed at efficient resource allocation

2001: Revised National Guideline - To upgrade fairness and transparency - To strengthen link with budgeting - To establish resource base for evaluation

2005: Revised National Guideline - To encourage researchers’ challenge for creativity - To develop evaluation skills to increase reliability - Evaluation results should be utilized more strictly

MEXT has just established “Guideline for Evaluation of Research and Development in MEXT” in Sep.2005

8

Ⅱ. R&D Evaluation in MEXT

9

5. Summary of the MEXT Guideline

Goals of the evaluation

To promote Research developing new frontier of S&T Research contributing to the society/economy

Significance of the evaluation Realizing open, flexible and competitive R&D environment Allocating R&D resources efficiently and effectively Obtaining public support through disclosure of R&D achievements

5-2 Basic Concept

5-1 Role- Sets out the basic approach for conducting evaluations on R&D which fall under MEXT’s jurisdiction. Internal divisions of MEXT should conduct evaluations based on this guideline while R&D institutions should refer to this guideline for conducting appropriate evaluations

10

5-3 Direction of Reform

Encouraging researchers’ challenge for creativity, promoting new frontier research

Recognize that former evaluation sometimes discourage researchers Continuation of the excellent projects through evaluation Announce the results of reflection

Ensuring the evaluation resources and improving evaluation system Prepare necessary budget for evaluation process Foster personnel for evaluation by training Develop the National R&D database

Develop efficient evaluation Eliminate too much procedure and too many data for evaluation

11

5-4 Common Principles

Setting the specific objective and aim of the evaluation Selection and assignment of reviewers

Avoid conflict of interest Timing of evaluation

Follow-up activity after the evaluation Method of evaluation

Announcing the procedure, criteria, etc. to the object Evaluating “quality” rather than “quantity” Expanding objective indicators use taking consideration for their limit Revealing criteria for evaluation Securing flexibility and avoiding burden

Utilization of the results Reflection to the distribution of resources

12

5-5 Category of Evaluation

Research and Development Policy Program, system etc.

Research and Development Theme Competitive fund Project

Research and Development Institute Independent Administrative Institutions National Laboratories Public Corporations Universities

Achievement of researcher

13

6. Current System of R&D Evaluation (1)

CST:Council for Science and TechnologySAC:Space Activity Commission

R&DTheme

○Grants- in- Aid for Scientific Research○Special Coordination Funds for Promoting S&T

ＤＯ ＳＥＥ

Ex- post Evaluation

ＰＬＡＮ

Research Supervisorsand Research

Directors

Research Supervisorsand Research

Directors

CST CST

SACSpace Development

R&D Projects

NationalR&D

projectsetc.

CompetitiveFunds

J ST Basic ResearchProgram

CST, etc. evaluation should be conducted before budget allocation for all newprojects over 1 billion yen the total amount.

SAC SAC

Interim Evaluation

CST CST

Prior Evaluation

Research Supervisorsand Research

Directors

CST

CST

13

14

IAI:Independent Administrative InstituteNUC:National University CorporationIAIEC: Independent Administrative Institute Evaluation CommitteeNUCEC: National University Corporation Evaluation CommitteeMPHPT:Ministry of Public Management ,Home Affairs,Posts and Telecommunications

6. Current System of R&D Evaluation (2)

ＰＬＡＮ

Ex- post Evaluation

NUC

IAI

CST,etc..(Attainment of achievement goal set

beforehand is evaluated)

NUCEC(each year)

NUCEC(after period over

medium- termplans)

In the authorization ofmedium- term plans,

IAIEC advices to MEXT.

IAIEC(each year)

ＤＯ ＳＥＥ

R&DInstitute

R&DPolicy

Prior Evaluation Interim Evaluation

CST,etc..(projects and programs

over 1 billion yen)

Minitriries' Policy, R&D Program,R&D System

IAIEC(after period over

medium- termplans)

In the authorization ofmedium- term

plans,NUCEC advicesto MEXT.

※ Evaluation results are checked by MPHPT IAIEC secondarily.

15

7. Evaluation System in Competitive funds - Special Coordination Funds for Promoting R&D -

　

Hearing

examination(consultation)

Documentary examination

Peer reviews

Working groups(=expert panel)

Acceptance of

applicationsEx-post

EvaluationR&D( 5 years )

Interim Evaluation

WGs, are established in the every fields and subjects are examined.

POINTS○SCF is fund for developing new frontier of S&T and contributing to the society/economy. ○ Evaluation for various aspects . Scientific evaluation by Peer reviews Final Selection by Expert Panel (including industry researchers)○ Working Groups are supported by Scientific stuff Program Officers,Program Director

Selection of

subjects

by CST

3rd year

established in JST(=Japan S&T corporation)

Screening(=Prior

Evaluation ）

16

(1) Comprehensive research and testing

(a) basic science research; (b) promotion of accelerator scientific research; (c) promotion of bioresources businesses; (d) promotion of synchrotron radiation research; (e) promotion of integrated collaborative in research; (f) promotion of comprehensive brain science research; (g) comprehensive genome science research; (h) promotion of botanical science research; (i)comprehensive research in developmental biology; (j) promotion of SNP research; (k) promotion of comprehensive research into immunology and allergy science; and (l) promotion of strategic research

(2) Dissemination and active utilization of research achievements

Presentation of research reports, access to resources in biogenetics, establishment of intellectual property rights on scientific achievements, licensing of patents, etc.

(3) Shared-use of facilities and equipment

(4) Training of researchers and engineers and advancement of their qualities and skills

(5) Promotion of shared use of Spring-8

Principal operations

Ryoji Noyori, Dr. Eng.President

2,835 full-time employees (of which 692 are executives and Mandatory Retirement Age Contract staff)Employees as of end of 2004

¥83,956 million (of which ¥74,920 million is funded from the government subsidy)

¥86,796 million (of which ¥75,882 million is funded from the government subsidy)

2004 budget

2005 budget

To conduct comprehensive research in science and technology (excluding humanities and social sciences) and to foster advances in scientific research and technological standards

Objective

October 1, 2003Date of Establishment

RIKEN Law (Independent Administrative Institution founded under Law No. 160, December 13, 2002)

Basis for Establishment

8. Evaluation System of RIKEN 8-1 Profile of RIKEN

17

8-2 Eight Features of RIKEN Operations

1. Longstanding reputation as a comprehensive research institute

2. Broad range of research

3. Versatile research organization adapted to the nature of research

4. Promotion of alliances with industry and technology transfer

5. Competitive and diversified research funding

6. Open research system

7. International Institute

8. Research evaluation on multiple stages

・ History of 88 years since 1917・ Japan’s only comprehensive research institute of natural science

・ Large number of fields (physics, chemistry, engineering, biology, medicine, etc.)・ Interdisciplinary research・ From basic science to applied research 　　

・ Institution Laboratory (for emerging research area, compound research and integrated research areas; hub organization) ・ Frontier Research System (new areas of research & target-oriented)・ Center system (centralized action in prioritized fields) ・ Availability of research platform (bioresources, SPring-8, etc.)

・ Joint research, patent licensing, etc.

・ Acquisition of external funding that is competitive and diverse・ Creation of competitive environment for funding inside the Institute

・ Joint research and broad-ranging personnel exchanges with universities, business enterprises, etc.・ Cooperation Program for graduate school withuniversities in Japan

・ Large number of foreign researchers・ Three overseas research centers・ Cooperation with foreign research institutes (including NIH & Max Planck)

・ First systematic evaluation by external sources, including foreign researchers ・ Evaluation on research unit revels (by Centers, by laboratories, etc.)・ Evaluation of research themes & research performance

18

Executive Directors　 Kenji Okuma　 Tsutomu Shibata Motohide Konaka　 Yoshiharu Doi, Dr.Eng.　 Kenji Takeda, Dr.Eng.

Auditors　 Takashi Fujii　 Tsuyoshi Hayashi

President　 Ryoji Noyori, Dr. Eng.

Research Priority Committee

Harima Institute (Tetsutaro Iizuka, Dr.Sci.)Research organization (laboratories, research technology development office, etc.), Harima Research Promotion Division, Harima Safety Center

　 Yokohama Institute (Tomoya Ogawa, Dr. Agr.)Genomics Sciences Center, Plant Science Center, SNP Research Center, Research Center for Allergy and Immunology, Yokohama Research Promotion Division, Safety Center

　 Kobe Institute (Masatoshi Takeichi, Ph.D.)Center for Developmental Biology, Kobe Research Promotion Division, Kobe Safety Center

Tsukuba Institute (Tsutomu Shibata)Bioresource Center, Research Collaborative Group, Tsukuba Research Promotion Division, Tsukuba Safety Center

Wako Institute (Koji Kaya, Ph.D.)Discovery Research Institute, Frontier Research System, Brain Science Institute. Initiative Research Unit, Special Laboratories, DRI/FRS Promotion Division, Brain Science Promotion Division

　　　＜ Overseas Centers ＞　　　 RAL Laboratory, UK, BNL Research Center, US (Discovery Research Institute)　　　 MIT Brain Science Research Center, US (Brain Science Institute)

RIKEN HeadquartersPolicy Planning Division, Public Relations Office, General Affairs Division, Personnel Division, Finance Division, Contract Management Division, Technology Transfer and Research Coordination Division, Facilities and Utilities Division, Safety Division, Auditing and Compliance Office, Internal Communications and Systems Support Office , Advanced Center for Computing and Communication, Center for Intellectual Property Strategies

8-3 RIKEN Organization & Research Centers

18

19

RIKEN OverviewRIKEN OverviewRIKEN Advisory Council (RAC) = Evaluation of R&D Organization

Discovery Research Institute & Harima Institute

Centers

Evaluation of R&D Themes

8-4 Evaluation System of RIKEN

Recommendation to Center Directors on research activities and management of Center.（ Advisory Council ）

Evaluation by external assessor of each research group as a unit

・ RAC: advisory body to the President on overall research activities and management in RIKEN

・ External evaluation of research laboratories (research performance review since 1972)

・ Internal and external evaluation by research themes (internal evaluation since 1965)

Evaluation of Researchers’ performance

Internal evaluation committee to assess promotion for researchers (ex. research → senior researcher)

Evaluation conducted by Head of each research unit at the end of each fiscal year, when contracts are renewed.

The Evaluation System is essential to make appropriate judgments on R&D activities, hence to increase work efficiency and vitalize research activities to achieve higher goals.

Evaluation of R&D Organizations

20

RIKENAdvisoryCouncil

PresidentPresident

The Board ofThe Board ofExecutive DirectorsExecutive Directors

AC

AC

AC

RIKEN Frontier Research SystemDirector ， FRS

RIKEN Brain Science InstituteDirector ， BSI

RIKEN Center for Developmental BiologyDirector ， CDBAC

RIKEN Genomic Sciences CenterDirector ， GSCAC

RIKEN Plant Science CenterDirector ， PSCAC

RIKEN SNP Research CenterDirector ， SNP

Review Report Advice

RecommendationRecommendation

ACRIKEN Discovery Research Institute &Harima InstituteDirectors, DRI & HI

RIKEN Research Center for Allergy and ImmunologyDirector ， RCAI

RIKEN BioResource CenterDirector ， BRC

AC

AC

Response

8-5 Advisory Council System of RIKEN

21

8-6 Distinctive Features of the Advisory Council System

1. Advisory Council (AC) Members are carefully, as well as internationally, selected from different academic fields to cover RIKEN’s broadly conducted research activities.

　　　 → More than half of the members are foreign nationals. All five RAC meetings of the past

were chaired by foreign nationals.

2. AC chairman of each Center to participate in the RAC meetings as a member.　　　→ Appropriate for RAC which conducts umbrella evaluation of RIKEN.

3. Reports are produced only by AC members so that activities in Centers are directly and openly incorporated.

　　　 → Assurance of objectivity and fairness (however, RIKEN might present its arguments

vis-à-vis those reports)

4. Responses to the RAC recommendations are compiled in a report immediately after the RAC meeting, which is sent to RAC Members. The final progress of actions taken to respond to the RAC recommendations is to be evaluated at the next RAC meeting.

　　　 → “ Recommendation Response Reporting” makes a perfect flow to contribute ⇒ ⇒to the 　　　 RIKEN’s development.

22

8-7 Advisory Council Status (Past 5 Years)

RAC (RIKEN Advisory Council） Fourth meeting: 4-7 June 2000Fifth meeting: 6-9 June 2004

ILAC (Institute Laboratories AC) First meeting: 6-9 February 2000Second meeting: 1-4 February 2004

FRAC (Frontier Research System AC )

First meeting: 11-12 May 2000Second meeting: 7-8 June 2001Third meeting: 6-8 April 2004

BSAC （ Brain Science Institute AC）

Third meeting: 12-14 April 2000Fourth meeting: 4-6 April 2001Fifth meeting: 18-20 September 2002Sixth meeting: 12-14 April 2004

BRAC （ BioResource Center AC) First meeting: 1-3 March 2004

GSAC （Genomic Sciences Center AC）

First meeting: 21-23 March 2000Second meeting: 30 November - 2 December 2003

PSAC （ Plant Science Center AC）

First meeting: 12-14 March 2003Second meeting: February - March 2004 (mail review)

SRAC （ SNP Research Center AC）

First meeting: 30 May 2001Second meeting: 11-13 December 2002Third meeting: 8-10 February 2004

AIAC （ Research Center for Allergy and Immunology AC）

First meeting: 26-28 April 2004

DBAC （ Center for Developmental Biology AC）

First meeting: March 2001 (mail review)Second meeting: 23-25 April 2002Third meeting: 1-4 June 2004

23

The RIKEN Advisory Council conducts an overall review of RIKEN and makes recommendations and proposals to the President on managerial issues including management of the Board of Executive Directors.

1. Recommendations on policies to further vitalize research activities2. Identifying research areas where resources should be increased and where

further development is required3. Recommendations on new areas of development, especially on how to embark on

unexplored fields4. Suggesting a policy to promote internationalization in RIKEN and international

cooperation by RIKEN5. Any other recommendations on other issues and measures for the advancement of

RIKEN

8-8 What is the RAC?

Meetings First meeting: 21-24 June 1993Second meeting: 26-29 June 1995Third meeting: 31 May - 5 June 1998Fourth meeting: 4-7 June 2000Fifth meeting: 6-9 June 2004Sixth meeting: 6-9 June 2006 (tentative)

24

8-9 Terms of Reference to the 5th RAC Meeting- Noyori Initiative -

1. Visibility of RIKEN Improve the image and public recognition of RIKEN Researchers and administrative staff should make the public aware of the importance of science and t

echnology

2. Maintaining RIKENs Outstanding History of Achievement in Science and Technology Sustain and build the RIKEN research spirit Emphasize quality of research. Maintain excellence and prestige of the RIKEN brand Strengthen action further to achieve intellectual properties and to make contributions to society and i

ndustry

3. RIKEN that Motivates Researchers Curiosity-driven concepts Presenting challenges that are unique and high in risk level Fostering promising human talent

4. RIKEN that is Useful to the World Close ties with industry and society Science and technology that supports civilized society (beyond the boundaries of universities or indu

stry)

5. RIKEN that Contributes to Culture RIKEN must raise the level of its own culture Dissemination of knowledge to humanities and social science

25

8-10　 RAC Recommendations of the 5th RAC Meeting

1. Build a strong scientific vision for the future of RIKEN 1a) Revise the scientific governance structure 1b) Develop a long-term plan for the support of basic research activities at RIKEN2. Strengthen the role of RIKEN President 2a) Set up an external advisory board reporting to the president 2b) Strengthen the position of the center and institute directors3. Increase efforts to build strategic relationships 3a) Broaden the scope of internal strategic program4. Develop programs to increase the quality of post-doctoral staff and graduate students w

orking at RIKEN 4a) Create a pre-doctoral fellowship program to support graduate students working at RIKEN 4b) Provide support for all staffers seeking employment after their RIKEN contracts ends5. Reassess RIKEN’s technology transfer regime 5a) Develop a strategic framework to support translational research6. Increase the number of foreign scientists working at RIKEN 6a) Increase the number of Japanese women scientists in leading positions at RIKEN 7. Implement best practices in management and administrative services 7a) Develop a coherent long-term personnel management strategy 7b) Develop best practices for the management of large facilities and resource collections

26

8-11 Issues under Evaluation

・ Combine information overlapped among various evaluation activitiesThe common information such as research performance is repeatedly gathered by different persons for various evaluation activities as the information is required at a different timing and by different evaluation organizations, i.e. IAI (Independent Administrative Institution) Evaluation and RAC Meeting. Having that in mind, it is advisable to establish an efficient data-management system for combining and/or sharing such common information to avoid duplicate work and reduce workload of both researchers and back-office staff.

・ Pressure from Evaluation

Evaluation imposes pressure on researchers, especially in research organizations consisting of Fixed-term Contract Researchers. This seems to be increasing tensions among researchers beyond what is necessary.

　　→　 Discourage researchers to take on challenges

27

$173.5 million (of which $146.5 million is funded from the government subsidy and $23.2 million is income for entrusted research (ex. Competitive fund))

2005 budget

1. Conduct “Basic” and “Generic and Infrastructural” research2. Promote dissemination of research outcomes and their utilization3. Open advanced facilities and equipments to outside researchers4. Educate materials scientists and oegineers

Mission

April 1, 2001Date of Establishment

NIMS Law (Independent Administrative Institution)Basis for Establishment

9. Evaluation System of NIMS 9-1 Profile of NIMS

Nanomaterials

Safe Materials

Environment and Energy Materials

Improvement of Research and Intellectual Infrastructure

Focused R&D areas under Five-Year Program at MINS

(2001-2006)

KISHI Teruo, Dr. Eng.President

1,538 full-time employees (of which 692 are permanent staff)Employees as of end of 2004

28

9-2 Organization of NIMS

President

Advanced Materials Laboratory

Nano-materials Laboratory

Materials Engineering Laboratory

Biomaterials Center

Superconducting Materials Center

Computational Materials Science Center

Steel Research Center

Ecomaterials Center

High Magnetic Field Center

Materials Information Technology Station

Material Analysis Station

High Voltage Electron Microscopy Station

Nanotechnology ResearchersNetwork Center of Japan

International Center for Young Scientists

Cooperative Graduate SchoolNIMS-Course, Univ. of Tsukuba

29

4. R & D Evaluation・ based on “R & D Evaluation

Procedure”

1. Evaluation of NIMS

・ by MEXT IAI Committee (Annual Evaluation of the Mid-term Program Achievement)

Use of Evaluation Results

2. Making Self-rating Sheets by NIMSSecretariat: ISO and REO

iv) R&D projects by NIMS Competitive Funds

・ by Internal Committee (advance and ex post facto)

i) R&D Projects by External Funds

ii) Major Research Projects

iii) Exploratory Researches

・ by Funding Agency

・ by External Committee 　　　　　(advance, mid-term, ex post facto and follow-up)

・ by Director General of Each Unit (advance and ex post facto)

Secretariat: REO

5. Researchers Personal Evaluation

・ based on “Researchers Personal Evaluation Procedure”

6. NIMS and ResearchUnit Evaluation

Secretariat: ISO

Secretariat: ISO

Secretariat: MEXT

Evaluation Results are Reflected on Management and R & D Activities through Reallocation

of Budget by the Executive Meeting

Reflection of Evaluation Results

Secretariat: ISO and REO

MEXT: Ministry of Education, Culture, Sports, Science and TechnologyMIAC: Ministry of Internal Affairs and Communications

IAI: Independent Administrative InstitutionISO: Integrated Strategy OfficeREO: Research Evaluation Office

R & D: Research and Development

3. MIAC Evaluation Committee for Policy and IAI

・ by the NIMS AdvisoryBoard

Submit self-rating sheets

9-3 Schematic Image of Evaluation System in NIMS

30

Management of research organization

I. Improving the service to the people and other management

I. 1. Basic research and development

• 1. 1 Major research projects

I. 1. 1. 1 Nanomaterials

I. 1. 1. 2 Environment and energy materials

I. 1. 1. 3 Safe materials

I. 1. 2 Improvement of research and intellectual infrastructure

I. 1. 2 1) Improvement of research infrastructure

I. 1. 2 2) Improvement of intellectual infrastructure

I. 1. 3 Exploratory researches

I. 1. 4 Proposal to external funds and acceptance of entrusted researches

9-4 Headings of NIMS evaluation as an independent administrative institute (1)

31

I. 2. Dissemination of research outcomes and their utilization

I. 2. 1 Dissemination of research outcomes and promotion of public relations

I. 2. 1 ① Research products

I. 2. 1 ② Public relations

I. 2. 1 Publishing materials data bases and data sheets③

I. 2. 2 Promotion of technology transfer

I. 3 Making facilities and equipments open to non-NIMS researchers

Number of patent applications Number of licensed patents

32

I. 5. 2 Research exchanges

I. 5. 2 1) Implementation of joint research and promotion of cooperation

I. 5. 2 2) Reception of researchers from other institutes

I. 5. 2 3) Dispatch of NIMS researchers to other institutes

I. 5. 3 Cooperation in accident-cause-finding

I. 5 Others

I. 5. 1 Improvement of survey and coordinate functions

I. 4 Training and education of researchers and techniciansI. 4. 1 Reception of traineesI. 4. 2 Participation and dispatch of lecturers to scientific societies and academic meetings

Number of participants to the scientific societies and academic meetings Number of lecturers dispatched to scientific societies and academic meetings

33

II. Steps to achieve goals for efficient management

II. 1. Organizational and management systems

II. 1. 1 Fundamental policy for research organization

II. 1. 2 1) Expansion of project leaders discretion II. 1. 2 2) Rational arrangement of human resources

II. 1. 2 Fundamental policy for management

II. 1. 2 3) Efficiency of management

II. 1. 1 Construction of research system ①

II. 1. 1 Construction of research support system ②

II. 1. 1 Construction of technology transfer system③

II. 1. 2 3) Simplicity, rapidity and efficiency of administrative procedures ① II. 1. 2 3) ②Outsourcing II. 1. 2 3)

Efficient management of operating subsidy③

34

III. Budget, income and expenditure and fund plan

IV. Amount of short-term loan payable

V. Plan for disposal or mortgage of important assets if planned

VI. Use of surplus fund

VII. Others concerning management specified in MEXT ordinanceVII. 1. Planning for facilities and equipments

VII. 2. Planning for personnel affairs

35

Times of evaluation ： Advanced/ mid-term (for 5-years projects)/ ex post facto/ follow-up (for selected projects)

Members of evaluation committee: (1) Candidates of about 10 specialists outside NIMS are proposed. (2) Persons concerned are excluded. (3) 4 or 5 are to be selected.

Preparation: Materials for evaluation and evaluation sheets are sent beforehand to committee members.

(The NIMS rule is based on the government outline and MEXT guidelines for evaluation.)

(25 Major research projects are evaluated.)

9-5 Outline of Evaluation of Major Research Projects

36

Evaluation committee: Project leaders explain the aim, objective, plan, results and so on. Committee members and the project leaders have discussion.

Evaluation results: Evaluation sheets compiled by the chairperson are submitted to the project leaders. If the leader is against the results, discussion will be held between committee members and the project leaders through the secretariat.

Reflection of results: The president of NIMS confirms the final evaluation results. The results are reflected on the budget and to improve the project.

Official announcement: The final evaluation results are disclosed to public via internet.

37

9-6 Performance-based Personnel System

Objective Maximize the research achievements under a competitive environment to

accomplish the institute’s goals Framework

Develop achievement-oriented pay and results-driven achievement systems

Reward high-performing researchers Achievements to evaluate

3P (Papers, Patents, Products) Scored automatically by on-line system

Paper = 3 + Impact factor x 2 Activities contributed to NIMS’ missions except 3P （management,

academic activities, collaborations, research assistance, etc.） Rated by supervisor

38

9-7 Performance-based Pay System

Both monthly pay and bonus reflect Individual achievements

Performance-based pay: determined by individual achievements over last several years

Director-general: 18 ~ 40 % of basic pay Director: 14 ~ 25 % Senior researcher: 12 ~ 20 %

Performance-based bonus: determined by individual achievement of the year before

Director: 4.25 months + a (average: 0.4 months) a is proportional to individual achievement point