10 best practices research and development portfolio management

8/12/2019 10 Best Practices Research and Development Portfolio Management

1/4

Across industries, organizations vary

widely when it comes to R&D perform-

ance. Though many factors contribute

to R&D performance gaps, organiza-

tions typically ignore or overcomplicate

how they manage their R&D portfolios. A set

of best practices is typically a part of those

organizations that manage their R&D portfo-

lios successfully. Theres no magic here. It

takes commitment, time, and experience.

A systemic approach to R&D portfolio

management enables you to see how the

portfolio is performingthe entire portfolio,

linkages across the portfolio, and drilling

down to the project level.

In considering the ten best practices,

keep in mind these imperatives:

Ask tough questions. Initiate an

internal dialogue. Dont stop asking important

strategic questions that hold significance forthe future of your enterpriseand focus on

assessment criteria that will shed light on

those questions.

Take a systemic view. Put tools in

place that will let you look at the entire

R&D portfolio at any time with new eyes

including links across the whole portfolio

and by discrete areas of activity.

Fully inform all decisions. Have all the data,

information, and graphical characterizations you need

to make effective decisions across the entire portfolio.

Know your organizational culture. Everyorganization has i ts own culture, including possible

barriers to change. Manage within that

culture, and have the right people, tools,

and approaches to overcome impediments.

Best Practices...

1R&D investment decisions are made as part of

a systemic, structured, and carefully defined

approach used across the whole organization.

By using the same approach to manage

the entire R&D portfolio, the organization

can see the portfolio from many

perspectives. Not only can the reviewers

examine how a particular project is

performing against objectives and other

projects, or what R&D linkages exist across

the portfolio, they can also see snapshots

of how the whole portfolio is performing in

relation to important assessment criteria.

This report card for the entire portfolio is

extremely valuableparticularly when theprocess is tailored around criteria that are

considered the most important investment

measures for the organization.

Surprisingly, many organizations (even big

R&D spenders) use only a bottom-up, project-

management approach. They look at each

R&D project through a soda straw and ignore

the important strategic parameters of cross-

portfolio analysis and management. This discrete approach

is appropriate, but only as part of a comprehensive analysis

that includes a top-down systemic view.

From the outset, the organizational team mustunderstand the underlying purpose for using a

Ten Best Practices in

R&D Portfolio ManagementA consistent, systemic approach to managing and assessing a portfoliocan lead to a higher yield on R&D investments.

John A. Parmentola is Director

for Research and Laboratory

Management for the U.S. Army.

John T. Walker is Director of

Science & Technology Manage-

ment for Navigant Consulting.

Executive

TMNovember 2004 www.rdmag.com

WHERE INNOVATION BEGINS A Supplement to R&D Magazine


2/4

"systemic" approach for portfolio management. Then

the process can be structured for the organization to

realize that benefit. For example, one public-sector

organization uses a systemic methodology as a change

management tool. Because the organization was formed

from a number of distinct labs, this process helps the

team develop a common perspective and shared

language to move the set of labs toward one ultimate

goal: to be a preeminent national laboratory.In the commercial sector, users often think of a

systemic approach in terms of efficiency, effectiveness,

new product development, and return on investment.

Other organizations use the approach as a communications

tool. And some use it for all these reasons.

2The people who will use the R&D management system

also design and implement it.

Key stakeholders from across the company help

create the assessment methodology, implement the

process, and participate in investment decision-making.

They also build a common language or taxonomy tofacilitate dialogue.

Even in a small organization, such collaboration is

difficult. But in a large organization, it can be the

source of tremendous frustration. Researchers,

management, and funding authorities all have different

perspectivesparticularly in how they look at the long-

term strategy for a healthy technology pipeline versus

the customers short-term needs. Funding authorities

often believe they are getting too little return on their

investments. Researchers may find themselves on

short leashesunable to do the fundamental research

they believe will strengthen competitive advantages

over the long haul. And the program managers are

stuck in the middle. They all need to be involvedand

dont forget the customers. They have a perspective too!

[Worst Practice: Letting IT people set the rules and build

the approach. IT people tend to think that R&D portfolio

management can be automatedbut that is wrong

thinking. Research is complex and true assessment

requires dialogue.]

3

The R&D approach balances purposes, timeframes, risks,

and rewards.

At one end of the spectrum, where fundamental

research has a long-term horizon, risk is high. Risk is

high at the other end too, as an organization transitions

its new products into a commercial environment. An

effective portfolio methodology lets you see the trade-

off between risk and reward, as well as short-, medium-,

and long-term results. The ability to capture the data

graphically to portray the results across the portfolio in

many dimensions is critical. This allows you to assess

the contributions of each discrete project, of a set of

related projects across different criteria, and of the

entire portfolioby risk and reward, short- and long-term timeframes, and other dimensions.

4Assessments are used to validate resource allocation as

part of the management decision-making process.

Start. Stop. Speed Up. Slow Down. Re-scope.

Organizations continually make decisions about how to

allocate investment dollars across the portfolio. Decisions

made in one year affect the portfolio in the next year,

and so on. But without a critical assessment of the entire

portfolio, organizations are hard-pressed to understand

the effect a complete set of decisions has on the portfolio

and its return on investment (ROI). An annualassessment captures all decision-making and its impact

on the portfolio constitution and its ROI over time.

You can also see how market changes, customer

demands, and company policies have affected the

research emphasis. Moreover, since an organization

continually risks losing institutional memory as key

staff leave or retire, a consistent approach and a common

language let stakeholders view valuable information year

over year, regardless of who is at the table.

Importantly, overall assessment dimensions should

touch upon some reasonable combination of

value/mission/strategy/impact/benefit, feasibility/risk,and cost. Other important criteria can support the

Looking at the effectiveness of an organization's R&D activities by sector

for major industrial companies (top chart) reveals that organizations exhibit a

wide range of R&D performance, with many companies showing clear

opportunities for significant improvement. An organization's R&D portfolio

should have the right balance of overall risk and reward (bottom chart) withcareful attention made to the strategic significance of particular criteria.


3/4

assessmentthough typically, only a few are needed to

answer the critical strategic questions. Additional concepts

include technology maturity, competitive impact,

innovation, and uniqueness of the activity.

[Worst Practice: Spending a lot of money and working

hard to build a systemic approachonly to slip back into

ad hoc decision-making.]

5Power is not in the criteria or data format, but in how

you use the tool to drive high-quality discussions and

decision-making.

A systemic approach to R&D planning produces a

lot of datafrom objective and subjective analyses of

discrete projects and the portfolio as a whole. The data

allow the assessors to create hundreds of snapshots

that characterize the portfolio. But neither data nor

snapshots are the complete answer.

The real answer is in the combination of the

questions you are trying to answer, the data required to

answer these questions, and the perspectives shared

around the table by those who have a stake in theoutcome. Be sure to listen to all players, regardless of

function or seniority. Organizational hierarchy can get

in the way of robust conversations.

[Worst Practice: An organization that underestimates

how difficult portfolio analysis really is. Capturing the

data is the easy part. Making sense of the data is the

hard part. Some organizations will move through all the

steps, but won't take the time to understand the data.]

6Though responsive to continuous improvement, the

approach does not change often or fundamentally so

that year-over-year comparisons are difficult.

Its fine to make minor changes to the methodology to

accommodate an evolving organization, changing external

factors, or errors in the way the original methodology was

constructed. But one of the values of applying a consistent

process is to see how the portfolio migrates over time. If

you change the methodology dramatically from year to

year, you cannot compare results. The lesson is simple:

Construct the methodology. Work out all bugs in the first

year. Make only minor changes in subsequent years. Some

organizations find it helpful to first conduct a pilot, correct

the flaws, and then roll it out.

7The systemic approach uses illustrations and graphical

results to help expedite and communicate R&D priorities

throughout.

Pictures are critical. People gain insights more

rapidly from graphics or cross-portfolio snapshots, than

from quantitative analyses or long explanations. Once

the software has completed its calculations, the ability

to generate a series of snapshots to illustrate the

portfolio performance can help answer strategic

questions and educate investment decisions. Pictures

also help the research group communicate withmanagement executives who are removed from the

R&D process, but are focused on return on investment

and other measures. An illustration is also a valuable

communication tool in the R&D funding process.

8Weighting factors are used to recognize the strategic

significance of particular criteria relative to others.

Many criteria are used to assess a portfolio. Each over-

arching dimension (benefit, risk, cost) has supportive

criteria. When building an approach to R&D portfolio

management, start with an understanding of your

organizations goalsand determine which assessment

criteria to use to measure how the organization is doing

against those goals. Remember, all criteria are not created

equal. So use importance factors to weight criteria in

the assessment framework. For example, uniqueness

may be a very important criterion for a company

struggling to define itself in the marketplace.

[Worst Practice: Spending too much time worrying about

the weighting factors. One organization spent so much

time arguing over weighting factors that they couldn't

move forward. If they become a roadblock, move on

without them.]

9A mechanism is in place to allow a rapid roll-up of

informationand also to let reviewers dig deeply into

the portfolio by posing key questions.

A research portfolio is complex. On average, hundreds

of discrete units of activity make up the portfolioand

many horizontal and vertical linkages exist across those

activities. So the portfolio methodology and analysismust enable the investigators to characterize the overall

Six Key Reasons Why R&D Portfolio Management Can Fail

Weve been doing that for years.

Portfolio analysis takes too longand I cant invest the time.

I have a whole suite of tools for doing portfolio analysis.

I can tell you exactly how that project generates value.

The research we do is too diverse to think about it systemically.

Youve got to be kidding; that program is too important to cancel.

Don't Overcomplicate the Process & Analysis

Don't bring too many people to the table.

You only need 10-12 people during the actual portfolio assessment process, not 40.

Don't include too many criteria.

It is burdensome (and overly time-consuming) to include too many criteria in

the portfolio assessment methodology.

Don't spend too much time over-defining the criteria and the scoring guidelines.

However, do make sure that everyone participating agrees that the important

strategic questions have real meaning and significance to the future health and

performance of the enterprise. Minimize wordsmithing. Initially, people will

ascribe their own interpretation to key concepts like "innovation"but along the

learning curve, interpretations are put on the table and shared, and commoninterpretations emerge.

Don't spend months building the process.

Though you need to get the process right from the outset, don't spend too much

time building it. Don't spend too little time either. Do enough to get it right the

first time.


4/4

portfolio across its major dimensions, and create an

efficient way to dig deeply into the portfolio, see the

linkages, and assess how even one unit of activity is

contributing to the portfolio success.

By reviewing a range of graphic displays, one after

the other and in logical combinations, reviewers start to

build an appreciation for the strengths and weaknesses

of the whole portfolio. Patterns emerge. For example,

certain work units in the portfolio may routinely appearin sub-optimal (underperforming) quadrants of the

chartshighlighting those units for analysis. Maybe the

activity is poorly designed, underfunded, or led by an

inexperienced research team requiring guidance. Many

times, underperformers are stars elsewhere. The ability

to dig deep and roll up the analysis is critical to

understanding the productive value of discrete parts and

the entire portfolioand what problem areas to address.

10External input and participation are part of the

assessment process

For fear of losing control of the decision-making process,organizations are often reluctant to involve internal or

external customers in this process. Thats unfortunate

because organizations are biased by nature, and sometimes

too close to the action to effectively gauge the quality of

work under way. If you dont involve the customer or gain

some kind of external perspective, bias can creep into the

system and corrupt the results. Though remember that

customers have a bias too: to get what they need now. So

many companies use a two-step process. First, get the

kinks worked out of the process and the internal

organization up the learning curve to a shared perspective.

Then, invite the customer to participate in the process.

[Worse Practice: Refusing to allow customers into the

practice. For one organization, their thinking was so

biased that they were no longer capable of evaluating

their work. Over the years, they have lost tremendous

market share and credibilitysome of that because of

their "closed" approach.]

One last note

Some organizations overcomplicate the process and

analysis. This is not rocket science. You are putting

together a common language, an integrated assessment

framework, and an experiential understanding of what

constitutes good R&D. You can use the analysis results

to create graphic illustrations that provide a strategic

perspective on the entire portfolio. Over time, this

approach helps strengthen the portfolio's composition

and improve its overall performance-year in, year out.

About the Authors

John T. Walker is a Director of Navigant Consulting's Science

and Technology Management Practice in Boston. His primary

specialty lies in working with government and private sector

companies to incorporate best practices in R&D portfolio

analysis and management. Mr. Walker has extensive

experience in the aerospace, automotive, defense, and

healthcare industries, along with experience in venture

capital, private equity, and corporate investment communities.

Contact: [email protected]; 781.564.9726

John A. Parmentolais Director for Research and

Laboratory Management for the U.S. Army in Washington,

D.C. He has responsibilities for the Armys Basic and

Applied Research Programs of Army Research Labora tor y,

Army Research Insti tute, Corps of Engineers, and

Simulation, Training, and Instrumentation Command. Dr.

Parmentola was previously a science and technology

advisor to the Dept. of Energy.

Contact: [email protected]; 703.601.1521

A graphical representation of the strengths and weaknesses of an R&D port-

folio can quickly reveal where more attention should be paid to specific areas

or projects. The results of these analyses should be used as a guide to

decision making, not as a replacement.

Reprinted from Research & Development, November 2004 by Valeo Intellectual Property, Inc.Copyright Reed Business Information, a division of Reed Elsevier, Inc. All rights reserved.For reorders call Valeo IP 651.582.3800. For subscription information call 630.320.7165.

408701

10 best practices research and development portfolio management

Documents