Measuring IT InnovationBenefits

Evaluation of IT innovation

Why measure IT benefits?

• A new IS/IT system is an investment; it must be strategically/financially justified.

• To evaluate an already existing system: Were the targets achieved?

• To enable comparison of different proposals• To learn and create tools for managing IT projects• Are we being conned by IT salespeople?

• Investments in IT are difficult to evaluate

• Both costs and benefits are difficult to quantify; intangible factors are likely to be significant.

• It is difficult to follow (and plan for) the extremely rapid developments in IT.

Measuring IT benefits

?$$$$$

An IT assessment should answer three main questions:

1. How does IT improve business performance?

2. How to decide the IT projects in which to invest?

3. How to assess the performance of systems after implementation?

IT project costs (examples)HardwareSoftwareProgrammingProblem analysisProject

managementTraining

Operation‘Hidden’ costs,

such as users helping other users in the period after implementation, or troubleshooting

IT project costs

“Companies spend ten times more money making a software work

than on the software itself”

Joe Levy

President Quadragon Group

Comdex 2002

Tangible benefits

Tangible benefits can be measured directly. Tangible benefits of IT/IS are improved business processes (as we have discussed throughout the course). These translate directly into reduced cost, - and indirectly into more business.

Intangible benefits

• Better information and decision-making

• Better coordination and supervision

• Improved morale

• Ability to evaluate more alternative

• Organisational learning

Intangible benefits are those that cannot be measured in monetary terms:

To consider...

IT is very often only one part of a larger BPR which may include changes in organisation, IT/IS, logistics, etc.

With falling prices of IT hardware and software, and similarity between products, IT can often be considered a commodity; - in some cases staff training may cost more than the implemented system

Traditional measurement methods

• Return on investment (ROI)• Net present value (NPV)• Cost-benefit• ‘Informal’ methods

Traditional project appraisal methods evaluate financial ratios only (monetary value). Intangible benefits are ignored or converted to a monetary value.

$$ $$

• There are at least 30 current available evaluation methods for IT benefits (Andreasen 1999). They can be split into:

• Objective methods, that quantify inputs and output

• Subjective methods, relying on attitudes and opinions of users and system builders

Alternative measurement methods

Alternative measurement methods

• Information Economics (IE)

• Return on Management (ROM)

• SESAME

• ... etc.

These methods are rarely used in construction, due to little awareness of these methods and complexity of the methods

In construction...

• Research show that there is some awareness of alternative IT evaluation methods, but very few companies use them.

• A general lack of evaluation methodologies with focus on IT, many rely on ‘gut feel’.

• IT evaluation techniques are considered more costly than the value they generate.

Pittard 1995

Awareness of IT appraisal techniques used in sample of UK construction companies

Andresen 1999

Procedures used in sample of Danish construction firms

Measuring IT benefits

There is a need for construction-specific tool/framework that enables all parties to communicate and exchange information on possible IT investments in a form that is readily recognisable within the culture of the construction sector.

IT benefits measurement

output

inputRate

One approach measures value in distinct areas in which IT innovation can benefit a business:Efficiency:

Effectiveness: Output / planned output

Performance: Level of new outputs enabled, i.e. ~ the strategic impact of IT

IT benefits measurement

• Efficiency money• Effectiveness a score/index of improved

operations• Performance qualitative evaluation of

improved business performance (long term)

The framework will measure value, but not only in quantifiable terms:

IT benefits measurement process

IT benefits measurement

Construction business Construction business processprocess

Efficiency Efficiency benefitsbenefits

Effectiveness Effectiveness benefitsbenefits

Performance Performance benefitsbenefits

•Business planning•Marketing•Information management•Procurement•Finance•Client management•Design•Construction•Operation &

Maintenance•Human resources

Example of typical benefits

ProcessProcess EfficiencyEfficiency EffectivenessEffectiveness •PerformancePerformance

Construction •Reduced construction times

•Improved productivity

•Reduced waste

•Improved quality of output

•Reduced technology risks

•Ability to exchange data

•Improved idea-sharing among project teams

•Improved integration

•Improved project relationships with strategic partners

IT benefits measurement

• The method makes use of sets of tables that must be filled out before and after an IT project.

• Full paper available at:

http://www.itcon.org/2000/4/

‘Conclusion’

1. Decide key targets; monetary as well as non-monetary performance indicators

2. Assign a manager responsible for evaluating benefits (a champion)

3. Evaluate results against targets

4. Evaluate the measurement method and feedback

A simplified procedure for any IT innovation benefit measurement may be concluded as: