7 evaluation and measurement · a fuller search for relevant theories using the extensive online...

B830 Making a Difference

Guest Lecture 7

Evaluation andMeasurementPrepared for the Course Team by Pauline Gleadle

Masters Copyright ª 2007 The Open University *SUP995624* 1.1 SUP 99562 4

The B830 Guest Lectures are an optional resource. They are not required reading, although you may find them useful and interesting at an early stage of the course, when you are considering some of the issues associated with your Evidence-Based Initiative (EBI).

The B830 Course Team invited a group of colleagues to provide something for B830 students on an aspect of management particularly relevant to the processes of managing implementation. The invited ‘lecturers’ were given a free hand to present their chosen topic in their own way. Some dealt with particular questions, others gave a general overview; some provided already published material, others produced more personal, exploratory discussions; some provided teaching activities, others wrote essays.

The topics chosen for the seven Guest Lectures were:

l Making decisions

l Management ethics and corporate social responsibility

l Power and politics

l Cross-boundary management

l Leadership

l Managing innovation

l Evaluation and measurement

These are significant as it is a requirement of B830 that you make use of management theories in your EBI, drawn from at least one of these topic areas. You may, therefore, find some of the Guest Lectures helpful as a starting point for your search for suitable pieces of theory.

You are under no obligation, however, to use the Guest Lectures at all, and you certainly must not use them as the source of authoritative ideas and approaches that you have to use in your EBI. Indeed, it will be unacceptable in TMA 02 and TMA 03 merely to draw on and quote from the Guest Lectures.

What you are obliged to do during your work on B830 is use a few substantial pieces of management theory that you have identified, accessed and considered for yourself. The theories you will need to use will depend on the issues you encounter in your EBI, the problems you need to solve and the options you wish to explore. There will be several opportunities early in the course – in Issues and Approaches, in TMA 01, at the Residential/Online School and in TMA 02 – to explore the sorts of theories that you are likely to need for your EBI and to consider ways in which you can search for them.

So beyond the Guest Lectures, the real work of finding theories for the aspect(s) of implementation relevant to your own EBI will probably begin with the links to a selection of professional and academic articles that are provided in the appropriate areas of the B830 website. Hopefully, those links will then prompt you to make

a fuller search for relevant theories using the extensive online databases provided by the Open University Library and also accessible through the B830 website. And, of course, you can also draw on other sources of models, frameworks and theories that are available to you.

Enjoy the Guest Lectures; they may be helpful with some of your exploratory ‘scouting’ for TMA 01. But thereafter, do make sure that you get into the habit of looking for more substantial bits of theory for yourself.

CONTENTS

1 Introduction 7

1.1 Some key definitions 7

1.2 Timing 8

2 Measurement 11

2.1 An example of a proposed project 11

2.2 Assumptions of the Net Present Value model 12

2.3 Strategic investment appraisal 13

2.4 Rationality and investment appraisal 15

3 Performance evaluation 21

3.1 An example of a proposed project to be evaluated 21

3.2 Evaluation explored further 22

4 Measuring and evaluating difference 27

4.1 Problems of definition 27

5 Some routes forward 29

6 Conclusion 31

7 References 33

8 Appendix: Review of the measurement literature 35

Acknowledgement 43

1 INTRODUCTION

1 INTRODUCTION

OU BUSINESS SCHOOL 7

Why are measurement and evaluation undertaken? You may well reply with the common-sense answer of ‘What gets measured, gets done.’ Although this can be a valid view, later in this Guest Lecture I will suggest that there are various problems with this commonsensical answer. I start this lecture by exploring some key issues around performance measurement. (Note that this is a huge topic in its own right, so I have provided an Appendix which reviews performance measurement. You should view the Appendix as a resource in case you want to investigate performance measurement in more depth.) Once I have highlighted some of the key assumptions of measurement, I will explore evaluation. Finally, I will complete this lecture by comparing measurement and evaluation and proposing some routes forward.

During this lecture, you should have an opportunity to:

l consider the relevance and importance of evaluation and measurement issues in a range of contexts

l explore diverse strategies and pathways through which evaluation and measurement may be achieved

l identify some sources of complexity of evaluation and measurement and relate these to your own situation

l draw on a range of theoretical/conceptual perspectives and relate these to your own context

l identify the practical issues, challenges and contradictions that surround specific evaluation and measurement contexts.

SOME KEY DEFINITIONS 1.1

Before we start looking at evaluation and measurement, we need to consider some key definitions.

Performance measurement provides us with the descriptions of the actual outputs of a system, enabling us to compare them with desired outputs. This can include providing a means by which an organisation is able to identify how well it is performing and whether or not it has improved its performance.

Performance management uses this output information to inform decisions about interventions to change the inputs or processes.

Evaluation involves a review of both the output information and the interventions, from the perspective of one or more stakeholders, frequently leading to recommendations for further action.

GUEST LECTURE 7

We can begin by exploring a key issue associated with both evaluation and measurement: timing.

TIMING 1.2

Before During After

Performance data

Actions

Manager’s evaluation*

Figure 7.1 Key timings associated with evaluation and measurement

In Figure 7.1, you can see that evaluation by a manager questions the before/during/after criteria against which evaluation is being made. Performance data is provided relating both to the ‘before’/ prior period and to the ‘during’ period or to the period in question, and sometimes to the period afterwards.

These aspects of timing are important to bear in mind when thinking about evaluation and measurement. Performance data can be gathered, analysed and acted upon during one period in question alone. An example of this practice includes the French Tableau de bord (literally, dashboard or instrument panel in a car or aeroplane). This is a collection of tactical and operating measures individual to each organisation, which tells management on an ongoing basis how the business is doing. It is quite different from the case of capital investment appraisal, where the starting point is the present, designated ‘Time Zero’ (TO), which is the time when we are making our decisions whether or not to invest. Future cash flows extending over a year and beyond are then estimated and converted back to money in today’s terms. As you can appreciate, you need to be quite clear in your own mind about timing in terms of the time periods you are dealing with in evaluation and measurement.

In order to explore measurement, we will now look at a common example: the case of capital investment appraisal. Unlike measurement and evaluation, which can also encompass the medium or shorter term, this is concerned with the long term, as it involves the evaluation of cash flows stretching one year or more into the future. We are exploring capital investment appraisal in


1 INTRODUCTION

some depth, as it is often undertaken with respect to highly visible projects within an organisation. Accordingly, many of the issues that arise – such as the political nature of such practices – are common to other types of evaluation and measurement.

��


GUEST LECTURE 7

10 OU BUSINESS SCHOOL

2 MEASUREMENT

2 MEASUREMENT


AN EXAMPLE OF A PROPOSED

PROJECT 2.1

You may remember that the Net Present Value (NPV) technique evaluates a long-term project in terms of its future cash flows, which are discounted back into present-period cash flows. This discounting of cash flows explains why the technique is also known as ‘discounted cash flow’ or DCF. I refer to the NPV technique in the following example.

Fred Smart is the ambitious manager of an upmarket gym in Paris on the Champs Elysées. He is keen to get the leisure company’s general manager to invest in a new ‘Super Abs’ machine, currently available only to very select establishments in the USA. According to reports, the Super Abs machine gives its users the type of toned abdominal muscles (or ‘abs’) usually possible otherwise only by undergoing the very painful ‘tummy tuck’ operation.

However, the machine is very expensive and Fred is able to establish the following facts about what the investment would entail:

l The machine costs E500,000.

l Fred believes that he can introduce a new premium-rate membership category to the gym, allowing these particular members and nobody else access to the machine. Moreover, installing the machine would also give the club an advantage over its competitors, thereby ensuring continued membership if other clubs opened in the area.

l Based on the extra revenue generated from the new premium-rate membership over a period of five years and the leisure company’s cost of capital of 7 per cent, the Net Present Value (NPV) of the project is E25,000. This positive NPV suggests the machine should be bought.

l However, using other criteria of payback period (PB) and internal rate of return (IRR), the project is not worthwhile.

You will remember from your earlier studies that the cost of capital reflects the required return that all successful projects must earn. Any project earning more than the cost of capital will show a positive NPV and so, according to the theory, should be undertaken as it will increase the wealth of the firm. However, the situation is confusing because of the outcomes calculated for PB and IRR.

Our question is: what should the general manager do? Invest in the machine and charge premium membership rates, or continue as at present?

This is now a good point to explore the assumptions underlying the important measurement technique of NPV.

ASSUMPTIONS OF THE NET

PRESENT VALUE MODEL 2.2

According to Watts (1996, p. 700), the assumptions of NPV are:

l Perfect competition including perfect knowledge, so that if an organisation estimates cash flows for any subsequent period, these can be ‘verified’ in a way similar to past transactions. This therefore assumes away risk and uncertainty.

l Perfect competition ensures that there is never a shortage of funds (in effect, no capital rationing). With perfect knowledge as an additional assumption, lenders can be certain of the cash inflow for repayment of their loans so that obtaining finance is never a problem.

l Given these assumptions, all investors and borrowers will face a single, market-determined interest rate. Accordingly, all a business needs to know is whether the NPV is positive. As Watts notes, if it is, then the proposal is profitable and the project should be undertaken.

ACTIVITY 7.1

What other assumptions can you think of that are not spelled out above, but that have very real implications for investment appraisal?

DISCUSSION

You may have replied that there are often very real issues around the validity of the base data. How reliable are the estimates of cash flows in the Super Abs question? In the real world, managers do not operate with the benefit of perfect knowledge. Instead, as you are likely to have found in the course of your work experience to date, people often display what Schuster (1997) terms an ‘entrepreneurial response’. Accordingly, people subject to performance measurement are often extremely creative. Particular managers such as Fred Smart may have a vested interest in the acceptance of a project and so produce unrealistically optimistic cash-flow estimates in order to justify it. As you may have witnessed yourself, individuals may be keen to claim the credit for sponsoring new prestigious projects, but move to another job before the full impact of such decisions is apparent.

GUEST LECTURE 7


The ceteris paribus assumption

Another more theoretical assumption is rarely commented upon and derives from the NPV model’s origins in economic theory. This is the ceteris paribus (‘all other things remaining equal’) assumption, which in effect assumes that if the investment proposed is not undertaken, then the organisation will continue in existence as before. However, in the real world we cannot simply assume that if we do not invest, the organisation will face no threat or change. Instead, as you saw in Guest Lecture 6, failure to keep up to date with changing markets and technologies can mean that a firm will go out of business. Accountants in particular therefore tend to be blamed for making this implicit assumption and for preventing projects going ahead because ‘the numbers don’t look good enough’.

Because of such concerns, one remedy for this shortcoming is ‘strategic investment appraisal’ (Bromwich and Bhimani, 1991), which we shall examine now.

STRATEGIC INVESTMENT

APPRAISAL 2.3

Under the strategic investment appraisal approach, a ‘high-tech’ investment is evaluated in terms of its contribution to the organisation’s strategies. Table 7.1 sets out an example of a strategic planning matrix.

2 MEASUREMENT


GUEST LECTURE 7

)1991

him

ani,

B

and

Bro

mwich

(fro

m

matrix

planning

strategic

A

.1

7Table

Strategies/

Benefits

Improved

revenu

es

Lower

costs

Higher

reliability

Better

supp

lyrespon

se

Meeting

custom

errequ

irem

ents

ithw

Fit

other

prod

ucts

Enh

anced

image

New

skills

Better

inform

ation

ofRisks

no

tinvesting

of

Cost

investment

fo

Costs

operation

Organisational

plans

Total

Produ

cten

hanc

emen

tX**

z

X*

X**

X**

X*

X*

prod

ucts

New

Risk

redu

ction

X**

Cos

tad

vantag

es

XX

Improv

edorga

nisa

tiona

lstructure

Com

pany

im

pact

wide

Mon

etary

items

which

eIte

ms b

can

expres

sed

mon

etary

in term

s*

Sco

red

Items**

TOTALS

enabling

in

tool

useful

a be

could

it extent,

this

To

investing.

not

of

costs

the

also

plus

issues

’soft

‘

10) so

-called

to bo

th

(1

scale

account

’‘points into

a takes

onNP

V,scored

terms

terms

conventional

monetary context.

monetary

unlike

in appraisal,

ininto

appraisal

converted

expressed

be investment

be in

vestment

can cannot

address

strategic

which

which

to

*Item

s that

**Items

Note

managers


The various benefits are then classified according to those that can be captured in monetary terms. However, there are also two further categories of benefits. The first category includes benefits that can be classified with difficulty in monetary terms. The second are benefits that are not possible to state in monetary terms – for instance, risk reduction and the consequent enhanced organisational image believed to be generated by greater acceptance of the product (Bromwich and Bhimani, 1991).

The final score to be used in appraisal of the project is obtained by weighting the various marks, which depends on the centrality of the strategic objectives to which the project contributes. While this might strike you as a highly subjective method of appraising projects, at least it goes some way to addressing the inherent ceteris paribus mindset we met above. Specifically, it forces management to think about strategy and to relate investment appraisal to this strategy. Strategic investment appraisal could be a useful tool to evaluate the Super Abs machine in terms of how it accords with the leisure company’s overall strategy.

However, to date I have addressed only some of the generally more explicit assumptions of the NPV technique. We will now explore some of the more implicit assumptions of NPV.

RATIONALITY AND INVESTMENT

APPRAISAL 2.4

The two articles summarised in Boxes 7.1 and 7.2 examine NPV at a more fundamental level, that of the model of rationality implicit in DCF models. Models of rationality constitute important implicit assumptions of theoretical models.

BOX 7.1

RATIONALITY AND DECISION MAKING IN CAPITAL BUDGETING

Northcott argues that NPV concepts within capital budgeting came out of the economics literature and so stressed notions of economic rationality. Under such models, decision makers are able to define and order goals, foresee all possible courses of action, anticipate all possible outcomes from these actions and to make a ‘rational’ choice of action which will optimise attainment of the decision maker’s predetermined goals. Northcott quotes Hollis and Nell (1975) who describe ‘rational economic man’ [sic] as follows.

‘He lurks in the assumptions leading an enlightened existence between input and output, stimulus and response ... We do not know what he wants. But we do know that, whatever it is, he will maximize ruthlessly to get it ... As producer he maximises market-share or profit. As consumer he maximises utility by omniscient (or all-knowing) and improbable comparison of, for

2 MEASUREMENT


instance, marginal strawberries and marginal cement ... Unlike the man in the street, he never misses an opening, ignores a price change, overrates the short-run or turns a blind eye to the unquantifiable’ (Hollis and Nell, 1975, pp. 53–4).

Accordingly, Northcott argues that in order to perform ‘sophisticated’ NPV analyses as set out by many textbooks, decision makers must possess some of the attributes of ‘rational economic man’. This means in effect that organisational goals must be known and the single goal of maximisation of organisational wealth must be shared by all decision makers. Moreover, NPV models reflect the assumption that maximisation of organisational wealth can be achieved by means of the wealth maximisation of any one individual investment. According to this view, there is no possibility of dysfunctional interrelationships between individual projects, nor is there any possibility of unforeseen consequences from combining a range of positive NPV investments.

(Source: adapted from Northcott, 1991)

In the next box, I summarise an article that develops some of these ideas of rationality associated with investment appraisal and examines how five very different accountants view these issues.

BOX 7.2

ACADEMIC AND PRACTITIONER RATIONALITY: THE CASE OF INVESTMENT APPRAISAL

Jones and Dugdale note that much of the literature on investment concentrates on the techniques that are used, or should be used, by accountants. Most of this literature sees NPV as theoretically sound and so as superior to other techniques such as payback (PB). Moreover, many academics are too ready to blame practitioners for failing to use recommended appraisal practices. However, Jones and Dugdale argue that, in order to address the issue of a gap between theory and practice, it is necessary to understand the perspectives of practitioners in their organisational context.

Accordingly, they solicit the views of five accountants who occupy very distinct positions in their organisations. These are:

1 a university lecturer in accounting (‘Adam’)

2 a chief accountant based in one small factory (‘Frank’)

3 a senior finance manager located in the head office of a chemical company (‘Simon’)

4 a junior finance manager located in the same head office (‘Judy’)

5 a finance officer in a local authority (‘Len’).

GUEST LECTURE 7


Frank is the only qualified accountant at a small factory which is part of a division of a larger group and he operates as a member of a fairly small management team. Consequently, he jokes that he even has to ‘change the light bulbs in the gents’ toilet’.

Jones and Dugdale go on to examine the views of three of the other accountants. Simon is found to believe that no calculative technique is of great importance. Instead, what matters is the strategic nature of capital decision making and his long business experience of using all the techniques. (He might find Bromwich and Bhimani’s technique of strategic investment appraisal useful, which we looked at earlier.) By way of contrast, Judy, the more junior accountant in Simon’s organisation, who is recently qualified, believes that managers should use NPV, but that requests from senior executives for other information cannot be ignored. Finally Len, who works for a local authority, believes that he is duty-bound to use Best Practice, which he sees as being NPV. However, he is quite clear that decisions such as building a new school are always political. Len uses both PB and NPV to advise the Council, but sees decision making as a political duty of members, rather than something in which he himself should get involved.

(Source: adapted from Jones and Dugdale, 1994)

Jones and Dugdale conclude that there are broadly two types of rationality displayed by the five accountants:

Academic rationality – this places a central importance on accounting because appraisals are seen as determinants of decisions. Accordingly, if it can be demonstrated that proposals are financially worthwhile, preferably by showing a positive NPV, then investment will go ahead.

Practitioner rationality – this sees proposals as originating outside accounting and that they are already filtered before formal appraisal begins. Proposals are not judged on purely financial grounds, but instead according to strategic criteria in business, or political reasons in the case of the local authority.

Accordingly, practitioner perspectives differ from the dominant academic view as regards both the ‘how’ and the ‘why’ of investment appraisal. This perhaps goes a long way towards explaining the theory–practice gap. Most textbooks support the view that NPV is the most ‘sophisticated’ form of investment appraisal and that outcomes of these calculations will determine whether investment goes ahead. However, surveys of practice generally indicate that a variety of measures are used, as in the Jones and Dugdale article, providing some explanation for the theory–practice gap.

2 MEASUREMENT


ACTIVITY 7.2

As Jones and Dugdale argue for the necessity of locating the views of accountants within their organisational context, how would you expect Adam to view the way in which investment appraisal should be undertaken?

Given that Frank works in a factory which is part of a division of a larger group, what might affect the way he thinks about investment appraisal? (You might consider whether the assumption of no capital rationing within NPV theory is likely to hold in Frank’s case.)

DISCUSSION

Adam assumed that financial measures are the sole criterion in investment appraisal and that NPV should be widely used in preference to other methods such as PB. This is perhaps to be expected as Adam operates within the world of academic accounting alone.

As those of you who have worked in a division of a larger group may know, a major factor affecting capital investment in these circumstances is the amount of funding available from Head Office. This applies to Frank. Interestingly, he favours PB because it is easily understood by non-accountants and he remains sceptical of NPV as being over-theoretical. To quote him ( Jones and Dugdale, 1994, p. 9) with respect to NPV:

‘It’s so theoretical that – you know, you work out a cash flow going forward – and then it seems to me, there’s an element of spurious accuracy that it can give when really the important thing is that cash flow statement.’

Jones and Dugdale therefore conclude that investment appraisal cannot be regarded as a neutral technical activity, divorced from its organisational context. This seems like a good place to revisit the view, ‘What gets measured, gets done.’

ACTIVITY 7.3

As a result of reflecting on the Northcott and Jones and Dugdale readings, what problems can you find with the justification of measurement in terms of ‘What gets measured, gets done’?

GUEST LECTURE 7


2 MEASUREMENT

DISCUSSION

The simple ‘What gets measured, gets done’ view ignores various key issues such as:

l Who does the measuring?

l What determines which proposals get so far as to be measured in the first place?

l How is the measurement carried out: what is the rationality underlying the models guiding the accountants/ evaluators/other technicians?

l Who makes the decision to take action following the process of measurement or evaluation?

So, we have now explored some important issues around measurement to do with investment appraisal that you may not have thought about before working through this lecture. However, we need to reflect on one last issue to do with the NPV technique.

ACTIVITY 7.4

What other implicit assumptions of the NPV technique can you think of that we have not covered? (You could think about how, up to now, we have been assuming in effect that we are looking at a firm undertaking investment appraisal.)

DISCUSSION

By thinking of the cost of capital as rate of return that must be paid to the providers of finance, we have been privileging the shareholder/financing perspective. Accordingly, this raises the issue of how to undertake project evaluation when we are taking into account a range of organisational stakeholders. This leads next to our discussion of performance evaluation itself.

��


GUEST LECTURE 7


3 PERFORMANCE EVALUATION



Performance evaluation is used widely – from the relatively circumscribed evaluation of the shopping-centre case set out below to assessing the socio-economic impact of basic or ‘pure’ scientific research on the economy at large. The shopping-centre example discussed next is important in highlighting key distinctions between performance measurement, performance management and performance evaluation.

AN EXAMPLE OF A PROPOSED

PROJECT TO BE EVALUATED 3.1

Managers in a shopping centre in Amsterdam want to evaluate the business impact of providing a childcare facility for children accompanying adult shoppers. Six months after opening this new facility, they asked the shoppers who had left children there during one week in the autumn about their satisfaction with the service. They combined this customer-satisfaction information with financial data about levels of expenditure by shoppers on other facilities at the shopping centre, such as at the main restaurant, and the cost of providing childcare (staff, equipment, insurance, and so on). Fortunately, the shopping centre has an effective performance measurement system in place so that it can make comparisons with data from the previous six months and twelve months. It concluded that providing childcare generates additional revenue that outweighs the costs.

The shopping centre introduced a few changes to make this facility even more appealing to shoppers, such as extending its opening hours. However, a manager who has overheard customers talking in the main restaurant wonders whether enough shoppers know about the childcare facility – indeed, perhaps there is scope to attract new shoppers by more active promotion of this benefit? But what about demand for childcare in school holidays – will the shopping centre be able to cope if it rises dramatically? Is there a risk that some shoppers may be discouraged from visiting the shopping centre if there is a lot of noise from the childcare facility? Answering one set of questions has led to more, which need to be answered before the shopping centre can draw firm conclusions that would have an impact on any large-scale or long-term changes.

Note that in this example of the shopping centre we have evaluation, performance measurement and performance

management all playing a role. Restating the definitions I gave you earlier:

Performance measurement provides us with the descriptions of the actual outputs of a system, enabling us to compare them with desired outputs. This can include providing a means by which an organisation is able to identify how well it is performing and whether or not it has improved its performance. In this case, we are told that the shopping centre has an effective performance measurement system in place.

Performance management uses this output information to inform decisions about interventions to change the inputs or processes.

Performance evaluation involves a review of both the output information and the interventions, from the perspective of one or more stakeholders, frequently leading to recommendations for further action.

EVALUATION EXPLORED FURTHER 3.2

A fuller definition of evaluation is as follows:

The term evaluation refers to the activity of systematically collecting, analyzing and reporting information that can then be used to change attitudes or to improve the operation of a project or program. The word systematic stipulates that the evaluation must be planned.

(Alkin, 1990, p. 81)

According to Guba and Lincoln (1989), what they term ‘fourth-generation’ evaluation is quite explicitly a socio-political process. This recognises that all forms of enquiry are bounded and framed by elements of culture and politics. Guba and Lincoln therefore see evaluation as a joint collaborative process aiming at the production of an agreed outcome as to the project being evaluated. Box 7.3 summarises Guba and Lincoln’s fourth-generation evaluation.

BOX 7.3

SOME PRINCIPLES FOR FOURTH-GENERATION EVALUATION

1 Evaluation creates reality which comes about through joint collaborative action by evaluators and stakeholders. The outcome of evaluation is subject to continuous refinement, revision and, if necessary, replacement.

2 No portion of the evaluation can be considered value free.

3 Evaluation is a specifically local process so that data from other settings cannot be transposed to local settings.

4 The socio-political aspects of evaluation are at least as important to the process as issues of technical adequacy.

GUEST LECTURE 7


5 Evaluation is a teaching/learning process so that evaluators, clients, sponsors and all stakeholders learn from each other.

6 Evaluations must be continuously recycled and updated so that a good evaluation raises more questions than it answers. Additionally, a good evaluation has no ‘natural’ end point.

7 As evaluation is an emergent process, it has unpredictable outcomes.

8 Evaluation is a process for sharing accountability rather than assigning it.

9 Evaluation is a joint process aiming at building a consensus but still requiring the clarification of competing views of reality. As evaluation is a collaborative process, Guba and Lincoln argue that it is an empowering activity for all. Evaluation should simultaneously aid understanding and clarify the nature of needed action. Importantly, Guba and Lincoln argue that evaluation requires that the evaluator engage in face-to-face interactions with stakeholders, with the result that the activity cannot be accomplished at a distance.

10 While the evaluator should be a technician in the sense of being able to carry out the evaluation activities, she or he must always remember their own humanity in the judgemental process.

11 Accordingly, evaluators must possess not only technical expertise but also relevant interpersonal qualities, including patience, humility, openness, adaptability and a sense of humour.

Note that as evaluation is regarded as a continuous process, objectives can change without invalidating the evaluation. This is different from the NPV example we explored earlier as here the objective remains the same throughout: to evaluate the project against the organisation’s cost of capital. Any projects showing a positive NPV should be undertaken – this does not change.

(Source: adapted from Guba and Lincoln, 1989, Table 9.1, pp. 263–4)

ACTIVITY 7.5

Compare and contrast the following aspects of performance measurement as in NPV and evaluation:

l views of reality; that is, the possibility of outcomes being objective and value-free

l role of technique

l nature of rationality assumed in NPV/evaluation approaches

l role of the accountant/evaluator



GUEST LECTURE 7

l qualities needed by the accountant/evaluator

l assumptions as to who are the stakeholders involved in the decision.

DISCUSSION

This is my answer. You may have come up with something different.

Question Traditional NPV Evaluation

1 View of resultant outcomes

Arrived at as a result of objectively defensible technique.

Truth/the best outcome is defined as the best-informed, most sophisticated construction on which there is agreement (see Guba and Lincoln, 1989, p. 84). Evaluation can be carried out on an ex ante basis during the project and on an ex post basis. Ex ante here means before the project is undertaken and ex post means afterwards.

2 Role of technique Paramount The evaluator and the project evaluated are tightly interlinked.

3 Nature of rationality Economics-based rationality

Constructivist: there are multiple socially constructed realities.

4 Role of the accountant/evaluator

Central: accountant as technical expert.

Collaborative with all stakeholders.

5 Qualities needed by the accountant/ evaluator

Technical expertise and professional judgement.

Technical expertise but also important interpersonal qualities such as humility and sense of humour.

6 Assumptions regarding stakeholders

Unitary nature of goals: all organisational stakeholders are united behind the goal of maximising NPV so as to provide an adequate return to the providers of capital to the business.

Recognises that stakeholders have different concerns.



You may be slightly puzzled by the term ‘constructivist’ in Row 3 in the discussion above. According to Paton (2003, 2004), constructivist approaches do not imply that measurement involves ‘arbitrary subjectivism’. Instead, constructivism emphasises the social processes involved, starting with the proposition that the meaning of performance usually differs at different levels and in different domains so that measurement and management should change accordingly. It then becomes important to establish connections between the different levels – between information, for example, given by centrally designed systems and the local operating context. In this connection, Ahrens and Chapman (1998) argue that this does not happen simply by driving down abstracted measures throughout the organisation. Instead, a lot of ‘talking through’ is required among managers at the different levels. (Note that there is also another approach to evaluation: ‘Realistic Evaluation’ [Pawson and Tilley, 2000]. Such an approach queries whether evaluations can be said to be negotiated; instead it highlights the impact of asymmetries of power, which they see as remaining untouched by the vast majority of policy initiatives.)

So now that we have explored one example each of measurement and evaluation, how do we measure and evaluate difference?

��


GUEST LECTURE 7


4 MEASURING AND EVALUATING DIFFERENCE

4 MEASURING AND EVALUATING DIFFERENCE

��


This is a complex issue, as recognised by Kondrasuk in a classic early review article on performance measurement (Kondrasuk, 1981). This article critiqued 185 studies for the effects of ‘management by objectives’ (MBO) on employee productivity and/or job satisfaction. As some of you may remember, MBO was a management technique in the 1970s and 1980s, which continues to influence managers today.

PROBLEMS OF DEFINITION 4.1

Kondrasuk concluded that problems included issues around the questionable nature of MBO effectiveness given problems of a common definition. Additionally, and particularly importantly, as you will recall from Figure 7.1, timing is important; Kondrasuk raised concerns around the temporal duration of MBO. This was because experts in the area usually claimed that it took two to five years to fully implement an MBO programme in an organisation. However, there is other evidence that in time, these positive effects of MBO evaporate. Accordingly, when trying to evaluate MBO in the real world, timings become confused. Finally, Kondrasuk argued that it was difficult to evaluate the nature of success when outcomes are mixed; for example, sales may go up, but profits might go down, as may job satisfaction.

More recent studies, such as Otley (1999), conclude that there has been little systematic evaluation to date of the totality of organisational performance measurement systems, with studies instead tending to concentrate on discrete elements. Ittner and Larcker (1998) also comment that despite increasing adoption of such approaches as improved financial metrics, economic value and Balanced Scorecard of integrated financial and non-financial measures, relatively few studies have examined these approaches in any detail. Accordingly, there is little evaluation of the new measures in terms of their economic relevance, the implementation issues arising from their adoption or the performance consequences from their use.

This is frankly rather depressing reading for individuals wanting to make a difference through using measurement in particular. However, there are some possible routes forward.

GUEST LECTURE 7


5 SOME ROUTES FORWARD

5 SOME ROUTES FORWARD

There is some practitioner-based literature on ‘what top firms do’ in terms of performance measurement. This tends to be largely anecdotal, although Lingle and Schiemann (1996) do report on a more comprehensive (US) national survey of senior executives. However, hopefully you will agree, after having worked through this section, that remedies cannot be simply imported from other organisations and applied blanket-fashion to your own employer. This is because it is naïve to assume that one organisation’s remedies can be imported to another without taking into account differences in context. Moreover, it is also worth bearing in mind that Chakravarthy (1986) comments that no single measure of profitability is capable of discriminating ‘excellence’. As I noted earlier, individuals tend to be ‘entrepreneurial’ in the face of performance measurement so that many authorities on the subject advocate introducing a range of relevant measures rather than relying on one measure alone.

These measures can include non-financial performance measures. Indeed, Ittner and Larcker (2003) advocate such an approach and recommend that managers should first develop a model of key cause-and-effect relationships affecting the organisation in its environment. The next step should be to undertake a careful inventory of all their databases and then to reassess the results of their causal modelling. Reasons for actually preferring non-financial performance measures to accounting data include that non-financial data is often regarded as more difficult to manipulate than profit-based figures. Moreover, it is also argued that non-financial data is more immediate and so not distorted by accounting conventions.

In the case of non-profit and public-sector organisations, measurement and evaluation issues can be even more complicated than in the private sector because it can be tricky reconciling the needs of quite diverse stakeholders. However, Paton (2004) sees Kanter and Summers (1987) as relating the constructivist idea of levels to this sector in a useful fashion. They see the levels as follows:

l the institutional level concerned with legitimacy in the eyes of major external stakeholders

l the managerial level concerned with resource use

l the technical or professional level concerned with service quality and outcomes.


GUEST LECTURE 7

Paton (2004) suggests that although these conceptions of performance will be related, they cannot be aggregated and disaggregated in a simplistic fashion. Instead, it has to be expected that performance measurement will be characterised by plurality, contingency and debate. Paton argues that this is an inevitable outcome unless one or other group at the various levels dominates the others.

We have covered a lot of ground as regards both evaluation and measurement. Where does this leave us?

��


6 CONCLUSION

6 CONCLUSION

At different points in this lecture, we have explored the expression ‘What gets measured, gets done’. This expression suggests that measurement does make a difference in that if you want to change something, you need data in the first place, so you therefore need to measure whatever you want to change. You will remember that in Figure 7.1, I set out time periods affected by evaluation and measurement and concluded that these could vary substantially according to different situations. Accordingly, this can result in what actually gets measured being not always what ostensibly gets measured. However, a very simplistic interpretation of the saying might view performance measurement techniques and evaluation approaches as being universally applicable across different contexts. Instead, I have suggested a much more contingent view that involves carefully exploring issues of organisational context and of the nature of the stakeholders involved. I suggest that in order to be able to make a difference using evaluation and measurement, you have to reflect quite carefully on all the issues we have explored in this lecture. Cause and effect are difficult to untangle, particularly in evaluation, and ‘good’ performance in any case is multidimensional. I hope that at the end of this lecture you feel confident about the following:

l exploring and assessing the impact of both explicit and implicit assumptions underlying the measurement models

l understanding the effect of any practical issues such as the adequacy of the base data used in performance measurement and evaluation

l considering carefully how you are going to operationalise or render measurable favourable performance for evaluation or measurement purposes

l and finally, possibly feel inspired at the end of this lecture to explore the topic further.

��


GUEST LECTURE 7


7 REFERENCES

7 REFERENCES

Ahrens, T. and Chapman, C. (1998) ‘Sustaining antagonistic harmony-food margin control in a UK restaurant chain’, Performance Measurement – Theory and Practice; The First International Conference on Performance Measurement, Cambridge, Centre for Business Performance, Cambridge University.

Alkin, M.C. (1990) Debates on evaluation, London, Sage.

Bromwich, M. and Bhimani, A. (1991) ‘Strategic investment appraisal’, Management Accounting, 72 (9), pp. 45–8.

Chakravarthy, B.S. (1986) ‘Measuring strategic performance’, Strategic Management Journal, 7, pp. 437–58.

Guba, E.G. and Lincoln, Y.S. (1989) Fourth Generation Evaluation, Newbury Park, CA, Sage.

Hollis, M. and Nell, E.J. (1975) Rational Economic Man: a philosophical critique of neo-classical economics, London, Cambridge University Press.

Ittner, C.D. and Larcker, D.F. (1998) ‘Innovation in performance measurement: trends and research implications’, Journal of Management Accounting Research, 10, pp. 205–38.

Ittner, C.D. and Larcker, D.F. (2003) ‘Coming up short on non-financial performance measurement’, Harvard Business Review, 81 (11), pp. 88–95.

Jones, T.C. and Dugdale, D. (1994) ‘Academic and practitioner rationality: the case of investment appraisal’, British Accounting Review, 26 (1), pp. 3–25.

Kanter, R.M. and Summers, D.V. (1987) ‘Doing well by doing good: dilemmas of performance measurement in non-profit organisations, and the need for a multiple contingency approach’ in W.W. Powell (ed.) The Non-profit Sector: a research handbook, New Haven, CT, Yale University Press.

Kondrasuk, J.K. (1981) ‘Studies in MBO effectiveness’, Academy of Management Review, 6 (3), pp. 419–30.

Lingle, J.H. and Schiemann, W.A. (1996) ‘From Balanced Scorecard to strategic gauges: is measurement worth it?’, American Management Association Management Review, March, pp. 56–61.

Northcott, D. (1991) ‘Rationality and decision making in capital budgeting’, Journal of Business Finance and Accounting, 18 (4).


GUEST LECTURE 7

Otley, D. (1999) ‘Performance management: a framework for management control systems research’, Management Accounting Research, 10, pp. 363–82.

Paton, R. (2003) Managing and Measuring Social Enterprises, London, Thousand Oaks, CA, and New Delhi, Sage.

Paton, R. (2004) ‘“Constructive constructivism” – a new perspective on the design and use of performance measures’, unpublished working paper, Milton Keynes, The Open University.

Pawson, R. and Tilley, N. (2000) Realistic Evaluation, London, Sage.

Schuster, J.M. (1997) ‘The performance of performance indicators in the arts’, Nonprofit Management and Leadership, 7 (3), pp. 253–69.

Watts, J. (1996) Accounting in the Business Environment, 2nd edn, London, Pitman.

��


8 APPENDIX: REVIEW OF THE M EASUREMENT LITERATURE

8 APPENDIX: REVIEW OF THE MEASUREMENT LITERATURE

Introduction 35

A brief recent history of measurement 36

Some non-operationally based criticismsof performance measurement 38

Measuring and evaluating difference 39

Conclusion 40

References 40

INTRODUCTION

Performance measurement is a wide-ranging inter-disciplinary field. This is borne out by my (Pauline Gleadle) experience when I did a search in early January 2002 of one electronic database on performance measurement for the period 1999 to date. The result was that I found 528 references existed in this field on that particular database alone. The current summary review will concentrate on financial aspects, although non-financial measurement issues in this area will also be raised.

There appears to be general agreement that there has been an increase in performance measurement during the period from approximately the late 1980s to date across a wide variety of organisations (Ittner and Larcker, 1998; Bourne, 2000; Heery and Salmon, 2000; Otley, 2003). Accounting often plays a central role in evaluation and measurement so that it is perhaps helpful to consider what Burchell et al. (1980), in a widely cited paper, have to say about the role of finance. They note that accounting developments are being increasingly associated with the creation of particular patterns of organisational visibility so that what is accounted for can shape organisational participants’ views of what is important. Given that accounting frameworks and their sometimes only implicit assumptions thereby help create a particular conception of economic reality, finance can no longer be seen as a mere assembly of calculative routines. Instead, Burchell et al. argue that accounting now functions as a cohesive and influential mechanism for economic and social management. Accordingly, the Critical Accounting school sees accounting as almost always necessarily saturated with significant issues around power, another theme of B830.


GUEST LECTURE 7

A BRIEF RECENT HISTORY

OF MEASUREMENT

Ittner and Larcker (2001) follow the International Federation of Accountants (1998) in tracking measurement and evaluation from around 1950 to date. Before 1950, they see the primary focus of managerial accounting practice as having been on cost determination and financial control through the use of budgeting and cost accounting systems. By the mid-1960s, the focus had shifted to the provision of information for management planning and control. Otley (2003) cites the main intellectual foundation for these developments in the 1960s as Anthony (1965), in which he separated management control from both operational control and strategic planning. Additionally, around this time Hofstede’s The Game of Budget Control (1967) was published, a study of how budgetary systems distorted economically rational behaviour. As such, it followed Argyris’ (1952) The Impact of Budgets on People and anticipated Schiff and Lewin’s (1970) ‘The impact of people on budgets’.

This body of work was largely universalistic in nature in that it sought the one best way of designing control systems that would lead to desired results. It was only in the 1970s that contingency approaches began to appear, although these had been used in organisational theory since the mid-1950s. Prominent contingent factors included external environment (simple versus complex; static versus dynamic), technology (production interdependencies, automation), competitive strategy and mission (e.g. low cost versus innovation), business unit and industry characteristics (e.g. size, diversification, firm structure, regulation) and knowledge and observability factors (e.g. knowledge of the transformation process; outcome observability; behaviour observability). Systems theory also emerged around the 1970s. This emphasised the connection of local systems of control with the wider environment and had been applied earlier to organisational theory in terms of the ‘open systems’ movement.

In the 1980s there were two major developments in the field. First, critical approaches became prominent. Second, from the mid-1980s there was a new emphasis on the reduction of waste occasioned by adoption of TQM as well as cost of quality measurement, Activity-Based Costing and strategic cost management.

By the mid-1990s the emphasis had become more strategic, with major interest centred on the creation of value for the firm through the identification, measurement and management of the drivers of customer value, organisational innovation and shareholder returns. These approaches included the Balanced Scorecard, economic value measures such as Economic Value Added1 and strategic management accounting systems providing information about current and expected states of strategic uncertainties (e.g. Simons, 1995). Accordingly, in many of such approaches financial controls



have been complemented by a range of non-financial performance measures, as in the Balanced Scorecard.

Additionally, there has been growing interest in corporate governance and the external control of organisations, generally focused around the theme of ensuring that senior managers work in the interests of shareholders and so deliver appropriate value to the owners of a business. In Europe, the emphasis is usually on trying to use monitoring arrangements outside the influence of senior managers, while in the USA more attention is paid to performance measures such as Economic Value Added1. Less attention is paid to budgeting, both because of the rise of these alternative controls and because of the high levels of unpredictability and uncertainty in many business environments. Simons (1995), with his emphasis on the role of boundary, belief and interactive control systems as well as the more traditional diagnostic control, is an important example of such more holistic frameworks. What is particularly interesting about Simons’ approach is not only his wide-ranging approach, which assigns an important role to ‘soft’ controls such as belief systems, but also his distinction between the actual control information and the use made of this. Accordingly, profit-planning information in his view can be used diagnostically, as is traditionally the case. However, in more innovation-centred firms, the very same type of information can be used interactively by senior management in order to probe the environment for emerging threats and opportunities. The focus therefore shifts from the actual performance measures to the use of such measures in practice.

Given perceived substantial shifts in the business environment since the heyday of nationally based manufacturing in the West in the early twentieth century, there have been further developments in performance measurement and evaluation. One major shift concerns the trend away from manufacturing and to increased employment in the service sector, thereby rendering redundant much of traditional management accounting (Johnson and Kaplan’s 1987 Relevance Lost critique). Reflecting this shift, Fitzgerald and Moon (1996) have studied performance measurement in the service sector, covering such firms as TNT, a logistics company, and the Peugeot dealership network. Another major development concerns the advent of the so-called ‘new economy’ characterised by hyper-competition, significant uncertainty and a general shortening of product life cycles. Nixon, an authority on accounting and New Product Development, and colleagues (Nixon et al., 2002) argue that these shifts necessitate altogether new types of performance measurement and control. Whereas the ‘old’ environment had benefited from detailed, structured tracking of performance targets, under the new economy this has to change. What is called for now is a combination of controls linked to strategy formulation and risk-management processes with a new emphasis on the cost and value of options and on metrics to support rapid action and flexible responses (Nixon et al., 2002). The advent of the new economy has triggered practitioner interest in so-called intangibles


GUEST LECTURE 7

such as brands and knowledge, distinguished from plant and machinery as examples of tangible assets. Intangibles (e.g. within the resource-based-view literature) are argued to be central to the creation of sustainable competitive advantage so that current performance measurement debates include consideration of the valuing and management of intangibles (e.g. see Power, 2001, for an overview).

A final, major, recent feature of organisational life includes internationalisation and globalisation. This development is reflected in the literature by such studies of international management control as Hofstede’s (1980) seminal work and the large body of research engendered, which is both friendly and hostile to this approach. Critics of Hofstede contend, among other issues, that national culture cannot be reduced to a set of so-called simplistic dimensions and, moreover, that these dimensions themselves are culturally biased. However, note that this literature on international accounting and management control tends to highlight the central role of accounting in the life of UK organisations. This contrasts strongly with the situation in other countries where accounting tends to be much more marginalised. Other work in the area of international management control and performance measurement includes textbooks such as that by Gray et al. (2001), who address issues of control in multinational enterprises (MNEs), including from a strategy-based literature perspective.

As well as these largely accounting- and control-based approaches to performance management and evaluation, there has been increased interest in the role of non-financial indicators. These are regarded by some (e.g. as noted in Ittner and Larcker, 1998) as inherently superior to traditional accounting data, as they are often less amenable to manipulation by management and more connected to business realities than are historically based and financial reporting-oriented measures. However, as well as unitarist (explanations adopting a managerialist perspective) criticisms of performance measurement in general, there is also a literature addressing the employee perspective, which is discussed next.

SOME NON-OPERATIONALLY BASED

CRITICISMS OF PERFORMANCE

MEASUREMENT

Heery and Salmon (2000) write of the coming about of an insecure workforce in the UK as forces from the outside market are brought inside the organisation, thereby causing significant changes in the psychological contract. Against the background of widespread downsizing, a major feature of this changed psychological contract includes increased measurement of employee performance, leading to an increase in perceived job insecurity. Moreover, other more critical work, such as that of Deetz (1995) and Alvesson and Willmott (2002), locates such developments in terms of senior management within a wide variety of organisations seeking to



regulate more than just the work behaviour of employees. Instead, it is argued that senior management may seek to ‘manage the insides’ (Deetz, 1995) of employees in terms of their hopes, fears and emotions, a development that is argued to go far beyond earlier approaches of normative control. While such theories might be regarded as unduly alarmist, Longenecker and Ludwig (1990) argue that there is indeed an ethical dimension to performance measurement and evaluation, something that is often ignored in the more mainstream performance-measurement literature.

However, while the current review performed for B830 has covered a wide range of developments within the field, it has not addressed the issue of measuring and evaluating difference as a result of implementing performance measures of whatever nature.

MEASURING AND EVALUATING

DIFFERENCE

You will remember that Kondrasuk (1981) concluded that there were many problems in evaluating the effectiveness or otherwise of MBO. While Kondrasuk might be regarded as somewhat dated, it is considered a seminal article by authorities in the area of performance measurement. Moreover, you will recall from Section 4 that more recent studies, such as that by Otley (1999), conclude that there has been little systematic evaluation to date on the totality of organisational performance measurement systems, with studies instead tending to concentrate on discrete elements. Accordingly, Otley proposes a five-question framework to evaluate the entirety of organisational control systems, a call that is beginning to lead to the adoption of such more holistic research approaches. Ittner and Larcker (1998) also comment on the scarcity of research examining in any detail such approaches as Economic Value Added1 or the BSC. Accordingly, there is little evaluation of the new measures in terms of their economic relevance, the implementation issues arising from their adoption or the performance consequences from their use.

However, there is some more practitioner-based literature on ‘what top firms do’ in terms of performance measurement. This tends to be largely anecdotal, although Lingle and Schiemann (1996) do report on a more comprehensive (US) national survey of senior executives. In the survey, measurement was found to play a crucial role in translating business strategy into results and certain findings emerged, such as the need to have agreed upon measures that managers understand and which balance financial and non-financial measurement. However, crucially it was also found that not many executives responding to the survey would stake their jobs on the information that is available to them! This then raises another major issue around performance measurement, namely that of the validity or otherwise of the base data. Finally, it is worth noting that with regard to performance measurement techniques, Chakravarthy


GUEST LECTURE 7

(1986) comments that no single measure of profitability is capable of discriminating ‘excellence’.

In conclusion to this section, a final disparate body of literature comprises generally positivist studies exploring the impact of maybe one variable, such as employee commitment, on organisational performance. Such approaches are beset by very real problems associated with measurement and with over-simplifying complex situations. However, these studies are interesting in their own right and students may like to follow up this line of enquiry (e.g. see Lau et al., 1995; Chong, 1996; Benkoff, 1997; Mia and Clarke, 1999; Scott and Tiessen, 1999).

CONCLUSION

Performance measurement is a truly enormous area which is difficult to summarise adequately. However, hopefully this Appendix will act as a useful resource for those wanting to explore these issues further.

REFERENCES

Alvesson, M. and Willmott, H. (2002) ‘Identity regulation as organisational control: producing the appropriate individual’, Journal of Management Studies, 39 (5), pp. 619–44.

Anthony, R.A. (1965) Planning and Control Systems: a framework for analysis, Boston, MA, Division of Research, Harvard Graduate School of Business.

Argyris, C. (1952) The Impact of Budgets on People, Ithaca, NY, The Controllership Foundation.

Benkoff, B. (1997) ‘Ignoring commitment is costly: new approaches to establish the missing link between commitment and performance’, Human Relations, 50 (6), pp. 701–26.

Bourne, M.C.S. (2000) ‘Success and failure of performance measurement system design interventions,’ PhD presented to the University of Cambridge.

Bromwich, M. and Bhimani, A. (1991) ‘Strategic investment appraisal’, Management Accounting, March, pp. 45–8.

Burchell, S., Clubb, C., Hopwood, A., Hughes, J. and Nahapiet, J. (1980) ‘The roles of accounting in organisations and society’, Accounting, Organizations and Society, 5 (1), pp. 5–27.

Chakravarthy, B.S. (1986) ‘Measuring strategic performance’, Strategic Management Journal, 7, pp. 437–58.

Chong, V.K. (1996) ‘Management accounting systems, task uncertainty and managerial performance: a research note’, Accounting, Organizations and Society, 21 (5), pp. 415–21.



Deetz, S. (1995) Transforming Communication, Transforming Business: building responsive and responsible workplaces, Cresskill, NJ, Hampton Press.

Fitzgerald, L. and Moon, P. (1996) Performance Measurement in Service Industries: making it work, Chartered Institute of Management Accountants (CIMA).

Gray, S.J., Salter, S.B. and Radebaugh, L.H. (2001) Global Accounting and Control: a managerial emphasis, New York, Wiley.

Heery, E. and Salmon, J. (ed.) (2000) The Insecure Workforce, London, Routledge.

Hofstede, G. (1967) The Game of Budget Control, Assen, Netherlands, Koninklijke Van Grocum.

Hofstede, G. (1980) Culture’s Consequences: international differences in work-related values, Beverly Hills, CA, Sage.

International Federation of Accountants (1978) International Management Accounting Practice Statement: management accounting concepts, New York, International Federation of Accountants.

Ittner, C.D. and Larcker, D.F. (1998) ‘Innovation in performance measurement: trends and research implications’, Journal of Management Accounting Research, 10, pp. 205–38.

Ittner, C.D. and Larcker, D.F. (2001) ‘Assessing empirical research in managerial accounting: a value-based management perspective’, Journal of Accounting and Economics, 32, pp. 349–410.

Johnson, H.T. and Kaplan, R. (1987) Relevance Lost: the rise and fall of management accounting, Boston, MA, Harvard Business School Press.

Kondrasuk, J.K. (1981) ‘Studies in MBO effectiveness’, Academy of Management Review, 6 (3), pp. 419–30.

Lau, C.M., Low, L.C. and Eggleton, I.R.C. (1995) ‘The impact of reliance on accounting performance measures on job-related tension and managerial performance: additional evidence,’ Accounting, Organizations and Society, 20 (5), pp. 359–81.

Lingle, J.H. and Schiemann, W.A. (1996) ‘From Balanced Scorecard to strategic gauges: is measurement worth it?’, American Management Association Management Review, March, pp. 56–61.

Longenecker, C. and Ludwig, D. (1990) ‘Ethical dilemmas in performance appraisal revisited’, Journal of Business Ethics, 9, pp. 961–9.

Mia, L. and Clarke, B. (1999) ‘Market competition, management accounting systems and business unit performance’, Management Accounting Research, 10, pp. 137–18.

Nixon, W.A., Laitinen, E.K. and Wingren, T. (2002) ‘Implications of the ‘New Economy’ for control,’ Paper given to the 3rd Conference


GUEST LECTURE 7

on New Directions in Management Accounting: Innovations in Practice and Research, Brussels, Belgium, 12–14 December.

Otley, D. (1999) ‘Performance management: a framework for management control systems research’, Management Accounting Research, 10, pp. 363–82.

Otley, D. (2003) ‘Management control and performance management: whence and whither?’ The British Accounting Review, 35 (4), pp. 309–26.

Power, M. (2001) ‘Imagining, measuring and managing intangibles’, Accounting, Organizations and Society, 26, pp. 7–8, 691–3.

Schiff, M. and Lewin, A.Y. (1970) ‘The impact of people on budgets’, The Accounting Review, pp. 45, 259–68.

Scott, T.W. and Tiessen, P. (1999) ‘Performance measurement and managerial teams’, Accounting, Organizations and Society, 24, pp. 263–85.

Simons, R. (1995) ‘Control in an age of empowerment’, Harvard Business Review, 73 (2), pp. 80–8.

Two other useful articles:

Atkinson, A.A., Waterhouse, J.H. and Wells, R.B. (1997)‘A stakeholder approach to strategic performance measurement’,Sloan Management Review, 38 (3), pp. 25–37.

Ittner, C.D. and Larcker, D.F. (1997) ‘Quality strategy, strategiccontrol systems and organizational performance’, Accounting,Organizations and Society, 22, pp. 3–4, 293–314.


ACKNOWLEDGEMENT

ACKNOWLEDGEMENT

Grateful acknowledgement is made to the following source for permission to reproduce material within this product.

Table

Table 7.1: Bromwich, M. and Bhimani, A. ‘Strategic investment appraisal’, Management Accounting, March, 72 (9), pp. 45–8.

Every effort has been made to contact copyright holders. If any have been inadvertently overlooked the publishers will be pleased to make the necessary arrangements at the first opportunity.


The OU Business School is accredited by

AACSB, EQUIS, CEL and AMBA

oubs.open.ac.uk The Open University

Printed in the United Kingdom

7 evaluation and measurement · a fuller search for relevant theories using the extensive online...

Documents