benefit realization from public projects full paper

BAM2013 This paper is from the BAM2013 Conference Proceedings

About BAM

The British Academy of Management (BAM) is a learned society dedicated to developing the

community of management academics. To find out more about BAM, please visit our website at

http://www.bam.ac.uk/

1

Benefit Realisation from Public Projects: A Theoretical Framework for the Quality of Target Benefits

Ying-Yi Chih, Ofer Zwikael

The Australian National University, Canberra, ACT, Australia

Contact details:

Dr Ying-Yi Chih

Research School of Management

ANU College of Business and Economics

The Australian National University

Canberra, ACT 0200

Australia

T: + 61 2 61257919

Email: [email protected]

Dr Ofer Zwikael

Research School of Management

ANU College of Business and Economics

The Australian National University

Canberra, ACT 0200

Australia

T: + 61 2 61256739

Email: [email protected]

Track: Public Management and Governance (Track Chair: Dimitrios Spyridonidis)

2


Abstract

Benefit realisation from public projects is strongly associated with the efficiency of public sector management. Along with the trends moving toward the Public Value Management, the literature has placed increasing emphasis on benefit realisation from public projects. The process of defining target benefits from public projects was considered as a critical but difficult step. Unfortunately, issues related to this subject remain unexplored in literature.

This article addresses this gap by presenting a theoretical framework for the quality of target benefits of public projects through a literature review and interviews with senior managers in Australian Government. Specifically, we introduce a construct of ‘quality of target benefits’ and suggest a set of criteria for its assessment. We also identify four antecedents and moderating variables of this construct and develop a series of propositions for further empirical studies. Lastly, we discuss the implications for public sector managers and directions for future research.

Key Words: Target Benefits; Public Projects; Benefit Realisation

Word Count: Abstract (149); Body of paper: 6,685 (excluding tables and references)

3


1. Introduction

Public sector management efficiency is strongly associated with local, regional and national development (e.g., Rhys & Boyne 2010). Public management activities, such as the improvement of public services, the implementation of new policies and the introduction of new ‘e-government’ systems, are conducted through projects and programmes (Dunleavy, Margetts, Bastow & Tinkler 2006). The performance of these public projects and, in particular, failure in delivering their target benefits has a negative effect on national growth, not to mention the waste of public resources and tax payer money.

Benefits are the flows of value that arise from a project (Zwikael & Smyrk 2012). As projects normally involve a high level of risk due to their innovative nature (Damanpour 2010) and need to balance various stakeholder perspectives (Devinney, Yip & Johnson 2010; Bryde 2005), most of them fail to achieve their target benefits (Shenhar & Dvir 2007). Public management literature shows us that public projects worldwide are no exceptions to this widely prevalent phenomenon (Kwak & Smith, 2009). For example, in the UK, it was found that ‘deficiencies in benefits capture bedevils nearly 50% of government projects’ (UK Office of Government Commerce (OGC) 2003) and ‘30–40% of systems to support business change deliver no benefits whatsoever’ (OGC 2005). Similarly, in Europe, it was found that ‘after at least a decade of large investments (running into billions of Euro) at digitalizing the public sector, governments in Europe are still mostly unable to objectively quantify and show the benefits and returns of such investments’ (eGEP 2006). In Australia, a nation-wide review on the management of public projects revealed that ‘agency governance mechanisms are weak in [the]…understanding of organisational capability to commission, manage and realise benefits from ICT-enabled projects’ (Gershon 2008, p.2). Such evidence indicates a pressing need in the public sector to ensure that benefits are realised from public projects.

The importance of realising benefits from projects is also recognised in the literature (e.g., Love & Irani 2004; Lin & Pervan 2003), which specifically emphasises the identification and definition of target benefits as a first and critical step towards their realisation (Ward, Taylor & Bond 1996). Target benefits are those that are formally anticipated by the funding public agency, and are set when projects are approved (Zwikael & Smyrk 2012). Better-defined target benefits will not only improve the quality of a decision regarding whether to approve a proposed project, but also improve its public transparency and support a more reliable setting of its deliverables (outputs) and hence its cost, duration and level of risk (Zwikael & Smyrk 2011). In this article, we use the term ‘benefit definition’ to refer to the practices of identifying and defining target benefits submitted for approval by a potential government funder as part of a project proposal.

Benefit definition is a challenging task, as benefits are usually dynamic, uncertain, long term and intangible, and can have different meanings to various stakeholders. Thus, it is hard to create a comprehensive list of target benefits that satisfies conflicting stakeholder needs. This difficulty is amplified in the public sector because the government context is characterised by ‘uncertainty, ambiguity, and stakeholder management issues that are multifaceted and complex’ (Crawford, Costello, Pollack & Bentley 2003, p. 443). With this high level of complexity, it is not surprising that most organisations fail to properly define target benefits (Flak & Gronlund,

4

2008; Lin, Pervan & McDermid 2005; Norris 1996). For example, Lin and Pervan (2003) found that the current project justification processes used by large Australian organisations failed to provide a proper definition of target benefits from a project. Even more stunning is that 30-40% of ICT projects in Australia were found to be implemented without perceptible benefits (Young 2006). In addition, target benefits are often inflated to secure project approval (Jenner 2009). In Australia, Lin and Pervan (2003) found that some Australian organisations overstated benefits to get project proposals approved. Similarly, 20 per cent of managers in Norwegian government agencies admitted that they had overestimated target benefits (Flak & Gronlund 2008). In the UK government, it was found that ‘There is a demonstrated, systemic, tendency for project appraisers to be overly optimistic…appraisers tend to overstate benefits, and underestimate timings and costs’ (Her Majesty's Treasury 2003). Unfortunately, despite consensus of its importance and the poor performance stated above, issues related to benefit definition are still poorly understood in literature. The objective of this article is to bridge this gap by addressing the following research questions: ‘how can target benefits from public projects be assessed?’ and ‘what are the factors that contribute to better-defined target benefits for public projects?’

We answer these research questions by presenting a theoretical framework for the quality of target benefits from public projects, which is developed based on a review of the state-of-the-art literature and analysis of practices using a qualitative study. We divide the remainder of this article into five parts. First, we discuss the trend towards project benefit-oriented management in the public sector along with the movement towards Public Value Management (PVM; a post-New Public Management (NPM) model). Second, we analyse the literature on benefit realisation management (BRM). Third, we describe the research methodology and present interview results. Then, we present the theoretical framework and a series of propositions. Last, we list managerial implications and suggest directions for future research.

2. Trend towards benefit-oriented project management in the public sector

2.1 From New Public Management to Public Value Management

In past decades, public management has undergone tremendous changes. For some time, NPM has been viewed as a new paradigm in public administration, which aims to improve efficiency in the use of resources and effectiveness in achieving organisational objectives (O’Flynn 2007; Stoker 2006). Two examples of its underlying concepts are to implement private sector management practices in the public sector and to increase emphasis on the controls of inputs and outputs (Hood 1991). However, NPM has been criticised because private competitive models have failed to capture the interconnected and interdependent nature of the public sector (O’Flynn 2007). In particular, this competitive government approach failed to ‘understand that public management arrangements not only deliver public services, but also enshrine deeper governance values’ (Organisation for Economic Cooperation and Development 2003, p. 3). As a result, several post-NPM approaches have been proposed. One that attracts considerable interest in both practice and academic communities (e.g., Alford 2002; Moore 1994; Spano 2009) is PVM, which is ‘a new “post-competitive” paradigm [that] could signal a shift away from the primary focus on results and efficiency toward the achievement of the broader governmental goal of public value creation’ (O’Flynn 2007, p. 358).

While PVM uses terms such as ‘use value’ and ‘exchange value’, they are not well defined in the literature (Bowman & Ambrosini 2000). Public value can be created by the

5

government through services, law regulations and other actions (Kelley, Mulgan & Muers 2002) and is achieved when the needs of citizens are satisfied (Spano 2009; Moore 1995). As these endeavours are usually implemented in the form of public projects and programmes, the creation of public value is argued to significantly depend on the extent to which their target benefits are realised. Kelly, Mulgan and Muers (2002, p. 1) best described this relationship thus: ‘the concept of public value is an attempt to measure the total benefits which flow from government action…[Public value] could be considered a more complex tool for assessing the total value of government services’.

2.2 From output-focused to benefit-oriented project management in the public sector

As there is a strong relationship between benefit realisation from public projects and the creation of public value, and due to increased pressure to improve public management efficiency, governments worldwide have started to devote significant efforts to ensuring that target benefits are successfully realised from public projects. For example, the UK OGC was established to ‘help Government deliver best value from its spending’, with a specific emphasis on ‘realizing benefits’ (OGC 2009). Consequently, the OGC developed a project management methodology called ‘PRojects IN Controlled Environments’ (PRINCE2), where benefit realisation was emphasised as an important part of the business case and critical for subsequent projects’ reviews (OGC 2009). This methodology is also being utilised by the majority of Australian government agencies that believe that more focus should be placed on improving benefit realisation from projects (Gershon 2008).

This trend in practice towards emphasising benefit realisation has received further support from the management literature. Researchers have recently recognised the limitations of the traditional output-focused project management approach and have shifted towards a benefit-oriented theory. The traditional view, which focuses on efficient delivery of outputs (e.g., an artefact such as a bridge) on time, on budget and according to specifications (so called ‘iron triangle’), has been criticised for neglecting the processes of benefit generation and realisation (Remenyi, Sherwood-Smith & White 1997; Ashurst, Doherty & Peppard 2008). As the only purpose of a project is to realise target benefits that are sought by its funding organisation (Shenhar & Dvir 2007), a view overlooking benefit realisation can be misleading. As supporting evidence, literature has shown that a project can still be a failure even if the iron triangle is met (Zwikael & Smyrk 2012). A case in point is the Los Angeles Metro, which was completed on time and under budget, and met all operational, safety and service requirements. However, the project was declared a failure since the residents of LA refused to leave their cars at home and use the Metro for their transportation needs. Thus, the target benefits from the project have not been realised. As a result, the plan to expand the rail network was abandoned (Shenhar & Dvir 2007).

To overcome the limitations of the traditional output-focused project management approach, research efforts have been devoted to promote and develop theoretical frameworks focusing on benefit realisation. As a first step, researchers (e.g., Shenhar & Dvir 2007; Turner 2009) have redefined ‘projects’ by integrating benefit-emphasis terms. Examples include the definition of a project as a ‘unique process intended to achieve target outcomes’ and the distinction of project investment success and project ownership success from the traditional iron-

6

triangle-based project management success (Zwikael & Smyrk 2012). A comparison of the output-focused and benefit-oriented project management paradigms is presented in Table 1.

< Table 1 about here >

An analysis of the underlying concepts of benefit-oriented project management, described in Table 1, reveals consistent similarities with those of PVM. For example, benefit-oriented project management suggests that all practices should support the ultimate realisation of benefits rather than narrowly focus on the controls of inputs and outputs. This principle is in line with Moore (1995), who argued that strategic management practices should be geared towards the generation of public value. Benefit-oriented project management also suggests that project performance should be gauged from different aspects concerning both output delivery and benefit realisation. A similar concept is also found in PVM, which suggests that multiple performance objectives (e.g., service outputs, stakeholder satisfaction and outcomes) should be met (Moore 1995). In addition, both approaches emphasise the critical role of beneficiaries and the importance of stakeholder participation. Finally, both disciplines agree that accountabilities for project output delivery and benefit realisation are clearly defined (Zwikael & Smyrk 2012).

3. Benefit realisation management

Within the trend towards benefit-oriented project management, a specific research focus on BRM is of special interest. Studies in this area can be grouped into two categories based on their objectives. One is to develop theoretical models on how BRM should be undertaken over the course of a project life, starting with benefit definition and ending with an assessment of its realisation (e.g., Ward et al. 1996; Bradley 2010). The other category is to study how benefits are managed in practice and suggest ways that organisations can improve their BRM practices. Examples include investigation of IS/IT benefits management practices in large Australian organisations (Lin & Pervan 2003) and development of benefit realisation capabilities for specific organisations (Ashurst et al. 2008). Although providing some insights into BRM, these prior studies are limited because they mostly focus on IS/IT projects and the private sector.

Another limitation of the literature is a lack of research focusing on the process of benefit definition, although it is recognised as important in successful benefit realisation. Researchers argue that the lack of theory on the benefit definition process presents a great barrier for effective BRM, as evident by poor current practice, which finds most target benefits are vague (Norris 1996) and inflated to secure their approval (Jenner 2009). For example, 20 per cent of managers in Norwegian government agencies admitted that they had overestimated target benefits (Flak & Gronlund 2008). In the UK government, it was found that ‘There is a demonstrated, systemic, tendency for project appraisers to be overly optimistic…appraisers tend to overstate benefits, and underestimate timings and costs’ (Her Majesty's Treasury 2003). To address these issues, this study aims at increasing the knowledge on assessment and improvement of target benefits in the public sector.

4. Methodology

In this study, we develop a theoretical framework based on the integration of prior knowledge and interviews (Zikmund, Babin, Carr & Griffin 2010). This research design allows interviewees to reflect on, and evaluate, their own practices, as well as supporting researchers in

7

bridging the gap between theory and practice (Ashurst et al. 2008; Breese 2012). This research design was implemented in this study through a multi-stage iterative process, as illustrated in Table 2.

In Stage 1, we reviewed literature on public sector management, BRM and interrelated disciplines (e.g., strategic management and organisational behaviour) to clarify definitions of key terms and develop a list of criteria for assessing target benefits and identifying contributing factors.

In Stage 2, a series of semi-structured interviews were carried out amongst Australian Government senior managers to ascertain the current realities of benefit definition in the public sector. These interviews also served the purpose of validating the assessment criteria of target benefits and contributors derived from the literature and identifying missing criteria. Purposive sampling was used to recruit Australian Government managers that were involved in defining target benefits. Potential participants were invited for interviews via email and follow-up phone calls. Fifteen managers from eight government agencies participated in the interviews, comprising two Directors, one Assistant Director, one Executive Secretary, seven National Managers, three General Managers and one Programme Manager. The heterogeneous roles played by different interviewees allowed us to understand how the benefit definition process is performed in practice. Each interview lasted 40 minutes. All interviews were audio-taped, transcribed and analysed with NVivo software.

In Stage 3, a theoretical framework and a list of propositions were developed. A further literature review was undertaken during this stage to ensure the solidity of the proposed framework and propositions.


5. Interview results

This section summarises the results of our interviews and compares them to the literature. Before the interviews, we identified a set of criteria in the literature for assessing target benefits, as described in the left-hand column of Table 3. Similarly, we identified factors associated with an effective benefit realisation process, which is listed in the left-hand column of Table 4. We then validated these criteria and factored in the interviews with a focus on the process of benefit definition. The interview results are summarised in the right-hand columns of Tables 3 and 4 respectively, with more detailed discussions provided in the following section. These analyses show not only that the general management literature is relevant in the context of defining target benefits for public projects, but also that reflections of best practice contribute to existing knowledge.



6. Theoretical framework and propositions

Focusing on ICT investments in the public sector, Jenner (2009) argued that target benefits should be ‘robust and realisable’. A consensus was also found among our interviewees on this point. However, it remains unclear in the literature how we can distinguish these target benefits from those that are not. To address this issue, in this section we introduce and define the notion of ‘quality of target benefits’ and suggest a set of criteria for its assessment. We also

8

identify four constructs that may influence the quality of target benefits and describe a full theoretical framework with supporting propositions. Detailed discussions are provided below.

6.1 Quality of target benefits

The term ‘quality’, though commonly used, is not consistently defined in the literature. Its definition ranges from excellence, value and conformance to specification and meeting/exceeding customer expectations (Reeves & Bednar 1994). As target benefits are the goals set for a project, defining target benefits is similar to setting goals. Goal-setting theory suggests that, in order to achieve better performance, goals must meet the criteria of Specific, Measureable, Achievable, Relevant and Time targeted (SMART) (Doran 1981), and the value of applying SMART criteria in goal setting has been confirmed in a myriad of studies (e.g., Boyne & Chen 2007; Verbeeten 2008). Based on the goal-setting theory, we assess the quality of target benefits in terms of whether they meet a set of criteria derived from both literature and practices.

Our analysis shows that five of the criteria for assessing the quality of target benefits that emerged from our interviews are consistent with SMART criteria, as well as the recent management literature. Two other criteria—accountable and comprehensive—have been continuously mentioned by interviewees as specifically important in the context of public projects and, although they are not included in the SMART framework, they have strong support from the general management literature. Table 3 introduces the comparison between the management literature and interview results, while the next paragraphs provide a detailed description of each of the criteria for assessing target benefits.

Specific. A vague statement of benefits can lead to an uncertain allocation of resources and responsibility for managing their delivery (Norris 1996). Therefore, target benefits must be specifically defined to prevent different interpretations by various stakeholders (Breese 2012; Ward & Daniel 2006). To meet this specificity requirement, target benefits should have a baseline and target value that can be in absolute (e.g., achieve a customer satisfaction score of eight on a 10-point scale) or relative terms (e.g., increased current value by 10 per cent). In our interviews, National Manager B supported this view by saying that ‘the benefits need to be well defined—that they have a baseline, they have an interim and an end target’ (see Locke, Chah, Harrison and Lustgarten (1989) for further discussion of high and low specificity goals).

Measurable. Target benefits must be measurable to allow managers to determine whether they have been realised (Ward & Daniel 2006). That is, they must have agreed measures (Ashurst et al. 2008), as consistently emphasised by the majority of our interviewees. However, several interviewees indicated that it is difficult to set measures for some target benefits because of their complexity and dynamic nature. For example, Director A mentioned this difficulty: ‘Some [benefits] are easy to do metrics for, we can quantify them and some less easy…some are long term…and the benefits are ongoing’ . Therefore, in setting the measures, managers should ensure that measures for target benefits could allow future assessments of benefit realisation (Melnyk, Stewart & Swink 2004; Zwikael & Smyrk 2012) and comply with regular reporting requirements in the public sector (Heinrich 2002).

Achievable. Target benefits should be realisable (Jenner 2009); that is, they should be ‘realistic given the context in which the organisation is operating and the constraints it has’ (Ward & Daniel 2006, p. 29). This realism of target benefits was reflected by National Manager

9

E: ‘I think too it’s about being realistic…about what you can achieve…so not over promising and I guess, yeah, being realistic about having…knowing where you’ve started from…having that baseline data and measuring…knowing what you are trying to achieve’.

Relevant. Target benefits should be relevant and important to the funding organisation (Ward & Daniel 2006) and thus be aligned with its strategy (Lederer & Mirani 1995). During the interviews, it became clear that benefit definition could be triggered either from top-down directions (e.g., compliance with a new law approved in the parliament) or bottom-up suggestions (e.g., from the operational areas). When the latter is the case, a ‘retro-fitting’ consideration is given to ensure that the target benefits are in line with the agency’s strategic objectives. This was described by National Manager F: ‘we work in both directions…so there’s an element of retro-fitting to make sure people are focusing on the right outcomes…rather than just paying money out’ .

Time targeted. Target benefits should have a set target date for their realisation (Breese 2012; Ward & Daniel 2006). A target date is important for continuous monitoring and tracking as well as the evaluation of benefit realisation. As pointed out by General Manager A, ‘there’s a work plan with a start and delivery date…it’s all clear…from what date…we start measuring the benefit now and how well we are tracking’ .

In addition to the above five literature-based criteria for assessing the quality of target benefits, our interviewees highlighted the importance of having clear accountability for the realisation of target benefits and including a comprehensive list of target benefits, which are described below.

Accountability. The need to establish clear lines of accountability for benefit realisation was highlighted by our interviewees. It can help organisations to envisage how they may effectively realise the benefits, which is expected to further enhance the achievability of target benefits (Breese 2012; Lin & Pervan 2003). For instance, Director A stated that ‘the more clear and visible the accountability is…the more people can keep a handle on the benefit’. Assigning a benefits owner—‘an individual or group who will gain advantage from [the target benefit] and who will work with the project team to ensure that the benefit is realised’ (Ward & Daniel 2006, p. 144)—is considered an effective way of addressing this accountability issue (Ashurst et al. 2008). As a result, each target benefit should have an accountable person’s name listed next to it.

Comprehensive. Our interviewees also considered it important to include a complete web of benefits in the defined target benefits, which is in line with the literature (e.g., Breese 2012; Jenner 2009; Truax 1997). This is particularly important for public projects, as they are normally concerned with various organisational, social and legal issues and need to meet the objectives of various stakeholder groups. This is reflected in the diverse groups of benefits (e.g., social benefits and efficiency benefits) that were mentioned in the interviews. Unfortunately, there is no universal answer as to what can be considered ‘comprehensive’, because it varies from one case to another. Commonly discussed categories of target benefits may provide some guidance in this regard, including financial/non-financial, direct/indirect, short/long term, internal/inter-organisational and economical/cultural benefits (Henderson & Ruikar 2010).

Proposition 1: The quality of target benefits can be assessed according to whether they have a target value, a target date and accountability, and the extent to which they are measurable, realistic and comprehensive, and whether they fit into organisational strategies.

10

Table 5 provides a summary of the criteria for assessing the quality of target benefits along with public sector examples.


6.2 Constructs that influence the quality of target benefits

Our interviewees across government agencies and departments also suggested factors that may influence the quality of target benefits. These factors, while consistent with general factors derived from the literature, also added more specificity to our understanding of the benefit definition process (as illustrated in Table 4). In order to develop a theoretical framework, similar factors were grouped together in an exercise that ended in four proposed constructs, as described in the next paragraph.

Some of the factors mentioned in the interviews are related to the benefit definition process, for example, to engage stakeholders and collaborate across functions. These factors were grouped under the construct heading of ‘a formal benefit definition process’. Other factors mentioned by our interviewees related to the way in which the benefit definition process is led; hence, it is grouped into the heading ‘leadership’. Examples include having clear governance structure and strong leadership. Factors related to mangers’ mindsets, including a public service and long-term mindset, were put together in the construct of ‘public service motivation’. Interviewees suggested that it is also important to have a senior executive holding an overview and supporting benefit definition practice, which leads to the fourth construct of ‘senior executive support’. In addition to the above four constructs, organisational culture and project type were also suggested as contextual variables that may affect the way BRM is viewed and utilised in the public sector (Ashurst et al. 2008; Breese 2012). The theoretical framework is illustrated in Figure 1, and the following paragraphs discuss each proposed construct.

Proposition 2: The quality of target benefits from public projects will be influenced by four constructs: (1) a formal benefit definition process, (2) leadership, (3) public service motivation and (4) senior executive support. The extent to which these constructs contribute to the quality of target benefits will be moderated by organisational culture and project type.

< Figure 1 about here >

6.2.1 A formal benefit definition process

The need for proactive benefit planning and formal methodologies to ensure benefit realisation is supported in the literature (e.g., Doherty, Ashurst & Peppard 2012; Lin & Pervan 2003). Similarly, our interviewees emphasised the importance of having a formal benefit definition process. Nonetheless, we found that the availability and use of such a process varies across public agencies. In some agencies, such a process was simply unavailable, while in others it was either embedded in existing budgeting and costing systems or under development. This finding is in line with those of Lin and Pervan (2003), which revealed the differences in the level of formality in managing benefits among large Australian organisations.

Despite the differences in benefit definition practices across public agencies, several process-related factors have been highlighted by our interviewees and will be discussed next.

11

These include stakeholder engagement, cross-functional collaboration, benchmarking and supporting guidelines.

Stakeholder engagement. To engage stakeholders is thought by almost all interviewees to be important while defining target benefits. Director A said that ‘…in that blueprinting process we will get all the key people in the room and get them to workshop what it means…what are the outcomes and how will that translate into a benefit and how will we align our outputs to that as well’. This is in line with Doherty et al.’s (2012) study, which suggested that all relevant stakeholders, not just users, should be engaged. Given that stakeholders may vary from one project to another and change over time, careful consideration should be given to engage the ‘ right stakeholder for the right reason at the right time’ (National Manager D).

Cross-functional collaboration. The fragmentation of goals can negatively affect the BRM process, even when a dedicated project team is formed (Doherty et al. 2012). This issue can be compounded in the public sector context, where the overall governance structure is complex, involving national, regional and local government levels. In addition, in many cases, there may be a number of public projects across agencies sharing the same set of target benefits. For these reasons, many of our interviewees consistently highlighted the need to have cross-functional collaboration while defining target benefits to develop a ‘shared view’. National Manager C best described this as ‘having to work across…having to keep in mind the government outcomes that are required and have that as the very key shared understanding of what the outcomes should be as a starting point and then if we have to go and renegotiate it…at least we’ve got a view of that…’.

Benchmarking. Having an ongoing benefits review is considered important in ensuring benefit realisation (Ashurst et al. 2008; Doherty et al. 2012). Our interviews also show that, after target benefits are defined, many organisations perform a review to ensure the quality of target benefits before they are submitted for approval. This review is normally conducted by benchmarking the target benefits with those of the agency’s other current or past initiatives, or by seeking feedback from a third party (e.g., external consultancy) to benchmark against external projects. For example, National Manager D mentioned the use of external consultancy: ‘we had a full evaluation done by a consultant firm, who came in and looked at our data and said: “yes, we think you will get these benefits out of it”’. General Manager B described their benchmarking practices: ‘We haven’t engaged consulting houses for that but we are working with [name of another unit within that agency] to do some benchmarking against what is going on with general elements of the sector’ . Despite this, some interviewees found it difficult to identify suitable subjects to benchmark against, especially when the services to be delivered are unique.

Supporting guidelines. Some of our interviewees emphasised the advantage of developing supporting guidelines with standardised tools and templates. One example mentioned the use of a transparent benefit register that documents the target benefits for future monitoring and facilitating cross-function collaboration.

Proposition 3: The use of a formal benefit definition process can enhance the quality of target benefits. Such a process should enable stakeholder engagement, facilitate cross-function collaboration, integrate quality assurance undertakings (e.g., benchmarking and third party evaluation) and establish supporting guidelines.

12

6.2.2 Leadership

As a change-driven process, leadership in BRM is critical (Doherty et al. 2012). Project team members can struggle when their senior managers are unwilling or unable to provide direction through active business leadership. This view was reflected in our interviews, which suggested that it is important to have a clear governance structure of the project proposal development team and a senior executive leading the benefit definition process. National Manager D emphasised this need to have a ‘a single accountable lead’ and further elaborated that ‘if you have more than one lead, more than one area, if you try and do things jointly, things often fall through the cracks, you need to have one responsible senior manager who’s leading it’.

Proposition 4: Strong leadership in the benefit definition process can enhance the quality of target benefits. This involves having a clear governance structure of the project proposal development team and a senior executive leading the benefit definition process.

6.2.3 Public service motivation

Public service motivation (PSM), which is ‘a particular form of altruism or pro-social motivation that is animated by specific dispositions and values arising from public institutions and missions’ (Perry, Hondeghem & Wise 2010, p. 682), is an important driver for performance in the public sector (Perry et al. 2010; Park & Rainey 2008). Given that the majority of prior research on BRM focuses on the private sector, the role of PSM in benefit realisation in the public sector remains unexplored in the literature. In our interviews, PSM was specifically mentioned by several interviewees as being important for benefit definition in the government context. With high levels of PSM, managers will more likely do things for the public good and give ‘frank and fearless advice’, which may lead to better-defined target benefits. Assistant Director A observed that ‘it’s more of a motivation…so I would like to put an emphasis on that and the motivation for some people might only be to just get the process done…many people in the [agency name] so I believe, they put a lot of emphasis on that because they know that their work and efforts will serve the public, so that’s what we call “public goods”’.

The operation in the public sector is often subject to political pressures (Crawford et al. 2003). This imposes a dilemma to managers of whether to seek the long-term good for the public or short-term gains for the minister in charge. Sometimes, managers may be forced to compromise long-term benefits for short-term achievements and/or to scarify difficult-to-measure benefits for easy-to-measure benefits. This dilemma is reflected in a comment provided by Programme Manager A: ‘I think having a long-term focus is important but I think it’s difficult in a political environment where you’re working on budget cycles and election cycles…’

It remains unclear how this dilemma can be best addressed. However, some of our interviewees suggested that organisations have clear and consistent long-term strategic objectives (or a roadmap). Within this broad picture, managers may seek further opportunities, which can be subject to short-term election cycles. Two interviewees best described this: ‘we’ve got the same [overall] goals and objectives [of the programme] in place after they were set five years ago with obvious reviews in the meantime…so while everything else has been changing, those have remained, they’ve been tested but remained relatively consistent and I think that’s important’ (National Manager C). ‘We do have a long-term plan, but the long-term plan is just a

13

roadmap for us…but what we need to do on the ground is take the opportunity that’s coming for us to enhance that roadmap’ (General Manager A).

Proposition 5: The quality of target benefits can be improved if managers possess high levels of PSM. It is equally important that the agency develops and maintains clear and consistent long-term strategic objectives while leaving some flexibility for managers to seek short-term opportunities.

6.2.4 Senior executive support

Ongoing organisational commitment is critical to BRM (Ashurst et al. 2008). Similarly, our interviews suggested that senior executive support is important, which includes a senior executive holding an overview of the agency and providing support to the benefit definition process. Such support can direct the agency’s focus on benefits, which in turn improves the agency’s overall benefit definition process. This is described by National Manager F: ‘We have very strong support from our senior executive who are very keen to do this…we run what we call our ‘ginger group’, consisting of heads of division and we’ve got a project going at the moment to build our capability here’.

Our finding about the influence of senior executive support on the quality of target benefits is not surprising. It is in line with the organisation behaviour and performance measurement literature (e.g., Kirman & Benson 1999; Riggle, Edmondson & Hansen 2009), which suggest a positive effect of senior executive support (or organisational support) on employees’ (or team) performances.

Proposition 6: Senior executive support can direct an agency’s focus on benefits and affect its benefit definition process. The quality of target benefits is likely to be improved when strong executive support is in place.

6.2.5 Organisational culture and project type

Contingency theory suggests that organisational effectiveness is dependent upon the organisation’s ability to adjust to its environment (Thorgren, Wincent & Anokhin 2010). Two contextual variables—organisational culture and project type—have also been integrated into the proposed theoretical framework. Organisational culture has been widely recognised in the literature (e.g., Ogbonna & Harris 2000; Marcoulides & Heck 1993) to affect the performance of employees and the long-term effectiveness of organisations. It is also found critical in explaining why some organisations are more successful in securing benefits from their projects than others (Ashurst et al. 2008; Doherty et al. 2012). Therefore, it is not surprising when the potential moderating effect of organisational culture on the benefit definition process was confirmed by our interviewees. We found that organisations with a supportive culture, that is, encouraging, open, relationship oriented and collaborative (Wallach 1983), are more committed to benefit definition, for example, by establishing standardised processes and documentations. Organisational culture can also affect the way managers approach the benefit definition process, as described by General Manager B: ‘It’s a very ‘can do’ culture…people will collaborate to get things done…that there are tough issues but we will collaborate to get good answers…I think as it comes to being better in recognising and understanding benefits…that’s acknowledged, and

14

people are working very actively on that…and seeing the benefit of doing that…understanding the good of what you do’.

Further, it is widely recognised that differences in project characteristics can affect the ways they are planned and managed (Shenhar & Dvir, 2007). Doherty et al. (2012) suggested that, because no two projects are the same, approaches to managing benefits should be tailored to the specific organisational and project contexts.

Proposition 7: Organisational culture and project types interact with all four constructs to influence the quality of target benefits.

7. Discussion

Successful public projects are linked to the effective delivery of government services and policies; hence, the creation of public value. Benefit realisation from public projects is therefore critical for a nation’s social and economic development. This paper focuses on the process of defining target benefits from new initiatives, which is undertaken by senior public sector servants before projects are approved. Despite the recognised importance of this process in the literature (e.g., Ward et al. 1996; Bradley 2010), very little research has been devoted to developing a theory for this process in the public sector. In part, this research fills this gap in the literature by proposing a theoretical framework, introducing the notion of ‘the quality of target benefits’ and suggesting a set of criteria for its assessment based on the integration of related theories (e.g., goal setting and contingency) and practice (e.g., inputs from public sector managers). We argue that, to be considered quality, target benefits must be measurable, realistic and comprehensive. They should also have target values and target dates, as well as clear accountability, and they should fit into an organisation’s strategic objectives. We also argue that using a formal benefit definition process, having strong senior executive support and leadership in the process, and having managers with high levels of public service motivations can enhance the quality of target benefits. The extent of such contributions will be moderated by the organisational culture and project type. The present framework has advanced our knowledge on how the quality of target benefits from the public projects can be gauged and improved.

Our findings also have critical managerial implications for the public sector. First, public agencies need to establish an organisational structured process for benefit definition. This process should enable stakeholder engagement and cross-function collaboration and integrate benchmarking or third party evaluation to assure the quality of target benefits. Moreover, agencies should create a culture that emphasises benefit management. This may be achieved by providing strong executive support and commitment on benefit management practices. In the meantime, in defining target benefits, public sector managers should be more broadly concerned with the organisation’s strategic objectives instead of narrow task-oriented project goals. In part, this reflects the need for managers to proactively negotiate and engage with different constitutions, including operational units within and/or across agencies and various stakeholder groups. This would require them to develop a new set of communication and managerial skills.

8. Conclusions

In order to improve the value generated from public projects, literature and practice suggest increasing the emphasis on benefit realisation management. This paper advances our

15

knowledge on how the quality of target benefits from public projects can be assessed and improved. High-quality target benefits can ensure easier management of projects and programmes, clear expectations from various stakeholders and hence a higher chance of benefit realisation.

Given the exploratory nature of this study, further work is needed in several directions. First, more effort is needed to develop comprehensive scales for assessing individual constructs. These scales should consist of several items rather than factors that have been currently included. Second, it is necessary to empirically investigate the relationship between the individual construct and the quality of target benefits, and explore in more detail which of the proposed constructs are more important.

In addition, even though the constructs in the framework are presented independently, they may be interdependent. Future studies can examine how these constructs interact with each other. For example, senior executive support may be more effective if it is supported by the use of a formal benefit definition process. To give one example, senior executives that provide the project proposal team with gateways to other functions may improve cross-functional collaboration, which may in turn lead to the improved quality of target benefits. Future research may also validate the proposed theoretical framework in government agencies in various countries and suggest amendments for the private sector context. Such studies would be useful to establish the extent to which the present framework reflects practice.

References Alford, J 2002, ‘Defining the client in the public sector: a social exchange perspective’, Public

Administration Review, vol. 62, no. 3, pp. 337–346. Ashurst, C, Doherty, NF & Peppard, J 2008, ‘Improving the impact of IT development projects:

the benefits realization capability model’, European Journal of Information Systems, vol. 17, pp. 352–370.

Bowman, C & Ambrosini, V 2000, ‘Value creation versus value capture: towards a coherent definition of value in strategy’, British Journal of Management, vol. 11, pp. 1–15.

Boyne, GA & Chen, AA 2007, ‘Performance targets and public service improvement’, Journal of Public Administration Research and Theory, vol. 17, no. 3, pp. 455–477.

Breese, R 2012, ‘Benefit realisation management: panacea or false dawn?’, International Journal of Project Management, vol. 30, pp. 341–351.

Bradley, G 2010. Benefit Realisation Management: A Practical Guide to Achieving Benefits through Change. Surrey: Gower Publishing Ltd.

Bryde, DJ 2005. ‘Methods for managing different perspectives of project success’, British Journal of Management, vol. 16, pp. 119–131.

Crawford, L, Costello, K, Pollack J & Bentley, L 2003, ‘Managing soft change projects in the public sector’, International Journal of Project Management, vol. 21, pp. 443–448.

Damanpour, F 2010. ‘An integration of research findings of effects of firm size and market competition on product and process innovations’, British Journal of Management, vol. 21, pp. 996–1010.

Devinney, TM, Yip, GS & Johnson, G 2010. ‘Using frontier analysis to evaluate company performance’, British Journal of Management, vol. 21, pp. 921–938.

16

Doherty, N, Ashurst C & Peppard, J 2012. ‘Factors affecting the successful realization of benefits from systems development projects: findings from three case studies’, Journal of Information Technology, vol. 27, no.1, pp. 1–16.

Doran, G 1981, ‘There’s a SMART way to write management’s goals and objectives’, Management Review, vol. 70, no. 11, pp. 35–36.

Dunleavy, P, Margetts, H, Bastow, S & Tinkler, J 2006. Digital Era Governance. Oxford: Oxford University Press.

eGovernment Economics Project (eGEP) 2006. Measurement Framework. Prepared for the eGovernment Unit, DG Information Society and Media, European Commission.

Flak, LS & Gronlund, A 2008. ‘Managing benefits in the public sector: surveying expectations and outcomes in Norwegian government agencies’, Journal of E-Government, vol. 5184, pp. 98–110.

Gershon, P 2008. Review of Australian Government’s Use of Information and Communication Technology. Parkes: Department of Finance and Deregulation.

Heinrich, CJ 2002, ‘Outcomes-based performance management in the public sector: implications for government accountability and effectiveness’, Public Administration Review, vol. 62, no. 6, pp. 712–725.

Her Majesty’s Treasury 2003, The Green Book Appraisal and Evaluation in Central Government. London: HM Treasury.

Henderson, J & Ruikar, K 2010. ‘Technology implementation strategies for construction organisations’, Engineering, Construction and Architectural Management, vol.17, no. 3, pp. 309–327.

Hood, C 1991, ‘A public management for all seasons?’ Public Administration, vol. 69, no. 1, pp. 3–19.

Jenner, S 2009. Realising Benefits from Government ICT Investment—A Fool's Errand? Reading: Academic Publishing.

Kelly, G, Mulgan, G &Muers, S 2002, Creating Public Value: An Analytical Framework for Public Service Reform. Discussion paper prepared by the Cabinet Office Strategy Unit, London.

Kirkman, BL & Benson, R 1999. ‘Beyond self-management: antecedents and consequences of team empowerment’, Academy of Management Journal, vol. 42, no. 1, pp. 58–74.

Kwak, YH & Smith, B 2009, ‘Managing risks in mega defense acquisition projects: performance, policy, and opportunities’, International Journal of Project Management, vol. 27, no. 8, pp. 812–820.

Lederer, AL & Mirani R 1995, ‘Anticipating the benefits of proposed information systems’, Journal of Information Technology, vol. 10, pp. 159–169.

Lin, C & Pervan, G 2003. ‘The practice of IS/IT benefits management in large Australian organisations’, Information & Management, vol. 41, no. 1, pp. 13–24.

Lin, C, Pervan, G & McDermid, D 2005, ‘IS/IT investments evaluation and benefits realisation issues in Australia’, Journal of Research and Practices in Information Technology, vol. 37, no. 3, pp. 235–251.

Locke, EA, Chah, DO, Harrison, S &Lustgarten, N 1989. ‘Separating the effects of goal specificity from goal level’, Organizational Behavior and Human Decision Processes, vol. 43, no. 2, pp. 270–287.

17

Love, PED &Irani, Z 2004. ‘An exploratory study of information technology evaluation and benefits management practices of SMEs in construction’, Information and Management, vol. 43, no. 2, pp. 227–242.

Marcoulides, GA & Heck, RH 1993, ‘Organizational culture and performance: proposing and testing a model’, Organization Science, vol. 4, no. 2, pp. 209–225.

Melnyk, SA, Stewart, DM &Swink, M 2004. ‘Metrics and performance measurement in operations management: dealing with the metrics maze’, Journal of Operations Management, vol. 22, no. 3, pp. 209–218.

Moore, M 1994. ‘Public value as the focus of strategy’, Australian Journal of Public Administration, vol. 53, no. 3, pp. 296–303.

Moore, M 1995. Creating Public Value: Strategic Management in Government. Cambridge, Massachusetts: Harvard University Press.

Norris, GD 1996, ‘Post-investment appraisal’. In L. Willcocks (ed), Investing in Information Systems: Evaluation and Management, Chapter 9, pp. 193–223. London: Chapman & Hall.

Ogbonna, E & Harris, LC 2000. ‘Leadership style, organizational culture and performance: empirical evidence from UK companies’, International Journal of Human Resource Management, vol. 11, no. 4, pp. 766–788.

Office of Government Commerce (OGC) 2009. Managing Successful Projects with PRINCE2. The Stationery Office, Great Britain.

OGC 2003, Gateway News, December. London: OGC. OGC 2005, OGC Successful Delivery Toolkit. London: OGC. O’Flynn, J 2007, ‘From new public management to public value: paradigmatic change and

managerial implications’, Australian Journal of Public Administration, vol. 66, no. 3, pp. 353–366.

Organisation for Economic Cooperation and Development (OECD) 2003. Public Sector Modernisation. OECD Policy Brief. Paris: OECD.

Park, SM &Rainey, HG 2008, ‘Leadership and public service motivation in U.S. federal agencies’, International Public Management Journal, vol. 11, no. 1, pp. 109–142.

Perry, JL, Hondeghem, A & Wise, LR 2010. ‘Revisiting the motivational bases of public service: twenty years of research and an agenda for the future’, Public Administration Review, vol. 70, no. 5, pp. 681–690.

Reeves, CA & Bednar, DA 1994. ‘Defining quality: alternatives and implications’, Academy of Management Review, vol. 19, no. 3, pp. 419–445.

Remenyi, D, Sherwood-Smith, M & White T 1997, Achieving Maximum Value from Information Systems: A Process Approach. Chichester: Wiley.

Riggle, RJ, Edmondson DR & Hansen, JD 2009. ‘A meta-analysis of the relationship between perceived organizational support and job outcomes: 20 years of research’, Journal of Business Research, vol. 62, no. 10, pp. 1027–1030.

Rhys, A & Boyne, GA 2010, ‘Capacity, leadership, and organizational performance: testing the black box model of public management’, Public Administration Review, vol. 70, no. 3, pp. 443–454.

Shenhar, AJ & Dvir, D 2007, Reinventing Project Management: The Diamond Approach to Successful Growth and Innovation. Boston, Massachusetts: Harvard Business School Press.

Spano, A 2009, ‘Public value creation and management control systems’, International Journal of Public Administration, vol. 32, no. 3-4, pp. 328–348.

18

Stoker, G 2006, ‘Public value management: a new narrative for networked governance?’ American Review of Public Administration, vol. 36, no. 1, pp. 41–57.

Thorgren, S, Wincent, J & Anokhin, S 2010, ‘The importance of compensating strategic network board members for network performance: a contingency approach’, British Journal of Management, vol. 21, pp. 131–151.

Truax, J 1997. ‘Investing with benefits in mind: curing investment myopia,’ DMR White Paper, pp. 1–6.

Turner, JR 2009. The Handbook of Project-Based Management. 3rd ed. London: McGraw-Hill. Verbeeten, FHM 2008, ‘Performance management practices in public sector organizations:

impact on performance’, Accounting, Auditing & Accountability Journal, vol. 21, no. 3, pp. 427–454.

Wallach, EJ 1983. ‘Individuals and organizations: the cultural match’, Training & Development Journal, vol. 37, no. 2, pp. 28–37.

Ward, J, Taylor, P & Bond, P 1996, ‘Evaluation and realisation of IS/IT benefits: an empirical study of current practice’, European Journal of Information Systems, vol. 4, no. 4, pp. 214–225.

Ward, J & Daniel, E 2006. Benefits management: Delivering value from IS & IT investments (Vol. 30). New York, NY: John Wiley & Sons Inc.

Zikmund, WG, Babin, BJ, Carr, JC & Griffin, M 2010. Business Research Methods. 8th Edition. Mason, Ohio: South-Western Cengage Learning.

Zwikael, O & Smyrk, JR 2011. Project Management for the Creation of Organisational Value. London: Springer-Verlag.

Zwikael, O & Smyrk, JR 2012. ‘A general framework for gauging the performance of initiatives to enhance organizational value’, British Journal of Management, vol. 23, pp. S6–S22

19

Table 1. Output-focused vs. benefit-oriented project management Output-focused project

management Benefit-oriented project management

Managerial focuses

Managing inputs and outputs Multiple focuses: managing inputs and outputs with a focus on the ultimate realisation of project benefits

Performance measures

Iron triangle (time, budget and scope/quality)

Multiple measures: distinguish project success and project management success, where iron triangle is used for measuring project management success and benefit realisation is used to measure project success

Strategic goal Meet agreed efficiency targets measured by the iron triangle

Multiple objectives: response to stakeholder needs, improve organisational capacity and implement strategic plans

Accountability The project manager is accountable for output delivery

Multiple accountability: project owner, the project funder’s agent, is accountable for outcome realisation; project manager remains accountable for output delivery

20

Table 2: Theoretical Framework and Proposition Development Process Stage 1 2 3

Title Preliminary literature review

Semi-structured interviews

Theoretical framework and propositions development

Outputs • Definitions of key terms

• Review of studies on benefit definition

• Preliminary lists of criteria for assessing target benefits and contributing factors

• Analysis of benefit definition practices in government

• Validated lists of criteria for assessing target benefits and contributing factors

• Theoretical framework linking various constructs to the assessment of target benefits

• List of propositions

21

Table 3. Criteria for assessing target benefits Criteria from the literature

References Reflections in the interviews with a focus on the benefit definition process

Clearly defined Breese (2012) Have target value Ashurst et al. (2008); Breese (2012)

Specific (target value)

Agreed measures Ashurst et al. (2008) Measurable Realisable Jenner (2009); Ward and Daniel (2006) Achievable (realistic) Strategic alignment Lederer and Mirani (1995) Relevant (strategic fit) Have realisation timeline

Breese (2012) Time targeted (target date)

Accountable Ashurst et al.(2008); Breese (2012); Ward and Daniel (2006)

Accountability

A complete web of benefits

Breese (2012); Jenner (2009); Truax (1997)

Comprehensive

22

Table 4. Factors contributing to better-defined target benefits Contributing

factors from the literature

References Reflections in the interviews with a focus on the benefit

definition process Stakeholder involvement

Ashurst et al. (2008); Doherty et al. (2012); Remenyi et al. (1997)

• Stakeholder engagement

Ongoing benefits reviews

Ashurst et al. (2008); Doherty et al. (2012); Lin and Pervan (2003)

• Benchmarking • Supporting guidelines

Establish coherent governance structure

Ashurst et al. (2008); Doherty et al. (2012); Lin and Pervan (2003)

• Cross-functional collaboration

Active business leadership

Doherty et al. (2012) • Clear governance structure • Strong leadership

Ongoing organisational commitments

Ashurst et al. (2008) • Senior manager support • A senior executive holding an

overview Public service motivation

Perry et al. (2010) • Public service mindset • Long-term mindset

Tailor to context Doherty et al. (2012); Truax (1997)

• Organisational culture • Project type

23

Table 5: Criteria for assessing the quality of target benefits Criteria Brief Description of Target Benefits Example Target value Have a baseline, a target value with

specific (positive or negative) direction A reduced time of 25% for finalising a rebate file at the tax office

Measurable Be measurable through the use of either a direct measure or an indirect indicator

Time (in minutes) per tax rebate file

Realistic Be realistic, given the context in which the organisation is operating and its constraints

Reflected in the target value to which it is assigned, or the target benefit itself

Strategic fit Fit into the organisation’s strategy Align with the organisation’s strategy to enhance operational efficiency

Target date Have a set date for realisation One year from project approval

Accountability Have a benefits owner National tax office manager Comprehensive Be considered from a variety of

aspects. Include broad categories of benefits

24

Stakeholder engagement

Cross-functional

collaboration

Benchmarking

Supporting guidelines

Clear governance structure

Strong leadership

Public service mindset

Long-term mindset

Senior executive supports

A senior executive holding

an overview

Target value

Measurable

Realistic

Accountability

Strategic fit

Target date

Comprehensive

P3

P4

P5

P6P7

P7

A formal benefit

definition process

Leadership

Public service

motivation

Senior executive

support

Quality of target

benefits

P1/P2

Organisational

culture

Project type

Figure 1. Theoretical framework for the quality of target benefits from public projects

benefit realization from public projects full paper

Documents