1

PERFORMANCE MEASUREMENT IN HUMANITARIAN LOGISTICS: A LITERATURE

REVIEW Hella Abidi*

Matthias Klumpp**

*) FOM University of Applied Sciences, Institute for Logistics and Service Management,

Leimkugelstraße 6, D-45141 Essen, Germany & VU University Amsterdam, Amsterdam, Netherlands

E-mail: hella.abidi@fom-ild.de, Tel: +49 201 81004 991

**) FOM University of Applied Sciences, Institute for Logistics and Service Management, Leimkugelstraße 6, D-45141 Essen, Germany

E-mail: matthias.klumpp@fom-ild.de, Tel: +49 201 81004 554

ABSTRACT Purpose Performance measurement in humanitarian logistics is crucial and is a critical element of successful humanitarian relief operations on strategic, tactical and operational levels. The purpose of this paper is to provide a literature review on performance measurement and key performance measures in humanitarian logistics and to outline future research opportunities in performance measurement in humanitarian logistics.

Design/methodology/approach For the literature review a qualitative content analysis is used to determine main managerial problems and major findings in this area. The existing literature is categorized in several attributes like general trends of performance measurement in humanitarian logistics, stage of disaster, research methodologies, key performance measures, performance measurement systems and the relationship with internal and external stakeholders. Findings Findings reveal that performance measurement in the humanitarian logistics is still a gap within logistics research, especially compared to the commercial logistics sector. However, the major findings from existing literature are presented and recommendations for further research are proposed.

Practical implications The result helps the humanitarian logistics community to conduct further research in this area and to develop key performance measures that suit the humanitarian logistics sector. Original/value The research paper shows how essential performance measurement in humanitarian logistics is and what the state-of-the-art in this area includes.

Keywords: Humanitarian logistics, performance measurement.

2

1. INTRODUCTION

In today´s environment the number of natural and man-made disasters has increased significantly. Due to climate change there will be even more disasters in the future according to experts prognoses (Oloruntoba, 2005; Dupont and Pearman, 2006). Thomas and Kopczak expect a steady increase by five times for the number of natural disasters over the next fifty years (Thomas and Kopczak, 2007). In 2006, the United Nations also confirmed that the natural disasters over the next years will become more severe, often and destructive (UN, 2006). In the existing Annual Disaster Statistical Review 2011 it is reported that 332 natural disasters were registered. In 2011, natural disasters killed 30,773 people and caused 244.7 million victims worldwide. Economic damages from natural disasters were estimated US$ 366.1 billion (Guha-Sapir et al., 2012). The increasing number of natural disasters and the resulting complex humanitarian emergencies put pressure on humanitarian organizations (in the following HOs) to deliver humanitarian aid in an appropriate and cost-effective way (van Wassenhove, 2006; Oloruntoba and Gray, 2006; Kovacz and Spens, 2007). HOs are faced with logistics complexity, destabilized infrastructure and environment and the HOs staff work in most cases in a chaotic environment (Cassidy, 2003). Humanitarian logistics is essential for disaster relief operations because it is characterized by effectiveness and speed in supplying beneficiaries with health, food, shelter, water, medicines and sanitation in case of disaster (Thomas and Kopczak, 2005). Humanitarian logistics is defined as „[…] the process of planning, implementing and controlling the cost effective flow and storage of goods and materials, as well as related information, from the point of origin to the point of consumption for the purpose of alleviating the suffering of vulnerable people. The function encompasses a range of activities, including preparedness, planning, procurement, transport, warehousing, tracking and tracing, and customs clearance” (Thomas and Kopczak, 2005, p.1).

Tomasini and van Wassenhove state that around 80% of the costs for relief operations consist of logistics costs in the form of procurement and transportation (Tomasini and van Wassenhove, 2009). Nevertheless, the significant increase of natural disasters, complex and cost intensive humanitarian logistics operations, responsibility, and reporting towards donors and beneficiaries are reasons for HOs to become more efficient in their operations. For an effective logistics management in particular, humanitarian logistics management, performance measurement and measures are crucial. The HOs are facing challenges in developing suitable and common performance measures: In practice 55% of HOs do not monitor and report any performance measurement measures, 25% declare to control only a few measures and 20% measure performance consistently (Blecken, 2010). The lack of performance measures has been a standing problem (Davidson, 2006). There are various critical elements that counteract enhancing the chances in this area which include (Blecken et al., 2009; Davidson, 2006; Widera and Hellingrath, 2011; Tatham and Hughes, 2011; Jahre and Heigh, 2008):

Nonexistence of centrally captured data from operations, Limited information technology capacity and infrastructure, Chaotic environment, Lack of motivation for measurement in the non-profit sector, Potentially negative media exposure, Human resource issues, General reluctance regarding performance measurement, Long-term versus short-term goals of disaster response, High complexity of performance measurement in this sector,

3

The ability of field workers to capture accurate data while working under significant time pressure,

No recognition of the key role of the logistic community as an essential part of NGO´s and humanitarian relief operations, No link between funding and humanitarian logistics, especially regarding performance measurement metrics.

The main target of performance measurement and suitable financial and non-financial measures is to inform decision makers at the strategic, tactical and operational level in the commencement of high quality goods, processes and services (Gunasekaran and Kobu, 2007) during humanitarian operations relief in disaster cases (Long, 1997). Performance measurement is fundamental for improvement (Kaplan, 1990), for making decisions (Gunasekaran and Kobu, 2007; Long, 1997), for simplifying communication among supply chain actors and for an increased transparency of supply chain and logistics processes (Gunasekaran and Kobu, 2007). This research paper attempts to identify literature on performance measurement and key measures in humanitarian logistics and to provide new research directions for performance measurement in humanitarian logistics. The paper is structured as follows: Section 2 presents the basic definition of performance measurement measures and systems in logistics. In Section 3 details of the review methodology are indicated; furthermore section 4 provides an overview of the existing literature on performance measurement in humanitarian logistics as well as the findings and discusses the results. Section 5 shows a research agenda that gives an overview for future research and regarding this subject and section 6 presents a conclusion.

2. PERFORMANCE MEASUREMENT

Chow et al. indicated that performance is multi-dimensional, because one measure is not sufficient for a logistics performance – logistics performance has to be seen as subsection of the larger conception of firm or organizational performance (Chow et al., 1994). To know the meaning of performance there are two central organizational and logistics goals which have to be defined. These are divided in two dimensions: The simplest dimension and which affect the performance – in particular logistics performance – is to differentiate between (i) efficiency and (ii) effectiveness (Gleason and Barnum, 1986) in performing logistics activities. Generally efficiency is “doing the things right” and effectiveness is defined as “doing the right thing” (Gleason and Barnum, 1986). Logistics effectiveness has to be viewed as the extent to which the logistics function’s goals – e.g. fulfillment time or in-stock availability – are accomplished (Mentzer and Konrad 1991). Logistics efficiency is the ratio of resources utilized against the results achieved. In a broader sense it indicates the measurement of how well the resources consumed are utilized (Mentzer and Konrad, 1991). Neely et al. supported this two dimensional perspectives later on (Neely et al., 1995). Sink et al. defined the meaning of performance in an extended view: They illustrate the seven dimensions of effectiveness, efficiency, quality, productivity, quality of work life, innovation and profitability (Sink et al., 1984). Gunasekaran and Kobu identified criteria for commercial logistics to show the purpose of performance measurement that can be adapted the humanitarian supply chain and logistics as follows (Gunasekaran and Kobu, 2007):

Identify success, Identify whether customers’ (donors’ and beneficiaries’) needs are met,

4

Help the organization to understand it’s processes and to confirm what they know or reveal what they do not know,

Identify where problems, bottlenecks, waste exist and where improvements are necessary,

Ensure that decision are based on facts, not on supposition, emotion, faith or Intuition, Track progress (show if improvements planned actually happened), Facilitate a more open and transparent communication and co-operation.

The outcomes of recent literature provide valuable research concept, approaches as well as case studies that analyze the presented metrics and system to evaluate the performance of supply chain as well as logistics e.g. production, distribution or inventory and implement performance measures (Gunasekaran and Kobu, 2007; Akyuz and Erkan, 2010; Gopal and Thakar, 2012). Hereby is to add that the existing research in performance measurement metrics and systems focuses on analyzing current and in the practice used performance measurement systems and in studying the measures (Beamon, 1999). All these argue for the complexity and difficulty in developing performance measures and systems for firms, organizations and their logistics activities. Gunasekaran et al. classified the performance measures on strategic, tactical and operational level to clarify the appropriate level of management authority and responsibility for performance (Gunasekaran et al. 2004); Lai et al. as well as Huang et al. present performance measures that are based on reliability, responsiveness, costs and assets (Lai et al., 2002; Huang et al. 2005); the work of Beamon shows three different measure categories like resources, output and flexibility (Beamon, 1999). Otto and Kotzab focus on the performance measure towards the goals of supply chains (Otto and Kotzab, 2003); Giannakis, Simatupang and Sridharan determine performance measures to evaluate the collaboration within a supply chain (Giannakis, 2007; Simatupang and Sridharan, 2005); a performance management process for delivery services is set by Forslund and Jonsson (Forslund and Jonsson, 2007). Furthermore, there are general methodologies developed to measure supply chain and logistics performance, namely the balance scorecard (Kaplan and Norton, 1992), supply chain council´s SCOR model, logistics scoreboard, activity-based costing and economic value added (EVA) (Lapide, 2000).

3. REVIEW METHOD

The literature review is defined as a content analysis for analyzing e.g. documents and identifying the conceptual content of the field by conducting a clear and systematic procedure (Mayring, 2003). Based on Seuring´s et al. recommendation a process model for content analysis that is developed by Mayring is used for the literature review in this research paper (Seuring et al., 2005). The process model for content analysis is as follows:

1) Material collection: The material to be collected is defined and delimitated. This might include taking a look at how the material emerged and the unit of analysis is defined.

2) Descriptive analysis: Formal aspects of the materials are assessed, e.g. the number of publications per year. This description forms the background upon which the theoretical analysis is conducted.

3) Category selection: Structural dimensions and related analytic categories are tested, which are to be applied in the literature review to structure the field. Structural

5

dimensions form the major topics of analysis, which cover various analytic categories, e.g. the year across a time period.

4) Material evaluation: the material is analyzed and stored according to the structural dimensions and categories built. This should allow identification of relevant issues and interpretation of results.

4. LITERATURE REVIEW

4.1. Material collection We collected the material during four months. The unit of analysis was set and different academic journal articles were analyzed. The collected articles were defined and delimited. The following subject terms and Boolean operators were searched for within six research databases namely EBSCOHOST, ABI/Informs, Academic Search Premier, textbooks, doctoral dissertations and Google Scholar for the time period from 1970 until 2012: ‘humanitarian logistics’, ‘humanitarian supply chains’, ‘performance’ ‘performance measure’, ‘performance measurement’, ‘performance evaluation’, ‘emergency logistics’, ‘emergency operations’, ‘disaster relief operations’, ‘performance model’, ‘performance system’, ‘KPIs’, ‘Indicators’, ‘Commercial logistics’, ‘business logistics’. Further, as articles were reviewed other cited articles were added.

4.2. Descriptive analysis The objective of the analysis was to position this research in the body of the literature. Hereby this step is used to categorize the articles. The main criteria were the number of publications and their distribution per year since 1970 to 2004 and since 2005 to 2012 after the disastrous logistics execution after the Indian Ocean tsunami in 2004 e.g. to evaluate the relevance of the topic performance measurement in the humanitarian logistics sector (figure 1). The second criterion was to identify what research method was used and in which journal the articles were published (table 1).

Altogether 27 empirical and analytical articles have been found. Out of these 27 a first subgroup of 15 articles are targeting an improvement-oriented approach. These articles deal with improving and promoting efficiency and effectiveness in the humanitarian logistics sector and exploring manifold logistics concepts to achieve the best logistics performance and are described exemplary as follows: Lin et al. propose an establishment of temporary depots in the affected areas by a disaster with the required vehicles and resources (Lin et al., 2012); Balcik and Beamon consider facility location decision for humanitarian relief chain as a response for quick-onset disaster, the presented results indicate the effects of pre- and post-disaster relief funding on relief chains performance in the response time and the proportion of demand satisfied (Balick and Beamon, 2008). Holguin-Veras et al. evaluate the performance of humanitarian logistics structure after the Port-au-Prince earthquake and defines three structure for comparative purposes (Holguin-Veras et al., 2012); Pettit and Beresford identified critical success factors for the humanitarian logistics sector (Pettit and Beresford, 2009); Scholten et al. investigated the agility concept in the context of humanitarian supply chains where the HO´s can use their limited resources more strategically (Scholten et al., 2010); Kovacz and Tatham explored the contribution of logistics skills and their effect on humanitarian logistics performance (Kovacz and Tatham, 2010); Blecken identified the state of practice of supply chain management in HO´s in order to develop a reference task model (Blecken, 2010); based on the example of the Tsunami

6

disaster in 2004 Whiting and Ayala-Öström investigated the means of advocacy that focus on promoting the logistics in providing humanitarian aid (Whiting and Ayala-Öström, 2009). The work of Gatignon et al. concentrates on design and implementation of supply chains. They evaluated the decentralized supply chains performance in responding to humanitarian disasters through an analysis of the International Federation of Red Cross and Red Crescent Societies (IFRC) operation during the Yogyakarta earthquake in 2006; they presented few measures that measure the efficiency of the decentralized supply chain (regional logistics unit (RLU)) of IFRC. Furthermore they introduced the resulted advantages such as time (reduced response time) and cost (reduced transport cost and reduced storage cost and other synergies) savings; hereby the new IFRC concept allow to increase the output and outcomes (provide service to a huge amount of beneficiaries) with the same input (amount of money) (Gatignon et al., 2010). A second subgroup of 12 articles are found with the specific focus on performance measurement systems 10 journals articles, one master thesis and one book chapter; these indicate a high relevance for the topic performance measurement in humanitarian logistics. Nevertheless, 12 articles about performance measurement and metrics in humanitarian logistics illustrate a low number compared to the counterpart commercial logistics (figure 1).

Figure 1: Allocation of the articles across the period 1970-2012 The topic performance measurement in humanitarian logistics gained a high importance in 2009. Based on the basic classification namely topic, methodology and characteristics the articles are categorized as presented in table 1. The topic performance measurement in humanitarian logistics as well as humanitarian supply chains is not sufficient explored, hereby is to assume that the complexity of humanitarian logistics activities, chaotic environment, limited access to data, unusual and uniqueness of logistics process and high challenges hinder the research in this topic. Case studies and conceptual models and frameworks are the most used methods. The articles indicate different new procedures, concepts, models and cases or field studies and these are supplemented by a small number of performance measures and performance measurement system and tools. The current research on performance measurement in humanitarian logistics illustrates an analysis and categories of performance measures and performance measurement systems that are already in use i.e. in HOs or in the commercial logistics (figure 2).

7

4.3. Category selection The selection of the categories was based on our analysis as well as the existing literature. The first category of the articles was based on general trends and issues of performance measurement in humanitarian logistics, which provide a comprehensive and general overview about the current used measures and system, the challenges and requirements to develop and implement accurate performance measures and systems as well as determine the relevance of the topic performance measurement in humanitarian logistics.

The second category is stage of disaster that indicates the process of disaster reliefs. The focus was on the used or developed performance measurement measures and systems in the three phases for specific disaster management preparation (i.e. evacuation plans, prepositioning of relief items, staff training etc.), immediate response (i.e. evacuation, rescue, distribution of relief items, coordination etc.) and reconstruction (i.e. reconstruction of buildings, development of the economy etc.) that were underlined by Kovacz and Spens (Kovacz and Spens, 2007). Haddow and Bullock or Tomasini and van Wassenhove mention the mitigation phase (Haddow and Bullock, 2004; Tomasini and van Wassenhove, 2009) that deals with mitigation and avoidance of disasters. In this research paper the mitigation phase is not considered due to the limited application only to some types of disasters.

The third category was based on research methodologies that are subdivided in two groups in analytical methods (conceptual model and mathematical model) and empirical methods (case study, interview, field study, review and survey). The fourth category is subdivided into the two subcategories performance measures and performance measurement system and the relationship with internal and external stakeholders (Zanjirani, 2009). The first subcategory performance measures and performance measurement system focus on what performance measures are used? Are these measure strategic and/or tactical and/operative? What are they used for? Does the performance measurements system consider qualitative and quantitative as well as financial and non-financial elements? The second subcategory characterizes the relationship with internal and external stakeholders and it investigates if the measures and system support the long-term and

Figure 2: Allocation of the articles by methodologies and characteristics

8

short-term objectives of the HOs and what benefit were provided? And do the measures and system match the HOs structure?

Table 1: Methodology and characteristics classification

Year Source Author Title Methodology Characteristics

Analytical Empirical PM indicators Description

Con

cept

ual m

odel

Mat

hem

atic

al m

odel

Cas

e/Fi

eld

stud

y/ In

terv

iew

Rev

iew

Surv

ey

stra

tegi

c

Tact

ical

Ope

rativ

e

Tech

nolo

gy

Proc

ess

PM sy

stem

2001 Transportation Research: Part A: Policy and Practice

Chang and Nojima

Measuring Post-disaster Transportation System Performance: The 1995 Kobe Earthquake in Comparative Perspective

x

x x

2006 Master Thesis (MIT)

Davidson Key Performance Indicators in Humanitarian Logistics

x

x x

x

2007 Journal of the Operational Research Society

Medina-Borja et al.

Large-scale data envelopment analysis (DEA) implementation: a strategic performance management approach

x

x

2007 Disaster Prevention and Management

Moe et al. Balanced scorecard for natural disaster management

x

x x x

x

2008 International Journal of Public Sector Management

Beamon and Balcik

Performance measurement in humanitarian relief chains

x

x x

x

2009 International Journal of Services Technology and Management

Blecken et al.

A humanitarian supply chain process reference model x

x

x

2009 Management Research News

Schulz and Heigh

Logistics performance management in action within a humanitarian organization

x

x x x

x

2009 International Journal of Risk Assessment & Management

Van Del Laan et al.

Performance measurement in humanitarian supply chains

x

x

x

2009 Management Research News

McLachlin et al.

Not-for-profit supply chains in interrupted environments The case of a faith-based humanitarian relief organisation

x

2010 IFIP Advanced in Information and Communication Technology (Book chapter)

de Leeuw Towards a reference mission map for performance measurement in humanitarian supply chains

x

x x x x

2011 Book chapter Tatham and Hughes

Humanitarian logistics metrics

x

2011 International Journal of public administration

Rietjens et al.

Measuring the immeasurable? The effects-based approach in comprehensive peace operations

x

x

x

9

4.4. Material evaluation The articles was analyzed and sorted according to the four categories namely general trends and issues of performance measurement in humanitarian logistics, stage of disaster, research methodologies and performance measures and performance measurement system and the relationship with internal and external stakeholder. The whole lecture of each article was done because a judgment and categorizing the articles based on reading the abstracts, title, keywords and conclusion is not enough. The 12 articles did not illustrate a consideration on the three different phases and/or different disaster types and/or different HOs type and their logistics structure and processes as presented in section 4.4.2. We determined that the humanitarian logistics and humanitarian supply chain terms are used sometimes interchangeably; hereby the presented performance measures and performance measurement system are developed for the humanitarian logistics field and do not cover the whole humanitarian logistics process. This produces a huge number and a variety of qualitative and quantitative performance measures that is only applicable in specific logistics processes and to specific humanitarian logistics structure as introduced in section 4.4.4. The reason is that they were not tested and adjusted in different HOs or in the three different phases. Furthermore the access to real data is hindered due to the HOs structure, chaotic environment as well as the complexity of the humanitarian logistics structure in such cases (e.g. Pettit, Beresford, 2005; Thomas, Kopczak, 2005; Tufingki, 2006). Based on the findings in the sections 4.4.1.-4.4.4. and table 1 the development and implementation of performance measures and performance measurement is generally a complex process. The performance measures, they are crucial for measuring the efficiency and effectiveness of logistics system and should base on the goals and key success factors of the humanitarian logistics field.

4.4.1. General trends and issues of performance measurement in humanitarian logistics

Following articles provide a comprehensive state of the art, challenges and trends as well as issues of performance measurement in humanitarian logistics. Beamon and Balcik compared the commercial supply chain purpose with humanitarian logistics purpose by defining goals and using the three-part performance measurement framework. Therefore they pointed the difficulties related to the measurement outcomes and impacts in humanitarian relief, and then the HOs strive to measure performance on inputs rather than outputs like in the non-profit organizations (Beamon and Balcik, 2008). The work of McLachlin et al. focused on using the managerial tools of commercial logistics in humanitarian logistics. Hereby the balanced scorecard was presented as a possible tool, but the HOs cannot implement this tool easily due do the different objectives. The commercial logistics focus on economic objectives and the humanitarian logistics focus on social and economic objectives. Further they highlighted that a peace HO does not measure their performance with a process measure such as tonnes per week, they measured their success of a project by its contribution to the promotion of their main activity i.e. peace (McLachlin et al., 2009). Tatham and Hughes investigate the challenges of developing the core measures of profitability that non-profit organizations are faced with. Further they show the high need of performance measures in the humanitarian logistics sector and the difficulties to capture robust data in such complex and chaotic environment with destructed information and communication network system. However, the authors outline that for a better understanding

10

of relief operations outcome or effectiveness measures have to be developed (Tatham and Hughes, 2011).

4.4.2. Stages of disaster management In the articles the three phases for the disaster management preparation (i.e. evacuation plans, prepositioning of relief items, staff training, etc.), immediate response (i.e. evacuation, rescue, distribution of relief items, coordination, etc.) and reconstruction (i.e. reconstruction of buildings, development of the economy, etc.) did not dominate in the research about the subject performance measures and performance measurement system. The researches investigated are based mostly on case studies and conceptual model where already different occurred disasters were analyzed. However, Rietjens et al. focused on performance measurement in peace operations by testing the Effected-Based Approach to Operations (EBAO) to measure the performance of sectors such as security, governance and socio-economic development and shows technical as well as fundamental i.e. lack of targets, benchmarks etc. challenges that facilitate the implementation of EBAO and the performance measurement of such operations (Rietjens et al., 2011).

Moe et al. had applied the balance scorecard to maximize the outcomes for the HOs in a case of natural disasters. They defined four perspectives such as donors’ perspective, the target beneficiaries’ perspective, the internal process perspective and the learning and innovation perspectives for the four phases preparedness, early warning, emergency relief, rehabilitation and recovery. For achieving the outcomes Moe et al. observed natural disaster management as a project management in which the government is a major stakeholder.

Hereby is to criticize that in such natural disasters the focus is not mainly on the government. The focus of HOs and generally humanitarian logistics is to alleviate suffering, vulnerability and save lives of affected people that occurred by disasters such as natural or man-made disasters (Moe et al., 2007).

4.4.3. Research methodologies To date the capture of data to measure logistics performance is hampered by the complex logistics structure of HOs structure and the challenging operations reliefs as well as the chaotic environment (e.g. Pettit, Beresford, 2005; Thomas, Kopczak, 2005; Tufingki, 2006). These are reasons that explain why the results of the majority of articles are explored by using case studies or conceptual models (Table 1 and figure 2). The majority of data used based on reports, magazines or data bases or few interviews with HOs staff that are not working in the field. Due to the time pressure that HO´s are faced with there is no possibility to contribute on surveys or long time period for an interview. Furthermore the steadily increasing interest in the field humanitarian logistics sector the academia execute manifold of interviews and surveys that HOs cannot effort it due to the limited staff, budget and other resources capacity. Chang and Nojima had used a mathematical model to evaluate the transportation system after the Kobe earthquake in 1995. Davidson; Medina-Borja et al.; Beamon and Balcik; Blecken et al.; Schulz and Heigh; Tatham and Hughes had applied conceptual model to identify performance measures and performance measurement system. Van Del Laan et al.; Mclachlin et al., de Leeuw had used empirical methods such as case studies. Rietjens et al. had conducted an interview to implement EBAO for measuring peace operations.

11

4.4.4. Performance measures and performance measurement system and the relationship with internal and external stakeholder

Based on interviews with professional staff at the IFRC Davidson had understood and structured the supply chain and the goals of IFRC. Davidson had examined the procurement and distribution of relief items data of the south Asia earthquake in 2005 from the Humanitarian Logistics Software. Based on these data Davidson had developed four performance measurement indicators such as appeal coverage (percent of appeal coverage, percent of items delivered) that determine how well the HOs is meeting its appeal for an operation; donation-to-delivery time indicate the delivery time of relief items in the destination country after a donor has assured to donate it; financial efficiency consist of three metrics, two metrics measure the budgeted prices compared to the actual prices paid for delivered items, the third incorporate the transportation cost of delivering the goods to the beneficiaries and precise the ratio of total transportation costs to total relief items costs for delivered goods at destination in time; assessment accuracy express the speed and the accuracy of pledged donation and delivered relief items to beneficiaries and how accurate the field staff have assessed the need of beneficiaries, therefore the measure is how much the operations final budget changed over time from the operations original budget (Davidson, 2006). Davidson had not only developed the indicators, she investigated and evaluated the scorecard at different point in time after a relief operation and indicate the relevance of an information system to capture data timely (Davidson, 2006). Beamon and Balcik discussed the difficulties of performance measurement in HOs because they tend to be input-oriented rather that output-oriented in measuring the performance. In 1999 Beamon developed a three-part performance measurement framework for the commercial supply chain that consist of resource metrics i.e. total cost of resources used, overhead costs, inventory investment, cost of supplies, annual costs etc., output metrics i.e. total amount of disaster supplies and total amount of disaster supplies each type that delivered to recipient, target fill rate etc. and flexibility metrics i.e. minimum response time, maximum proportion of emergency orders cycle and is applicable to humanitarian logistics. The presented measures of Beamon and Balcik are clear in itself but they were not empirically tested, only three measures (annual cost, response time and maximum proportion of emergency orders cycle) were used in mathematical modeling of inventory in relief operation.

The work of van der Laan et al. identified based on literature review necessary conditions for an effective performance measurement and used a case study to explore whether these conditions are met or not. Van der Laan et al. emphasized that the immense challenge in implementing a performance measurement and developing and determining performance measures lies on the accuracy of the data. Van der Laan et al. recommended following non-financial service performance measures if a HOs strategic focus on service rather than cost performance and who are faced with the lack of accurate data. The set of performance measures are accuracy of stock records gives information about what to order when; realized service level is categorized in monitoring responsiveness and indicate the percentage of relief items that were delivered in the promised time and with stock efficacy a calculation of the expected inventory turnover rates can be conducted based on the chosen policy parameters because different product types require different order policies and/or different parameter settings regarding order size and safety stock (van der Laan et al. 2009). The main purpose of the article of Schulz and Heigh is to present the design and testing procedure that consists of four phases (1. first draft of indicators based on the balanced scorecard concept, 2. tool improvement based on RLU feedback, 3. piloting, 4. continuous review and training) of the “Development Indicator Tool” as well as the objective of the tool that developed by the IFRC Red Crescent Societies to monitor the improvement of their

12

decentralized supply chain. Furthermore they showed the selected set of performance measures and discuss the importance of integration of stakeholders that is relevant for the relationship with internal and external stakeholder and lead to the success of the process (Schulz and Heigh, 2009). The used balanced scorecard and the presented set of performance measures in % that are found in the article of Schulz and Heigh indicate four perspectives namely customer service i.e. delivery performance, stocks managed by service agreements etc., financial control i.e. deviation from unit budget, service turnover versus plan, cost recovery etc.), process adherence i.e. available stock capacity to supply 5,000 families in 48 hours, relief stock turnover rate etc. and innovation and learning i.e. staff development, on time reporting etc. (Schulz and Heigh, 2009). Also de Leeuw work presents the balanced scorecard with the perspectives of the humanitarian supply chain based on the literature review in the humanitarian logistics and supply chains field. The four perspectives presented in the balanced scorecard; customer (i.e. speed of delivery, quantity etc.), internal (environmental and compliance issues, the use of pledged donation etc.), learning and growth (required knowledge, staff development etc.), financial (efficiency, flow of donations, track budget etc.); are tested based on case studies (de Leeuw, 2010). The article of Blecken et al. presents a conceptual research, it specify an outline of the categories and characteristics of the required indicators. This research delivers some examples of performance measures that are subdivided in three categories responsiveness i.e. donation-to-delivery-time, order fulfillment cycle, supply chain adaptability, assessment accuracy, output i.e. cost efficiency, on time delivery, target population coverage rate, order fulfillment rate) and resources i.e. human resources efficiency and utilization of fleet and warehouse (Blecken et al., 2009).

5. FUTURE RESEARCH

The review of a few numbers of literatures about performance measurement in humanitarian logistics indicates that researches try to construct performance measurement systems and metrics for the humanitarian logistics sector. In fact, the field performance measurement in humanitarian logistics is still in its infancy. The analysis of the paper leads to new directions. The research should be designed around the following questions as broad thematic suggestions:

Which methods and areas of business science (controlling, IT, change management, value chain management) can be used to overcome the identified difficulties in establishing performance measurement systems in practice?

Which types of organizations can be categorized regarding performance measurement and have to be subject to adaptions?

Are there national/cultural/regional influences regarding performance measurement in humanitarian logistics or is it feasible to establish a coherent standardized international concept?

How we can improve the inter-agency and donor collaboration? The link between funding and humanitarian logistics is essential, which measures have

to be determined in this area? How we can improve information flow and forecasting e.g. to avoid the nonexistence of

data?

13

Should the implementation of performance measurement consider the three phases for the disaster management preparation (i.e. evacuation plans, prepositioning of relief items, staff training, etc.), immediate response (i.e. evacuation, rescue, distribution of relief items, coordination, etc.) and reconstruction (i.e. reconstruction of buildings, development of the economy, etc.)

6. CONCLUSION

The presented literature review revealed many interesting facts but also gaps in the research output regarding performance measurement in humanitarian logistics so far. A significant limitation of this study is that validity and reliability in qualitative research is controversial. Besides this general method restriction, the following aspects can be highlighted from the analysis:

The main body of publications regarding performance measurement in humanitarian logistics emerged after the 2004 Indian Ocean tsunami disaster.

The total number of research articles in this specific field is still very low and should be increased, especially in comparison to other areas of logistics research.

A systematic approach as provided here in order to categorize research output regarding performance measurement in humanitarian logistics does not yet exist.

Furthermore three articles present the balanced scorecard (McLachlin et al., 2009; Schulz and Heigh, 2009; de Leeuw, 2010) as a managerial tool that can be adopted in the humanitarian logistics setting with determining suitable objectives. Based on the results of the literature review and the different presented performance measures (appendix 1), we suggest a first draft of the balanced scorecard for the humanitarian logistics sector that have to be tested in a HO by using the real-life data and evaluate the HO´s objectives. The balanced scorecard should cover four levels such as organizational level, network level, project level and process level. The organizational level measures the achievement of the target of an organization e.g. donations and represents their strategy. The network level is characterized by knowing and understanding the strategy of the complete supply chain network. It deals with measuring the common targets of the overall supply chain network by applying a common method. At the project level the each process of the project can be measured. The project in the humanitarian logistics can be subdivided in three categories based on the phases for the disaster management preparation, immediate response and reconstruction. The process level measure the logistics process e.g. transportation, warehouse and inventory.

The appendix 1 presents different key performance measures that are selected from the literature review. We have categorized each key performance measures to organizational level, project level, process level and network level. We find out that the most developed key performance measures can be classified to organizational and process level. Performance measures at network and project are rare. The determining of performance measurement system and measures in the humanitarian logistics sectors at organizational level, project level and process level can be developed based on the key success factors, phases for the disaster management (preparation, immediate response and reconstruction) and goals of each HO. In this respect, the network level seems to be a challenging issue for the humanitarian logistics sector due to a lack of collaboration and partnership in this sector. Furthermore it requires an extensive coordination and communication between the members of the supply chain network. Currently such collaboration and partnership at this dimension does not exist compared to the counterpart commercial logistics. Common goal of the overall supply chain

14

network is relevant to determine performance measures and these require that the members of the supply chain network have similar key success factors.

ACKNOWLEDGEMENTS

This research contribution is connected the German national research cluster for logistics (Effzienzcluster “LogistikRuhr”, www.effizienzcluster.de), funded by the federal department for research (BMBF), project number 01|C10L19D (“WiWeLo”). The authors are grateful for this research support.

REFERENCES

Akyuz, G.A. and Erkan, T. E. (2010), “Supply Chain performance measurement: a literature review“, International Journal of Production Research, Vol. 48 No. 17, pp. 5137-5155.

Beamon, B. M. (1999), “Measuring Supply Chain Performance”, International Journal of Operations and Production Management, Vol. 19 No. 3, pp. 275-292.

Beamon, B. M. and Balcik, B. (2008), “Performance Measurement in Humanitarian Relief Chains”, International Journal of Public Sector Management, Vol. 21 No. 1, pp. 4-25.

Blecken, A. (2010), Humanitarian Logistics. Modelling Supply Chain Process of Humanitarian Organisations, Bern.

Blecken, A. (2010), Humanitarian Logistics: Modelling Supply Chain Processes of Humanitarian organisations, Haupt Verlag, Bern et al.

Blecken, A., Hellingrath, B., Dangelmeier, W. and Schulz, S.F. (2009), “A humanitarian supply chain process reference model”, International Journal Services Technology and Management, Vol. 12 No. 4., pp. 391-413.

Cassidy, W.B. 2003. A logistics lifeline, Traffic World. October 27, p.1.

Chang, S.E. and Nojima, N. (2001), “Measuring post-disaster transportation system performance: the 1995 Kobe earthquake in comparative perspective”, Transportation Research Part A: Policy and Practice, Vol. 35 No. 6, pp. 475–494.

Chow, G., Heaver, T.D. and Henriksson, L.E. (1994) “Logistics performance: definition and measurement”, International Journal of Physical Distribution & Logistics Management, Vol. 24 No. 1, pp. 17-28.

Davidson, A. L. (2006), Key Performance Indicators in Humanitarian Logistics, Master Thesis, Massachusetts Institute of Technology, Boston.

De Leeuw, S. (2010), “Towards a reference mission map for performance measurement in humanitarian supply chains”, IFIP International Federation for Information Processing, pp. 181-188.

Dupont, A. and Pearman, G. (2006), Heating up the planet: climate change and security, Lowy Institute Paper 12. Double Bay, The Lowy Institute.

Forslund, H. and Jonsson, P. (2007), “Dyadic integration of the process management process management process”, International Journal of Physical Distribution & Logistics Management, Vol. 37 No. 7, pp. 546-567.

15

Gatignon, A., van Wassenhove, L.N., Charles, A. (2010), “The Yogyakarta earthquake: Humanitarian relief through IFRC`s decentralized supply chain”, International Journal Production Economics, Vol. 126, pp. 102-110.

Giannakis, M. (2007), “Performance measurement of supplier relationships”, Supply Chain Management International Journal, Vol. 12 No. 6, pp. 400-411.

Gleason, J.M. and Barnum, D.T., (1986), “Toward Valid Measures of Public Sector Productivity: Performance Measures”, Urban Transit, Management Science, Vol. 28 No. 4, pp. 379-386.

Gopal, P.R.C. and Thakar, J. (2012), “A review on supply chain performance measures and metrics: 2000-2011”, International Journal of Productivity and Performance Management, Vol. 61 No. 5, pp. 518-547.

Guha-Sapir, D., Vos, F., Below, R. and Ponserre, S. (2012), Annual Disaster Statistical Review 2011: The Numbers and Trends, published by the Centre for Research on the Epidemiology of Disasters (CRED) Brussels.

Gunasekaran, A. and Kobu, B. (2007), “Performance measures and metrics in logistics and supply chain management: a review of recent literature (1995-2004) for research and applications”, International Journal of production Research, Vol. 45 No. 12, pp. 2819-2840.

Gunasekaran, A., Patel, C. and McGaughey, R.E. (2004), “A framework for supply chain performance measurement”, International Journal of Production Economics, Vol. 87, pp. 333-347.

Haddow, G.D. and Bullock, J.A. (2004), Introduction to emergency management, Butterworth-Heiemann, Amsterdam.

Holguín-Veras, J., Jaller M. and Wachtendorf, T. (2012), “Comparative performance of alternative humanitarian logistics structures after the Port-au-Prince earthquake: ACEs, PIEs, and CANs”, Transportation Research Part A, Vol. 46, pp. 1623-1640.

Huang, S.H., Sheroan, S.K. and Keskar, H. (2005), “Computer-assisted supply chain configuration based on supply chain operations reference (SCOR) model”, Computers & Industrial Engineering, Vol. 48, pp. 377-394.

Jahre, M. and Heigh, I. (2008), “Does failure the current constrains in Funding promote failure in humanitarian supply chains?”, International Journal Supply Chain Forum ,Vol. 9 No. 2, pp. 44-45.

Kaplan, R.S. (1990), Measures for manufacturing excellence, Harvard Business School Press, Boston.

Kaplan, R.S. and Norton, P.D. (1992), “The balanced scoreboard measures that drives performance”, Harvard Business, Vol.70 No. 1, pp. 71-79.

Kovács, G. and Spens, K.M. (2007), “Humanitarian logistics in disaster relief operations”, International Journal of Physical Distribution & Logistics Management, Vol.37 No.2, pp. 99-114.

Kovacz, G. and Tatham, P. (2009), “Humanitarian performance in the light of gender”, International Journal of Productivity and Performance Management, Vol. 2 No. 58, pp. 174-187.

16

Kovacz, G. and Tatham, P. (2010), “What is Special about a Humanitarian Logistician? A Survey of Logistics Skills and Performance”, Supply Chain forum an International Journal, Vol. 11 No.3, pp. 32-41.

Lai, K.H., Ngai, E.W.T. and Cheng, T.C.E. (2002), “Measures for evaluating supply chain performance in transport logistics”, Transport Research Part E, Vol. 38, pp. 439-456.

Lapide, L. (2000), “What About Measuring Supply Chain Performance?”, available at http://ftp.gunadarma.ac.id/idkf/idkf-wireless/aplikasi/e-commerce/lapide.pdf.

Lin, Y.H., Batta, R., Rogerson, P.A., Blatt, A. and Flanigan, M. (2009), “Application of a humanitarian relief logistics model to an earthquake disaster”, Transportation Research Record: Journal of the Transportation Research Board.

Long, D.C. (1997), “Logistics for disaster relief: engineering on the run”, IIE solutions, Vol. 29 No.6, pp. 26-29.

McLachlin, R., Larson, P.D. and Khan, S. (2009), “Not-for-profit supply chain in interrupted environments: the case of a faith-based humanitarian relief organization”, Management Research News, Vol. 32 No. 11, pp. 1050-1064.

Medina-Borja, A., Pasupathy, K.S. and Triantis, K. (2007), “Large-scale data envelopment analysis (DEA) implementation: a strategic performance management approach”, The Journal of the Operational Research Society, Vol.58 No.8, pp.1084-1098.

Mentzer, J.T. and Konrad, B.P., “An Efficiency/Effectiveness Approach to Logistics Performance Analysis”, Journal of Business Logistics, Vol. 12 No. 1, pp.33-61.

Moe, T.L., Gehbauer, F., Senitz, S. and Mueller, M. (2007), “Balanced scorecard for natural disaster management”, Disaster Prevention and Management, Vol.16 No.5, pp. 785-806.

Neely, A., Gregory, M. and Platts, K. (1995), “Performance measurement systems design: a literature review and research agenda”, International Journal of Operations & Production Management, Vol.15 No.4, pp. 80-116.

Oloruntoba, R. (2005), “A wave of destruction and the waves of relief: issues, challenges and strategies”, Disaster Prevention and Management, Vol. 14 No. 4, pp. 506-521.

Oloruntoba, R.and Gray, R. (2006), “Humanitarian aid: an agile supply chain?”, International Journal of Supply Chain Management, Vol. 11 No. 2, pp. 115-120.

Pettit, S.J., Beresford, A. (2005), “Emergency relief logistics: an evaluation of military, non-military and composite response models”, International Journal of Logistics: Research & Applications, Vol. 8 No. 4, pp. 313-331.

Pettit, S. and Beresford, A. (2009), “Critical success factors in the context of humanitarian aid supply chains”, International Journal of Physical Distribution & Logistics Management, Vol. 39 No. 6, pp. 450-468.

Rietjens , S., Soeters, J. and Klumper, W. (2011), “Measuring the immeasurable? The Effects-Based Approach in Comprehensive Peace Operations”, International Journal of Public Administration, Vol. 34, pp. 329-338.

Scholten, S., Sharkey Scott, P. and Fynes, B. (2010), “(Le)agility in humanitarian aid (NGO) supply chains”, International Journal of Physical Distribution & Logistics Management, Vol. 40 No. 8/9, pp. 623-635.

17

Schulz, S.F. and Heigh, I. (2009), "Logistics performance management in action within a humanitarian organization", Management Research News, Vol. 32 No. 11, pp.1038 – 1049.

Seuring, S., Mueller, M., Westhaus, M. and Morana, R. (2005), “Conducting a Literature Review-The Example of Sustainability in Supply Chains”, Kotzab, H., Seuring, S., Müller, M., Reiner, G. (Eds.), Research methodologies in Supply Chain Management, Physica, Heidelberg, pp. 91-106.

Sharahi, S. and Abedian, M. (2009), “Performance measurement”, Zanjirani Farahani, R. et al. (Eds.), Supply Chain and logistics in National, International and Governmental Environement, Springer, Berlin Heidelberg, pp. 21-42.

Simatupang, T.M. and Sridharan, R. (2005), “The collaboration index: a measure for supply chain collaboration”, International Journal of Physical Distribution & Logistics Management, Vol. 35 No. 1, pp. 44-62.

Sink, D.S., Tuttle, T.C. and De Vries, S.J. (1984), “Productivity Measurement and Evaluation: What is Available?”, National Productivity Review, Vol.4 No. 13, pp. 265-387.

Tatham, P. and Hughes, K. (2011), “Humanitarian logistics metric where we are and how we might improve”, Christopher, M. and Tatham, P. (Eds.), Humanitarian Logistics: Meeting the Challenge of Preparing for and Responding to Disasters, Philadelphia, London, pp. 249-263.

Thomas, A. and Kopczak, L.R. (2007), “Life-Saving Supply Chain – Challenges and the Path Forward”, Lee. H.L. and Lee, C.Y. (Eds.), Building Supply Chain Excellence in Emerging Economies. Springer Science & Business Media. New York, pp. 93-111.

Thomas, A.S. and Kopczak, L.R. (2005), From logistics to supply Chain Management: The Path Forward in the Humanitarian Sector. Fritz Institute. San Francisco, CA.

Tomasini, R., van Wassenhove (2009), Humanitarian Logistics, Palgrave MacMillan, Basingstoke.

Tufinkgi, P. (2006), Logistik im Kontext internationaler Katastrophenhilfe: Entwicklung eines logistischen Referenzmodells für Katastrophenhilfe, Haupt Verlag, Bern 2006.

UN (2006), “Strengthening of the coordination of emergency humanitarian assistance of the United Nations”, available at http://www.un.org/docs/ecosoc/documents/2006/reports/ECOSOC%202006%20-%20Humanitarian%20Coordination.pdf.

Van Der Laan, E.A., De Brito, M.P. and Vergunst, D.A. (2009), “Performance measurement in humanitarian supply chains”, International Journal of Risk Assessment and Management, Vol. 13 No. 1, pp. 22-45.

Van Wassenhove, L.N. (2006), “Humanitarian aid logistics: supply chain management in high gear”, Journal of Operational Research Society, Vol. 57 No. 5, pp. 475-489.

Whiting, M.C. and Ostrom, B.E.A. (2009), “Advocacy to promote logistics in humanitarian aid”, Management Research News, Vol. 32 No. 11, pp. 1081-1089.

Widera, A. and Hellingrath, B. (2011), “Performance Measurement Systems for Humanitarian Logistics”, Proceedings of the 23rd Annual NOFOMA Conference, Harstad, Norway, pp. 1327-1342.

18

APPENDIX 1 Table 2: Performance measures from current literature Year References Performance measures Description Level

Org

anis

atio

nal

Proc

ess

Proj

ect

Net

wor

k

2006 Davidson

Appeal coverage

No. of items required that have been covered by donations; the no. of items reaching beneficiaries as a percentage of the total no. requested in appeal x

Donation-to-delivery-time

Capture the throughput speed of goods in the supply network,; average lead time from donation release to final delivery to beneficiaries x

Financial efficiency Cost of providing goods to beneficiaries x

Assessment accuracy Measurement of the extent to which the budget for a particular disaster changes over time x

2008 Beamon and Balcik

Resources

Total cost; overhead cost; total cost of distribution; inventory investment; inventory obsolescence; fund received per time period x

Output

Total amount of disaster supplies; total amount of disaster supplies of each type; total amount of disaster supplies delivered to each recipient; target fill rate achievement; stock-out probability; no. of backorders; no of stock-outs; average backorder level; average response time; minimum response time x

Flexibility

No. of individual units of supplies that an organization can provide in time period; Minimum response time; Mix of different type of supplies that the relief chain can provide in a specified time period; no. of individual units of supplies that an organization can provide in time period x

2009 Blecken et al.

Order fulfilment cycle time Average lead time from field order to final delivery x

Supply chain adaptability Percentage of upside capacity x

Assessment accuracy Percentage of unnecessary supplies; no. of return; spoilage rate x

Order fulfillment rate Percentage of order fulfillment x

On time delivery Percentage of on time deliveries x

Cost efficiency

Transportation cost versus total product costs; bugeted versus actual costs, percentage of price premium on product cost x

Human resources efficiency Average working hours per day; staff turnover rate; unfullied postings time x

OS utilisation Degree of warehouse utilisation; degree of fleet utilisation x

2009 van der Laan et al.

Accuracy of stock records What to ordet when x

Realised service level Percentage of demand; insufficent replenishments; delivery delay; order lead time x

Stock efficacy Expected inventory turnover rates

19

2009 Schulz and Heigh

Cus

tom

er se

rvic

e Delivery performance Identification of lead time issues x

Orders with agreed delivery time in days

Force customers and procurement to plan needs and agree on delivery date x

Stocks managed by service agreements Drive use of service agreements x

Monthly reports to customers on time Drive process stabilization for customer reports x

Fina

ncia

l co

ntro

l Deviation from unit budget Deviation from project budget x Deviation from project budget Stay within budget lines x Service turnover versus plan Service income versus plan x Cost recovery Drive invoice management, timely cost recovery x

Proc

ess a

dher

ence

Available stock capacity to supply 5,000 families in 48 h (segmented by ownership of stock) x

Available stock capacity to supply 15,000 families in 14 days x

Relief stock turnover rate Drive utilization and fix cost digression of warehouse x

Procurement transactions using Humanitarian Logistics Software Promote usage of HLS software x

Operational vehicles using It software fleet wave Promote usage of fleet wave software x

Average procurement cost per transaction Drive cost efficiency and process improvement x

Average warehouse cost per m2 stored Drive cost efficiency and process improvement x

Average % of transport cost of total order cost for airfreight Drive cost efficiency and process improvement x

Average % of transport cost of total order cost for other transport mode Drive cost efficiency and process improvement x

Inno

vatio

n an

d le

arni

ng

Staff development Drive staff development through use of development tool x

Actual project time versus planned project time Drive good project management x

On time reporting Drive process stabilization for internal reports x 2010 de Leeuw

Cus

tom

er

Product and service level speed delivery; reliable delivery; sufficiant quantity; high availibility; high quality x

Customer relationship (to donors) Ensure efficient and effective processes; provide feedback and information x

Customer relationship (to intermediaries)

Provide relvant and timely service; provide feedback and information x

Customer relationship (to beneficiaries) Provide relevant and timely service x Image Provide sustainbale relief aid x

Inte

rnal

Operations management Create operational excellence in processes from needs assessment to delivery x

Donor management Target, acquire and retain donors x Partner management Build partnerships with local government x

Innovation Identify, manage, and develop new product and services x

Regulatory and social Ensure compliance with regulations x

Lear

ning

and

G

row

th Human capital

Build and expand supply chain skills and comeptencies x

Information capital Develop, utilise and capitalize on supply chain ICT x

Organisational capital Nurture local leadership and foster team work internally and with partners x

Fina

ncia

l Bugeting Develop, monitor and adjust budgets x Funding management Ensure steady and timely flow of donations x

Cost management Ensure efficient use of funds in a transparent manner x

#