method of evaluating the impact of erp implementation critical success factors – a case study in...

25
This article was downloaded by: [UQ Library] On: 06 November 2014, At: 16:46 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Enterprise Information Systems Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/teis20 Method of evaluating the impact of ERP implementation critical success factors – a case study in oil and gas industries Gordana Gajic a , Stevan Stankovski b , Gordana Ostojic b , Zdravko Tesic b & Ljubomir Miladinovic c a Petroleum Industry of Serbia , Narodnogfronta 12, Novi Sad , Serbia b Department for Industrial Engineering and Management , Faculty of Technical Sciences, University of Novi Sad , Trg Dositeja Obradovica 6, Novi Sad , Serbia c Faculty of Mechanical Engineering, Belgrade University , Kraljice Marije 16, 11120 Belgrade 35, Serbia Published online: 24 May 2012. To cite this article: Gordana Gajic , Stevan Stankovski , Gordana Ostojic , Zdravko Tesic & Ljubomir Miladinovic (2014) Method of evaluating the impact of ERP implementation critical success factors – a case study in oil and gas industries, Enterprise Information Systems, 8:1, 84-106, DOI: 10.1080/17517575.2012.690105 To link to this article: http://dx.doi.org/10.1080/17517575.2012.690105 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,

Upload: ljubomir

Post on 13-Mar-2017

218 views

Category:

Documents


4 download

TRANSCRIPT

This article was downloaded by: [UQ Library]On: 06 November 2014, At: 16:46Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Enterprise Information SystemsPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/teis20

Method of evaluating the impact of ERPimplementation critical success factors– a case study in oil and gas industriesGordana Gajic a , Stevan Stankovski b , Gordana Ostojic b ,Zdravko Tesic b & Ljubomir Miladinovic ca Petroleum Industry of Serbia , Narodnogfronta 12, Novi Sad ,Serbiab Department for Industrial Engineering and Management , Facultyof Technical Sciences, University of Novi Sad , Trg DositejaObradovica 6, Novi Sad , Serbiac Faculty of Mechanical Engineering, Belgrade University , KraljiceMarije 16, 11120 Belgrade 35, SerbiaPublished online: 24 May 2012.

To cite this article: Gordana Gajic , Stevan Stankovski , Gordana Ostojic , Zdravko Tesic & LjubomirMiladinovic (2014) Method of evaluating the impact of ERP implementation critical successfactors – a case study in oil and gas industries, Enterprise Information Systems, 8:1, 84-106, DOI:10.1080/17517575.2012.690105

To link to this article: http://dx.doi.org/10.1080/17517575.2012.690105

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,

systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Method of evaluating the impact of ERP implementation critical success

factors – a case study in oil and gas industries

Gordana Gajica, Stevan Stankovskib, Gordana Ostojicb*, Zdravko Tesicb andLjubomir Miladinovicc

aPetroleum Industry of Serbia, Narodnogfronta 12, Novi Sad, Serbia; bDepartment for IndustrialEngineering and Management, Faculty of Technical Sciences, University of Novi Sad, TrgDositeja Obradovica 6, Novi Sad, Serbia; cFaculty of Mechanical Engineering, Belgrade

University, Kraljice Marije 16, 11120 Belgrade 35, Serbia

(Received 24 November 2010; final version received 28 April 2012)

The so far implemented enterprise resource planning (ERP) systems have in manycases failed to meet the requirements regarding the business process control,decrease of business costs and increase of company profit margin. Therefore,there is a real need for an evaluation of the influence of ERP on the company’sperformance indicators. Proposed in this article is an advanced model for theevaluation of the success of ERP implementation on organisational andoperational performance indicators in oil–gas companies. The recommendedmethod establishes a correlation between a process-based method, a scorecardmodel and ERP critical success factors. The method was verified and tested ontwo case studies in oil–gas companies using the following procedure: the modelwas developed, tested and implemented in a pilot gas–oil company, while theresults were implemented and verified in another gas–oil company.

Keywords: enterprise resource planning (ERP); ERP system implementation;critical success factors (CSF); key performance indicators (KPI)

1. Introduction

A large number of companies in today’s market face global competition anddemands for higher profits – all in a constantly changing environment. In order forthe companies to be able to respond to challenges, and transform their businesseswith minimum impact on profitability and transparency, it is ideal to invest inreliable information technologies and specific software solutions for a particularbranch of industry (Stefanou 2001). The integration problem is crucial today becausethe applications and the information systems of companies increasingly need to worktogether (Izza 2009). Enterprise resource planning (ERP) systems are informationsystems which integrate all business information within an organisation, allowingprocess control and an integral information system (Fontana and Neto 2009, Niuet al. 2011). Through implementation of ERP system, companies get an efficientsoftware solution for the control of operative and functional processes, as well asplanning and consolidation of company resources (Peslak 2006).

*Corresponding author. Email: [email protected]

Enterprise Information Systems, 2014Vol. 8, No. 1, 84–106, http://dx.doi.org/10.1080/17517575.201 .

© 2013 Taylor & Francis

2 690105

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Besides the significant advantages of the implementation of ERP systems, thereare some drawbacks as follows: ERP implementation can take many years tocomplete and can cost tens of millions of dollars to companies and even withsignificant investments in time and resources, there is no guarantee of a successfuloutcome. Mabert et al. (2003) concluded that in companies that tended to haveshorter completion times and smaller budgets the implementations have becomemore ‘efficient’.

For the companies which have already implemented ERP systems, it is of utmostimportance to establish the influence of ERP system and information technologieson the organisational and business performances of their systems (Fontana and Neto2009, Sun and Bhattacherjee 2011). The assessment of the influence of informationtechnologies on organisational characteristics of a company is not a simple task(Maksimovic et al. 2010), bearing in mind the lack of adequate methods and toolsfor such a task (Kohli and Grover 2008, Tan and Takakuwa 2011). Themethodological assessment of the influence of ERP system on the key performanceindicators (KPI) of a system should rely on the measurements of relevant parametersand indicators of the success in implementation of an ERP project (Irani 2010).

Proposed in this article is a method for the assessment of ERP implementationsuccessfulness regarding the organisational and operational characteristics ofbusiness in oil–gas companies. The assessment shall take into consideration theirorganisational environment and assess the performances from several aspects. Thisimproved method is the result of two case studies conducted in two oil–gascompanies which have already implemented ERP systems in their production. Thesuccessfulness of ERP system shall be regarded as the relevant measure of successfulimprovement of business performances of the two business systems. The success ofERP system implementation is determined by the influence of the critical successfactors (CSFs) which are defined during ERP system implementation (Plant andWillcocks 2007, Snider et al. 2009). Many CSFs have been identified in literature(Esteves and Pastor 2000, 2006, Loh and Koh 2004, Nah and Delgado 2006, Plantand Willcocks 2007, Li et al. 2008a,b, Fryling 2010). However, this study focuses on15 specific CSFs, considering their pronounced influence on the implementation ofERP system modules, aimed specifically at oil–gas companies (as defined in thefeasibility study for the project of ERP system implementation in a pilot company –which shall be referred to as the ABC Company). Performance indicators (PIs)which were used in this research were defined by the management of the ABCCompany based on their business strategy.

2. Theoretical basis – a literature review

2.1. Methods for assessment of the influence of the implemented ERP system on thecharacteristics of the business system

According to the findings of Uwizeyemungu and Raymond (2009), the assessment ofthe influence of information technologies on the system performance can be groupedinto four categories: (1) causal models, (2) contingency models, (3) process modelsand (4) scorecard models. Causal models, better known as variant models, tend toestablish a cause–effect relationship between the investment into informationtechnologies and the organisational characteristics of a company. The contingencymodels simulate the influence of information technologies on the organisationalcharacteristics of a company through assessment of the degree of fitting the

85Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

information technologies into some other aspects of organisation, such as strategy,structure and business processes. The concept of the process models is based onreviewing the influence of information technologies on system characteristicsthrough a series of connected effects upon the business characteristics of a system,as well as the assessment of its losses (Salmela 2008). The scorecard models (Kaplanand Norton 1996) emphasise multiple dimensions of system characteristics and theirapplication in the assessment of the influence of information technologies in general,as well as the influence of ERP system in particular.

In this article, the influence of ERP system is assessed based on a model which isa combination of process and scorecard model (Uwizeyemungu and Raymond2009). Scorecard model is used for its ability to define multi-dimensional features ofsystem organisational characteristics on a global level, as well as the multiplepotential effects of ERP implementation. On a global level, a score is generatedwhich represents the sum of local impacts of ERP implementation on the system’sorganisational characteristics, measured by KPI. Such an approach to assess ERPinfluence underscores the transformation of ERP investment into the added value fora company, considering the multiple aspects and dimension of the added value. Theprocess methods allow the connection between the ERP system and theorganisational characteristics, through the automated information and transforma-tion effects that they exert on the operational and control processes (Uwizeyemunguand Raymond 2009).

2.2. Critical success factors of ERP implementation success

Enterprise resource planning implementation projects are considered risky, firstlybecause the project is large in scale and secondly because it causes changes in theindividuals’ tasks (Vilpola 2008). The success of the very implementation of an ERPsystem depends on the influence of the CSFs. In this article, the CSFs which hadbeen defined until the 2006 (Estaves and Pastor 2006, Nah and Delgado 2006, Soja2006, Plant and Willcocks 2007, Wickramasinghe and Gunawardena 2010), whichcoincides with the period of ERP implementation in the two reviewed companies,were reviewed. According to the relevant sources for the period until 2006 (Estavesand Pastor 2006, Nah and Delgado 2006), the critical factors of ERP implementa-tion are classified into the group of strategic and tactical factors, as shown in Table 1.

Some specific CSFs of successful ERP implementation have been identified in theoil–gas companies, considering the fact that they use the modules which are specificto oil–gas industrial companies (as defined in the ERP implementation feasibilitystudy for the ABC pilot company, by the ERP implementation team). The specifickey parameters of success are:

. Complete finances for the whole company.

. Budgeting for Fiscal year for the whole company on a single, integrated basis.

. Cost planning and collection on cost centres and for defined measures.

. Legacy systems connected via XI interfaces.

. Centralised payment (in-house cash).

. Credit limits check on a corporate level.

. Complete inventory (volumes and valuation).

. Closed purchasing loop (from requisition to payment).

. Harmonised material master records.

86 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

. IS-Oil basic functionality.

. Integrated and harmonised processes (from order to cash).

. Closed loop for asset management lifecycle.

. Closed loop for demand and supply chain planning.

. Closed loop order-to-cash including service station network.

. Well level production and revenue analysis.

In addition, one of the most important strategic parameters is the influence of topmanagement on ERP system implementation, considering that top managementdefines the strategic business goals in a company which impact the subsequentdefinition of the indicators of business system characteristics (Dong et al. 2009). Thementioned CSFs were identified through questionnaires filled out by top managersand members of the project implementation team: IT managers, production,financial and logistics managers, process owners, key users, end users of ERP systemand external implementers of ERP system. The analysis of interview data was cross-linked with the list of KPIs which were affected by implementation of modulesspecific for oil and gas companies, as well as with the list of mandatory functions of aspecific ERP module which must be implemented as a prerequisite for theimplementation of the next module. Through each step of the analysis, the time ofbusiness process occurrence and project milestones were taken into consideration,since they influence the execution of ensuing steps. The data collected byquestionnaires were checked by triangularisation in order to increase the reliabilityof survey. In addition, in the course of the definition of the CSFs, projectdocumentation, presentations and meeting records were also available for reviewing.

2.3. Business PIs

Through ERP implementation, business processes are subject to changes orcancellations, or may be introduced (Aldin and de Casare 2011). Every change in

Table 1. Critical ERP implementation success factors.

Strategic critical success factors Tactical critical success factors

Top management commitment and support User training and educationBusiness process reengineering Use of ERP’s consultantsUse of project management to manage

implementationMonitoring and evaluation of performanceManagement structure

Effective organisational change management Formalised project plan/scheduleInter departmental cooperation and

Change management culture and program communicationClear goals, focus and scope (business plan and

vision)Vendor package selectionAppropriate business and IT legacy systems

Selecting the right team (competence) Use of vendors’ development toolsProject champion role Effective communicationVendor partnership User participationAvoidance customisation Technical and business knowledgeAdequate ERP implementation Integration of the systemAdequate ERP version Software development, testing and

troubleshootingLegacy systems knowledge

87Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

the business process is aimed at providing an added value for the company business.This added value is measured by business performance indicators. In oil–gascompanies, the business performance indicators are projected based on the businessstrategy, which also defines the goals of ERP implementation. Performanceindicators were determined using a process-oriented Supply Chain OperationsReference (SCOR) model which is based on five business processes: Plan, Source,Manufacture, Delivery, Return on the company’s map of supply chain (Supply-Chain Council Inc, 2006, Li et al. 2011). The SCOR model is structured into threelevels of detail with respect to the description of business processes. All three of themare defined by the Supply-Chain Council (2006).

Level 1 deals with the types of processes and defines the framework and contentsfor the SCOR model. This is a prerequisite for meeting the goal characteristics of thebusiness system. Level 2 deals with the level of process configuration, i.e. it definesprocess categories (e.g. companies implement their operational strategy throughconfigured acquisitional processes of their choice). Level 3 represents the level ofprocess elements. At this level, company’s ability to achieve market competitivenessis defined, i.e. the company performs precise adjustment of its operational strategy.

For every level and process, the model contains precisely defined PIs for thebusiness system. The SCOR model which was used to determine process perfor-mance indicators is shown in Figure 1 (Supply-Chain Council 2006).

For the logistics processes which are crucial for the business of oil–gascompanies, the SCOR model was used to define recommended PIs (FeasibilityStudy and Business Case 2005, Supply-Chain Council 2006).

In this article, standard financial performance indicators were used (EarningsBefore Interest and Tax/Earnings Before Interest, Taxes, Depreciation andAmortisation, Return of capital Employed, Return on assets, Return on Equity,Self-financing ratio, Current ratio, Liquidity index, Cash ratio, Debit equity ratio,Payables-to-sales ratio), as well as other process-oriented business performanceindicators which are specific to oil–gas systems (Table 2).

Table 2 shows business performance indicators for an oil–gas company (asdefined in the Feasibility Study and Business Case 2005).

2.4. A proposal for an advanced model

We propose an advanced model for the evaluation of the impact of ERPimplementation on PIs. This model is an extension of the combined model

Figure 1. SCOR model (Supply-Chain Council 2006).

88 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

(Uwizeyemungu and Raymond 2009) and the CSFs of ERP implementation, whichis customised for oil–gas companies. These factors were identified during ERPimplementation in the ABC Company (Figure 2). The combined method unifiesprocess model and scorecard model to evaluate the impact of ERP implementationon organisational performance indicators, as shown in Figure 2. The process modelwas chosen because of its ability to establish relationships with organisational systemparameters through automational, informational and transformational effects onoperating and control processes (Mooney et al. 2005). Scorecard model was used toshow the multi-dimensional character of organisational performance indicators in abusiness system.

In Figure 2, dotted lines represent the elements which are in the focus of thisresearch, while solid lines indicate the data which came from the ABC Company.Following data were collected from the ABC Company:

. CSFs for the implemented ERP system.

. Characteristics of the implemented system.

. Organisational processes which are affected by the implemented ERP system.

. Process and sub-process performance indicators.

Table 2. PI parameters for an oil–gas company.

Process Value driver PIs

Supply chain planning andoptimisation

Forecast accuracy Days of supply

Operations and schedulingExecution Reporting andanalytics

Inventory reductionInventory visibility

Days of supply (inventoryturn-over) Number of stock-outs

Opportunity to cash Number of deliveries in fullNumber of lost business

Terminal management Inventory Visibility Days of supply Order fill rateSales Planning and account

managementPlanning accuracy Days of supply

Service station fuelsmanagement

Stock-outs Number of stock-out Value oflost business process

Breakdown maintenance Equipment reliability MTBF (Mean Time BetweenFailures)e MTTR (MeanTime To Repair)

All maintenance processes Equipment availability Technical availabilityPreventive maintenance Planning effectiveness PMs overdueAll maintenance processes Maintenance cost

managementTotal cost of maintenance

Shutdown/turnaroundplanning

Planning effectiveness Turnaround duration

Bank reconciliation process Reference data information % of matched open itemsFinancial closing Documents posted on-time Closing cycle timeCorporate finance

managementImprove payments and cash

managementAdministrative costs to deliver

cash flow reportCorporate finance

managementImprove payments and cash

managementLiquidity index

Dunning process Improve automated dataflow including dunningletters

Days of sales outstanding

89Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

The following elements are subject of interest in this article:

. Impact of CSFs on the characteristics of the implemented ERP system.

. Impact of ERP characteristics on process effects (automation, informational,transformational).

. Impact of process effects on the process and sub-process performanceindicators.

. Impact of process effects, process and sub-process performance indicators andCSFs on the organisational performance indicators.

The combined model encompasses investigation of the impact of ERPcharacteristics on the process effects, the impact of process effects on the processand sub-process performance indicators, and the influence of process effects, processand sub-process performance indicators on the organisational performance

Figure 2. The modified Evaluation model of ERP system effects.

Critical Success Factors of the ERP

Implementation

Process effects

Process and sub-process performance indicators

Organizational performance indicators

Organizational processes

affected by the ERP system

Characteristics of the ERP

system implemented

Transformational effects

Automational effects

Informational effects

Financial perspective

Customerperspective

Innovation and

Learning perspective

Internal Business

perspective

Financial perspective

Customerperspective

Innovation and

Learning perspective

Internal Business

perspective

Evaluation model of ERP system effects (Uwizeyemungui Raymond, 2009).

90 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

indicators. The advanced model encompasses investigation on the impact of CSFson the combined model, and the organisational performance indicators.

To define the relationship between the combined model and CSFs of the ERPimplementation, a matrix CSFs vs. combined model was used. Within the combinedmodel the scorecard method and PIs (as defined for oil–gas companies in FeasibilityStudy and Business Case 2005) were used (Table 2). The matrix is the result ofcrossing the key ERP implementation parameters, with the business performanceindicators, by implementing the ‘open coding’ part of the Grounded theory(Suranjan et al. 2010).

3. Research methodology

The strategy used in this research relies on two case studies of two companies – thefirst company (designated ABC) was used as a pilot case, i.e. the reference model thatproduced the results, which were subsequently applied on the second company(designated DEF). Such an approach allows for better structure in the process ofinvestigating the case study in the second company. Case studies are well suited foranalysis in cases when the subjects of investigation cannot be clearly distinguished –as is the case with information technologies (IT) and ERP systems. The followingcriteria were used for selecting cases, according to Uwizeyemungu and Raymond(2009):

. The company must be a production one in the oil–gas sector.

. The company must have at least two years of previous history of using ERP.

. The company must be using ERP system in at least two of its core businessprocesses, including the processes of ‘delivering goods and services’.

Adoption and implementation of ERP system already begins at the productionstage, which, in the case of oil–gas companies, means the logistics and production ofoil, gas and their derivatives. Enterprise resource planning systems which aredesigned for production companies are more deployed than those for the service andpublic sectors. Accordingly, the cases which can be found in the production sectorprovide larger and better data samples for the researchers. Previous studies haveshown that ERP system must be in implementation for approximately two yearsbefore the first benefits can be noted (Weider et al. 2006). Two large oil–gascompanies expressed their willingness to participate in this research, on the conditionof anonymity. For the purpose of this report they shall be named company ABC andcompany DEF. Both companies are located in the Republic of Serbia.

Data were collected through interviews during the period December 2008–October 2009. The structure of employees from companies ABC and DEF is shownin Table 3. The interviews were conducted in person or in group, depending on therelative importance of the questionnaire, and the relative importance of theinterviewee. Group interviews were conducted with a maximum of 10 participants.In order to analyse results from the previous interview, some participants wereinterviewed a second time. Company ABC which represents the referential case,made its project documentation available, as well as other business documentationrequired for analysis of ERP influence.

The analysis of the collected data and the questionnaires filled by the intervieweeswas based on the grounded theory introduced by Glaser and Strauss (1967). More

91Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

precisely, the open coding method was used as part of the grounded theory(Suranjan et al. 2010). ‘Open coding’ method was used to analyse the data with pre-defined codes which were based on the principles of the influence assessment model.Both cases, ABC and DEF were first processed individually and then analysed inparallel.

Exactness of methodology, and exactness and quality of comparative study can bediscussed from aspects of credibility, portability, reliability and conformability. Thefirst measure that was undertaken in order to provide credibility is the authenticitywith which this research was conducted. The results of the research were presentedand confirmed by interviewees, triangulation of collected data, methods and datasources was performed, and opposing statements were reviewed and analysed.Portability was increased in three ways: (1) by detailed description of the case underexamination, (2) by comparing the results with the available theoretical knowledge,(3) by the generic (general) approach to the case in hand. Reliability was provided bythe complete documentation of the conducted research, and the original results,interview protocols and confirmations of the answers given by the participants.

4. Results of research

The comparative analysis of organisational characteristics of companies ABC andDEF is presented in Table 4. The pilot-case company ABC is a large-enterprisecompany with 11,000 employees, and annual revenue of $4.7 billion, while thecompany DEF is a medium-sized enterprise with 5000 employees, and an annualrevenue of $3.2 billion.

The companies were chosen to operate in the same market but are not in directcompetition with each other. Within their organisational structure both companieshave sectors which deal with production, transport and sales. Introduction of ERPsystem into the production sectors of a company represents a major challenge. Forcompanies ABC and DEF, optimisation of market supply, material resourceplanning, as well as production and transport are a priority, which is one of thereasons for ERP implementation in those companies. Within the optimisation themajor challenge is the optimisation of production planning process and reduction ofproduction costs.

Table 3. Detailed structure of employees who participated in the interview.

ABC (Pilot study) 10,000–11,000 employees DEF 2500–3000 employees

Interviewee No. Interviewee No.

CEO (Chief Executive Officer) 1 CEO 1CIO (Chief Information Officer) 1 CIO 1VP finance 2 Production manager 2VP production 2 ERP system provider 2VP sales 2Production manager 2Planning manager 2Purchasing manager 2Key user – sales 1Consultant 1Total 10 6

92 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Table 5 shows the comparative review of characteristics of ERP systemsimplemented in companies ABC and DEF.

The approach defined in Section 2.4 allows better review of the impact of ERPsystem on the operational and organisational performance indicators of company.The relationship between the effects of ERP implementation and operational andorganisational performance indicators of a company is defined in the following way:it rarely occurs in practice that the effects of ERP implementation can be treated asindicators of operational and organisational performance. In other words, the effectsof ERP implementation are rarely the PIs which are used by the company to assessits business activities. On the example of the ABC Company, this means thefollowing: during their interviews, a certain number of managers expressed theopinion that the effect of implementation should be the ‘standardisation ofinformation’ which in no way represents any of the PIs of a business system. It istherefore necessary to establish relationships between the effects of ERPimplementation and the PIs they affect. Tables 6 and 7 show all the relationships.The measurements taken also took into consideration the importance of eachindicator – from 1¼ insignificant to 5¼ very significant and the extent to which avariation in the indicator is attributable to the ERP effect from 0¼ no impact,1¼weak impact, 2¼ average impact, 3¼ strong impact (with the minus sign in caseswhen the variation is negative). The impact of ERP CSFs on ERP impact isevaluated as: 1¼ no impact, 2¼ normal impact and 3¼ strong impact.

The effects of ERP implementation were identified through interviews with themembers of ERP implementation project team in ABC Company. Their role was todefine business process and sub-processes – business owners, process owners. Theprocess and sub-process owners who take care of and are responsible for the dailyoperations of ABC Company, explained in detail the changes and gave theirevaluation of the impact of ERP implementation on each PI. They also evaluated theimpact of ERP implementation CSFs on the impact of ERP implementation. In thisway, a single product (a6 b)6 c is obtained for each (ERP effect–PI) couple andCSF. The aggregate result comprises the sum of (ai6 bi)6 ci products. The general

Table 4. Organisational characteristics of the companies which participated in the research.

Organisationalcharacteristics ABC DEF

Size1. No. of employees 10,000–11,000 50002. Annual sales revenue $4.7 billion $3.2 billion

Business area Oil and oil derivatives Natural gas and liquid oil gasMarket Domestic – Serbian market

Exporting services toTurkmenistan and Angola

Domestic – Serbian marketMarket of Bosnia andHerzegovina

Main goals ofproduction system

Optimisation of supply system Optimisation of supply system

Main strategic goals Competitiveness withmultinational companies

Competitiveness withmultinational companies

Main strategicorientation

Rationalisation ModernisationContinuous improvementTowards open market

Rationalisation ModernisationContinuous improvement

93Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

formula for the calculation of the local and global impacts of ERP implementationon KPIs is given in the following form:

Xn

i¼1ðai � biÞ � ci;

where i is the numerical order of couple (ERP impact – PI performance indicator); ais the importance of PI performance indicator: from 1¼ insignificant to 5¼ verysignificant; b is the degree of change of PI performance indicator due to ERP impact:from 0¼ no effect, 1¼weak effect, 2¼ average effect, 3¼ strong effect (with negativesign for negative variation); c is the indicator of the impact of ERP implementationCSFs on the impact of system PI (1¼ no impact, 2¼ normal impact, and 3¼ strongimpact)

The member of the total sum, (ai6 bi)6 ci, represents the local impact of ERPsystem implementation on the PIs, while the sum of local impacts,Pn

i¼1 ðai � biÞ � ci, represents the global impact of ERP system implementation onthe PIs. The sequence of steps required to perform this calculation is given below:

(1) Determine the parameters of PI significance (ai) and parameters of the degreeof change of PIs which are the result of ERP implementation (bi).

Table 5. Comparative review of characteristics of the implemented ERP systems.

ERP implementation ABC DEF

The year of ERP systemimplementation (go-live)

2009 2008

Previous ERP experience No NoImplemented ERP system SAP SybaseProvider of ERP system SAP Consulting Institute for information

technologyInitial ERP investment 12.4 mil. EUR 1 mil. EURProject manager CIO CIOProject team 300 – 350 members 20–30 membersExternal implementors ERP system provider ERP system providerDuration of implementation 3 years 2 yearsCriteria for ERP selection Preliminary study Preliminary studyNumber of sites Single site Single siteMain modules Finance and controlling

Accounting (general ledger,account payable accountreceivable)

Finance reportingBusiness intelligenceLogistics (purchasing,

material resourceplanning – in progress)

Cache managementInvestment managementHR localised payroll

Finance and controllingAccounting (general ledger,

account payable accountreceivable)

Finance reportingHRProduction planning and

scheduling

Elements retained fromthe legacy system

Process software modules Process software modules(SCADA system)

Other implemented systems No No

94 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Table 6. ERP effects and their influence on performance indicators in ABC Company.

ERP effects

Critical SuccessFactors of ERPimplementation

Performance indicator (PI)

PI affected i a b c a*b (a*b)*c

Automational effectsProductivity of

organisationalprocess

User training andeducation

Productivity ratios 1 3 2 3 6 18

Personal rotation 2 3 1 2 3 6Business process

reengineeringProductivity ratios 3 3 2 3 6 18

User participation Productivity ratios 4 3 2 2 6 12User resistance to

changePersonal rotation 5 3 1 3 3 9

Integration of system Labourimprovements

6 4 2 3 8 24

Use of Projectmanagement tomanageimplementation

Labourimprovements

7 4 2 3 8 24

Effective organisationalchange management

Productivity ratios 8 4 3 3 12 36

Improve paymentsand cachemanagement

Centralised payment Reducing operatingcosts

9 5 3 3 15 45

Percent of matchedopen items

10 4 2 1 8 8

Cost planning andcollection on costcentres

Current ratio 11 3 3 2 9 18

Credit limit check oncorporate level

Cash ratio 12 4 3 3 12 36

Better managementof warehouse

Closed loop order-to-cache and servicepetrol stations

Mean wait time ofcustomers

13 4 3 3 12 36

Closed loop for demandand supply chainplanning

Shipping delays 14 3 2 3 6 18

Productivity ratio 15 3 3 3 9 27Planning accuracy Closed loop order-to-

cache and servicepetrol stations

Days of supply 16 3 3 3 9 27

Automational effectsClosed loop for demand

and supply chainplanning

Perfect orderfulfilment

17 3 2 3 6 18

Connectivity withcustomers

Integration of system Returns- % ofproduct returnedby customer

18 4 3 3 12 36

Adequate ERPimplementation

Productivity ratios 19 5 2 3 10 30

Change managementculture

Customersatisfaction

20 5 1 2 5 10

Cancelled orders(lost sales)

21 3 0 1 0 0

Use of Projectmanagement tomanageimplementation

Productivity ratios 22 5 3 3 15 45

(continued)

95Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Table 6. (Continued).

ERP effects

Critical SuccessFactors of ERPimplementation

Performance indicator (PI)

PI affected i a b c a*b (a*b)*c

Integration ofresources

Integrated andharmonised processes(from order to cache)

Productivity 23 3 1 3 3 9

Centralisation ofoperations

Centralised payment EBTID/EBITDA 24 5 1 3 5 15

Cash flow 25 5 2 3 10 30Liquidity index 26 5 1 3 5 15

Centralised purchasing Perfect orderfulfilment

27 5 1 2 5 10

Informational effectsImprovement of

operationscheduling

IS-Oil basic functionimplemented(Inventory reductionand visibility)

Delivery lead time 28 5 3 3 15 45

Delivery quantities 29 5 2 2 10 20Shipping delays 30 5 2 1 10 10Production cycle

time31 4 2 2 8 16

Inventory turnover 32 3 2 1 6 6Richness of

informationextracted fromthe data

Technical and businessknowledge

Inventory turnover 33 3 2 3 6 18

Informational effectsAppropriate business

and IT legacy systemsInventory level 34 4 3 3 12 36

Adequate ERP version Return on capitalemployed

35 4 2 3 8 24

Use of Projectmanagement tomanageimplementation

Cash ratio 36 5 3 2 15 30

AppropriateManagementexpectation

Cash ratio 37 5 3 3 15 45

Legacy systemsknowledge

Inventory level 38 5 3 3 15 45

Breakdown ofmanufacturingcosts

Cost planned andcollection on costcentres and fordefined measures

Gross margin 39 5 3 2 15 30

Reliability of data Harmonised materialmaster records

– 40 5 3 3 15 45

Accuracy ofinventory

Complete inventory Sales mix 41 4 1 3 4 12

Closed loop for assetmanagement lifecycle

Inventory level 42 4 2 2 8 16

Transformational effectsFlexibility in pricing Closed purchasing loop Order fulfilment 43 3 2 3 6 18

Centralised payment Cash ratio 44 3 2 2 6 12Improvement of

production rangeClosed purchasing loop Reduction in

operating costs45 3 2 1 6 6

Complete inventory Reduction inoperating delays

46 1 1 1 1 1

Development ofcross-functionalcompetences

Inter departmentalcooperation andcommunication

Inter-functionalcooperation

47 3 3 2 9 18

(continued)

96 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

(2) Calculate the number and values of (ai6 bi) couples.(3) Define the indicator of CSFs of ERP implementation on system PIs (ci).(4) Sum the results, i.e. calculate the product (ai6 bi)6 ci in order to establish

the global impact of ERP system implementation on PIs.

4.1. Interpretation of results

In an ideal case, one would expect a strong ERP impact on each related PI. In anideal case of a negative impact, no impact on PIs should be expected. By keeping aiconstant and substituting all bi with constant values (3¼ strong change for positiveimpact and 0¼ no change for negative impact), a potential change of the system willbe made that will serve as a basis for comparison. Since a total of 52 couples wereidentified (ERP effect–PI) and CSF for ABC, and 42 for DEF, there follows theaggregate result:

X52

i¼1ðai � biÞ � ci ¼ 1077 for ABC;

X42

i¼1ðai � biÞ � ci ¼ 742 for DEF:

The score of 1077 represents the maximum impact of CSF of ERP systemimplementation on the global characteristics of business performance for the ABCCompany, while the score of 742 represents the same for the DEF Company. Thismeans that the scores represent the maximum CSF values (for the CSFs consideredin this case study) of ERP system implementation on the global characteristics of thetwo companies. Since the authors had access only to the maximum score values forthe non-oil and gas companies (Uwizeyemungu and Raymond 2009), the score of theABC Company is used as the referential value for the DEF Company.

Table 6. (Continued).

ERP effects

Critical SuccessFactors of ERPimplementation

Performance indicator (PI)

PI affected i a b c a*b (a*b)*c

Transformational effectsDevelopment new

applicationAvoidance

customisation– 48 – – – – –

Legacy systemconnected via XIinterface

– 49 – – – – –

Software development,testing andtroubleshooting

– 50 – – – – –

Revision of processand structures

Business processreengineering

Closing cycle time 51 4 1 2 4 8

Use of Projectmanagement tomanageimplementation

Number of lostbusiness

52 4 3 3 12 36

Sum (a*b)*c 1077

Notes: ‘i’ is a numerical order of the pair (ERP effect - PI). ‘a’ is a PI importance: from no importance (1)to strong importance (5). ‘b’ is a degree of PI influence on ERP effects: no influence (0), weak influence (1),medium influence (2), strong influence (3), (with the minus sign if the variance is negative).

97Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Table 7. ERP effects and their influence on performance indicators in DEF Company.

ERP effects

Critical SuccessFactors of ERPimplementation

Performance indicator (PI)

PI affected i a b c a*b (a*b)*c

Automational effectsProductivity of

organisationalprocess

User training andeducation

Productivity ratios 1 3 1 3 3 9

Personal rotation 2 3 2 2 6 12Business process

reengineeringProductivity ratios 3 3 2 2 6 12

User participation Productivity ratios 4 3 3 3 9 27User resistance to

changePersonal rotation 5 1 3 2 3 6

Increase in the riskslinked tointegration

Productivity ratios 6 3 72 3 76 718

Use of Projectmanagement tomanageimplementation

Labourimprovements

7 3 1 3 3 3

Effectiveorganisationalchangemanagement

Productivity ratios 8 3 2 3 6 18

Improve paymentsand cachemanagement

Centralised payment Reducing operatingcosts

9 3 4 3 12 36

Percent of matchedopen items

10 4 2 3 8 24

Cost planning andcollection on costcentres

Current ratio 11 3 2 2 6 12

Credit limit check oncorporate level

Cash ratio 12 4 3 2 12 24

Productivity ratio 13 2 3 3 6 18Planning accuracy Closed loop for

demand and supplychain planning

Perfect orderfulfilment

14 3 2 3 6 18

Centralisation ofoperations

Centralised payment EBTID/EBITDA 15 1 5 3 5 15

Cash flow 16 5 2 3 10 30Liquidity index 17 2 5 2 10 20

Automational effectsConnectivity with

customersChange management

cultureCustomer

satisfaction18 4 1 2 4 6

Cancelled orders(lost sales)

19 3 2 1 6 6

Use of Projectmanagement tomanageimplementation

Productivity ratios 20 4 3 3 12 36

Informational effectsImprovement of

operationscheduling

IS-Oil basic functionimplemented

Delivery lead time 21 5 3 3 15 45

Delivery quantities 22 5 2 2 10 20Shipping delays 23 5 2 1 10 10Production cycle

time24 4 2 2 8 16

(continued)

98 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Table 7. (Continued).

ERP effects

Critical SuccessFactors of ERPimplementation

Performance indicator (PI)

PI affected i a b c a*b (a*b)*c

Richness ofinformationextracted fromthe data

Adequate ERPversion

Return on capitalemployed

25 2 4 3 8 24

Use of Projectmanagement tomanageimplementation

Cash ratio 26 5 3 2 15 30

AppropriateManagementexpectation

Cash ratio 27 5 2 3 10 30

Breakdown ofmanufacturingcosts

Cost planned andcollection on costcentres and fordefined measures

Gross margin 28 5 3 2 15 30

Reliability of data Harmonised materialmaster records

– 29 3 4 3 12 36

Closed loop for assetmanagementlifecycle

Inventory level 30 4 2 2 8 16

Legacy systemknowledge

Inventory level 31 4 3 3 12 36

Informational effectsInformation on

human errorsChange management

cultureHabilitation of

employees32 4 3 3 12 36

Transformational effectsFlexibility in pricing Closed purchasing

loopOrder fulfilment 33 3 2 3 6 18

Centralised payment Cash ratio 34 3 2 2 6 12Improvement of

production rangeClosed purchasing

loopReduction in

operating costs35 3 2 1 6 6

Complete inventory Reduction inoperating delays

36 1 1 1 1 1

Development ofcross-functionalcompetences

Inter departmentalcooperation andcommunication

Inter-functionalcooperation

37 3 3 2 9 18

Development newapplication

Avoidancecustomisation

– 38 – – 3 0 0

Legacy systemconnected via XIinterface

– 39 – – 3 0 0

Softwaredevelopment,testing andtroubleshooting

– 40 – – 3 0 0

Revision of processand structures

Business processreengineering

Closing cycle time 41 4 1 2 4 8

Use of Projectmanagement tomanageimplementation

Number of lostbusiness

42 3 4 3 12 36

Sum (a*b)*c 742

Notes: ‘i’ is a numerical order of the pair (ERP effect – PI). ‘a’ is a PI importance: from no importance (1)to strong importance (5). ‘b’ is a degree of PI influence on ERP effects: no influence (0), weak influence (1),medium influence (2), strong influence (3), (with the minus sign if the variance is negative).

99Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

A general formula which yields the aggregate result is:

Xn

i¼1ðai � biÞ � ci ¼ 1077

which is the maximum result for the given enterprise size, that is,Pn

i¼1 ðai � dÞ wherei, a, b and care variables already mentioned in above formula while n is the totalnumber of ERP impact – PI couples; d is a constant; if bi� 0 and ci4 1, d¼ 6; ifbi5 0 and ci¼ 1, d¼ 0.

Calculation of sumPn

i¼1 ðai � dÞ, yields a total of 1128 for ABC and 786 forDEF.

To simplify interpretation, one can transpose the preceding scores to a usualcomparison scale by changing the denominator to a more meaningful number suchas 5, 10 or 100. Here, the results are mapped on the scale with five points, while thedesignated degrees of influence of ERP and CSFs on organisational performance areorganised as follows: very weak (1), weak (2), average (3), important (4) and veryimportant (5). The total result for ABC was calculated as 5 (1077/1128)¼ 4.8, whichis an average influence of ERP on the system characteristics. For DEF, the totalresult is 5 (742/786)¼ 4.7 which equals the ABC.

5. Discussion of results

The steps in the development of the method for the evaluation of ERP implemen-tation proposed in this article are shown in Table 8.

The method described is process-oriented, and it connects the influence of CSFsto the ERP system characteristics, on the one side, with operating and organisationalparameters, on the other. Should such an approach to evaluation of ERP influenceshow very little influence (or none at all) on business parameters which managers useto control business activities in a company, the following questions arise: is theimplemented ERP suitable for the company? Is the system being exploited to its fullpotential? Is the company using adequate parameters to express success of itsbusiness operations? as well as whether the investment into ERP system was justified.

The proposed method for the evaluation of ERP influence allows companies toconnect the influence of IT technologies on business parameters through the effect ofimplemented IT technologies on the business processes. The organisationalcharacteristics are defined by numerous factors, and the methodology proposed inthis article confirms that it is sufficient to evaluate the influence of ERPimplementation, since it is almost impossible to encompass all possible factors ofinfluence which determine a company’s system.

In ABC and DEF Companies, the model has been implemented without majordifficulties, but there are still some challenges that have emerged during theimplementation of the model. The challenges that have arisen during the applicationof the model are user resistance to ERP implementation and the impacts oncompany systems in the environment in which companies operate. The incurredchallenges appeared due to the lack of preparation and motivation of the users forthe implementation of ERP systems as well as external influences on the system.Specifically, the activities carried out by ABC and DEF Companies belong to the oiland gas sector, but differ greatly in the organisational structure, manpower and thetechnology they use. ABC Company has a much larger number of employees in

100 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Table 8. Realisation steps for the proposed evaluation method.

Step Description

1 A repository of all operating and control processes and their sub-processes wasdefined with the required level of detail

2 All processes and sub-processes which are in any way affected by ERPimplementation were defined

3 Performance indicators were identified and then implemented on the modifiedprocesses and sub-processes (local level), as well as on the organisationalperformance indicators (global level); Degree of importance was defined for everyindicator (from (1): insignificant, to (5): very significant)

4 ERP implementation critical success factors were identified for oil–gas companies5 Effects of ERP implementation on various processes were identified by end-users

(managers and system users): assessment of direct effects vs. indirect effects,expected effects (realised or not, or not fulfilling expectations) vs. the unexpectedeffects, and existing effects vs. potential effects

6 Connections were established between critical success factors, ERP effects, andvarious indicators of business system characteristics

7 The level of ERP impact on performance indicators (no impact (0), weak impact (1),medium impact (2), strong impact (3), with the negative sign if the variance isnegative) was established

8 Results were analysed: comparison with the expected, questions with explanationswhy the expected effects failed to materialise, why they were over- orunderestimated; analysis of possibilities for extending the ERP system to cover theunaffected processes, possible effects on the system and the prerequisites for theirrealisation: definition of measurements which are to be used in order to advancethe exploitation of business system; setting goals for future evaluations

Given below is the graphical representation of steps from Table 8.

Step 1: Organisational processes – repository of all operating and control processes and their sub-processes.

Step 2: All processes and sub-processes affected by ERP implementation were detected.

Step 3: Performance indicators were identified and then implemented on the modified processes and sub-processes (local level), as well as on the organisational performance indicators (global level).

ERP system

Local level Global level

ERP system

...

.........

...

.........

Org.Perf..Indicators

...

...

...

...

...

.........

...

...

101Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Step 4: ERP implementation critical success factors were identified for oil–gas companies.

Step 5: Effects of ERP implementation on various processes were determined.

Step 6: Connections were established between critical success factors, ERP effects and various indicators ofbusiness system characteristics.

Step 7: The level of ERP impact on performance indicators were established.

Step 8: Analyse results.

ERP system

..

......

Global effects

........

.... .... ....

...

.........

...

.......

...

...

Org.Perf..Indicators

....

ERP system

...

...

...

...

...

...

...

.........

....

.............. ....

....

...

.........

...

.......

...

...

Org.Perf..Indicators

....

ERP system

...

...

...

...

...

...

...

.........

....

............ ....

....PIPI

PI PI

PI

ERP system

...

...

...

...

...

.........

...

.......

...

...

Org.Perf..Indicators

....

...

.

........ ....

....

(0-3)

-

(0-3)(0-3)(0-3)

(0-3)

(0-3)

ERP system

...

...

...

...

Org.Perf..Indicators

....

........

....(0-3)

(0-3)

(0-3)(0-3)

(0-3)

....

Conclusions of the evaluation

...

.........

...

.......

...

...

....(0-3)

102 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

comparison with the DEF Company, different levels of IT knowledge andpreparedness for the introduction of changes in organisation and business processesresulting from the implementation of ERP systems. The employees in DEF weresignificantly better prepared for the implementation of ERP systems. The ABCCompany during the implementation of the proposed method detected the userresistance to change business processes and organisational changes resulting fromthe implementation of the ERP system, which was a challenge during the final stagesof implementation of ERP systems in post-productive period. The user resistancedetected in the ABC Company has caused a slight decrease in the value of PIs. Userresistance was not detected in DEF Company, so the values obtained are within theexpected limits of PIs. The emergence of user resistance in the implementation ofERP systems has been observed in organisations where users have little knowledge ofIT technologies as well as in organisations where the average age structure of theuser is over 40 years. In addition, user resistance was observed as resistance tochanges that cause the changes in organisational structure and business processes.The reluctance of users and poor motivation for adopting ERP systems use also ledto the emergence of user resistance. DEF Company prior to the implementation ofthe ERP system has organised a series of sessions in which implementation necessityand achieved benefits were explained to the users. Sessions in DEF Company wereattended by the members of top-management as well. Thus, it emphasises the needfor the management of companies, that implement ERP systems, should be preparedfor the user resistance and consequently develop the strategy to overcome thisphenomenon.

One of the challenges that generally occur during the evaluation of the impact ofIT technology on organisational performance is the impact of exogenous variables,i.e. external factors on the system. In the case of method application in ABC andDEF Companies, this difficulty was overcome by the model that evaluates the impactof the realised ERP system which was implemented at the organisational features asit is impossible to control all the external factors that can affect the system. Theresults from such a model would be useless if we take into account that all theexternal factors would make the model too complex. During the implementation ofthe model it is observed that the values fell short of PIs that are used for theevaluation of the profitability of both companies. At first, two questions arose:whether the good performance of the system was observed, and whether the generalimplementation of ERP systems suits the needs of ABC and DEF Companies. Theanalysis of external influences on the system detected a drop in global sales in amarket in which ABC and DEF Companies operate, which explains the occurrenceof a short-term decline in the value system of PIs that are used for the evaluation ofthe profitability of the company. The influence of user resistance to the success of theimplementation of ERP systems and PIs may be the subject of further research.

The advantage of the proposed method is reflected in four aspects:

(1) The proposed method focuses on the goal of ERP implementation, because itallows the advancement of business characteristics through IT investment tobe measured. The method also encompasses characteristics of businesssystem on the process level, as well as on the global (corporate) level.

(2) The proposed method is action-oriented because it identifies areas inoperating and organisational company characteristics which are not affectedor are only slightly affected by ERP implementation. This allows the

103Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

managers to define corrective measures, i.e. implement additional ERPmodules.

(3) The proposed method allows the success of operation of organisational unitswithin a company to be compared, even though their areas of operationdiffer.

(4) The proposed method boosts project managers’ efficiency when it comes tomanagement of ERP implementation projects.

Another important feature introduced by this method is that managers andresearchers are allowed to measure the value of IT investments.

Being project-oriented, the proposed model allows managers of ERP implemen-tation projects to review and control the success of ERP implementation in order toimprove business system performance (Zafeiropoulos et al. 2009).

As an action-oriented method, it allows identification of the operating andorganisational characteristics which remain unchanged or are affected very little byERP implementation. This allows managers to apply corrective measures wherenecessary, through introduction of new software ERP modules, and investment intoadditional measures aimed at improving the application of the already used ERPmodules. This method allows comparison between business performances ofcompanies which differ regarding their area of business, company size, appliedtechnology, etc. By focusing on the goal of implementation, the proposed methodallows assessment of specific situations in companies, where the normalised data canbe compared. This fact can contribute to future research of benchmarking theassessment of IT investment.

6. Conclusion

The evaluation of the impact of information technologies on organisationalcharacteristics represents a priority for the related research. According to majorityof IT experts, the existing methods for the evaluation of ERP implementation impacton enterprise business performance are not efficient enough. As a result, ITinvestments have been less evaluated than some other types of investments. Inaddition, the changes in organisational and technological environments stimulatedevelopment of novel methods. This is particularly true for ERP systems, becausetheir complexity prevents evaluations of IT impact on business performance to beautomatically applied to them.

Presented in this article is an advanced model for the evaluation of ERPimplementation on the business performance of an enterprise, using a combinedmethod (Uwizeyemungu and Raymond 2009) and CSFs which are defined for oil–gas companies.

The results show that the advanced model allows companies to define the degreeof influence of critical successes factors of ERP implementation on businessperformance (PIs) through automation, integration and transformation of businessprocesses.

Using this method, companies can integrate operating and organisational systemcharacteristics – which influence the implemented ERP characteristics – with thecharacteristics of implemented ERP and CSFs. In case it is established that ERP hasa small or non-existent influence on operating and organisational systemcharacteristics, justification of ERP implementation should be considered,

104 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

accompanied by the introduction of an additional functionality or new modules ofERP system. Furthermore, there is a question as to whether the project ofimplementation was well managed or there were some omissions in implementation.

The proposed method presents the basis for further research regarding ERPimplementation and its influence on business performance of oil–gas companies. Itshould also help the advancement of methodologies for ERP implementation projectmanagement when it comes to defining CSFs of implementation which influence theexpected improvement of business performance during the productive phase of ERPsystem. Further research should be aimed at reversing this method: goal values oforganisational performance parameters should be defined, and then the character-istics of a corresponding ERP system should be determined. In addition, effortsshould be focused on widening this research to encompass the oil and gas companiesfrom the region, as well as to include additional CSFs of ERP systemimplementation.

References

Aldin, L. and de Cesare, S., 2011. A literature review on business process modelling: newfrontiers of reusability. Enterprise Information Systems, 5 (3), 359–383.

Dong, L., Neufeld, D.J., and Higgins, C. 2009. Top management support of enterprise systemsimplementations: two cases. Journal of Information Technology, 24 (1), 55–80.

Esteves, J. and Pastor, J., 2000. Towards the unification of critical success factors for ERPimplementations. In: Proceedings of the 10th annual business information technology (BIT)conference, 1–2 November 2000. Manchester, UK. (CD-ROM Paper No. 9). Availablefrom: http://www.jesteves.com/bit2000.pdf [Accessed 18 May 2012].

Esteves, J. and Pastor, J.A., 2006. Organizational and technological critical success factorsbehavior along the ERP implementation phases. In: I. Seruca, ed. Enterprise informationsystems VI. the Netherlands: Springer, 63–71.

Feasibility Study and Business Case, 2005. Project implementation ERP systemdocumentation.

Fontana, M.R. and Neto, I.A., 2009. ERP systems implementation in complex organization.Journal of Information Systems and Technology Management, 6 (1), 61–92.

Fryling, M., 2010. Estimating the impact of enterprise resource planning project managementdecision on post-implementation maintenance costs: a case study using simulationmodelling. Enterprise Information Systems, 4 (4), 391–421.

Irani, Z., 2010. Investment evaluation within project management: an information systemsperspective. Journal of the Operational Research Society, 61, 917–928.

Izza, S., 2009. Integration of industrial information systems: from syntactic to semanticintegration approaches. Enterprise Information Systems, 3 (1), 1–57.

Glaser, B. and Strauss, A., 1967. The Discovery of Grounded Theory: Strategies for QualitativeResearch. Chicago: Aldine.

Kaplan, R. and Norton, D., 1996. Using the balanced scorecard as a strategic managementsystem. Harvard Business Review, Jan – Feb, 75–85.

Kohli, R. and Grover, V., 2008. Business value of IT: an essay on expanding research direc-tions to keep up with the times. Journal of the Association for Information Systems, 9 (1),23–39.

Li, L., et al., 2008a. TQM – A predecessor of ERP implementation. International Journal ofProduction Economics, 115 (2), 569–580.

Li, L., et al., 2008b. Assessing the effects of manufacturing infrastructure preparation prior toenterprise information systems implementation. International Journal of ProductionResearch, 46 (6), 1645–1665.

Li, L., Su, Q., and Chen, X., 2011. Ensuring supply chain quality performancethrough applying the SCOR model. International Journal of Production Research, 49(1), 33–57.

105Enterprise Information Systems

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014

Loh, T.C. and Koh, S.C.L., 2004. Critical elements for a successful enterprise resourceplanning implementation in small and medium-sized enterprises. International Journal ofProduction Research, 42 (17), 3433–3455.

Mabert, V.A., Soni, A., and Venkataramanan, M.A., 2003. Enterprise resource planning:managing the implementation process. European Journal of Operational Research, 146 (2),302–314.

Maksimovic, R., et al., 2010. Complexity and flexibility of production structures. Journal ofScientific and Industrial Research, 69 (2), 101–105.

Mooney, J.G., Gurbaxani, V., and Kraemer, K.L., 1995. A process oriented framework forassessing the business value of information technology. In: J.I. DeGross, G. Ariav, C.Beath, R. Hoyer and C. Kemerer, eds. Proceedings of the sixteenth international con-ference on information systems, 10–13 December 1995. Amsterdam: ICIS, 17–27.

Nah, F.F.-H. and Delgado, S., 2006. Critical success factors for enterprise resource planningimplementation and upgrade [online]. Journal of Computer Information Systems, Availablefrom: http://www.highbeam.com/doc/1P3-1171027711.html [Accessed 4 August 2010].

Niu, N., Jin, M., and Cheng, J-R.C., 2011. A case study of exploiting enterprise resourceplanning requirements. Enterprise Information Systems, 5 (2), 183–206.

Peslak, A.R., 2006. Enterprise resource planning success: an exploratory study of the financialexecutive perspective. Industrial Management and Data Systems, 106 (9), 1288–1303.

Plant, R. and Willcocks, L., 2007. Critical success factors in international ERP implementa-tions: a case research approach. Journal of Computer Information Systems, 47 (3), 60–70.

Salmela, H., 2008. Analysing business losses caused by information systems risk: a businessprocess analysis approach. Journal of Information Technology, 23, 185–202.

Snider, B., da Silveira, G., and Balakrishnan, J., 2009. ERP implementation at SMEs: analysisof five Canadian cases. International Journal of Operations & Production Management, 29(1), 4–29.

Soja, P., 2006. Success factors in ERP systems implementations lessons from practice. Journalof Enterprise Information Management, 19 (4), 418–433.

Stefanou, C.J., 2001. A framework for the ex-ante evaluation of ERP software. EuropeanJournal of Information Systems, 10 (4), 204–215.

Sun, Y. and Bhattacherjee, A., 2011. Multi-level analysis in information systems research: thecase of enterprise resource planning system usage in China. Enterprise InformationSystems, 5 (4), 469–494.

Supply-Chain Council Inc. 2006. Supply Chain Operations Reference-model (SCOR) 8.0.Pittsburgh: Supply-Chain Council Inc.

Tan, Y. and Takakuwa, S., 2011. Use of simulation in a factory for business continuityplanning. International Journal of Simulation Modelling, 10 (1), 17–26.

Suranjan, C., Sarker, S., and Sarker, S., 2010. An exploration into the process of requirementselicitation: a grounded approach [online]. Journal of the Association of InformationSystems, 11 (4), Available from: http://aisel.aisnet.org/jais/vol11/iss4/1 [Accessed 4 August2010].

Uwizeyemungu, S. and Raymond, L., 2009. Exploring an alternative method of evaluating theeffects of ERP: a multiple case study. Journal of Information Technology, 24, 251–268.

Vilpola, I., 2008. A method for improving ERP implementation success by the principles andprocess of user centered design. Enterprise Information Systems, 2 (1), 47–76.

Weider, B., et al., 2006. The impact of ERP system on firm and business process performance.Journal of Enterprise Information Management, 19 (1), 13–29.

Wickramasinghe, V. and Gunawardena, V., 2010. Critical elements that discriminate betweensuccessful and unsuccessful ERP implementations in Sri Lanka. Journal of EnterpriseInformation Management, 23 (4), 466–485.

Zafeiropoulos, I., et al., 2009. Installing an ERP system with a methodology based on theprinciples of goal directed project management. Journal of Information Systems andTechnology Management, 6 (3), 357–394.

106 G. Gajic et al.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

16:

46 0

6 N

ovem

ber

2014