Performance Evaluation of Distribution Utilities based on DEA & AHP models

Zhang Lizi Research Center for Electricity Market North China Electric Power University

Beijing, China lizizhang2001@sina.com

Wang Qingran Research Center for Electricity Market North China Electric Power University

Beijing, China wangqingran@163.com

Abstract—This paper proposes a procedure and method for assessing the operation performance of distribution utilities which is the hierarchical yardstick competition method. The hierarchical analysis of factors reflecting the operation situation of distribution utilities is inosculated in the overall goals, the criteria layer and the factor layer by employing the analytic hierarchy process. The judgment matrix and data envelopment analysis are used to gain the weights and performance factor of assessment criteria in the criteria layer and factor layer respectively. An example is provided to clarify the method suggested by this paper.

Keywords- analytic hierarchy process; data envelopment analysis; efficiency factor; performance assessment

I. INTRODUCTION With the electricity market development farther, the

research of regulation theories and methods has become an important topic[1]. Power sector reformation are transforming the structures and operating environments of electricity industries across developed and developing countries in order to increase competition[2]. Performance evaluation plays a crucial role in structural reforms in facilitating an understanding of the behavior of electric utilities, and also in defining regulatory policies for distribution utilities[3]. Benchmarking models for electricity distribution have been introduced in UK and US[4][5], and have become common throughout Latin America and Europe[6], and have become common throughout the Latin America [7][8]. However, for developing countries, few studies have so far been reported. No detailed performance analysis has so far been reported, despite the fact that the sector has undergone reforms, and has further accelerated the process of change[9].

This paper proposes a procedure and method for assessing the operation performance of distribution utilities which is the hierarchical yardstick competition method. Firstly, the hierarchical analysis of factors reflecting the operation situation of distribution utilities is inosculated in the overall goals, the criteria layer and the factor layer by employing the analytic hierarchy process. Secondly, the judgment matrix and data envelopment analysis are used to gain the weights and performance factor of assessment criteria in the criteria layer and the factor layer respectively. An example is provided to clarify the method suggested by this paper.

II. THEORIES OF AHP AND DEA

A. Analytic Hierarchy Process(AHP) The AHP enables the decision makers to structure a

complex problem in the form of a simple hierarchy and to evaluate a large number of quantitative and qualitative factors in a systematic manner under multiple conflicting criteria. The AHP method makes use of pair-wise comparisons matrix, hierarchical structures, and ratio scaling to apply weights to attributes. Problems are decomposed into the hierarchy of a goal, attributes, and alternatives by the AHP method [10] shown in Figure. 1.

Figure 1. The AHP method

Table I shows the scale for comparisons. The numbers 1, 3, 5, 7 and 9 are used as scaling ratios, corresponding to the strength of preference for one element over another. For example, number 9 represents extreme importance over another element. Generally, the 9-point scale is used because the qualitative distinctions are meaningful in practice and have

978-1-4244-4639-1/09/$25.00 ©2009 IEEE

an element of precision when the items are compared with one another. The ability to make qualitative distinctions is well represented by the 5 possible attributes of equal, moderate, strong, very strong, and extreme.

TABLE I. SCALE FOR PAIRWISE COMPARISIONS

Important scale Definition Explanation

1 Equal important Two elements contribute equally.

3 Moderate important One element is slightly favored over another.

5 Strong important One element is strongly favored over another.

7 Very strong important An element is very strongly favored over another.

9 Extreme important One element is the most favored over another.

When we apply the AHP method to take the weights of criteria and alternatives, the decision maker should be consistent in the preference ratings. The equation below describes the process of taking the overall weights of alternatives.

1 1 1

1 21 1

2 2 22 2

1 2

1 2

n

n

n nn n n

n

w w ww w w

w nww w w

w nww w w AX nX

w nww w ww w w

⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥= ⇒ =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎣ ⎦⎢ ⎥⎢ ⎥⎣ ⎦

(1)

Where ija represents the importance of alternative i over alternative j and ika represents the importance of alternative i over alternative k , ij jka a⋅ must be equal to ika that is an estimate of the ratio /i kw w for the judgments.

B. Data Envelopment Analysis(DEA) Comparing the relative efficiency of Decision Making Unit

(DMU) through mathematical programming, DEA is the method that assesses efficiency of decision-making unit based on the concept of relative efficiency. Based on input and output data of comprehensive analysis, DEA can draw the efficiency of each DMU comprehensive quantitative indexes. Accordingly, the DMU classification ranking will determine effective (that is, relative efficiency of the highest) DMU, and point out other non-effective DMU on the causes and extent [11][12]. 2 2C GS model is chosen in this paper. 2 2C GS model is one of DEA basic models, which is an ideal method that evaluates relatively effective technology for multiple inputs and multiple outputs decision-making units. It may involve the production set which is more than one set of convex set to meet convex, invalid and the smallest assumptions of justice system [13].

The so-called technical effective means "production" in the ideal state, compared to the existing input, which will get the biggest output. The relationship between input indexes and output indexes in this paper can meet the request. So 2 2C GS model is chosen.

Assuming there are n decision-making units, each unit has m “input”(X), as well as s “output”(Y). If assess the efficiency of the 0j decision-making unit, 2 2C GS model is as follows:

0 0max( )TpY Vμ μ+ = （2）

. .s t

0

0

0

10, 0

T Tj j

T

X Y

X

ω μ μ

ωω μ

⎧ − − ≥⎪⎪ =⎨⎪ ≥ ≥⎪⎩

When the efficiency index 1pV = is effective relative to other elements, this means that there are no other decision-making units or their linear combination that can use less production inputs to get equivalent outputs.

From above we can see that DEA method can give more objective assessment for evaluating the decision-making units through the actual value of the input and output indexes. However, from the EDA results, those relatively ineffective decision-making units ( 1pV < ) can be sorted accordance with the size of sequencing pV . But for a relatively effective decision-making unit ( 1pV = ), that can’t be carried out further sorting. So if several decision-making units are relatively effective, the optimal decision-making unit will not be selected.

III. PERFORMANCE EVALUATION OF DISTRIBUTION UTILITIES BASED ON DEA & AHP MODELS

A. Core Idea The core idea of this paper is to combine AHP with

yardstick competition regulation method of distribution utilities. DEA is an effective mathematical tool to achieve yardstick competition which is dependent on the input and output indicators and is objective completely. A reasonable classification of different assessing indicators is necessary because of the large-scale calculation and the absence of the importance of different indicators.

When the indicators are designed according to AHP, a reasonable determination is made by regulators in accordance with the actual situation in the regions. The structure of judgment matrix is mainly based on subjective judgments in which the importance of the indicators is difficult to validate. Therefore, the DEA model which is objective completely is used to analyze the indicators. Combining the expertise of the former method with the objectivity of the latter method, the computational complexity of DEA is reduced effectively and the model is applied properly.

B. Hierarchical Model of Distribution Utilities Considering the actual situation of the power regulation and

combining with the implementation of power utilities performance standards, the regulation indicator system is selected as follows:

wholesale price, distribution line length, substation capacity, net assets, loss rate, electricity quantity, supply area, number of users, load factor, peak load, supply reliability rate, voltage passing rate, frequency passing rate, power program, complaint handling cases.

The indicators above are divided into three categories for utilities evaluation such as economic operation, supply quality and services.

In this model, the relative operating performance of the distribution utilities is the overall goal, three categories of evaluation criteria are the criteria layers. Every evaluation criteria is the corresponding factor layer. The hierarchical method for assessing operation performance of distribution utilities is established as shown in Figure 2 for the hierarchical structure model and table Ⅱ.

Figure 2. Layered structural method for assessing distribution utilities’

relative performance

TABLE II. EACH CRITERIA’S INPUTS AND OUTPUTS

Criteria Indicators

Economic operation

Inputs Wholesale price, Net assets, Loss rate

Outputs Total revenue, Load factor, Peak load

Supply quality

Inputs Line length, Substation capacity

Outputs Voltage passing rate, Supply reliability rate, Frequency passing rate

Services Inputs Supply area, Number of users

Outputs Power program, Complaint handling cases

The hierarchical method for assessing operation performance of distribution utilities is achieved combining AHP and DEA. The steps of specific assessment methods are as follows:

• Constructing the judgment matrix of criteria layer and calculating the corresponding weights.

• Calculating the efficiency evaluation factors of DEA.

• Calculating the overall operating performance factors according to the corresponding weights of distribution utilities.

IV. APPLICATION EXAMPLE In this paper, six regional distribution utilities in China are

researched to test the validity of the model which is to assess the operating performance of the distribution utilities. The known operation data is shown in TableⅢ, TableⅣ and TableⅤ.

TABLE III. ECONOMIC OPERATION DATA OF SIX UTILITIES

Utility Wholesale

price (yuan/MWh)

Net assets

(G yuan)

Loss rate (%)

Total revenue (G yuan)

Load factor (%)

Peak load

(MW)

1 300.12 112.87 3.24 15.68 75.44 9870

2 278.98 219.87 4.65 34.68 69.32 8330

3 189.99 133.24 1.55 33.35 60.66 6990

4 300.23 47.56 5.43 25.90 59.11 4760

5 267.88 215.34 3.66 33.48 65.41 6590

6 298.33 123.55 2.69 22.26 76.44 4457

TABLE IV. SUPPLY QUALITY DATA OF SIX UTILITIES

Utility Line

Length (km)

Substation Capacity (MVA)

Voltage passing

rate (%)

Supply reliability

rate (%)

Frequency passing

rate (%)

1 9241 83550 99.83 99.90 99.92

2 5643 23630 99.92 99.87 99.85

3 7754 54790 99.82 99.89 99.81

4 8743 33690 99.85 99.87 99.89

5 3908 16720 99.96 99.91 99.92

6 4092 45892 99.67 99.85 99.87

TABLE V. SERVICES DATA OF SIX UTILITIES

Utility Supply

area (km2)

Number of Users (M persons)

Complaint handling cases (%)

1 172300 19.235 0.32

2 190100 8.96 0.22

3 512300 4.46 0.37

4 225200 2.47 0.45

5 78500 3.45 0.76

6 199200 6.43 0.11

Taking into account the specificity of the power industry, supply quality contains some technical indicators which are the supply security and the electricity quality. Electricity quality is the most important assessment criteria of the distribution utilities. On the other hand, reducing costs and improving efficiency are laid store by distribution utilities in which economic operation is more important than services. With the specificity of electric power industry, supply quality and service quality are two most important indicators. Judgment matrix is as follows according to local conditions:

Supply Economic

Servicesquality operation

Supply qualityEconomic operation

Services

1 2 31/ 2 1 21/ 3 1/ 2 1

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

Figure 3. Judgment matrix of the criteria lay

The weights of the criteria in judgment matrix are 0.633, 0.301, 0.198 in accordance with AHP method. According to the observing and analyzing the data, supply efficiency factor

Qθ , economic efficiency factor Eθ and service efficiency factor Sθ are calculated of DEA model in which θ is the synthesis efficiency factor of AHP model of assessing operation performance of distribution utilities.

TABLE VI. THE DISTRIBUTION UTILITIES’ PERFORMANCE FACTORS

Utilities Qθ Eθ Sθ θ

1 0.761 0.711 1 0.782

2 1 0.911 1 0.993

3 1 1 1 1

4 1 1 1 1

5 0.689 1 1 0.864

6 1 1 1 1

The economic operation, supply quality and services in

utility 3,4,6 are 1 which show that operation development of the utilities are well-balanced and the performance are better than other ones. The supply quality and service indicators of utility 2 are outstanding, but the economic operation indicators are weak, which impact the overall efficiency . The supply quality indicators of utility 5 are weak which are so important that the overall efficiency factors are affected. The overall efficiency factors of utility 1 are the worst.

AHP evaluation indicators and judgment matrix are adjusted in accordance with the actual situation of distribution utilities by regulators in which regulation methods and the development of distribution utilities are synchronized.

V. CONCLUSION This paper proposes a procedure and method for assessing the

operation performance of distribution utilities which is a hierarchical yardstick competition method. The hierarchical analysis of factors reflecting the operation situation of distribution utilities is inosculated in the overall goals, the criteria layer and the factor layer by employing the analytic hierarchy process. A numerical example is given to prove its feasibility and validity. This project is expected to provide scientific and practical reference to supervisors as well as managers of electric power companies for decision-making.

REFERENCES [1] F. R. Forsund and S. A. C. Kittelsen, “Productivity development of

Norwegian electricity distribution utilities,” Resource and Energy Economics, vol. 20, pp. 207–224, 1998.

[2] Su Desheng,Gu Xueping,Zhao Shuqiang. “Development of a balck start decision support system for the southern power network of hebei province,” Automation of Electric Power Systems, Vol.28, No.12,pp.45-49,2004.

[3] Lou Ping, Chen Youping.” An AHP/EDA method for vendor selection in the agile supply chain,” Journal of Huazhong University of Science and Technoloy(Nature Science), Vol.30, No.4,pp.29-31,2001.

[4] Yue Yiding,Zhang Yi, “ The application of AHP model based on DEA in the measuring and selecting of enterprise technic strategic assets,” Statistics & Information Forum, Vol.19, No.5,pp.5-9,2004.

[5] P. J. Agrell, P. Bogetoft, and J. Tind, “Efficiency and incentives in regulated industries: The case of electricity distribution in Scandinavia,” in Proc. Sixth Eur. Workshop on Efficiency and Productiv. Anal., Copenhagen, Denmark, Oct. 29–31, 1999.

[6] A. Charnes, W. W. Cooper and E. Rhodes, “Measuring the efficiency of decision making units,” European Journal of Operational Research, vol. 2, pp. 429-444, 1978.

[7] R. D. Banker, A. Charnes and W. W. Cooper, “Some models for estimating technical and scale efficiencies in Data Envelopment Analysis,” Manage Science, vol. 30, no. 9, pp. 1078-1092, 1984.

[8] R. D. Banker, A. Charnes and W. W. Cooper, “An introduction to Data Envelopment Analysis with some models and their uses,” Research in Governmental and Nonprofit Accounting, vol. 5, 1989.

[9] F. Y. Lo, C. F. Chien and J. T Lin, “A DEA study to evaluate the relative efficiency and investigate the district reorganization of the Taiwan power company,” IEEE Trans. Power Systems, vol. 16, pp. 170-178, Feb. 2001.

[10] J. Partanen, J. Lassila, S. Viljainen and S. Honkapuro, “Data Envelopment Analysis in the benchmarking of electricity distribution companies,” International Conference on Electricity Distribution. CIRED Barcelona, Espa. 2003.

[11] J. Partanen, J. Lassila and S. Viljainen, “Analysis of the benchmarking results of the electricity distribution companies in Finland,” IEEE Postgraduate Conference on Electric Power Systems. Budapest, Hungary. 2002.

[12] TANUREJ., E. P. S , “Comparative Analysis of the Distribution Companies in the Establishment of Quality Targets in Terms of Continuity Indices,” Master Dissertation, IEE/DET, Escola Federal de Engenharia de Itajubi - EFEI, Nov 2000, Brazil (in Portuguese).

[13] JOSEPHF. HAIR,~ L P HE. ANDERSON~ ,N A L DL. TATHAEM W ILLIACM. RACK,“ Multivariate Data Analisis With Radings,” Fourth Edition, Prentice Hall, Englewood Cliffs, New Jersey.