operational conditions in regulatory benchmarking – a monte-carlo simulation stefan seifert &...

Operational Conditionsin Regulatory Benchmarking – A Monte-Carlo Simulation

Stefan Seifert & Maria Nieswand

Workshop: Benchmarking of Public UtilitiesNovember 13, 2015, Bremen

1 Motivation and Literature

2 Methodologies

3 The DGP

4 Simulation Design and Performance Measures

5 Initial Results

6 Conclusion and Outlook

Agenda

Stefan Seifert & Maria Nieswand2Benchmarking of Public Utilities

Motivation1

Stefan Seifert & Maria NieswandBenchmarking of Public Utilities

3

Regulatory Approaches for Electricity DSOsSource: Agrell & Bogetoft, 2013

Motivation

• Benchmarking widely used in regulation – sectors in which environmental factors play an important role

• Accuracy of estimates influences revenue caps, industry performance, firm survival, and ultimately customers via prices

• Methodological advances to account for environmental factors and heterogeneity

• Non-parametric approaches: z-variables in 1-stage DEA (Johnson and Kuosmanen, 2012), conditional DEA (Daraio & Simar, 2005 & 2007), …

• Parametric approaches: Latent Class (Greene, 2002; Orea & Kumbhakar, 2004), Zero-inefficiency SF (Kumbhakar et al., 2013), …

• Semi-parametric approaches: StoNEzD (Johnson & Kuosmanen, 2011), …

• BUT: Regulatory models typically based on standard DEA or SFA

1


4

Motivation

Aim of this study• Systematical performance evaluation of Latent Class, StoNEzD and conditional DEA in

the presence of environmental factors

• Generalization of results via Monte-Carlo-Simulation

Guidance for regulators to choose estimators given industry structure and industry characteristics

Scope of this study• Consideration of different model set-ups imitating real regulatory data

Cross section with variation in sample sizes, noise and inefficiency distributions and in terms of the true underlying technology

Consideration of different cases of impact of environmental variables

1


5

Related Literature

Monte Carlo Simulation Studies• Basic MC evidence in original research papers

• Andor & Hesse (2014): StoNED vs SFA vs DEA

• Henningsen, Henningsen & Jensen (2014): multi-output SFA

• Krüger (2012): order-m vs order- vs DEA

• Badunenko, Henderson & Kumbhakar (2012): KSW bootstrapped DEA vs FLW

• Badunenko & Kumbhakar (forthcoming): persistent and transient ineff. SFA

Few studies focusing on environmental variables• Cordero, Pedraja & Santin (2009) – z-variables in DEA

• Yu (1998) – z-variables in DEA and SFA

1


6


Methodologies

2

Methodology – Notation

• Production function • observations,

• Input to produce output

• Deviation from the frontier • ,

• Expected inefficiency

• Environmental factors• Vector of environmental factors with impact

2


8

Methodology – conditional DEA

• DEA with firm specific reference sets (Daraio & Simar, 2005, 2007) depending on realization of s.t.

• Estimation of the reference set: Kernel estimation

• Frontier reference point is (output oriented for comparability)

2


9

Methodology – Latent Class

• LC SFA tries to account for unobserved factors and heterogeneity in technologies(Greene, 2002; Orea & Kumbhakar, 2004)

• Consideration of J classes to estimate – class-specific shape of

• Endogenous selection of class membership: multinomial logit model with

2


10

Methodology – Latent Class

Estimation: ML or MSL - Likelihood function () as function of

• - parameters of the technology – pre-specified functional form

• - parameters describing class membership

• Posterior class membership probability can be calculated as

• This class membership probability can then be used to either weight the efficiency scores – or the frontier reference points

Weighted frontier reference point:

2


11

Methodology – StoNEzD

StoNEzD for Normal-Half-Normal Noise / Ineff.

1. Stage QP: Estimation of average function

• No functional form (but piece-wise linear)

• is common to all firms

2. Stage: Decomposing residuals of first stage

• MM estimator to derive

• Shift of by expected value of inefficiencyto derive frontier estimate

Frontier reference point:

2


12

E [u ]=𝜇=𝜎𝑢√2/𝜋

Methodology – Comparison of cDEA, LC SFA and StoNEzD for production function2


13

cDEA LC SFA StoNEzD

Type Non-parametric Parametric Semi-parametric

Error / Inefficiency

Deterministic Stochastic Stochastic

Shape Constrained Parametrically constrained

Constrained

Scaling assumption

Necessary Possible Possible

Convexity of T Yes No Yes

Reference set Observation specific

All observations, weighted

All observations

Effect of z on frontier

Observation specific

Grouped, but observation specific via weighting

General effect


The DGP

3

Data Generating Process

• DGPs are created to replicate real world regulatory data

General relationship

Sample Size + 4% observations twice as large in terms of inputs

Inputs 4 correlated Inputs for small and for large firms

3


15


Functional form of • Translog

Inefficiency and Noise , with

• Noise-to-Signal:

3


16


Environmental Factors 4 different distributions considered, 1 symmetric, 3 skewed, 1 correlated

with inputs

3


17


Simulation Design and Performance Measures

4

Simulation Design

Scenarios• So far only two different scenarios: Baseline (BL) and High Impact (HI) scenarios

• Only one -Variable considered each, variation in impact

• Each scenario estimated with variation in sample size, and , for each estimator

3


19

Simulation Design

Implementation Replications: 100*9*5 = 4500 data sets for 3 estimatorsR samples for u and v for each scenariox,y and z are constant over one scenario Samples with strong deviations from the DGP

are discarded (correlations in , wrong skewness in )

StoNEzD Implemented with Sweet Spot Approach (Lee et al. 2013)MoM with set to -0.0001 if wrong skewness

occurs

Latent Class CD estimation Estimation with 2 - 4 classes, reported is max BIC5 repetitions with „randomized“ starting values

cDEA Least squares cross validation, Epanechnikov kernel

3


20

Performance measures

Performance Evaluation Evaluated at frontier reference points corrected for

Performance Measures

Equally weighted deviation in percentage points

Bias > 0 overestimation of the frontier and of inefficiency

Average squared deviation, higher impact of larger deviation

3


21


Initial Results

5

Initial Results5


23

Generally…• LC most often outperforms cDEA and StoNEzD

• Distribution of z does not seem to matter concerning bias

• Correlation of z & x has only little effect (BL4 vs. the others)

• Also magnitude of environmental effect seems to play a minor role (HI vs BL)

Initial Results5


24

• LC SFA• Performs generally well, stable and efficient

• Frontier overestimation tendecies in higher noise cases

• cDEA• High sensitivity against noise

• Underestimation of frontier in small samples, overestimation in larger samples

• StoNEzD• General underestimation of the frontier favorable for firms

• Performs well with low inefficiency and small samples

• But problems with high inefficiency

• … but does not seem to be generally efficient


Outlook

6

Conclusion and Outlook

• Additional Scenarios• Scenarios with multiple z variables

• Scenarios with heterogeneity in technologies induced by zs

• Misspecified scenarios?

• Estimation• Optimization of optimization routines – still failed estimations although the

estimated model is the true underlying model

• Suggestions?

5


26

Vielen Dank für Ihre Aufmerksamkeit.

DIW Berlin — Deutsches Institutfür Wirtschaftsforschung e.V.Mohrenstraße 58, 10117 Berlinwww.diw.de

Redaktion

Initial Results5


28

Initial Results5


29

Initial Results5


30

Initial Results5


31

Initial Results5


32

Initial Results5


33

Initial Results5


34

Initial Results5


35

References

• Agrell, P. and Bogetoft, P. (2013). Benchmarking and Regulation. CORE Discussion Papers 2013008.• Andor, M. and Hesse, F. (2014). The stoned age: the departure into a new era of efficiency analysis? a monte carlo comparison of stoned and the

oldies (SFA and DEA). JPA, 41(1):85-109.• Badunenko, O., Kumbhakar, S. (2015) When, Where and How to Estimate Persistent and Time-Varying Efficiency in Panel Data Models. WP.• Cordero, J. M., Pedraja, F., and Santin, D. (2009). Alternative approaches to include exogenous variables in DEA measures: A comparison using

Monte carlo. Comput. Oper. Res., 36(10):2699-2706.• Daraio, C. and Simar, L. (2005). Introducing Environmental Variables in Nonparametric Frontier Models: a Probabilistic Approach. JPA, 24(1):93-121.• Daraio, C. and Simar, L. (2007). Conditional nonparametric frontier models for convex and nonconvex technologies: a unifying approach. JPA,

28(1):13-32.• Greene, W. H. (2005). Reconsidering heterogeneity in panel data estimators of the stochastic frontier model. Journal of Econometrics, 126(2):269-

303.• Haney, A. B. and Pollitt, M. G. (2009). Efficiency analysis of energy networks: An international survey of regulators. Energy Policy, 37(12):5814- 5830. • Johnson, A. and Kuosmanen, T. (2011). One-stage estimation of the effects of operational conditions and practices on productive performance:

asymptotically normal and efficient, root-n consistent StoNEzD method. JPA, 36(2):219-230.• Jondrow, J., Knox Lovell, C. A., Materov, I. S., and Schmidt, P. (1982). On the estimation of technical inefficiency in the stochastic frontier production

function model. Journal of Econometrics, 19(2-3):233-238.• Krüger, J. J. (2012). A monte carlo study of old and new frontier methods for efficiency measurement. EJOR, 222:137-148.• Kuosmanen, T. (2012). Stochastic semi-nonparametric frontier estimation of electricity distribution networks: Application of the stoned method in

the Finnish regulatory model. Energy Economics, 34(6):2189-2199.• Lee, C.-Y., Johnson, A. L., Moreno-Centeno, E., and Kuosmanen, T. (2013). A more efficient algorithm for convex nonparametric least squares. EJOR,

227(2):391-400.• Orea, L. and Kumbhakar, S. C. (2004). Efficiency measurement using a latent class stochastic frontier model. Empirical Economics, 29(1):169-183.• Yu, C. (1998). The effects of exogenous variables in efficiency measurement - a monte carlo study. EJOR, 105(3):569-580.

0


36

operational conditions in regulatory benchmarking – a monte-carlo simulation stefan seifert &...

Documents