nathan economic consulting india pvt. ltd. research methods for the crew diagnostic country reports,...
TRANSCRIPT
Nathan Economic Consulting India Pvt. Ltd.
Research Methods for the CREW Diagnostic Country Reports, August 9, 2013 Zambia
2
NATH
AN IN
DIA
Introduction
This study aims to assess the state of competition and
benefits of competition reforms on consumers and producers
in selected product markets and countries, especially focusing on
the impact that past and existing public policies, regulations,
private sector business practices, institutional arrangements and
other factors have had on consumer and producer welfare*.
The goal of the project is to better demonstrate measurable
benefits from effective competition reforms in DCs, for ensuring
long-term support for competition.
*Due to discussions and concerns about the term “producer welfare” we
recommend using the term producer benefits.
3
NATH
AN IN
DIA
Introduction: Three Phases
The first phase of this project is the Diagnostic phase, which started with
the selection of four countries and two sectors for the study:
The next stage in the Diagnostic phase is to identify a standardized
framework in which the analysis is to be conducted through Diagnostic
Country Reports (DCRs).
In the Design phase, developing the toolkit to do the impact analysis.
The impact analysis will be done and quantified in the Validation phase.
Country Staple Food Bus Transport
Ghana Maize/RiceInter-City
transport and Intra-City
transport routes
India Wheat
Philippines Rice
Zambia Maize
4
NATH
AN IN
DIA
We are currently still in the Diagnostic Phase
Diagnostic
Design Validation
5
NATH
AN IN
DIA
Conceptual Framework for Phase I
6
NATH
AN IN
DIA
Objectives of the DCR
1. At the first level it will provide a detailed background on the market
structure of the 2 sectors in the 4 countries, including the regulatory
structure, using the DFID Competition Assessment Framework to answer
questions about the competitive landscape of the market.
2. Next, the DCR will provide a criteria for identifying the competition-
enhancing reforms that have already taken place, or could take place,
which would promote competition.
3. Third, after identifying the reforms, the DCR will provide the analysis
framework to:
1. define the relevant market,
2. identify players and stakeholders who will/could be affected including
both consumers and producers,
3. frame the hypotheses for testing and define the counterfactual,.
4. identifying the data required, data sources, and methods to test the
hypotheses.
It is important to remember that the DCR just provides a framework and the
actual analysis comes later.
7
NATH
AN IN
DIA
Step 1-Competition Analysis
The DFID Competition Assessment Framework will be the basis for assessing
the competitive landscape in the chosen sectors in each of the four countries.
Identifying the markets and competitors
Examining the market structure
Looking for barriers to entry (or exit)
Looking for anti-competitive conduct
Considering vested interests and the principal beneficiaries
Identifying government policies or institutions that limit competition
Use “Porter’s Five Forces of Competition” to recommend industries’ strengths
and weaknesses:
1. Nature of competition in the industry
2. Potential of new entrants into industry
3. Power of suppliers
4. Power of customers
5. Threat of substitute products
8
NATH
AN IN
DIA
Step 2-Identifying Competition Enhancing Reforms
Identify elements of competition reforms to be examined in each sector
starting with identifying reforms which are targeted or likely to have an impact
on competition. “Reforms” include:
Enabling government policies that promote competition in markets.
Appropriate regulatory framework and institutions for promoting competition in
sectors including sector-specific regulations
Effective competition enforcement institutions
The criteria for selecting the reforms/recommended reforms will be based on
the OECD’s Competition Assessment Toolkit looking at whether the
reforms/recommended reforms:
o Remove barriers to entry and increase number and range of suppliers
o Allow suppliers to compete
o Incentivize suppliers to compete
o Increase investment
Ex-post vs Ex-ante analysis
9
NATH
AN IN
DIA
Step 3 – Impact Analysis Framework
Defining the relevant market
The relevant market has to be defined so that impact analysis can be
focused.
Definition of the relevant product market
Definition of the relevant geographic market
Will likely be a qualitative analysis
Next, identify the stakeholders that could be affected:
Consumers
Focusing on only End Consumers
Producers
Incumbents
New entrants
10
NATH
AN IN
DIA
Identification of Stakeholders – Staple Food
A non-exhaustive list for discussion:
CONSUMERS PRODUCERS OTHER
General public (preferably differentiated by income
group)Farmers
Think Tanks/University Research Centers
Exporters Millers Government regulatory bodies
Distributors (Public or private) Farmers/Business Associations
Input Suppliers (such as fertilizer or seed suppliers)
Consumer Advocacy Groups
Pre-production: banks/MFIs lending to farmers
11
NATH
AN IN
DIA
Identification of Stakeholders – Bus Transport
Map relevant stakeholders along the supply chain to clearly illustrate the particular stage of the pre-production, production, or distribution stages where the impact of a particular reform measure is likely to be felt.
CONSUMERS PRODUCERS OTHER
General PublicBus operators
(Public and Private)Regulatory Bodies
Input suppliers (e.g. tires, petrol/diesel) Bus terminal operators
Think Tanks/University Research Centers
Consumer advocacy groups
12
NATH
AN IN
DIA
Determination of Hypotheses – Staple Food
A pertinent hypothesis should ask to what extent a reform measure
affected competition in the market, and how it affected consumers
and producers.
Some potential hypotheses:
Staple Food NULL HYPOTHESIS FOR STAPLE FOOD
The effect of reform has no impact on the quantity of goods sold.
The effect of reform has no impact on the quality of goods sold.
Improvements in the quantity/quality of goods rendered have no impact on the health outcomes.
The reform has no impact on intermediaries’ (millers, traders, etc) business by sales volume.
A reduction (increase) in prices has no impact on the quantity of goods sold.
The reform has no impact in number of players in the market.
The effect of entry of a new player does not have an impact on the price of the goods provided.
The effect of entry of a new player does not have an impact on the quality of the goods provided.
The reform has no effect on innovations in the sector.
13
NATH
AN IN
DIA
Determination of Hypotheses – Bus Transport
Bus Transport
It is also important to define the Counterfactual –would the
result have been the same regardless of the reform?
NULL HYPOTHESIS FOR BUS TRANSPORT
The reform has no significant impact in the quality of service.
The reform has no significant impact on consumer satisfaction.
The reform has no significant impact in the frequency of services (inter/intra state).
The reform has no significant impact in the consumers income level/other demography indicators
The reform has no significant impact in the bus manufacturers/operators financial indicators.
The reform has no impact on the price of services rendered.
The reform has not reduced bus related injuries and fatalities.
14
NATH
AN IN
DIA
Definition of Benefits to Consumer and Producers
Impacts on consumer welfare will focus on the end consumer
On the producer side, the project will look at reforms throughout
the various levels of the supply chains
Impact on producers should be measured as benefits accruing to
them from:
Ease of entry,
Ease of doing business,
Operational efficiencies.
15
NATH
AN IN
DIA
Identification of Variables and Data to be Collected
Potential variables capturing consumer welfare and benefits to
producers include:
BENEFITS TO CONSUMER/CONSUMER WELFARE
BENEFITS TO PRODUCERS
Access to goods/services Access to essential services
Quality Free movement of goods and services
Price changes Predictability of regulatory actions
Choices of new goods/services Cost savings
Time savings Fair market processes
Access to financing Productivity gains
Level-playing field
Transparency in market
Profit
16
NATH
AN IN
DIA
Identification of Variables and Data to be Collected
Potential sector -wise data to collect from primary and secondary sources:
STAPLE FOOD BUS TRANSPORT
Price data Price data
Costs of production Costs of production
Quantities produced and sold Frequency and Number of buses
Availability of substitute products Number of passengers
Changes in health outcomes Available roads and infrastructure
Number of distribution channels Substitute products and their details
Number of players in the market
Changes / reduction in fatalities and injuries
Passengers per bus/capacity of bus
Length of bus routes
17
NATH
AN IN
DIA
Data Collection - Data Sources
Data is available from a combination of
Secondary sources (census, government records, international
statistics)
Primary sources (survey, focus groups)
Secondary sources may include:
• Competition Authority, if existent
• Ministry of Commerce/Trade
• Government censuses
• Business Associations
• International donor agencies such as the World Bank
• Local research institutes
• Consumer Protection Agencies
• Publicly available online sources
If no secondary data are available, turn to primary data collection, such as
surveys or focus group discussions.
Keep in mind the variables to be measured and budget/resource constraints
18
NATH
AN IN
DIA
Primary Data Collection: Sampling Methods and Strategies
A single methodology should be employed across countries when possible
Probability sampling: random selection of sample elements from sampling frame
o Simple Random Sampling
o Random Sampling
o Stratified Random Sampling
Non-Probability sampling: should be used as a secondary option when
probability sampling is not possible
o Accidental sampling (such as ‘man on the street’)
o Snowballing
o Purposive sampling
A broad strategy including a stratified random sampling method, with
clustering to control costs if the geographic areas are large, could be utilized in this
study.
o Stratification divides elements into population subgroups, ensuring
representation from each; strata here could be defined based on region and
urban/rural
o Clustering lowers costs when populations are spread across large geographic
areas
For bus transport a non-probability sampling method might be most feasible
Nationally representative sample vs. cost constraints
19
NATH
AN IN
DIA
Primary Data Collection: Sampling Size
The sample size should be large enough for the analysis to be statistically
significant while balancing survey and project costs.
Formulas calculate the sample size (n) based on the following parameters:
o Population of the sampling frame (N)
o Degree of variance (P)
o Precision desired (α)
o Confidence interval (Z)
o Response rate (R)
Below is an illustrative example:
EXAMPLE 1EXAMPLE
2
Total population of people in the sampling frame (N) 100,000 500
Degree of variance (P) 0.5 0.5
Precision desired (α) 0.05 0.05
Confidence interval (Z) 1.96 1.96
Response rate (R ) 0.7 0.7
Sample size (n) 547 310
20
NATH
AN IN
DIA
Primary Data Collection: Survey Design
Primary data for the analysis will be collected through a survey.
Questionnaires will be designed for both producers and consumers in each
sector
o Slight modifications , including translation to local languages, will be
made on a country-specific basis, but the questions in general should be
similar across countries
Survey canvassing via a combination of email, phone and in person
interviews
Data collection using either pen and paper interviewing (PAPI) or computer
assisted personal interviewing (CAPI).
o CAPI via PDAs is recommended
Surveyor training
Piloting
Follow-up and spot checks for quality control
21
NATH
AN IN
DIA
Types of Analysis
Quantitative Analysis
oTime Series Variation
oSpatial Variation Analysis
oDifference-in-Difference
oCost-Benefit
Qualitative Analysis
oFocus Group Discussions
o In-depth Individual Interviews
oCase Studies
Cross Country Analysis
22
NATH
AN IN
DIA
Quantitative Analysis Techniques
Time Series Analysis compares data before and after a regulatory reform is
introduced
o Data availability limitations
o Carefully choose data time periods
o Use econometric rigor to avoid estimation errors
• Spatial variation is a method that compares two sets of markets which are
identical or similar in all respects, except for the regulatory constraint introduced.
o Produces more precise estimates of the effect of policy reform
o Very data intensive.
o Often difficult to find two sample sets that are characteristically similar,
except for the regulatory reform being studied.
o Difference-in-Difference combines time series analysis with spatial variation to
compare two different markets at two time periods, with a reform occurring in one
o Use multiple regression techniques to control for external factors
23
NATH
AN IN
DIA
Quantitative Analysis Techniques (cont’d)…
Regression analysis:
o Methods include ordinary least squares (OLS), two-stage least squares
(2SLS), and limited dependent variable techniques (logit or probit
models).
o Dependent variable such as price, rate of technical change
o Modeled as a function of exogenous independent variables
o Dummy variable indicates the “regulated” period; the sign and
coefficient of the dummy corresponds to the impact and size impact of
the regulation.
If data constraints don’t allow for regression analysis, other tests many be
used:
Parametric Tests
Non-parametric tests
24
NATH
AN IN
DIA
Quantitative Analysis – Cost Benefit Analysis
Measures the impact of an intervention in terms of the
Economic benefits it generates or gains to the economy relative to
Actual and opportunity costs of the intervention or losses to the
economy
Steps to undertake
• Quantify the impacts, positive and negative, tangible and intangible
• If the reform has not yet occurred, estimate the likely impact of the
expected reform
• Calculate of social costs, both the tangible and the intangible.
• Determine which are attributable to the reform
• Convert all costs and benefits to USD
• Discount the future value of costs and benefits as they accrue over time
• Compare of the costs and benefits to determine the net social rate of
return
• Use a qualitative approach whenever benefits would be intrinsic and not
measurable in monetary terms such as improvements in quality,
innovations, and new products.
25
NATH
AN IN
DIA
Qualitative Analysis
• Used in the absence of sufficient data to undertake a purely
quantitative evaluation
• Poses open-ended questions about reform strengths and shortfalls to
provide a source of new information on benefits or potential ideas for
future reforms.
• Cover all levels of stakeholders
• Focus Group Discussions (FGD) and In-depth Individual
Interviews
• Discussions with an individual or sample group in order to
understand their in-depth experiences
• Provides insight explaining the results of the quantitative
analysis
• Provide a basis for case studies
• Case study: an individual study highlighting the experience of an
individual, group, or company to give context and a personal touch to
the numbers
26
NATH
AN IN
DIA
Potential Issues
Data unavailability
Cost of collecting nationally representative data samples
How to measure: Improvements in quality, innovations, and new products
Tradeoffs between price and quality
Tradeoffs between producer and consumer welfare
Determining pass-on of benefits between various levels of consumers
How to treat transfers of producer to consumer surplus and how to define net welfare
Weighing short-run vs. long-long tradeoffs of welfare
Controlling for external factors
Attributing impact to a certain reform
Access to consistent measures across countries
Contact Us
Nathan India Nathan Economic Consulting India Private Ltd.
Chennai New DelhiNo. 45, George Ponnaiya Building, Level 4,
Rectangle 1,
TTK Road, Alwarpet, Commercial Complex D-4, Saket,
Chennai 600 018 New Delhi 110 017
T: +91 44 4293 7700 T: +91 11 4051 4076
F: +91 44 4293 7777 F: +91 11 4051 4052
Nathan Associates Inc.2101 Wilson Boulevard
Arlington, VA 22201
Washington DC Metro Area
T: 703-516-7700
F: 703-351-6162
e-mail: [email protected]
web: www.nathaninc.com
NATH
AN IN
DIA