'Best practices in application of sampling under PoA framework: DOEs perspective'

Workshop on Programme of Activities (PoA) under the CDM:Challenges and Road Ahead

07th - 8th May 2011, Bonn

UNFCCC Secretariat

2

Contents

Data sampling – Approach (general)

Data sampling & Verification - Best practice example

CDM Methodologies (general)

Sampling plan- issues to be addressed

3

Data sampling – Approach Sampling determines the reliability of the parameter value estimate expressed in terms of probability

of a parameter falling within a specified interval around a parameter’s true value

Tools/guidance (sampling guidance EB50, annex 30)

• Methodological guidance, otherwise 90/10 confidence/precision as the criteria for reliability of sampling

Over estimation of the mean value should be avoided, unbiased and reliable results ensured

Simple random sample

• most commonly used approach

• each observation chosen randomly and entirely by chance

• presents unbiased estimate of true population,

• suitable for relatively homogeneous population

Systematic sampling, Stratified random sample, Cluster sampling, Multi stage sampling

Explanation of why the sample size is reliable and accurately reflects the population

Statistically sound sampling• 95/5 precision (95% confidence interval and 5% margin of error)

• 90/10 precision (90% confidence interval and 10% margin of error)

• 90/30 precision (for eg. leakage)

Unique IDs- double counting avoided, verification status can be determined at all times

4

Real Projects- Verification Approach

Gold Standard methodology for Improved Cook-stoves and Kitchen Regimes V.01 ( CDM Meth AMS IIG)

6627 stoves were installed during the first year of the crediting period

Scalable to 1 Million plus installations

Verification challenges (MP1) Access, Travel time Limited annual ERs (~15000) – Costs On site - Man days Population characteristics, record and

installation database of every stove constructed incl. family name, ID number, location, and date of construction for all the households that receive a stove

1,788 monitoring surveys that include leakage, sustainability and qualitative fuel use data

Province Municipalities Approximate population

Santa Barbara 28 342,054

Copan 23 288,766

Lempira 28 288,766

Intibucá 17 179,862

5

Verification approach

Statistical justification, Reasonable Assurance

Key monitored parameters Number of stoves installed (~350 samples) Continued use of stoves over time – Drop off rate (alteration from original

configuration) – PDD states that this shall be done through a survey of first 50 beneficiaries who

had stoves installed within the first 12 months of the start crediting period– MR1 stated that a total of 1787 households were surveyed in 2009 and 2010; of

these only 28 stoves were found to be out of use - drop off rate of 1.57% – Does the sample selected represent the entire population (spatial and temporal) -

first 50 beneficiaries only?– Verification of 30 households in one locality (1/2 man day) –random sample

indicated 3 drop off (indicating 10% drop off)– 3 more localities selected, 30 h/h each– Sample size increased to 120 households across three randomly selected

locations (samples) – Drop off rate found constant (7.5%) across the selected samples

(90% confidence level, 5-10% error margin)

6

Verification Approach

Statistical Justification, Reasonable AssuranceEmission reduction achieved per stove per year

» Registered PDD - 2.23 mtCO2e/year per stove (qualification attached- paired sample test)

» MR1 - 2.73 mtCO2e/year per stove

Paired sample Kitchen Test for annual emission savings per stove (2010 Paired Fuelwood Consumption Study)

Measure daily fuel consumption over a 4-day period in 50 households stoves, random selection, monitored before adoption of the La Justa 2x3 vis-a vis traditional stove (fogon)

 Actual sample size taken by PP was larger (reaching n=55); and wood was weighed over a 5-day period (not just a 4-day period as required), resulting in four 24-hour periods of fuelwood consumption data rather than 3

Confidence level- 90%, SD of population- 0.168 Reliability directly proportional to numerical sample size Large sample size and paired design, sampling approach, assumptions & justification

of approach transparently reported Eliminated systematic underestimation Original datasets available, Yale university staff (Third Party) - on site

7

The sampling plan submitted by project proponents is reviewed using to assess a range of issues and questions, such as:

Does the sampling plan present a reasonable approach for obtaining unbiased, reliable estimates of the variables?

Is the data collection/measurement method likely to provide reliable data given the nature of the parameters of interest and project, or is it subject to measurement errors?

Is the population clearly defined and how well does the proposed approach to developing the sampling frame represent that population? Does the frame contain the information necessary to implement the sampling approach?

Is the sampling approach suitable, given the nature of the parameters, the data collection method, and the information in the sampling frame?

Is the proposed sample size adequate to achieve the minimum confidence/precision requirements? Is the ex ante estimate of the population variance needed for the calculation of the sample size adequately justified?

Are the procedures for the data measurements well defined and do they adequately provide for minimizing non-sampling errors? Is the quality control and assurance strategy adequate? Are there mechanisms for avoiding bias in the answer, including possible fraud?

Are the persons conducting the sampling activities qualified?

(Ref. EB50, annexure 30)

8

Methodologies (CDM Portfolio)

AMS ID : Hydro Projects, AMS IIIF (Municipal Waste), AMS IIG (Efficient cook stoves), AMS IC (eg. Solar thermal), AMS II J (CFL Lighting) Sampling approach justified, consistent monitoring plan applied across

all CPAs, technical specifications are similar, operational criteria/equipment provider same

Statistically representative Issues: temporal and spatial aspects

AMS ID & AMS IF (Hydro, Wind, Biomass, Photovoltaic) & other methodological combinations More complex Sampling approach for each category

9

Employs 59,000 people and operate a network of more than 1,000 offices and laboratories around the world

Largest verifier of CERs issued under CDM

Questions?

Thank You

Siddharth.Yadav@sgs.comContact +44 7712785772