Module 2:Basic analyses

Module 2: Learning Objectives

Identify approaches for setting targets

Understand common analyses that calculate program coverage and retention

Calculate program coverage and retention

Terminology

Indicator

Target

Program coverage

Service availability

Service utilization

Program retention

Indicator

Program element that needs tracking

Measures an aspect of a program’s performance

Measures changes over a period of time

• # of new family planning users

• # of clients currently on ART

Expressed as a number or percentage

Target Definition

A specified level of performance for a measure (indicator), at a predetermined point in time (i.e., achieve ‘x’ by ‘y’ date)

Overall target

Annual targets

Why Set Targets?

Targets help program staff with: Planning

– Staffing and service delivery– Commodities

Monitoring progress – Break long-term goals into manageable pieces – Check progress on indicators

Setting Reasonable Targets The range of values for a given indicator

can be from 0% to 100%. • Example: The theoretical range for the Polio

indicator is between 0% of children immunized (bad) and 100% immunized (ideal)

• Is it appropriate to set the Polio indicator target at 100% for a given program? Why/why not?

Setting Reasonable Targets

Example: In Somalia, the national CPR from 2007 to 2009 was15%. The following year, a national target was set for 70%.

Is it appropriate to set the CPR target for Somalia at 70%? Why/why not?

Overall Target Setting Approaches There are three approaches to set a target :

Established long-term goals by contacting that national program

Past performance (of your program, increasing by no more than 10%)

Local high performer (a stellar program nearby)

Consider the number of clients your program can realistically expect to serve during a given period of time

Annual Target Setting

Determine the increase your program needs to gain to reach your overall target

Divide that number by the number of years in which you would like to achieve the target

Add the number to your baseline indicator for each year

Considerations for Target Setting

Ensure you have an agreed-upon and realistic definition of target population

Set a realistic target to achieve in the long term and short term

Importance of Defining the Target Population: Case Example

Target was 372 children to be immunized

Actual was 488 children immunized

To calculate the % target achieved, use (Actual/Target) * 100

488/372 = 1.31*100 = 131%

How could the clinic have surpassed its target by so much?

Implications of Incorrect Target Setting: Case Example

You don’t really know to what extent you’re fully immunizing the children in your setting

If your program purchases commodities (e.g., vaccines) based on the target set, supply could run out

If you set your target too low, you may not have enough vaccines, leading to disease outbreaks

Common Analyses

Program Coverage

Extent to which a program reaches its intended target population, institution, or geographic area

Compare current performance to prior year/quarter

Compare performance between sites

Program Retention

Extent to which the range of services is being delivered as initially intended so that client drop-outs are minimal

15

Why do we need to measure coverage? To understand program progress

To determine if the target is reached Clients, commodities, adherence…

To determine if one target is reached more effectively than another

• Are there underserved area/regions, subpopulations?

Program coverage

Extent to which a program reaches its intended target population, institution, or geographic area

Utilization:

Is the target population utilizing services, accessing commodities, being reached with services?

Availability:

Are the services available where there is a need?

Utilization calculation

Percentage of the target population utilizing services

# of individuals in target population

using a service

------------------------------------------- X 100

# of individuals in target population

Utilization calculation: Example

No. of persons educated as of 6/12/09 = 300

Goal for 12/31/09 = 900

300

900

You have reached 33% of your target group with education messages

= 0.33 x 100 = 33%

Comparison of time periods

Compare percentage achieved toward target for different time periods, different sites, etc.

Rate of increase

As of January, 70 people educated; by June, 300 people

300 – 70 = 230 increase in people educated

230/6 = 38.3 new people educated per month over the 6 months

Sought prenatal care (600)

All pregnant women (2,000)

PMTCT Target (1,000)

Utilization =

Service users

Target population

Counseled & Tested for HIV (500)

Utilization of PMTCT Programs

Utilization =

600/1,000 = 0.6

0.6 x 100 = 60%

Program coverage

Extent to which a program reaches its intended target population, institution, or geographic area

Utilization:

Is the target population utilizing services, accessing commodities, being reached with services?

Availability:

Are the services available where there is a need?

Availability calculation

Number of service outlets available per target population

# of clinics with PMTCT per # of pregnant women

Expressed as a ratio

PMTCT clinic availability

There are 8 clinics offering PMTCT & 100,000 pregnant women in region X.

Ratio of clinics to pregnant women 8:100,000

Reduce to (1:12,500) pregnant women

The standard recommendation is 1 clinic with PMTCT services per 10,000 pregnant women

Clinic availability is not reaching the target

Availability + Utilization = Coverage

Service availability is 1:12,500

Service availability target is 1:10,000

PMTCT service utilization is 25% off the target

What can we conclude?

Service availability and utilization are too low; the program is not meeting the needs of pregnant women.

25

Program retention

Measures if the range of services are being delivered as initially intended

Determines program retention, i.e., is the project keeping clients through entire package of services?

• Important in clinical programs where drug adherence is an issue (TB, HIV/AIDS, immunization) and there are multiple steps (PMTCT)

Utilization

Retention example: Immunization

Completion

Tested for HIV (500)

Sought prenatal care (600)

All pregnant women (2,000 women)

PMTCT Target(1,000)

40 received prophylaxis

350 received HIV- result or no result

100 received HIV+ result

PMTCT Program Retention

Tested for HIV

Sought prenatal care

All pregnant women (2,000 women)

40 received prophylaxis

350 received HIV- result

100 received HIV+ result

1,000

500

PMTCT Program Retention

Tested for HIV (500)

Sought prenatal care (600)

All pregnant women (2,000 women)

PMTCT Target(1,000)

40 received prophylaxis

350 received HIV- result

100 received HIV+ result

PMTCT Program Retention

Tested for HIV (500)

Sought prenatal care (600)

All pregnant women (2,000 women)

PMTCT Target(1,000)

40 received prophylaxis

350 received HIV- result or no result

100 received HIV+ result

PMTCT Program Retention

Tested for HIV (500)

Sought prenatal care (600)

All pregnant women (2,000 women)

PMTCT Target(1,000)

40 received prophylaxis

350 received HIV- result or no result

100 received HIV+ result

PMTCT Program Retention

Key messages

Target Setting – A specified level of performance for a measure (indicator) at a predetermined point in time. Both overall and annual targets are set

Coverage – extent to which a program reaches its intended target population, institution, or geographic area

Retention – the extent to which the range of services are being delivered as initially intended, with clients retained throughout the full package of services