seda impact evaluation western cape (south africa) varsha harinath (the dti) francisco campos (world...
Post on 29-Dec-2015
217 Views
Preview:
TRANSCRIPT
SEDA IMPACT EVALUATIONWESTERN CAPE(SOUTH AFRICA)
Varsha Harinath (the dti)Francisco Campos (World Bank)
Finance and Private Sector Development IE Workshop
Dakar, SenegalFebruary 2010
Content
Background
Objective of the impact study
Impact Evaluation methodology
Process and timelines
Challenges and lessons
Next steps
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
South Africa background
Unemployment is high: 19% for males 25% for females
South Africa has identified the development of small, micro and medium enterprises (SMMEs) as means to mitigate the unemployment crisis in the country, as well as to ensure a more equitable distribution of income (Department of Trade and Industry, 1995).
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
SEDASmall Enterprise Development Services
The Small Enterprise Development Agency (seda) offers non-financial business development services to small businesses in South Africa.
The agency, overseen by the Department of Trade and Industry (the dti), was created in 2004 and operates in all 9 provinces of South Africa.
Business assessments, training as well as assistance with business plans, promotional material, technology systems, registering process are among a range of non-financial services that seda offers to clients.
Eighty percent of its support to enterprises is focused on businesses with less than 20 employees.
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Objective of the impact evaluation
The impact evaluation seeks to test if seda services in the Western Cape Province lead to improved business knowledge and practices by SMMEs
and hence have a positive impact on growth, competitiveness, sustainability and formality.
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Key outcomesImpact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Outcomes Indicators
1. Growth 1a. Increase in turnover1b. Increase in number of employees
2. Competitiveness 2a. Increase in productivity2b. Increase in net profits2c. Improvement in operational margins2d. Increase in number of customers
3. Sustainability 3a. Survival rate
4. Formality 4a. Registered4b. Certification of tax payment
Methodology
To measure the impact of seda services, we would like to observe the performance of the same enterprise with seda’s intervention and without seda’s intervention over the same period in time……….
Both states of the world do not exist at the same time!
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Methodology
We “construct” these states of the world by creating an “experiment”, whereby we randomly assign a representative group of small business to either:
A treatment group - those that receive seda’s services. A control group - those that do not receive seda’s services.
The treatment and control group is obtained from a representative sample of formal and informal small businesses in the Western Cape Province of South Africa.
To date, there is no extensive list of formal and informal small businesses in the Western Cape. Hence, for this study a multi-stage sampling technique was used , including the sampling of households and the identification of small enterprises within those households.
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
MethodolodyImpact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Populations of SMMEs
SMME Sample(906 businesses)
Control group(453 businesses)
Treatment group(453 businesses)
Average outcome for treatment group
Average outcome for control group
Impact of seda’s intervention
_ __
Process
1. Sample identification - ~9,500 sampled households surveyed to identify SMMEs owned by household members
2. Baseline survey - ~900 SMME owners surveyed on business characteristics, financial accounts, support from government, registration status
3. Intervention - seda contacts the businesses in the treatment group to complete an office visit, assess their needs and provide appropriate services.
4. Follow-up survey - information collected on outcomes for treatment and control group 12 months after the intervention
5. Analysis - impacts measured with difference in differences
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
TimelinesImpact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
2008-2009 2010 2011
Set Objectives Roll out of intervention Follow-Up survey
Design Methodology Baseline survey reports Impact evaluation final report
Budget and Implementation Plan
Design Questionnaire
Sampling
Baseline survey
Roll out of intervention
Baseline reports
Baseline data can be useful for decision-making by assessing the status of entrepreneurship, as well as differences between sub-groups and their corresponding needs
Growth Series: Profile of businesses Profile of entrepreneurs Awareness of SEDA Informality Access to finance Performance (financials) Labor Markets Challenges to growth
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Baseline dataImpact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Variable Treatment Control Difference
Male business owners (%) 53.0% 53.0% 0pp
Non-white business owners (%) 80.6% 79.7% 0.9pp
High-school business owners (%) 50.0% 52.5% -2.5pp
Average age business owner 40.9 40.2 0.7
Average number of employees 5.2 5.1 0.1
Wholesale & retail (%) 54.1% 53.4% 0.7pp
Unregistered (%) 72.9% 72.1% 0.8pp
Number of observations 453 453 0
Challenges and lessonsImpact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
Challenges Lessons
First rigorous IE study for the dti:
buy-in for a large study is difficultPolitical head lobbying for IE fast tracks the process of buy-in
IE can at first glance look expensive (especially if multi-stage sampling is done)
Multi-stage sampling improves external validity and provides rich data at the household and enterprise level
Pilot encouragement design, inviting treatment group to an event by seda at cinemas in the Western Cape yielded low turn-out rates and even lower take up rates of seda services
Significant brainstorming with programme managers & marketing teams is crucial and flexibility when implementing an encouragement may be necessary. Encouragement can be a challenge, especially when invitees are dispersed geographically. Randomised outreach intervention techniques are also usually more successful
Next steps
Office visit intervention for the Western Cape to be completed by March 2010.
Following the interest of this study in the Western Cape, the dti was also requested by seda to do an impact evaluation for the agency on a national scale. The baseline data for national seda study is piggybacking of a national small business study that is presently being conducted for South Africa.
Follow–up survey for Western Cape and for the National study in 2011.
Impact Evaluation South Africa
WB FPD Workshop, Dakar
February 2010
top related