dpme’s role re evidence, - the dpsa - department of ... dpme psrf.pdf · dpme’s role re...
TRANSCRIPT
PSRF Conference
Dr Ian Goldman Head: Evaluation and Research
28 November 2014
The Presidency Department of Planning, Monitoring and Evaluation
DPME’s role re evidence, including research
The Presidency: Department of Planning, Monitoring and Evaluation
Summary
What is the challenge
DPME’s role re evidence
Evaluations – using research for specific ends
Research – now and way forward
2
The Presidency: Department of Planning, Monitoring and Evaluation
Type of evidence most often used for policy decision-making
(interviews with 55 senior managers, PSPPD, 2012)
3
0 5 10 15 20 25
None
Opinion
Informal
Substantive
Formal Research
Scientific Research
Research Synthesis
All
More of
Current
The Presidency: Department of Planning, Monitoring and Evaluation
The main factors that influence policy (interviews with 55 senior managers, PSPPD, 2012)
4
0 5 10 15 20 25 30
Lobbyists & Pressure Groups
Habits & Traditions
Values
Resources
Judgement
Experience & Expertise
Pragmatics & Contingencies
Evidence
Future
Current
The Presidency: Department of Planning, Monitoring and Evaluation
DPME’s role re evidence
Establishment of PCAS and system of 5 year reviews sought to use evidence
NPC with diagnostic
Range of DPME’s work – outcomes monitoring, MPAT, FSD, evaluations etc seek to use evidence to inform government’s work
Explicit EBPM focus
Project in Presidency on Pro-Poor Policy Development (PSPPD) since 2007 focusing on promoting use of evidence
DPME/PSPPD on training in evidence for DGs/DDGs – run two so far – last October 2014 - 140 applied for this, so running 2 in 2015, as well as range of other initiatives
5
The Presidency: Department of Planning, Monitoring and Evaluation
Operational plan and resourcing
Review, refine and continue
What is known about the
problem
What is the change – desired
and undesired?
Options for addressing the
problem
PLANNING
Policy/Programme planning and
budgeting
OUTPUT
Implementation and monitoring
OUTCOME & IMPACT
DIAGNOSING
Analysis of the problem and
options
Evidence cycle
Theory of change
Design
Implementing the plan
Monitoring the plan, environment and budget
Understand-ing the root causes
Value for money?
Are planned outcomes
being achieved?
Document, evaluate, reflect &
learn
Agenda
Intervention Evaluations Diagnostic
evaluation/research
Smallholder
SR
Repository
Research
Evaluations -
Implementation,
Impact,
Economic
Budgeting?
EPRs
Management
Phakisa/prog
planning
guide
The Presidency: Department of Planning, Monitoring and Evaluation
Evaluations M&E key part of evidence system
Evaluations a form of research, applied to a specific end
39 evaluations completed or underway covering R50 billion of government expenditure
11 selected for 2015/16
16/46 national departments now involved in evaluations, 2 provinces have provincial evaluation plans and 4 more developing them in 2014/15
5 departments have departmental evaluation plans
Systems developed – standards, competences, 18 guidelines, trained >600 staff, QA
Cabinet taking very seriously
Evaluation Repository with >130 evaluations on it as basis for knowledge
7
Stage No
Approved reports 11
Improvement plans
being implemented
8
Served at Cabinet 6
Draft reports 10
Research underway 13
TORs approved 2
Preparation stage 11
Stuck 2
The Presidency: Department of Planning, Monitoring and Evaluation
Evaluations and MTSF
8
Outcome 7 Rural
7.1 Land admin/spatial planning
7.2 Land reform RECAP (2013/14) Restitution (2013/14) Impact of Restitution (2014/15)
7.3 Food security Nutrition for Children Under 5 (2012/13) CRDP (2013/13 CASP (2013/14) Ilima Letsema (2014/15)
7.4 Smallholder farming RECAP (2013/14) Restitution (2013/14) Impact of Restitution (2014/15) CRDP (2013/13 CASP (2013/14) Ilima Letsema (2014/15) Policy evaluation on Smallholders (2014/15) Extension Recovery Programme (2015/16)
7.5 Infrastructure and services
7.6 Non-farm rural enterprises
Outcome 8 Human settlements
8.1 Adequate housing Integrated Residential Development Programme (2012/13) Urban Settlements Development Grant (2012/13) Access to the City (2013/14) Asset Poverty (2013/14) Informal Settlements (2013/14)
8.2 Equitable residential market Social Housing (2014/15)
8.3 Coordination of spatial investment Urban Settlements Development Grant (2012/13)
And
EPRs
The Presidency: Department of Planning, Monitoring and Evaluation
First signs of impacts from evaluations
ECD – new ECD policy been developed including new focus on children from conception (first 1000 days)
Grade R – not having impact in poor schools - moving from emphasis on quantity to quality
Nutrition for Under 5s – increased emphasis, target to reduce stunting from 21-10% in MTSF
Business Process Outsourcing Scheme – scheme redesigned and relaunched
Gov Coordination Systems (clusters) – included in MTSF, developing new guidelines, strengthening secretariats and role of Presidency
Rural (several) – having to rethink and integrate programmes 8 improvement plans being implemented so far
9
The Presidency: Department of Planning, Monitoring and Evaluation
Research to date
Range of research conducted under PCAS, NPC and DPME, especially linked to 5 year reviews and NDP (eg 24 studies commissioned for 20YR)
Supported the NIDS (now going to 4th wave) – longitudinal panel study of 28 000 people where able to track changes in individuals
Set of basic data and indicators Development Indicators – working on data quality
PSPPD supporting use of evidence, notably research, funding set of research studies around social and economic development
Specific research studies, eg on norms and standards, DG time….
Establishing repository of studies in Presidency to complement evaluation repository
10
The Presidency: Department of Planning, Monitoring and Evaluation
Way forward
DPME/PSPPD doing a diagnostic on the research system, what are the systems/skills, how to strengthen, how to ensure it supports the content areas of the NDP/outcomes – how can we influence the system to deliver research related to policy needs. Bringing together Centre of Government departments including NT, DST and DPSA to support the diagnostic and see what configuration may be needed to support this
Introducing innovative tools to support including evidence/gap maps, Rapid Evidence Assessments (REAs), systematic reviews. Piloting these.
Repository of research around outcomes
Supporting departments on development of research (and evaluation) agendas around specific outcomes (already strong re human settlements, rural, economic)
Focusing evaluations on some gaps. Public Sector Research Forum to also suggest areas for evaluations
11
The Presidency: Department of Planning, Monitoring and Evaluation
We welcome the renewed focus on research and evidence.
Thank you
Ian Goldman [email protected]
www.thePresidency-dpme.gov.za
12
The Presidency: Department of Planning, Monitoring and Evaluation
Additional slides
13
The Presidency: Department of Planning, Monitoring and Evaluation
NEP 2012/13 Dept Title of evaluation Status as 5 May
dti Implementation/ design evaluation of the
Business Process Services Programme
Final report approved. Report on
way to Cabinet
DBE Impact Evaluation of Grade R Final report approved. Report
been to Cabinet
Health Implementation Evaluation of Nutrition
Programmes addressing Children Under 5
Final report approved
DRDLR Implementation Evaluation of the Land
Reform Recapitalisation and Development
Programme
Final report approved
DRDLR Implementation Evaluation of CRDP Final report approved
DHS Implementation Evaluation of the Integrated
Residential Dev. Programme
Evaluation underway. DHS
procurement has delayed
DHS Implementation Evaluation of the Urban
Settlements Development Grant (USDG)
Evaluation underway. DHS
procurement has delayed
DBE Impact Evaluation of the National School
Nutrition Programme (NSNP)
DBE requested to drop.
Reallocated to 2014/15.
14
Completing Underway About to
start
Delays!
The Presidency: Department of Planning, Monitoring and Evaluation
NEP 2013/14 Dept Title of evaluation Presidency Implementation Evaluation of Government’s Coordination Systems
dti Evaluation of Export Marketing Investment Assistance Incentive programme (EMIAI)
dti Evaluation of Support Programme for Industrial Innovation (SPII)
dti Impact Evaluation of Technology and Human Resources for Industry Programme
(THRIP)
Military
Veterans
Evaluation of Military Veterans Economic Empowerment and Skills Transferability
and Recognition Programme.
DST Evaluation of National Advanced Manufacturing Technology Strategy
SARS Impact Evaluation on Tax Compliance Cost of small businesses
COGTA Impact evaluation of the Community Works Programme (CWP)
DRDLR Evaluation of the Land Restitution Programme
DAFF Impact Evaluation of CASP
DAFF Implementation Evaluation of MAFISA
DHS Baseline for informal settlements targeted for upgrading
DHS Evaluating interventions by DHS to facilitate access to the city.
DHS Diagnostic of whether the provision of state-subsidised housing has addressed
asset poverty for households and local municipalities
DPME Impact Evaluation of the Outcomes Approach
15
Completing
by August Underway About to
start
Delays!
The Presidency: Department of Planning, Monitoring and Evaluation
2014/15
16
Dept Evaluation
DEA Evaluation of the Effectiveness of Environmental Governance in the Mining Sector
(EEGM)
DHET Design Evaluation of the Policy on Community Education and Training Colleges
(PCETC)
DHS Impact Evaluation of the Social Housing Programme (SHP)
DST Evaluation of the Indigenous Knowledge Systems Policy (IKSP)
DSD Diagnostic Evaluation/Programme Audit for Violence Against Women and Children
(AVAWC)
DSD Diagnostic Review of Coordination of the Social Sector Expanded Public Works
Programme
SAPS Economic Evaluation of the incremental investment into the SAPS Forensic Services
DAFF/DRDLR Impact Evaluation of the Ilima Letsema Programme and Irrigation Schemes
DAFF Impact evaluation of MAFISA (quantitative) – through 3ie
DAFF/DRDLR Policy Evaluation of Small Farmer Support
DBE Evaluation of the Funza-Lushaka Bursary Scheme
DBE Impact evaluation of National School Nutrition Programme
DRDLR Impact evaluation of Land Restitution Programme – through 3ie
DPME Impact/implementation evaluation of the MPAT system
DPME Implementation evaluation of the dept strategic planning and APP system
Underway TORs
developed
No TORs
yet
Delays!
The Presidency: Department of Planning, Monitoring and Evaluation
Evaluations selected for 2015/16
Impact/implementation evaluation of the evaluation system (DPME).
Implementation evaluation of citizen-based monitoring programme (DPME).
Agricultural Extension Recovery Plan (DAFF).
Evaluation of the New School Curriculum (DBE).
Evaluation of the Asset Forfeiture Unit Sub-programme (NPA).
Diagnostic evaluation of the Non-Profit Organisations Regulatory Framework and Legislation (DSD).
Implementation Evaluation of the National Drug Master Plan in addressing all forms of Substance abuse (DSD).
Evaluation of the effectiveness of the South African Post-School Education and Training (PSET) quality assurance regime (DHET).
Evaluation of Early Grade Reading in SA (DBE).
Environmental Impact Assessment (submitted for 2014/15 and need to confirm with DEA that they still wish to have the evaluation in 2015/16).
17