funding was contributed by the conflict and governance facility, a project of national treasury,...
TRANSCRIPT
Funding was contributed by the Conflict and Governance Facility, a project of National Treasury, which is funded by the European Union under the European Programme for
Reconstruction and Development
University of Cape Town
Department of Statistical Sciences
Monitoring and evaluating policy across the spheres and functions of government
M&E
Monitoring: A continuing function that uses systematic collection of data on specified indicators to provide management/stakeholders with indications of the extent of progress and achievement of objectives
Evaluation: The systematic and objective assessment of an ongoing or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability.
Adapted from OECD; http://www.oecd.org/dataoecd/29/21/2754804.pdf 17 Oct 2005
M&E
Motherhood and apple pie? Is there anything to discuss?
Does access to data lead to better information, lead to better decision making, lead to better performance?
Monitoring involves selecting an indicator that can provide evidence of progress towards some objective (value judgements)Assumptions: causality and the links between actions and results. To provide evidence:(1)Articulate the framework within which monitoring will occur. (2)Clarify the logic that will be applied in making the claim for evidence (of progress).
Evidence of what?
Purpose: To develop a critical appraisal of the current system of
monitoring and evaluation established by government around the Programme of Action (POA)
To propose a set of indicators that can be used to appraise performance.
To investigate the alignment of M&E processes across national and provincial spheres of government.-
To investigate the existence of appropriate data sets that can be used to calibrate indicators.
M&E of Policy in South Africa: Alignment across the Spheres and Functions of Government
Goal setting and prioritisation
MCDA Problem Structuring : defining the problem
arena, including stakeholders Specifying criteria (in M&E these could be
referred to as objectives, goals or outcomes)
Development of alternative courses of action
Preference modelling Assigning weights to the criteria
A strategic framework for M&E
Articulating Goals/ Values/ High level Objectives (or, put simply “what is the problem?”)
Developing a ‘theory of change’ on how interventions will tackle the problems within the problem context
Determining what activities will be undertaken to address the problems.
Outcomes: if all interventions are successful where will we be? The answer is the (medium to long-term) outcome. NB: Check that activities outcomes objectives/ goals.
Strategic M&EProblem
Objectives/ Goals Activities:
Programmes projects, etc
Theory of
Change
Inputs
Outputs
OutcomeDoes outcome lead to goal being fulfilled?
Ind
Ind
Ind
Ind
Ind
High level objectives for the Province
To ensure the integrity of the
physical environment
To enhance economic participation, shared growth and development
To enhance spatial integration and provide sustainable service delivery
A Home for ALL
To develop social and
human capital
To practice holistic
governance and
administratn
A set of desired outcomes for each objective
To ensure the integrity of the
physical environment
To enhance economic participation, shared growth and development
To enhance spatial integration and provide sustainable service delivery
A Home for ALL
To develop social and
human capital
To practice holistic
governance and administration
Enhanced broad-based economic participation
Enhanced provincial economic growth
Adequate spatial integration and transport infrastructure to support a growing and integrated economy
Pro-poor, shared economic growth
Significantly increased employment
Skills profile which matches the needs of the economy
Share of poorest quintile in provincial consumption (MDG).
% real GDPR growth per capita p.a.
% black ownership of companies
% of individual income spent on transport
% of Science, Engineering and
Technology graduates
Unemployment rate
To ensure the integrity of the
physical environment
To enhance economic participation, shared growth and development
To enhance spatial integration and provide sustainable service delivery
A Home for ALL
To develop social and
human capital
To practice holistic
governance and administration
Enh
ance
d br
oad-
base
d ec
onom
ic p
artic
ipat
ion
Enh
ance
d pr
ovin
cial
eco
nom
ic g
row
th
Ade
quat
e sp
atia
l int
egra
tion
and
tran
spor
t in
fras
truc
ture
to s
uppo
rt a
gro
win
g an
d in
tegr
ated
ec
onom
y
Pro
-poo
r, s
hare
d ec
onom
ic g
row
th
Sig
nific
antly
incr
ease
d em
ploy
men
t
Ski
lls p
rofil
e w
hich
mat
ches
the
need
s of
the
econ
omy
Spa
tially
inte
grat
ed,
sust
aina
ble
hum
an s
ettle
men
ts
Bas
ic s
ervi
ces
and
adeq
uate
bu
lk in
fras
truc
ture
whi
ch le
ads
to im
prov
ed q
ualit
y of
life
A p
hysi
cal e
nviro
nmen
t whi
ch
enha
nces
qua
lity
of li
fe
An
envi
ronm
ent w
hich
pro
mot
es
sust
aina
ble
livel
ihoo
ds
Sus
tain
able
acc
ess
to s
afe
ener
gy r
esou
rces
Sus
tain
able
acc
ess
to s
afe
wat
er r
esou
rces
An
effe
ctiv
e so
cial
saf
ety
net
Impr
oved
hum
an c
apita
l an
d ed
ucat
iona
l sta
tus
An
impr
oved
hea
lth p
rofil
e
Saf
e co
mm
uniti
es
Hig
h le
vels
of
soci
al c
apita
l in
com
mun
ities
Government operates in a holistic mannerConstitutional rights are protectedGovernment is effective, efficient and has good customer (citizen) relationsThere is political stability and peaceThere is democracy and accountabilityThere is adherence to the rule of lawGovernment is ethical and free of corruption
National value tree
Create a single & integrated economy that benefits all.
Impr
ove
hous
ehol
d &
com
mun
ity
asse
ts
Alleviate poverty
Reduce unemployment
Red
uce
deat
h fr
om r
oad
acci
dent
s
Add
ress
soc
ial r
oots
of
crim
inal
ity
Continue economic growth (6, 14, 22)
Enc
oura
ge la
bour
inte
nsiv
e m
etho
dsD
eal w
ith n
egat
ive
impa
ct o
f ca
sual
isat
ion
Ext
end
Exp
ande
d P
ublic
Wor
ks P
rogr
am (
28-2
9)Build a healthier
nation
Cre
ate
new
jobs
Impr
ove
oppo
rtun
itie
s fo
r se
lf-e
mpl
oym
ent &
Pro
mot
e su
stai
nabl
e co
mm
unit
y li
veli
hood
s (2
7)
Build more subsidised housing, medium den-sity housing closer to
work & provide serviced stands
Spe
ed u
p de
liver
y of
wat
er,
sani
tatio
n, p
hone
s,
heal
th,
elec
tric
ity,
hous
ing,
edu
catio
n
Pro
vide
fre
e ba
sic
wat
er &
ele
ctric
ity
Red
uce
awai
ting
tria
l num
bers
Str
engt
hen
pros
ecut
ion
& S
corp
ions
, sp
ecia
l cou
rts
Impr
ove
coor
dina
tion
amon
g re
leva
nt a
genc
ies
Mor
e, b
ette
r tr
aine
d po
lice
Pro
tect
bor
ders
(ill
egal
imm
igra
tion,
crim
e, t
erro
rism
, m
arin
e re
sour
ces)
Impr
ove
anti-
corr
uptio
n st
ruct
ures
Bui
ld c
omm
unit
y, g
ood
citi
zens
hip,
sol
idar
ity
Impr
ove
serv
ices
in g
over
nmen
t
Bet
ter
inte
ract
ion
betw
een
govt
& p
eopl
e (1
0, 5
0, 7
1)B
ette
r in
ter-
govt
. coo
pera
tion
Bet
ter
impl
emen
tati
on, m
onit
orin
g, e
valu
atio
nIn
tegr
ate
trad
itio
nal l
eade
rshi
p
Bet
ter
acce
ss to
gov
t inf
orm
atio
n
Stre
ngth
en d
emoc
rati
c in
stit
utio
ns in
cl. c
ourt
s
Help consolidate peace, security & development in southern Africa, Africa &
globe & Position SA as an effective global player
Improve safety & security
Broaden access to services & consti-tutional rights and
provide good governance
Fac
ilit
ate
broa
d-ba
sed
BE
E E
nsur
e lo
w in
tere
st &
infl
atio
n ra
tes
Spee
d up
rur
al d
evel
opm
ent &
urb
an r
enew
al
Pro
vide
com
preh
ensi
ve s
ocia
l sec
urit
y (1
0) a
nd e
nsur
e pe
ople
ac
cess
them
(6)
Promote social cohesion (14) & a non-
racist, non-sexist, democratic society
Impr
ove
eff
icie
ncy
of ju
stic
e sy
stem
(9,
55)
Inte
rven
e in
cri
me
area
s-en
hanc
e no
rmal
com
mun
al li
fe (
53)
Red
uce
crim
e &
m. d
eath
fro
m c
rim
e
Pro
vide
eff
icie
nt, s
afe
road
&ra
il tr
ansp
ort (
9, 2
6, 5
2)
Sup
port
mic
ro-lo
an f
inan
cing
Ref
orm
sup
port
age
ncie
sC
hang
e to
fin
anci
al s
yste
m a
nd in
stitu
tions
Pro
mot
e gr
owth
of
smal
l & m
icro
-ent
erpr
ises
(30
,31)
Lear
ners
hips
Edu
catio
n sy
stem
gea
red
for
prod
uctiv
e w
ork
Ski
lls t
rain
ing
Com
preh
ensi
ve a
ssis
tanc
e to
em
ergi
ng f
arm
ers,
ex
pand
agr
icul
tura
l sup
port
pro
gram
s (3
7-39
)
Com
plet
e la
nd r
estit
utio
nC
ontin
ue la
nd r
efor
m
Fac
ilit
ate
crea
tion
of
jobs
, ski
lls
and
wor
k ex
peri
ence
(27
,29,
32-
36,4
3,45
)
Pro
vide
eff
icie
nt, s
afe
tran
spor
t (9,
26, 5
2)
Com
preh
ensi
ve c
are
& t
reat
men
t of
HIV
& A
IDS
.
Exp
and
prim
ary
scho
ol n
utri
tion
pro
gm (
43)
Exp
and
prim
ary
heal
th c
are
(42)
Red
uce
prev
enta
ble
caus
es o
f de
ath-
TB
, dia
bete
s, m
alnu
trit
ion,
m
ater
nal d
eath
s, H
IV/A
IDS,
mal
aria
(9)
Rec
ruit
& r
etai
n he
alth
per
sonn
elIm
prov
e se
rvic
es in
hea
lth
faci
liti
es
Com
plet
e te
rtia
ry in
fras
truc
ture
impr
ovem
ents
(42
)
Pro
vide
the
skil
ls r
e-qu
ired
by
the
econ
omy
(27,
29,3
2-36
,43,
45)
Inve
st in
roa
d, r
ail &
air
tran
spor
t, te
leco
mm
unic
atio
ns &
ene
rgy
Incr
ease
pub
lic
& p
riva
te o
f in
vest
men
t (24
-26)
Fac
ilit
ate
econ
omic
act
ivit
y w
ithi
n th
e se
cond
eco
nom
y, P
ro-m
ote
invo
lvem
ent o
f m
argi
nali
sed
in e
cono
mic
act
ivit
y (2
7-39
)
A society that truly cares & a better life for all (as in the Bill of Rights& Constitution)
Ens
ure
invo
lvem
ent
of c
omm
uniti
es in
loca
l eco
nom
ic
initi
ativ
esP
rovi
de e
ffic
ient
, saf
e tr
ansp
ort (
9,26
, 52)
All
child
ren
with
dec
ent
clas
sroo
ms
Red
uce
teac
her/
pupi
l rat
ioS
pend
in f
avou
r of
stu
dent
s fr
om p
oor
hous
ehol
ds
Ens
ure
peop
le a
cces
s gr
ants
Impr
ove
UIF
Exp
and
scho
ol n
utrit
ion
prog
ram
.
Intr
oduc
e na
tiona
l hea
lth in
sura
nce
Fre
e he
alth
ser
vice
to
disa
bled
All
ocat
e m
ore
reso
urce
s to
wel
fare
gra
nts
for
3 yr
s (4
4)
Mul
tipu
rpos
e ce
ntre
s, e
-gov
t, et
c. (
49)
Cre
ate
a co
mpa
ssio
nate
gov
t ser
vice
(9)
Impr
ove
stat
e ca
paci
ty f
or g
row
th &
dev
elop
men
t (47
)
Impr
ove
loca
l gov
t cap
acit
y (4
8)
Impr
ove
prov
isio
n of
info
rmat
ion
(49)
Stre
ngth
en S
A’s
Ear
ly W
arni
ng M
echa
nism
Eco
nom
ic c
oope
rati
on in
sou
ther
n A
fric
aR
eali
se N
EP
AD
, str
engt
hen
AU
, SA
DC
(62
)
Coo
pera
tion
wit
h So
uth
Coo
pera
tion
wit
h in
dust
rial
ised
nat
ions
Pur
sue
mul
tila
tera
l glo
bal a
ppro
ach.
Dem
ocra
tise
UN
, IM
F, W
B,
etc.
Im
prov
e re
lati
ons
wit
h C
hina
(64
)
Eff
icie
nt lo
gist
ics
and
infr
astr
uctu
re (
59)
Stre
ngth
en d
emoc
racy
esp
. Zim
babw
e, D
RC
, Ang
ola,
als
o C
ongo
, Sw
azil
and
also
Tan
zani
a, K
enya
, Bur
undi
, Rw
anda
, Gha
na,
Sene
gal,
Ivor
y C
oast
, Sud
an, E
thio
pia,
Som
alia
, Mah
greb
) (6
0,
61)
Red
uce
Cor
rupt
ion
(10)
Create a single & integrated
economy that benefits all.
Alleviate poverty
Reduce unemployment
Continue economic
growth
Build a healthier
nation
Help consolidate peace, security & development in
southern Africa, Africa & globe & Position SA as
an effective global player
Improve safety & security
Broaden access to services & constitutional
rights and provide good governance
Promote social cohesion & a non-racist, non-sexist, demo-cratic society
A society that truly cares & a better life for all (as in the Bill of Rights& Constitution)
Create a single & integrated
economy that benefits all.
Reduce unemployment
Enc
oura
ge la
bour
inte
nsiv
e m
etho
ds
Dea
l wit
h ne
gati
ve im
pact
of
casu
alis
atio
n
Ext
end
Exp
ande
d P
ubli
c W
orks
P
rogr
am
Create new jobs Improve opportunities for self-employment & Promote sustainable community livelihoods
Sup
port
mic
ro-l
oan
fina
ncin
g
Ref
orm
sup
port
age
ncie
s
Cha
nge
to f
inan
cial
sys
tem
and
in
stit
utio
ns
Promote growth of small & micro-enterprises
Lea
rner
ship
s
Edu
cati
on s
yste
m g
eare
d fo
r pr
oduc
tive
wor
k
Ski
lls
trai
ning
Com
preh
ensi
ve a
ssis
tanc
e to
em
ergi
ng f
arm
ers,
exp
and
agri
cult
ural
sup
port
pro
gram
s
Com
plet
e la
nd r
esti
tuti
on
Con
tinu
e la
nd r
efor
m
Facilitate creation of jobs, skills and work experience
Facilitate economic activity within the second economy, Pro-mote involvement of marginalised in economic activity
A society that truly cares & a better life for all (as in the Bill of Rights & Constitution)
Ens
ure
invo
lvem
ent o
f co
mm
unit
ies
in lo
cal e
cono
mic
in
itia
tive
s
Pro
vide
eff
icie
nt, s
afe
tran
spor
t
Build a healthier
nation
Com
preh
ensi
ve c
are
& tr
eatm
ent
of H
IV &
AID
S.
Exp
and
prim
ary
scho
ol n
utri
tion
pr
ogra
m
Exp
and
prim
ary
heal
th c
are
Reduce preventable causes of death-TB, diabetes, malnutrition, maternal deaths, HIV/AIDS, malaria
Recruit & retain health personnel
Improve services in health facilities
Complete tertiary infra-structure improvements
A society that truly cares & a better life for all (as in the Bill of Rights& Constitution)
Improve services in govt
Better interaction between govt & people
Better inter-govt. cooperation
Better implemen-tation, monitoring, evaluation
Integrate traditional leadership
Better access to govt information
Strengthen democratic institutions incl. courts
Broaden access to services & constitutional rights and provide good
governance
A society that truly cares & a better life for all (as in the Bill of Rights& Constitution)
Multipurpose centres, e-govt, etc.
Create a compassio-nate govt service
Improve state capacity for growth & develop-ment
Improve local govt capacity
Improve provision of information
Reduce corruption
Create a single & integrated economy
that benefits all.
Alleviate poverty
Reduce unemployment
Build more subsidised housing, medium density housing closer to work& provide serviced stands
Improve household & community assets
A society that truly cares & a better life for all (as in the Bill of Rights & Constitution)
Sp
ee
d u
p d
eli
ve
ry o
f w
ate
r, s
an
ita
tio
n,
ph
on
es
, h
ea
lth
, e
lec
tric
ity
, h
ou
sin
g,
ed
uc
ati
on
Pro
vid
e f
ree
ba
sic
w
ate
r &
ele
ctr
icit
y
(a) Percentage of population housed in informal/formal dwellings.(b) Percentage of the population which is homeless.
(a) Proportion of population with access to an improved water source (both urban and rural) (Target 10 Indicator 30)(b) Proportion of population with access to improved sanitation (Target 10 Indicator 31)(c) % of gdp spent on sanitation infrastructure(d) Proportion of households with access to electricity
The following is the heading and definition for ALL of the indicators mentioned on this page:Housing: These indicators measure the marginality of human living conditions
Co
mp
reh
en
siv
e
as
sis
tan
ce
to
e
me
rgin
g f
arm
ers
, e
xp
an
d a
gri
cu
ltu
ral
su
pp
ort
pro
gra
ms
Co
mp
lete
la
nd
re
sti
tuti
on
Co
nti
nu
e l
an
d r
efo
rm
Improve opportunities for self-employment & Promote sustainable community livelihoods
(e) Proportion of HH with access to secure tenure (owned or rented) (Target 10 Indicator 32)
Links to GW M&E system
Create a single & integrated economy that benefits all.
Impr
ove
hous
ehol
d &
com
mun
ity
asse
ts
Alleviate poverty
Reduce unemployment
Continue economic growth (6, 14, 22)
Enc
oura
ge la
bour
inte
nsiv
e m
etho
dsD
eal w
ith n
egat
ive
impa
ct o
f ca
sual
isat
ion
Ext
end
Exp
ande
d P
ublic
Wor
ks P
rogr
am (
28-2
9)Build a healthier
nation
Cre
ate
new
jobs
Impr
ove
oppo
rtun
itie
s fo
r se
lf-e
mpl
oym
ent &
Pro
mot
e su
stai
nabl
e co
mm
unit
y li
veli
hood
s (2
7)
Build more subsidised housing, medium den-sity housing closer to work & provide serviced stands
Spe
ed u
p de
liver
y of
wat
er,
sani
tatio
n, p
hone
s,
heal
th,
elec
tric
ity,
hous
ing,
edu
catio
n
Pro
vide
fre
e ba
sic
wat
er &
ele
ctric
ity
Fac
ilit
ate
broa
d-ba
sed
BE
E E
nsur
e lo
w in
tere
st &
infl
atio
n ra
tes
Spee
d up
rur
al d
evel
opm
ent &
urb
an r
enew
al
Pro
vide
com
preh
ensi
ve s
ocia
l sec
urit
y (1
0) a
nd e
nsur
e pe
ople
ac
cess
them
(6)
Sup
port
mic
ro-lo
an f
inan
cing
Ref
orm
sup
port
age
ncie
sC
hang
e to
fin
anci
al s
yste
m a
nd in
stitu
tions
Pro
mot
e gr
owth
of
smal
l & m
icro
-ent
erpr
ises
(30
,31)
Lear
ners
hips
Edu
catio
n sy
stem
gea
red
for
prod
uctiv
e w
ork
Ski
lls t
rain
ing
Com
preh
ensi
ve a
ssis
tanc
e to
em
ergi
ng f
arm
ers,
ex
pand
agr
icul
tura
l sup
port
pro
gram
s (3
7-39
)
Com
plet
e la
nd r
estit
utio
nC
ontin
ue la
nd r
efor
m
Fac
ilit
ate
crea
tion
of
jobs
, ski
lls
and
wor
k ex
peri
ence
(27
,29,
32-
36,4
3,45
)
Pro
vide
eff
icie
nt, s
afe
tran
spor
t (9,
26, 5
2)
Com
preh
ensi
ve c
are
& t
reat
men
t of
HIV
& A
IDS
.
Exp
and
prim
ary
scho
ol n
utri
tion
pro
gm (
43)
Exp
and
prim
ary
heal
th c
are
(42)
Red
uce
prev
enta
ble
caus
es o
f de
ath-
TB
, dia
bete
s, m
alnu
trit
ion,
m
ater
nal d
eath
s, H
IV/A
IDS,
mal
aria
(9)
Rec
ruit
& r
etai
n he
alth
per
sonn
elIm
prov
e se
rvic
es in
hea
lth
faci
liti
es
Com
plet
e te
rtia
ry in
fras
truc
ture
impr
ovem
ents
(42
)
Pro
vide
the
skil
ls r
e-qu
ired
by
the
econ
omy
(27,
29,3
2-36
,43,
45)
Inve
st in
roa
d, r
ail &
air
tran
spor
t, te
leco
mm
unic
atio
ns &
ene
rgy
Incr
ease
pub
lic
& p
riva
te o
f in
vest
men
t (24
-26)
Fac
ilit
ate
econ
omic
act
ivit
y w
ithi
n th
e se
cond
eco
nom
y, P
ro-m
ote
invo
lvem
ent o
f m
argi
nali
sed
in e
cono
mic
act
ivit
y (2
7-39
)
A society that truly cares & a better life for all (as enshrined in the Bill of Rights, constitution)
Ens
ure
invo
lvem
ent
of c
omm
uniti
es in
loca
l eco
nom
ic
initi
ativ
esP
rovi
de e
ffic
ient
, saf
e tr
ansp
ort (
9,26
, 52)
All
child
ren
with
dec
ent
clas
sroo
ms
Red
uce
teac
her/
pupi
l rat
ioS
pend
in f
avou
r of
stu
dent
s fr
om p
oor
hous
ehol
ds
Ens
ure
peop
le a
cces
s gr
ants
Impr
ove
UIF
Exp
and
scho
ol n
utrit
ion
prog
ram
.
Intr
oduc
e na
tiona
l hea
lth in
sura
nce
Fre
e he
alth
ser
vice
to
disa
bled
All
ocat
e m
ore
reso
urce
s to
wel
fare
gra
nts
for
3 yr
s (4
4)
Measurement issues
Simple vs composite indicatorsParsimonious modelsRobust or sensitive to changeLocal or global viewsError (measurement, sampling, statistical
bias, interpretation)Measuring (step 1) and valuing (step 2)
Some conclusions…
Pervasive lack of clarity on high level objectives… value trees might be a useful tool
Priority. What’s that? Danger of over-engineering Danger of separating M&E from strategic
planning (activities may not be aligned with desired outcomes)
Tremendous benefits of awareness created by MTEF…. but
Lack of quantitative/ problem solving/ modeling skills
There is a strong need for a common strategic planning process which is individually tailored by each department/ govt entity within the context of clear national, provincial, local objectives.
Strategic M&E