human capacity challenges in the implementation of a...
TRANSCRIPT
University of the Witwatersrand
Graduate School of Public & Development Management
(P&DM)
Human Capacity Challenges in the
Implementation of a Monitoring and Evaluation
System
By
Mompheleng Ernest Maphunye
(9813052R)
A Research Thesis Submitted in Partial Fulfilment of the
Requirements for a Masters Degree in Management and
Public Policy (MM-PP)
2013
TABLE OF CONTENTS
ABSTRACT ................................................................................................................... iii
ABBREVIATIONS AND ACRONYMS ......................................................................... iv
LIST OF TABLES .......................................................................................................... v
CHAPTER ONE ............................................................................................................. 1
1.1 INTRODUCTION ................................................................................................... 1
1.2 RESEARCH BACKGROUND .............................................................................. 3
1.3 RESEARCH PROBLEM (PROBLEM STATEMENT).......................................... 5
1.4 PURPOSE OF THE RESEARCH ......................................................................... 6
1.5 PRIMARY AND SUPPLEMENTARY RESEARCH QUESTIONS .......................... 6
1.6 STRUCTURE OF THE STUDY (OUTLINE OF THE CHAPTERS) .................... 7
CHAPTER TWO ............................................................................................................. 9
LITERATURE REVIEW .............................................................................................. 9
2.1 INTRODUCTION .................................................................................................. 9
2.2 THEORETICAL FRAMEWORK .......................................................................... 9
2.3 CONCEPTUAL FRAMEWORK ........................................................................... 16
2.4 CONCLUSION: OBSERVATIONS ON THE REVIEWED LITERATURE ........ 24
CHAPTER THREE ....................................................................................................... 26
RESEARCH METHODOLOGY ................................................................................ 26
3.1 INTRODUCTION ................................................................................................ 26
3.2 RESEARCH APPROACH .................................................................................. 26
3.3 RESEARCH DESIGN ......................................................................................... 28
3.4 SECONDARY SOURCES .................................................................................. 33
3.5 CONTENT ANALYSIS ....................................................................................... 34
3.6 RELIABILITY AND VALIDITY ........................................................................... 36
3.7 RESEACHER’S BIASES ................................................................................... 37
3.8 CONCLUSION .................................................................................................... 37
CHAPTER FOUR ......................................................................................................... 38
PRESENTATION, ANALYSIS AND DISCUSSION OF FINDINGS ....................... 38
4.1 INTRODUCTION ................................................................................................ 38
4.2 PRESENTATION OF THE FINDINGS .............................................................. 39
4.3 ANALYSIS OF THE FINDINGS (DISCUSSIONS) ............................................ 47
4.4 SECONDARY DATA ANALYSIS ...................................................................... 50
4.5 CONCLUSION .................................................................................................... 51
CHAPTER FIVE ........................................................................................................... 52
SYNTHESIS OF THE ANALYSIS TO THE FINDINGS .......................................... 52
5.1 INTRODUCTION ................................................................................................ 52
5.2 CONCLUSION ..................................................................................................... 54
CHAPTER SIX ............................................................................................................. 56
CONCLUDING REMARKS ...................................................................................... 56
REFERENCE LIST........................................................................................................ 58
ANNEXURE A: The Department of Local Government and Traditional Affairs
(DLGTA) Proposed Municipal M&E System ........................................................... 63
DECLARATION
I Mompheleng Ernest Maphunye (student no. 9813052R) hereby declare that this
thesis, in partial fulfilment of the requirements for a Masters Degree in Management
and Public Policy (MM-PP) 2013, submitted by myself, has not previously been
submitted for a degree at this or any other University. It is my own work in design
and in extension and all reference material contained therein has been duly
acknowledged.
…………………. ………………
Signature Date
ACKNOWLEDGEMENTS AND DEDICATIONS
This work was made possible by the supervision, guidance, assistance and valuable
input of Mr. Stephen Porter. Thank you kindly for taking me under your wing and
embarking on this journey with me. I would also like to humbly thank my former and
current colleagues at the Gauteng COGTA who took their valuable time to assist in
the data collection process. To family, friends and loved ones, thank you kindly for
the support, understanding and words of encouragement from the beginning right
until the end of this journey.
Omphemetse Goene Suping...you are an inspiration.
iii
ABSTRACT
A Monitoring and Evaluation (M&E) system is important in a country or public sector
organisation as it can assist in the understanding of a country or organisation’s M&E
efforts. Information acquired from using the M&E system will then lead to a clearer
understanding of the current M&E initiatives, the overall environment where the
system is used and some of the institutional arrangements involved. Using the
system can also be looked at from a viewpoint of improving the current M&E
initiatives and using M&E information obtained from using the system to assist and
benefit the intended stakeholders.
Based on the above remarks around M&E systems, this study explored the
challenges faced by the Gauteng Department of Local Government and Traditional
Affairs (DLGTA), in implementing a municipal M&E system. In a probe of the
challenges related to the implementation of this system, the study found that
department’s lack to properly capacitate the municipal service delivery M&E unit with
the required and skilled personnel mostly impacted the department’s efforts to
implement a municipal M&E system.
The study also found that the challenges of not implementing a municipal M&E
system was as a result of the DLGTA as an organisation not properly viewing itself
as the coordinating body of municipal M&E in the Gauteng province. By not
recognising itself as the coordinating body of M&E in the province, the DLGTA ’s
efforts were minimal in terms of capacitating its service delivery M&E unit with the
right number of personnel, but also highly skilled personnel who have a solid M&E
background, training and who could ensure the implementation of a municipal M&E
system.
A major consequence of the above departmental shortcomings is that they lead to
the human capacity challenges of the service delivery M&E unit of the DLGTA that
contributed to the non-implementation of a municipal M&E system. This is the
system that should be used to conduct municipal M&E and also assist Gauteng
municipalities with their M&E functions as mandated by legislation.
iv
ABBREVIATIONS AND ACRONYMS
AO: Administration Officers
AG: Auditor General
CABRI: Collaborative Africa Budget Reform Initiative
DLGH: Department of Local Government and Housing
ECB: Evaluation Capacity Building
DLGTA: Gauteng Department of Local Government and Traditional Affairs
GCOGTA: Gauteng Department of Cooperative Governance and Traditional
Affairs
GNU: Government of National Unity
GWM&E: Government Wide Monitoring and Evaluation Policy Framework
IGR: Intergovernmental Relations
INTRAC: International NGO Training and Research Centre
KPA: Key Performance Area
MEC: Member of the Executive Council
M&E: Monitoring and Evaluation
MFMA: Municipal Finance Management
MSA: Municipal Systems Act
NPM: New Public Management
PMER: Planning, Monitoring, Evaluation and Reporting
PRC: Presidential Review Commission
v
LIST OF TABLES
Table 1: Components of a functional M&E system
Table 2: Components of a functional M&E system
Table 3: Attributes or competencies of a capacitated M&E official
Table 4: Questionnaire Example
Table 5: Dimension and Purpose for asking Questions
Table 6: Proposed DLGTA Municipal M&E System
1
CHAPTER ONE
1.1 INTRODUCTION
This study explored the challenges faced by the Gauteng Department of Local
Government and Traditional Affairs (DLGTA), municipal service delivery monitoring
and evaluation (M&E) unit, in implementing a municipal M&E system. As an attempt
to probe these challenges, the study found how the lack to focus on the “human
capacity element for M&E systems” by the DLGTA, impacted the municipal service
delivery M&E unit’s efforts to implement a municipal M&E system. The inability of the
municipal service delivery M&E unit of the DLGTA in not implementing a municipal
M&E system has largely resulted in municipalities of the Gauteng province not being
assisted with certain elements of their M&E functions by the DLGTA, as mandated
by legislation.
The study further explored whether can the challenges faced by the service delivery
M&E unit of the DLGTA in the implementation of a municipal M&E system be rooted
either to the institutional (organisation) make up of the DLGTA or the inability of the
DLGTA to prioritise the capacitation of the service delivery M&E unit in terms of
human capacity and the required skills. In the same sentiments, the study also
explored whether the DLGTA as an organisation has been aware of the importance of
M&E systems, human resource capacity requirements within M&E systems and the
importance of capacity building in M&E systems.
In an attempt to understand the challenges encountered by the Gauteng DLGTA
municipal service delivery M&E unit in implementing its municipal M&E system, this
study was premised around the theoretical underpinnings of the New Public
Management Theory (NPM). For the purposes of this study and related to the M&E
phenomenon, the NPM theory and its public sector reforms are significant as they
put forward a new wave of thinking for governments which is based on a broader
level, results-based, turnaround strategy of improving efficiency, effectiveness and
overall, the general performance of the public service. Within this discussion, the
phenomenon of M&E can be said to be an important part of this turnaround strategy.
2
The study also touched on the public sector reform phenomenon of Evaluation
Capacity Building (ECB) that can be linked to the provisions of the NPM theory. The
NPM theory and the ECB component were used as examples to indicate how
government administrations should improve themselves and how the DLGTA as an
organisation has not applied their provisions by improving itself and strengthen its
human capacity element for the implementation of a municipal M&E system by the
service delivery M&E unit.
An M&E system is important in a country as it leads to an understanding of a
country’s socioeconomic and political M&E efforts. The analysis from these M&E
efforts will then lead to a clearer understanding of the current M&E initiatives, the
overall public sector environment, its institutional arrangements and opportunities for
strengthening and improving the current M&E initiatives, as well as using M&E
information to benefit the intended stakeholders. The importance of this information
is that it can be used for core government functions such as budget decision making
and the ongoing management of programs and projects. More importantly, this
analysis will help key role players in government and the donor community discover
the strength and weaknesses of M&E as well as the institutional arrangements
(MacKay, 2007: 3).
At a municipal level, a part of M&E functions includes, amongst others, municipalities
reporting on their performance to the respective national and provincial government
departments, legislature as well as other related local government stakeholders. This
means municipalities are supposed to produce credible quarterly, midyear and
annual performance information reports that will be used for performance review and
assessment by the above mentioned stakeholders. As guided by the Government
Wide Monitoring and Evaluation policy framework (GWMEPF), the DLGTA’s
mandated role to municipalities involves the provision of assistance for them to
effectively perform their M&E functions. The assistance will also play a role in
municipal reporting and enable municipalities to produce credible, consistent and
reliable performance reports as part of their M&E functions. This assistance to
municipalities will also mean that the DLGTA is also broadly implementing the
GWM&E policy framework.
3
Due to a lack of a municipal M&E system which is made up of all the components of
a functional M&E system especially the human capacity element which will also
specify how the unit’s M&E functions towards municipalities should be carried out,
municipalities in the Gauteng province continued to struggle with the above
mentioned M&E functions. Though some support was provided to municipalities on
other levels, municipalities were still not provided with the assistance they require
that will enable them to perform a crucial part of their M&E functions which includes
producing performance information reports that are of acceptable and required
standards. This study then investigated how the challenges related to the lack to
focus on the “human capacity element for functional M&E systems”, largely
contributed to the inability of the municipal service delivery M&E unit of the DLGTA
to implement a municipal M&E system.
1.2 RESEARCH BACKGROUND
On the 27th of February 1998, a report was presented to the then President of the
Republic of South Africa, President Nelson R. Mandela, by a commission named the
Presidential Review Commission (PRC). The importance of this report was that it
highlighted that government’s progress in transforming public service delivery was
severely lacking and this called for a development of a performance monitoring and
evaluation white paper. In some of their findings, the PRC indicated that amongst
other challenges facing the Government of National Unity (GNU)1 was a lack of
effective M&E mechanisms that would regularly monitor, evaluate and review the
performance of government. From the PRC findings, it can be concluded that
institutional arrangements within government lacked M&E systems to measure
performance and evaluate government policy outcomes.
Related to this study, concerns raised by the PRC were that annual reports that are
used as part of M&E reporting mechanisms by national, provincial and local
governments were largely scanty and not always helpful. The PRC also indicated
that being reporting mechanisms, these annual reports largely failed with regard to
objectivity in outlining the successes and failures of previous years. In addition,
1 “That took over from the National Party after 1994”
4
annual reports were not flexible enough to give immediate feedback to policy makers
and implementers, in terms of performance (PRC, 1998: 27). As intervention
strategies, the PRC recommended that there must be mechanisms in place to review
legislation, in particular, ensuring compliance with the then new constitution as well
as improved systems of M&E in order to ensure efficiency and effectiveness within
government (PRC, 1998: 27-28).
With the above in mind, in 2004 South Africa’s cabinet initiated plans for an M&E
system for government. The presidency subsequently developed the GWM&E policy
framework. The foundations of this policy framework were largely rooted in the
resolutions of the PRC. The GWM&E policy framework was initiated to support the
improvement of the collection and collation, analysis, dissemination and application
of information on the progress and impact of government, including its programmes.
This was done in order to ensure transparency and accountability and to promote
service delivery improvement and compliance with statutory and other requirements,
as well as to create a learning culture in the public sector. The GWM&E policy
framework was also initiated for programme managers in the public sector as the
first comprehensive guideline after the cabinet of the Republic of South Africa
approved a process to plan an M&E system for use across government. During its
development the GWM&E policy framework was meant to deliver useful information
and analysis and, at the same time, improve M&E practices, mechanisms and
capacity, and in the long run contribute to better public management in South Africa
(Presidency, 2004: 1-6).
Legislatively speaking, Section 155 (6) (a) of the Constitution of the Republic of
South Africa (1996) serves as one of the foundations of the GWM&E policy
framework as it states that “each provincial government must by legislative or other
measures provide for the monitoring and support of local government in the
province”. In addition, Section 92 of the same constitution provides that, “members of
the cabinet are accountable collectively and individually to parliament for the
exercise of their powers and the performance of their functions, in addition they must
provide parliament with full and regular reports concerning matters under their
control” (Constitution of the Republic of South Africa, 1996, Section 155 (6) & 92).
5
To complement the Constitution, Section 105 (1) of the Municipal Systems Act
(MSA) obliges the Member of the Executive Council (MEC) for local government in a
province to establish mechanisms, processes and procedures in terms of section
155 (6) of the constitution to:
(a) Monitor municipalities in the province in managing their own affairs,
exercising their powers and performing their functions,
(b) Monitor the development of local government capacity in the province
and,
(c) Assess the support needed by municipalities to strengthen their
capacity to manage their own affairs, exercise their powers and perform
their functions (MSA, 2000, section 105).
1.3 RESEARCH PROBLEM (PROBLEM STATEMENT)
Due to a lack of a municipal M&E system which is made up of all the requirements of
a functional M&E system (especially the human capacity component), the municipal
service delivery M&E unit through the DLGTA as the department responsible for
local government in the Gauteng province, has not been in a position to properly
execute its legislated mandate. This mandate includes assisting Gauteng
municipalities with their reporting M&E functions, as mandated by the Constitution
and the MSA.
The improper, flawed and inconsistent performance information reports produced by
municipalities in the Gauteng province highlighted a broader problem which is the
way in which the lack to focus on the human capacity element for functional M&E
systems by the municipal service delivery M&E unit of the DLGTA, impacted their
implementation of a municipal M&E system. This can be considered as one of the
reasons on why municipalities were not been assisted with some of their M&E
functions as mandated.
Through an institutionalised, and later implemented, municipal M&E system, made
up of all the components of a functional M&E system, especially the human capacity
component, the municipal service delivery M&E unit of the DLGTA was going to be
in a better a position to guide and assist municipalities in the Gauteng province with
6
their M&E functions, which include compiling and producing credible performance
information reports that are of acceptable standards. In a broader sense, this could
also be considered as assisting municipalities with their M&E functions, as mandated
by legislation.
1.4 PURPOSE OF THE RESEARCH
The purpose of this study therefore, became to firstly explore how the lack to focus
on and strengthening of the “human capacity element for functional M&E systems”,
by the municipal service delivery M&E unit of the DLGTA impacted in the
implementation of a municipal M&E system. The study therefore used the NPM
theory to explore how M&E capacity building can contribute towards understanding
how governments can use M&E to improve their performance. This was done
through exploring how the DLGTA as an organisation was not in line with its
provisions by improving itself to strengthen its human capacity element for the
implementation of a municipal M&E system. The study also used the provisions of
the NPM theory as part of a measure to propose or draft an M&E system centred on
the human capacity element that the municipal service delivery unit of the DLGTA
can implement in their efforts to carry out the M&E of municipalities in the Gauteng
province.
1.5 PRIMARY AND SUPPLEMENTARY RESEARCH QUESTIONS
With the above reiterations in mind, the study was guided by the following question
as the primary research question:
What were the challenges faced by the municipal service delivery
M&E unit of the DLGTA in implementing a municipal M&E system?
Furthermore, the study also used the following questions as supplementary questions to
further unpack the above primary question:
1. How did the DLGTA as an organisation view the role of the service
delivery M&E unit?
2. Was the service delivery M&E unit properly capacitated to carry out
its M&E functions?
7
3. How skilled are members of the service delivery unit in carrying out
their M&E functions?
1.6 STRUCTURE OF THE STUDY (OUTLINE OF THE CHAPTERS)
As a follow up to this chapter, the second chapter of the study is a literature review
based on establishing a conceptual framework for the topic, definition of key terms,
related terminology and concepts, identification of models supporting the study as
well as defining or establishing the study’s stance on the literature review. In
establishing a theoretical framework for the study, the theory of NPM was considered
as key in conceptualising the study. The study also looked at the phenomenon of
Evaluation Capacity Building (ECB) public sector reform linked to the theory of NPM.
Related to the phenomenon of M&E as a turnaround strategy, the NPM theory and
the ECB component were used as benchmarks or examples indicating how
governments should turn themselves around and also improve themselves through
capacitation efforts. This helped to explain how the DLGTA did not apply their
provisions by improving and turning itself around in the form of strengthening its
human capacity element for the implementation of a municipal M&E system that will
enable municipalities to be assisted with their M&E reporting functions.
The literature review chapter also included a section that discussed a conceptual
framework that guided the study’s research. The purpose of this discussion was to
locate the relevance of the theories and concepts used as departure points for the
research. In this section, key theories and concepts used in the research and their
relationship were unpacked, for example, the relationship between the theory of
NPM and M&E, as well as how the human capacity element and the phenomenon of
M&E are related.
Chapter 3 of the study was based on the research design that the researcher
followed or used concerning the research approach. The chapter touched on data
collection methods regarding primary data collection, secondary sources, content
analysis and data validation methods embarked upon in the collection, analysis and
validation of the study’s data. The researcher’s biases were also included in the
chapter.
8
Chapter 4 of the study presents the findings from implementing the methodology, as
well as an interpretation of the close findings with evidence. Chapter 5 of the study
was a synthesis of the analysis to the findings raised in chapter 4. The aim was to
present what was made out of the findings, their impact and implications. This
included proposed measures as the study also proposed an M&E system centred on
the human capacity element that the DLGTA can follow for the fulfilment of its role in
assisting municipalities with their M&E functions and in the broader sense also to
implement the GWM&E policy framework. A part of this included a reflection of why
the DLGTA needed to strengthen the human capacity element of its municipal M&E
system. Chapter 6 of the study was based on concluding remarks for the study and
in the same context focusing on the study’s main findings.
9
CHAPTER TWO
LITERATURE REVIEW
2.1 INTRODUCTION
This chapter presents the surveyed literature that guided this study’s research. The
chapter also established a conceptual framework, defined key terms, related
terminology and concepts, identified models supporting the study as well as
established the study’s stance on the surveyed literature. The chapter also reflected
on the critical internal documents of the service delivery M&E unit that had a bearing
on the study as a way to substantiate some of the arguments brought forward. A
discussion of the key concepts used in the research and their relationship was also
unpacked as part of the conceptual framework. The purpose of this literature review
was also to locate the relevance of the theories and concepts used as departure
points for the research.
There exists a substantial amount of literature on the subjects of M&E, M&E
systems, their importance, as well as the human capacity element for such systems.
In essence, the review encapsulated views about the critical issues involved in M&E,
M&E systems, the importance of the human capacity element for these systems
including the phenomenon of ECB. In terms of literature review types, this was a
“Context Review”. This is “a common type of review in which the author links a
specific study to a larger body of knowledge. This type of review often appears at the
beginning of a research report and introduces the study by situating it within a
broader framework and showing how it continues or builds on a developing line of
thought or study.” (Neuman, 2006: 112).
2.2 THEORETICAL FRAMEWORK
2.2.1 NEW PUBLIC MANAGEMENT (NPM)
To begin with, and in establishing a theoretical framework for the M&E subject area,
the review looked into the theory of New Public Management (NPM). According to its
advocates, NPM is based on an approach in public administration that employs
knowledge and experiences acquired in business management. Vigoda (2003)
10
emphasised that NPM should represent an approach in public administration that
employs knowledge and experiences acquired in business management and other
disciplines to improve efficiency, effectiveness, and general performance of public
services in modern bureaucracies (Vigoda, 2003: 1). The term “New Public
Management” was coined in 1989 by Christopher Hood to denote “shifts from an
emphasis on policy towards an emphasis on measurable performance and from
reliance on bureaucracies toward loosely coupled quasi- autonomous units and
competitively tendered services (Lynn, 2006: 107).
The theory of NPM can also be associated with a set of operating principles and
practical solutions to the operational problems confronting governments, as put
forward by Miller and Dunn (2006). They argued that these principles were
generated as remedies for a broken system of government (Miller & Dunn, 2006: 2).
Operationally speaking, NPM is relevant for the public sector as it proposes for
governments being driven by their missions of improving their effectiveness and
efficiency (Osborne and Gaebler: 1993). On this point, it is worth noting that the
theory of NPM signifies an ideological thought system characterised by the
importation of ideas generated in private sector settings within public sector
organisations. This means that NPM advocates for a fusion of private and public
sector management ideas which aim to re-energise public sector managers by
outlining a distinct public service mission which is compatible with received notions
of high quality management derived from transferable good practice in the private
sector (Osborne & Gaebler, 1993). In the long run, it is believed that “legitimacy is
conferred on a new style public sector which claims to have broken with the
pathological aspects of the past, but which retains a sense of distinctive identity and
purpose” (Ferlie, Ashburner, Fitzgerald & Pettigrew, 1996: 10–15).
New Public Management theory has been around since the beginning of the 1980s
to describe a new way to study and manage public sector organisations. As an
activity and as a discipline, it is presented as being different from traditional public
administration, public policy and private or business management (Adei & Badu,
2006: 102). Adei and Badu also offered an exhaustive list of techniques that
constitute NPM in practice. The popular known ones were privatisation, contracting
out, decentralisation, merit pay, partnerships and management by results and
customer orientation. From their argument, important points put forward are that the
11
implementation and success of NPM is possible only if there is a cultural and
behavioural shift in the management of government away from bureaucratic
government towards an entrepreneurial type of government (Adei & Badu, 2006: 102
– 103).
In recent times, the changed role of the state gave birth to NPM which advocated for
de–bureaucratisation, flexibility, innovation, reliance on the private sector, creation of
an enabling environment for private enterprise growth and use of means other than
public bureaucracy for service delivery other than public bureaucracy. Within this
debate, Sharma argued that the significance of new public management is that its
emphasis on results, distinguished it from the traditional public administration
process orientation (Sharma, 2006: 131).
For the purposes of this study, NPM was meant to highlight a new wave of thinking
or results driven approach that the public sector can apply upon the
institutionalisation of its policies, programmes and processes. This study was
therefore premised on the operational notions that allude that NPM is relevant for the
public sector as it proposes for governments to be driven by their missions of
improving their effectiveness and efficiency. This formed part of the section that will
be discussed later which highlights the relationship between the theory of NPM and
the phenomenon of M&E.
2.2.2 NEW PUBLIC MANAGEMENT (NPM) CRITIQUES
The review was also aware and recognised critiques of NPM that began by arguing
that as a term NPM has probably outlived its analytic usefulness. Within these
discussions, the term NPM was deemed to be ambiguous because the agenda of
public sector reform was considered to have moved in some respects beyond the
traits identified by scholars of public management in the 1990s reflecting the various
cultural cross-currents that have swept through managerial debate (Hood, 2001:
125). The term was also considered to be too basic for the distinctions between
different sorts and themes of managerialism that academic scholars need to make
as the study of public services developed and the public sector reform movement
became professionalised. This then led to an argument that there was a need for
more words to describe the cultural and technical variety of contemporary
12
managerialism. Therefore it was not surprising that there were numerous attempts to
proclaim a move beyond NPM (Hood, 2001: 12555).
Furthermore, critics of NPM doctrines saw them as too heavily based on business-
school and private-sector management perspectives whose key parts of the reform
message have been reversed because they lead to policy disasters. According to
Hood (1994), even the strongest advocates of NPM expected its doctrines to have
little impact on altering the overall effectiveness of government. This was based on
the argument that NPM practices were extensively institutionalised and will continue
just as NPM itself did not displace large elements of previous public management
orthodoxies, sometimes characterised as ‘progressive public administration’ or PPA
(Dunleavy, Margetts, Bastow & Tinkler, 2005: 468).
It was further put forward that the torch of leading edge change has passed on from
NPM and will not return, simply because NPM was now an almost two decades old
set of public management ideas. Even analysts sympathetic to NPM were driven to
acknowledge that it was middle-aged and generated adverse by-product outcomes,
while still resisting evidence of its provisions (Hood, 2001: 12555).
Nevertheless this review acknowledges that in spite of its above-mentioned
proclaimed death, NPM theory refused to lie down and continued to be widely used
by practitioners and academics alike. As it was the case in this study, NPM theory
was deemed important because it advocated for business and private-sector
management perspectives that are required and could be useful in governments. To
be exact, its operational notions argued that NPM is relevant for the public sector as
it proposes for governments to be driven by their missions of improving their
effectiveness and efficiency. In this case the DLGTA adopting a new wave of
thinking or a result driven approach that can be applied upon the implementation of
its policies, programmes and processes in line with Governments being driven by
their mission of improving their effectiveness and efficiency.
13
2.2.3 MONITORING AND EVALUATION PHENOMENON
The phenomenon of monitoring involves reporting on actual performance against
what was planned or expected according to pre-determined standards. Monitoring
generally involves collecting and analysing data on implementation processes,
strategies, results and recommending corrective measures. In the same context, the
Presidency (South Africa) refers to evaluation as a time bound exercise that
systematically and objectively assesses the relevance, performance, challenges and
successes of programmes and projects. Evaluation can also address outcomes or
other development issues. Additionally evaluation usually seeks to answer specific
questions to guide decision-makers or programme managers. Evaluation should
advise whether underlying theories and assumptions were valid, what worked, what
did not work, and why. Evaluation commonly aims to determine relevance, efficiency,
effectiveness, impact and sustainability of a programme or project (Presidency,
2007: 5).
Combined, monitoring and evaluation are distinct but connected analytical exercises.
Monitoring instruments provide standing checks to assess progress against financial
and non-financial performance targets. The information that flows through these
instruments reflects the extent and type of actual spending against budgeted
projections, goods and services purchased under the spending plans, and whether
the intended objectives of expenditure were achieved. These instruments are
designed to operate at set intervals, whether monthly, quarterly or annually. The
Collaborative Africa Budget Reform Initiative (CABRI) initiative (2006) also put
forward that evaluation goes a step further as it considered not only whether
spending plans are being implemented, but whether they are being implemented
correctly and are suited to overall policy objectives. Time bound evaluation exercises
measure the relevance efficiency, effectiveness, impact and sustainability of
government programmes. Evaluations may be carried out internally by ministries,
departments and agencies, or externally by central ministries, the Auditor General,
the Cabinet and Legislature. There is also space for independent evaluations to be
conducted by commissions, civil society organisations, research institutions and
development partners. Together, these elements form part of a system to improve
governance within the public sector (CABRI, 2006: 156 -157).
14
A monitoring and evaluation system, hence, is important in a country as it usually
leads to a clear understanding of the current M&E efforts, the overall public sector
environment, its institutional arrangements and opportunities for strengthening and
improving the current M&E initiatives. This also includes using M&E information to
benefit the intended stakeholders. The importance of this information is that it can be
used for core government functions, such as budget decision-making and the
ongoing management of programmes and projects. More importantly and linking the
above provisions to this study, this analysis will help key role players in government
and the donor community on the strength and weaknesses of M&E, as well as the
institutional arrangements (MacKay, 2007: 3). In all, the successful
institutionalisation of M&E involves the creation of a sustainable, well functioning
M&E system within a government where good quality M&E information is used
intensively (MacKay, 2007: 23).
2.2.4 THE RELATION BETWEEN NPM AND M&E
The theory of NPM in this study was meant to highlight how the DLGTA
management has not used its provisions as part of a results based turnaround
strategy towards improving its efficiency and effectiveness in line with Governments
being driven by their missions of improving their effectiveness and efficiency. The
Presidency’s Government Wide Monitoring and Evaluation Policy Framework noted
that Government’s major challenge was to become more effective. Monitoring and
Evaluation processes in South Africa were introduced as a way to assist the public
sector in evaluating its performance and identifying the factors which contribute to its
service delivery outcomes. M&E is uniquely oriented towards providing its users with
the ability to draw causal connections between the choice of policy priorities, the
resourcing of those policy objectives, the programmes designed to implement them,
the services actually delivered and their ultimate impact on communities
(Presidency, 2007: 1).
Taking note of the NPM theory which directs towards a result based turnaround
strategy, based on improving efficiency and effectiveness towards Gauteng
municipalities, the DLGTA management did not create an enabling environment
based on effective internal processes that will ultimately assist Gauteng
15
municipalities with their M&E functions, as required by legislation. A part of the
turnaround of internal processes was meant to commence with the DLGTA firstly
focusing on the implementation of a municipal M&E system. This system in turn
would lay a foundation for how the municipal service delivery M&E unit would roll out
their assistance to municipalities in the Gauteng province with their M&E functions.
The absence of a municipal M&E system within the DLGTA was documented in the
“Planning, Monitoring, Evaluation and Reporting Policy Framework” as put together
by the municipal service delivery M&E unit. To begin with, the M&E framework noted
that “an organisation that is outcome, or service delivery focused requires a
Planning, Monitoring, Evaluation and Reporting system that is aligned to its needs”.
The aim behind the Planning, Monitoring, Evaluation and Reporting Framework was
also to promote, support and devise a system that would link and appreciate the
relationship and interdependence that exists between planning, monitoring,
evaluation and reporting (PMER) in the DLGTA and municipalities throughout the
Gauteng province. The framework was premised on the understanding and
appreciation that the increasing recognition of the importance of PMER has however
not always translated into capacities to develop systems and a culture of
performance and accountability in organisations. It was also noted in the framework
that, across the world, organisations lacked sufficient dedicated financial and human
resources to be able to make a significant impact on improving the practice of
PMER. These resource constraints continued to hamper the quick roll out of the
various initiatives aimed at improving effectiveness of programmes and increasing
efficiencies in organisations’ planning, monitoring, evaluation and reporting process
(DLGH, 2011: pp 9-10)
The absence of the municipal M&E system has therefore resulted in Gauteng
municipalities struggling with some of their M&E functions, especially their reporting
duties. Related concerns regarding municipal inadequate reporting were also noticed
by the Auditor General. In his assessment of the City of Tshwane Metropolitan
Municipality’s annual report for the 2007/2008 financial year, the AG highlighted a
number of concerns and problems based on municipal reporting. These problems
were mostly related to how the municipality did not properly report its performance
information (City of Tshwane, 2008, pp 82-83). The AG was mostly concerned with
16
the inconsistencies regarding reporting principles and the reported performance
information with the planned objectives.
Additionally, a concern was also raised in one of the quarterly reports compiled by
the service delivery M&E unit of the DLGTA. The concern was as follows:
“Some of the reports submitted by municipalities for the period under
review are not based on their Service Delivery Budget Implementation
Plans and that made it very difficult to assess performance as the
performance reported cannot be linked to any measurable
performance objectives. The data submitted usually does not respond
to the common set of indicators to make the report conclusive about
municipal achievements or challenges thereof” (GDLGH, 2009, p.7).
The above provisions were meant to highlight how, throughout the years,
governments (in this case the DLGTA) challenge has been to become more
effective. Instead of abiding by the principles of NPM that advocate for governments
being driven by their mission of improving their effectiveness and efficiency, the
DLGTA instead did not create an enabling environment based on effective internal
processes that commenced with an internal municipal M&E system that would
ultimately assist Gauteng municipalities with their M&E functions as required by
legislation.
2.3 CONCEPTUAL FRAMEWORK
2.3.1 MONITORING AND EVALUATION SYSTEMS
An M&E system can be defined as a system that spells out how the current M&E efforts
of an organisation will be undertaken including a definition of the roles and
responsibilities of the members who will be involved in the running of the system.
International experience has shown that there are a number of generic system
requirements for a functional M&E system. The significance of these system
requirements is that if they are in place, the implementation aspect surrounding M&E
strategies and policies might be less complex. According to the Global Fund (2009),
some of the important dimensions and associated performance goals in an M&E system
can be summarised as follows:
17
Component Performance Goal For This Component
Organisational with M&E functions Establish and maintain a network of organisations
responsible for M&E at the national, sub national and
service delivery levels
Human capacity for M&E Ensure adequate skilled human resources at all levels of the
M&E system to ensure completion of all tasks defined in
the annual M&E work plan. This includes sufficient
analytical capacity to use the data and produce relevant
reports
Evaluation and research Identify evaluation and research questions, coordinate
studies to meet the identified needs and enhance the use
of evaluation and research findings
Data dissemination and use Disseminate and use data from the M&E system to guide
the formulation of policy and the planning and
improvement of programs
National, multi-sectoral M&E plan Develop and regularly update the national M&E plan,
including identified data needs, national standardized
indicators, data collection procedures and tools and roles
and responsibilities for implementation
Table 1: Components of a functional M&E system (The Global Fund, 2009: 15)
According to the 1998 Presidential Review Commission (PRC), the significance of
having M&E systems in place is that they enable a systematic and objective
assessment of ongoing and completed projects, programmes or policies, their
design, implementation and overall results. The aim was to determine the relevance
and fulfilment of programme or policy objectives, their development efficiency,
effectiveness and sustainability. Systematic evaluation programmes or mechanisms
could provide the government of the day with information that is credible and useful ,
that will also enable the incorporation of the lessons learned into the decision making
process going forward (Scott and Joubert, 2005: 2).
Additionally Gorgens and Kusek (2009) argue that sustaining an M&E system that
can produce trustworthy, timely and relevant information on the performance of
government, civil society or private sector projects, programmes and policies
requires the overcoming of many M&E system challenges. The implementation of
such a system would therefore require an approach based on experience, skill and
real institutional capacity. Gorgens and Kusek further put forward 12 components of
a functional M&E system and they argue that for any M&E system to be fully
18
functional, it is important that these components are in place (Gorgens and Kusek,
2009: 7-8). The components are as follows:
1. Structure and organisational alignment for
M&E systems
2. Routine Monitoring
3. Human Capacity for M&E systems 4. Periodic surveys
5. Functional M&E partnerships 6. Databases useful to M&E systems
7. M&E work plans 8. Supportive supervision and data auditing
9. Costed M&E work plans 10. Evaluation and research
11. Advocacy, communication and culture for
M&E systems
12. Using information to improve results
Table 2: Components of a functional M&E system (Gorgens & Kusek, 2009)
2.3.2 EVALUATION CAPACITY BUILDING (ECB)
Within the discussion on M&E, M&E systems and human capacity for M&E, the
phenomenon of ECB needs consideration. It could be defined as referring to
capacity building plans that provide improvement or training on a range of M&E
tools, methods, approaches, and concepts. MacKay (2002) suggests a more
complete definition of evaluation capacity, by referring to ECB as:
“An organisation’s ability to bring about, align and sustain its objectives,
structure, processes, culture, human capital and technology to produce
evaluative knowledge that informs on-going practices and decision making
in order to improve organisational effectiveness”. (MacKay, 2002: 83)
Along the same sentiments, a broader definition by MacKay (2002) suggests that
ECB should ensure that knowledge from M&E is applied as part of sound public
governance and it should encompass ”a broad range of evaluative tools and
approaches that include, but go beyond programme evaluation, capacity building
being one step along a “results chain” (MacKay, 2002: 83).
It also came about that one of the key challenges for everyone involved in capacity
building for M&E was agreement on what is meant by the term. For definitional
purposes and at its most basic, capacity can be understood as the ability of people,
organisations and society as a whole to manage their affairs successfully. On the
other hand, they suggest that organisational capacity can be defined as the
19
capability of an organisation to achieve effectively what it sets out to do.
Furthermore, the capacity of an individual, an organisation or a society is not static
as it can change over time, and is subject to both internal and external influences.
Many of these changes are unplanned. For example, an organisation can lose
capacity if key individuals leave or change positions within that organisation.
Capacity development on the other hand can be seen as a more deliberate process
whereby people, organisations or society as a whole create, strengthen and maintain
capacity over time (Simister & Smith, 2010: 3).
In relation to the question “why build evaluation capacity?” Boyle, Lemaire and Rist
(1999) noted that governments build evaluation systems because they believe that
such a system will help them improve their means and methods of governance.
Additionally, a system which is understood, credible and used enhances the
performance of public sector management. Furthermore it is noted that in the
absence of such a system, there is little objective evidence that one can turn to in
order to ascertain the consequences of governmental actions and programmes
(Boyle, Lemaire & Rist, 1999: 6).
Evaluation capacitation can also refer to the provided skills on “human capital (skills,
knowledge and experience) and financial/material resources”. This may also include
“activities and initiatives taken to implement an evaluation programme” (Boyle and
Lemaire, 1999: 5- 6). Stokdill, Baizerman and Compton define ECB as “a context-
dependant, intentional action system of guided processes and practices for bringing
about and sustaining a state of affairs in which high quality programme evaluation
and its appropriate uses are ordinary and on-going practices are within and/or
between one or more ordinary organisations/programmes/sites”. (Stokdill, Baizerman
& Compton, 2002: 8).
According to Rist, Boily and Martin (2011), evaluation capacity can be assessed in
terms of four dimensions:
1. Institutional Capital
2. Human Capital
3. Technical Capital
4. Financial Capital
20
This review however shall focus on the “Human Capital or Capacity” dimension of
evaluation capacity for its relevance to this study. This dimension is also important
because the municipal service delivery unit of the DLGTA has been challenged on it
and also because it is a part of the capacity requirements for the implementation of a
municipal M&E system that will lay a foundation for how the unit will assist Gauteng
municipalities with their M&E functions. In relation to the human capital dimension,
Rist, Boily and Martin argue that human capital is of primary importance for the
production of M&E results. To begin with, there needs to be good capacity in terms
of quantity and quality of M&E human resources, both within the organisation (M&E
staff) and outside of the organisation (external evaluators). Additionally, proper
training and experience is needed to ensure good M&E as well as the set up of a
good multidisciplinary team. Furthermore human resource management is also
important to ensure the stability of M&E personnel through greater retention,
including the implementation of measures such as tailor made career paths and
individual incentives (Rist, Boily & Martin, 2011: 8).
2.3.3 HUMAN CAPACITY AS PART OF MONITORING AND EVALUATION
An understanding of the skills needed and the capacity of people involved in the
M&E system and addressing capacity gaps should be at the heart of the M&E
system. Gorgens and Kusek (2009) denote that focusing on human capacity for M&E
will improve the quality of the M&E system. Ideally, the M&E system would be
designed in advance, skill requirements for it established and human capacity
development should be planned and undertaken before the M&E system is
implemented (as indicated in the above tables) (Gorgens & Kusek, 2009: 94).
2.3.4 HUMAN CAPACITY ELEMENT IN M&E SYSTEMS
Important to note is that an M&E system may not function properly without skilled
individuals who effectively perform their M&E tasks for which they are responsible
and for any M&E system to perform well, it is necessary to have well-trained officials
who are highly skilled in M&E. For this reason, most capacity-building plans should
place considerable emphasis on provision of training in a range of M&E tools,
methods, approaches, and concepts. Those governments that contract out their
21
evaluations need to ensure that their officials possess the skills and experience
necessary to oversee and manage evaluations. This requires a broader set of
competencies than the ability to simply conduct an evaluation. They also need to
understand the strengths and limitations; the relative cost-effectiveness of various
types of M&E. Introductory training in M&E can also raise awareness of and demand
for M&E information. Training should also extend to the use of M&E information
(Mackay, 2007: 58).
With all the above in mind, it is important that all members of an M&E unit possess
M&E competencies in order to effectively execute their part in the operation of the
M&E system. It has been proven that there is a need for holistic M&E capacity
development programmes that focus on the technical skills that would enable other
components to be functional, including managerial aspects of M&E systems
(Gorgens & Kusek, 2009: 94).
As indicated in the above tables in yellow, the Global Fund, as well as Gorgens and
Kusek, consider the human capacity element as one the important requirements of
an M&E system. An effective and functional M&E system is therefore highly
dependent on the human resources that would keep the system running. Capacity is
required for the design, planning, development of tools, implementation,
maintenance, coordination, harmonisation and utilisation of the products to enhance
results based management and rational evidence-based decision making (Curricula,
2010: 1).
Within an M&E system, human capacity is important because it is one of the core
requirements in any functional M&E system. According to the Global Fund (2009),
this component is premised on ensuring that adequate human resources skills at all
levels of the M&E system are present to ensure completion of all tasks defined in the
annual M&E work plan. This includes sufficient analytical capacity to use the data
and produce relevant reports (Global Fund, 2009: 15).
The human capacity element of an M&E system can be considered part of the
people, partnerships and planning ring of the 12 components of an M&E system. It is
also essential that M&E responsibilities of an organisation and its staff are clearly
defined and authorised, so it is essential for staff to have the necessary skills to
execute their tasks well. According to Gorgens and Kusek (2009), the capacity
22
element relates to the ability to perform appropriate tasks effectively, efficiently and
sustainably on the following three levels: system capacity, organisational capacity
and individual capacity. System capacity relates to the ability of a system to deliver
the goal and objectives of a process and thereby contribute toward fulfilling the
organisation’s objectives (Gorgens & Kusek, 2009: 91).
Hopkins (1994) adds by denoting that in a system context, capacity is defined as a
set of entities that operate to achieve a common purpose and according to certain
rules and processes (Hopkins, 1994). Organisational capacity on the other hand
relates to the capacity of an organisation and its processes to deliver the
organisation’s goals and development objectives. Finally and most significant to the
study, individual capacity is about the ability of individuals to perform functions
effectively, efficiently and sustainably (Gorgens & Kusek, 2009: 91-92).
Amongst the three mentioned levels of capacity, the significant one which the study
is premised on is the third one which is the individual or human level. This level
consists of individuals functioning within the various organisations. Gorgens and
Kusek (2009) argue that a major dimension of capacity is at the individual level,
meaning people. This covers individuals within organisations involved in executing
M&E functions, and those who are beneficiaries or are otherwise impacted by the
M&E system or the things it measures (Gorgens & Kusek, 2009: 92).
In relation to the study, human and skills capacity challenges facing the DLGTA
service delivery M&E unit were highlighted in a document aimed at repositioning the
municipal service delivery M&E directorate. Repositioning the unit was meant to
make the unit more consistent with the departmental reconfigurations, legislative
mandate, current demands and practices of M&E in relation to the departmental
vision of a leading intellectual edge of local government and championing of
municipal performance assessment. The document further highlighted that while the
department has been able to execute its legislative mandate to a certain extent, the
municipal service delivery M&E arm of the department has not been effective
enough to expose some of the weaknesses in local government performance as well
as to propose remedial action. The document put forward that municipal M&E work
has become routine, more about compliance because it has not been prioritised. The
Repositioning Document identified human resources, skills, competencies and
expertise required as some of the key challenges facing the municipal service
23
delivery M&E unit. The argument is that there has been very little effort to build the
unit’s skill base, and where there are skills development programmes, they are not
aligned to the needs of the unit (skills mismatch). The document further puts forward
that the skills challenge is also exacerbated by the lengthy turnaround times in filling
vacancies of the directorate (Gauteng COGTA, 2013).
For the DLGTA municipal service delivery M&E unit, important requirements within
the human capital or capacity dimension of evaluation capacity need to firstly entail
key staff with M&E responsibilities needing to be ensured that they possess the
knowledge, skills, tools and support to carry out their respective M&E tasks.
Secondly, a well functioning M&E system requires human resources, training,
materials and financial resources. The Director of the municipal service delivery M&E
unit will be required to lead the orientation on the M&E system by ensuring that the
M&E unit is well capacitated and also ensuring that staff members of the unit are
familiar and have the necessary skills on how to implement a municipal M&E system
so they can effectively use the system to assist municipalities with their M&E
functions as required (www.ausaid.gov.au). On an individual level, the Director will
also need to ensure that already existing and new staff members of the M&E unit are
capacitated with the following individual competencies or attributes:
Competency / Requirements
Professional Practice Application of professional evaluation standards
Acting ethically and striving for integrity and honesty when conducting evaluations
Understanding the knowledge base of evaluation (terms, concepts, theories, assumptions)
Knowledgeable about quantitative and qualitative research methods
Ability to develop evaluation designs
Ability to frame evaluation questions
Ability to collect data
Ability to analyse and interpret data
Ability to assess validity and reliability of data
Ability to compile evaluation reports
Ability to develop recommendations from evaluation reports
Ability to supervise others involved in conducting
Ability to train others involved in conducting evaluations including
24
Table 3: Attributes or competencies of a capacitated M&E official (Source: Stevahn, King, Ghere & Minnema, 2005: 49-51)
As part of sustaining the M&E system and after ensuring that the above mentioned
competencies/requirements are in place, the Director of the municipal service
delivery M&E unit will be required to ensure that the above mentioned individuals
constantly participate in M&E training programmes that will ensure that their already
acquired skills are improved and maintained on an up to date basis.
2.4 CONCLUSION: OBSERVATIONS ON THE REVIEWED LITERATURE
A number of common viewpoints were identified from the reviewed literature and the
most important ones are that to begin with and on a theoretical level, there is a
relation between the NPM theory and the phenomenon of M&E. This review
encapsulated how NPM provides for governments to be driven by their mission of
improving their effectiveness and efficiency. The DLGTA has not been in line with
NPM provisions by improving and turning itself around in the form of strengthening
its human capacity element for the implementation of a municipal M&E system that
will enable municipalities to be assisted with their M&E reporting functions.
In relation to the phenomenon of M&E and the human capacity element, the review
realised the importance to indicate the agreement in terms of how the human
capacity element is considered to be important in an M&E system as it ensures the
completion of all tasks defined in the annual M&E work plan. The review also
showcased how the human capacity element needs to be accompanied by
evaluation capacity building based on capacity building plans that provide training on
a range of M&E skills, tools, methods, approaches, and concepts, and these will
enable the production of good M&E results. Furthermore, there needs to be an
understanding of the skills needed and the capacity of people involved in an M&E
system and addressing capacity gaps should be at the heart of the M&E system. It is
also important that all members of an M&E system need to have certain
evaluations evaluation practices
Ability to serve the information to intended users of evaluation reports
Ability to manage evaluation projects
25
competencies in order to effectively execute their part in the operation of the M&E
system.
Most importantly, the review focused on the “Human Capital or Capacity” dimension
of evaluation capacity. The review argued for the importance of this dimension
because the municipal service delivery unit of the DLGTA has been challenged on it
and also because it is the most important requirement for the institutionalisation of a
municipal M&E system that will broadly lay a foundation for how the unit can assist
Gauteng municipalities with their M&E functions.
The study therefore aims to add to the existing literature by using the reviewed
themes of M&E, monitoring and evaluation systems, human capacity elements within
M&E systems and evaluation capacity building. These themes were used when an
M&E system centred on the human capacity element was designed that the
municipal service delivery unit of the DLGTA can implement in their efforts to assist
municipalities in the Gauteng province with their M&E functions.
26
CHAPTER THREE
RESEARCH METHODOLOGY
3.1 INTRODUCTION
This chapter discusses the research methodology that was followed in the study.
The first part discusses the generic conceptualisation related to the qualitative social
science research method and the rationale on why this method was selected. The
second part then outlines a research design that the researcher followed for data
collection, analysis and validation. This is an indication of how the data collection
was undertaken, identification of primary and secondary sources of data, including
analysis. Furthermore the researcher discusses how the data was analysed and how
the reliability and validity were maintained.
3.2 RESEARCH APPROACH
3.2.1 QUALITATIVE RESEARCH METHOD
Due to the nature of the study which is based on social phenomena, a qualitative
approach and method of conducting research was followed. Qualitative research is a
type of scientific research which consists of an investigation that seeks answers to a
question and systematically uses a predefined set of procedures to answer a
particular question. Qualitative research also collects evidence, produces findings
that were not determined in advance and findings that are also applicable beyond
the immediate boundaries of the study. Additionally, it seeks to understand a given
research problem or topic from the perspectives of the individuals it involves.
Qualitative research is especially effective in obtaining culturally specific information
about the values, opinions, behaviours, and social contexts of particular individuals
or populations (Mack, Woodsong, MacQueen, Guest & Namey, 2005: 1).
Leedy and Ormrod (2005) argue that qualitative research seeks to answer questions
on complex social reality from a participant’s viewpoint rather than that of the
researcher (Leedy & Ormrod, 2005). Arguments from qualitative research literature
indicate that qualitative researchers are concerned with meaning in context or in the
real world, thus highlighting the fact that research is context bound and
encompasses views as expressed by individual human beings. In the qualitative
27
research approach, the researcher constructs the reality and applies his/her
understanding of the social phenomenon using the respondent’s appreciation of
social reality as the lens. Thus the qualitative approach acknowledges the fact that
multiple interpretations can be derived from any given social situation (Kalof, Dan &
Dietz, 2008: 85).
Creswell (1994) advises that qualitative researchers should therefore be honest
when interpreting social reality and rely on voices and interpretations from the
informants. In the qualitative approach, the researcher is very active and engages
with the informants to generate a detailed understanding of the social reality under
study. The distance between the researcher and the researched is reduced as the
researcher can sometimes live with the informants while studying them (this is called
ethnography). Therefore, this makes qualitative research more personal and informal
(Creswell, 1994: 6).
As a category of research designs or models which focus on understanding and
explaining the meaning of social phenomena (in this case the challenges around the
institutionalisation of an M&E system), a qualitative method of conducting research
was then deemed appropriate for the study. Qualitative researchers use verbal,
visual or tactile data in the form of descriptive narratives such as field notes, reports,
transcriptions, audio or video recordings, artefacts or other written recordings.
Relating to the study, the characteristics and strengths of qualitative research that
helped in achieving the goal of the study are that the strength of qualitative research
is its ability to provide complex textual descriptions of how people experience a given
research issue. Qualitative research also provides information about the human side
of an issue, and that is the often contradictory behaviours, beliefs, opinions,
emotions, and relationships of individuals. Qualitative methods of research are
therefore effective in identifying intangible factors, such as social norms,
socioeconomic status, gender roles, ethnicity, and religion, whose role in the
research issue may not be readily apparent. When used along with quantitative
methods, qualitative research can help to interpret and better understand the
complex reality of a given situation (Mack, Woodsong, Macqueen, Guest & Namey,
2005: 1- 2).
28
3.3 RESEARCH DESIGN
3.3.1 PRIMARY SOURCES
For conceptual clarification, primary sources refer to sources that appear first in time.
In addition, primary sources are generally being regarded as being closer to
whatever is, or was true. They take such diverse forms as interviews and their
scripts, questionnaires, letters, diaries, sermons, laws, census reports, immigration
records and probate documents to mention but a few (Leedy & Ormrod, 2005: 165).
As part of the qualitative research approach, the researcher constructs the reality
and applies his/her understanding of the social phenomenon using the respondent’s
appreciation of social reality as the lens. At the same time, the researcher is very
active and engages with his/her informants to generate a detailed understanding of
the social reality under study. It was under this onus that the researcher decided to
use an open-ended questionnaire as a primary source of data collection. A
questionnaire can be referred to as a set of questions on a particular issue sent to
individuals for the purposes of gathering information on that issue. Questionnaires
usually form an integral part of descriptive and opinion related surveys.
Administered by electronic mail, the researcher used a questionnaire in which
respondents were requested to complete a number of questions on key issues
pertaining to the study. The questionnaire was sent to respondents to complete in
their own time and this process was also managed by electronic mail. As time
constraints also had some role to play, a questionnaire was deemed appropriate as it
is a practical data collection method which is cost effective to collect data from a
number of people in a short period of time. The researcher also preferred to use a
questionnaire because its results can be analysed more objectively than other forms
of research. It should also be noted that the researcher did not have sufficient time to
be able to test the questionnaire before sending it out to respondents. However the
researcher did rely on the supervisor’s opinion and approval of the questionnaire,
comments and inputs before sending it out to respondents.
To encourage responses the researcher firstly called each of the respondents to
explain the purpose of the call, including the research project. After a number of
respondents agreed to assist in the project, the researcher sent them the
questionnaire and attached the following cover letter to the questionnaire which
29
stated the purpose of the project and also humbly requested their participation. The
letter was as follows:
Dear Respondent:
As a present or past member of the Municipal Service Delivery Monitoring and Evaluation
(M&E) Unit of Gauteng COGTA, formerly known as the Gauteng Department of Local
Government and Traditional Affairs (DLGTA), you are kindly requested to assist the
researcher (Mr. M.E Maphunye) regarding information for completion of his dissertation on
the unit’s challenges in executing some of its mandated M&E duties towards Gauteng
municipalities. Mr. Maphunye is currently a student with the University of the
Witwatersrand Business School where he is pursuing a Masters Degree in Management and
Public Policy (MMPP). This exercise is part of the requirements for completion of a research
dissertation in line with the MMPP Degree.
As a brief synopsis of the dissertation, the researcher explores the challenges faced by the
municipal service delivery M&E unit in implementing a municipal M&E system. As an
attempt to probe the challenges, the dissertation aims to explain how the lack to focus on
the “human capacity element for functional M&E systems”, by the municipal service
delivery M&E unit of the DLGTA has had an impact in the efforts to implement a municipal
M&E system. The absence of the municipal M&E system has somewhat resulted in
municipalities of the Gauteng province not being assisted with certain elements of their
M&E functions by the DLGTA municipal service delivery unit as mandated by legislation.
The researcher aims to use your responses in an attempt to understand whether the
challenges faced by service delivery M&E unit in the implementation of a municipal M&E
system are rooted to the institutional (department) or human capacity and skills
dimensions. Based on your responses, the researcher also aims to propose and design an
M&E system that the municipal service delivery M&E unit can implement and base their
efforts for assisting municipalities in the Gauteng province with their M&E functions.
With the above in mind, the researcher therefore would like to humbly thank you for taking
your time and becoming a part of this endeavour.
Kind Regards
M. E Maphunye (Researcher)
30
3.3.2 SAMPLE SIZE
As a primary source of data collection, the researcher therefore sent out the
questionnaire to current and former officials of the service delivery M&E unit in the
DLGTA for their hands-on insight regarding the problem. Designations of the
respondents ranged from Chief Director, Director, Deputy Directors and Assistant
Directors. In ensuring that all questions were interpreted in the same way, the
researcher firstly realised the importance of asking questions that are clear and
specific and that each respondent would be able to answer. As the questionnaire
questions are open ended, the researcher ensured that respondents would be able
to answer the questions in their own words. The researcher also realised the
importance of asking one question at a time as opposed to double barrelled
questions. In general, questions that use simple and concrete language are more
easily understood by respondents. The researcher also avoided the use of words
that may be viewed as biased or potentially offensive to some respondents.
Questions in the questionnaire were developed in relation to the literature review and
their rooting in the primary research question which is trying to understand some of
the challenges encountered by the municipal service delivery M&E unit of the
DLGTA in implementing its M&E system that will guide the carrying out and
performing of municipal M&E tasks. Related to the reviewed literature on M&E
systems and the human capacity required for such systems (competencies or
attributes of M&E practitioners), the following is an example of the questionnaire
containing questions that the researcher sent out to past and present members of
the municipal service delivery M&E unit of the DLGTA.
31
3.3.3 QUESTIONNARE EXAMPLE
Dimension: Institutional (Organisational)
Questions
1. How did/does the DLGTA as an organisation view its role as the coordinating body of M&E in the province?
2. What efforts did the DLGTA embark upon to prioritise the capacitation of the service delivery M&E unit?
3. What were/are the efforts in place to try and institutionalise a municipal M&E system? How successful are these efforts?
4. How important was/is the work produced by the service delivery unit towards the attainment of the overall goals of the DLGTA?
5. What were/are the roles and responsibilities of the members of service delivery M&E unit, and are/were they clearly defined?
Dimension: Human Capacity and Skills
Questions
1. How capacitated were/are unit members to properly carry out their M&E functions? 2. What abilities did/do members of the unit possess related to the implementation of a municipal
M&E system? 3. What abilities did/do members of the unit possess related to the management of
evaluation projects? 4. What M&E training or capacity building courses did/do members of the M&E unit attend on
a regular basis? 5. How skilled were/are the members of the unit in carrying out their functions effectively,
efficiently and sustainably? 6. How knowledgeable were/are members of the unit about quantitative and qualitative
research methods? 7. What abilities did/do members of the unit possess related to supervising others (service
providers) involved in conducting evaluations? 8. How skilled are/were the unit members in presenting M&E reports?
Table 4: Questionnaire Example
Related to the study’s conceptual framework, the questionnaire attempted to probe
whether the challenges faced by service delivery M&E unit of the DLGTA unit in the
institutionalisation of a municipal M&E system could be rooted either to the
institutional (organisation) or human capacity and skills dimension. In addition, the
questionnaire was also meant to capture responses with regard to establishing whether
the DLGTA has been aware of the earlier mentioned provisions of the importance
M&E systems, human resource capacity requirements within M&E systems and the
importance of capacity building in M&E systems.
32
3.3.4 THE DIMENSION AND PURPOSE FOR ASKING QUESTIONS
The above section can be summarised into the following table which highlights the types
of questions contained in the questionnaire and the purpose or why the researcher
decided to pose these questions.
Question Purpose
Dimension: Institutional (Organisational)
1. How did/does the DLGTA as an
organisation view its role as
the coordinating body of M&E
in the province?
To establish the whether the DLGTA as an organisation
is aware of the importance of its mandated role in the
coordination of municipal M&E in the province
2. What efforts did the DLGTA
embark upon to prioritise the
capacitation of the service
delivery M&E unit?
To establish the prioritisation of the capacitation the
service delivery M&E unit by the DLGTA
3. What were/are the efforts in
place to try and institutionalise
a municipal M&E system? How
successful are these efforts?
To establish whether were there any existing measures
or efforts in the institutionalisation of an M&E system
(from an departmental point of view)
4. How important was/is the
work produced by the service
delivery M&E unit towards the
attainment of the overall goals
of the DLGTA?
To establish how the DLGTA as an organisation views
the work undertaken by the service delivery M&E unit
towards the attainment of its goals as an organisation
5. What were/are the roles and
responsibilities of the members
of service delivery M&E unit,
and are/were they clearly
defined?
To establish whether members of the service delivery
M&E unit are aware of their roles and responsibilities
Dimension: Human Capacity and Skills
Question Purpose
6. How capacitated were/are unit
members to properly carry out
their M&E functions?
To establish the human capacity levels of the service delivery M&E unit, whether unit members are well capacitated with the right numbers, knowledge, tools and support to carry out their respective M&E tasks
7. What abilities did/do members
of the unit possess related to the
implementation of a municipal
M&E system?
To establish abilities related to the implementation of
M&E systems that unit members are in possession of
33
8. What abilities did/do members of the unit possess related to the management of evaluation projects?
To establish the M&E skills levels of members of the
service delivery M&E unit in carrying out their M&E
tasks
9. What M&E training or capacity
building courses did/do
members of the M&E unit
attend on a regular basis?
To establish the rate at which the DLGTA prioritises M&E
capacity building to improve the M&E skills of unit
members
10. How skilled were/are the
members of the unit in carrying
out their functions effectively,
efficiently and sustainably?
To establish the M&E skills levels of members of the
service delivery M&E unit in carrying out their M&E
tasks
11. How knowledgeable were/are members of the unit about quantitative and qualitative research methods?
To establish the M&E skills levels of members of the
service delivery M&E unit in carrying out their M&E
tasks
12. What abilities did/do members
of the unit possess related to
supervising others (service
providers) involved in
conducting evaluations?
To establish the M&E skills levels of members of the
service delivery M&E unit in carrying out their M&E
tasks
13. How skilled are/were the unit members in presenting M&E reports?
To establish the M&E skills levels of members of the service delivery M&E unit in carrying out their M&E tasks
Table 5: Dimension and Purpose for asking Questions
3.4 SECONDARY SOURCES
As part of secondary sources, the researcher reflected on the reviewed literature
firstly to benchmark the DLGTA as an organisation against the provisions of the
theory of NPM that advocates for a results-based turnaround strategy towards
improving its efficiency and effectiveness. The essence was to showcase from a
theoretical level whether the DLGTA as an organisation has been using NPM
provisions as part of a results based turnaround strategy towards improving its
efficiency and effectiveness or not. Secondly, secondary sources in the form of the
reviewed literature were used to showcase how the DLGTA fared in attempts to
institutionalise the following concepts related to the phenomenon of M&E:
Monitoring and evaluation system,
Prioritising human capacity element as part of the requirements of an M&E
system,
34
Evaluating capacity building for the capacitation of members of the M&E unit
with the required skills as they will be the ones carrying out M&E functions.
3.5 CONTENT ANALYSIS
3.5.1 PRIMARY DATA ANALYSIS
To analyse data collected from the questionnaires, the researcher used an analysis
method called thematic analysis. Thematic analysis is the most commonly used
method of data analysis in qualitative research. According to Braun and Clarke
“thematic analysis is a qualitative analytic method for identifying, analysing and
reporting patterns (themes) within collected data. It minimally organises and
describes data sets in detail. However frequently it goes further than this and
interprets various aspects of the research topic” (Braun & Clarke, 2006: 79).
This type of analysis moves beyond counting explicit words or phrases and focuses
on identifying and describing both implicit and explicit ideas within the data (themes).
From the themes, codes are then developed to represent the identified themes and
are then applied or linked to raw data as summary markers for later analysis. This
process is now referred to as thematic coding. Thematic coding refers to any method
of categorising segments of collected data into meaningful themes with the aim of
identifying patterns to draw conclusions.
For this study, the researcher coded the questions directed to officials of the DLGTA
M&E unit into themes. Related to the literature review, and also linked to the
questionnaire, the themes put forward or aimed to probe whether the challenges
faced by the service delivery M&E unit of the DLGTA unit in the implementation of a
municipal M&E system could be rooted either to the “institutional (organisation)” or
“human capacity and skills dimension”. To begin with and related to the institutional
(organisational) theme, the researcher benchmarked the DLGTA against provisions
of the NPM theory that advocates for governments to be driven by their mission of
improving their effectiveness and efficiency. The researcher analysed whether the
DLGTA had been in line with NPM provisions by improving and turning itself around
as an organisation and also strengthening its human capacity element for the
35
implementation of a municipal M&E system that would enable municipalities to be
assisted with their M&E reporting functions.
Secondly and related to the human capacity and skills dimension theme, the
researcher analysed whether the DLGTA has been considering the human capacity
element as an important part of an M&E system (human capacity in terms personnel
and the skills required). The researcher analysed this theme to check firstly, whether
the DLGTA has ensured that the service delivery M&E unit is capacitated in numbers
in terms of the right number of personnel. Secondly, the researcher analysed
whether in strengthening human capacity and skills, the human capacity element in
the DLGTA has been accompanied by evaluation capacity based on capacity
building plans that provide training on a range of M&E skills, tools, methods,
approaches, and concepts as these will enable the production of good M&E results.
Primary data analysis of the study firstly involved reading through responses from
the questionnaires sent out to respondents. As the questionnaires were already
designed into themes relating to the research problem, the researcher’s next step
was to draw connections and patterns amongst the collected information. Over and
above answering the questions, the researcher also looked for underlying
subthemes and patterns in the data and also attempted to interpret these underlying
patterns. Themes that the researcher identified were interpreted in relation to the
research questions and study objectives. Direct quotations were also taken from the
transcripts and integrated in this report as evidence to support the process of
reflection and describing the patterns where there was divergence of patterns from
the data. Reflecting on the data was also done to validate the primary data with
literature and to draw conclusions from the data.
3.5.2 SECONDARY SOURCES ANALYSIS
As earlier mentioned, the essence of highlighting secondary sources in this study
was to find out whether the DLGTA as an organisation has been using NPM
provisions as part of a results-based turnaround strategy towards improving its
efficiency and effectiveness. It is worth noting that it is important for the DLGTA to
apply provisions of the NPM theory because legislatively, this department is
mandated to monitor municipalities in the province in managing their own affairs,
36
exercising their powers and performing their functions. Legislatively, the DLGTA is
also expected to monitor the development of local government capacity in the
province and to assess the support needed by municipalities to strengthen their
capacity to manage their own affairs, exercise their powers and perform their
functions. With the above in mind, it is important that the DLGTA as an organisation
abides by the provisions of the NPM theory especially with regard to applying
turnaround strategies or mechanisms that will assist in improving its efficiency and
effectiveness.
In that case, secondary sources will be analysed in order to benchmark how the
DLGTA fared with regard to the following: institutionalisation of M&E systems,
prioritisation of the human capacity element as part of the requirements of an M&E
system and lastly, prioritisation of the evaluation capacity building element for the
capacitation of members of the M&E unit with numbers and with the required skills.
3.6 RELIABILITY AND VALIDITY
Patton (2002) stated that validity and reliability are two factors which any qualitative
researcher should be concerned about while designing a study, analysing results
and judging the quality of the study. This corresponds to the question “How can an
inquirer persuade his or her audiences that the research findings of an inquiry are
worth paying attention to?" (Lincoln & Guba, 1985: 290). To answer this question,
the quality of a study in each paradigm should be judged by its own paradigm's
terms. For example, while the terms reliability and validity are essential criteria for
quality in quantitative paradigms, in qualitative paradigms the terms credibility,
neutrality, consistency or dependability and applicability are to be the essential
criteria for quality (Lincoln & Guba, 1985).
The basis of reliability and validity for secondary sources in the study was the
literature review and the concepts that were used and applied. As advised by Lincoln
& Guba, for quality the researcher relied on the credibility, neutrality, consistency,
dependability and applicability of the concepts of M&E, M&E systems, human
capacity element within M&E systems and evaluation capacity building. These are
concepts or terms that were used and then applied in the study. Different sources of
information, including available literature on the theory of NPM, M&E, M&E systems,
37
human capacity element within M&E systems and evaluation capacity building, were
also used in the study to better inform the researcher when drawing conclusions.
For primary sources and in an effort to minimise bias in this study and as a form of
reliability and validity, the researcher relied on the primary data collection instrument
(questionnaire) and its consistency in producing credible data, appropriateness of
the data analysis technique and the degree of relationship between the conclusions
drawn and the data upon which they were drawn. In addition, direct quotes from the
questionnaires were used as part of the report to illuminate an argument and
illustrate that it was the voice of the respondent rather than that of the researcher.
3.7 RESEACHER’S BIASES
As the researcher is also an official of the DLGTA, the researcher tried by all means
to be as objective as possible for the purposes of not compromising the outcomes of
the study. The researcher also considered the ethics within the research process
thereby assuring that the researcher did not take any sides and was also not
prejudiced in any way or have any preconceived notions.
3.8 CONCLUSION
This chapter discussed the research methodology that was followed in the study.
The first part of the chapter discussed the generic conceptualisation related to the
qualitative social science research method and the rationale on why this method was
selected. The second part then outlined a research design that the researcher
followed for data collection, analysis and validation. This was an indication of how
the data collection was undertaken, identification of primary and secondary sources
of data, including analysis. Furthermore, the researcher discussed how the data will
be analysed and how the reliability and validity were maintained.
38
CHAPTER FOUR
PRESENTATION, ANALYSIS AND DISCUSSION OF FINDINGS
4.1 INTRODUCTION
The aim of this chapter is to present, analyse and discuss the findings that were
gathered pertaining to the challenges faced by the municipal service delivery M&E unit
of the DLGTA in implementing a municipal M&E system. This chapter presents,
analyses and discusses the findings that were gathered through the used primary
method of data collection (questionnaire). This presentation, analysis and discussion
of the findings also takes into account the provisions of the surveyed literature
(secondary sources) on whether the DLGTA as an organisation has been using NPM
provisions as part of a results-based turnaround strategy towards improving its
efficiency and effectiveness. This also includes an analysis of how the DLGTA fared
with regard to the following: institutionalisation of a municipal M&E system,
prioritisation of the human capacity element as part of the requirements of an M&E
system and lastly, prioritisation of the evaluation capacity building element for the
capacitation of members of the M&E unit with the right number of personnel who
also possess the required M&E related skills. The above also relate to the analysis
being framed by an assessment of the critical concepts of inputs/resources related to
the implementation of systems. This means assessing how the DLGTA fared in
attempts to capacitate the municipal service delivery unit initially with personnel in
terms of numbers and personnel who possess the required M&E skills.
Before presenting the findings, the researcher would like to give a reflection
regarding the questionnaires sent out. These are the viewpoints of respondents and
how some of them received the questionnaires. Worth noting, and as earlier
indicated, the researcher was unable to test the questionnaire as he was faced with
time constraints, however with all this in mind the researcher did make the necessary
changes to the questionnaire before sending it to respondents, based on the
supervisor’s input on the questionnaire. The researcher also kept in mind the
importance of asking questions that are clear and specific and that each respondent
would be able to answer without any challenges. From an identified sample of eight
respondents who were sent the questionnaires, the researcher only managed to
receive five responses. From the beginning, the researcher was a bit concerned with
39
the knowledge and understanding of respondents regarding the phenomenon or
discipline of M&E and capacity requirements for functional M&E systems as none of
them were qualified M&E practitioners with related M&E credentials.
However, the received five responses managed to assist the researcher a great deal
with regard to the required information for analysis purposes. The researcher relied
on the fact that some of the respondents identified have been members of the
municipal service delivery M&E unit for close to four years. This is the main reason
why they were selected as they were in a better position than most people in the
department to provide the researcher with valuable information that could be used
for analysis purposes regarding the challenges faced by the municipal service
delivery M&E unit in attempts to implement a municipal M&E system.
4.2 PRESENTATION OF THE FINDINGS
As indicated in the previous chapter and as a primary source of data collection, the
researcher sent out a questionnaire to current and former officials of the service
delivery M&E unit in the DLGTA for their hands-on insight regarding the problem. For
analysis purposes, the researcher also took into consideration the importance of
presenting the questionnaire in themes. The proposed themes aimed to probe
whether the challenges faced by the service delivery M&E unit of the DLGTA in the
implementation of a municipal M&E system could be rooted either in the “institutional
(organisation)” or “human capacity and skills dimensions”. In addition and related to
the surveyed literature, the questionnaire also meant to capture responses with regard to
establishing whether the DLGTA had been aware of the earlier mentioned provisions of
the importance M&E systems, human resource capacity requirements within M&E
systems and the importance of capacity building in M&E systems.
40
4.2.1 DIMENSION ANALYSIS (INSTITUTIONAL/ ORGANISATIONAL)
The aim of this dimension of the questionnaires was to explore whether the
challenges experienced by the DLGTA in the implementation of a municipal M&E
system are rooted in the institutional/organisational make up of the DLGTA. This
section of the questionnaire comprised five questions that can be presented into the
following headings: (1) How the DLGTA views its role as the coordinating body of
M&E in the province, (2) Departmental efforts embarked upon to prioritise the
capacitation of the service delivery M&E unit, (3) Efforts embarked upon in the
implementation of an M&E system, (4) Importance of the work produced by the
service delivery M&E unit towards the attainment of the overall goals of the DLGTA,
and (5) The roles and responsibilities of members of the service delivery M&E unit
and their definition.
(1) How the DLGTA views its role as the coordinating body of M&E in the
province
A number of responses on this question seemed to agree that to a certain extent,
there is some level of understanding of the role of the department in terms of it being
the coordinating body of municipal M&E in the province. Respondents indicated that:
“To a certain extent, there is some level of understanding of the role of
the department in terms of it being the coordinating body of municipal
M&E in the province”. (Maphunye, “Data Collection Guide”,
Questionnaire, 19 December 2013).
Additionally respondents also indicated that:
“The department is mandated to monitor and support the local
government sphere in the province, as such, there is common
understanding of the centrality of M&E in the department”. (Maphunye,
“Data Collection Guide”, Questionnaire, 19 December 2013).
Concerns that were raised however were that the department does not recognise the
work produced by the service delivery M&E unit as central to the core mandate of
41
the department. Another major concern that was highlighted across the responses
was that from a departmental point of view there seems to be a disjuncture between
the importance and the understanding of the actual institutionalisation of M&E
including what capacity and resources must be considered in order to ensure an
efficient M&E unit and system in the department.
(2) Departmental efforts embarked upon to prioritise the capacitation of the
service delivery M&E unit
In line with this question, responses seemed to agree that from a departmental point
of view, there has been very little effort to ensure full capacitation of the M&E unit,
even where numbers have been filled, the skills and experience cannot match the
required outputs in terms of the understanding of the content and the monitoring
output.
“The department has made little effort in relation to the capacitation of
the unit even though the nature and amount of the work indicated that
the unit is in serious need of capacitation, not only in terms of the
number of personnel but highly skilled personnel who have a solid M&E
background. This also reflected how little attention has been paid to the
M&E unit which is supposed to form a part of the core of the
departmental mandate”. (Maphunye, “Data Collection Guide”,
Questionnaire, 19 December 2013).
(3) Efforts embarked upon in the implementation of a M&E system
Most responses to this question pointed to the development of a departmental M&E
framework:
“The M&E framework that the department has developed and currently
attempting to rollout to municipalities is the only effort in attempts to
institutionalise a municipal M&E system. The framework however does
spell out exactly how the unit intends to approach its work in relation to
42
the M&E of municipalities”. (Maphunye, “Data Collection Guide”,
Questionnaire, 19 December 2013).
A concerning point that was noted about the framework in the questionnaires was
that:
“Municipalities in the province however still need to be taken through
the framework and their approval is also required”. There also has been
a recent capacitation of the unit in terms of personnel and some work
has gone in assessing the effectiveness of the M&E system, which forms
part of the new departmental strategy, which must culminate into a
new service delivery model and the structure that better responds to
the gaps identified in the assessment”. (Maphunye, “Data Collection
Guide”, Questionnaire, 19 December 2013).
(4) Importance of the work produced by the service delivery M&E unit towards
the attainment of the overall goals of the DLGTA
There seemed to be agreement from the respondents that:
“The work of the unit should be considered very important to the
attainment of the goals of the department as the unit’s work is based on
identifying Gauteng municipality’s areas of non-performance. This will
then lead to the Department supporting municipalities in the identified
areas of non-performance as well as assisting them with regard to their
integrated development planning processes.” (Maphunye, “Data
Collection Guide”, Questionnaire, 20 December 2013).
Respondents considered the work of the unit to be of utmost importance as its aim is
to also identify municipal service delivery challenges, early identification of matters
that could block service delivery and measures that could unblock them.
43
(5) The roles and responsibilities of members of the service delivery unit and
their definition
There seems to be agreement from respondents that the roles and responsibilities
are to some extent clearly defined, according to the defined posts in terms of what is
expected from individuals. It was noted that:
“Some roles are also interchangeable in nature and are also clearly
communicated when necessary in terms of the content/subject matter.
However, there has been no effort to really concretise the roles per
individual to ensure maximum outcomes. Some of these roles include
what regions (municipalities) an individual must coordinate, which Key
Performance Area (KPA) each individual or a unit member should
specialise on”. (Maphunye, “Data Collection Guide”, Questionnaire, 19
December 2013).
Additionally and as much as there are job descriptions for unit members, these
however cannot be fulfilled as unit members sometimes find themselves in a position
where they are overwhelmed with ad-hoc duties and thereby neglect some of their
roles and responsibilities.
4.2.2 DIMENSION ANALYSIS: HUMAN CAPACITY AND SKILLS
The aim of this dimension of the questionnaires was to explore whether the
challenges experienced by the service delivery M&E unit of the DLGTA in the
implementation of a municipal M&E system could be rooted to the neglect to
capacitate the M&E unit firstly with personnel in terms of numbers as well as
personnel who are also highly skilled with regard to M&E practices. This section of
the questionnaire comprised eight questions that can be presented into the following
headings: (1) Capacitation of the unit to properly carry out its M&E functions, (2)
Ability of unit members related to the implementation of municipal M&E systems, (3)
Ability of unit members related to the management of evaluation projects, (4) Attendance
of M&E training or capacity building courses (5) Ability and skills of members of the
unit to carry out their functions effectively, efficiently and sustainably, (6) Knowledge of
unit members in relation to quantitative and qualitative research methods, (7) Ability of
44
members of the unit related to supervising service providers, (8) Ability of unit
members in presenting M&E reports.
(1) Capacitation of the unit to properly carry out its M&E functions
On the capacitation of unit members, respondents agreed that currently the unit
seems to be capacitated in terms of the number of personnel and they are able to
carry out their M&E functions towards municipalities; however this was not the case
in years gone by. A number of respondents also noted that
“The unit is not properly capacitated in line with the departmental
strategic outlook and the expected monitoring outcomes in terms of
responding properly to various strategies such as outcomes, National
Development Plan and the Gauteng City Region strategy”. (Maphunye,
“Data Collection Guide”, Questionnaire, 20 December 2013).
Respondents also agreed that
“it would have been appropriate if each municipal district would have
an assistant manager, and in addition to this there would also be
officials who would be specialising in monitoring as to whether
municipalities do comply with legislative prescripts (legislative
compliance)”. (Maphunye, “Data Collection Guide”, Questionnaire, 20
December 2013).
An important point to note about the above response relates to what was mentioned
earlier about the knowledge and understanding of respondents regarding capacity
requirements for functional M&E systems. A number of respondents seemed not to
have grasped this question correctly as the researcher was only interested in finding
out whether the service delivery M&E unit had been properly capacitated in terms of
the right number of personnel who also possess the proper M&E skills to be able to
carry out their M&E functions and thereby implement a municipal M&E system.
45
(2) Ability of unit members related to the implementation of municipal M&E
systems
There was an agreement amongst respondents about members of the unit
possessing a somewhat slight background related to M&E, however as it was also
indicated that:
“The implementation of a municipal M&E system will also be proved by
the implementation of the Departmental M&E framework includes a
system that needs to be followed in relation to the M&E of
municipalities. The departmental M&E framework firstly has to be rolled
out to municipalities; this will then determine whether unit members
are in a position to implement a municipal M&E system”. (Maphunye,
“Data Collection Guide”, Questionnaire, 21 December 2013).
(3) Ability of unit members related to the management of evaluation projects
Generally, respondents were concerned with the level of abilities of unit members in
relation to managing evaluation projects as members hold qualifications in various
fields of study, some unrelated to the M&E field. It was also noted that some unit
members may have conducted evaluation studies in some area or the other;
therefore generally there is an inadequate ability to manage evaluation studies
internally. Worth noting as well, is that respondents alluded to the training or
capacitation on M&E that unit members need to undergo in order to improve on this.
(4) Attendance of M&E training or capacity building courses
A concern was raised by most respondents with regard to the department not
prioritising the attendance of M&E training or capacity building courses for M&E unit
members. Respondents regularly noted that
“In the last 5 years, a number of unit members only attended one week
long training and capacity building course on M&E. Members have also
been offered general report writing courses and training in developing
an M&E system. However no M&E courses have been offered on a
regular basis”. (Maphunye, “Data Collection Guide”, Questionnaire, 19
December 2013).
46
In relation to this response, the respondents also seemed to agree that the onus lay
with the Director of the municipal service delivery M&E unit to lead the orientation on
the M&E system by ensuring that firstly, the M&E unit is well capacitated and also,
ensuring that staff members of the unit attend M&E related training on a regular
basis in order to improve their skills.
(5) Ability and skills of members of the unit to carry out their functions
effectively, efficiently and sustainably
Respondents indicated that even though skill levels are very low, however certain
members of the unit do possess somewhat relevant skills and expertise in relation to
carrying out their duties effectively. It was also noted that as M&E is considered as
an evolving discipline, it is important that members of the unit attend M&E training
and capacity building courses on a regular basis in order to build and improve on
their skills.
(6) Knowledge of unit members in relation to quantitative and qualitative research
methods
This question intended to explore how knowledgeable unit members are regarding
qualitative and quantitative research methods as part of carrying out their M&E tasks
and also as part of individual competencies or attributes for M&E practitioners.
Respondents generally agreed that members of the unit are knowledgeable about
qualitative and quantitative research methods.
(7) Ability of members of the unit related to supervising service providers
Respondents indicated that some members have such experience and have
supervised service providers before, however the majority would not have been
exposed to such, but could handle it well given the opportunity. Additionally
respondents agreed that:
“Certain members of the unit have proven themselves to have
supervision abilities relating to service providers, as the projects that
were once embarked upon when service providers were involved turned
out to be successful”. (Maphunye, “Data Collection Guide”,
Questionnaire, 20 December 2013).
47
(8) Ability of unit members in preparing and presenting M&E reports
Respondents generally agreed that unit members do possess the ability to prepare
and present M&E reports:
Certain members of the unit are skilful in presenting M&E reports; however these skills
need to be constantly improved upon as M&E is an evolving discipline. Just like in all
the skills related questions, this also needs to be improved upon through the M&E
training and capacity building courses” (Maphunye, “Data Collection Guide”,
Questionnaire, 22 December 2013).
4.3 ANALYSIS OF THE FINDINGS (DISCUSSIONS)
INSTITUTIONAL/ ORGANISATIONAL DIMENSION ANALYSIS
As a departure point, there was a level of understanding on the role of the
department in terms of it being the coordinating body of municipal M&E in the
Gauteng province. However the DLGTA, by not properly viewing itself as the
coordinating body of municipal M&E in the province, has somewhat led to the
organisation not prioritising the capacitation of the municipal service delivery M&E
unit. This is indicated in the responses as they pointed out the little effort that was
made with regard to capacitation of the municipal service delivery M&E unit of
personnel who also posses the required M&E skills. Another major concern worth
highlighting is that, from a departmental point of view, there seems to be a
disjuncture between the importance and the understanding of the actual
institutionalisation of M&E, including what capacity and resources/inputs must be
considered in order to ensure an efficient M&E unit and system in the department.
Other than the departmental M&E framework that the department is in the process of
rolling out to municipalities, there was no other indicated initiative that aims to
implement a municipal M&E system. To this day, the framework remains the only
effort in attempts to implement a municipal M&E system. The framework, however,
does not spell out exactly how the unit intends to approach its work in relation to the
M&E of municipalities. Municipalities in the province also need to be taken through
the framework and their approval is also needed as it will have some bearing on their
reporting duties towards the department. The framework still also has to be rolled out
48
to municipalities and this will determine whether the current unit members are in a
position to implement a municipal M&E system or not.
The work produced by the service delivery M&E unit seems not to be considered as
very important towards the attainment of the overall goals of the DLGTA even though
this work is based on identifying Gauteng municipality’s areas of non-performance,
which is one of the core mandated aspects of the department. This will then lead to
the Department supporting municipalities in the identified areas of non-performance,
as well as assisting them with regard to their integrated development planning
processes. The unit’s work should be considered to be of utmost importance as it is
based on identifying municipal service delivery challenges, early identification of
matters that could block service delivery and measures that could be embarked upon
to unblock them.
The roles and responsibilities of members of the service delivery unit and their
definition seem to be clearly defined according to the defined posts in terms of what
is expected from individuals. However, there has been no effort to really concretise
the roles per individual to ensure maximum outcomes, for example dividing members
of the unit to focus on different municipal regions and specialisation in areas of work
or municipal KPAs. Additionally, and as much as there are job descriptions for unit
members, these however cannot be fulfilled as unit members sometimes find
themselves in a position where they are overwhelmed with ad-hoc duties and
thereby neglect some of their roles and responsibilities.
HUMAN CAPACITY AND SKILLS DIMENSION ANALYSIS
The challenges faced by the service delivery M&E unit of the DLGTA in the
implementation of a municipal M&E system seem to be rooted within the institutional
make up of the DLGTA as an organisation and this has led the DLGTA not to
capacitate the human and skills dimensions of the municipal service delivery M&E
unit. There seems to be a link between the institutional makeup of the DLGTA and
the lack of prioritising the capacitation of the M&E unit in terms of personnel who are
highly skilled in M&E practices. This alludes to points raised earlier that the DLGTA
still does not view itself as the coordinating body of municipal M&E in the province
49
and also that this organisation does not view the work undertaken by the service
delivery unit as core to achieving its legislated mandate. Currently the municipal
service delivery M&E unit seems to be capacitated only in terms of the number of
personnel and they are still unable to carry out their M&E functions towards
municipalities as required. Municipal service delivery M&E unit members still need to
be taken for M&E related courses to improve their knowledge and skills base for
them to properly execute their M&E duties and also to implement a municipal M&E
system.
Other concerns highlighted were that members of the municipal service delivery unit
also lack proper evaluation skills as currently most of the unit’s work is confined to
the monitoring aspect and it does not cover the evaluation aspect. The department
has also made little effort in relation to the capacitation of the unit even though the
nature and amount of the work indicated that the unit is in serious need of
capacitation, not only in terms of personnel but highly skilled personnel who have a
solid M&E background. This also reflected how little attention has been paid to the
M&E unit which is supposed to form a part of the core of the departmental mandate.
Though some efforts have been made recently towards capacitating the service
delivery M&E unit, there remains some ground to be covered in the implementation
of a municipal M&E system. This will also largely depend on the DLGTA prioritisation
of the evaluation capacity building element for the capacitation of members of the
M&E unit with the right personnel who possess the required M&E skills. It was
indicated that even though skill levels are very low, however certain members of the
unit do posses somewhat relevant skills and expertise in relation to carrying out their
duties effectively. It was also noted that as M&E is considered as an evolving
discipline, it is important that members of the unit attend M&E training and capacity
building courses on a regular basis in order to build and improve on their skills.
50
4.4 SECONDARY DATA ANALYSIS
The little effort that was made towards demonstrating a new wave of thinking or
results driven approach that could have been applied in relation to the
institutionalisation of policies, programmes and processes shows that the DLGTA as
an organisation has not been in line with provisions of the NPM theory. This was also
highlighted by not taking a stand as the coordinating body of M&E in the Gauteng
province. The lack of prioritising the capacitation of the municipal service delivery
M&E unit also means that the DLGTA has not been driven by government’s mission
of improving its effectiveness and efficiency as advocated by the NPM theory. Taking
note of the NPM theory which directs towards a results-based turnaround strategy
based on improving efficiency and effectiveness towards Gauteng municipalities, the
DLGTA management has not created an enabling environment based on effective
internal processes that will ultimately assist Gauteng municipalities with their M&E
functions, as required by legislation. Secondary sources were analysed in order to
benchmark how the DLGTA fared with regard to the following: institutionalisation of
M&E systems, prioritisation of the human capacity element as part of the
requirements of an M&E system and lastly, prioritisation of the evaluation capacity
building element for the capacitation of members of the M&E unit with numbers and
with the required skills. Worth noting is that in the department, little effort was made
in relation to the above.
It is worth noting that it is important for the DLGTA to apply provisions of the NPM
theory because legislatively, this department is mandated to monitor municipalities in
the province in managing their own affairs, exercising their powers and performing
their functions. Legislatively the DLGTA is also expected to monitor the development
of local government capacity in the province and to assess the support needed by
municipalities to strengthen their capacity to manage their own affairs, exercise their
powers and perform their functions. With the above reiterations in mind and from an
analysis of secondary sources, it is worth noting that the DLGTA has also not been
abiding by the provisions of the NPM theory, especially with regard to applying
turnaround strategies or mechanisms that will assist in improving its efficiency and
effectiveness as the challenges regarding institutionalising a municipal M&E system
still persist.
51
4.5 CONCLUSION
The aim of this chapter was to present, analyse and discuss the findings that were
gathered pertaining to the challenges faced by the municipal service delivery M&E unit
of the DLGTA in implementing a municipal M&E system. This chapter presented,
analysed as well as discussed the findings that were gathered through the
questionnaires. The presentation, analysis and discussion of the findings also took
into account the provisions of the surveyed literature (secondary sources) on
whether the DLGTA as an organisation had been using NPM provisions as part of a
results-based turnaround strategy towards improving its efficiency and effectiveness.
A discovery was made that the DLGTA management has not created an enabling
environment based on effective internal processes that will ultimately assist Gauteng
municipalities with their M&E functions as required by legislation.
52
CHAPTER FIVE
SYNTHESIS OF THE ANALYSIS TO THE FINDINGS
5.1 INTRODUCTION
This chapter aims to present a synthesis of the analysis of the findings raised in this
study regarding the challenges faced by the municipal service delivery M&E unit of
the DLGTA in the implementation of a municipal M&E system. This will be followed
by proposed measures that can be adopted by the municipal service delivery M&E
unit which may assist in overcoming the challenges encountered in implementing a
municipal M&E system. A part of the proposed measures includes a municipal M&E
system (attached as Annexure A) that the researcher proposes which the DLGTA
service delivery M&E unit can implement in their efforts to conduct municipal M&E
and also to assist Gauteng municipalities with their M&E functions. To begin with, it
is worth noting that after analysing the findings presented in chapter 4, the
researcher realised that the inability of the municipal service delivery M&E unit of the
DLGTA in the implementation of a municipal M&E system can be attributed to two
challenges. The challenges are with regard to “The lack of the DLGTA to abide by
provisions of the NPM Theory” and “The institutional (organisational) make up of the
DLGTA”.
5.1.1 The lack of the DLGTA to abide by provisions of the NPM Theory
It was indicated that for this study, the NPM theory and its provisions were reflected
upon to highlight a new wave of thinking or results driven approach that the public
sector can apply for the institutionalisation or implementation of its policies,
programmes and processes. This study was therefore premised on the operational
notions that allude to the fact that NPM is relevant for the public sector as it proposes
that governments should be driven by their mission of improving their effectiveness
and efficiency. Despite critics that saw NPM doctrines as too heavily based on
business-school and private-sector management perspectives whose key parts of
the reform message have been reversed because they lead to policy disasters, NPM
theory was still considered relevant for the study as it is also continues to be widely
used by practitioners and academics alike. The theory of NPM was deemed
important for the study because it advocates for business and private-sector
53
management perspectives that are required and could be useful in governments. For
specificity, its operational notions argue that NPM is relevant for the public sector as
it proposes that governments should be driven by their mission of improving their
effectiveness and efficiency. Also in line with being driven by this mission,
governments will be adopting a new wave of thinking or results driven approach that
can be applied upon the implementation of their policies, programmes and
processes.
Being cognisant of the NPM theory provisions which direct towards a results-based
turnaround strategy based on improving efficiency and effectiveness, the researcher
realised that the DLGTA did not create an enabling environment based on effective
internal processes that demonstrate the willingness to be driven by its mission of
improving its effectiveness and efficiency towards Gauteng municipalities. This is
going against the provisions of NPM that advocate for governments to be driven by
their mission of improving their effectiveness and efficiency. A part of the turnaround
of internal processes was meant to commence with prioritising the implementation of
a municipal M&E system. This system, in turn, would lay a foundation for how the
municipal service delivery M&E unit would roll out assistance to municipalities in the
Gauteng province for their M&E functions. This would be taken as the DLGTA being
efficient and effective in Gauteng municipalities as it would be acting according to its
mandate which is to monitor municipalities in the province in managing their own
affairs, exercising their powers and performing their functions.
5.1.2 The institutional (organisational) make up of the DLGTA
By not taking a stand or viewing itself as the coordinating body of municipal M&E in
the Gauteng province, the DLGTA opened a door for not recognising the work of the
service delivery M&E unit as core to the attainment of the departmental mandate and
objectives. This is part of the gap that exists between the institutional makeup of the
DLGTA and the lack of prioritising the capacitation of the service delivery M&E unit.
The institutional makeup of the DLGTA by not properly viewing itself as the
coordinating body of municipal M&E in the province contributed immensely to this
organisation not prioritising the capacitation of the service delivery M&E unit.
Because the DLGTA does not view itself as the coordinating body of M&E in the
province, there has been very little effort to ensure full capacitation of the service
54
delivery M&E unit, even though the nature and amount of the work of the unit
indicates that the unit is in serious need of capacitation, not only in terms of
personnel but highly skilled personnel who have a solid M&E background. This also
reflected how little attention has been paid to the M&E unit which is supposed to
form a part of the core of the departmental mandate. By taking a stand as the
coordinating body of municipal M&E in the province, the DLGTA will be in a position
to realise the capacity requirements that need to be a part of a functional M&E
system. A part of these requirements would entail capacitation of the service delivery
M&E unit with personnel who are highly skilled in relation to M&E practices.
To bridge the gap earlier mentioned, the DLGTA would then begin by viewing itself
as the coordinating body of municipal M&E in the province and this would lead to a
realisation of the type of personnel and the skills required for capacitation of the
service delivery M&E unit. This capacitation in terms of personnel who possess
proper M&E skills would ultimately lead to the implementation of a municipal M&E
system. Capacitation of the service delivery M&E unit however has to go hand in
hand with the aspect of evaluation capacity building which will involve unit members
undergoing related M&E training and skills development courses on a regular basis
to improve their skills as M&E can be considered as an evolving discipline.
As earlier mentioned, a part of this chapter also includes a municipal M&E system
that the researcher proposes which the DLGTA service delivery M&E unit can
implement in their efforts to conduct municipal M&E and to assist Gauteng
municipalities with their M&E functions. The researcher therefore designed a system
(attached as Annexure A) which can be used by the service delivery M&E unit of the
DLGTA in its efforts to conduct municipal M&E in the Gauteng province.
5.2 CONCLUSION
The first part of this chapter aimed at presenting a synthesis of the analysis to the
findings raised in this study regarding the challenges faced by the municipal service
delivery M&E unit of the DLGTA in the implementation of a municipal M&E system.
In the first part of the chapter, the researcher realised that the inability of the
municipal service delivery M&E unit of the DLGTA in the implementation of a
municipal M&E system can be attributed to two challenges which were discussed as
55
“The lack of the DLGTA to abide by provisions of the NPM Theory” and “The
institutional (organisational) makeup of the DLGTA”. Part two of the chapter then
proposed a municipal M&E system (attached as Annexure A) that the researcher
designed which the DLGTA service delivery M&E unit can implement in their efforts
to conduct municipal M&E and also to assist Gauteng municipalities with their M&E
functions. The researcher therefore proposes this system which can be used by the
service delivery M&E unit of the DLGTA in their efforts to conduct municipal M&E in
the Gauteng province.
56
CHAPTER SIX
CONCLUDING REMARKS
The aim of this study was to explore the challenges faced by the municipal service
delivery M&E unit of the DLGTA in their efforts to implement a municipal M&E
system. The inability of the municipal service delivery M&E unit of the DLGTA in the
implementation of a municipal M&E system was attributed to two challenges which
were discussed as “The lack of the DLGTA to abide by provisions of the NPM
Theory” and “The institutional (organisational) makeup of the DLGTA”.
Because the DLGTA does not abide by provisions of NPM, the study firstly began by
indicating that the NPM theory and its provisions were reflected upon to highlight a
new wave of thinking or results driven approach that the public sector can apply
upon the institutionalisation or implementation of its policies, programmes and
processes. The theory of NPM was deemed important for the study because it
advocates business and private-sector management perspectives that are required
and could be useful in governments. This relates to operational notions of the NPM
theory that argue that NPM is relevant for the public sector as it proposes that
governments should be driven by their mission of improving their effectiveness and
efficiency.
Taking note of the NPM theory provisions which direct towards a results-based
turnaround strategy based on improving efficiency and effectiveness, the researcher
realised that the DLGTA did not abide by these provisions to create an enabling
environment based on effective internal processes that demonstrate its willingness to
be driven by its mission of improving its effectiveness and efficiency towards
Gauteng municipalities. A part of the turnaround of internal processes was meant to
commence with prioritising the implementation of a municipal M&E system. This
system in turn would lay a foundation for how the municipal service delivery M&E
unit would roll out the assistance of municipalities in the Gauteng province with their
M&E functions. This would also mean that the DLGTA is broadly exercising its
mandated role of monitoring municipalities in the province in managing their own
affairs, exercising their powers and performing their functions.
57
In relation to the institutional (organisational) makeup of the DLGTA, the researcher
realised that the DLGTA by not properly viewing itself as the coordinating body of
municipal M&E in the province, contributed to this organisation not prioritising the
capacitation of the service delivery M&E unit. By not recognising itself as the
coordinating body of M&E in the province, the DLGTA therefore could not see or
recognise the importance of the service delivery M&E unit as core to the attainment
of the departmental mandate and objectives. It also reflects how little attention has
been paid to the M&E unit which is supposed to form a part of the core of the
departmental mandate. Though some efforts have been made recently towards
capacitating the service delivery M&E unit, there remains some ground to be
covered in the implementation of a municipal M&E system. There has been very little
effort to ensure full capacitation of the service delivery M&E unit even though the
nature and amount of the work indicated that the unit is in serious need of
capacitation, not only in terms of personnel but highly skilled personnel who have a
solid M&E background. A consequence of the above departmental shortcomings is
that they lead to the challenges of the DLGTA service delivery M&E unit to
implement a municipal M&E system that will be used to conduct municipal M&E and
also assist Gauteng municipalities with their M&E functions.
58
REFERENCE LIST
Adei, S, & Badu, Y.A. (2006). Capacity Building in the Public Sector: Lessons from
the Transformation of GIMPA, in Awortwi N, & Sitoe E. (Ed). African Perspectives
on New Public Management: Implications for Human Resource Training,
Amsterdam: Shaker Publishing, pp 99-126.
Boyle, R & Lemaire, D. (1999). Building Effective Evaluation Capacity: Lessons
from Practice. New Brunswick: Transaction Publishers.
Braun, V. & Clarke, V. (2006). Using Thematic Analysis in Psychology:
Qualitative Research in Psychology. Bristol: University of West England.
City of Tshwane Metropolitan Municipality (2008). City of Tshwane Annual Report
2007/2008 Financial Year. Pretoria: The City.
Collaborative Africa Budget Reform Initiative (CABRI). (2006). Bridging the Gap:
From Policies to Budget, 3rd Annual CABRI Seminar, 28-30 November, Addis
Ababa, Ethiopia, German Technical Cooperation (GTZ).
Cresswell J.W. (1994). Research Design: Qualitative and Quantitative
Approaches. London: Sage Publications.
Department of Local Government and Housing (2011). Planning, Monitoring,
Evaluation and Reporting Policy Framework 2011/12 Financial Year.
Johannesburg: The Department.
Dunleavy, P., Margetts, H., Bastow, S. & Tinkler, J. (2005). New Public Management
is Dead: Long Live Digital Era Governance, in Journal of Public Administration
Research and Theory, Volume No. 16 (3), London: Oxford University Press, pp 467
– 494.
Ferlie, E., Ashburner, L., Fitzgerald, L. & Pettigrew, A. (1996). The New Public
Management in Action. Oxford: Oxford University Press.
59
Gauteng Department of Cooperative Governance and Traditional Affairs, (2013).
Repositioning of Municipal Service Delivery Monitoring & Evaluation
Directorate, GCOGTA: 2013. Johannesburg: The Department.
Gauteng Department of Local Government and Housing, (2009). Municipal
Quarterly Performance Report (April – June) 2008/2009 Financial Year.
Johannesburg: The Department.
Global Fund (2009). Monitoring and Evaluation Toolkit: HIV, Tuberculosis,
Malaria and Health Systems Strengthening, Part 1: The M&E System and
Global Fund M&E Requirements. 3rd ed. Washington, DC: The Fund.
Gorgens, M. Kusek, J.D. (2009). Making Monitoring and Evaluation Systems
Work: A Capacity Development Toolkit. Washington, DC: The International Bank
of Reconstruction and Development.
Hood, C. (2001). New Public Management, in International Encyclopaedia of the
Social & Behavioural Sciences (Volume 14) Lisse: Elsevier, pp 12553 -12556.
Kalof, L., Dan, A & Dietz, T. (2008). Essentials of Social Research. New York:
Open University Press.
Leedy, P.D. & Ormrod, J.E. (2005). Practical Research: Planning and Design, 9th
Ed. Boston: Pearson.
Lincoln, Y. S, & Guba, E. G. (1985). Naturalistic Inquiry. Beverly Hills, CA: Sage.
Lynn, E. L. (2006). Public Management: Old and New. London: Routledge.
Mack, N., Woodsong, C., Macqueen, K., Guest, G. & Namey, E. (2005). Qualitative
Research Methods: A Data Collector’s Field Guide. New Yourk: Family Health
International.
Mackay, K. (2002). The World Bank’s ECB Experience, in New Directions for
Evaluation, Volume No. 93 (43), pp 81 - 100.
MacKay, K. (2007). How to Build M&E systems to Support Better Governance.
Washington, DC: World Bank.
60
Miller, D.Y, & Dunn, W.N. (2006). Critical Theory of New Public Management,
Pittsburgh: Graduate School of Public and International Affairs & Macedonia
Graduate Centre for Public Policy and Management, University of Pittsburgh.
Neuman, W.L. (2006). Social Research Methods, Qualitative and Quantitative
Approaches (6th Ed) Boston: Pearson.
Patton, M. Q. (2002). Qualitative Evaluation and Research Methods (3rd Ed.)
Thousand Oaks, CA: Sage.
Presidential Review Commission [PRC] (1998). Developing a Culture of Good
Governance, Report of the Presidential Review Commission on the Reform
and Transformation of the Public Service in South Africa. Presented to the
President of South Africa, Mr. N.R. Mandela - 27 February1998. Cape Town:
Government Printer.
Republic of South Africa (1996). Constitution of the Republic of South Africa.
Cape Town: Government Printer.
Republic of South Africa (2004). Government Wide Monitoring and Evaluation
Framework: Proposal and Implementation Plan for a Government Wide
Monitoring and Evaluation System, Policy Coordination and Advisory Services
(PCAS). Cape Town: Government Printer.
Republic of South Africa. (2000). Local Government: Municipal Systems Act 32
of 2000, as amended. Cape Town: Government Printer.
Republic of South Africa (2007). Policy Framework for the Government Wide
Monitoring and Evaluation System. Pretoria: Government Printer.
Rist, C.R., Boily, M.H. & Martin, F. (2011). Influencing Change: Building
Evaluation capacity to Strengthen Governance. Washington, DC: World Bank.
Scott, L. & Joubert, A. (2005). The Monitoring and Evaluation of Government Policy:
Appropriate Alignment across the spheres and functions of government,
Working Paper no. 1: Monitoring and Evaluation as Tool of Governance. Cape Town:
Department of Statistical Sciences, University of Cape Town.
61
Sharma, K. C. (2006). Improving Public Administration through New Public
Management: The Experience of Botswana, in Awortwi N, & Sitoe E. (Ed) African
Perspectives on New Public Management: Implications for Human Resource
Training. Amsterdam: Shaker Publishing, pp 127-145.
Simister, N. & Smith, R. (2010). Monitoring and Evaluating Capacity Building: Is
it really that difficult? New York: International NGO Training and Research Centre
(INTRAC).
Stevahn, L., King, J.A., Ghere, G. & Minnema, J. (2005). Establishing Essential
Competencies for Program Evaluators. American Journal of Evaluation, Volume
26 (1), pp 43-59.
Stokdill, S.H., Baizerman, M. & Compton, D.W. (2002). Toward a Definition of
Evaluation Capacity Building Process: A Conversation with the Evaluation Capacity
Building Literature, in New Directions for Evaluation, Volume No. 93 (39), pp 7-26.
United Nations Development Programme Evaluation Office (2002). Handbook on
Monitoring and Evaluation for Results. New York: United Nations.
Vigoda, E. (2003). New Public Management, in Encyclopaedia of Public
Administration and Public Policy, Volume No. 10 (1081), New York: Marcel
Dekker, pp 812 – 816.
World Bank (2004). Monitoring and Evaluation: Some Tools Methods and
Approaches, Washington, DC: World Bank.
Internet Material
Australian Government: Department of Foreign Affairs and Trade, 2012, Proposed
Monitoring and Evaluation System, accessed 07 March 2012,
<http://www.ausaid.gov.au/files.html>. Australian Government Web Services,
2013.
Prennushi, G., Rubio, G. & Subbarao, K. (2002), Monitoring and Evaluation, World
Bank Poverty Library, Volume 1 – Core Techniques and Cross Cutting Issues in
Poverty Reduction Strategy Paper, accessed 26 May 2013,
<http://povlibrary.worldbank.org/files.html>. World Bank Web Services, 2013
62
Hopkins, T.J. (1994). Handbook on Capacity Assessment Methodologies: An
Analytical Review, accessed 4 June 2013,
<http://www.pogar.org/publications/other/undp.html>. United Nations
Development Programme Web Services, 2013.
63
ANNEXURE A: The Department of Local Government and Traditional Affairs
(DLGTA) Proposed Municipal M&E System
As mentioned in the introduction as well as in chapter five, the study also intended to
design and propose an M&E system that the municipal service delivery M&E unit of
the DLGTA could implement in their efforts to carry out the M&E of municipalities in
the Gauteng Province. This system can also assist Gauteng municipalities with their
M&E functions. The researcher therefore designed the following system, attached as
Annexure A that can be used by the service delivery M&E unit of the DLGTA in their
efforts to conduct municipal M&E in the Gauteng province.
The purpose of developing the M&E system is mostly founded around the following
South African National Government legislation and documentation: Constitution of
the Republic of South Africa (Act 108 of 1996), Local Government Municipal
Systems Act (MSA, Act 32 of 2000), Municipal Finance Management Act (MFMA,
Act 56 of 2003), Performance Monitoring and Evaluation White Paper, Government
Wide Monitoring and Evaluation System (GWM&ES), Monitoring and Evaluation
Framework in Support of Cooperative Governance (2007), The Presidency Draft
Policy Framework for the GWM&ES 2007, and the National Treasury Framework for
Managing Programme Performance Information (2007). The above documentation
advocate for the need to initialise and institutionalise M&E and related systems
across all government spheres. For the DLGTA, M&E functions, roles and
responsibilities are founded in the following documents: The Gauteng M&E
Provincial Framework and The DLGTA municipal M&E Framework. In the
development of this system for the DLGTA, the researcher was mostly guided the
work of Kusek and Rist (2004), based on their proposed ten steps to a results based
M&E system.
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
64
Strategy/System:
Department of Local Government and Traditional Affairs (DLGTA Municipal M&E System
Overall Outcome:
Improved Municipal M&E in the Gauteng Province
Specific Objectives/Goals:
Monitoring, Evaluation and reporting on municipal performance throughout the 5 Key Performance Areas (KPA)
1. Institutional Transformation and Organisational Development
2. Service Delivery and Infrastructure Development
3. Local Economic Development
4. Financial Viability and Management
5. Good Governance and Intergovernmental Relations (IGR)
1. Readiness Assessment
The DLGTA municipal M&E unit will be the champions of this system
The system will assist in making municipal M&E in the Gauteng province more effective
The system is intended for the following recipients
DLGTA internal units
Gauteng Municipalities
Gauteng Premier’s Office
Provincial M&E Coordinating Forums
National Department of Performance Monitoring and Evaluation
National Department of Cooperative Governance and Traditional Affairs
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
65
The following have been identified as impediments to the current system
Human resource capacity (in terms of numbers and skills, including implementation skills)
The DLGTA not viewing itself as the coordinating body of M&E in the province
The DLGTA not viewing the work produced by the municipal M&E unit as central to the attainment of Departmental goals
Unidentified roles and responsibilities of M&E unit members
Lack of internal M&E coordination
Lack of DLGTA senior management involvement in M&E processes
Lack of adequate tools of trade
2. Indicators
Producing accurate and concise quarterly, midyear and annual municipal performance information reports
Increase evaluation studies to assess and verify municipal performance information
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
66
3. Gathering Baseline Data
Commissioned research studies
Desktop research
Evaluation studies
Municipal Reports analysis
4. Setting Realistic Targets
The DLGTA plans to address the above set indicators by producing the
following documents in a financial year:
Quarter 1
(Oct – Dec)
1 Concise
quarterly
performance
information
report
1 report
engagement
and discussion
meeting
Quarter 2
(Jan – March)
1 Concise
Midyear
performance
information
report
1 Evaluation
study
1 report
engagement
and
discussion
meeting
Quarter 3
(April – Jun)
1 Concise
quarterly
performance
information
report
1 report
engagement
and discussion
meeting
Quarter 4
(Jul – Sep)
1 Concise
Annual
performance
report
1 Evaluation
study
1 report engagement and discussion meeting
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
67
5. Monitoring for results:
Data Collection Evaluation Roles and Responsibilities of
M&E Unit Members
The compilation of
the above stated
reports will rely on
primary and
secondary methods
of data collection
Data collection shall
occur every quarter
for municipal
quarterly reports,
mid-yearly for the
The purpose of the
evaluation studies is to play
a central role in the
verification of the
information contained in the
collected municipal reports
The significance is to ensure
that the reported municipal
information is reliable and
accurate
The essence is to ensure
that upon compiling our
reports, the information
Administration Officers (AO):
Collection or solicitation of
municipal quarterly, midyear,
annual and all other municipal
information related reports
Assistant Directors: Analysing of
the information contained
(Findings) in the municipal
reports and compiling the
targeted quarterly, midyear and
annual reports.
Deputy Directors: Inputting and
finalising the draft report to a
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
68
midyear report and
annually for the
annual report
used is reliable and accurate
Evaluations will also serve as
measures to determine the
relevance, efficiency,
effectiveness, impact and
sustainability of a programs
or projects
final one, drawing up the terms
of reference or proposal for the
evaluation studies
Director: Approval of the final
draft report, prepare the report
to be disseminated to internal
and external role players, drive
the engagement meetings or
process with internal or related
stakeholders to interrogate the
report (e.g. senior management
forums), input on the proposal
for the evaluation studies and
initialise that process
Lead the orientation on the
M&E system by ensuring that
firstly the M&E unit is well
capacitated and also ensuring
that staff members of the unit
are familiar and have the
necessary skills on how to
implement a municipal M&E
system
This will be done to effectively
use the system to assist
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
69
municipalities with their M&E
functions as mandated
Chief Director: Sign off the final
draft and oversee the following;
internal and external
dissemination of the report,
senior management
engagement process/ meetings
and presentation of the reports
to senior management forums
Oversee the process of
ensuring that the M&E unit is
well capacitated and also
ensuring that staff members of
the unit are familiar and have
the necessary skills on how to
implement a municipal M&E
system
6. Evaluation information to Support Decision Making
The significance of the performance evaluation studies is that they will play a central role in the verification of the information contained
in the collected municipal reports
Evaluations will also serve as measures to determine the relevance, efficiency, effectiveness, impact and sustainability of a programs or
projects
The findings of the evaluation studies will also determine whether the information used by municipalities in their quarterly, midyear and
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
70
annual reports is accurate and reliable
After the information verification process from evaluation studies, the purpose is to ensure that upon compiling the unit reports, the
information used is reliable and accurate
The reliable and accurate information produced from the unit reports will largely influence the decision making process in senior
management meetings/forums
The support and intervention measures to be proposed in senior management meetings/forums after going through M&E unit reports
will largely impact on municipalities improving on their mandate across the 5 KPAs.
7. Analysing and Reporting Findings
Findings will be reported or communicated in the following formats and timelines:
Quarter 1 (End of)
1 Concise quarterly
municipal performance
information report
Quarter 2 (End of)
1 Concise Midyear
municipal
performance
information report
Quarter 3 (End of)
1 Concise quarterly
municipal performance
information report
Quarter 4 (End of)
1 Concise municipal annual performance
report
8. Using the Findings
The reliable and accurate information produced from M&E unit reports will largely influence the decision making process in senior
management meetings/forums
The support and intervention measures to be proposed by the DLTA senior management need to be largely informed by M&E unit
reports
Annexure A: Department of Local Government and Traditional Affairs (DLGTA) Municipal M&E System
71
Table 6: Proposed DLGTA Municipal M&E System
The support and intervention measures will also largely impact on municipalities improving on their mandate to deliver services across
the 5 KPAs.
9. Sustaining the M&E System It is worth noting that the above system will not sustain itself, however for the system to be sustainable and fully functional, the following
have to accompany this system:
Inputs:
Human resource capacitation (personnel and skills)
Abiding by the roles and responsibilities
The DLGTA taking a stand as the coordinating body of M&E in the province
Internal M&E coordination
Senior management involvement and support in the municipal M&E processes
Adequate tools of trade to conduct M&E functions
10. Outcomes
Effective and efficient monitoring of municipal performance across the 5 KPAs
Effective evaluation of municipal performance information across the 5 KPAs
72