European Consortium for Political Research – General conference 2013

Bordeaux, 4-7 September

What does Transnationalisation do with Sovereignty and the State? Normative

Implications and Reconfigurations of Politics (Section 31, Panel 412)

“The ‘Internationalization’ of the State in Europe: the Experience of the SIGMA

Programme

(Support for Improvement in Governance and Management, OECD-EU)”

Magdaléna HADJIISKY

Senior Lecturer at the Université de Strasbourg – Institut d’Etudes Politiques

Member of SAGE (Sociétés, Acteurs et Gouvernement en Europe), UMR-CNRS n°7363

Introduction

Since the end of the Second World War, and particularly since 1989 and the end of the

Europe’s division into two blocs, International Organizations (IOs) have played an

increasingly visible role in international public policy transfer processes. Today, these

transfers are no longer limited to the economic and financial fields or to cross-border sectoral

policies whose international dimension meets a manifest functional requirement of

interdependence, such as transport and environmental protection. International policy transfer

and learning processes now also apply to government expertise and practices that used to fall

under state sovereignty (Cassese, Wright, 1996; Nolan, 2001; Ougaard, Higgott, 2002; Bézès,

2007; Pal, 2010; King, Le Galès, 2011). Though recent, the “internationalization” of state

reform is a historically momentous development. It challenges national governments to deal

with external evaluations of their very organization and management. But it also challenges

multilateral actors to evidence legitimate models in an area that had once contributed to

defining state power. This paper analyzes this phenomenon from the IOs point of view. Based

on the case of the OECD (Organisation for Economic Co-operation and Development), it

aims at exploring the strategies of legitimization of external expertise on state administration,

and at identifying some of the consequences of this evolution on the IO’s activities.

IOs are frequently called upon by political and economic actors to produce and diffuse

informed and “objective” analysis on public administration (expertise), and in some cases to

help national actors in the implementation of administrative reform (assistance). The OECD

has been a key actor in the internationalization of expertise on state reform (Mahon, McBride,

2008; Pal, 2012), first through its abundant written production, but also due to its central role

in the dissemination of expertise and the building of new networks and expert organizations

(Porter, Webb, 2008).

Since the late 1990s, much scholarly literature has acutely raised the question of the

contextualization of international transfer processes. A growing number of studies have

criticized what they see as the homogenizing approach of IOs, accused of looking at

international transfers as the mere reproduction of ‘one-size-fits-all models’. They point out,

for instance, the lack of compatibility between the institutional and administrative spheres and

the logic that prevails in technological transfers. In an analysis of the regulatory impact

assessments’ (RIA)1 internal diffusion, Radaelli argues that “the ‘one-size fits all’ approach to

reform seems to ignore that RIA is not about products traded in a market. The definition of

success is easier in the case of a product; where administrative processes are concerned,

however, the very definition of the “success” of a reform may vary depending on the context

(Radaelli, 2004, p.729). The distinctive feature of this approach lies in the effort to replace the

states concerned by these international transfers at the heart of the analysis by attempting to

assess the effects of reforms inspired by external influences at national level. The aim is to

understand “how context matters” (Radaelli), and to evidence the “translations” and

“interpretations” resulting from transfer processes in different national contexts (Hood,

1998a).2 These studies point out that, as they adopt a global approach, the actors of

international transfer often miss their target: their method generally results in “diffusion

without convergence” (Radaelli, 2006, p.165). In one of his papers, Radaelli uses an extended

metaphor to show that, depending on national contexts, the “new bottle” of RIA, which

“everyone wants to have […] at home” (ibid, p.167) “may contain either old or new wine, or,

in case of symbolic politics, even no wine at all” (p.163).3

In most studies, IOs are generally reported to adopt a hegemonic and prescriptive

stance. The OECD, in particular, is singled out as one of the main international actors in the

diffusion of “one best way”-style models in the field of public administration. Refuting the

idea of an administrative convergence following a universal reform movement (the OECD’s

“reform story”), Swedish political scientist Rune Premfors prefers to speak of “reform

trajectories”, largely determined by the historically constructed relations between state and

society (Premfors, 1998). Most authors emphasize the role played by the OECD in the 1990s,

1 RIAs are NPM tools initially launched by the Thatcher and Reagan administrations to reduce the “burden” of

legislation on economic activity. Systematized and formalized as checklists by the OECD, RIAs have also spread

in the EU through the Lisbon process (Radaelli, 2004, 2006). For more information on the EU actors involved in

this process: European Commission (2004) “Who is doing what on better regulation at EU level – organization

charts”, Commission Working Document compiled by the Secretariat General TFFU-2 (Task Force Future of

Union), Brussels, 1 June. 2 This is for instance the case of Hood’s comparative analysis of the British and Dutch reforms of contracts for

top civil servants. Though these reforms initially shared the same impetus – contractualization – they yielded

contrasting outcomes due to the transformations undergone by the initial model in the acclimatization process

(Hood, 1998). 3 Following a comparative analysis of the introduction of RIAs in various European states, Radaelli goes on to

elaborate: “the language of RIA has produced a community of discourse for policy-makers and has stimulated

the introduction of some instruments which are labelled ‘impact assessment’ but which in some cases exist only

on paper, and in other cases disguise different practice under the same label” (Radaelli, 2006, p.167)

through the PUMA (Public Management Committee, now GOV), in the rise of an

international “community of discourse” on public management reform. The PUMA is

described as one of the “nodal points” in the creation of this community, emphasizing “good

governance” as “crucial for long-term economic, social and environmental development”

(Pollitt, Bouckaert, 2000; see also Hood, 1998b). Some scholars also hold it responsible for a

“de-contextualized benchmarking”: “OECD precepts suggest (…) that the same elements will

deliver success independently of the context. Benchmarking exercises that take place at the

OECD (…) do not provide a real forum for learning from different, context-sensitive national

experiences” (Radaelli, 2004, p.728). Initially an arena for discussion between states, the

OECD – and here PUMA in particular – is described as an agent of international

standardization in matters of public administration (Sahlin-Andersson, 2000). Whether it is

seen in a positive or negative light, the OECD has generally been described as one of the

driving forces behind the emergence of a universalizing global discourse on state reform since

the 1990s.

This paper is based on an empirical analysis of the OECD’s SIGMA programme,

launched within the framework of GOV (formerly PUMA).4 Since its inception in 1992, it has

been one of the few public international bodies with an explicit mandate to assist central state

administrations in the reform process. The initiative, whose acronym is quite similar to that of

its hierarchic superior (SIGMA stands for “Support for improvement in governance and

management”), is more geographically specialized: it only works with candidate countries and

EU neighborhood countries. In light of the above, we may wonder if SIGMA can truly be

described as the “implementing arm” of PUMA and subsequently of GOV, the agency that

transfers the models elaborated by the latter in the countries concerned. Despite what we

might have expected, this study shows that SIGMA has develop an administrative assistance

method characterized by the ambition to adjust to national contexts as much as possible. As

many international actors are currently rediscovering the virtues of contextualization, SIGMA

appears to be a case apart, since they have tried to develop a contextualized approach to

international transfers for a long time.5

Leaving aside preconceptions on the purported hegemonic action of the OECD, this

paper retraces the genesis and development of the OECD’s involvement in assistance in the

field of public administration. How has transnational expertise gained legitimacy on an area

such as central public administration? At this point, there has been little “insider” sociological

4 The empirical data collected in this still ongoing study is drawn from observation, interviews and archive

research, conducted in Paris (OECD headquarters) and Brussels (EC). I have participated in formal events

organized by OCDE-GOV (including meetings of the Public Governance Committee) and in similar but less

formal events. I made several visits to the OECD archives to view documents from the 1970s to the 1990s not

available for consultation online. I have conducted several sets of interviews with current and former OECD-

SIGMA advisors (12 individuals, sometimes interviewed on several occasions), with OECD-GOV advisors (5)

national representatives of the Public Governance Committee (3), former PUMA advisors and national

representatives (5) and, in the European Commission, agents in charge of relationships with SIGMA and with

the new EU Member States at DG Enlargement and DG DEVCO (7 interviews). 5 The reception of this method by the national governments concerned will be the subject of a complementary

research. This paper focuses on the sociology of the construction of an international transfer method and makes

no claim to assess its impact.

research on International Organizations6. This blind spot is a factor in the confusion between

audience and influence and in mistaken identifications of sources of international transfers.7

The role of IOs in international transfers cannot be boiled down to the action of a unified

“global elite”; rather, it gains from being analyzed as a collective construct involving actors

whose conceptions and interests vary. IOs are indeed complex spaces where partially

contradicting rationales may coexist.8 That is the reason why this study focuses on the very

actors of the international side of the transfer processes, on the way they perceive and practice

their activities.

Having initially described SIGMA’s assistance methods, I will then retrace the

conditions of their emergence and legitimization. This case study exemplifies the way in

which approaches emphasizing sensitivity to national contexts are understood and sometimes

valued within international arenas. This will also be an opportunity to shed light on the range

of constraints that make national contextualization often difficult and in some cases

conflictual in practice. Using the example the SIGMA, this paper examines the origins and

effects of the “contextualization” of international policy transfers in the multi-organizational

European framework. It shows how, under certain circumstances, IOs administrations don’t

act only as diplomatic extensions of their member states, but become key players in the trans-

nationalization of national policies.

1/ Context-driven international assistance

As explained in one of its Implementation Reports, SIGMA was created by the signing

of a Convention between the OECD and the European Commission (EC) on 21 May 1992.

Bulgaria, the Czech Republic, Hungary, Poland, Romania and the Slovak Republic were the

recipient countries; Albania, Estonia, Latvia, Lithuania, and Slovenia joined in 1994

(OCDE/GD(95)120, p.5). Since 2006, SIGMA has had a mandate to work not only within the

framework of pre-accession policies, but also in EU neighborhood countries.9

6 Some studies have begun to fill this gap. See in particular, on the OECD and the Public Governance issue: Pal,

2012. More generally on IOs: B. Reinalda, B. Verbeek, eds., 2004; M. Barnett, M. Finnemore, 2004; Gayon,

2009; J. Trondal, M. Marcussen, T. Larsson, F. Veggeland, 2010; O. Nay, 2012. 7 The approach advocated here aims at going beyond the top-down view and accounting for the interactions

between multiple actors in the process. Just as Europeanization should not be seen as merely an EU initiative,

internationalization should not only be studied as the result of the IOs’ action. (In this perspective, on

Europeanization, see Saurugger, Surel, 2006). 8 Olivier Nay and Franck Petiteville suggest considering International Organizations as “concrete systems of

action” in which various categories of actors participate, forming configurations involving constant “social

dynamics interweaving games of cooperation and games of competition, mechanisms of solidarity and of power

relations”. See O. Nay, F. Petiteville, « Elements pour une sociologie du changement dans les organisations

internationales », Critique Internationale, 2011/4, n°53, p.16. 9 Due to this new development, SIGMA has considerably extended its scope of action. As of 2012, SIGMA

supported the public governance reform efforts of the following countries: EU Candidates and Potential

Candidates – Albania, Bosnia and Herzegovina, Croatia, the former Yugoslav Republic of Macedonia, Kosovo,

Montenegro, Serbia and Turkey; European neighbors – Algeria, Armenia, Azerbaijan, Egypt, Georgia, Jordan,

Lebanon, Moldova, Morocco, Tunisia and Ukraine.

The geographical specialization of SIGMA is to be related to its specific nature:

legally, it is a joint initiative of the OECD and the EC, but it is financed mainly by the EC.10

It

was created on the initiative of the EC after the fall of the Berlin Wall, when the EU’s

PHARE programme assisted Central European countries in the political and economic

transition process. Rather than undertaking the difficult task of creating a specialized

department from scratch within the EU institutions, the executives of DG1A settled for an

alliance with the OECD and its young department PUMA (created in 1989-1990). Thus, this

particular programme results from an original agreement between the OECD and the EC,

which is renewed every two years.11

SIGMA, whose permanent staff works in the OECD headquarters in Paris, is a

modestly-sized structure employing twenty senior advisers and seventeen project assistants,12

hired by the OECD as international civil servants with open-ended contracts. SIGMA has a

mandate to assist EU candidate countries (and since 2006, neighbor countries) in the reform

and modernization of their central administrations and of the overall functioning of public

administration. The general approach (so-called horizontal) of administration is thus at the

core of SIGMA’s activities. In a 1995 document, SIGMA presents itself as follows: “SIGMA

was created to work on reform of central public administration in Central and Eastern Europe.

More precisely, five work areas were defined at its creation: reform of public institutions;

management of policy-making; expenditure management; human resource management; and

administrative oversight.” (OCDE/GD(95)120). Though they have been refined and widened,

SIGMA’s competence areas have not fundamentally changed since then13

.

In these different areas, SIGMA’s activities take various forms. SIGMA publishes

memorandums and assessment reports on the reform projects and bills of the SIGMA states.

Working documents are also published in the SIGMA Papers collection.14

The bulk of its

activities consists in organizing workshops and training seminars with the representatives of

partner countries engaged in the reform process: national or multinational seminars providing

training or information to civil servants in partner countries; meetings between executives in

recipient countries and former OECD members; formalization of networks of civil servants

from partner countries, etc.

10

Currently, 98% of pre-accession-related missions are funded by the EC (Description of the Action SIGMA, Sub

Programme for strengthening institutional capacity of IPA region beneficiaries. 2012). I do not have any data

concerning neighbor countries, but interviewees suggested that the proportion of EC funding is similar. 11

I collected this information in interviews conducted with SIGMA and EC staff and by consulting the relevant

legal documents: European community contribution agreement with an international organisation, 2012;

Description of the Action SIGMA; Project Fiche. See also Pal, 2012. 12

As of February 2013. In November 2012, SIGMA employed nineteen senior advisers and fifteen project

assistants. In the past few years, SIGMA staff has not grown to the same extent as its geographical scope has

extended. A number of recruitments were made in the 1990s, however, since in 1992 SIGMA only included

twelve employees, advisors and assistants included. 13 In 2012, SIGMA provided assistance in the following areas: “Legal Framework and Civil Service

Management: Civil service and human resources management; Administrative law and justice; Regulatory

policy; Public Finance Management: Public expenditure management; Public internal financial control; External

audit; Public Procurement; Policy-making and Co-ordination: Policy development, co-ordination and

implementation; Strategy and Reform.” (official website, accessed November 2012). 14

These papers can be accessed free of charge at:

http://www.oecd.org/site/SIGMA/publicationsdocuments/SIGMApapers.htm

Securing the participation of “partner” states

In terms of method, SIGMA promotes a demand-driven and contextualized assistance,

which it presents as the only way to ensure the progressive appropriation of the reforms by the

states concerned.

This is nowadays a relatively widespread discourse among IOs, which emphasize the

importance of “ownership” and the necessity to ensure the appropriation of the reforms

engaged under external influence in the countries concerned. The diffusionist approach has

not only been criticized in the academic spheres. Some IOs have made similarly critical

assessments of their own practices and preconceptions; yet these have not necessarily

fundamentally changed. Assessments of numerous NPM-inspired reforms, in particular, have

changed the terms of the debate since the early 2000s. Many national and international reports

now evidence a number of unanticipated effects of the NPM reforms, specifically regarding

the actual final cost of change (studies, direct and indirect costs of reorganization) and the

biases resulting from the unreliability of the performance indicators. The OECD’s Uses and

Abuses of Governance Indicators report, for instance, warns against the lack of transparency

of good governance indicators. Along the same lines, the World Bank has developed what it

calls a “learning process” after having observed the failures of the structural adjustment

policies and officially rejected the universal convergence model that used to underpin its

strategy (Cling, 2011).

In the case of SIGMA, the emphasis placed on the need to secure the active

participation of the states concerned by policy transfers does not result from negative

feedback; on the opposite, this approach was openly asserted as soon as the Programme was

launched. For instance, it is worth noting that during an initial period, representatives of

partner countries from Central and Eastern Europe were responsible for coordinating the

programme, together with EC representatives in the Liaison Group. Indeed, from 1992 to

1996, SIGMA was managed through a Liaison Group of representatives of recipient countries

and funders, in which representatives of CEECs were the majority. This Liaison Group was

“responsible for reviewing needs, advising on programme content and effective management

and reporting on progress” (CCEET/SIGMA/PUMA(92)1, p.3). SIGMA’s first orientation

programme justified the establishment of this group in the following terms: “A main principle

of SIGMA is that it should be a flexible resource for national groups responsible for designing

and carrying out national programmes to transform public institutions.”

(CCEET/SIGMA/PUMA(92)1, p.3).

From the beginning, SIGMA’s approach was not unilateral but openly multilateral in

order to encourage the active participation of Central and European professionals. A

significant number of events organized by SIGMA during its first year were multi-country

thematic workshops in which representatives of several “SIGMA countries” participated.15

15

During the year 1993, SIGMA organized 15 multilateral events on subjects such as “Strengthening

Government Policy-making”, “Allocation and Control of Government Resources”, “Ensuring the Regularity of

Administrative Acts”. Five of those events were organized in Paris (where the headquarters of SIGMA are

Remaining faithful to the peer exchange method promoted within PUMA, SIGMA added: “it

also provides a forum for practitioners in central and eastern European countries (CEECs) to

exchange experiences amongst themselves and with the professional communities linked

through PUMA.”16

Indeed, representatives of CEECs soon became involved as “experts” –

i.e., they were invited to bring their contribution to the workshops: “We increasingly used

SIGMA country practitioners as experts in both multi-country and national work and their

inputs were greatly valued by the recipient countries.” (OCDE/GD(95)120, p.7).

SIGMA mentions this interactive approach in the presentation of the Programme in its

publications: “Throughout its work, SIGMA places a high priority on facilitating co-operation

among governments. This includes providing logistical support to the formation of networks

of public administration practitioners in Central and Eastern Europe, and between these

practitioners and their counterparts in OECD Member countries” (OCDE/GD(96)47 p.2). In

the same perspective, SIGMA contributed in 1993-1994 to the creation of a network specific

to CEECs in this field: the Network of Institutes and Schools of Public Administration in

Central and Eastern Europe (NISPAcee).17

At odds with a common preconception on international expertise, SIGMA does not

present itself as having readymade solutions to provide to receptacle states. One of the main

sources of the “strength of the SIGMA programme” is, according to Derry Ormond in his

introductory speech to the Prague seminar in 1993, “the SIGMA approach, which is to

provide mere advice, discussions, comments and deductions based on experience. The

solutions, or the models, are your responsibility”.18

Bob Bonwitt, one of SIGMA’s founders and its director from the beginning to 2011,

also openly embraces this approach. At a conference bringing together practitioners and

analysts of international expertise,19

Bonwitt was quick to dismiss the terms “expert” and

“expertise” to refer to SIGMA’s activities: “It’s a type of action that I reject. The only experts

on the Czech civil service are the Czech, not us!” In our face-to-face interview, this theme

came up again spontaneously. Following a question on co-financing: “do countries sometimes

contribute to the financing of your actions?” Bob Bonwitt laughed and replied: “My answer is

yes, they provide the majority of the funds. Traditionally, co-funding is usually 50/50, it

always is. This is a way to understand if the government [of the partner country] participates.

We have a different vision. If a country asks us for help with administrative procedures, our

way of working with them consists in first asking them to put together a team with national

located); the others took place in various other European capitals. SIGMA 1993 Implementation Report, OCDE,

Paris, 1994 (OCDE/GD (94)48), p.40 16

The last two excerpts are taken from OCDE/GD(95)120, 1994 SIGMA Implementation Report, Paris, OECD,

1995, p.5 17

SIGMA’s support to the creation of the NISPACee is for instance mentioned in SIGMA 1994 Implementation

Report (OCDE/GD(95)120). 18

Notes prepared by S. Synnerstrom for Derry Ormond’s introductory speech to the Prague Seminar on the

Implementation of civil service laws for a reformed human resources management, 29-30.06.1993. Dated

22.06.1993, p.3 of the speech 19

Conference “Les réformes de l’Etat en Europe centrale et orientale : regards croisés sur l’expertise

international”, Maison Interuniversitaire des Sciences de l’Homme d’Alsace, Strasbourg, organized jointly with

Valérie Lozac’h, GSPE-PRISME, 23 June 2011.

experts so that we can then assist them: we’ll advise them on law-making, on the acceptation

of the law by the government and by the West and on implementation. Our role is minimal; we

can bring in a few days’ worth of work by our advisors. So the partner country is in charge of

the work and of the cost. (…) As far as I’m concerned, the government is the main source of

financing [indirectly, through the salaries of civil servants, the conduct of some studies and

preliminary consultations, etc.]”20

Bob Bonwitt sees the attention to the demands by governments of “partner” countries

(this telling phrase is used by SIGMA advisors to refer to the countries in which they work) as

an important defining feature of SIGMA, of which he is proud: “SIGMA’s culture is about

helping and trying to give answers. We like our countries, our interlocutors”. Demands may

be partially reformulated by SIGMA advisors to better fit the organization’s propositions, but

ideally, the starting point remains the expression of a need by representatives of the partner

country.

Bonwitt claims “radical modesty” is SIGMA’s “motto”.21

Is this merely a public

posture? Does this emphasis on humility essentially serve to secure the participation of

member states, which is indispensable in an inter-state organization such as the OECD? These

strategic dimensions should admittedly not be overlooked. Yet, I argue that this

contextualized approach is derived from structural elements that relate to SIGMA’s

functioning and history

The effects of a special status

First, some effects of SIGMA’s special status deserve emphasis. SIGMA is a specific

programme focused on an issue deemed as high priority (the administrative transition initially,

and then after 1997, the “administrative capacity” required for EU accession) that only targets

a small number of states. At the time of its creation, some of SIGMA’s first activities

consisted in providing advisors with specialized knowledge on “SIGMA countries”: visits to

these countries were organized between November 1991 and February 1992, exchanges took

place in the Liaison Group, and the first SIGMA publications were intended to assess the

legacy of the Communist period and of the administrative reforms undertaken in various

CEECs.22

On a very practical level, this geographical specialization gave SIGMA advisors more

leeway than their counterparts in generalist OECD departments usually have. They can for

instance organize workshops of varying sizes, changing the list of guest states according to

20 Interview with Bob Bonwitt, July 2011. In order to maintain the discretion that befits this type of study, I have

preserved the anonymity of the interviewees who discussed contemporary data. They are however named when

discussing historical or more general data. 21

Talk given by Bob Bonwitt, then director of SIGMA, at the Conference “Les réformes de l’Etat en Europe

centrale et orientale : regards croisés sur l’expertise international”, op.cit., 23 June 2011. 22

See the two sets of Public Management Profiles published in 1993 and 1995 for each of the SIGMA partner

countries.

the issues under discussion, which enables them to hold work seminars that are considered as

more productive than the usual large-scale events. An OECD department that constitutes a

“committee” cannot do this as easily, since the invitation of all member states is required to

all events.

SIGMA’s special status gives partner states a significant importance and margin of

choice. Until 1997-1998, any SIGMA action depended directly on the demands of partner

governments, since assistance was exclusively demand-driven. A negotiation could refine the

content of the collaboration, but the beneficiary government retained the initiative. Today, this

initial procedure remains the most frequently used,23

even if the European Commission now

steers activities within the framework of the pre-accession process (since 1997) and of the

neighborhood policy (since 2006).24

Partner countries are considered as the main “beneficiaries” of the programme (this

term is used to refer to them in legal documents on the agreement between the OECD and the

EC). As such, they are involved in the evaluation of SIGMA’s activities. The Programme is

evaluated every two years on average by an independent company or by the EC.25

Feedback

from partner countries that have worked with SIGMA advisors is systematically used as a

source, generally in the form of satisfaction surveys.26

International contextualization in practice

SIGMA’s relative dependence on the expectations of “partner” countries is reflected in

the orientations of its main methods of action.

First, attention to the national context in partner countries partly structures the

activities of SIGMA advisors. Each adviser becomes specialized not only in one of the

23

The Interim evaluation of SIGMA Programme provides elements to assess the proportion of missions initiated

by beneficiary countries, as it includes responses to the questionnaires sent to these countries. The evaluation

concerns 85% of all eligible projects according to the methodological section of the evaluation report. Among

the missions carried out by SIGMA between 2009 and 2011, 89% are reported to have been directly requested by

beneficiary countries, against only 11% of “project definition sheets”. At the time being, we cannot however

corroborate these numbers with other sources, since current SIGMA archives are not available. Interim

Evaluation of SIGMA Programme, September 28th 2012. 24

For instance, the EC can ask SIGMA to evaluate bills on public administration, independently of the country

concerned. 25

Information on the frequency and on the authors of the evaluations was provided by SIGMA advisors during

interviews. It was confirmed informally at the OECD archives (however, I was not allowed to consult the

document that reviews SIGMA evaluations and audits). 26

Attention to feedback from partner countries in evaluations was discussed in interviews, including with former

director Bob Bonwitt and current director Karen Hill. Depending on the case, feedback is given as responses to

anonymous questionnaires or in interviews conducted by the organization in charge of the evaluation. The 1995

Implementation Report, for instance, indicates that “the Warsaw-based Policy Education Centre on Assistance to

Transition (PECAT) was chosen to undertake an independent monitoring and evaluation of the Programme. The

evaluation [which proved positive], used four sources of data: SIGMA documentation and interviews with

SIGMA staff; data from a statistically random sample drawn of SIGMA’s universe of actions; data from over

100 on-site interviews conducted in the partner countries; and data from questionnaires directed to persons who

participated in SIGMA actions.” OCDE/GD(96)47, p.42. More recently, the Interim Evaluation of SIGMA

Programme (September 28th 2012) also used this source, in the form of responses to questionnaires by

representatives of SIGMA beneficiary countries (39 respondents, accounting for 85% of eligible projects).

Programme’s activity areas, but also in the collaboration with one or several partner countries,

of which he or she is in “charge” for a few years. In 2011, for instance, one of the senior

advisers was specialized in “policy-making and coordination” and also in charge of the

“country desk Albania”, which means not only that he chiefly operationalized his specific

competency in Albania, but also that he had to ensure the coherence of SIGMA activities

within the framework of the collaboration with this country. As of 2013, the same adviser was

still in charge of Albania.27

Team members are thus supposed to constantly maintain working

relations with the country or countries in question. The explicit goal is to build trust in

interactions with the governments and civil servants of partner countries – and of course, to

expand the advisors’ list of contacts.

In addition to the permanent staff, external experts are also hired for more or less short

stints in the Programme. They are generally either academics or higher civil servants, or

former academics or civil servants who have become private experts. These external experts

are the most frequent contributors to the working documents published since 1995 on

SIGMA’s website, the SIGMA Papers. They assist SIGMA advisors in their activities in the

field (the latter never go alone). They are also called to contribute in the workshops organized

by SIGMA for the benefit of partner countries. SIGMA advisors generally ask them to do

more than just provide a reference model; they are to present a variety of examples that makes

up a sample of possible solutions. Here are the recommendations sent by a SIGMA advisor to

the guest experts before a multilateral workshop in 1993:

“Objectives: 1.Create understanding of that law is only a base for reform; it does

not change anything by itself. 2. Create awareness of the various issues that should

be addressed in the implementation of a law and in the institutionalization of the

Human Resource Management. 3. Expose possible methods/techniques both for

establishment and for continuance of the HRM systems for understanding and

assessment by the SIGMA countries. 4. Establish a bridge to the future SIGMA

actions on the national level.

Standards for the various expert contributions:

1. If possible, give reference to different solutions from OECD.

2. Make clear the importance of objectives and the linkage between

objectives and chosen solutions.

3. Concentrate more on flows, interactions and processes than organizational

structures.

4. Make it possible for the participants to understand why things are done in a

certain way rather than how it is done in detail.

5. Make it possible for the participants to make conclusions for their own

context rather than give them models for copying.”28

As this example shows, the recommendations sent to external experts during this

founding period aimed at avoiding contributions based on a ready-made, detailed model,

instead preferring those proposing a range of possibilities. They also emphasized

27

SIGMA’s official website includes a “SIGMA staff” page, which is exhaustive and regularly updated. The

information provided in this instance was collected on the website in July 2011. 28 Fax sent by Staffan Synnerström on 7 April 1993 during the preparation of the seminar on implementation of

Civil Service Laws held in Prague on 29-30th June 1993.

understanding of processes and of reform implementation rather than of legislative texts. This

concern was frequently expressed by SIGMA executives: legislative texts are considered as

useful but clearly insufficient to ensure actual change in practice.

There is however an important hindrance to the contextualization logic: the advisors

are generally unfamiliar with the language of the countries they are entrusted with.

Interactions therefore require translation into English of legislative texts and contributions in

workshops and meetings. This not only creates risks of misunderstandings, but also imposes

limitations on the type of actor liable to collaborate with SIGMA in partner countries

(English-speaking individuals are generally favored). SIGMA advisors also do not have the

time to collect information on the national history and the recent political developments of the

relevant countries (nor do they necessarily see it as useful). Their sources of information are

essentially internal: selected SIGMA papers on one or several countries,29

the Country profiles

of 1993 and 1995, and the Assessment Reports published since 1999. They then refine their

views on the local context through interaction with their contacts in partner countries. We

may thus consider that the approach at play here aims not so much at understanding these

countries from within, but at establishing relationships of trust with local reformers. SIGMA

actually does not particularly consider the individuals’ knowledge of the partner countries in

the recruitment process.

SIGMA’s recruitment policy ties in with the ambition to build a relationship of trust

with representatives of partner countries in a different way. Indeed, SIGMA recruits as many

applicants who have accumulated significant experience of the administration in their home

country as possible. Based on the peer exchange approach, the assumption here is that

SIGMA advisors will have work more easily and efficiently if they have already “got their

hands dirty” (to use a phrase that often comes up in interviews with SIGMA advisors), if they

share experience of and in civil service with their interlocutors. Thus, traditional criteria for

recruitment in IOs (language skills, multinational background or ability to do away with

political and cultural preconceptions from one country’s of origin) are complemented with a

more distinctive criterion relating to the thematic and technical specialization of the

Programme. Former director Bob Bonwitt claims this is meant to ensure a “concrete

approach of administrative activity, which does not remain theoretical. Whenever possible,

we have chosen to hire ‘seniors’, people who almost have their career behind them, old

hands” (interview with Bob Bonwitt, July 2011).

This approach comes with a defiance toward “theoretical models” imposed from the

outside, with no relation to the national contexts. Some interviewees were even somewhat

jubilant as they discussed cases where “experts” made missteps and had to “eat humble pie”

because their peremptory general recommendations turned out to be ill adjusted and counter-

29

For instance: Civil service professionalisation in the Western Balkans (Meyer-Sahling, J. (2012), "Civil

Service Professionalisation in the Western Balkans", SIGMA Papers, No. 48, OECD Publishing) or Country

Profiles of Civil Service Training Systems (OECD (1997), SIGMA Papers, No. 12, OECD Publishing.).

productive.30

SIGMA advisors are similarly defiant toward NPM (New Public Management),

even though the latter partly served as a basis for numerous PUMA activities in the 1990s.

Finally, it is worth noting that advisors assign great value to fieldwork. They name

field missions and interactions with the governments of partner countries as their preferred

activities in their everyday work (administrative tasks at the desk are the least prized; the

drafting of evaluation reports or SIGMA papers are considered as interesting activities, but to

a lesser extent). It is in the field that they believe they are making an undeniable difference

compared to others (they judge consulting firms very harshly, for instance). It is also during

these missions that the job is at its most stimulating and leaves indelible memories – some of

the senior SIGMA advisors I spoke to talked about wanting to “write a book” about these

experiences.31

Advisors use the term “assistance” more frequently than “expertise” to refer to their

activities. Though seemingly anodyne, this distinction is an important one. In interviews, it

appears that the term “assistance” is preferred because it relates to the advisors’ eagerness to

take into account the actual expectations and needs of the countries, always referred to as

“partners”.

The rejection of the term “expertise” has to do with the line frequently drawn between

SIGMA and private consulting firms. In the words of Derry Ormond: “you may hire any

number of persons who’ve been experts in Bangladesh, in Somalia, consultants for consulting

firms who were out there to sell their products. They knew exactly what [model] they had to

give. But that doesn’t work.”.32

Here it is worth recalling the distinction between demand-

driven assistance and supply-driven assistance, which provides a pre-constructed service or

product. The countries mentioned in this excerpt are significant: the interviewee meant to

highlight the contextual distance between them and OECD countries. This critique of the role

played by international experts relates to the view of the post-1989 period as a boon for

development aid experts, hastily converted to transition assistance in East-Central Europe. It

manifests the existence of an implicit hierarchy for OECD advisors, enabling them to distance

themselves from such practices and present themselves as advocates of an assistance that does

not target profit but seeks to meet the needs of the countries concerned.

30

Quotes from an interview with a SIGMA advisor, December 2011. 31 Through my interviews, I noticed that the hierarchy which prevails here differs from that of other OECD

departments. For comparison purposes, I have conducted a dozen interviews in the other departments that make

up GOV. Discourses are quite contrasted. The attachment to the OECD “brand” is more immediate and

spontaneous (requiring no incentive in the questions asked) and the interviewees refer to themselves more

readily as experts or analysts. They see preparation and elaboration of OECD reports as the most stimulating part

of their work. When, as has been the case since Spring 2012, the Secretariat-General asks directorates to improve

reform assistance in member states, the agents often perceive it as a denial of the distinctiveness of the OECD’s

work. More prosaically, they raise the issue of human resources that such a policy will entail for the entire

organization if it is to be implemented. 32

Interview with Derry Ormond, November 2012.

2/ The emergence of an assistance programme in an expert organization

How are we to understand the emergence of these international assistance methods

within the OECD, an organization that has built its legitimacy in the field of expertise by

collecting and analyzing data rather than by helping on the field? SIGMA’s method, indeed, is

an atypical one in comparison to the quantitative, model-based approaches that are practiced

by most OECD directorates.

More broadly, the concern for contextualization questions the very foundations of the

IOs contemporary practices. The international circulation of the same norms and practices is

frequently presented and perceived as one of the IOs’ main tools to construct a unified world,

and therefore (it is believed) a more peaceful one. This is particularly true in specialized IOs,

which work in limited geographic areas or have an economic vocation. More than universal

and generalist IOs such as the United Nations, these specialized IOs can take liberties with the

inter-state approach: their mission is not formulated in mainly diplomatic terms, but in global

economic and social terms. Leslie A. Pal presents a very documented analysis of the

underlying tension in the functioning of the OECD as a “hybrid organization” characterized

by a “combination of seniority and diplomacy” and by the coexistence of a diplomatic

rationale and of a think tank approach, between inter-state organization and international

transfer agent (Pal, 2012, p. 23).

In this context, the use of international comparison for appraisal purposes has become

widespread, to the extent that it has been of the IOs’ main tools in the contemporary period.

According to this logic, the internationalization of administrative principles and instruments

requires them to be presentable as rationally elaborated products, and accordingly universally

exportable ones – not socio-historical constructs proper to each country. Yet, the

contextualization agenda questions this assumption of the interchangeability between the

spaces being compared (states, regions, cities, institutions, organizations). The construction of

a shared “international language” is the result of a social construction based on a number of

artifacts and presuppositions enabling comparability between the spaces. How is it possible to

reconcile contextualization and comparability, to account for the unicity of a historically

constructed case without giving up the comparative approach? These long-time

methodological concerns in comparative politics are raised very acutely for these international

actors. How can they combine the demand for national contextualization and evaluation-

oriented, if not prescriptive international comparison?

The OECD’s mark

The main reason for the development of a method combining comparison and

contextualization within SIGMA is arguably a historical or more precisely a genetic one. A

number of elements in this study suggest the following hypothesis: the inclusive, non-

hierarchical international approach defended by SIGMA was strongly influenced by the

context in which it was elaborated, characterized by the “vital” need to convince OECD

member states. Despite what the EC’s investment in the creation of SIGMA may suggest, and

despite PUMA’s current widespread image, the OECD’s mark can clearly be pinpointed in

the early stages of SIGMA.

In order to understand this, we need to remember the ties of filiation between SIGMA

and PUMA. The contrast that now exists between the generalist machinery that GOV has

become33

and the modest specialized department that SIGMA has remained should not

mislead us. There has been a clear continuity in terms of structure and staff between PUMA

and SIGMA, which were created in an interval of three years only (1989 for PUMA, 1992 for

SIGMA). One of the creators of PUMA (before he went on to initiate SIGMA) was no other

than Bob Bonwitt. Derry Ormond, who was the director of PUMA from 1989 to 1998, was

also involved in the creation of SIGMA and later supported its evolution. Francis Hénin, one

of SIGMA’s current advisors, was also one of its founders after having worked for PUMA.

The structure behind both of these departments was actually the Technical Cooperation

Committee (TECO), set up by OECD members in the 1960s to assist Southern European

countries in economic development.34

The origins of PUMA and SIGMA thus date back to

TECO. Arguably, SIGMA’s approach to assistance was for the most part elaborated during

the long and difficult gestation of PUMA by the TECO executives starting in the mid 1970s.

TECO (created in 1966 within the Development Division) was a technical assistance

programme specifically aimed at Southern European countries: Spain, Portugal, Greece,

Turkey, and less frequently Yougoslavia. Though TECO initially had economic and technical

objectives, its leaders developed an interest in the institutional aspects of development in

Southern countries. In the mid 1970s, the rapid transformations in the political and

institutional context of these countries raised a new challenge for TECO advisors: assisting

countries in the process of political transition.35

These countries (with the exception of

Yugoslavia) also quickly entered the EEC accession process. The TECO participated in this

process, with a very “cooperative” approach according to Bob Bonwitt, as they were OECD

and NATO member states. This political normalization of the TECO countries helped giving

new legitimacy to a department that was until then merely tolerated and seen as a means to

assist dictatorships in avoiding the Communist influence.

TECO executives appear to have taken advantage of this change of context to draw

consequences from what they saw as the cause of their dissatisfaction with TECO activities.

In interviews with the advisors working at the time, the same image frequently comes up: that

33

The GOV Directorate has a staff of 120 advisors (according to the GOV directory, last consulted in November

2012) working in eight departments, including SIGMA. Each Directorate in the OECD supports one or a number

of committees. GOV supports the Public Governance Committee, the Territorial Development Policy

Committee, and the Regulatory Policy Committee. 34 This transpires from the accounts of the main actors at that time and from their professional trajectories. Before

he obtained the creation of PUMA, Derry Ormond was the director of TECO (since 1973). Bob Bonwitt, who

was hired in 1976, was Derry Ormond’s deputy director at TECO, and then PUMA, before he founded and

directed SIGMA. Francis Hénin was also a TECO recruit, as was one of the founders of PUMA and SIGMA, the

late Ravi Kapil. See Pal, 2012, chapter 2. 35 This was the time of the end of the Greek Military Junta (1974), of the Carnation Revolution in Portugal

(1974) and the early stages of the Spanish democratic transition.

of the disheartening amount of expert reports stored away on the shelves of Turkish or

Spanish officials.36

It was inferred, first, that “the success of a reform hinges of the degree of

political support for that reform” and, second, that “in order to make progress with a reform,

regardless of the field, an administration that functions well is necessary”. In other words,

“without an adequate and active institutional and administrative framework, no advice,

however valuable, can ever be useful”.37

Dissatisfaction about the experience of pre-1975 TECO also justified the rejection of

what was called at the time the “donor-recipient model” of technical assistance. Finding

assistance methods liable to encourage the countries concerned to engage in the reform

process became a priority.38

In a 1977 document, the relevance of “joint activities” bringing

together several countries is justified in the following terms: “The basic idea behind the joint

activity is that for many of the key problems facing countries, (…) there is no accepted

solutions or models to be imported from abroad. Every country has to identify its own

particular batch of problems, and work out an answer which pays heed to the special

conditions of that country” (OECD, 1977, “The TECO”, OECD Informations). One can easily

recognize here the origins of the multi-country strategy promoted afterwards by SIGMA.

Converting the OECD to public administration

Beyond the Southern countries, TECO leaders had the ambition to create a new

generalist committee working on public administration with all OECD member states. An

internal document from 1975 indicates that the involvement of member states in public

management-oriented bureaucratic reform was already in the works at the time, apparently

under the influence of Ormond’s close advisor Ravi Kapil.39

This would prove an arduous

task, as TECO leaders were touching on an exclusively national competence. There was for a

long time much resistance from a few of the leading OECD states. As Bob Bonwitt recalls:

“the OECD’s main donors didn’t want the creation of a Public Management Committee. The

French, the British, the Americans didn’t want the OECD to interfere in their public

administration.” This powerful resistance may to some extent explain the slow birth of

PUMA, which was only created at the very end of the 1980s. Bonwitt goes on to explain that

“it was a long struggle, from my arrival in 1976 until 1990. But we had internal support from

the other OECD committees (economics, education, environment…) who had problems with

36 This negative image of the reports piling up without being read or acted upon is also found in the archives for

that period. For instance, a brochure on TECO published by the OECD in 1997 insisted on the fact that the

Committee was being reformed to act as an “incentive for national officials to bring about change, rather to

produce reports”. OECD, 1977, “The Technical Co-operation Service”, OECD Information, n°40, December

1977. 37

Excerpts of interviews with SIGMA founders. 38 As the document cited above announces, “in recognition of (…) the more co-operative character of activities,

the name of the Programme has been changed to the Co-operative Action Programme”. OECD, 1977, « The

Technical Co-operation Service », doc. cit. 39

Note entitled “Public Management”, by Derry Ormond and Ravi Kapil, 22 September 1975, c.c. G. Piperno

and A. Grinkow, RK/fma. This document was kindly sent to me by Derry Ormond, November 2012.

the implementation of public policy. So there was a growing interest for the institutions that

carried the sectoral policies recommended by the OECD committees.” Although this last

point would deserve investigation and verification, this interview excerpt gives an idea of the

way in which TECO leaders made their ambition legitimate to the other OECD committees,

by emphasizing the benefits in terms of policy implementation.

Even before PUMA was formally created, however, TECO’s activities changed to the

extent that all the initiatives launched by PUMA at the time of its creation had already been

introduced in TECO. A SIGMA advisor who was present over the entire period sums this up

humorously: when PUMA was created, “we had already stopped handling the cultivation of

carrots and peas, we took care of public administration, ‘joint activities’ between countries’”.

During that decade (1979-1989), the TECO leaders were not in a strategic position to

impose any kind of “model” on partner countries. They were engaged in the dynamic of

constructing state reform as a legitimate activity area for an organization with originally

economic objectives. None of their initiatives could progress without the states’ agreement. A

classic illustration of the inter-state approach, this dependence on the goodwill of member

state representatives explains the adoption of a prudent approach, consisting essentially in

persuading, or even seducing these countries.

My interview with Derry Ormond is a worthwhile illustration of this. Having

described the resistance of some OECD member states to the creation of a Public

Management Committee, he discussed the “‘soft’ elements that were enormously helpful in

bringing about all this change”. He claims the decisive move was “going to these countries,

speaking with [the countries’ representatives to the OECD and the ministers and higher civil

servants in charge of administrative questions], actually listening to what they had to say”.

Ormond emphasizes the methodological dimension of this approach: “I heard some other

OECD colleagues in other areas who went to these countries, and deep down they were just

telling them what they had to do, or what the OECD thought without listening to them. You

had to actually listen to them, since each country is different and also it’s political, we all

know there’s an important political side to administration.”

During the interviews the founders of PUMA recall that one of the most frequent

reaction of the country representatives was to insist on the specificity of their country, in order

to discredit the value of what was experimented elsewhere: « we are different », « we are

unique”, the delegates used to say. One of the key issues of this founding period for the future

founders of PUMA was to make central national administrations comparable, to legitimate

their comparability, without offending national sensibilities. It was therefore indispensable if

not “vital” for TECO leaders to listen to member state representatives, to “understand what

could interest them” (Derry Ormond, November 2012). Francis Hénin also recalls this

uncertain time: “It’s true that, traditionally, ‘public management’ was considered as an

internal matter, it was nobody else’s business. We had to raise awareness of the interest of

delving into these issues together to try and come up with solutions – because the problems

were rather widely shared, but the solutions weren’t, for reasons relating to history, to

administrative culture, etc. We had to convince them.” (Francis Hénin, December 2011).

Beyond the approach focused on listening, a strategy of cooptation and involvement of

higher civil servants and political leaders in member states was also put in place: “So we had

to go to those countries. For instance, for the Madrid meeting (1979),40

we had to involve the

people in charge, the higher civil servants leading the institutions (…). We had to see those

people, to be able to see them and draw them in. We actually used emulation to convince them

[by telling them]: ‘you know, that other guy is coming too’.” This is an example among others

of the seduction strategies developed by the TECO director to capture the interest of higher

civil servants and political leaders.

Succeeding in coopting national leaders was essential because, at that time, the

initiators of the future PUMA committee were attempting to construct their own position as

analysts and advisors through the interaction with member states: “We made significant

progress. First, we realized that bringing the countries together to report on what they did

was not enough. (…) So we moved on from merely exchanging experiences to analyzing

problems, which required that we analyzed problems first, we conducted the analysis and we

presented it. But in order to conduct an analysis, we had to hear out these countries, to go to

these countries, there was discussion in the analysis process. That was the real leap forward

of the 1980.” (Derry Ormond). TECO’s institutional fragility and the politically sensitive

character of the issue thus contributed to shaping an international expertise method with an

emphasis on hearing out national representatives.

It was during that period that the networks of exchange between practitioners

(ministers and civil servants specialized in these issues in member states) were experimented

with for the first time, with the agreement of then Secretary-General Emiel van Lennep. The

“JAPMI” (Joint Activities on Public Management Innovation) were conceived as informal

groupings allowing for a new socialization between ministers and higher civil servants of

OECD member states. The Working Party of Senior Budget Officials, for instance, celebrated

its thirtieth anniversary in 2009.41

Derry Ormond insists: “Practitioners; we had to get

practitioners involved. (…) Because they could speak more clearly and on even footing [with

the partners of member states].” This is how Ormond justifies a recruitment policy that

(already) targeted practitioners: “After a while, we recruited practitioners to come and work

in the PUMA secretariat for two or three years”. To develop its strategy, the team around

Derry Ormond had to quit diplomatic interstate rationale (linked with the accreditation system

and with the principle of interchangeability between the national delegates) to try to generate

trans-governmental and trans-administrative networks based on a form of professional

collegiality.

40

The symposium organized in Madrid in 1979 can be considered as the first visible manifestation of the

investment of the technical cooperation committee in institutional and administrative matters. Entitled

“Managing change in the public administration” (5-9 February 1979), it was aimed at all OECD member states,

as opposed to just the “TECO countries”. See Pal, 2012, p.40-42. 41

For the occasion, a historical report was published: Chronology of SBO meetings and articles in the OECD

journal on budgeting. 30th Annual Meeting of OECD Senior Budget Officials (SBO)

GOV/PGC/SBO/RD(2009)1/REV).

This strategy of attraction focused on national practitioner had a second explanation,

also linked with the institutional fragility of the Ormond’s team. The annual working agenda

of the whole organization is decided by the Council at ministerial level. It is thus crucial for

all the OECD Directorates to be identified as a “priority” by the national officials during the

Council meetings. As one of the founders of the PUMA explained to me: “Without this kind of

‘clientelist relation’, you are dead...”, and, remembering the TECO period, he went on

saying: “Our purpose at that time was to go in the national administrations and pick up the

persons who could afterwards enter into this type of relationship with us”.

At the time of its creation, SIGMA built up on TECO’s experience in field assistance

aimed at a limited number of states facing issues judged as similar. This enabled it to acquire

specialized competences while remaining a relatively mobile and reactive small structure. The

programme also bears the mark of PUMA’s early stages, both due to its particular interest in

public administration and to the quest for a “cooperative” assistance method accounting for

the largely political character of administrative questions.42

SIGMA picked up the logic of an

inter-state “club” in which it is essential that each national representation feels it is on an

equal footing with the others. Indeed, the secretariats of the OECD committees can be quickly

reprimanded if national delegates consider that their initiatives go beyond the scope of the

organization’s prescribed role as facilitator and organizer.

3/ The problems of custom-made international assistance

Did this inclusive and contextualized assistance method, so strongly asserted by the

SIGMA founders, stand the test of time? Did it catch on in the long term despite the costs in

terms of time and staff43

and the difficulty of quantifying and objectifying its results?

Their double institutional affiliation appears to give a degree of leeway to the SIGMA

executives. According to our early investigations in GOV and the EC, it appears that SIGMA

is perceived as a distinct programme that is left with significant room to maneuver in the

name of respecting the specificity of the other mother ship organization.

Additionally, it is considered both in GOV and in the EC that long-term investment at

national level is one of SIGMA’s key assets. As SIGMA’s latest framework document

explains: “The successive SIGMA Programmes have demonstrated efficiency, quality and

effectiveness. Findings of an evaluation of SIGMA amongst beneficiaries provide evidence

that its very specific approach characterised by long-term working relationships and a

continuous partnership with key institutional stakeholders at different hierarchical levels is

very well appreciated. In addition, it is stated that an intense dialogue and exposure to fellow

42 SIGMA’s search for an inclusive method involving national representatives does not mean that this method

was perceived as such by the actors concerned. An ongoing complementary study of OECD member states’

representatives aims at addressing this very question. 43

The need for continuity and frequent presence on the field requires more staff.

peers and practitioners enables intensive learning experiences that have a lasting impact on

stakeholders and lay the foundation for sustainable change.”44

Yet, the methods defended by the SIGMA founders have also been met with

reservations, if not negative reactions. Though overall positive, the regular evaluations of the

Programme call for more tangible results.

This pressure manifested itself early on in the history of SIGMA. In 1993-1994, the

first evaluation of the programme suggested refocusing on national activities and the

production of written outputs. In the words of the 1994 Assessment Report: “We adapted our

work to follow recommendations made in the 1993 Assessment Report. We moved our focus

from ‘awareness raising’ actions to more intensive work on reform policy development and

implementation; produced more written outputs such as comparative papers, checklists,

national studies, and three reference publications for direct use by governments”. Among

others, one of the main features of the SIGMA method, multilateralism, had been called into

question in the 1993 assessment: “SIGMA directly focused more of its inputs on the national

level in comparison with 1993. We organised approximately 250 country-specific actions

(e.g. participation in seminars, panel reviews of draft laws, study visits, advisory visits). On

the multi-country level, we organised eight meetings (for government representatives and for

country experts; about 180 in total) which often stimulated national actions in countries.” The

report nevertheless adds a justification of the multi-country method: “They also proved to be a

cost-efficient approach for exposing SIGMA country practitioners to the range of reform

approaches and administrative systems in OECD and SIGMA countries.” (OCDE/GD(95)120,

p.7).

Beyond this pressure exerted on a regular basis in the assessment reports, the

Programme has experienced more fundamental upheavals, of specific interest within the

framework of this research as they precisely concerned the contextualization of assistance.

In the late 1990s, SIGMA went through a particularly rough patch, which, according

to some interviewees, even threatened the very existence of the Programme. It was at that

time that the EC decided to have SIGMA assist countries not only in the transition, but also in

the pre-accession process. Behind this apparently harmless and even positive development, a

major transformation of SIGMA’s method would occur. SIGMA was indeed asked by the EC

to provide thematic yearly country-specific assessments; the EC would use them as sources to

draft the political and administrative chapters in the Progress Reports of candidate countries.

This new development took place at an already tense juncture. The new director of DG

Enlargement, Catherine Day, had made it known through a number of channels that she was

unhappy with SIGMA’s work in terms of output and pace. As a then DG Enlargement agent

explains (interview in January 2012): “Catherine, like others anyway, though SIGMA was not

‘efficient’ enough: not very fast, not very eager to plan ahead, to set objectives, to make

governments decide on reform options… Catherine could be impatient, particularly on

matters of ‘administrative capacity’”.

44

EC, Sub Programme for strengthening institutional capacity of IPA region beneficiaries, “Description of the

Action SIGMA”, December 2012.

Bob Bonwitt also recalls that “Catherine Day expected these countries (CEEC) to go

faster; there were great expectations because of the hastiness of the enlargement.

Additionally, the EC was facing pressure from member states to mitigate as much as possible

the risks related to the accession of the new Eastern countries in the fields of environment,

chemicals, industry, etc. Whereas at SIGMA we had a systematic view of administration, we’d

tell them ‘it’ll take generations’…” (interview with Bob Bonwitt, Strasbourg, 1/10/2012).

From 1998 on, SIGMA advisors were asked to conduct a yearly assessment of “their”

countries to evaluate their progress on the basis of the enlargement policy criteria.

The SIGMA advisors who were already there at the time express great reluctance

towards this change in interviews. According to Bob Bonwitt: “We used to do peer reviews on

specific subjects – civil service, external audit – on the beneficiaries’ request. With the

assessments, we work directly for the EC and we’re supposed to have a more critical, more

‘photo-objective’ outlook. Whereas peer reviewing is more analytical and constructive, since

the objective is to pinpoint the problem and solve it, it’s forward-looking. Criticizing is not

necessarily the best way of doing things. There’s a conflict of interest of sorts, it’s difficult to

combine the two.” Most of these “senior” advisors still consider that the assessment part of

their activity contradicts their approach to fieldwork.

Within this framework, the European Commission asked SIGMA to produce

“baselines” or minimum institutional requirements for key areas of public management.

Prepared in co-operation with Commission services, these baselines were used as yardsticks

for assessing the institutional situation in the (then) ten central and eastern European

candidate countries. SIGMA delivered assessments in June 1999, and, after having updated

them, published a Report untitled “SIGMA Baselines” in October 1999.45

The idea was also

to elaborate criteria allowing for comparison between candidate countries, as the SIGMA

newsletter explains: “Assessing the institutional situation in the central and eastern European

candidate countries against the baselines criteria provided SIGMA and the Commission with

valuable comparative information” (“Baseline for Financial Control in EU Candidate”, p.7),

which nourished the Commission’s Regular Reports of October 1999 on the progress made by

the candidate countries in applying the acquis communautaire.

With these new tasks, two logics that had been until then implicit or absent were

introduced into SIGMA’s activities. First, they led to establishing a normative and

asymmetric relationship with partner countries; secondly, they required the existence of

predefined criteria, or in other words yardsticks defining the goals to achieve. In effect, the

reform introduced an exogenous logic and pace that were proper to the pre-accession process.

As many scholars have shown, the EU’s Eastern enlargement has been characterized by a

particularly decontextualized approach to conditionality. The “accession criteria” established

in the Councils of Copenhagen and Madrid were defined within the framework of an

asymmetric relationship in which the expectations of the countries concerned were not taken

45

SIGMA Baselines – October 1999. Available in the DVD Document Collection 2010 Edition, OECD,

SIGMA.

into account.46

From 1997, the actors of DG enlargement were embedded in a network of

constraints characterized by reluctance from some member states toward the accession of

countries considered as “unreliable”. During the same period, a set of reforms of EU methods

of cooperation with third party countries were introduced following criticisms by institutions

such as the European Court of Auditors and the European Parliament (Tulmets, 2006, p.163).

After the European Council of Luxembourg, the cooperation policy was no longer demand-

driven but accession-driven (ibid). Yearly assessments and objective and homogeneous

“progress criteria” (DG Enlargement now uses the term “opening benchmarks”) were added

to the toolkit developed to ensure the “feasibility” of the Eastern enlargement.47

While it

emphasizes the importance of “ownership” (according to the terminology used by DG

Enlargement), the EC tends in practice to favor a conditional and standardized approach to

assistance to candidate countries.

The surreptitious shift toward a prescriptive logic

Since then, a new trend has quietly shifted the type of assistance proposed by SIGMA

away from its initial ambition.

First, assessments seem to have become progressively part and parcel of the job. The

SIGMA advisors who were hired after the late 1990s appear to have found it easier to

integrate this task into their activity. In interviews, they present support and evaluation as two

sides of the same reform assistance process. Some also argue that evaluation, or at least the

accompanying criteria, is an expectation of partner countries. Anticipating pre-accession

negotiations, the governments of candidate countries expect SIGMA advisors to shed light on

the yardsticks used by the EC to assess their progress. The hard power of conditionality seems

to have deeply informed the relationships between SIGMA and partner countries that are also

accession candidates.48

In terms of content, SIGMA documents still make sure to avoid the promotion of a

self-described “model” (“continental”, “Anglo-American”, “managerial” or “bureaucratic”

model). They do so all the more readily as the administrations of EU member states are far

from homogeneous. However, SIGMA participates in the clarification of accession criteria in

46 Studies on the accession process of CEEC have tended to emphasize the asymmetrical character of

conditionality, insofar as unlike member states, candidate countries had no influence whatsoever on the rules that

were proposed to them. They argue that the Eastern enlargement was characterized by a particularly conditional

approach described by some as a « regate » (Schimmelfennig, Sedelmeier, 2005, p.11): candidates are assessed

on the basis of their “progress” (the word used in the titles of the yearly reports published by the Commission

during the pre-accession process) and can only be eligible for accession to the EU once they have fulfilled all the

“criteria” defined by the EC (Dimitrova, 2002; Schimmelfenning, Sedelmeier, 2005). As a matter of fact, even

though negotiations were conducted on an individual basis with each candidate country, they were all subjected

to the same accession criteria and the same evaluation grid. The latter are not adjusted to national contexts, even

though a given dimension of the acquis may particularly resonate in a given country and require a significant

adjustment to be understood. 47

I will not elaborate on the specific constraints of the EC and its DGs at the time of the enlargement to the

CEEC. See in particular Robert, 2001, and Tulmets, 2006. 48

The network of relationships is different for neighbor countries, which do not have to fit the requirements of

EU accession.

its activity areas. Without constituting models as such, these productions attempt to define

European standards.

This trend is illustrated by a comparison of the SIGMA Papers. Most of these

publications consist in comparative analyses that are neither presented as guides for good

practices nor as ranking tools – they do have a prescriptive dimension, but it remains implicit.

These papers, frequently drafted by external experts, address specialized administrative

structures in EU member states (for instance OECD (2007), "Central Public Procurement

Structures and Capacity in Member states of the European Union", SIGMA Papers, No. 40,

OECD Publishing), the relations between “politics” and “administration” in various countries

(OECD (2007), "Political Advisors and Civil Servants in European Countries", SIGMA

Papers, No. 38, OECD Publishing) or ongoing reform processes in candidate countries

(Meyer-Sahling, J. (2009), "Sustainability of Civil Service Reforms in Central and Eastern

Europe Five Years After EU Accession", SIGMA Papers, No. 44, OECD Publishing;49

Meyer-Sahling, J. (2012), "Civil Service Professionalisation in the Western Balkans", SIGMA

Papers, No. 48, OECD Publishing). Though they weigh on very varied subjects, these reports

share a comparative approach that is used for purposes of analysis – not assessment. They fit

within the classic view of expertise, according to which political decision-making requires

validation with an informed and politically neutral analysis.

Other reports, generally drafted by SIGMA advisors, have a different purpose: they are

meant to clarify or even define EU accession criteria. The SIGMA Paper European Principles

for Public Administration (OECD (1999), SIGMA Papers, No. 27) for instance, is an attempt

to define the lowest common denominator between the administrations of European countries.

Its author, Francisco Cardona, does not propose a “model” but establishes general

“principles”; the goal is to give “baselines” for conditionality in terms of “administrative

capacity” introduced in the 1995 European Council of Madrid. This ties in with his talk at a

conference organized for candidate countries in Montenegro in 2009. Cardona emphasizes the

difficulty of progressing on the path toward accession in the absence of a unified European

model for candidate countries. He goes on to argue that “however, in the course of time, a

relatively wide consensus on key criteria [has emerged], which by now can be considered as

part of the acquis communautaire and can be grouped into the following four categories, with

the rule of law in a prominent position”. These categories reflect the principles laid out in

SIGMA Paper n°27: “1. Rule of law, i.e. legal certainty and predictability of administrative

actions and decisions, which refers to the principle of legality as opposed to arbitrariness in

public decision-making and to the need for respect of legitimate expectations of individuals;

2. Openness and transparency, aimed at ensuring the sound scrutiny of administrative

processes and outcomes and its consistency with pre-established rules; 3. Accountability of

49 This report assesses the civil service reforms initiated by the new Member States before their accession, with a

view toward understanding the reasons for what is considered as a relative failure of European assistance in this

matter. In the concluding section, the authors question the model-based approach and argue that depending on

their respective experiences, CEECs have often followed original pathways. Meyer-Sahling,

J. (2009), "Sustainability of Civil Service Reforms in Central and Eastern Europe Five Years After EU

Accession", SIGMA Papers, No. 44, OECD Publishing.

public administration to other administrative, legislative or judicial authorities, aimed at

ensuring compliance with the rule of law; 4. Efficiency in the use of public resources and

effectiveness in accomplishing the policy goals established in legislation and in enforcing

legislation” (Cardona, 2009).

In another SIGMA activity area, the SIGMA Papers attest to an even more direct role

in the conditional monitoring of pre-accession. In the late 1990s and early 2000s, SIGMA

actively contributed to the elaboration of the new chapter of the acquis communautaire on

“financial control”. Initially chapter 28, this chapter on the structures, organizations, processes

and methods for controlling the use of public funds in public administration then became

chapter 32 of the acquis communautaire. The imminent accession of the CEEC contributed to

making this issue a priority for the EC and the EU member states. As summed up by a 1999

SIGMA internal publication, “all are concerned about the ability of aspiring Member States to

protect the Union’s financial interests when managing EU funds” (“Protecting the financial

interests of the state and of the European Union”, 1999, p.1). In this context, the elaboration

of precise, unified criteria meeting the requirements of the European treaties was considered

as an urgent matter. It also allowed for the consolidation and legitimization of this activity

area, so that it was introduced into the acquis.

SIGMA was very active on this subject in the late 1990s. Two important SIGMA

Papers dealt with financial control and external audit (SIGMA Papers n°19 and n°20, 1997).

Remaining faithful to its method, SIGMA analyzed the various solutions adopted by member

states without drawing an actual “model” from them. As the abstract for these two papers

indicates: “The purpose of these publications is to assist central and eastern European

countries that have applied for membership of the European Union in discerning the ideas at

stake, to give comparative information on the various approaches and solutions used by

Member States”. The Baselines document cited above obviously adopted a more directive

approach, and brought about a surge of activity in this field. Control of public finances in the

broader sense (also including public procurement) is addressed in four of the document’s six

themes.50

The first ten-page-long chapter, which focuses on “public sector financial control”

conducts an exhaustive review of “EU regulations concerning budgeting, financial control and

external audit”; it was probably used as a basis for the elaboration of the acquis chapter.

There is a notable difference between these two examples of contributions to European

standardization (here, in the sense of the definition of “European standards” in certain fields

deemed as priorities). The second example of “standardization” on public financial control

does not participate in the definition of general principles, but in the elaboration of a genuine

unified assistance tool. In the process, an existing European model was clearly favored to the

detriment of another. From 1997 to 2002, SIGMA was indeed involved in the elaboration of a

doctrine now known as PIFC (Public Internal Financial Control). Considered in the current

enlargement process as a requirement for candidate countries, PIFC recommends a reform

50

“Public sector financial control baseline; public procurement management systems baseline; public

expenditure management systems baseline; public sector external audit baseline”. The last two were: “civil

service baseline” and “policy-making and coordination machinery baseline”.

based on managerial accountability and functionally independent internal audit,51

which

means that a choice has been made between the preexisting European models in that field.52

.

This choice benefited the model that the Commission was in 1999-2000 in the process of

adopting in the wake of the Santer Commission scandal, i.e. the “management accountability”

model (internal EC administration reform implemented in 2000). Quite significantly, this

SIGMA activity area is one of those where normative, “evolutionist” formulations are most

frequently used (they feature more sparsely in other activity areas). While elsewhere there is a

search for consensual “principles” in Europe, the goal here is to adopt a “more modern”

system complying with “good practices” in the field.53

This suggests the hypothesis that in

areas where the EC most closely collaborated with SIGMA in the pre-accession process, the

Programme moved toward a standardized, normative form of assistance. This trend is partly

the result of constraints proper to the EC in the Eastern enlargement process.

The trend is however neither linear nor univocal. It elicited debates between SIGMA

departments: some advisors thought that PIFC encouraged an overly homogenizing approach

of diverging national contexts at the risk of not achieving the main objective, i.e. a cost-

efficient system for using public funds. In an otherwise praiseful paper on Robert de Koning’s

work, Bob Bonwit for instance writes: “As for a number of new policies on which candidate

countries embarked in the accession process, where it was necessary to ‘close the

(negotiating) chapter’ and show progress on the way to accession, the introduction of PIFC

may raise questions of ownership and sustainability. Compliance may sometimes be confused

with excellence.” The question of appropriation is thus clearly raised (and Bob Bonwitt could

almost as used Radaelli’s metaphor of the “empty bottle”), in relation to the temporal and

material conditions of the pre-accession process.

The issue of the appropriation and the durability of the reforms engaged during the

pre-accession process is then linked to the method of transmission of what is presented as a

“model”. According to Bonwitt, “another possible point of further discussion is about how

detailed a prescribed model of PIFC can be. (…) The texts produced often displayed a

multitude of detailed prescriptions, which sometimes made their adjustment to practical

51

For an expert analysis in favor of PIFC, see Koning, 2007. Professionally invested in the issue in the EC (he

was then Principal Administrator at the DG-AUDIT, now DG-Budget), Robert de Koning was in touch with

SIGMA advisors on a regular basis in the late 1990s-early 2000s. The author in fact underlines the ties with

SIGMA in his account of the elaboration of PIFC, along with the contributions of the Institute of Internal

Auditors and of INTOSAI (International Organisation of Supreme Audit Institutions). It is worth noting that his

previous publication on the subject was in the SIGMA newsletter (Koning, 1999). 52

Broadly speaking, two approaches can be identified among the EU Member States. One is the “third party ex-

ante approach”, which can be found for example in France, Spain or Portugal with the system of the centrally

placed Inspectorate General for Finance. The other model, the “management responsibility approach”, can be

found in Great Britain or the Netherlands. It emphasises the responsibility of the head of the line ministry and

includes the creation of Internal Audit Services in each of the line ministries. 53

In 1999, two SIGMA advisors specialized in financial control wrote: “This article addresses (…) good

European practices that are contained in the baseline on financial control.” (Larsson, Madsen, 1999, p.4). More

recently, one of the senior advisors of SIGMA’s Financial control and external audit department wrote:

“Traditional internal audit focused on the audit of individual transactions, their accuracy and compliance with

relevant rules and regulations. This contrasts with the current, more modern, internal audit approach (…)”. The

conclusion added the caveat that “there is no quick-fix solution” and that reform must rely on an approach that is

not merely technical but global. (Klingenstierna, 2007).

situations problematic.” While acknowledging the value of de Koning’s synthesis and

analysis, Bonwitt has serious reservations on its usefulness as a prescriptive model. While the

former SIGMA director commands de Koning for the “remarkable and rapid construction of

what most interested parties look at as a model”, the use of the periphrasis indicates that he

distances himself from this approach in terms of “models”. This is confirmed by his

conclusion: “As its main contributor and promoter himself recognised, the model is

nevertheless not a fully fixed thing, and with the spread of PIFC-oriented systems, it will be

more and more possible to rely on practice and assessment to improve, refine or simplify, and

generally speaking adjust the initial design of PIFC, to better take into account the reality of

implementation.”54

.

Conclusion

The construction of a context-driven method of international assistance on such a

sensitive issue as is the central public administration has been the result of a long maturation.

It was made possible by the convergence of several conditions. Partly based on a negative

experience of expert advice in the form of reports (assistance on paper), the method devised

and experimented in the years 1975-1990 emerged for strategic reasons in a period of

institutional fragility and uncertainty. The dependence of the OECD team on the goodwill of

member state representatives explains the creation of an inclusive method where national

officials are actively involved. It is tricking that it is precisely because of this inter-state

context that the founders of PUMA had to develop a method based on trans-administrative

networks which over time gained a real autonomy.

Even though the international approach adopted is sensitive to national contexts and

traditions, the normative dimension is not necessary altogether absent. The range of proposed

solutions is not infinite; generally a desirable “middle way” is traced for countries eager to

stay in the OECD “club”. Lacking financial or political instruments to prevail, the

administrations of the various OECD committees develop methods that allow them to attract

and, if need be, convince representatives of member states. Peer reviews and joint activities

are cornerstone of this approach, as prescription entails exerting an attraction on member

states.55

Because of its strong presence in the field and of its initial recruitment choices

(favoring former civil servants from member states), the SIGMA programme is probably one

of the OECD departments where the normative dimension is least asserted and where

attention to national contexts has been most systematically put in practice. In this particular

instance, the cohabitation with the EU enlargement policy created more tension in terms of

method than the relationship with other OECD departments. After 1997, there was a clear

strain due to the difficult coexistence between two different international approaches of

54

Comment by Bob Bonwitt, 1/03/2007. Website http://www.pifc.eu, last accessed 20 May 2013. 55 As Guy Peters notes (Peters, 1997, p.185): “the individual member countries want to appear as modern and

innovative as possible to their peers”.

“prescription” for member states. SIGMA appears caught between the different operational

methods of the OECD and the EU; the former established its relationships with member states

with a diplomatic (inter-state) and incentive-based approach, while the latter is characterized

by a supranational ambition and a prescriptive and conditional approach to the relationship

with candidate countries and, in some areas, with member states. The legacy of the origins

(TECO and PUMA) encourages the SIGMA’s advisers to create trans-governmental and

trans-administrative networks between practitioners, while the European pre-accession

process influences the SIGMA’s activities toward a more procedural and normative trend.

References

Barnett Michael, Finnemore Martha, Rules for the World: International Organizations in

Global Politics. Ithaca: Cornell University Press, 2004.

Bézès Philippe, 2007, « Construire des bureaucraties wébériennes à l’ère du New Public

management ? », Critique internationale, 35, p.9-29.

Cassese Sabino, Wright Vincent, 1996, La recomposition de l’Etat en Europe, Paris, La

Découverte.

Cling Jean-Pierre, 2011, “La Banque mondiale, entre transformation et résilience”, Critique

Internationale, 2011/4, n°53, p.43-65.

Carroll Peter, Kellow Aynsley, 2011, The OECD: A Study of Organisational Adaptation,

Edward Elgar Publishing.

Dimitrova, 2002, ‘Governance by Enlargement? The case of the administrative capacity

requirement in the EU’s Eastern enlargement’ West European Politics, 25:4, October 2002.

Gayon Vincent, 2009, « Le crédit vacillant de l’expert. L’OCDE face au chômage dans les

années 1990 et 2000 », Cultures & Conflits, n°75, p. 53-73.

Hood, Christopher, 1998a, “Individualized contracts for top public servants: copying

business, path-dependent political reengineering or Trobriand cricket”, Governance (Oxford)

1998-10 11 (4), p.443-462

Hood Christopher, 1998b, The Art of the State: Culture, Rhetoric, and Public Management.

Oxford: Clarendon Press.

King Desmond, Le Galès Patrick, ed(s). 2011. "Conceptualiser l'Etat contemporain."

Revue française de sociologie, 52 (3).

Koning Robert de, 2007, Public Internal Financial Control, a European Commission

Initiative to Build New Structures of Public Internal Control in Applicant and Third-Party

Countries. Sint Katharina Lombeek, Belgium.

Mahon Rianne, McBride Stephen, 2008, The OECD and Transnational Governance, UBC

Press.

Nay Olivier, 2012, «How Do Policy Ideas Spread among International Administrations?

Policy Entrepreneurs and Bureaucratic Influence in the UN Response to AIDS», Journal of

Public Policy, 31, 1, p. 53-76.

Nay Olivier, Petiteville Franck, 2011, « Elements pour une sociologie du changement dans les

organisations internationales », Critique Internationale, 2011/4, n°53

Nolan B. C., ed., 2001, Public sector reform. An international perspective, Basingstoke,

Palgrave.

Ougaard M., Higgott R., eds., 2002, Towards a global polity. London, Routledge.

Pal Leslie A., 2010, «The OECD and Global Public Management Reform», in Nemec J.,

Peters B. G., State and Administration in a Changing World, Bratislava, NISPAcee.

Pal Leslie A., 2012, Frontiers of Governance: The OECD and Global Public Management

Reform, New York, Palgrave Macmillan

Pollitt Christopher, Bouckaert Geert, 2000, 2004, Public Management Reform: a

Comparative Analysis.

Porter T., Webb M., 2008, « Role of the OECD in the orchestration of global knowledge

networks », in R. Mahon, S. McBride, The OECD and Transnational Governance,

Vancouver, Toronto, UBCPress, p.43-59

Premfors Rune, 1998, “Reshaping the democratic state: Swedish experiences in a comparative

perspective”, Public Administration, 76, Spring, p.141-159.

Radaelli Claudio M., 2004, “The Diffusion of Regulatory Impact Analysis. Best Practice or

Lesson-Drawing?”, European Journal of Political Research, 43 (5), p.723-747.

Radaelli Claudio M., 2006, “Diffusion without convergence: how political context shapes the

adoption of regulatory impact assessment (RIA)”, in Christopher Knill, ed., Cross-National

Policy Convergence, p.161-179.

Reinalda B., Verbeek B., eds., 2004, Decision-Making within International Organizations,

Abingdon, Routledge.

Robert Cécile, 2001, « L’Union européenne face à son élargissement à l’Est : incertitudes

politiques et construction d’un leadership administratif », Politique européenne, n°3, p. 38-60.

Sahlin-Andersson Kerstin, 2000, « Arenas as standardizers », in Brunsson Nils, Jacobsson

Bengt, eds., A World of Standards, Oxford University Press, p.101-113.

Saurugger Sabine, Surel Yves, 2006, « L’européanisation comme processus de transfert de

politiques publiques », RIPC, 13 (2), p.185-186 et p.193-197

Schimmelfennig Frank, Sedelmeier Ulrich, eds., 2005, The Europeanization of Central and

Eastern Europe, Cornell University Press.

Trondal J., Marcussen M., Larsson T., Veggeland F., 2010, Unpacking International

Organizations : The Dynamics of component Bureaucracies, Manchester, Manchester

University Press.

Tulmets Elsa, 2006, « L’adaptation de la méthode ouverte de coordination à la politique

d’élargissement de l’UE : l’expérience des jumelages institutionnels en Estonie et en

Hongrie », Politique européenne, 18, p.155-189

Woodward Robert, 2009, The Organisation for Economic Co-operation and Development,

Routledge.

Institutional sources

Cardona Francisco (SIGMA), “Integrating national administrations into the European

administrative space”, conference on Public Administration Reform and European

Integration, Budva, Montenegro, 26-27 March 2009.

CCEET/SIGMA/PUMA(92)1

Comment by Bob Bonwitt, 1/03/2007. Website http://www.pifc.eu, last accessed 20 May

2013.

European Commission, Sub Programme for strengthening institutional capacity of IPA region

beneficiaries, “Description of the Action SIGMA”, December 2012

Interim Evaluation of SIGMA Programme, September 28th 2012

Klingenstierna Ulrika , 2007, “Review of Robert de Koning’s book”, SIGMA, 15 March

2007.

Koning R. de (1999), “Public internal financial control (PIFC) in the context of EU

enlargement”, Public Management Forum, Vol. V, no. 6, November-December 1999,

SIGMA, Paris.

Larsson Kjell, Madsen Johannes Stenbæk, 1999, “Baseline for Financial Control in EU

Candidate”, Public Management Forum, nov/déc., vol. V, n°6.

Meyer-Sahling, J. (2009), "Sustainability of Civil Service Reforms in Central and Eastern

Europe Five Years After EU Accession", SIGMA Papers, No. 44, OECD Publishing.

Nicolaidis, K. and R. Kleinfeld (2012), "Rethinking Europe's « Rule of Law » and

Enlargement Agenda: The Fundamental Dilemma", SIGMA Papers, No. 49, OECD

Publishing.

OCDE/GD(95)120, 1994 SIGMA Implementation Report, Paris, OECD, 1995

OCDE/GD(96)47, “SIGMA Implementation Report 1995”, OECD, Paris, 1996

OECD (1999), SIGMA Papers, No. 27, “European Principles for Public Administration”,

OECD Publishing)

OECD, "Effects of European Union Accession: Part I Budgeting and Financial Control",

SIGMA Papers, No. 19, OECD Publishing, 1997

OECD, 1977, “The Technical Co-operation Service”, OECD Information, n°40, December

1977

OECD, "Effects of European Union Accession: Part II External Audit", SIGMA Papers, No.

20, OECD Publishing, 1997

“Protecting the financial interests of the State and of the European Union”, special issue by

Kjell Larsson and Johannes Stenbaek Madsen (SIGMA), Public Management Forum, vol.V,

n°6, nov/dec 1999.

Public Management, note by Derry Ormond and Ravi Kapil, 22 September 1975, c.c. G.

Piperno and A. Grinkow, RK/fma. Document kindly sent by Derry Ormond, November 2012.

SIGMA Baselines – October 1999.