Rome, july 5, 2006

Observing project implementation and conducting project analysis

(UVER)

Presentation byLuigi Guerci

Public Investments Verification Unit (UVER)

The Unit performs controls during implementation and after completion of projects with:

• On site control

• Desk monitoring

Public Investments Verification Unit (UVER)

During the implementation

On site control

The activity consists of inspections on the construction sites and periodic contacts with project managers to find out potential problems and suggest solutions.

Public Investments Verification Unit (UVER)

During the implementation

On site control

Provides support and technical assistance to selected implementing administrations to accelerate the execution of projects.

Public Investments Verification Unit (UVER)

During the implementation

Single project solution• facilitation on project achievement• raising up potential problems• speed up procedures

General solution • feed back on general procedures • suggestion of changes in laws

Public Investments Verification Unit (UVER)

Projects are usually grouped with regard to the financial sources e.g. :

• National additional resourses

• EU funds

• Legge Obiettivo funds

During the implementation

Public Investments Verification Unit (UVER)

During the implementation

Framework programme agreements consist of:

• more than 60 bil €

• more than 10.000 projects

Public Investments Verification Unit (UVER)

During the implementation

-All projects are verified with a monitoring procedure (performed by SPTSI)

- 450 projects (selected among the most critical and important) were verified on site

Framework programme agreements

Public Investments Verification Unit (UVER)

Checking 450 projects we found out some common operational impediments as:

•financial problems (flows and coverage)

•design deficiency

•failure of the permission procedure

•inability of the implementing

administration

Public Investments Verification Unit (UVER)

During the implementation -Effect of the operational impediments

Some results579

730

472

519

448

315

1308

1663

1171

1921

722

830

962

1367

948

1218

395

380

0 500 1000 1500 2000 2500 3000 3500 4000

Media prevista

Media effettiva

Media prevista

Media effettiva

Media prevista

Media effettiva

Inter

venti

verifi

cati

Inter

venti

non

verifi

cati S

up. 5

Meur

o

Inter

venti

non

verifi

cati I

nf. 5

Meur

o

Predisposizione ed approvazioneprogetto preliminare

Periodo intercorrente tral'approvazione del preliminare el'inizio dei lavoriEsecuzione lavori

Public Investments Verification Unit (UVER)

During the implementation

In the period after the preliminary design (which usualy represents the inclusion of the project in the Framework Programme Agreement) and before the start of the construction work, there are nearly no expenditures

Public Investments Verification Unit (UVER)

After the preliminary design andbefore the construction

292

411

234

323

238

484

165

283

183

266

59

154

0 500 1000 1500 2000 2500

Media prevista

Media effettiva

Inte

rven

ti no

n ve

rific

ati S

up. 5

Meu

ro

Progettazione definitiva Approvazione progettazione definitiva Progettazione esecutiva

Approvazione progettazione esecutiva Aggiudicazione Approvazione aggiudicazione

Tot. 3,2 Anni

Tot. 5,2 Anni

Public Investments Verification Unit (UVER)

Risk

Financial programmes can be ineffective with the assigned resourses unemployed for 2 years (average).

Public Investments Verification Unit (UVER)

Feed back

The ex ante evaluation must consider the real capacity of a project of being implemented in a specific period.

Public Investments Verification Unit (UVER)

After the completion of a project

To answer to the questions • “ Was it worth it?”• “Is the service offered congrous with

the preset target ?”• “Are users satisfied?”

Effectiveness Evaluation (in progress activity)

Why ?

Public Investments Verification Unit (UVER)

GENERAL GOALS

Disseminating effectiveness evaluation culture (self monitoring)

Improve the governance of projects

Building a participatory process of evaluation (involvement of key stakeholders at all stages)

Create a public interest disseminating evaluation results

Providing guidelines for decision making by selecting best practices and quantitative indicators with their targets

Public Investments Verification Unit (UVER)

After the completion of a project

Verify and measure actual product and impact of concluded projects in terms of improved or added services to beneficiary communities.

Effectiveness Evaluation(in progress acitivity)

Public Investments Verification Unit (UVER)

After the completion of a project

Check of physical realization, direct results and indirect results with quantitative indicators.

Effectiveness Evaluation (in progress acitivity)

Public Investments Verification Unit (UVER)

HOW DO WE DOApply a standardized but flexible conceptual framework

through the following stages

Desk reviews of project related documents

Brainstorming with key stakeholders (objectives and needs identification)

Selection of performance indicators (process, outputs, outcomes, impacts)

Data collection (administrative and statistical sources, structured interviews, ad hoc questionnaires)

Fill up standardized projects forms containing summary information

Presentation and dissemination of evaluation results in specific and general reports

Public Investments Verification Unit (UVER)

After the completion of a project

Effectiveness Evaluation (in progress acitivity)

Project selection from a group of 310 called “Completamenti” (1,8 bil €).It is a group of non completed projects, that in 1999, we decided to carry on after an ex ante evaluation made by UVAL.

Public Investments Verification Unit (UVER)

After the completion of a project

Effectiveness Evaluation (in progress acitivity)

Totale

Non avviati

In corso

Conclusi non in esercizio

Conclusi in esercizio

Interventi di completamento

310 7 82 74 147

Interventi oggetto di valutazione d’efficacia

228 7 // 74 147

Interventi esaminati al 28/02/2006

60 7 // 9 44

Public Investments Verification Unit (UVER)

After the completion of a project

Effectiveness Evaluation (in progress acitivity)

Interventi conclusigià in esercizio

(44 casi)

Valutazionedi efficacia

Positiva40 casipari a

167,91 Mln €

Negativa4 casipari a

3,30 Mln €

Public Investments Verification Unit (UVER)

After the completion of a project

Effectiveness Evaluation (in progress acitivity)

For the uneffective projects the department suggested Cipe to deduct an equal amount of funds from the next assignment to the administrations involved.

Public Investments Verification Unit (UVER)

Quality results

• General achievement of results related to the construction

• Lack of targets related to operational results and impact

• Presence of gaps between construction and operation

Public Investments Verification Unit (UVER)

Basic Ideas

• Evaluate the real capacity of a project of being implemented in a specific period

• Keep control on the main step of a project, especially during the period after the preliminary design and before the start of construction (with nearly no expenditures)

• Fill the gap between the physical implementation and the working phase thinking from the beginning the services the project will provide and finding indicators to qualify the targets to be achieved