BMH4-98-3669

CLINICAL GUIDELINES IN EUROPE

OUTLINE

Background to the project

Objectives

The AGREE Instrument: validation process and results

Outcomes and impact of AGREE

Research & policy implications

Future developments

BACKGROUND to AGREE PROJECT

Clinical guidelines play a key role in healthcare improvement in most countries

Ensuring their quality is a shared concern

There is a need for consistent international guidance on guideline development and assessment

‘The whole is greater than the sum of its parts’

OBJECTIVES

Develop compatible approaches for the creation of clinical guidelines

Establish a structure for the appraisal and monitoring of clinical guidelines

Define relevant quality criteria

Promote and encourage the diffusion of these criteria through international exchanges and collaborative links

APPRAISAL OF GUIDELINES fOR RESEARCH & EVALUATION

(AGREE) INSTRUMENT

The AGREE Collaboration

September 2001

PURPOSE OF THE AGREE INSTRUMENT To provide a systematic framework for

appraising the quality of clinical guidelines

To help guideline developers follow a structured and rigorous methodology

To help policymakers decide which guideline to recommend for use in practice

To help health care providers assess guidelines before adopting recommendations in practice

WHAT IS A GOOD GUIDELINE?

‘Quality of clinical guidelines’ is the confidence that:

the potential biases of guideline development have been addressed adequately

the recommendations are both internally and externally valid, and are feasible for practice

STRUCTURE

23 items 4-point

Likert Scale

Six Six domainsdomains

1. Scope & purpose (3)

2. Stakeholder involvement (4)

3. Rigour of development (7)

4. Clarity & presentation (4)

5. Applicability (3)

6. Editorial independence (2)

User guide

Overall assessment

DEVELOPMENT PROCESS

Subgroup reviews literatureFirst draft circulated for comments First workshop, instrument modifiedFirst validation study Second workshop, instrument revised

Final validation study

VALIDATION PROCESS (1)

Phase 1Phase 1

The instrument was applied to 100 guidelines from 11 countries

All countries followed the same protocol

Each guideline was independently assessed by 4 appraisers (194 altogether)

Instrument was used in English

VALIDATION PROCESS (2)

Phase 2Phase 2

Refined instrument tested on 33 guidelines randomly selected from 1st study

Same countries participated

New set of appraisers recruited (70)

Each country assessed 3 guidelines

RESULTS

Over 90% of appraisers found the instrument and the user guide useful

Reliability was satisfactory for each domain

- Cronbach’s Alpha 0.64 - 0.88 - ICC 0.57 -0.91

There was suggestion of criterion validity

OUTCOMES

The AGREE Instrument has been translated into 7 European languages, more in preparation

Six papers have been published

All EU-funded projects are requested to assess guidelines with the AGREE Instrument

WHO has endorsed the AGREE Instrument

The Council of Europe has formally recommended its use

IMPACT OF AGREE

Guideline Assessment

• Europe• South East Asia• Middle East• Latin America• Australia• Canada

Education and training

Postgraduate courses

(Mc Master University, New Zealand, UK, France)

Guideline development

European Society Cardiology

National guidelines programmes

ResearchUSAUKCanada

“Both NICE and SIGN aim to meet the international

standard set by the AGREE guideline appraisal

instrument”

Joint NICE/SIGN statement on working together to improve patient careNICE website 18 April 2002

RESEARCH & POLICY IMPLICATIONS

The AGREE Instrument creates a common framework for assessing guidelines that can be applied internationally.

Can be used to compare quality of guidelines across countries and disease areas.

Provides standards for reporting of clinical guidelines published in medical journals.

Governmental agencies can use AGREE as basis for making policy decisions about guidelines

FURTHER DEVELOPMENTS: AM Project

Structured visits to 7 central & East

European countries

Dissemination workshops in 15

European countries

AGREE Training manual

& Evaluation form

Research programmeEU 6th Framework

G.I.N. The Guidelines International Network

The Association for Guideline REsearch and Evaluation

http://www.agreecollaboration.org

Website address