MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s

MARQuIS:WP4: The Questionnaire

Niek Klazinga

Kiki Lombarts

Ines Rupp

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s ObjectivesThe strategic objective of work package 4 is to describe in a sample of EU member states how hospitals have applied national quality strategies, to what extend they meet the defined requirements of cross-border patient care and what organizational and/or methodological variables are associated with (non-)compliance of requirements.

Further, WP4 is to validate the statistical results provided by WP3 concerning types and quantity of health services used by patients from another European member state.

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s WP 4: Questionnaire Purpose

• measure quality improvement: • the application of Q policies and procedures,

Q governance structure as well as Q activities, used to close the gap between current and expected levels of quality

• measure quality: • quantifying the current levels of performance

or compliance with expected requirements.

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s Three main phases

1. Defining a concrete set of measures (development of a questionnaire) to identify quality strategies in hospitals and quality requirement for hospital care delivered to EU cross-border patients

2. Collecting data through a questionnaire survey in an ad random sample of hospitals in 8 EU member states

3. Analysing and reporting the data

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s Focus of the questionnaire in

measuring quality (improvement) in hospitals

Strategic level, Hospital-wide policies,

structures and activities

Tactic level, hospital (supporting)

services, policies, structures

and activities

Operational level, Management of patients

suffering from AMI

Operational level, Management of patients

suffering from acute appendicitis

Operational level, Management of women

during pregnancy and labor

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s

Section S1 S2 S3 S4

Numberof questions 63 42 48 46

Number of items appr. 500 in total

QuestionnaireDesign

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s Recruitment & response

Hospitals visiting quest.

Hospitals responded

UK 41 14

Ireland 29 25

Neth 12 10

Belgium 33 25

France 100 78

Spain 131 113

Poland 84 80

Czech Rep 53 44

Total 483 389

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s Challenges for analyses

• Frequency distribution answers• Testing of hypotheses• Contributing to the audit tool• Developing a QI-maturity

classification for• sampling for the audit

• testing of hypotheses

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s QI-maturity classificationPolicy, planning, documents 20 items α=0.86

Leadership 36 items α=0.89

Structure 19 items α=0.69

QI activities 8 items α=0.75

QI activities (labs) 20 items α=0.85

Patient involvement 6 items α=0.82

Accountability 4 items α=0.54

→ scores combined in an overall classification score

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s

Hospital

response

Hospitals in overall QI classification

Belgium 25 24

Czech Republic 44 38

France 78 65

Ireland 25 23

The Netherlands 10 8

Poland 80 76

Spain 113 105

UK 14 10

Total 389 349

Response QI-maturity classification

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s Variance overall

classification scoremin. max.

Belgium (n=24) 2.10 - 3.10

Czech Republic (n=38) 1.50 - 3.00

France (n=65) 1.87 - 3.25

Ireland (n=23) 1.90 - 3.08

The Netherlands (n=8) 2.09 - 2.52

Poland (n=76) 1.86 - 3.45

Spain (n=105) 1.70 - 3.26

UK (n=10) 1.77 - 2.37

Total (n=349) 1.50 - 3.45

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s Hospital distribution per

country based on the classification score

Belgium 3 16 5

Czech Republic 10 22 6

France 11 32 22

Ireland 8 12 3

The Netherlands 4 4 -

Poland 22 33 21

Spain 20 55 30

UK 9 1 -

Total 87 175 87*most mature: ≤25e percentile; least mature: > 75e percentile (entire sample)

“least mature”*“most mature”*

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s

WP 5: Hospital audits• On-site hospital audits

• Verify questionnaire responses• Obtain details of quality methods

• 105 hospitals, 8 countries• Recruiting hospitals that have returned WP4

questionnaire• Sample based on QI-maturity classification

(most / least mature)• Belgium 6, Czech Republic 15, France 21,

Ireland 8, The Netherlands 4, Poland 15, Spain 30, United Kingdom 6

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpr o

vem

en

t S

trate

gie

s Further validation QI-maturity classification

• Reliability study• Consistency over dimensions

• Exploring the “QI-index” in other datasets (e.g., PATH)

• Validity study• Comparison with maturity classification

based on audit

• Linking to data on accreditation / performance indicators (e.g. France, Spain)