Performance Monitoring:Thoughts, Lessons, and Other Practical

Considerations

slide 2

Objectives

Identify key concepts Discuss applications Address questions and concerns

slide 3

Key Concepts

Performance monitoring What you do

How well you do it

Do you accomplish something

Process, quality, capacity, outcomes The window Baselines and standards Risk or case mix adjustment

slide 4

Performance Monitoring

Part of much larger cycle of program design and implementation

Performance - this is about definitions Inputs, outputs, or the relationship

between inputs and outputs?

Monitoring - This is about data collection and analysis Important with respect to investment - are

you getting something back?

slide 5

The Framework

slide 6

Process of care

Referral, intake, and assessment Service planning, link to

interventions Reassessment, follow-up, case

closure

slide 7

Quality of Care

Human resources Physical plant and equipment Practice protocols - evidence base Supervision Consumer feedback Agency management around

practice model fidelity

slide 8

Capacity

Enough trained workers Enough office space Enough funding Enough information Enough is defined by the relationship

between process, quality, and outcomes

slide 9

Outcomes

Depends on the program and intervention Well-being

Safety

Family provides stable nurturing

Education

Health

Behavioral health

slide 10

Process, Quality, and Outcomes

Highly interdependent Quality depends on a process

Process is different than quality

Quality without outcomes is ‘inefficient’

Agencies invest in process, quality, and capacity

slide 11

The Window

Performance happens in time Improvement is change in

performance over time Sampling in time is difficult but

critical

slide 12

Clinical Experience in Time:(Each line represents the start and end of service within

the window)

Jan. 1, 2000 Jan. 1, 2001 Jan. 1, 2002

slide 13

Sampling

Inception Process vs child

How much time do you have to observe the process?

slide 14

Baselines and Standards

Baseline is a measure taken prior to intervention

Standards of practice and performance The usual as in standard practice

Fidelity or compliance

Standards are better suited to process and quality; baselines are better suited to outcomes

slide 15

Risk or Case Mix Adjustment

An important question when facing variation in performance:Is the variation a function or performance or

the result of client differences

Children/families have different outcomes for reasons that are intrinsic to them Baseline mortality rates differ by age

Adjustment for case mix refers to taking the intrinsic differences into account somehow when measuring outcomes

slide 16

Case Mix Adjustment Applied

Case mix adjustment makes more sense for outcomes, less so for process and quality

Process/quality standards apply to all children, given the process standard applies in the first instance (differential diagnosis)

Baselines for outcomes should be adjusted

Standards don’t work as well for outcomes because of the random component.

slide 17

Comments, Questions, Concerns

Thank you!