1 © 2009 council of juvenile correctional administrators performance-based standards (pbs) for...

1© 2009 Council of Juvenile Correctional Administrators

Performance-based Standards (PbS) for Youth Correction and Detention Facilities

Using Performance-based Standards (PbS) to

Measure Evidence-based Practices

Correctional Education AssociationMarch 31, 3009Annapolis, MD


2

PbS: Measuring Performance and Improvement

Ned Loughran, executive director/project director CJCA

National non-profit organization dedicated to improvement of youth correctional services and practices so youths succeed when they return to the community

Unites nation’s youth correctional CEOs to promote best practices, address common concerns and provide leadership for juvenile justice

Directs several grant projects: MacArthur Foundation Model for Change, CJCA Yearbook OJJDP Mental Health Model, New Directors Seminar Committed to expanding the adoption of PbS as a best

practice to improve conditions of confinement


Presentation Overview

How PbS was developed The scope of PbS: Standards and Outcomes How PbS works PbS Tools for the Field Outcome Measure Examples NDTAC Recommendations for PbS/CJCA Surveys

Questions and Answers

www.pbstandards.org for more information


Performance-based Standards (PbS)

Information, outcome measures, reports:What’s going on

Injuries, climate, fairness, staff-youth relationships

Risk managementSuicidal behavior, screening, isolation

What works, what doesn’tEducation, programming, health/mental health

services

Evaluation and evidence-based practicesTreatment, behavior management


Performance-based Standards (PbS)

Conditions of Confinement

Quality of life for youths and staff

Making sure facilities are safe, hold youths accountable and provide rehabilitation that prevents future crime


1994 Conditions of Confinement Study

About 1,000 secure facilities and found “substantial and widespread deficiencies:” High rates of youths and staff getting hurt High rates of suicidal behavior Few timely or professional health screenings High levels of staff turnover Adherence to existing standards did not mean

“better” facility


Participants: October 2008

178 facilities 25 states, 9 counties

116 correction 48 detention 14 assessment centers 23 all-female facilities 73 facilities < 50 youths 26 facilities 50-99 youths 79 facilities > 100 youths 14,000 youths


How PbS Works: The Blueprint• A set of seven goals and 27 standards for corrections and

19 standards for detention facilities to assess:• Safety• Order• Security• Health and mental health• Programming • Justice • Reintegration (Corrections Only)Performance toward meeting each standard is measured

using one or more outcome measures, which are compared over time and to other participating facilities.

PbS is a standards based performance improvement measurement system


Each of the seven areas are formatted in the following manner:

Area: XXXGoal

XXX1 Performance Standard

PbS Standard Layout

Outcome Measures Expected Practices Processes

Rates, frequencies, numbers that show change in status,

occurrence or prevalence.

Practices expected to produce higher quality

outcomes based on available research,

consensus of professional opinion

and past PbS experience.

Policies, forms, tests used to describe or perform practices.


PbS Goals• Safety: To engage in management practices that promote

the safety and well-being of staff and youths.

• Order: To establish clear expectations of behavior and an accompanying system of accountability for youths and staff that promote mutual respect, self discipline and order.

• Security: To protect public safety and to provide a safe environment for youths and staff, an essential condition for learning and treatment to be effective.

• Programming: To provide meaningful opportunities and services to youths to improve their educational and vocational competence, to effectively address underlying behavioral problems and to prepare them for responsible lives in the community.


PbS Goals

Justice: To operate the facility in a manner consistent with principles of fairness and that provides the means of ensuring and protecting youth’s and family’s legal rights.

Health and Mental Health: To identify and effectively respond to residents’ health, mental health, and related behavioral problems throughout the course of confinement through the use of professionally appropriate diagnostic, treatment, and prevention protocols

Reintegration: To prepare youths for successful reintegration into the community while they reside at the facility


PbS Outcome Measures

Area Corrections DetentionSafety 14 14

Order 12 12

Security 7 7

Health 14 7

Programming 27 12

Justice 7 7

Reintegration 24 0

Totals 27 Standards/ 105 OMs

19 Standards/

59 OMs


April and October are data collection months.

Draft Site report is received within 48 hours of close of data entry.

Data corrections are made to anomalies, not recorded and outliers

Final Site Reports are generated along with summary reports for all data collection forms 14 days after receipt of the draft reports

Improvement plan is developed and entered into website with targeted outcome measures.

How PbS Works: Improvement Cycle


Data Collection Administrative Form

1 per site, 46 questions Incident Reports

All incident reports for data collection period (12,849)

Youth Record 30 random YR, 93 questions (5274)

Youth Climate Survey 30 random youths, 38 questions

(5668) Staff Climate Survey

30 random staff, 38 questions(5999)

Youth Exit Interview All youths released since last

data collection, 24 questions (5899)

Ongoing data entry


Site Reports and Analysis

Divided Between Areas: Safety, Order, Security,

Health/ Mental Health, Programming, Reintegration

Corrections: 105 outcomes

Detention: 59 outcomes

Compare over time, to the Field Average


Components of a Sites FIP Targeted Outcome

Measures Targeted Outcome Measure

Goals What is the problem? Action Steps Progress Notes Ongoing Review by Facility

Administrator , PbS State Coordinator, and PbS Coach

Facility Improvement Plans


Measuring Performance

PbS outcome measures adhere to definition of performance outcomes as rates, frequencies, numbers that show change in status, occurrence or prevalence

PbS outcomes are measured twice a year to reflect change from one data collection period to the next

PbS reports performance to users in many ways: performance over time, in comparison to the field average of other participating facilities, performance on outcomes targeted for improvement and outcomes critical to safe and effective operations


Facility Site Reports and Performance Profiles

Provide a compass for future improvement opportunities

Create a culture where it is accepted that data was not going to be used to “catch” people

Reliable, accurate data will be used to manage and drive improvement


PbS Coaches use Performance Profiles and Site Reports to guide sites to opportunities for improvement

Sites develop Facility Improvement Plans after each data collection

Ask sites to develop PbS sampling, definition, comprehensive reporting, and thorough data collection competencies first.


PbS Sustained Improvement


Components of a Sites FIP FIP Status:

Open or Closed Targeted Outcome

Measures Targeted Outcome Measure

Goals What is the problem? Action Steps Progress Notes Ongoing Review by Facility

Administrator and PbS State Coordinator



PbS Technical Assistance

Web site, data collection and data quality Technical assistance with data collection, site

reports and facility improvement plans, both by phone and email

Linkage to OJJDP Training and Technical Assistance

Distance Learning Calls

Consultants/coaches, staff Assist with Site Report Analysis, FIP

development, coaching Data quality review site visits


Why PbS Works:

PbS gives facilities data, twice year, showing levels of safety, order, security, programming, health/mental health services, justice and reintegration

PbS improvement process guides facilities in ways to use the information to identify what works and celebrate success as well as see what is not working and a structured path to create change

Improves accountability: facilities have data to share Prevents future incidents, lawsuits by improving compliance with

best practices and high standards Identifies progress over time, compared to other facilities Ongoing, personalized customer support Access to resources, networking across USA

(Yale School of Management’s Survey of PbS Users)


How do we know?

Field testimonials: word-of-mouth and money

2004 Innovations in American Government Award

CRIPA analysis: Legal protection Mapped 10 years of investigations with PbS Conclusion: Facilities that implement PbS as intended and

should not expect a CRIPA investigation because CRIPA investigates based on failure to meet the bare minimum Constitutional standards and PbS sets the highest standards for operational success.

PbS Research: Each kid’s individual experience best way to impact facility safety


Field Responses:

Directors, managers and staff are using data to understand facility performance and services

PbS demonstrates data can be provided within the context of improvement: PbS site reports identify the good and not-so-

good; Expected practices and processes provide a

roadmap of ways to improve FIP process integrates continuous work and

monitoring over time to demonstrate accountability and effectiveness


Benefits of PbS

Provide sites and agencies with a system of continual improvement

Tool to measure and track key indicators of facility performance and services

Able to compare performance to similar participating facilities across the country

Allow facilities to demonstrate success with data rather than anecdotes

Become more accountable and gain public support


Outcome Measure Example: Order

Goal: To establish clear expectations of behavior and an accompanying system of accountability for youths and staff that promote mutual respect, self discipline and order.

Standards: Maximize responsible behavior by youth and staff and

conformance to facility rules; Minimize the facility’s use of restrictive and coercive means of

responding to disorder; Maximize opportunities for youths to participate in activities

and programs. Outcome Measures: Incidents of youth misconduct; use of

physical restraint; use of mechanical restraint; use of isolation or room confinement and; average duration of isolation or room confinement.


Order 3: Physical Restraint Use

The denominator is constructed using the total number of youth population for the month (2533)

The numerator is constructed from incident reports of all physical restraints

Raw score for the October 2008 data collection was 23 physical restraints


Outcome Measure Example: Programming

Goal: To provide meaningful opportunities and services to youths to improve their educational and vocational competence, to effectively address underlying behavioral problems and to prepare them for responsible lives in the community.

Standards: Provide an education program that is tailored to each youth’s

education level, abilities, problems, and special needs, and improves education performance and vocational skills while confined.

Address the behavioral problems of confined youths by developing and implementing a level system and associated programming that prepares youths for progressively increased responsibility and freedom in the community and that promotes healthy life choices.

To provide a support system to ensure that services are gender‐specific, culturally sensitive, language appropriate and tailored to fit the individual needs to the youths.


Programming 2: Reading Scores

Percent of youths whose reading scores increased between admission and discharge


Programming 6: Vocational Programming

Percent of youths whose records indicate that they received the vocational skills programming prescribed by their individual treatment plans


NDTAC Recommendations for PbS/CJCA Surveys

Include education as a heading/category of its own, separated from the programming goal

Creating three education standards

1. General education

2. Special education

3. Vocational education Include additional items that focus on:

Special education services Advanced student outcomes Educational transition Teacher qualifications


Each of the seven areas are formatted in the following manner:

Area: XXXGoal

XXX1 Performance Standard

PbS Standard Layout


Rates, frequencies, numbers that show change in status,

occurrence or prevalence.

Practices expected to produce higher quality

outcomes based on available research,

consensus of professional opinion

and past PbS experience.

Policies, forms, tests used to describe or perform practices.


Standard 1: General Education

Outcome Measures Expected Practices Processes•Percent received a math/reading test at admission.•Percent received the educational programming prescribed by their individual treatment plans.•Percent of non-English speaking youth who have treatment plans written in the appropriate language.•Percent whose individual treatment plans have monthly progress notes.

•Receive math/reading tests to determine grade level within the 1st day•Trained and qualified staff applies the math/reading tests and interpret findings•The education program is provided 12 mos a year and for the number of hours per day specified by state law.•The education records contain info from their most recent school, arrived within 14 days•The facility uses aggregate and summary education data to develop a plan to improve education programming.•Youths held in isolation receive education programming and materials.•Written individual treatment plans are based on the results of reading and math tests and the education, social skills and vocational skills assessments.

•Written math/reading tests that help determine grade-level exist•Math/reading tests are age-appropriate and externally normed and validated.•Facility policy dictates that written individual treatment plans address education, social skills and vocational skills.•Case-level data on youths’ education status, needs, and performance are aggregated at least annually.•Facility policy provides for the education for youths held in isolation.•The provision of educational materials for youths held in isolation is stipulated in facility policy.•A record of materials provided to youths held in isolation exists.

Educational Programming: Currently in PbS



Outcome Measures Expected Practices Processes•Length of average school day (in hours)•Average class size for:oGeneral CurriculumoSpecial EducationoVocational

•Percent of high school-level courses from which credits are transferrable to community high schools.•Number of hours per week the school library is available to students.

•The school day is 6 hours exclusive of lunch and count related activities.•Age-eligible students are able to earn high-school course credits that can be transferred to local community high schools.•There is an active school library staffed by a full or part-time librarian.•Male and female residents have access to the same educational opportunities.

Educational Programming: Recommendations




•Percent of youths confined for over 6 months whose math/reading scores increased between admission and discharge.

•Youth are re-assessed in math and reading every six months and at discharge.

•Facility policy dictates that youths are re-assessed in math and reading.

Student Academic Outcomes: Currently in PbS




•Percent of age-eligible students earning a high school diploma in the most recently completed school year.•Percent of age-eligible students earning a GED in the most recently completed school year.

•Youth are tested in math and reading every 3 months.

Student Academic Outcomes: Recommendations




•Rate of percent of minority staff to percent of minority youth.

Teachers/Staff: Currently in PbS




•Percent of teachers that are certified in the area in which they are teaching.•Percent of teachers with any certifications.•Percent of ALL school staff that have college degrees.

•Students are taught by State Department of Education (SDE) credentialed teaching staff or students are taught by teaching staff that meet the highly qualified teaching standards set up by NCLB•A guidance counselor is a part of the educational staff.•A transition coordinator is a member of the educational staff.

Teachers/Staff: Recommendations



Outcome Measures Expected Practices Processes•Percent of youth who have an institutional education file in the school office.•Percent of youth enrolled in an educational program that issues a report card.•Percent of youth admitted whose educational records are requested within 3 days of admission.•Average number of days between a youth’s arrival and when the school record arrives from community school district.•Average number of days between a youth’s discharge and when the facility school records were released (mail/fax/hand carried) to his community school or district.•Percent of individual facility re-entry plans (individual treatment plans) that reflect educational re-entry activities/priorities/goals.

•Educational records are maintained on youth while enrolled in facility school, including the issuing of report cards every grading period.• The facility school establishes formal communication with the community school youth last attended.•Records are transferred from the facility to the new school within 7 days of discharge.•Future educational plans are considered by facility case management staff as demonstrated by youth’s re-entry plans.

Transition/Reintegration: Recommendations


Standard 2: Special Education


•Education and program staff sign off on the written individual treatment plans and/or the individual education plan.•Copies of IEPs are distributed to the staff responsible for implementing them.

•Facility policy dictates that individual education plans (IEPs) are developed for appropriate youth as mandated by federal education regulation.

Currently in PbS


Standard 2: Special Education

Outcome Measures Expected Practices Processes•Percent of students arriving with IEPs developed within last 12 months.•Percent of students needing new or updated IEPs based on intake assessment. •Percent of IEPs completely implemented within 10 calendar days of the IEP meeting. •Percent of IEP meetings held.•Percent of parents sent IEP notification letters of IEP meetings.•Percent of IEP meetings in which parents attended (in-person or by phone).•Percent of educational transitional services identified on the youth’s IEP that are coordinated with the youth’s institutional treatment plan.

•Existing student IEPs are reviewed for their currency and adequacy.• There exists an active child search/child find staff/committee to determine which youth are educationally disabled upon entry to the facility.•The facility has access to the state-wide special education database/network.•An IEP meeting is held within 30 days of the youth’s enrollment in facility school.•Parents are invited to their child’s IEP meeting•Parents provide input to their child’s IEP.

Recommendations


Standard 3: Vocational Education


•Percent of youths who received a vocational assessment by qualified staff.•Percent received the vocational skills programming prescribed by their individual treatment plans.•Percent completed a vocational skills curriculum.

•Youths receive a vocational assessment within 30 days after admission•Trained and qualified staff apply the vocational assessment and interpret the findings•Vocational skills classes are held as scheduled.

•A written vocational assessment for youths exists•The written vocational assessment is age-appropriate and externally normed and validated.•An age-appropriate vocational skills curriculum exists.

Currently in PbS


Standard 3: Vocational Education

Outcome Measures Expected Practices Processes•Percent of vocational courses that follow an established vocational education curriculum that is recognized by the state education agency.•Percent of youth enrolled in recommended vocational programming as determined/recommended by his vocational assessment. •Percent of vocational coursework hours that are transferrable to a career college or secondary vocational program of instruction. •Percent of youth in vocational programs that earned transferrable credit hours.

•Students are afforded vocational programming comparable to vocational programs operated in the community or community school.•Students receive appropriate vocational instruction that is aligned with interests and/or aptitude.

•A written policy/procedure for admission to vocational programs exists.•A written policy on the type of vocational offered: a secondary school model as offered in community schools, or a vocational/technical school model as offered in community/technical colleges.

Recommendations


Recommendations for PbS/CJCA Youth Surveys

Programs/Education• I am in class (6 hours or more), five days per

week.• I have what I need to complete my lessons in

my classes.• I have received a copy of my most recent

report card.• The school library is available for me to use

during school hours.



Classes and Teachers• Overall, the courses here are challenging.• Overall, the teachers here care about my

education.• Overall, the teachers here help me if I don’t

understand something in my lesson.



Transition• I have talked with a school staff member about

my educational plans when I am released.• School staff encourage me to get a high

school diploma.• School staff encourage me to get a GED.


PbS Website and Email Address

www.pbstandards.org

[email protected]

Availability of Trial Facility to experience PbS website design, data collections, outcome measures and reports.

1 © 2009 council of juvenile correctional administrators performance-based standards (pbs) for...

Documents