2012 qa report

Quality

Assurance

Report 2011-2012

Quality Assurance Overview 3

Service Output Statistics 4

File Audits 5

Serious Occurrences and Client Complaints 6

Service Wait Times Community Clinic 7

Referral Source Questionnaire 9

Client Feedback: Community Clinic Consultation Interview 11

Community Clinic Client Satisfaction Blitz 13

Brief Child and Family Phone Interview (BCFPI) 15

Child and Adolescent Functional Assessment Scale (CAFAS) 17

Client Feedback: Community Clinic Groups 18

Client Feedback: Prevention and Early Intervention Groups 20

Client Feedback: Ontario Early Years 21

Client Feedback: Special Needs Resourcing 23

Day Treatment Programs 24

Residential Programs 27

Highfield Community Enrichment Project 29

Family Group Conferencing Project of Toronto 31

Family Group Conferencing Ontario Provincial Resource 33

Toronto Preschool Speech and Language 35

Teaching and Training 37

TABLE OF CONTENTS

Background

The Quality Assurance (QA) process provides a means of reviewing and assessing service delivery and

administrative functions. Quality Assurance is a continuous process involving a cycle of evaluation and

improvement. It is a multi-faceted process emphasizing client-focused elements such as service output,

service access, client satisfaction, client outcomes and resource utilization as well as administrative

functions. This process provides the climate for policy change and service improvement.

Quality Assurance Structure

All Centre Quality Assurance activities are overseen by a Quality Assurance Committee (formed in September 2009).

The Quality Assurance Committee is chaired by the Director of Research. The Committee contains a minimum of four additional members and must include at least one person who has knowledge of the following areas of the Centre: Residential, Day Treatment, Community Clinic, and Central Administration.

The Quality Assurance Committee reports to the Directors’ Group and meets at minimum four times per year. During fiscal 2011-2012, the QA Committee met five times although two of those meetings contained only Clinical Records Subcommittee topics.

Reporting of Quality Assurance Activities

Many Quality Assurance reports are generated monthly (waitlist summaries, intake stats), or quarterly

(total clients served statistics).

A Quality Assurance Report is compiled annually which summarizes the Quality Assurance activities

for the fiscal period. This report is presented to the Management Group, the Directors’ Group and the

Centre’s Board of Directors. It is then distributed to all staff.

In order to facilitate ongoing communication regarding QA activities and the dissemination of QA

findings, QA is now a standing agenda item for all Team Meetings. Minutes of the QA Committee are

circulated to all Managers and Directors. Members of the QA Committee provide critical links to all

staff teams. Program Evaluation and Client Feedback is regularly provided to Teams through reports

and presentations.

Fiscal 2011-2012 Activities

Revised Centre file audit procedures and templates

Revised CYSIS reports to provide more detailed information to Clinic staff and managers

Executed the Community Clinic Client Satisfaction Blitz

Revised intake and clinical records processes and began drafting manuals

Created administrative procedures for new Centre programs and Clinic Group Treatment Programs

Quality Assurance Overview

Service Output Statistics 2010-2011

Service Output Statistics 2011-2012

Community Clinic Total Served

Total Unduplicated Clinical Clients 808

Total Workshop and Group Participants 890

Family Group Conferencing Total Served

Total Referrals 67

Total Clients Served 112

Toronto Preschool Speech and

Language (GHC Site Only)

Total Referrals Total Client

Contacts

Total Unique

Clients

GHC Site Only 205 2,470 416

Highfield Community Enrichment Project Total Served

Hours of direct service 11,181

Number of children served 1,736

Number of families served 2,020

OEYC Total Served

Number of parent/caregiver visits 9,400

Number of children served 1,150

Number of child visits 10,175

Number of parents/caregivers served 1,300

Special Needs Resourcing (Calendar Year) Total Served

Special Needs Resource Consultant 49

Intensive Resource Support 18

Number of professional/parent workshops 4

Total number of workshop participants 18

Direct Program Consultations 5

Residential Programs Total Served

Libby’s Place 19

Boys House 12

Day Treatment Programs Total Served

School Program 47

Clear Directions 11

Programs that retain clinical client files must have a regular audit process. The Community Clinic, Family

Group Conferencing, Special Needs Resourcing and Toronto Preschool Speech and Language undergo an

audit three times per year. Human Resource files are audited twice yearly. Residential files undergo an

external audit by MCYS as part of their annual licensing review.

Human Resource

The HR department was restructured in July 2011 following the resignation of the HR Specialist. The

administrative responsibilities of the HR Specialist position were combined with the responsibilities of the

Payroll position to create a new position HR/Payroll Administrator. No Human Resource File Audits were

conducted during the 2011-2012 fiscal year. It was determined an HR software program would be beneficial

to the centre, both for file audit purposes and to assist with streamlining the HR/Payroll workload

duplications. The implementation of the new software will begin in Sept 2012 and include an electronic

audit process. As a result of these changes, new Human Resource File Audit procedures will be required.

Community Clinic

The Community Clinic audit involves both closed and open files. The audit includes files that receive

Outpatient services as well as those that terminate at the stage of Consultation Interview and Short Term

Intervention. Community Clinic audits were performed in May 2011, October 2011 and February 2012.

The Community Clinic audit is a peer audit involving all clinicians. In May 2011, 35 files were audited and

14 files had deficiencies. In October 2011, 31 files were reviewed and 10 files had deficiencies. In February

2012, a total of 40 files were reviewed and 15 files had deficiencies. Following each audit, staff were shown

results and deficiencies were corrected where possible. Audit results were also discussed during team

meetings and addressed in supervision where required.

Special Needs Resourcing

Special Needs Resourcing audits involve closed files only and due to the small caseload in this program, all

closed files are audited. Audits took place in May 2011, October 2011 and February 2012. In May 2011,

one file was audited and no deficiencies were found. The October 2011 audit involved 9 files with 4

deficient files (8 deficiencies in total). In February 2012, 5 files were audited and 3 files were deficient (7

deficiencies in total). These deficiencies were addressed with staff and in one case the staff had left the

Centre.

Speech and Language

The Speech and Language audit involves 10 files (active and/or closed) randomly chosen by the program

manager. Speech and Language files were audited in May 2011, October 2011 and February 2012. In

total, 30 files were audited and 1 file had a deficiency that was corrected. The remaining 29 files were found

to be in complete compliance with the standards outlined in the audit forms.

Family Group Conferencing

The FGC audit involves closed files only. Two random files are selected for each coordinator from all the

closed files for the period. FGC audits took place in May 2011, October 2011 and February 2012. In May

2011, a total of 12 files were audited. Two files were deficient (total individual deficiencies two)). In

October 2011, 13 files were audited and deficiencies were noted in 1 file (11 deficiencies total). In February

2012, 8 files were audited and 3 files were deficient (3 total deficiencies). The FGC Team reviewed the

audit results and deficiencies were addressed with staff. In one case, deficiencies were due to a staff health

condition.

File Audits

Resolution of Client Complaints

The George Hull Centre has a formal policy for addressing complaints. All complaints are taken seriously and considered a high priority at all levels of the organization. The process of resolving complaints involves full written documentation and follow-up procedures. The goal is to resolve the issue in the spirit of cooperation with the complainant.

Due to the confidential nature of the issues, complaints are not reviewed by the Quality Assurance Committee. All complaint files are reviewed by the Executive Director and the Director of Research and Program Evaluation annually. During 2010-2011, a total of 7 formal complaints were received. In each case, complaints received documentation and thorough follow-up in accordance with the Centre’s policies.

Serious Occurrence Reporting

Serious Occurrences refers to a formal classification of incidents or events which must be reported to the Ministry of Children & Youth Services. Service providers who deliver any direct service to children and youth under the Child and Family Services Act (CFSA), and children under the Day Nurseries Act (DNA) are required to report all serious occurrences to the Ministry within 24 hours. In 2011, there were 38 serious occurrences. Three serious occurrences were reported for the Day Treatment program and the remaining occurrences were for Libby’s Place (25) and the Boy’s House (10). The high number of serious occurrences in part reflects Libby’s Place participation in the Quicker Access program (2 beds). The Quicker Access beds are available to child welfare programs in Toronto so that they have ready and speedy access to children’s mental health beds. These residents tend to have limited family involvement and the staff are not able to pre-screen these clients for appropriate fit or willingness to participate in a treatment program. The mandated nature of many of these admissions has resulted in extreme acting out behavior including running away, self harm and damaging property. In the case of all serious occurrences, an individualized plan was implemented to address the clinical needs of the clients involved. A total of 79% of reports were submitted within the 24 hour reporting period. Late reporting for the remaining occurrences was due to email issues and reporting delays during the Christmas break. The program staff, particularly new staff, have been reminded of the need to make submissions on time.

Serious Occurrences and Client Complaints

Background

The Centre is committed to providing the best

possible response time to client requests for service.

Due to limited resources, clients often experience a

wait time for services.

Monitoring wait list times and designing wait list

management strategies are important quality

assurance priorities for the agency.

Reporting on wait times for services is a complex

task because clients may wait at multiple points as

they move through stages of treatment. Wait times

do not only reflect limited service availability and

capacity. Clients may be offered services at an

earlier time but not be able to accommodate these

arrangements into their schedules.

In 2011-2012, Intake Services completed a total of

490 intakes.

Intake Department Response to Call Times

The Centre monitors the response time to intake

calls. For each fiscal year, a total of 4 months are

randomly selected for response time calculations.

In fiscal 2011-2012 only three months were

completed. The fourth month was not done due

to a change in intake worker. Compiled results

are presented in Figure 1.

The percentages in Figure 1 are based on 657

calls. A total of 78% of calls were returned the

same day and 91% of calls were returned within

24 hours (this latter percentage equals that of

2010-2011).

Wait Time for Intake

Every effort is made to provide an intake as

quickly as possible. Figure 2 displays the

aggregate wait times between the initial client

phone call and the completed intake (excludes

Shared Care). Wait-times in 2011-2012 are very

similar to 2010-2011.

Service Wait Times Community Clinic

Figure 2 - Client Wait Times for a

Completed Intake

0%

10%

20%

30%

40%

50%

60%

70%

Same day 19% 9%

1-7 days 58% 66%

8-14 days 16% 14%

15-21 days 5% 5%

22-30 days 2% 3%

Over 30 days 1% 3%

2010-2011 2011-2012

Figure 1 - Intake Response Time for

Calls

0%

20%

40%

60%

80%

100%

Same Day 78%

1 13%

2 2%

3 2%

4 1%

Over 4 days 4%

2011-2012

Wait Time for First Service

Figure 3 presents wait-time for first service for Community Clinic clients who received an intake between April 1, 2011 and March 31, 2012 (excluding clients who had residential and day treatment as first service). Of the 458 cases, 66 clients (14%) withdrew with no service and 6 clients are still waiting for services. The remaining 386 cases received services at the Clinic. Wait times are presented in Figure 3. Almost a third were seen within 2 weeks and 67% were served within 30 days (higher than 2010-2011 when this figure was 51%). The higher figure is likely due to new programs in which waitlists do not accumulate internally (Supervised Alternative Learning and School Focused). Waitlist Management

The Centre remains committed to providing timely service to all families. The Centre employs an “active waitlist management strategy” which involves the following activities:

Use of data management systems to support intake and clinical staff to track the presenting needs of clients so that they may be informed of opportunities for treatment

Employing a triage system to assign priority to cases based on high clinical needs after consultation with the multidisciplinary team

Regular reports to assess response times and determine the time it takes us to complete intake assessment and to provide families with service

Weekly Intake Committee meetings where files are reviewed by a team to ensure that the most optimal service is offered to each family.

Distribution of waitlist statistics and snapshot reports to management and senior management levels

Finally, the Intake Service Coordinator and the Manager of the Community Clinic have had numerous opportunities to disseminate information to community partners about the range of services being offered at the Centre and how they can best support families to access these services. Community partners welcomed the access to our Short Term Intervention Program, psychiatric consultations, our Family Wellness Series of groups, as well as trauma assessment and treatment programs.

Service Wait Times Community Clinic

Figure 3 - Wait Times For First Service -

Community Clinic

0%

5%

10%

15%

20%

25%

30%

35%

40%

0-14 days 23% 31%

15-30 days 28% 36%

31-60 days 36% 21%

61-90 days 9% 8%

Over 90 days 3% 4%

2010-2011 2011-2012

Note: Figure percentages do not equal 100% due to rounding

Background

As part of our accreditation requirement, the Centre completed a Referral Source Survey in the Spring of

2011. Centre Teams were asked to produce a list of referral sources for their programs, this list was merged

with Referral Source data for an 18 month period extracted from the central data system (CYSIS). A total

of 279 questionnaires were mailed and 58 responses were received (response rate 22%). Below is a

breakdown of respondents by type. The largest referral source category was doctors.

Between 61% and 69% of respondents agreed that they understood the referral process, that the intake

department was responsive to their requests and that the referral process was easy. About 20% of

respondents did not reply to the questions on the intake and referral process (see Figure 4). Those that

George Hull Centre: Referral

Source Questionnaire

Organization Total

Received

Percent

Child Welfare 9 15.5%

School/School Board 4 6.9%

Hospital, private practice, walk in clinic,

health centre / organization 36 62.1%

Child Care Agency 3 5.2%

Children's Mental Health Agency 3 5.2%

Community Organization 1 1.7%

Multiple entry 2 3.4%

Total 58 100.0%

0%

10%

20%

30%

40%

50%

60%

Figure 4 - Experience with the referral process

Strongly agree 12% 10% 9%

Agree 52% 59% 52%

Disagree 12% 7% 16%

Strongly disagree 3% 2% 5%

No answ er 21% 22% 19%

I understand the

process for making

the referral

Intake responsive to

my referrals

I found the referral

process easy

95% of respondents said they would refer to the GHC in the future (5% no response).

Referral sources noted many strengths of the Centre including: comprehensive range of services, flexibility,

strong client focus, professionalism, and dedication to child and family needs. Referral sources would like

to see development assessments, more psychiatry, walk-in clinic, services delivered in schools settings, more

services for teens and for those with special needs.

About 10% of referral sources mentioned the need for shorter wait-times. A considerable amount of the

feedback was related to improving communication e.g. stay in contact with practitioners, put things on web

page, more outreach and dissemination of information. Some referral sources lacked an awareness of some

Centre services. Management staff has discussed how to incorporate these changes. The Centre’s new

website design will fill many of these gaps.

George Hull Centre: Referral

Source Questionnaire

0%

10%

20%

30%

40%

50%

60%

Figure 5 - How the GHC is doing


Agree 21% 28% 55%

Disagree 53% 43% 22%

Strongly disagree 10% 16% 7%

No answer 5% 5% 3%

The GHC keeps me

informed of relevant

program developments

I know where to go or how

to be updated on the GHC

programs

I know which type of

clients to refer to the GHC

Background

Clients who are assigned to the Community Clinic for services receive a Consultation Interview

appointment to assess their needs. At the end of this interview, each client is asked to fill out a

questionnaire designed to rate how easy it was for clients to connect with services, get information, and

have their needs assessed.

Response Rates

In 2011-2012, a total of 209 consultation interviews were conducted by the Community Clinic and a total

of 138 completed questionnaires were received. This represents a response rate of 66%. Not all clients

respond to all questions. In this fiscal year many clients left forms unfinished. There was a high rate of

missing responses (13-20%) particularly the for questions on “process” and “progress”.

Ease of Connecting to Services

84% reported that it was “easy” for them to find out about services and how to connect with them. 11% said the process was “not too easy” and only 1% of clients found the process “very difficult” (4% missing).

84% reported that they did not have to keep calling different people to get the information and support they needed and 86% said they did not have to repeat the story of their situation too many times before getting help.

Involvement in the

Process

The majority of clients

said their choices and

preferences were

heard and understood

(79% “yes”, 8%

“somewhat”). One

respondent disagreed

and the rest did not

provide a response.

76% of clients felt

they were well informed about what was going to happen next and 11% reported that they were

“somewhat” informed about what was going to happen next (13% missing).

Client Feedback:

Community Clinic Consultation Interview

Yes Somewhat No Missing

2011-2012 67% 17% 2% 14%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Figure 6 - We got advice and recommendations that were useful for us

Support While Waiting

When asked if they were getting the support they needed while waiting for services to be available,

two thirds (57%) of respondents said “yes”, 20% said “somewhat” and 8% said “no” (17% missing).

Perceptions of Progress

83% of clients felt that progress was being made to get their family what they need. Only 2 clients

said “no” (16% missing).

In general, the Consultation Interview Questionnaire shows that, at this initial point of service, clients are

fairly satisfied with the wait times for consultation and ease of connecting to services. The qualitative

statements show that clients felt the staff were caring, helpful professionals who were very easy to talk to.

It was clear that clients felt very good about the initial experience, but several were very concerned with

the long wait time after consultation.

The missing responses and some of the qualitative feedback suggests that the Community Clinic and the

QA Committee should review the Consultation Interview form to determine if all of the questions are

appropriate. Several clients felt it was not possible to report on the questions given that they were just

beginning service.

Client Feedback:

Community Clinic Consultation Interview

“It is the beginning of our involvement with the Centre, but I feel hopeful for the

first time in a long time. I know they can help us and I am grateful.”

0%

10%

20%

30%

40%

50%

60%

Figure 7 - In an overall general sense, how pleased are you with

the service you received

Very satisfied 51%

Satisfied 28%

Dissatisfied 1%

Very dissatisfied 1%

Missing 20%

Background The Community Clinic has monitored Client Satisfaction for many years. This has primarily taken the form of a paper/pencil mail survey to closed clients. Historically, response rates for this survey rarely exceeded 20%. Between 2005 and 2009, response rates for this survey began to decline (2009 response rate 14%). Changes in mail survey processes and on-line response options failed to increase the response rates. In early 2011, the Community Clinic and the Research Department discussed options to obtain better feedback. The decision was made to do a Client Satisfaction Questionnaire (CSQ) Blitz during the month of November.

Scope The primary target for the CSQ Blitz was clients receiving long term Outpatient services. Clients who were receiving Outpatient services, Day Treatment services or Psychiatric services during this time period were included in the sample. Clients at consultation and receiving Short Term Intervention and group services were not included as these programs are evaluated in other ways. Intensive Child and Family Services (ICFS) clients were excluded because a quadrant-wide evaluation project was beginning in January 2012.

Results

87 CSQs were returned. This represents 68% of clients who had appointments during the two week period and 35% of the total active clients in service.

All respondents indicated that they were comfortable receiving services in English and no respondents indicated that that needed language support.

Overall, respondents felt that the location of the Centre’s services was convenient and accessible (75%

“yes, definitely”, 20% “yes, somewhat”). Only 5% replied “no, not really”.

Respondents indicated that services were “definitely” (71.3%) or “somewhat” (26.4%) available at times that were convenient for them Less than 3% said “no, not really”.

89% of respondents reported that they “definitely” felt welcome and comfortable and 10% said they “somewhat” felt welcome and comfortable.

A large percentage of

respondents felt that the Centre was helpful in recommending other services (47% “very helpful” and 25% “somewhat helpful”). A total of 20% felt it was too soon to respond and only 5% said we were “not really that helpful” in this respect.

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Figure 8 - We were served by knowledgeable and

competent professionals

Yes, definitely 87.4%

Yes, somewhat 11.5%

No, not really 0.0%

No, definitely not 1.1%

Knowledgeable and Competent

Professionals

Community Clinic CSQ Blitz

The majority of respondents reported that “yes, definitely” (79%) or “yes, somewhat” (20%) that the staff recognized the strengths of their family and child.

Overall, a high number of

clients reported that they would come back to our Centre (71% “definitely” and 23% “I think so”). Two clients (2%) reported “no, I don’t think so”.

Qualitative feedback was

overwhelmingly positive,

particularly with respect to

the knowledge and

professionalism of the staff and the useful solutions. A large number of clients felt the staff were caring,

and provided a safe, confidential and non-judgmental environment. Eight clients objected to long wait

times. Other comments included additional services (3) - (e.g. occupational therapist, more psychology),

concerns with progress (3), better appointment times (2), better building and facilities (2), an office in

Toronto (1). One

client mentioned initial

intimidation and

another client was

uncomfortable with the

two-way mirror.

There was a high level

of staff engagement

with this method and a

much larger response

rate was obtained using

this method rather than

the mail survey.

Future CSQ

evaluations will expand

the percentage of the

active client population

and include clients who

have recently exited from service. In January 2012, the George Hull Centre joined a MCYS funded

project to explore best practices for obtaining client satisfaction data. The Community Clinic and the

Research Department will incorporate information from this working group into future evaluation efforts.

Community Clinic Client Satisfaction Blitz

Felt involved in

making plans and

goals for treatment

We felt comfortable

asking questions

about the treat-

ment and/or

medication

Our cultural values

and experiences

were considered by

the GHC staff

Yes, definitely 73.6% 79.3% 83.9%

Yes, somewhat 20.7% 13.8% 6.9%

No, not really 2.3% 1.1% 0.0%

No, definitely 0.0% 0.0% 0.0%

No response 3.4% 5.7% 9.2%

0%

20%

40%

60%

80%

Increasing Hope

for Future

Suggesting

Useful Solutions

Understanding

Problems

Helping Your

Child

Figure 9 - How helpful have we been in...

Background

The Brief Child and Family Phone Interview (BCFPI) is a standardized intake interview being used in

children’s mental health centres across Ontario for youth between the ages of 6 and 18 years of age.

Administering this tool involves asking caregivers questions about their children’s behavioural and

emotional adjustment, and overall child and family functioning. The BCFPI is not used with children

under the age of six.

Completion Rates

The Centre began reporting monthly statistics on BCFPI completion rates. In the 2011-2012 fiscal year

there were a total of 435 clients eligible for BCFPI and the completion rate was 87%.

Reasons for non-completion are listed below. These categories were further refined in CYSIS based on data for the fiscal year.

BCFPI Profiles 2011-2012

The BCFPI reports produce a problem prevalence

percentage for a variety of categories. These values reflect

the percentage of clients with t-scores of 70 or above. The t-

scores are standardized to population norms where 98% of

the population would score below 70. A score of 70 or above

puts these children within the top 2% of the

population in terms of difficulties.

Figure 10 (next page) presents the

prevalence percentages.

Almost 60% of clients were in the

clinical range for Global Functioning

and half were in the clinical range for

Social Participation.

Roughly 40% of clients scored in the

clinical range for Managing Moods.

The lowest percentages were found in

the areas of Conduct (14%) and Separation (22%).

Brief Child and Family Phone Interview (BCFPI)

BCFPI Completion Rate

Total # eligible 435

Total # completed 380

Percent complete 87%

Reason for Non Completion Total

BCFPI completed < 1 year 6

Child with provider < 6 mos 3

Parent/Informant Refused 9

Language Barrier 15

Client withdrew request for services prior to

completing BCFPI 10

Other (e.g. Clinical decision, completed at

another agency, admin error ) 12

Atten

tio

n/

Imp

uls

ivity

C

oo

pera

tive

Co

nduct

Exte

rnaliz

Sep

ara

tio

nM

an

ag

ing

An

xie

tyM

an

ag

ing

Mo

od

sIn

tern

aliz

Ttl 6

MH

D

om

ain

sS

ocia

l P

art

icQ

uality

of

Rela

tio

ns

Sch

ool

Part

ic.

Glo

bal

Fun

ctio

nin

g

2009-2

010

34%

40%

21%

40%

19%

24%

38%

33%

41%

51%

34%

39%

50%

2011-2

012

32%

36%

14%

34%

22%

28%

39%

36%

39%

51%

25%

35%

59%

0%

10%

20%

30%

40%

50%

60%

70%

CAFAS Background

The Child and Adolescent Functional Assessment Scale (CAFAS) is a clinician-rated scale that is being

used in children’s mental health centres across Ontario for youth 6 through 17 years of age. The scale

assesses 8 areas of child functioning (role performance in school, home, and community, behaviour

towards others, substance use, mood, self-harm, and thinking), and 2 areas of caregiver functioning

(material needs and social support). Behaviour in these areas is rated as severe, moderate, mild, or

showing no impairment. Lower scores on the CAFAS indicate higher levels of functioning.

CAFAS for 2011-2012

Treatment Outcomes

In 2011-2012 a total of 86 Outpatient clients received an exit CAFAS evaluation. Full pre-post treatment

scores were available for all of these clients. Decreases in CAFAS scores represent improved functioning.

For the 2011-2012 data, three clients withdrew from treatment, 20 clients were classified as “treatment

interrupted” and 63 clients were classified as “treatment accomplished”. Comparisons of CAFAS change

are presented in Figure 9.

79% of clients in the treatment accomplished category had lower scores at exit. This figure is 55% for

clients who treatment was interrupted.

62% of the cases where

treatment was

accomplished show a

CAFAS improvement of

20 pts or more.

The percentage of clients in

the treatment interrupted

category who had higher

CAFAS scores at exit is

three times greater than

cases in the treatment

accomplished category

(30% versus 10%).

Child and Adolescent Functional

Assessment Scale (CAFAS)

Treatment Interrupted Treatment Accomplished

Higher Score at Exit 30% 10%

No Change 15% 11%

Lower Score at Exit 55% 79%

Lower Score at Exit by

20pts or more30% 62%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Figure 9 - CAFAS Outcomes by Closed Reason

Help I have Teens!!

Help I have Teens!! is a group designed for parents to learn how to improve their relationships with their

teenage children. The group teaches parents how to set reasonable limits, how to communicate effectively,

and how to prepare their teen for independence. The group was held for six sessions between January 31,

2012 and March 6, 2012. There were a total of 11 participants and 7 completed feedback questionnaires

(response rate 64%).

71% of the respondents rated the overall quality of the group as “excellent” (29% rated it as “good”).

The majority of respondents said that since being in the group, the way they view their teen’s behavior had

changed (86% “a lot”).

Participants were asked what they liked best about the group. Four respondents liked learning that they were

not alone and one parent felt relief from sharing. Respondents also liked the role plays, homework, learning

about the developmental stage, and how they need to change. Respondents requested handouts, more role

plays, and scenarios about out-of-control behaviour. As well, respondents suggested a follow-up group to

discuss results and an interactive email site for discussions.

Turning Down the Heat

“Turning Down the Heat” Art Therapy Group works with children between 7-12 years old who are having

difficulty coping with their anger. The group was held three times during the fiscal year and group feedback

was collected for all sessions. In addition to client satisfaction, an MSW student worked with the group leaders

to revise the logic model, compile program materials and create a binder of standardized measures that might

be useful in program evaluation for the upcoming year. A total of 18 participants completed a Client

Satisfaction Questionnaire, representing a response rate of 82%.

All child respondents said the group leaders knew a lot about anger and how to help them (78% of participants

said “yes, definitely”). When asked if they were happy with what they learned in the group, 72% of

respondents answered “yes, definitely” and 22% answered “yes, somewhat”.

Many child respondents (67%) found the concept of “when anger comes to visit” helpful. One participant said

it was helpful because it helped them to talk about their problems instead of yelling them and one participant

said the ‘recorder trick’ was helpful in avoiding fights.

83% of respondents said they liked learning about the tricks to deal with anger.

89% of respondents said they could use the tricks they learned in the group when anger comes to visit in

the future.

All parents felt the group helped them to understand anger triggers and gave them useful strategies to help

manage their own anger. The majority of respondents (64%) said since being in the group the way they viewed

their child’s behavior has changed “a lot” and 36% said it had changed a “a little”

79% said the group was helpful in improving their child’s behavior and 58% said the group had helped

improve the behavior of other children in the family.

The majority of suggestions for program improvement were related to increased program time and more one-

on-one support for families.

Client Feedback: Community Clinic Groups

Cognitive Behaviour Therapy Anxiety Group

Cognitive Behaviour Therapy (CBT) is a brief evidence-based intervention for treating anxiety in children. The Centre runs a CBT group for children experiencing issues with anxious/nervous feelings, thoughts or physical sensations. This group is divided into two sections – one group for the parents and one group for the children. Both parent and child must attend the group. Evaluation of the intervention includes a parent and child feedback questionnaire. Results of standardized measures of anxiety administered to group participants are not presented in the report due to difficulty in obtaining complete and reliable data. The group had two sessions ending December 2011 and May 2012. A total of 19 children and 19 parents participated in these groups and 15 children (79% response rate) and 18 parents (95% response rate) completed feedback questionnaires. Children rated the group as friendly and welcoming (67% “yes definitely” and 33% “yes somewhat”).

93% of child respondents agreed that they learned how to calm themselves when anxious and also how

to reward themselves when they have coped with an anxious situation.

73% of child respondents said that as a result of the group they were able to cope with their anxiety a

“little bit” better and 27% said they were able to cope with their anxiety “a lot” better.

All children agreed that being in a group made them feel supported. They all liked the use of art in the

group and the majority found the art helped them to understand what they were learning, release their

stress and help them talk about their feelings. Several children said they learned that they were not alone

with anxiety. Suggestions for improvement were more art (5), longer group (2) and more parent-child

activities (1). Parent feedback is provided in Figure 10.

Client Feedback: Community Clinic Groups

0%

10%

20%

30%

40%

50%

60%

70%

80%

Figure 10 - How Helpful was the CBT Parent's Group in:

Very helpful 72% 61% 39%

Helpful 28% 28% 56%

Somewhat helpful 0% 11% 6%

Not helpful 0% 0% 0%

Teaching strategies for

encouraging your child to face

their fears & worries

Prov iding relaxation

strategies to your child to use

in anxious situations

Help family dev elop parenting

approach for dealing with

your child's anxiety

Kindergarten Readiness

The Kindergarten Readiness Program is designed to meet the needs of families who feel their children

(between the ages of 3.5 to 5 years) require extra supports before entering the school system. The group

consists of up to 10 children who may be

presenting with developmental delays and/or

social or emotional struggles that could

potentially interfere with school adjustment. This

program enables children to develop and practice

the necessary school readiness skills in a

structured, safe, warm and nurturing group

environment with a low staff to child ratio (1:3).

Kindergarten Readiness held five sessions in

2011-2012. Four sessions received client

satisfaction questionnaires (39 participants).

Eighteen completed questionnaires were received

which represents a response rate of 46%.

A high majority of caregivers agreed that their

child is less anxious about starting

kindergarten because of this program, and

that they were less anxious as well (see Figure

11).

All respondents said that their child was more comfortable being away from them (72% “strongly agree”).

A total of 94% said their child was better prepared for school as a result of the program. All respondents

felt their children had learned new skills (83% saying “strongly agree”, 17% “agree”).

Nursery School Program

The Nursery School Program offers a structured, caring and warm group experience to young children

between the ages of 18 months to 3 years. These children may be displaying developmental delays, anxiety

separating from parents, and/or struggles in large group structures. Through staff support, the children

develop social skills such as turn taking, language skills, self help skills, and cognitive and motor skills. They

also develop an increased sense of self which leads to a healthy independence from parents.

The Nursery School Program held nine sessions in 2011-2012. Seven sessions received client satisfaction

questionnaires (68 participants). Thirty-four completed questionnaires were received (response rate 50%).

All respondents agreed or strongly agreed that the program was friendly and welcoming (91% “strongly

agree”). All of the caregivers felt their child had learned new skills in the program (71% “strongly agree”

and 29% “agree”).

All respondents said that they would recommend the program to a friend (91% “strongly agree”). Most

respondents agreed that their child is more comfortable being away from them as a result of the program

(79% “strongly agree”, 15% “agree”, 6% did not answer this question).

Parents observed many positive changes including improved confidence, language skills, social skills,

interest in activities.

Parents suggested both programs could be improved by increasing the number of days it is held per week.

Client Feedback:

Prevention & Early Intervention Groups

Figure 11 - Kindergarten Readiness Anxiety

Rating

0%

20%

40%

60%

80%

100%

Strongy agree 72% 78%

Agree 22% 17%

Disagree 6% 6%

Strongly disagree 0% 0%

Child Less AnxiousParent Less

Anxious

Background

The George Hull Ontario Early Years Centre offers free universal programming to the community for children from birth to age six and their caregivers. Programs range from structured parent/caregiver education sessions to drop-in programs. Programs are offered six days a week, during traditional and non-traditional hours.

Evaluation Structure

Prior evaluations have been done on a monthly schedule. In fiscal 2011-2012 the OEYC had an opportunity to participate in an evaluation process designed by the Family Resource Programs of Canada. Participants had the opportunity to complete feedback forms during a three-week period in November 2011.

Respondents rated their Centre experiences highly (see Figure 12). Almost all respondents felt welcomed and accepted when they came to the centre (89% “strong agreement”). Respondents said they were treated with respect by staff members (92% “strong agreement” 6% “moderate agreement”), and the centre does its best to be welcoming to the diverse groups of people who live in the community (81% “strong agreement” and 17% “moderate agreement”). The majority of respondents said that since coming to the Centre they have become more aware of the services and resources available in their community (50% “strong agreement” and 18% “moderate agreement”).

Areas for Growth

The distribution of responses also highlighted a few places where there is opportunity for growth. These included the following:

The creation of opportunities to involve parents and caregivers in decision making about the programming and operations of the Centre.

To help parents and caregivers deal more effectively with the day to day challenges encountered as a family.

Client Feedback: Ontario Early Years

Figure 12 - Participant Experience

0%

20%

40%

60%

80%

100%

Strong agreement 69% 83%

Moderate agreement 22% 17%

A little agreement 3% 0%

No agreement 0% 0%

Cannot say/ n/a 3% 0%

Missing 3% 0%

It is possible for me to

participate in programs

and activities

Staff & services are

available when I need

them

Respondents

reported positive

impacts on their

parenting as a result

of participating in

the Centre’s

programs (see

Figure 13).

Respondents

reported being more

aware of activities

that are appropriate

for their

child/children (72%

“strong agreement”

and 22% “moderate

agreement”) and

using activities at

home that were

learned at the

Centre (61%

“strong agreement”

and 25% “moderate

agreement”).

Participant Comments

Participants were asked “how has this program made a difference for you or your family?” The answers given

most frequently by respondents were that the program provided opportunities for social interaction for the

children. Respondents said that this interaction helped their child to learn to share, to play, and to improve

their interactions with other children. Respondents also said that the program helped them to meet, connect,

and share with other parents and caregivers. Respondents reported that the program helped them to develop

a routine, schedule, and provided structure to the day.

Participants were asked “what would you like us to do differently?” No patterns emerged from the many

suggestions provided. There were comments about the schedule of activities, such as to schedule programs

around typical routine times for infants and toddlers, to open earlier, and to start snack time earlier. However,

most comments were unique and varied. Respondents suggested offering coffee and tea during the morning

programs, to keep the website up-to-date, to have a larger space, more bathrooms, to provide more

educational toys, and to have a program for pre-school aged children where they can learn letters and

numbers.

Client Feedback: Ontario Early Years

Figure 13 - Since coming to the centre:

0%

10%

20%

30%

40%

50%

60%

70%

Strong agreement 44% 58% 47%

Moderate agreement 28% 33% 33%

A little agreement 6% 6% 11%

No agreement 11% 0% 6%

Cannot say/ n/a 8% 3% 3%

Missing 3% 0% 0%

Feel more confident

as a

parent/caregiver

More aware of what

to expect child to do

at their age

Use strategies for

guiding child's

behaviour

“The children look forward to coming each day and have learned to interact and share with other children.”

Background

The Special Needs Resource Consultant works with children who attend licensed group care settings and home child care. Consultation includes activities such as creating program plans for individual children; consultation on programming, customized workshops for providers and parents or information sessions. The Intensive Resource Support Worker is available to licensed child care programs to support the inclusion of children with complex and/or intensive developmental, social, emotional or behavioural needs.

Child Care Centre Feedback

Each year questionnaires are mailed to child care centres that receive services from both these programs. In

February 2012 questionnaires were mailed to 18 centres. A total of 15 completed forms were received (83%

response rate) and several contained missing data. Seven centres had used Special Needs Resourcing in the

past year. All respondents who had used services found them to be “helpful” or “very helpful.” Child

Specific Referrals and Program Development were rated highly, although several centres did not respond to

these questions (see Figure 12). All the centres who had used the services “definitely” had their needs met

and said that they would “definitely” use the services in the future.

Suggestions for program improvement included providing suggestions and feedback, strategies on how to

help children, and to offer workshops in the evening. One centre suggested offering an informal session at the

beginning of the school year to increase parent’s awareness of services and roadblocks to watch out for so

that parents may be more open to accepting outside help.

Intensive Resource Support A total of 3 child care centre questionnaires were received.. In all cases, workers “strongly agreed” that the

IRS staff shared information in a clear, direct and supportive manner and that IRS staff helped everyone

involved to openly discuss concerns. Two centres “strongly agreed” that as a result of services, child care

staff are better able to meet the child’s needs and

have learned new skills and ideas to use with

other children. One centre did not answer these

questions. One centre suggested the service

could better meet their needs by extending the

service to a longer period.

Program Review

The City of Toronto, Children's Services is

undertaking a review of the "Every Child

Belongs Model" that has been in place for the

past 7-8 years. This model focuses on the

Special Needs Resource Consultation and the

Intensive Resource Support service. The model

review is aligned with maintaining the key

priorities of City Council (customer service and

accountable and transparent service) and talks to

the 2010-2014 Service Plan goal to reconfigure

and adjust services for children with special

needs as required during the introduction of the

Full Day Early Learning Program.

Special Needs Resourcing

Figure 14 - Yearly Child Care Feedback

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Very H elpful 14% 29% 86% 14%

H elpful 43% 29% 14%

So mewhat H elpful 14%

N o t H elpful

D id N o t Use/ B lank 43% 43% 72%

T raining P ro g D evC hild

R eferralsIR S

School Program The School Program offers academic programming for up to 24 youth through three Section 23 classrooms. The youth who attend the school exhibit acute or chronic behavioral, emotional, learning or developmental difficulties. The data below reflects the evaluation completed for the 2011-2012 academic year. In the 2011-2012 academic year, 35 students entered the School Program and two students had two admissions during the academic year, for a total of 37 admissions to the School Program.

CAFAS Ratings Table 2 displays CAFAS change for males and females in the School Program. The CAFAS tool assesses 8 areas of child functioning (role performance in school, home, and community, behaviour towards others, substance use, mood, self-harm, and thinking). Total scores reflect overall functioning and lower ratings represent improvement. Data presented is for clients in the program not less than 60 days and with complete data (28 clients).

Additional Findings School anxiety in the School Program increased slightly by the end of the year. Pre-post data shows

that the percentage of students who were “not at all anxious” decreased from 14% to 0%. The

number of students who were “extremely anxious” increased from 3% to 14%.

69% of students earned academic credits during the school year and 49% of students earned 3 or more

credits. 11 students (31%) did not earn any credits during the school year. 10 of these students were

enrolled for 47% or less of the total academic year

Student Feedback

At the end of the 2011-2012 academic year, a survey was conducted to obtain feedback from the students.

A total of 35 participants were eligible for follow up at the end of the School Program. Twelve participants

provided feedback, representing a response rate of 34%.

The majority of respondents felt that the School Program was a welcoming and comfortable place

(58% “yes, definitely” and 33% “yes, I think so”), that the program staff listened to them (67% “almost

always), and treated them in a polite and friendly way (67% “yes, very much” and 33% “yes, kind of”).

The majority of respondents said the program helped them to better manage the problems that

brought them to the program (33% “yes, it helped a lot” and 42% “yes, it helped a little”).

Day Treatment Programs

Table 2: School Program - Total CAFAS Score Change Indicators By Gender

Females Males

Percentage with Higher CAFAS scores at exit. 38% 40%

Percentage with No Change in CAFAS total score at exit. 8% 7%

Percentage with Lower CAFAS score at exit. 54% 53%

Percentage with Lower CAFAS score at exit >= 20pts. 46% 53%

Percentage with Lower CAFAS scores at exit >=40pts. 31% 13%

92% of respondents were satisfied with the amount of help they received from the program.

67% of respondents said that the program helped them to meet their personal goals and 25% said the

program did not help them to meet their personal goals. 42% of respondents said the program helped

them to meet their academic goals and 50% felt it had not helped them in this area. 42% of

respondents said that the program helped them to understand their family differently and 42% felt it

had not helped them to understand the family differently.

42% of respondents said that

the daily program at school

was “excellent”, 50% said it

was “good” and 8% said it

was “fair”.

75% of respondents would

recommend the program to a

friend in need of similar help

and 25% would not

recommend the program.

92% of respondents were

happy with the program

(42% “very happy”).

The majority of respondents

said that the staff support,

academics, discussion

groups, physical activities,

and peer support were

helpful (see Figure 15).

Positive Aspects

of the Program

Participants were asked what they liked best about the School Program and what was good about the

program. They mentioned the supportive and empathetic staff and the support system. Other positive

comments were the opportunity to learn and not to be left out of everything, the exercise program, and the

students.

Suggestions for Improvement

Participants were also asked what they didn’t like about the School Program and what could be improved.

Respondents mentioned placing a greater emphasis on academics, improving the math program, fewer

interruptions to the physical education program, increased respect for students’ private and personal space,

and improving peer interactions. A few respondents said that the program did not need improvement.


Figure 15 - How helpful were the following:

0%

10%

20%

30%

40%

50%

60%

70%

80%

Very helpful 75% 25% 33% 50% 33%

Helpful 8% 33% 25% 25% 33%

Somewhat helpful 17% 42% 33% 17% 17%

Not helpful 0% 0% 0% 8% 17%

Missing 0% 0% 8% 0% 0%

Staff

Support

Academi

cs

Discussi

on

groups

Physical

activities

Peer

Support

Clear Directions The Clear Directions program is designed to assist youth, 18 years and under, who are struggling with

serious substance abuse, family difficulties, and mental health issues. The data below reflects the evaluation

completed for the 2011-2012 academic year. In the 2011-2012 academic year, 9 students were enrolled in

Clear Directions.

CAFAS Ratings Table 3 displays CAFAS change for males and females in Clear Directions. The CAFAS tool assesses 8 areas of child functioning (role performance in school, home, and community, behaviour towards others, substance use, mood, self-harm, and thinking). Total scores reflect overall functioning and lower ratings represent improvement. Data presented is for clients in the program not less than 60 days and with complete data (9 clients).

Additional Findings School anxiety ratings were available for 7 of the 9 students. School anxiety increased for Clear

Directions clients overall. 22% of clients were rated as “fairly anxious” or “very anxious” at the start of the year. This percentage increased to 67% (56% “fairly anxious” and 11% “extremely anxious”) by the end and the number of students who were not anxious at all decreased from 22% to 0%.

56% of students achieved 3 or more academic credits and one student achieved 6 credits during the

program year. 2 students (22%) did not earn any academic credits. These students were enrolled in the program for

32% or less of the total academic year. Student Feedback Only one client feedback form was received from Clear Direction in this fiscal year. This person indicated some positive changes as a result of the program, including academic achievement and decrease in substance use. No suggestions for program improvement were offered by this client.


Table 3: Clear Directions - Total CAFAS Score Change Indicators – By Gender

Females Males

Percentage with Higher CAFAS scores at exit. 33% 17%

Percentage with No Change in CAFAS total score at exit. 33% 17%

Percentage with Lower CAFAS score at exit. 33% 67%

Percentage with Lower CAFAS score at exit >= 20pts. 33% 67%

Percentage with Lower CAFAS scores at exit >=40pts. 33% 33%

The main Quality Assurance activity for the Centre’s Residential Programs is the formal licensing program

conducted by MCYS on an annual basis. This rigorous process involves determining compliance with

regulations through multiple sources: staff and resident interviews, file audits, and physical inspections.

Changes that came out of the licensing process were addition of a school reports section in all clinical case

files, staff review of the policy regarding dispensing of medication, staff review of the enhanced

serious occurrence policy, and a change in the procedures for providing medications to clients during home

visits. In June 2012, the Centre was found to be in compliance with all licensing requirements and was granted

a regular license with no term and conditions, expiring July 2, 2013.

Program evaluation for the Residential Programs continues to be a challenge due to the small sample and

complex nature of the population and the intervention.

Boys House

In the 2011-2012 fiscal year, four boys were discharged from the Boys House. One of the boys was discharged

to his own apartment with child welfare support. This young man had completed high school and was going

on to college. Two of the boys were discharged to their family homes and the fourth left the program to live

with family friends. Of the four discharges, two were planned and two were not. The unplanned discharges

came about due to their behaviour in the program. Despite this, they were both planning to move from the

residence within a few months, and had made many gains within the program. The boys who left in a planned

way had done extremely well in the program. Some of the outcomes present in this group were:

Significant increase in school attendance.

Resolution of difficult family issues that allowed for better connection to family and in some cases ability

to move home.

Increased skills in managing severe anxiety symptoms and a dramatic decrease in overall mental health

symptoms allowing for wider engagement in everyday and challenging activities,

Scores on the Child and Adolescent Functional Assessment Scale (CAFAS) showed clinically significant

improvements in overall functioning for 3 of the 4 clients.

Client Feedback

Client satisfaction forms are made available to clients for feedback at the plan of care meetings every 6 months.

Only three completed forms were received during the 2011-2012 fiscal year. Based on the form dates, it was

determined that these forms represented the views of three distinct clients.

The forms represented mixed views. All three agreed that family were encouraged to visit the program. All

three said they did not have a clear discharge plan. Two felt involved in discharge planning and the third did

not. Two boys felt the staff “usually listened” and one said “sometimes”. There was a range of responses in

terms of rating the helpfulness of various parts of the program with some saying ‘very helpful”, another saying

“a little” and one saying “not at all”. Two said the program had helped to meet their academic goals and one

said this was not applicable. All agreed that the Boys House was helping to improve their social relations with

peers.

Residential Programs

Libby’s Place

At Libby’s Place, 16 girls were discharged during the fiscal year 2011-2012. Of these 16 girls, 8 of them were admitted through the Quicker Access Program. The Quicker Access Program accepts clients from the child welfare system who child welfare believe are in urgent need of a mental health bed. This isn’t intended to be a crisis bed program, but the assessment process is forgone and admission generally happens within 24 hours of referral, often within hours. The Quicker Access clients at Libby’s Place have had a higher turnover than many of our clients. We believe this is due to the mandated nature of the treatment for these clients. We have participated in the program for a year and plan to move one of the Quicker Access beds out of Libby’s Place to the Boys House so each program will have one Quicker Access bed. The hope is that the Quicker Access client will adapt to the existing culture within the program and better engage in the treatment. All 8 Quicker Access clients who were discharged in the reporting period were in the program less than 4

months, and 50% of them were in the program less than 1 month. Two of these clients showed an improved

CAFAS score at exit. Program turnover was higher for regular clients, partly due to the instability generated by

the Quicker Access protocol. Of the 8 non-Quicker Access clients, four clients had planned discharges; two

returned to their family homes and two moved on to semi-independent programming. The two girls who

moved home continued to attend the School Program for additional support. Four of the eight clients

showed an improved CAFAS score (see table below).

CSQ

Only two client satisfaction questionnaires were received during the fiscal year. One reason for this is that

many clients did not remain in the program long enough to receive these forms which are given every six

months at plan of care meetings. The program is currently reviewing their process for obtaining feedback

from clients.

Residential Programs

CAFAS Change Category Quicker Access

Clients

Non Quicker Access

Clients

# with CAFAS scores showing a decline in

functioning

1 2

# with CAFAS showing an improvement in

functioning

2

4

# with CAFAS that showed no change in

functioning

1 1

# with no reliable CAFAS due to stay in

program less than one month

4 1

CAFAS Results—Libby’s Place

Background

The Highfield Project is a Better Beginnings, Better Futures site based at Highfield Junior School in Rexdale. The area is a high-need, low-income neighbourhood in Etobicoke that is populated by struggling newcomers. The Project offers multiple programs to children and families.

Highfield Summer Enrichment Program (SEP)

The Highfield Summer Enrichment Program (SEP) is a kindergarten readiness program that has run yearly since the summer of 1995. During the 2011 summer session, approximately 80 children took part and 55 caregivers completed a feedback from at the end of the program.

All respondents found the program friendly and welcoming, said that their child looked forward to coming to the program, and that they would recommend this program to a friend.

Respondents also said that they read or sing more with their child because of this program (52% “strongly agree” and 42% “agree”).

Almost all respondents supported the

idea that their child was better

prepared for school as a result of the

program (86% “strongly agree”, 14%

“agree”).

88% of caregivers reported that their

children were less anxious/nervous

about starting kindergarten because

of the Highfield program.

90% of caregivers reported that they

were less anxious about their children

starting school as a result of the program.

95% of caregivers said they made new friends because of this program and 97% of respondents said

their child had made new friends because of this program.

Program Strengths and Improvements Caregivers said that the strengths of the program were the teachers, that their children made friends, learned things such as shapes, counting, and how to recognize their name, and that their children are now prepared for kindergarten.

Highfield Community Enrichment Project

“I am confident she is ready to be alone in the class”

Figure 16 - My child is better prepared for

school as a result of this program

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Strongly agree 81%

Agree 19%

Kindergarten Readiness and Over a Cup of Tea

The Kindergarten Readiness program at Highfield provides preschool children the opportunity to experience attending school in a non-intimidating way to facilitate their transition from home to school life. Children attend the program while caregivers attend their own program, Over a Cup of Tea, that provides a socialization and learning opportunity for parents.

A total of 12 feedback forms (response rate of 71%) were received for Kindergarten Readiness. As shown in Figure 17, all respondents indicated that their child learned new things, had improved social skills and became more independent.

Respondents suggested that the Kindergarten Readiness Program could be improved by increasing the number of days it is held per week, by teaching table manners, and by having the children remove their hats and coats by themselves.

Almost all respondents reported positive benefits for caregivers for participating in the Over a Cup of Tea program.

92% agreed or strongly agreed that the program taught them new and important information and taught them parenting skills.

92% said the program helped them to make new friends and feel less isolated.

The caregivers in the program participated in a series of information sessions including—healthy eating,

child discipline, internet safety, job search, and preparing children for school. Qualitative feedback from

the participants indicated that they found these sessions to be very useful.

Highfield Community Enrichment Project

0%

10%

20%

30%

40%

50%

60%

Figure 17 - Kindergarten Readiness Program Outcomes:


Agree 50% 50% 42%

Disagree 0% 0% 0%

Strongly disgree 0% 0% 0%

No answer 0% 0% 0%

My child

learned new

things

My child's

social skills

improved

My child is

more

independent

Background Family Group Conferencing (FGC) is an alternative approach to working with and engaging families. The main objective of FGC is to give the extended family group (i.e., nuclear family, extended family, and friends) a meaningful voice in the decision-making process to ensure the safety and well-being of children who are at significant risk of or in need of protection from abuse and neglect.

Plan Development

The Family Group Conferencing Project of Toronto received 67 referrals in 2011-2012. A total of 43 conferences were held during the fiscal year and an agreed upon plan was produced for 43 conferences (100%). In total, 68 children were involved in these plans and for 64 of them (94%) the plan was for the children to remain with or be returned to kin.

Client Feedback

The project offers all family members and professionals the opportunity to fill out a Client Satisfaction Questionnaire at the end of the conference day. A total of 462 questionnaires were received during the fiscal year - 312 (67%) of the forms were received from kin participants and 114 (25%) forms were filled in by professionals. The remaining 36 (8%) forms either did not specify or were of an indeterminate

Family Group Conferencing Project of Toronto

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Figure 18 - Family Satisfaction with FGC Program

Yes, definitely 92.9% 76.7% 67.3% 86.9% 82.9% 91.2% 95.1%

Yes, somew hat 6.1% 19.1% 21.0% 11.8% 13.5% 6.5% 3.9%

No, not really 0.6% 3.6% 9.0% 1.0% 2.3% 1.3% 0.7%

No, definitely not 0.3% 0.6% 2.7% 0.3% 1.3% 1.0% 0.3%

Clear

Purpo

se

Adeq

Info

Every

one

there

Freed

om to

Expre

Freed

om to

Disagr

Cultur

al

Respe

Safety

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Figure 19 - Professional Satisfaction with FGC Program

Yes, definitely 97.4% 83.2% 48.7% 92.0% 93.8% 93.5% 94.6%

Yes, somewhat 2.6% 15.0% 37.2% 8.0% 6.2% 5.6% 5.4%

No, not really 0.0% 1.8% 8.0% 0.0% 0.0% 0.0% 0.0%

No, definitely not 0.0% 0.0% 6.2% 0.0% 0.0% 0.9% 0.0%

Clear

Purpo

se

Adeq

Info

Every

one

there

Freed

om to

Expre

Freed

om to

Disag

Cultur

al

Resp

Safety

The questionnaire addressed a variety of aspects of the FGC process (clear purpose, adequate information,

family members present, freedom to share opinions, freedom to disagree, cultural responsiveness, and

safety). (see previous page—Figure 18 for family responses and Figure 19 for professional responses).

There is a slight difference between families and professionals in their agreement levels for many areas.

Professionals were more likely than family members to select the higher level of satisfaction for all

dimensions except for “everyone who needed to be there was” and “I felt safe during the conference.”

Very few respondents expressed high disagreement with any area of the conference process. For all

questions except “everyone who needed to be there was”, the percentage of “no, definitely not”

responses was less than 5%.

Levels of satisfaction with the FGC coordinator

were high. Over 90% of family members and

professionals said the coordinator was definitely

“knowledgeable”, “well organized”, “respectful and

courteous” and able to “keep the group focused on

the purpose of the conference”.

Outcomes of the Process

Participants were asked about the outcomes of the

FGC process (see Table 4 for results).

The results show very high levels of satisfaction along

all dimensions.

There are not large differences between families

and professionals in their agreement levels. For the

most part, very few respondents expressed high

disagreement with any area of outcomes for the

conference process. For the majority of questions,

the percentage of “no, definitely not” and “no, not

really” responses was less than 5%.

All professionals and 99% of family members felt

the plan protects the child’s safety.

99.1% of professionals and 93.8% of family

members felt the FGC helped the family to get

along.

All professionals and 97% of family members said

they would definitely recommend FGC to others.

Family Group Conferencing Project of Toronto

Family Prof

Family made decisions

Yes, definitely 90.3% 91.0%

Yes, somewhat 8.7% 9.0%

No, not really 0.3% 0.0%

No, definitely not 0.6% 0.0%

FGC helped family get along





FGC helped family and

professionals get along





Decisions were respected





The plan protects the child's safety





The plan explains what is to be done





Table 4 - FGC Outcome Ratings

Background

At the provincial level, The George Hull Centre is mandated to lead the FGC Provincial Resource (PRG),

providing training, mentoring and consultation across the province. The Provincial Resource is

responsible for developing and managing a provincial roster of coordinators and mentors, a training

manual, and ongoing professional development activities.

This project ensures the integrity of the FGC service model through quality and consistency in training

and mentorship of new service providers.

Key Activities and Accomplishments

Completion of the revised FGC/Family Group Decision Making (FGDM) Coordinator Manual for

Ontario

Revision to the Basic and Advanced training slides

Participation in the organizing and delivering of the 1st Canadian Conference on FGC held in

September 2011 in collaboration with Ontario Association of Children’s Aid Societies (OACAS) and

the American Humane Association (AHA)

The program delivered a total of 977.9 hours of mentorship. As at March 31, 2012 the provincial roster

had 73 coordinators, 20 trainers and 19 mentors.

Youth in Transition Research Project

In March 2008, the Family Group Conferencing (FGC) Ontario Provincial Resource Advisory Group

identified the need to gain more knowledge about the use of FGC with youth in the Province of Ontario.

The review of outcome literature for youth involved in child welfare suggests that this group is at risk for

many negative outcomes. The literature also points to social support as a key variable in ameliorating

negative outcomes. Family Group Conferencing provides a promising intervention for youth because it

engages youth in a planning process for their future, encourages decision-making and youth voice, and

focuses on repairing relationships and expanding supports. Between 2008 and 2012, interview data was

collected from FGC coordinators and from youth at 6 months post-conference.

Although based on a small number of cases, almost all youth reported positive changes that emerged from

their experience with FGC. The conference process for the youth produced many changes including

affirmation and clarity on social support, improved contact and communication with family, increased

self-esteem, and various forms of practical support.

The coordinator data provides a rich source of information about working with youth in the FGC context

and generated many helpful suggestions that can be incorporated into FGC coordinator training materials.

Interim results of this study were presented to the Ontario FGC Provincial Resource Steering Committee

on March 5, 2012 and during the April 18, 2012 semi-annual FGC Ontario Provincial Resource

professional development day. The Provincial Resource has made a commitment to continue ongoing

training in this area by having workshops and training materials on how to provide FGC services to youth.


Ontario Provincial Resource

Basic Training

CSQs were collected at 5 of the 6 Basic Training Sessions held in fiscal 2011-2012. In total, 75 forms were

received (response rate of 93%)

The participants rated the trainers’ knowledge very highly. A total of 85% of the respondents said the

trainers’ knowledge was "Excellent," 13% said it was "Good," and 1% said it was “Fair”. A total of

88% said the trainer was "definitely" effective in helping them to learn more about FGC.

All participants said they would recommend FGC training to a colleague (71% “yes definitely” and

29% “yes, I think so”).

59% of participants were “very

satisfied” with the amount of

information they received in the

training and overall training

experience and 37% were “mostly

satisfied”.

Advanced Training

CSQs were collected at 5 of the 6

Advanced Training sessions held in

fiscal 2011-2012. A total of 71 CSQs

were collected, representing a response

rate of 86%.

78% of respondents said the

trainers’ knowledge was “excellent”

and 18% felt it was “good”.

Almost all respondents rated the

quality of training as “excellent”

(69%) or “good” (30%).

Most respondents (97%) indicated

they would be interested in

receiving additional training. Almost all participants said they would recommend training to a

colleague (79% “yes, definitely”, 17% “yes, I think so”). (see Figure 20, Column 2).

Suggestions for improvement included having longer training sessions, using more videos and more

role plays.

Orientation

14 CSQs were collected during the FGC Orientation Day held in fiscal 2011-2012. All respondents

said that the trainer was effective in helping them to learn more about FGC (36% “yes, definitely” and

64% “yes, somewhat”). 71% said they would consider using FGC in their work. Respondents said

they would use their experience from the orientation by keeping the service in mind when working

with families.


Ontario Provincial Resource

Figure 20 - After taking part in FGC Training

would you consider using the process in

your work?

0%

20%

40%

60%

80%

100%

No, definitely not 0% 3%

No, not really 1% 0%

Yes, I think so 21% 17%

Yes, definitely 75% 79%

No answer 3% 1%

Basic -

Use FGC in Work

Advanced -

Recommend FGC

Ministry Deliverables

Motor Speech Research Study The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.

Aravind Namasivayam, Ph.D. from the Dept of Speech-Language Pathology, U of T and The Speech &

Stuttering Institute. The study is aimed at (a) estimating the magnitude of treatment effects, (b) treatment

efficacy and (c) identifying the key factors that contribute to treatment effectiveness of motor speech

intervention for children presenting with speech-motor difficulties. The study is organized by the Ministry

of Child and Youth Services (MCYS) and approximately 52 specially trained clinicians from 32

Preschool Speech and Language (PSL) programs in Ontario, Canada will be participating. It is expected

that about 250 children (about 5 per clinician) with motor speech issues will be participating in this study.

Training for the TPSLS clinician who will participate in this study began in Feb 2011. Recruitment of

clients for this project will begin in the next fiscal year

Client Feedback Questionnaires

During 2009-2010 the Toronto Speech and Language team worked to design a feedback form for clients

exiting service. The feedback forms were administered to clients exiting service in the Spring of 2011

and results will be released in September 2011

More Than Words

The More Than Words Program is a parent training program for children who have Autism Spectrum

Disorder (ASD) and social communication delays. The More Than Words program ran from November

2, 2010 to February 8th 2011 and involved 9 children and their caregivers. Final evaluations were

completed by 89% of participants. All caregivers felt the child's communication had improved as a result of taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent". On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as “5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and getting kids together once a month.

Toronto Preschool Speech and Language Services


Motor Speech Research Study

The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.














More Than Words




completed by 89% of participants. All caregivers felt the child's communication had improved as a result of

taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how

satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent".

On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as

“5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with

kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and

getting kids together once a month.


Page 35

Background

The George Hull Centre is the West Quadrant Local Coordinating Agency for Toronto Preschool Speech and

Language Services (TPSLS), overseeing the services in seven different sites. TPSLS at the George Hull site

are provided in close collaboration with the Community Clinic multidisciplinary team and the Ontario Early

Years Centre (OEYC).

The program is collaboratively managed in partnership with the City of Toronto, providing community-based

speech and language services to children and their families before the children are eligible for senior

kindergarten. The range of services includes assessment, consultation and support to parents and staff in

child care settings, training for parents and professionals, and group or individual therapy. Speech and

Language Pathologists, Communications Disorder Assistants and Early Childhood Educators provide the

services of the TPSLS program.

* TPSLS provided a full day workshop to clinicians that focused on group interventions. Clinical quadrant discussions

continue to support group interventions as an effective intervention for certain communication disorders.

Speech and Language Outcome Measures

Effective December 2011, the Severity Rating Scales (SRS) that have been in use by Preschool Speech and

Language (PSL) clinicians since 2009 are no longer required. A workgroup of provincial PSL clinicians and

coordinators reported several persistent challenges

with the scales, including perception that the SRSs:

Do not capture changes in children with more

severe disorders;

Lack appropriate levels of regional and provincial

inter-rater reliability for staff using the scales;

Would require the development of additional

scales in order to capture change in certain

communication domains (e.g. fluency, emergent

literacy, dysphagia);

Do not capture changes in parent and caregiver

skills, despite this being a core mandate of the PSL

Program.

MCYS will replace the existing PSL Functional

Communication Questions (FCQs) with the

FOCUS©, beginning in the summer of 2012. The

FOCUS©, a tool developed collaboratively by a team

of researchers at the Bloorview Research Institute (led

by Prof. Nancy Thomas-Stonell), CanChild Centre for

Childhood Disability (led by Dr. Peter Rosenbaum) and Laurentian University (led by Dr. Bruce Oddson) has

been demonstrated to be a reliable and valid measure of communication participation for preschool children.

Program Deliverable Actual

45% of all initial assessments will be provided to children by 30 months of age 42%

100% of children who are discharged for the reasons "attending JK", "attending SK" and "attending

Grade 1" will receive transition plans

92%

Individual treatment with a speech language pathologist and individual treatment with a mediator together

will comprise no more than 35% of the total interventions provided *

23%

0 %

2 0 %

4 0 %

6 0 %

8 0 %

Figure 21 - How helpful have

we been in...

Strongly Agree 69%

Agree 25%

Somewhat

Disagree

6%

Strongly

Disagree

0%

I now understand my

child's speech and

language difficulties.


Motor Speech Research Study The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.














More Than Words




completed by 89% of participants. All caregivers felt the child's communication had improved as a result of taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent". On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as “5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and getting kids together once a month.



Motor Speech Research Study

The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.














More Than Words




completed by 89% of participants. All caregivers felt the child's communication had improved as a result of

taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how

satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent".

On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as

“5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with

kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and

getting kids together once a month.


Page 36

Client Feedback Survey

The Toronto Preschool Speech and Language Services (TPSLS) program conducted phone interviews with

caregivers of children completing a treatment cycle within the program from April 2011 to September 2011. A

total of 65 families participated in the program during this time and were eligible for the evaluation. Thirty six

caregivers provided feedback, representing a response rate of 55%.

Survey Findings

All respondents said they “almost always” or “usually” felt welcome and comfortable

94% said they would recommend the program to a friend

89% agreed that they were given accurate information about service wait times.

83% indicated their questions were almost always answered thoroughly (14% said “usually”)

81% felt the suggestions were almost always useful and 17% felt the suggestions were usually useful

Qualitative feedback from respondents showed a high appreciation for the staff. Staff were seen as caring,

professional and flexible. Caregivers liked the individual attention, and that programs were all located in the

same centre. Many respondents saw changes in their child’s skills and felt that they as caregivers learned as

well. Suggestions for program improvement included:

More notice prior to starting service

Shorter wait times for services

More sessions

Increasing upper age limit

Adjusting group size

“Approachable, flexible, down-to-earth and really focusing

on where the child is at. Very good service”

0%

20%

40%

60%

80%

Figure 22 - Outcomes

Strongly Agree 64% 80%

Agree 30% 11%

Somewhat Disagree 3% 3%

Strongly Disagree 3% 3%

No answer 0% 3%

My child's speech

and language has

I now have skills that

I can use to help my

Masters of Social Work Student Feedback

There were six MSW students at the GHC during 2011-2012 and all six students completed the evaluation All “agreed” or “strongly agreed” that they were provided with timely and appropriate constructive feedback and that staff were available to them for questions. Field instructors were seen as role models who facilitated achievement of the objectives of their respective learning plans. Five students “strongly agreed “ and one “agreed” that the placement provided the learning experience required to develop competency in this area of practice. The education seminars provided were highly regarded by the students. Areas of improvement were identified as more orientation to safety procedures and clinical recordings. Students also suggested the placement have additional opportunities to follow clients throughout the year, more contact with the short term cases, more opportunities to observe the work of other clinicians and additional group learning.

Child and Youth Work Student Feedback

A total of 5 CYW evaluation forms were received (response rate—71%). The feedback received was very

positive. All students “totally agreed” that they were valued as a team member. 80% of the students “totally

agreed” and 20% “somewhat agreed” that their experience increased their understanding of children and their

families . The students felt they were respected and able to share ideas with the team. 60% of the respondents

“totally agreed” that the program challenged their existing skills and allowed them to develop new ones (40%

somewhat agreed). All students “totally agreed” that their experience helped them to define future career

goals. All students “totally agreed” that they had an increased level of clinical knowledge of the clients and

their families. All agreed that they received regular supervision which was conducive to open communication.

Student suggestions for placement improvement included additional placement time, holding the student

support group meeting more frequently, having the supervisor directly in the classroom, and getting feedback

from other staff early in the program.

Psychiatry Rotation Feedback

The Centre receives a summary of the feedback from the University of Toronto on the residents’ experience

with their placements. Feedback was received in November 2011 and May 2012 from 2 of the 4 residents.

Areas of Strength

Good teaching – staff approachable, flexible, plenty of feedback and good multi-disciplinary rounds

Good breadth of cases and no difficulty getting cases

Good psychopharmacology

No difficulty getting psychotherapy patients

Good family therapy seminar/supervision and CBT

Community atmosphere

Focus on education (good balance between learning and service)

Flexible, e.g. off-site supervision

Supervisor that does 1 consult per week with young children (e.g. PDD)

Areas for Growth

No dynamic supervisors, difficult to find a case, need off-site dynamic supervision

Antiquated health records system

Improve scheduling, especially considering distance from core teaching

More physician-led rounds to focus on medical aspects of cases

Teaching and Training

2012 qa report

Documents

quality assurance reports

quality assurance qa

client feedback

quality assurance overview

quality assurance structure

qa committee

client complaints

client outcomes