otap data mf november 2016 · november 17, 2016 gayl bowser: [email protected] 2 guide informed...

OTAP Moving Forward Series: #2

Using Evidence

November 17, 2016

Gayl Bowser: [email protected] 1

Using Evidence Using Evidence Using Evidence Using Evidence to Strengthen to Strengthen to Strengthen to Strengthen Your AT ProgramYour AT ProgramYour AT ProgramYour AT Program

OTAP Moving Forward Series

November 18, 2016

11:30-12:30Instructor: Gayl Bowser

[email protected]

1

Discussion:Discussion:Discussion:Discussion:

:

What information and data do you currently collect about the work of your AT team?

How is it collected?

2

Three Types of EvidenceThree Types of EvidenceThree Types of EvidenceThree Types of Evidence

1. Data about general use of the intervention (AT Research)

2. Data about how AT works for an individual student

3. Data about what your team does

3

Using Research About Using Research About Using Research About Using Research About Assistive TechnologyAssistive TechnologyAssistive TechnologyAssistive Technology

Part 1:

Data About Use of the Intervention

(AT Research)

4


Using Evidence

November 17, 2016


Guide informed decision making

Guide purchasing

Set programmatic goals

Compare solutions

Why we need to know Why we need to know Why we need to know Why we need to know about researchabout researchabout researchabout research

5

1. Use of audio text alone

2. Use of text to speech software

3. Use of audio text + print text + training in note taking

What would you choose?

Example: How might you Increase Example: How might you Increase Example: How might you Increase Example: How might you Increase Reading Comprehension?Reading Comprehension?Reading Comprehension?Reading Comprehension?

6

300.320 (a) (4) A statement of the special education and related services and supplementary aids and services, based on peer-reviewed research to the extent practicable, to be provided to the child, or on behalf of the child, and a statement of the program modifications or supports for school personnel that will be provided to enable the child--

7

From Final Regulations:

Teacher surveys about knowledge, attitude, etc.

AT user surveys

Reviews/synthesis of research

Application research

Outcomes research

Types of ResearchTypes of ResearchTypes of ResearchTypes of Research

8


Using Evidence

November 17, 2016


Teachers who receive 40 or more hours of training about AT report that AT is essential to students’ daily activities.

Teacher who have not received training about AT report that AT is not important and are not confident in identifying and using AT with students.

◦Ashton, Lee & Vega (2005)

Survey: Need for trainingSurvey: Need for trainingSurvey: Need for trainingSurvey: Need for training

9

Students place importance on experiencing immediate benefit from use of AT;

AT was disliked if it interfered with peer relations;

The need to “take care of” the AT sometimes perceived as a burden;

Influential aspect: whether AT was accessible when needed and integrated into every day activities.

◦Hemmingsson, Lidstrom, Nygard. (2009).

Consumer SurveysConsumer SurveysConsumer SurveysConsumer Surveys

10

Reviewed 256 research articles about AT for VI

Found only two that used true research design. They were both single subject design.

They found, “A large percentage of the literature consists of anecdotal evidence of the impact of AT without evaluating the effectiveness of AT”.

◦ Kelly, S. & Smith, D., (2011). Impact of AT on Educational Performance of Students with

Visual Impairments.

Synthesis of Research: Synthesis of Research: Synthesis of Research: Synthesis of Research: Quality of Research in a specific areaQuality of Research in a specific areaQuality of Research in a specific areaQuality of Research in a specific area

11

AT not provided to majority of students with VI who could benefit from it.

AT more likely to be provided if parents are highly involved.

AT more likely to be provided if student is in residential placement.

◦ Kelly, S. (2009). Use of AT by Students with VI: Findings from a National Study.

Secondary Analysis Secondary Analysis Secondary Analysis Secondary Analysis SSSShows: hows: hows: hows: Big PictureBig PictureBig PictureBig Picture

12


Using Evidence

November 17, 2016


Can use of cognitive mapping improve grasp of critical information?

High school students with behavior outbursts and LD in reading

Use of Inspiration software changed scores on chapter quizzes from <15% to >75%

Decreased off task behavior and reliance on teacher for help

• Blankenship, T.L., Ayers, K. M., & Langone, J. (2005).

Application StudiesApplication StudiesApplication StudiesApplication Studies

13

Studied how students in special education are impacted by the inclusion of AT as part of an intervention;

Repeated-measures pretest–posttest quasi-experimental design;

AT’s contribution as an intervention strategy is greater than nine other possible intervention strategies.

◦ Watson, Ito, Smith & Anderson (2010)

Impact of ATImpact of ATImpact of ATImpact of AT

14

Augmentative and Alternative Communication-www.isaac-online.org/en/publications/aac.html

Assistive Technology-www.resna.org

Journal of Research on Technology in Education-www.iste.org

Journal of Special Education Technology-www.tamcec.org or jset.sagepub.com

Journals of your professional organizations

15

Journals with AT ResearchJournals with AT ResearchJournals with AT ResearchJournals with AT Research

16

MMMMore about ore about ore about ore about Research Research Research Research and how to and how to and how to and how to use ituse ituse ituse it

https://pantherfile.uwm.edu/edyburn/www/what/index.html


Using Evidence

November 17, 2016


Part 2:

Data About How AT Works for an Individual Student

17

www.educationtechpoints.org

http://[email protected]

Quality Indicators for the Evaluation of Effectiveness of AT Services

http://[email protected]

Evaluation of Effectiveness of AT Services

1. Shares responsibilities for data collection, evaluation, and

interpretation

2. Collects data on specific student achievement and related to goals

3. Includes quantitative and qualitative measurement of changes

4. Evaluates across environments during naturally occurring and

structured activities

5. Provides teams with a means for analyzing student achievement

6. Makes changes when indicated by data

7. Views evaluation as a dynamic, responsive, ongoing process

Student Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection Opportunity

Trial period plan

20


Using Evidence

November 17, 2016


Trial PeriodsTrial PeriodsTrial PeriodsTrial Periods

�Trial Period form

� Specifies the tool(s) on trial

�Describes the environment for use

�Dates of trial

� Indicators of effectiveness

�Identify staff/student training needs

�Identify staff person responsible for follow-up

21

Trial Period Plan

www.otap-Oregon.org

Results of Completed Trial Data

23

24


Using Evidence

November 17, 2016


Student Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection Opportunity

Trial Period Follow-up

25

Completed Trial Completed Trial Completed Trial Completed Trial

Scan

26

Student FollowStudent FollowStudent FollowStudent Follow----up up up up DocumentationDocumentationDocumentationDocumentation

Data/Work samples

What worked

AT considerations documented on the student’s IEP

What didn’t work

Feedback about the support you provide

27

Example Data Set:Example Data Set:Example Data Set:Example Data Set:Tools used by studentsTools used by studentsTools used by studentsTools used by students

28


Using Evidence

November 17, 2016


29

HIAT HIAT HIAT HIAT Online Online Online Online Effectiveness Effectiveness Effectiveness Effectiveness SurveySurveySurveySurveyHandoutHandoutHandoutHandout

Example Data Set:Example Data Set:Example Data Set:Example Data Set:Student changes as a result Student changes as a result Student changes as a result Student changes as a result of the trial Periodof the trial Periodof the trial Periodof the trial Period

Strategies were...

A. AT strategies not

implemented at this

time

B. AT trial stategies

ineffective

C. Some Trial stategies

effective

D. All trial strategies

effective

30

Student Outcomes of Student Outcomes of Student Outcomes of Student Outcomes of the Trial Periodthe Trial Periodthe Trial Periodthe Trial Period

�Student has or is developing use of a new tool or strategy

�Student does not require AT

�No changes for student

31

Data about changes in Data about changes in Data about changes in Data about changes in student performancestudent performancestudent performancestudent performance

�Quality of student work improved

�Student is more independent with identified task

�Productivity increased

�Time on task increased

�Time on task decreased

�Motivation improved

�Behavior improved

32


Using Evidence

November 17, 2016


Changes in Student Changes in Student Changes in Student Changes in Student BBBBehaviorehaviorehaviorehavior

33

If you don’t communicate the value of what you do in evidence- based ways, then no one can appreciate what you do.

Part 3:

Data About What Your Team Does

34

Commonly Used AT Teams DataCommonly Used AT Teams DataCommonly Used AT Teams DataCommonly Used AT Teams Data

Number of staff

Number of students served

35

Implementation Process Data

Training provided

Primary Contact Role

Hours of AT Team Support

Kinds of AT Support

36


Using Evidence

November 17, 2016


Keep a Team Data File!Keep a Team Data File!Keep a Team Data File!Keep a Team Data File!Be ready with answers!Be ready with answers!Be ready with answers!Be ready with answers!

How many students have you served so far this year?

What are the reasons that IEP teams request your services?

What types of services have you provided?

How many full assessments have you completed?

How much time do you spend with individual students?

How do your numbers for this year compare to last year’s numbers?

37

Opportunities for BuiltOpportunities for BuiltOpportunities for BuiltOpportunities for Built----in Data Collectionin Data Collectionin Data Collectionin Data CollectionHIAT Model of School SupportHIAT Model of School SupportHIAT Model of School SupportHIAT Model of School Support

School completes an online survey

describing

the type of support needed

Contact case

manager at school to

gather

additional relevant

information

Determine level of support required

Tool Consult

Phone Consult

SETT Meeting

Staff Training

Trial period with

training

Follow up/

documentation

38

Services Data Collection OpportunityServices Data Collection OpportunityServices Data Collection OpportunityServices Data Collection Opportunity

School completes an online referral form or survey describing the child and the type of support needed

39

40

Traditional Traditional Traditional Traditional Referral FormReferral FormReferral FormReferral Form


Using Evidence

November 17, 2016


An Alternative An Alternative An Alternative An Alternative Request for Support SurveyRequest for Support SurveyRequest for Support SurveyRequest for Support Survey

Reason for Request

What prompted you to make a request for AT services?

◦ (My idea, parent request, advocate request, administrator suggestion, central office suggestion, mediation or due process agreement, conversation with an AT Team member)

Information on the student for whom you are considering AT

◦ (Name, ID, grade, disability, program, reading level, math level, case manager)

41

OnlineSurvey

42

Questions about team’s Questions about team’s Questions about team’s Questions about team’s consideration/assessment skillsconsideration/assessment skillsconsideration/assessment skillsconsideration/assessment skills

How would you rate your team’s ability to meet and complete the following? (Scale: not comfortable, somewhat comfortable, comfortable, very comfortable)

◦ Identify and prioritize critical barriers to learning for a specific student

◦ Collaboratively problem solve as a team

◦ Identify a clear diagnostic question that can be answered during an AT trial period

43

Questions about team’s trial Questions about team’s trial Questions about team’s trial Questions about team’s trial period skillsperiod skillsperiod skillsperiod skills

How would you rate your team’s ability to do the following tasks related to AT trials?

(Scale: not comfortable, somewhat comfortable, comfortable, very comfortable)

◦ Identify available technology tools at your school and how they can support student needs

◦ Match potential AT tools to the identified student needs

◦ Develop a plan to use technology strategies to answer your diagnostic questions

◦ Ensure that staff and students involved can use the features of the tool(s) on trial for the identified student

◦ Use a team process to monitor the trial and make adjustments as needed

44


Using Evidence

November 17, 2016


Questions About Team’s Ongoing Questions About Team’s Ongoing Questions About Team’s Ongoing Questions About Team’s Ongoing Implementation SkillsImplementation SkillsImplementation SkillsImplementation Skills

How would you rate your team’s ability to do the following tasks related to ongoing AT implementation?

(Scale: not comfortable, somewhat comfortable, comfortable, very comfortable)◦ Use a team process to integrate technology tools into instructional

routines

◦ Document the rationale for the AT on the IEP and other student records

What is your knowledge of the following tools? (List of tools commonly used in the district)◦ (Scale: Do not know this well, know how this can help students, use

this regularly with students, can train other staff on this)

◦ List applications or other tools most commonly used in the district(s)

45

46

Agency Data from Your Agency Data from Your Agency Data from Your Agency Data from Your Online Request SurveyOnline Request SurveyOnline Request SurveyOnline Request Survey

Underlying Student Issues Identified by Teams.

�Math

�Language

�Spelling

�Reading

�Organization

�Behavior

�Composition

�Communication

47


Determine course of action with case manager.

48


Using Evidence

November 17, 2016


Initial ContactInitial ContactInitial ContactInitial Contact

Frame the question

Examples of items which you might request to better serve the process:

◦ Current levels of performance

◦ Written Productivity Profile (WPP)

◦ Protocol for Accommodations in Reading (PAR)

Determine course of action

49

Your Action OptionsYour Action OptionsYour Action OptionsYour Action OptionsProvide Resources

◦ Technology-options Handout

◦ Link to “Strategies for supporting struggling students in the classroom”

◦ Quick Guides

◦ Road Maps

◦ HIAT Help Blog

◦ Newsletter

Loan necessary tools when appropriate

Identify training needs

Attend IEP meeting or assessment meeting

Complete traditional multidisciplinary assessment

And……

50

Tracking the Actions Your AT Team Takes

• Led an assessment meeting

• Participated in assessment meeting

• Regular face-to-face with student

• Consult to IEP team

• Loaned equipment for trial period

• Loaned equipment for long-term use

• Training provided

• Consultation about implementation plan

51

Automated Referral: Automated Referral: Automated Referral: Automated Referral: Tracking services in new waysTracking services in new waysTracking services in new waysTracking services in new ways

52

Reasons for Request for Support School Contact

� Composition � Math Name:

� Handwriting � Language Delay Phone:

� Organization � Communication Email:

� Behavior � Mobility

� Spelling � Vision Tools Used/Added

� Reading � Hearing Tool #1

Services ProvidedTool #2

Tool #3

� Full SETT Meeting

Support

� Trial Period Consult Provided Level of Support

� Tool Consult Provided � Equipment Loaned

� Training Provided � Implementation Plan Consult


Using Evidence

November 17, 2016


Adapted from DeCoste, D. (2013) HIAT program data report

53

Intensity of services provided

Adapted from DeCoste, D. (2013) HIAT program data report54


Follow-up documentation

55

Interview feedback about Interview feedback about Interview feedback about Interview feedback about the support you providethe support you providethe support you providethe support you provide

◦ review goals of support relationship

◦ reach consensus on efficacy of process for the student

◦ Reach consensus on efficacy of process for staff

◦ decide to continue or discontinue support

56


Using Evidence

November 17, 2016


57

HIAT Online effectiveness SurveyHandout

Staff OutcomesStaff OutcomesStaff OutcomesStaff Outcomes

Staff learned our consideration and assessment and trial period process and will be able to complete it independently in the future.

Staff exposed to our consideration, assessment and trial period process but will need further support.

Staff learned new strategies to support the student.

Staff learned new strategies to incorporate and will be able to support multiple students in the future.

Consideration and Assessment process was not useful to the staff at this time

58

Common Issues Affecting Common Issues Affecting Common Issues Affecting Common Issues Affecting EffectivenessEffectivenessEffectivenessEffectiveness

• Strategies too dependent upon adult support

• Not enough opportunities to use strategies

• Strategies took too much time to implement

• Student did not use strategies appropriately or see them as useful

• Strategies did not result in improvement

• Strategies did not prove necessary

• Absenteeism

59

Staff OutcomesStaff OutcomesStaff OutcomesStaff Outcomes

Staff learned our consideration and assessment and trial period process and will be able to complete it independently in the future.

Staff exposed to our consideration, assessment and trial period process but will need further support.

Staff learned new strategies to support the student.

Staff learned new strategies to incorporate and will be able to support multiple students in the future.

Consideration and Assessment process was not useful to the staff at this time

60


Using Evidence

November 17, 2016


Staff outcomes as a result of SupportStaff outcomes as a result of SupportStaff outcomes as a result of SupportStaff outcomes as a result of Support

61

What are three types of data that you will plan to collect during this school year to help you understand and improve the effect that your AT team is having on AT use in your district?

MAKE A MAKE A MAKE A MAKE A PLAN !PLAN !PLAN !PLAN !

YOUR TURNYOUR TURNYOUR TURNYOUR TURN

62

63

Ideas? Questions? Comments?

otap data mf november 2016 · november 17, 2016 gayl bowser: [email protected] 2 guide informed...

Documents