Models of Practice

Lecture 7

A successful outcome begins with choosing the most appropriate AT for a person. How can we best do that?

Assistive technology is designed to provide functional benefits and to facilitate participation for a person with a disability (World Health Organization, 2002).

However, research shows that there is a high level of device abandonment, even with what appears to be a well matched device (M. J. Scherer & Craddock, 2002).

Studies on device abandonment, often explained by inefficient assessments and intervention processes (Judge, 2002; M. J. Scherer & Craddock, 2002), have led to the development of assistive technology specific outcome measures to evaluate the satisfaction and effectiveness of a device.

There is a lack of evidence- based procedures that are specific to assistive technology provision.

Although the International Classification of Functioning (ICF) was not specifically developed to guide assistive technology assessment, the literature shows that it lends itself as a descriptive model for the assistive technology assessment process.

ICF captures the complex aspects of the impact of assistive technology and its service delivery process and can assist the professional in decision-making (Bernd, Van Der Pijl, & De Witte, 2009).

When assistive technology is successful, it reduces or removes barriers, to allow the person to take part in activities (Jutai, Fuhrer, Demers, Scherer, & DeRuyter, 2005).

The ICF checklist assists the service provider to elicits what capabilities and limitations the user’s experience in activities and participation related domains.

Examples of relevant domains are: learning and applying knowledge; speaking; getting around inside and outside the home; self-care; interpersonal relationships; and social life etc

Matching people with assistive technology is complex because people’s expectations of and reactions to technologies are complex. Reactions are highlyindividualised. Scherer M. J. (2005). Assistive technology ineducation for students who are hard of hearing or deaf. Handbook of special education technology research and practice. Knowledge by Design. (2005). Whitefish Bay, WI.

Sometimes the evaluationis driven by a request for aspecific piece of equipment.In these instances, the focusis on the equipment, andthe student’s problem isnot identified.

Kurtz J. (2003). Assistive technology in schools: how do we make it work? OT Practice. Aug 18; 8 (15), 16-20.

Scherer, M., Jutai, J., Fuhrer, M., Demers, L. & DeRuyter, F. (2007). A framework for modeling the selection of assistive technology devices (ATDs). Disability and Rehabilitation: Assistive Technology, 2(1), 1-8.

Resources: * Family/Friends &

Significant Others

* Financial

Consumer

Assessment ofFunctional ATD

Need

 (‘Objective Need’)Knowledge and Information

Expectations

Personal Preferences and

Priorities

Provider  Assessment ofATD

Predisposition 

(‘Subjective Need’)

   

 Follow-Up

Trialling, Use and Realization of

Benefit

Environmental Factors Cultural and Financial Priorities

Legislation & PolicyAttitudes of Key Others

Personal Factors AT Decision-Making and Device Selection

ATD Selection Framework

Purpose of Outcomes measurement

Ensure good outcomes for individuals

Provide evidence for successful practices

Augment AT knowledge base Document need to funding and

policy makers

Consumer-Centered Outcome Measurement

to make informed choices to monitor how well solutions meet their

goals, preference and ongoing requirements to enable them to direct the process in

order to optimize their utilization on the solution.

An integrated approach which utilizes a range of mechanisms to provide consumers with adequate information

ENVIRONMENTAL

PERSONAL

Functioning & Disability

Activities & Participation

Body Functions &Structures

Context

DisabilityDisability

Environmental Factors

Personal Factors

Body Body function&structurefunction&structure

(Impairment(Impairment))

ActivitiesActivities(Limitation)(Limitation)

ParticipationParticipation(Restriction)(Restriction)

CONTEXT

INDIVIDUAL

The WHO’s ICF ClassificationQuality of Life = Performing Activities + Participating in Life Areas

QOL

Influences on Activities & Participation 

 

MilieuPolicies and mandatesFinancial/fundingProvider knowledgeAttitudes of others

Individual PredispositionResources and knowledgePersonal perspectives, prioritiesPrior experiencesExpectations

OutcomesDegree of AT usePerceived benefit/gain from use•Subjjective well-being/quality of life

CONTEXT

EnvironmentalFactors

PersonalFactors

It is no longer sufficient to show we have improved a person’s

functioning. We must show we have enhanced

participation.

Challenges to Evaluating Outcomes

•  AT is often provided as part of a number of interventions and it is therefore difficult to ascertain the degree to which the AT is responsible for the outcome

•  Difficult to define the expected outcomes

•  Consumer diversity and individualization makes comparisons difficult

We know what to measure, but how? With what tools?

PIADS for assessing the increase in the user’s sense of competence, self-esteem and adaptability

QUEST for assessing the person’s satisfaction with the device

MPT for assessing person-technology fit -- how well the device matches the needs, characteristics, preferences and expectations of the person -- and enhances participation

Plus others that were designed to address AT

PIADSPIADS 26 self-report items on a 7-point scale ranging from 26 self-report items on a 7-point scale ranging from

decreases (-3) to increases (+3). Items are: decreases (-3) to increases (+3). Items are: competence, happiness, independence, adequacy, confusion, efficiency, self esteem, productivity, security, frustration, usefulness, self confidence, expertise, skillfulness, well-being, capability, quality of life, performance, sense of power, sense of control, embarrassment, willingness to take chances, eagerness to try new things, ability to participate, adapt to activities of daily living and take advantage of opportunities.

3 scales, competence, self-esteem and adaptability3 scales, competence, self-esteem and adaptability

Jutai J & Day H. (2002). Psychosocial Impact of Assistive Device Scale (PIADS). Technology and Disability, 14, 107-111].

PIADSPIADS

+ Good psychometric properties after a slow start (contact + Good psychometric properties after a slow start (contact lens and eyeglasses use)lens and eyeglasses use)

- Quality of life is assessed with only 1 item and a separate Quality of life is assessed with only 1 item and a separate items exists for well-being items exists for well-being

- Some items lack face validity regarding impact on AT useSome items lack face validity regarding impact on AT use- Consumers report difficulty in distinguishing Consumers report difficulty in distinguishing powerpower and and

controlcontrol

QUESTQUEST 12 self-report items on a 5-point scale ranging from not at 12 self-report items on a 5-point scale ranging from not at

all satisfied (1) to very satisfied (5). Items are: all satisfied (1) to very satisfied (5). Items are: Dimensions, weight, adjustment, safety, durability, ease of use, comfort, effectiveness, service delivery, repairs & servicing, professional services, and follow-up

2 scales: Device and Service2 scales: Device and Service

Demers, L., Weiss-Lambrou, R., & Ska, R. (1997). Quebec User Evaluation of Satisfaction with assistive Technology (QUEST): A new outcome measure. In S. Sprigle (Ed.), Proceedings of the RESNA 97 Annual Conference (pp. 94-96). Arlington (VA): RESNA Press.

The only evidence based assistive technology specific model, developed to match the ICF and its checklist found in the literature, is the Matching Person and Technology (MPT) model

Bernd, et al., 2009; Karlsson, P (2006) ICF: A Guide to Assistive Technology

Decision-making University of Western Sydney

Matching Person with Technology

The MPT model explores assistive technology use and perceived quality of life/participation of predetermined assistive technology users and non-users. The foundation of the instrument is the user and their environments. It assists the assessment process as a collaborative decision-making tool designed to determine the most appropriate assistive technology solution for a given individual. Separate instrument for children and adults

Mapped on ICF Several instruments make up the MPT assessment package

with versions of each to be completed by the consumer and by the service provider. Depending on what is been assessed each scale can be used independently They include:

MPT assessment instruments

The Survey of Technology Use (SOTU) The Assistive Technology Device

Predisposition Assessment (ATD PA) The Workplace Technology Predisposition

Assessment (WPPA) The Health Care Technology Predisposition

Assessment (HCT PA) The Educational Technology Predisposition

Assessment (ET PA)

IMPT

The MPT was modified and re-validated for an Irish audience – Irish Matching Person with Technology

Impact of Assistive Technology on the quality of Life and participation, student self-esteem and autonomy of students (Craddock 2002)

Expanded to include subset on QOL & participation

45 students assessed using the IMPT, longitudinal study, pre and post

MPT (IMPT)

Environment – Educational Environment Subscale

Technology – Educational Technology Subscale

User-Capability Quality of Life Self-evaluation Educational Goal Additional information, transport, family

support etc

MPT & IMPT models

It was developed to address the environment, the person and the technology, factors that need to be considered when evaluating a person’s need for assistive technology

The MPT supports a collaborative partnership between the service providers and the user

The Assistive Technology Device Predisposition Assessment consumer form (ATD PA), a part of the MPT assessment battery, is compatible with ICF and measures the impact of technology using the ICF domains.

The ATD PA items ask the user to rate their predisposition to using the assistive technology that is being considered, to better match technology with the person and therefore minimize device abandonment. ATD PA is developed for adults

IMPT

Pre-test was used to capture the stage of technology experience, their quality of life, their degree of support and level of self-esteem before the assistive technology was introduced.

Post-test was used to measure these qualities after the client has used the technology for two years, in order to investigate if assistive technology had made a difference

MPT (Specifically the ATD PA)MPT (Specifically the ATD PA) It has 66 self-report items on a 5-point scale and yes/no It has 66 self-report items on a 5-point scale and yes/no

questions, all mapped to the ICFquestions, all mapped to the ICF

4 scales: Functional capabilities, Subjective well-being, 4 scales: Functional capabilities, Subjective well-being, personal factors, and person-device match with each item personal factors, and person-device match with each item mapped to the ICFmapped to the ICF

Scherer, M.J. (1989). The Assistive Technology Device Predisposition Assessment (ATD PA) Consumer Form. Webster, NY: The Institute for Matching Person & Technology, Inc.

Functional Abilities

ATDPA Section A: Abilities ICF Classification: Body Functions (b)________________________________________________________

1. Seeing b210 Seeing functions

2. Hearing b230 Hearing functions

3. Speech b3 Voice and speech functions

4. Understanding,remembering b144 Memory; b164 higher level cognitive

functions; b1670 reception of language 5. Physical strength/stamina b730, b735, b740 Muscle functions6. Lower body use b760 Control of voluntary movement functions

7. Grasping and use of fingers b760 Control of voluntary movement functions

8. Upper body use b760 Control of voluntary movement functions

9. Mobility b770 Gait pattern functions

Subjective Well-Being

ATDPA Section B. Well-Being, QOL ICF Classification: Activities & Participation (d)

_______________________________________________________________________

10. Personal care, household activities d5 Self-care; d630, d640 Household tasks11. Physical comfort & well-being b280 (pain)12. Overall health b4, b5, b6, b8 13. Freedom to go wherever desired d4 Mobility; d460 Moving around in different

locations, d470, Using transportation; d475 Driving14. Participation in desired activities d2 General tasks & demands; d9 Community,

social & civic life15. Educational attainment d810-d839 Education16. Employment status/potential d840-d859 Work and employment17. Family relationships d760, e310 Family relationships18. Close, intimate relationships d770 Intimate relationships, e320 Friends19. Autonomy, self-determination d177 Making decisions20. Fitting in, belonging d7 Interpersonal interactions, d910 Community life21. Emotional well-being b152 Emotional functions; d240 Handling stress and other psychological demands

Person Factors

ATDPA Section C: Psychosocial factors ICF Classification: Contextual Factors______________________________________________________________________

Attitudes and support from family, Support from family (e310, 410), friends Support from friends (e320,420)

Temperament Personal, Temperament & personality (b126)

Mood Emotional functions (b152)

Autonomy and self-determination Making decisions(d177), Higher cognitive functions (b164), Attitudes (e4)

Self-esteem Personal, Emotional functions (b152)

Readiness for technology use Incentive to act (b1301), Forming an opinion

(b1645)

Characteristics of the AT Device

ATDPA Section D. Device Match ICF: Products & Technology Matching (e115-e145)

_________________________________________________________________________________

Help achieve goals General tasks and demands (d2)

Improve QOL All Activities & Participation (d), Energy (b130), Sleep (b134), Emotional functions (b152)

Knows how to use Learning and applying knowledge (d1), Support (training) from health professionals (e355)

Secure with use Psychomotor function (b147), Emotional functions (b152)

Fits with routine Carrying out daily routine (d230)

Capabilities for use Specific mental functions (b140-bb180), Neuromusculoskeletal & movement –related functions (b7)

Supports for use Support and relationships (e3)

Will physically fit Moving around using equipment (d465), Domestic life (d6), Community life (d910), etc.

Comfort – family Emotional function (b152), family attitudes (e410)Comfort – friends Emotional function (b152), friends attitudes (e420)Comfort - school/work Emotional function (b152), peer attitudes (e425)Comfort - community Emotional function (b152), stranger attitudes (e445)

MPT (Specifically the ATD PA)MPT (Specifically the ATD PA)+ Good psychometric properties. Predictive of a match.+ Good psychometric properties. Predictive of a match.

+ Useful when evaluating a person’s device expectations + Useful when evaluating a person’s device expectations and realization of benefit with a specific device.and realization of benefit with a specific device.

+ Computerized scoring and interpretations available+ Computerized scoring and interpretations available

– – Requires a commitment of at least 45 minutes to complete Requires a commitment of at least 45 minutes to complete (longer if other forms are also used such as History of (longer if other forms are also used such as History of Support use) and to involving the consumer in the processSupport use) and to involving the consumer in the process

– – Many professionals are uncomfortable with asking Many professionals are uncomfortable with asking consumers personal questions.consumers personal questions.

More information on consumer AT experiences and the other measures

de Jonge, D., Scherer, M & Rodger, S. (2006)

Assistive Technology in the Workplace St Louis, Mosby.

Scherer, M. J. (2005). Living in the State of Stuck: How Assistive Technology Impacts the Lives of People with Disabilities, Fourth Edition. Cambridge, MA: Brookline Books.

 

 

                                       

                                       

COPMCOPM Canadian Occupational Performance Measure is an Canadian Occupational Performance Measure is an

individualized evaluation toolindividualized evaluation tool

uses a semi-structured interview to assist consumers to uses a semi-structured interview to assist consumers to identify specific problems in occupational performance identify specific problems in occupational performance areas such as self-care, productivity and leisure areas such as self-care, productivity and leisure

The importance of each problem is then rated on a scale of The importance of each problem is then rated on a scale of 1 (not important) to 10 (very important). Then, the client 1 (not important) to 10 (very important). Then, the client rates current level of performance and satisfaction with rates current level of performance and satisfaction with their performance on scales of 1(unable to perform, not their performance on scales of 1(unable to perform, not satisfied) to 10 (able to perform, extremely satisfied). satisfied) to 10 (able to perform, extremely satisfied).

COPMCOPM

+ Allows the client to reassess their performance on the + Allows the client to reassess their performance on the identified tasks at various intervals identified tasks at various intervals

+ Very individualized+ Very individualized

– – Requires considerable time Requires considerable time – – Not focused on ATNot focused on AT

IPPAIPPA Individualized Prioritised Problem Assessment (IPPA) is an Individualized Prioritised Problem Assessment (IPPA) is an

interview similar to format of COPM. interview similar to format of COPM.

clients identify problems and rate the importance and clients identify problems and rate the importance and degree of difficulty experienced in carrying out an activity degree of difficulty experienced in carrying out an activity on a 7-point scale (1= no importance at all, not at all on a 7-point scale (1= no importance at all, not at all difficult, 7=most important, too difficult to perform difficult, 7=most important, too difficult to perform activity). activity).

Provides a list of daily activities similar to activities listed Provides a list of daily activities similar to activities listed in the ICF. Asks the AT user to rate how the AT has in the ICF. Asks the AT user to rate how the AT has addressed each problem on a 5-point scale with –2 being addressed each problem on a 5-point scale with –2 being much less than expected and +2 being much more than much less than expected and +2 being much more than expected. expected.

IPPAIPPA

+ E+ Enables issues to be prioritised and the baseline nables issues to be prioritised and the baseline performance to then be compared with performance performance to then be compared with performance following acquisition of the devicefollowing acquisition of the device

– – Assesses Assesses activitiesactivities and not and not participation participation – – Requires consumers to be able to identify their problems Requires consumers to be able to identify their problems

– – Has not been used extensively in outcome studies.Has not been used extensively in outcome studies.

SCAI (SIVA Cost Analysis Instrument)

Designed to help clinicans estimate the economic aspects of AT provision

Using SCAI involved 3 steps – describing the objectives of the AT programme

Establishing the sequence and timing of interventions

Compiling cost for each AT solution

SCAI

The social cost is the main indicator of the economic significance of the AT solution

Alternative solutions must compared in terms of their social cost

Not a decision making tool, informative which adds to clinical assessment to make clinicans and users aware of economic consequences

SCAI – cost analysis

SCAI

AT paradox – Andrich, Renzo Cost analysis of AT, Portale Siva

Other MeasuresOther MeasuresCOMPASS is a software program to measures computer performance COMPASS is a software program to measures computer performance

(input, navigation and output). Provides quantitative data (input, navigation and output). Provides quantitative data regarding reaction time, typing speed, number of errors. regarding reaction time, typing speed, number of errors.

The Siva Cost Analysis Instrument (SCAI) detail/compares costs of The Siva Cost Analysis Instrument (SCAI) detail/compares costs of technology interventions. technology interventions.

Assessment of Life Habits (LIFE-H) questionnaire available in two Assessment of Life Habits (LIFE-H) questionnaire available in two forms: a 69- item screening tool to identify areas of life where forms: a 69- item screening tool to identify areas of life where participation in limited and a 240-item in-depth assessment across participation in limited and a 240-item in-depth assessment across 12 domains: 12 domains: nutrition, residence, responsibility, fitness, personal nutrition, residence, responsibility, fitness, personal care, communication, interpersonal relations, mobility, community, care, communication, interpersonal relations, mobility, community, education, employment and recreation.education, employment and recreation.

Other measures of participation have been developed for wheelchair Other measures of participation have been developed for wheelchair users, community activities (AM-PAC/PM-PAC is not an AT outcome users, community activities (AM-PAC/PM-PAC is not an AT outcome measure), measure),

FEW Functioning Everyday in a Wheelchair

Functional Evaluation in a Wheelchair (FEW) ICF coded Designed as a tool to measure basic wheelchair use

including such items as ability to reach form the wheelchair

Includes a mix of body function, activity and participation and environmental elements

Difficult measure to classify because all items are multi-barreled, eg respondents asked to rate their agreement with following statement “The size, fit, postural support and functional features of my wheelchair – response is on a 7 point scale

Wheelchair outcomes Measurment (WHoM)

Based on the ICF Uses items nominated and weighted

by the client Rater solicits information that is

participation focused Captures the satisfaction with

performance of activities or participation when using a wheelchair

At the end of the day…. A variety of processes and measures are used to address

AT outcomes. Thus, it is imperative that you know your consumers and what will work best for them!

As important as choice in quality measures is their appropriate use Know what you want to measure, and not want to measure

(competence, functioning, satisfaction participation) Determine how much time you want to devote to outcomes

assessment Know how specific or general you want to be

IPPA vs. QUEST

Evidence Based Practice

Evidence based practice is how Clinicians/researchers objectively provide evidence through scientific means on the outcome (positive and/or negative) of their intervention/s with their client group/s

Health Service ExecutiveWhy we need evidence based Practice.

To eliminate poor/unnecessary practice and promote good practice.

To promote evidence based medicine. To increase the accountability of services in line with the

key principles of the Health Strategy, "Quality and Fairness” (2001)

To develop means to evaluate services. To empower consumers and involve them in service

evaluation and planning. To evaluate new services. To inform priority setting and resource allocation. To help set, monitor and improve standards of care. To develop and share research

HIQA & HRB

The HSE through the Health Information and Quality Authority (HIQA) is responsible for making sure that the resources in our health services are used in a way that ensures the best outcome for the patient or service user.

They intend to do this by assessing the clinical and cost effectiveness of the medicines, devices, diagnostics, and health promotion used across the health system.

The Health Research Board (HRB) is the lead agency in Ireland supporting and funding health research. They provide funding, maintain health information systems and conduct research linked to national health priorities.

HSE

The outcomes of these assessments will allow the HIQA to support the Minister for Health & Children to make informed decisions on the desirability and effectiveness of investing in new therapies, drugs, equipment or health promotion activities.

HIQA will also advise on the rationale for continuing with existing practices to ensure that people are not being treated with outdated therapies, drugs or procedures.

Evidence Based Practice (EBP)

Where has this come from? This movement emerged first in the health

sciences in response to the demand for more accountability in professional practice

The goal of EBP is to bridge the knowledge gap between research and practice by providing guidelines based on the best available evidence from research

In the fields of EBP, methods have been developed, called “systematic reviews,” that insure that searches are thorough and reliable. Archives of systematic reviews have been developed that can be used as key resources for EBP.

Evidence Based Practice (EBP)

The Cochrane Library (health sciences) and the Campbell Collaborative (social sciences) are organizations devoted to supporting the development of EBP. Each maintains an archive of systematic reviews and has specific guidelines for conducting a review for their archive.

The National Center for Dissemination of Rehabilitation Research has published several concise articles describing the concept of systematic reviews and started an archive of systematic reviews in rehabilitation science

The Cochrane Collaboration

The Health Research Board have paid national subscription to make the Cochrane Database available to all clinicians/researchers in Ireland

5000 scientific papers published daily Clearing house for international evidence

based practice Information on evidence based practice Improving healthcare decision-making

globally, through systematic reviews of the effects of healthcare interventions

Measuring Outcomes

Quality of life indicators are now generally considered reliable measures to evaluate services, rather than merely questioning users as to their degree of satisfaction.

Traditionally the services received were considered as the standard for establishing social validity eg client surveys, waiting lists as opposed to establishing the success of intervention for the client’s needs etc

Assistive Technology Outcomes

Clinical Result Functional Status Quality of Life Consumer Satisfaction Cost Factors

DeRuyter 1998

Perspectives of Different Stakeholders: Importance of Various Outcome DimensionsDeRuyter 98

  Clinical result

Functional status

Quality of Life

Consumer Satisfaction

Cost Factors

Administrator

No Yes No Yes Yes

Client No Yes Yes Yes No

Clinician Yes Yes Yes Yes No

Payer-Funder

Yes Yes No Yes Yes

Outcomes based research

When choosing an instrument, careful consideration must be given to validity and reliability.

If the instrument is considered to be invalid or unreliable, the research is worthless

Validity refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure.

reliability is concerned with the accuracy of the actual measuring instrument or procedure, validity is concerned with the study's success at measuring

In validating an instrument, certain criteria must assessed through repeated testing

Validity & Reliability

Validity is key to effective research, In qualitative data validity might be

addressed through the honesty, dept, richness and scope of the data achieved, the participants approached, the researcher’s objectiveness

Quantitative data validity is addressed through careful sampling, appropriate instrumentation and appropriate statistical analysis

Measurement Tools

There are many standardised instruments available to clinicians

Assistive technology outcome measurement is commonly associated with a number of conceptual domains, including: device usability, user satisfaction, quality of life, social role participation, functional level and cost.

Most instruments focus on one or more of the above domains to measure a specific type of outcome of the application of assistive technology. The challenge for clinicians is in identifying the most appropriate tools for their clients and service.

Outcomes based measurement

A analysis of existing measurement tools should cover conceptual, reliability, validity and practical considerations

Comprehensive reviews have been undertaken using the ICF

Older instruments which focus on the construct of handicap may not fully capture the full scope of the concept of participation

Appraisal of Instruments

Conceptual comparisons can be classified according to the ICF, Cieza et al have proposed a process

Each measure can be classified in terms of whether it measures activity or participation or both eg items such as mobility, self care and domestic life could be deemed as activity. Items that assess interpersonal interactions, community, social life etc are deemed as participation

Outcomes based research

As the prescription process becomes more complicated and Health departments begin to demand evidence to support the need for equipment, outcome measurement is becoming necessary

Attempting to decide on an appropriate measure, researchers and clinicians must choose from an increasing array of potential instruments

Outcomes measurement

Provision of the right system is essential to reduce not only monetary cost but also person related cost

Garber et al found that 31% of their sample discontinued using their wheelchairs because the devices no longer met their needs

Kittel et al found that failure to consider important lifestyle issues was identified as primary factor leading to wheelchair abandonment

Many studies have documented abandonment due to mismatch of device

AT System for Education NZ

http://www.minedu.govt.nz/NZEducation/EducationPolicies/SpecialEducation/ServicesAndSupport/AssistiveTechnology.aspx

NDA Research on AT in 6 Countries and comparing it to

Ireland