k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

The Alberta Context Tool (ACT)

Carole A. EstabrooksProfessor &

Canada Research Chair

Janet SquiresDoctoral candidate

Faculty of Nursing University of Alberta

KU09Wales, UK

June 24-26, 2009

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

ACT overview• 59 items representing 8 modifiable dimensions

of context

• Currently 3 versions• Adults (acute care)• Pediatrics (acute care)• Long term care

6 groups• Healthcare Aides (LTC only)• Nurses• Allied• Physicians• Specialists/Educators• Managers

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Initial ACT development

1. Selection of a conceptual framework• Critical review of literature• Promoting Action on Research Implementation in

Health Services (PARiHS) framework as initial guide

2. Conceptual refinement• Addition of concepts to incorporate team’s

understanding of other relevant aspects of organizational context

• Conceptual and operational definitions developed

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Initial development

3. Survey construction• Item construction• Initially versions for 5 groups in adult acute care

(RN/LPN, MD, Allied, Managers, Educators)

4. Feasibility modifications• Survey administration and debriefing sessions were

held with members of the five groups (summer, 2006)• Modifications made based on debriefing feedback

• Question clarity• Revision of examples• Length/respondent burden

5. Response process validity evidence

Context

FacilitationEvidence

Research implementation

PARiHS Framework

Evaluation

Leadership

Culture

• Slack• Interactions• Resources• Social capital

• Attitude• PSA• Belief suspension• MBI• SF-8

Adding tocontext

Individual attributes

Kitson, et al., 1998, QSHCRycroft-Malone et al, 2002 (J Adv Nurs)

Framework

+

Context

Evaluation

Leadership

Culture

• Organizational slack• Formal interactions• Informal interactions• Structural & electronic resources• Social capital

Adding to context

ACT Framework

Structure of ACT1. Leadership

• 6 items scored on a 5-point Likert like scale• Concepts reflected in items: openness, optimism, self cont

rol,  empathic, developing  others, conflict management • e.g., The leader calmly handles stressful situations

2. Culture• 6 items scored on a 5-point Likert like scale• Concepts reflected in items: recognition, autonomy, workli

fe balance, development opportunity, focus on service/mission, support 

• e.g., I receive recognition from others about my work

3. Evaluation• 6 items scored on a 5-point Likert like scale• Concepts reflected in items: data access, informal data rev

iew, formal data review, action planning, performance monitoring,  benchmarking

• e.g., Our team routinely discusses this data informally

4. Formal Interactions• 5 items scored on a 5-point frequency scale• Concept reflected in items: Interactions with others

through engagement in formal organizational (unit) activities

• e.g., Team meetings about residents

5. Informal Interactions• 6-9 items scored on a 5-point frequency scale• Concept reflected in items: Interactions with others

through engagement in informal organizational (unit) activities

• e.g., Other professionals in my discipline

6. Social Capital• 6 items scored on a 5-point Likert like scale• Concepts reflected in items: bonding, bridging, linking• e.g., People in my group share information with

others in the group

Structure of ACT

7. Structural and Electronic Resources• Structural: 9 items scored on a 5-point frequency scale• Electronic: 3 items scored on a 5-point frequency scale• Concept reflected in items: availability/

use of structural and  electronic resources• e.g., Policy and procedure manuals   

8. Organizational Slack• 9-11 items consisting of 3 sub-scales

• Staffing (2-3 items scored on a 5-point Likert like scale assessing availability of adequate staffing resources

• Space (3-4 items scored on Likert and frequency scales assessing availability and use of space

• Time (4 items scored on a 5-point frequency scale assessing availability and use of time)• e.g., Have time to do something extra for

residents

Structure of ACT

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Validation

The Standards1 approach to validityA unitary approach to validity assessment where empirical results are classified as supporting (or refuting) validity

Validity Source

Explanation

Content To what extent does the content of the items represent the content of the domain? e.g., expert panel, CVI

Response Process

To what extent do the respondents’ responses fit the intended construct? Do respondents interpret, process, and elaborate upon item content and is this behaviour in accordance with the construct?e.g., revise items from pretest, interview to confirm instrument

Internal Structure

To what extent do the relationships among the items match the construct as operationally defined and do the data support dimensionality?e.g., factor analysis, item analysis (IRT)

Relationships with other Variables

What is the nature and extent of the relationships between instrument scores and:- variables it is expected to correlate with or predict - variables it is not expected to correlate with or predict e.g., correlations, regression

1 Standards for Educational and Psychological Testing. AERA; 1999.

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Initial validation(AKUTE1 study)

• Acute Care (Adult) setting• 4 teaching hospitals in Alberta• 5 professional provider groups (N=453)

• Reliability (alpha)• Range: .65 (formal interactions) to .92 (evaluation)• All but one >.70

• Validity (internal structure evidence)• Factor analysis (PCA with Varimax rotation)

14 factor solution; 67% of the variance ACT core (culture, leadership, evaluation – 31.95% of

variance)

1`AKUTE: Estabrooks, Norton, Birdsell, Cummings, Newton. AHFMR funded

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Second validation study(pilot + wave 1 CIHR Team in Children’s Pain)

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Second validation study• Validity (internal structure evidence)

• Item assessment• Item-total statistics remained stable

• Factor analysis• 14 factor solution; 61% of the variance• ACT core (culture, leadership, evaluation – 27.63%)• Some realignment of items (formal/informal

interactions)

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Second validation study• Validity (relationship with other variables

evidence)

• Correlation analysis• All ACT dimensions significantly correlated with IRU• Range: r=.052 (OS-staff) to .272 (structural/electronic resources)

• Regression Analysis• 7/8 ACT dimensions predict research use (at hospital level) at p<.10 (exception: structural/electronic resources)

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Additional Validation(CapitalCare studuy)

Pilot Study• 91 LPNs and HCAs in 4 long-term care units

• Extensive feasibility testing (response process validity evidence)

• Reliability:• Alpha > .70 for 7/8 dimensions (formal interactions

revised)• Alpha range 0.50 (formal interactions) to 0.96

(evaluation)

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Ongoing plans

The ACT currently under revision • i.e., harmonize across versions , reduce the number of

versions

Planned validation:• Psychometric assessment in new populations and in settings:

Further psychometric assessment:• Item Response Theory (IRT)• Confirmatory Factor Analysis (CFA)

Ongoing work re scoring of dimensions Assessment of translated versions (e.g., Swedish,

French)

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp