Big Data and Practice-based Evidence:

How EHR data is bringing the voice of nursing practice into policy and research

NNIC 2015

Karen A. Monsen

Lovely to be here!

Steeped in EBP• Joanna Briggs – Cochrane – NZ guidelines

Hendry, C. (2011). The New Zealand Institute of Community Health Care: REPORT TO THE MINISTRY OF HEALTH ON THE IMPROVING NURSING UTILISATION OF EVIDENCE TO INFORM CLINICAL PRACTICE SERVICES PROJECT. Available at: https://www.health.govt.nz/system/files/documents/publications/cdhb_report.docx

http://www.dilmah.co.nz/wp-content/uploads/2014/06/dimah-ranges.jpg

After 20 Years of EBP• Scholars are concerned.

• Sound clinical judgment is devalued• Mark Tonelli (1999)

• Evidence is limited• GoodyearSmith (2012). What is evidence-based practice and how do we get there? The Journal of Primary Healthcare, 4, 2, 90-91.

• Evidence is biased • Greenhalgh, T, Howick J, Maskrey N.Evidence based medicine: a movement in crisis? 2014 BMJ 348 :g3725

The Real World• The real world with all of its complexities and nuances is not

controlled

Practice-based Evidence is Needed

• Beyond algorithms, how do we provide the best personalized care for individuals in unique situations?

Gap in Commentary• Assumption that there is no data source that underlies and enables

study of practice-based evidence• This assumption is false• Nurses know what to do

What to do?

Learner Objectives• Describe the role of practice-based evidence as a necessary

component of nursing knowledge• Discuss possible sources of information that constitute practice-based

evidence• Provide examples of Big Data research using large nursing data sets

Practice• What expert nurses know and do every day to ensure wellbeing and

safety of patients• in the real world • for unique patients and situations

Practice-based Evidence• “How does adding X intervention alter the complex personalized

system of patient Y before me?”

Swisher, A. K. (2010). Practice-Based Evidence. Cardiopulmonary Physical Therapy Journal, 21(2), 4.

Practice-based Data• Data from nursing assessment and documentation that is

part of routine nursing care is an important source of practice-based evidence

Big Data• Large datasets of structured or unstructured information that may

require new approaches for analysis • Let the data speak

Garcia, A., L. (2015). How big data can improve health care. American Nurse Today. Available at:http://www.americannursetoday.com/how-big-data-can-improve-health-care/

Big Data Research in Nursing• Traditional and new methods for big data

• Using large data sets to examine important healthcare quality questions• Looking for hidden patterns in the data• Hypotheses generating vs. hypothesis testing• New voices for nursing and patients: Practice-based evidence

Big Data Studies in Nursing• All of the studies I’m about to share are examples of the rigorous

study of data – from practicing nurses – powerful observational datasets that speak for nursing and for patients alike.

Outcome Variability: Nurses and Interventions• Using a logistical mixed-effects model with nursing data

to evaluate outcome variability

This research is partially supported by the National Science Foundation under grant # SES-0851705, and by the Omaha System Partnership. Monsen, K. A., Chatterjee, S. B., Timm, J. E., Poulsen, J. K., & McNaughton,

D. B. (in review). Public health nurse, client, and intervention factors contribute to variability in health literacy outcomes for disadvantaged families.

• Client (50%)• Problem (17%)• Nurse (17%)• Intervention (17%)

Age was significantly positively associated with knowledge benchmark attainment

Implications• Research

• We need to incorporate the ‘nurse’ as an important part of the research model

• Policy• To ensure optimal outcomes, we need the best nurses

• Best fit with assigned patients • Support expertise• Ensure wellbeing

Mothers with Mental Health ProblemsMonsen, K. A. et al., 2014

Method: Data VisualizationEach image (sunburst) was created in d3 from public health nursing assessment data for a single patient. Data were generated by use of the Omaha System signs and symptoms and Problem Rating Scale for Outcomes

Key:•Colors = problems•Shading = risk •Rings = Knowledge, Behavior, and Status•Tabs = signs/symptoms

Documentation patterns suggest a comprehensive, holistic nursing assessment.

Kim et al. found that the presence of mental health signs and symptom tends to be associated with more diagnostic problems and worse patient condition

Kim, E., Monsen, K. A., Pieczkiewicz, D. S. (2013). Visualization of Omaha System data enables data-driven analysis of outcomes. American Medical Informatics Association Annual Meeting,

Washington D. C. Funded by a gift from Jeanne A. and Henry E. Brandt.

Implications• Research

• Visualization methods can help identify individuals with similar patterns in complex multidimensional data

• Policy• It is critical to identify and serve the individuals who most need our help

• Method: Generalized Estimating Equations for cohort comparison

• Results: Mothers with intellectual disabilities have twice as many problems as mothers without intellectual disabilities

• Receive more public health nursing service• Twice as many encounters and interventions

• Show improvement in all areas• Do not reach the desired health literacy benchmark in

Caretaking/parenting

Mothers with Intellectual Disabilities

Monsen, K. A., Sanders, A. N., Yu, F., Radosevich, D. M, & Geppert, J. S. (2011). Family home visiting outcomes for mothers with and without intellectual disabilities. Journal of Intellectual Disabilities Research, 55(5), 484-499. doi:10.1111/j.1365-2788.2011.01402.x

Implications• Research

• Large datasets will enable research into situations that are relatively rare and otherwise difficult to study

• Policy• Extra time and effort (and therefore funding) is needed to produce the

positive outcomes we desire for mothers with intellectual disabilities

Studies of Intervention/Outcome Patterns

• Problem-specific intervention patterns• Individual-specific intervention patterns• Population-specific intervention patterns

• Home care patients• Mothers with low health literacy

Using Kaplan-Meier Curves to Detect Problem Stabilization

This research was supported by the National Institute of Nursing Research (Grant #P20 NR008992; Center for Health Trajectory Research). The content is solely the responsibility of the authors and does

not necessarily represent the official views of the National Institute of Nursing Research or the National Institutes of Health. Monsen, K. A., McNaughton, D. B., Savik, K., & Farri, O. (2011). Problem

stabilization: A metric for problem improvement in home visiting clients. Applied Clinical Informatics, 2, 437-446 http://dx.doi.org/10.4338/ACI-2011-06-RA-0038

Using Data Visualization to Detect Nursing Intervention Patterns

Each image (streamgraph) was created in d3 from longitudinal public health nursing intervention data for a single patient. Data were generated by use of the Omaha System in clinical documentation

Key:•Colors = problems•Shading = actions (categories)•Height = frequency•Point on x-axis = one monthFrom 403 images, 29 distinct patterns were identified and validated by clinical experts

Documentation patterns suggest both a unique nurse style and consistent patient-specific intervention tailoring

Monsen, K.A., Hattori, K., Kim, E., Pieczkiewicz, D. (In review). Using visualization methods to discover nurse-specific patterns in nursing intervention data.

Streamgraph development funded by a gift from Jeanne A. and Henry E. Brandt.

Monsen, K. A. et al., 2014

COMPREHENSIVE WOUND CARE

BASIC WOUND CARE

Treatments & procedures

Case management

Surveillance

Monitoring

Teaching, guidance, & counseling

Informing

Providing Therapy

Using Inductive and Deductive Approaches to Create Overlapping Intervention GroupsRelationships between four intervention grouping/clustering methods for wound care.

Monsen, K. A., Westra, B. L., Yu, F., Ramadoss, V. K., & Kerr, M. J. (2009). Data management for intervention effectiveness research: Comparing deductive and inductive approaches. Research in Nursing and Health, 32(6), 647-656. doi:10.1002/nur.20354

Home Care Interventions and Hospitalization Outcomes

• Method: Logistic regression• Results: Too little care may result in hospitalization

when patients have more intensive needs • Frail elders are more likely to be hospitalized if they have

low frequencies of four skilled nursing intervention clusters

Monsen, K. A., Westra, B. L., Oancea, S. C., Yu, F., & Kerr, M. J. (2011). Linking home care interventions and hospitalization outcomes for frail and non-frail elderly patients.

Research in Nursing and Health, 34(2), 160-168. doi:10.1002/nur.20426. NIHMS274649

Knowledge scores across problems over time•Pre-intervention, patterns by race/ethnicity

•Post-intervention, patterns by problem

Health Literacy Outcomes

Benchmark = 3

Monsen, K. A., Areba, E. M., Radosevich, D. M., Brandt, J. K., Lytton, A. B., Kerr, M. J., Johnson, K. E., Farri, O, & Martin, K. S. (2012). Evaluating effects of public health nurse

home visiting on health literacy for immigrants and refugees using standardized nursing terminology data. Proceedings of NI2012: 11th International Congress on Nursing

Informatics, 614..

Implications• Research

• Nurses address multiple problems in different ways over time• Future research should take into account and evaluate factors of timing,

specific problem, and individual needs

• Policy• Encourage personalized interventions tailored to meet individual needs

Data Mining for Translation to Practice (Chih-Lin Chi et al., 2015)

Problem: A small percentage of clients consume a high percentage of service resources (80-20 rule)

20% patients use 70% of intervention resource

Research Question 1: Predict Intervention Usage• Regardless of outcome, who will need more interventions?

For 75% thresholdMaximal accuracy ~ 74%Maximal AUC ~ 77%

Prediction measured using receiver operating curves and area under the curve (AUC).

For 50% thresholdMaximal accuracy ~ 60%Maximal AUC ~ 75%

Research Question 2: Predict Responsiveness to Interventions

• Within the population, which individuals will be responsive to more interventions for this problem, compared to those who are less responsive?

More responsive Less responsive

Research Question 3: Predict Personalized Nursing Intervention

• How to personalize care planning based on an individual’s characteristics and what intervention patterns can be used to help personalization?

• Intervention patterns typically used in Oral health

Teaching, guidance, and counseling

Treatments and procedures

Case management Surveillance

Number of clients

A 0.00% 0.00% 0.00% 100.00% 24

B 0.00% 10.00% 0.00% 90.00% 2

C 0.00% 20.00% 0.00% 80.00% 285

D 30.00% 0.00% 30.00% 40.00% 1

E 30.00% 10.00% 10.00% 50.00% 1

F 40.00% 0.00% 10.00% 50.00% 210

G 50.00% 0.00% 10.00% 40.00% 234

H 60.00% 0.00% 10.00% 30.00% 1

Research Question 4: Predict Relative Improvement for Personalized Nursing Intervention

• Relative improvement is 51% (compared to maximum possible improvement for all clients)

• Choosing the right pattern can improve care (efficiency and effectiveness)

51%

Next steps• Nursing Big Data has been shown to enable the identification of

personalized algorithms to improve nursing care quality and efficiency• Practice-based dissemination and implementation research proposals in

development and review

Implications• Research

• It is becoming feasible to amass large quantities of data and create a pipeline for research into personalized care

• Policy• It is critical to support data sharing agreements and collaborations that

support use of clinical data for research

Nurses: Let the data speak!• Thank you!• Questions?• mons0122@umn.edu