amia 2018 clinical informatics conference multiple choice question (mcq… · 2018-05-03 · amia...
TRANSCRIPT
AMIA 2018 Clinical Informatics Conference
Multiple Choice Question (MCQ) Booklet
for Self-assessment
and
Maintenance of Certification (MOC) Part II Credit
[>280 pages in length – consider downloading to printing]
2 | P a g e
AMIA 2018 Clinical Informatics Conference Self-assessment and Maintenance of Certification (MOC) Part II Q&A booklet
Purpose AMIA invited the AMIA 2018 Clinical Informatics Conference workshop and session presenters to submit multiple choice questions (MCQ) associated with their presentations. This was to encourage interactivity as well as to comply with Maintenance of Certification guidelines from the American Board of Preventive Medicine (ABPM) and the American Board of Pathology (ABPath). All attendees of the AMIA 2018 Clinical Informatics Conference may use these MCQs to test their knowledge of what they learned in these sessions. All board-certified clinical informaticians must engage in answering the MCQs associated with the sessions for which they will claim MOC-II credit. American Board of Pathology-boarded clinical informaticians: acceptable performance in this self-assessment activity is defined as answering correctly 75% of the MCQs for the sessions you attended. Copy the text on page 4 to send an attestation of your self-assessment to [email protected]. The ACGME competencies addressed by this activity are: Patient Care, Practice-based Learning and Improvement, Interpersonal & Communications Skills, Professionalism, and Systems-based Practice. For a full explanation of the ACGME competencies in the context of clinical informatics, see Safran C et al. Program requirements for fellowship education in the subspecialty of clinical informatics. J Am Med Inform Assoc. 2009 Mar-Apr;16(2):158-66. MOC Committee Reviewers and Disclosures: The following MCQ reviewers disclose that they and/or their life partners do not have relevant financial relationships with commercial interests: Kamran Badizadegan, Laura Fochtmann, Martha Hellems, Robert Leviton, Robert C. Marshall; Lisa Masson, Karl Poterack, Matthew Rafalski, Sandeep Soman, Amy Y. Wang, Keith Woeltje
The following MCQ reviewers disclose that they and/or their life partners do have relevant financial relationships with commercial interests:
Name of Individual Name of Commercial Interest
Nature of the Relationship
Philip A. Smith MedMorph, LLC Stockholder
Applied Health IT Experts, LLC
Stockholder
3 | P a g e
Instructions The first half of this booklet of about 270 MCQs provides the question and answer options organized by day and by session number. The second half provides the correct answers per question. Each answer is followed by an explanation and reference(s). Clinical Informatics Diplomates: How to claim MOC-II credit • Log into your account at www.amia.org to access AMIA Central (upper right corner)
• On the right side, click “My Events”
• Click “Apply for Credits” under “AMIA 2018 Clinical Informatics Conference”
• There will be two drop-down boxes - for the first one, called “Select Credit Type” click “Physician” (Do not use “CME” as this is a placeholder not yet in use.)
• For the second one, “Select Physician Credit Type” click “MOC-II”
• Use the radio buttons to select the MOC-II sessions you attended
--You must claim your MOC-II credit first for AMIA to keep an accurate record of your MOC-II credits
• At bottom of page, click “Save and Add Additional Credit Type”
• Now, be sure both drop-down menus show “Physician” and claim your regular CME session (there will be only one, the keynote speaker session)
• Click Submit
• You will be prompted to download your certificate
Claim credit by Monday, June 11, 2018. AMIA reports MOC-II credits to the ABPM and to the American Board of Pathology.
If you are board-certified through the American Board of Pathology, please complete and send the form on the next page to [email protected] by June 11. 2018.
AMIA pulls data to send reports only within the same year as the activity.
4 | P a g e
Email attestation text for American Board of Pathology-boarded clinical informaticians:
From: [Use the email you have on file with the American Board of Pathology]
Subject: MOC-II credits CIC2018
I attest that I attended the live activity AMIA 2018 Clinical Informatics Conference and used the self-assessment booklet as my Self Assessment Module (SAM) for those sessions designated as offering MOC-II credit in the activity.
I attest that I achieved at least a 75% score on the questions I answered that were associated with the MOC-II designated sessions that I attended.
Diplomate’s ABPath ID number
First Name
Last Name
Date of Birth (Month and Day only, mm/dd format)
Activity Completion Date in mm/dd/yyyy format
ACCME Activity Program & Activity Reporting System (PARS) ID
201105920
Number MOC Credits earned from your claim at amia.org’s AMIA Central
5 | P a g e
Multiple Choice Questions for CIC 2018 Self-Assessment:
Tuesday, May 8th WS02: Organizational Issues and Informatics: Translating Theory into Practice
--Kim Unertl, Saira Haque
WS02-1: When implementing health information technology, why is it a good idea to consider organizational theory when developing an implementation plan?
A. Academics love theory
B. To help identify concepts that are relevant for informatics practice
C. It doesn’t make a difference
D. To inform your organization as a whole
WS02-2: What paradox led to the research that established Sociotechnical Systems Theory?
A. The Profit Paradox, that profits were elevated when new technology was implemented
B. The Happiness Paradox, that workers were happier when their work was going poorly
C. The Productivity Paradox, that despite improved equipment and better working conditions, productivity went down
D. The Despair Paradox, that the better technology got, the less happy workers became
WS02-3: Everett Rogers identified four major elements in how innovations are adopted in groups, as part of his Diffusion of Innovations Theory. What were the four elements?
A. The Innovation, Communication Channels, Time, A Social System
B. The Bosses, The Workers, The Tools, The Organizers
C. Advertising, New Technology, Coordinators, End Users
D. Great Ideas, Software Developers, Sales Representatives, Time
6 | P a g e
WS02-4: There are five stages in Rogers’ Innovation-Decision Process. What is a major influence on the Persuasion stage?
A. Whether people are really interested in making changes
B. How many people are interested in adopting the innovation
C. How people perceive the innovation
D. How long people have been using their past tools
WS02-5: In applying Lewin's Change Model to an organization that is considering a major health information technology change, what strategies might be useful in "unfreezing" an organization?
A. Making sure the opinions of people who oppose the new technology are ignored
B. Avoiding contact with the intended end users of the technology
C. Firing anyone who disagrees with the choice of new technology
D. Involving all levels of intended users, from front-line workers to management, in the decision-making process
WS02-6: Why is the low point of the Change Curve Model sometimes referred to as the "Valley of Despair"?
A. People like to whine
B. People worry for no reason when productivity goes down
C. When productivity dips because of new technology, people have a hard time understanding how the technology might help them in the future
D. The implementation productivity dip does not actually exist, but is rather just a theoretical model that has been disproven by years of technology implementation
7 | P a g e
WS02-7: Many organizations refer to the day a new technology system is turned on as a “Go Live” date. Why is this a term that we are better off avoiding?
A. Going “live” makes people think about death
B. Implementing technology occurs over time and “go live” implies it is a single point in time
C. Customers/patients might think we are talking about medical procedures
D. People are unnecessarily picky about language use
WS02-8: What is one reason that people worry about change?
A. No one worries about change – it’s just a part of life
B. Concerns about wasting time and effort
C. People who worry about change are irrational
D. People enjoy finding new things to worry about
WS03: How to Properly Onboard Doctors in the Age of Informatics & Digital Health
--Hamed Abbaszadegan, Nacchal Nachiappan, Shakaib Rehman
WS03-1: As a busy clinician, there is no time to learn a new EHR system in a three-day period. Patients need to be seen. What is your approach to onboarding this busy physician:
A. Cancel Clinic
B. Reschedule patients
C. Find coverage
D. Block time 4 months in advance
8 | P a g e
WS03-2: As a long-time provider at your local facility, you never really learned how to use the EHR nor keep up with new changing widgets and apps. How can you attend New Provider Orientation if you are not new?
A. Displace the new provider
B. Schedule special sessions
C. Advanced planning of participants
D. New Provider Onboarding is only for new providers
WS03-3: Every hospital has a unique set of features and health IT tools that they invested in. How is curriculum established?
A. Given dynamic changes in the health IT world, curriculum can change at any time
B. Teaching is universal and content for onboarding is usually uniform
C. New digital toys should be brought on display for new providers
D. Once you know which providers attend, content should be tailored to fit
WS03-4: Shorthand and macros can make any process faster. Given EHRs have more than one way to do the same thing, what approach should be taken when onboarding?
A. Show all methods of shortened processes
B. Discuss approaches to macros and let the providers decide what is best
C. Only show what works for the majority
D. Discourage using quick methods for processes
9 | P a g e
WS03-5: Health care providers are diverse in their workflows. How can this diversity be a part of your onboarding process?
A. Too much work flow diversity cannot be overcome
B. Customize a trainer depending on who attends class
C. Ensure representation of both in/outpatient settings for instruction
D. Any provider can teach class as long as they have EHR experience
WS03-6: Providing health care now involves utilization of advance messaging techniques such as secure messaging through web portals. How can doctors be more efficient in processing of secure messaging?
A. Show the providers in detail how to send messages
B. Review all bells and whistles with advanced secure messaging
C. Set up teams associated with providers who can help resolve appropriate messages
D. Ensure that rules of timeliness of secure messaging is well explained
WS03-7: Navigating a new health system can be daunting with all the rules. What approach will yield a higher likeliness of success?
A. Establish a buddy mentor and organize forms in easy-to-find locations
B. Show which forms matter to physicians and demonstrate how to complete them
C. Invite departmental secretaries to training sessions
D. Teach the providers medial assistant how to obtain forms
10 | P a g e
WS03-8: Pharmacy prescribing practices such as opiates have recently been highly publicized. Which approach will support providers as they start with your healthcare system?
A. Have a pharmacist team member teach how to write prescriptions in a rapid manner
B. Teach how to build quick order sets
C. Invite your pain management director along with your pharmacy team member to review support roles and processes available
D. Show providers how to rapidly access decision support tools via phone apps
WS04: Hands-On Full Life Cycle Data Science Workshop
--Steven Johnson, Lisiane Pruinelli, Tamara Winden
WS04-1: When using EHR data in an observational study, which of the following is correct? IRB review…
A. Does not apply since this is not a clinical trial.
B. Is not needed since the data is being collected in the EHR to document care
C. Is not needed since we are only accessing the EHR and not interacting directly with patients
D. Is required since we are using patient-specific health data
WS04-2: How do data quality issues impact data science results?
A. Because EHR data is used to document healthcare, there are no data quality issues
B. Data quality can significantly impact results, making them biased or even invalid
C. Data quality issues can be ignored because machine learning and AI will take care of it
D. Poor data quality doesn’t impact results, because the data quality issues disappear with enough data
11 | P a g e
WS04-3: A good practice for exploratory data analysis is to use a number of techniques to visualize your dataset. Which of the following is NOT a good reason to visualize your data?
A. Data visualization allows you to better understand and ”get to know” your data
B. Humans can spot visual patterns much better than looking at descriptive statistics
C. Visualization allows you to see large amounts of data summarized and simplified
D. Visualization looks nicer than just having tables of numbers
WS04-4: After receiving a dataset for a data science project, a researcher will spend a significant amount of the total project time performing data preparation and exploratory data analysis. Which of the following is NOT a data preparation task?
A. Assessing missing data and imputing appropriate values or removing records
B. Adding variables, transforming data and joining data tables
C. Ensuring all data is of the appropriate type and is consistent with the data mode
D. Obtaining the appropriate level of IRB approval for the project
WS04-5: Many datasets have imbalanced data where there is a much smaller amount of data for positive classes than negative ones. For example, the prevalence of a number of diseases is low in the overall population (i.e. Ebola). Which is the best method to evaluate the performance of a classification model for imbalanced data?
A. Accuracy
B. F1 Score
C. Precision
D. Recall
12 | P a g e
WS04-6: Once data are properly prepared and cleaned, researchers should select the most appropriate modeling algorithm depending on the data and the research question. Which modeling technique does NOT work well on categorical variables?
A. Classification
B. Linear Regression
C. Neural Network
D. Random Forest
WS04-7: Once you have a good performing model and you have tested it sufficiently on your training data, it is time to implement it in the real world. Which tasks are essential for successful model deployment?
A. Ensure the model is activated at the right time in the workflow, appropriately alerts the clinician and allows specific action
B. It is not necessary to get buy-in from the clinicians, since the model embedded in the EHR will tell them what to do
C. Such models should only be created for research purposes and should not be used to support direct patient care
D. The model should only be implemented in the latest version of the EHR software
WS04-8: Data science is fast becoming very useful in healthcare, but success requires close collaboration between people with clinical knowledge and people with data and computer expertise. Which of the following is true about data science in healthcare? Data science…
A. Can be understood sufficiently by everyone in the healthcare organization
B. Is easy since we give raw EHR data to machine learning algorithms to produce highly accurate results
C. Is somewhat of a fad and it won’t have a significant impact in improving healthcare
D. Only needs to be understood by a few people in the healthcare organization
13 | P a g e
WS05: Operational and Practical Aspects of Clinical Knowledge Management
--Dominik Aronsky, Saverio Maviglia, Asli Ozdas Weitkamp, Roberto Rocha, Dirk Wenke
WS05-1: What is a logical first step in starting a clinical knowledge management program within the iterative knowledge management framework?
A. Build standards-based interoperable knowledge assets
B. Centralize the process of creating knowledge assets
C. Create a catalog of existing knowledge assets including relationships with metadata
D. Establish a knowledge asset lifecycle
WS05-2: Which of the following activities is least likely within the scope of a clinical knowledge management framework?
A. Build a sustainable knowledge asset framework with reusable components
B. Integrate knowledge assets into clinical workflow
C. Maintain the representation of knowledge assets
D. Provide knowledge lifecycle support
WS05-3: Which topic is least likely associated with clinical knowledge management?
A. Institutional data governance
B. Lifecycle management of knowledge assets
C. Management of knowledge meta-data
D. Single-login management for clinical workstations
E. InfoButton
14 | P a g e
WS05-4: Which process is not directly supported by a clinical knowledge management system?
A. Centralized prioritization of knowledge management implementation tasks
B. Centralized request management for changes in the knowledge base
C. Monitoring clinicians’ usage of clinical decision support systems
D. Versioning of knowledge assets
WS05-5: Which characteristic is NOT considered best practice for creating knowledge assets?
A. Interoperable
B. Modular
C. Recursive
D. Reusable
WS05-6: Which of the following is the least critical method applied in clinical knowledge management?
A. Formal representation of knowledge assets
B. Lifecycle management of knowledge assets
C. Management of knowledge asset reference material
D. Versioning of knowledge assets
15 | P a g e
WS05-7: Select the most important role(s) domain experts (or subject matter experts) fulfill within the context of a knowledge management lifecycle?
A. Review quality measures and propose new knowledge assets
B. Create asset specifications based on existing knowledge representation standards
C. Author new knowledge assets using tools available within clinical knowledge management systems
D. Critique asset specifications and share expertise during knowledge engineering sessions
E. Support end-user adoption of new knowledge assets
WS05-8: Which of the following is NOT correct when deploying or updating vendor-based knowledge assets in a clinical information system environment?
A. Integration requires workflow adjustments, data mappings, and extensive testing
B. Assets cannot be easily customized to reflect local needs and priorities
C. Deploying updates of vendor-based knowledge assets prevents CDSS malfunctions
D. Local customizations might be lost when subsequent updates are deployed
E. Effective troubleshooting by vendor is associated with security vulnerabilities
WS06: CMIO Workshop - Leading Clinical Informatics Initiatives: How to Successfully Tackle Challenges & Build Resilience
--Julie Hollberg, Richard Schreiber
WS06-1: All hospitals with an EHR should follow the principles of EHR governance. A formal, written EHR governance plan is necessary for:
A. Only large university hospitals that have large information technology staffs
B. Hospitals that have a chief medical informatics officer who runs the EHR
C. Hospitals that manage their EHRs, but not those who contract EHR services
D. All hospitals of whatever size, leadership structure, or EHR design
16 | P a g e
WS06-2: The most critical success factor in EHR governance is:
A. Managing and leading change transformation processes
B. Assuring purchase of the newest technology
C. Establishing best infrastructure prior to and after EHR installation
D. Rewarding staff for meeting or exceeding goals
WS06-3: Effective governance requires effective communication for EHR optimization and healthcare transformation, as well as implementation. All the following have been shown to be successful forms of communication. The most effective is:
A. Rich channels such as personal interaction and audiovisual aids are better than email alone
B. Encouraging leaders to disseminate information to users by any means
C. Just-in-time training—not too soon, not too late; as well as at-the-elbow support
D. Involvement of clinicians as well as technical experts in creating the communication plans
WS06-4. Lack of available time, lack of political and communication skills, and lack of vision are examples of leadership failures which can damage EHR implementation or migration. How can one avoid such failures?
A. Have a mission statement, a long-term strategic plan, and governance
B. Create a project plan, use Gantt charts to determine time lines, and follow agile processes
C. Hire experienced managers to run the day-to-day operations of the implementation
D. Be sure to hire a chief medical informatics officer to manage the process
17 | P a g e
WS06-5. There are many factors which ensure a successful EHR implementation or transformation. Which of the following is most predictive of success?
A. Training for computer-savvy providers or those who have used the EHR before is not necessary
B. Reducing ambulatory productivity permanently will accommodate changed provider workflows
C. Managers supporting implementation should have no other responsibilities at go-live
D. Additional staff is required to execute the new workflows and provide at-the-elbow support.
WS06-6. A recent study showed that different vendor databases vary in their categorization of drug-drug interactions (DDIs). Different EHR vendor and home-grown products are configured differently, and different sites differ as to whether they modify the databases to alter the number of alerts displayed. Other studies show that despite these differences it is possible to decrease the burden of DDI alerting. This data suggests: A. It is too complicated and fraught with potential error to modify the vendor product’s data base B. It is best practice to accept the vendor product’s DDI database and alerting configuration
C. Meaningful use, which requires DDI alerting, has set a new standard of care for such alerts
D. It is time for a revamping of the international standards of care for DDI alert strategy WS06-7. The CIO calls you into her office. She tells you the hospital is going to select a new EHR (Electronic Health Record), implement it, and then optimize it. All is to be done within the span of 24 months. She wants to hire a Chief Medical Informatics officer (CMIO). You advise her that the best candidate is:
A. Physician with extensive EHR experience but not necessarily any informatics education
B. Clinical Informatician educated in and experienced with Informatics C. Physician with extensive Informatics education but limited EHR experience D. Only physicians, and absolutely no PhDs or Masters
18 | P a g e
WS06-8. Successfully passing the ABPM board exam in applied clinical informatics requires knowledge in a diverse set of disciplines. These include: A. Workflow analysis, health information technology, and clinical care B. Clinical care, workflow analysis, and information system lifecycles
C. Evidence basis for clinical care, change management, and decision science
D. Health systematics, clinical decision support, and computer programming
WS06-9. Yogi Berra once said, “When you come to a fork in the road, take it.” Many of us change our career trajectories, and this can be a difficult transition. Some are better at these transitions because:
A. Self-reliant physicians manage change well
B. Career and life coaches can benefit physicians
C. Physicians often change positions without difficulty
D. New positions are a welcome and exciting challenge
WS06-10. Book knowledge provides a solid basis for medical and informatics practice; experience provides real-world application of that knowledge. Since young informaticians lack that experience, the single best strategy to increase one’s knowledge is to:
A. Work long hours in the clinic and/or hospital as well as in informatics to augment their learning
B. Seek out a mentor to help with challenging situations and to help guide your career
C. Treat a transition into a new role as a temporary condition as things will work themselves out
D. Avoid revealing one’s difficulties to colleagues as this may jeopardize your job performance
19 | P a g e
WS06-11. Which of the following is not a key driver or domain of burnout and professional fulfillment?
A. Efficiency of practice
B. Practice productivity
C. A culture of wellness
D. Personal resilience
WS06-12. Tuckman’s model of team work posits that groups will develop in a predictable fashion. The usual order of that development is:
A. Norming, storming, forming, performing
B. Storming, forming, norming, performing
C. Forming, performing, storming, norming
D. Forming, storming, norming, performing
WS06-13. The best statement regarding IT governance is:
A. A single structure for good IT governance can keep pace with all the “wants” and “needs” of an organization
B. IT governance does not need to include research and educational needs in its structure—separate governance provides adequate oversight
C. How an institution makes IT investment decisions is often more important that what specific decision sare made
D. Proper leadership governance in IT is sufficient to overcome the lack of a clinical information systems advisory group
20 | P a g e
WS06-14. When facing conflict over very important issues, and there are no time pressures for resolution, you should first use which of the following leadership/management methods:
A. Accommodation
B. Compromise
C. Collaboration
D. Competition
WS06-15. Alert fatigue reduces efficiency of use of electronic health records. Many clinicians gloss over alerts, often without reading them. Research reveals various ranges of override rates, but the most widely quoted and likely most accurate overall rate is:
A. 60%
B. 75%
C. 96%
D. 99%
WS06-16. Which of the following leadership qualities would be most important for a chief medical informatics officer to display when implementing new systems and applications?
A. Charismatic
B. Task-oriented
C. Transformational
D. Supportive
21 | P a g e
WS07: Get Your Hands on FHIR®
--Laura Heermann, Russ Leftwich
WS07-1. What does the acronym FHIR® stand for?
a. Federal Health Information Record
b. Fast Healthcare Interoperability Resources
c. Fire Hot Information Response
d. Fast Health Information Record
WS07-2. FHIR® is designed to
a. Replace enterprise healthcare records
b. Be used exclusively for personal health records
c. Enable the exchange of healthcare-related information
d. Burn up all other attempts at interoperability
WS07-3. Why is FHIR® important to clinicians?
a. It leverages the latest web standards and applies a tight focus on interoperability across care settings and disciplines.
b. It is a standard specification specific to the needs of nursing interoperability.
c. FHIR® has specific resources that apply only to hospital care.
d. FHIR®has no implications for clinical interoperability
22 | P a g e
WS07-4. How is the Fast Healthcare Interoperability Resources specification accessed?
a. http://hl7.org/fhir
b. www.fhir.com
c. www.FIRE.org
d. http://fasthealthcareinteroperability.org/
WS07-5. What is a FHIR® Resource?
a. A system of interoperability gadgets that hook together
b. A logically discrete data concept that has a defined meaning, a known identity and location, and has meaning in healthcare.
c. Short snippets of reusable clinical themes used to comprise a medical record.
d. Small increments of reusable code.
WS07-6. What is an extension in FHIR®?
a. A way to limit the use of a specific resource
b. An adornment to a resource to make it more interesting.
c. A reference to an additional resource that makes it more meaningful.
d. A method to implement additional valid requirements not included in the core specification.
23 | P a g e
WS07-7. What does it mean to constrain a FHIR® resource?
a. To limit a resource to a specific definition fitting the intent of the implementation (such as limiting a pick list with the colors red, green, yellow, blue, orange and purple to red, yellow and blue only)
b. To allow for additional items to be added to a resource
c. To compel a user to document using FHIR resources only.
d. To discourage use of specific parts of a FHIR resource but allow them if absolutely necessary.
WS07-8. What is a FHIR® Profile?
a. A sketch of a drafted FHIR® resource prior to its being released for use.
b. A specification of a FHIR® resource or resources detailing constraints and extensions necessary for use in a particular use case.
c. A fully instantiated resource
d. A FHIR® resource in action sharing patient data from one clinical site to another.
WS07-9. Name 3 FHIR® Architectural Principles.
a. Fast, healthcare specific, reusable
b. Fast, easy to learn, scalable
c. Reusable, Scalable, Readily Implementable
d. Implementable, clinically sufficient, highly structured
24 | P a g e
WS07-10. FHIR® Resources are organized into 5 framework layers. These include:
a. Foundation, Base, Clinical, Financial and Specialized
b. A, B, C, D, E layers
c. Problems, Meds, Allergies, Orders and Labs
d. Structural, Clinical, Observations, Provider, Payer
WS07-11. Resources classified as “workflow” are meant to:
a. Support the healthcare process
b. Conform to a specific institution’s clinical workflow.
c. Be implemented using BPMN
d. Identify the users of a specific workflow
WS07-12. What is a FHIR® Implementation Guide?
a. A listing of FHIR® resources
b. A how-to guide on profiling FHIR® resources
c. A knowledgeable person available for new implementers for assistance as needed.
d. A publication outlining capabilities as defined by national standards, vendor consortiums, clinical societies, etc., regarding how FHIR® specifications are used to solve particular problems.
25 | P a g e
WS07-13. The Observation Resource should be used for what type of data?
a. Home meds, orders, MAR
b. Vital Signs, physical exam finding, pulmonary artery pressure reading, lab results
c. CT Scan, MRI, Xrays
d. Problem List, Chief Complaint, Past Medical History
WS07-14. The Condition Resource should be used for what type of data?
a. CT Scan, MRI, Xrays
b. Vital Signs, Physical Exam Findings, Pulmonary Artery Catheter reading
c. Diagnosis, Chief Complaint, Past Medical History
d. Home meds, orders MAR
WS07-15. The clinFHIR software tool requires what configuration parameters?
a. A use case type aligned with the categories of FHIR® resources
b. Selection of a set of capability statements from a library
c. Selection of LOINC and SNOMED value sets from the NIH Value Set Access Center
d. Setting compatible data, conformance, and terminology servers
WS07-16. How do I provide feedback to the FHIR development team regarding FHIR resources?
a. Send them an email to [email protected]
b. Call HL7 Headquarters
c. Submit them through the gForge tool
d. Don’t bother – the specification is done and published.
26 | P a g e
WS07-17. What is a FHIR® Maturity Level?
a. A method for the FHR specification developers to inform implementers how advanced and stable a FHIR Resource is.
b. How long the FHIR® resource has been in existence
c. How many times the FHIR® resource has been implemented
WS07-18. What is the interoperability paradigm in which FHIR® is operable?
a. The messaging paradigm, like HL7 version 2.
b. The document paradigm, like HL7 Clinical Document Architecture (CDA).
c. As a RESTful API.
d. FHIR® can be used in all of the above interoperability paradigms.
WS07-19. The Argonaut project is what type of initiative?
a. An initiative to develop FHIR® certification testing led by the federal government.
b. An initiative initially started to accelerate development of FHIR®profiles for the Common Clinical Data Set sponsored by a vendor collaborative.
c. An initiative to develop FHIR® translations led by the HL7 Greece affiliate.
d. An initiative that is part of SMART on FHIR®.
WS07-20. What is SMART on FHIR®?
a. An ONC project to create FHIR® profiles for CDS.
b. A platform architecture for portable healthcare apps utilizing FHIR® as an API for health data.
c. The organization which has published FHIR® profiles for the Common Clinical Data Set.
d. The organization which first developed the concept of FHIR®.
27 | P a g e
WS07-21. Which of the following HL7 standards include specifications for an application programming interface (API)?
a. HL7 Version 2.x
b. HL7 Version 3
c. HL7 CDA
d. HL7 FHIR®
WS07-22. FHIR resources can be linked to each other via what type of element in a resource definition?
a. Code
b. Codable Concept
c. Reference
d. Identifier
WS07-23. Which of the following represent types of terminologies that can be linked to a FHIR® Profile?
a. SNOMED and LOINC
b. SnowOwl and RF2
c. C4NNol1 and NaPoLeon
d. Java and Python
28 | P a g e
WS08: How to Effectively Negotiate the Outcomes You Really Desire?
Shakaib Rehman, Hamed Abbaszadegan
WS08-1: According to the Thomas-Kilmann Model, if your concern about your own outcome is low and your concern for the other party's outcomes is high; your approach would be:
A. Accommodating
B. Avoiding
C. Compromising
D. Problem-solving
WS08-2: Psychologist Stephen Covey’s research has shown that a certain percentage of life is under your control and is decided by how you react while the other portion of life is called destiny (no matter what do you do; it will happen because it is inevitable). Which of the following option is correct about his determination of life events decided by you over what happened to you?
A. 60/40
B. 70/30
C. 80/20
D. 90/10
WS08-3: Psychologist Abraham Maslow has described the “Human Hierarchy of Needs.” What is the highest need of individuals, according to his model?
A. Esteem
B. Love
C. Security
D. Self-Actualization
29 | P a g e
WS08-4: What one of the following best describe “Principled Negotiations”?
A. Focus on interests not positions
B. Focus on people not problems
C. Identify subjective criteria to determine success
D. Identify Worst Alternative to a Negotiated Agreement (WATNA)
WS08-5: Which one of the following is not part of the “Principled Negotiation” model?
A. Separate people from problem
B. Invent options for mutual gain
C. Insist on using subjective criteria
D. Focus on interests, not positions
WS08-6: The “Principled negotiation” model emphasizes the importance of the “BATNA.” What does BATNA stand for?
A. Best Action to a Negotiated Agreement
B. Best Alternative to a Negotiated Agreement
C. Best Alternative to a Nonsense Agreement
D. Best Action to a Nonsense Alternative
WS08-7: Complete the list of Thomas-Kilmann's five conflict management styles: Competing, accommodating, avoiding, ___ and ___.
A. collaborating, and compromising.
B. colluding, and connecting.
C. communicating, and compromising.
D. cooperating, and connecting
30 | P a g e
WS08-8: If a person typically tends to delegate controversial decisions and accept others’ decisions, which of the following styles best describes this behavior?
A. Accommodating
B. Avoiding
C. Collaborative
D. Competitive
WS09: Designing Outpatient Clinical Decision Systems for High Use Rates, Provider Satisfaction, and Improved Care: Lessons from Meta-Analyses and Experience
--Patrick O'Connor, JoAnn Sperl-Hillen
WS09-1: Outpatient Clinical Decision Support (CDS) has been shown to be least effective at improving which of the following clinical measures:
A. Adult Immunization rates
B. Blood pressure control in hypertensives
C. Diabetes eye exam rates
D. Lipid test rates
E. Screening Mammography rates
WS09-2: Which of the following statements is most true of the impact and cost of Clinical Decision Support and Nurse Case Management to improve the proportion of those with diabetes who reach recommended treatment goals:
A. Clinical Decision Support is as effective as Nurse Case Management
B. Clinical Decision Support and Nurse Case Management can reach similar proportions of adults with uncontrolled diabetes
C. On a population basis, the cost effectiveness of Clinical Decision Support and Nurse Case Management for diabetes care are similar
D. Nurse Case Management can reduce overall health care costs
E. Clinical Decision Support for diabetes care is cost saving to payers
31 | P a g e
WS09-3: Consider a CDS system that extracts clinical information from the EHR, sends it to a web service where it is processed using clinical algorithms, and then transmits CDS back to the clinician and the patient at the point of care. Barriers to broad dissemination this type of CDS system include all of the following except:
A. Getting physician leaders from many medical groups to agree on clinical goals for BP, A1c, lipid management, and aspirin use in various groups of patients.
B. Concerns about data security when transmitting personal health information using the cloud
C. The potential of EHR-derived clinical information to be inaccurate or incomplete
D. Desire to provide evidence-based care personalized to each patient
E. Low use of the CDS system by primary care providers
WS09-4: In a classic paper by McGlynn et al published in 2003, the number of evidence-based actions that apply to adults in primary care settings is approximately:
A. 60
B. 300
C. 600
D. 3000
E. 6000
WS09-5: In a time-motion study of primary care physicians using a popular electronic health record system, the number of clicks and minutes required to get 80% of relevant data on cardiovascular risk to assess control of BP, lipids, glucose, smoking, aspirin use, and weight/BMI during an encounter was:
A. 4 clicks and 20 seconds
B. 12 clicks and 50 seconds
C. 30 clicks and 2 minutes
D. 50 clicks and 4 minutes
E. 70 clicks and 6 minutes
32 | P a g e
WS09-6: The approximate annual payment from a medical group to an EHR vendor for use and maintenance of an EHR software system is approximately:
A. $500 per clinician per year
B. $1,000 per clinician per year
C. $2,000 per clinician per year
D. $5,000 per clinician per year
E. $50,000 per clinician per year
WS09-7: Advantages of web-based, rather than within-EMR clinical decision support systems include all the following except:
A. Many medical groups can use a single CDS installation
B. Updating CDS algorithms when guidelines change is cheaper in a web-based service
C. Updating CDS algorithms when guidelines change is less prone to error if only one installation requires updating
D. Data security is improved when CDS is web-based
E. The cost of shared CDS installations would likely be lower than the aggregate cost of each medial group doing their own CDS installation
WS09-8: In many cases, there are conflicting clinical guideline recommendations from different sources. For example, the 2018 American Diabetes Association recommends a BP goal for adults with diabetes of < 140/90 mm Hg, while the 2017 AHA/ACC hypertension guideline recommends a BP goal of < 130/80 mm Hg for adults with diabetes. The best way to handle this type of situation in primary care CDS systems is to:
A. Provide detailed information to clinicians at each encounter, explaining the different recommendations
B. Insert “hard stops” to assure that clinicians must pay attention to BP goals at every patient encounter
C. Provide no CDS on BP management until expert panels resolve their differences
D. Frame treatment recommendations using words like “consider” rather than words like “should,” to accommodate differences in guidelines
E. Give each hypertensive patient a written sheet explaining the situation
33 | P a g e
Wednesday, May 9th S01: Presentations - Integrating Quality and Workflow
Quality in Action, Adapting Quality Measures for Realtime Guidance
--Andrew Simms
S01-1: When are providers required to report electronic Clinical Quality Measures data to CMS?
A. 2017
B. 2018
C. 2019
D. 2020
S01-2: The Merit-Based Incentive Payment System (MIPS) uses a score system for calculating performance. What percentage of this score is based on quality improvement activities?
A. 10%
B. 15%
C. 25%
D. 60%
From Pathway to Bedside: Integrating Clinical Informatics in Quality Improvement.
--Eric Shelov
S01-3: The most valuable asset of a Clinical Informatician in a Quality Improvement project centered on Clinical Decision Support is their knowledge of:
A. Data Standards
B. Clinical & IS workflows
C. EHR functionality
D. Database structure
34 | P a g e
S01-4: The most fundamental strength of a clinical pathway is its ability to help ________ clinical care and best practices.
A. Improve
B. Disseminate
C. Standardize
D. Accelerate
How am I doing? Online, interactive quality measures for clinicians
--Mark Weiner
S01-5: Patients may see different providers over time, so attributing the quality of a patient’s care to a single provider can be challenging. Which of the following is the most sensible way to ensure that the association between patient and provider is appropriate for attribution of a quality measure?
A. Recent Provider
B. Listed Provider in EHR
C. Listed Provider on the Insurance Card
D. Most frequent Provider
S01-6: Providers who receive report cards on their quality of care express many concerns about the validity of the measures. Which of the following statements about quality measures is true?
A. If a quality threshold is relaxed, the proportion of a provider’s panel achieving that threshold is likely to increase
B. If a quality threshold is relaxed, a provider’s quality ranking is likely to improve
C. A higher-ranked (“better”) provider will have a greater number of patients meeting a quality threshold then a lower-ranked (“worse”) provider.
D. Stratifying a provider’s panel by the presence of diabetes will decrease the proportion of the cohort receiving a HBA1c test compares with the overall cohort.
35 | P a g e
S02: Presentations –- Standards and Best Practices for EHR Implementations
Come Together: Lessons Learned from a Health System EHR Migration and Proposal for Sharing Implementation Best Practices
--John McGreevey
S02-1: All of the following are limitations to the broad applicability of existing guidance about how to implement an EHR except:
A. Existing EHR implementation guidance is based on valid experience and research and comes from respected health care, academic, and government institutions
B. Work describing an implementation addresses a narrow population (example: small, rural primary care clinics)
C. Some literature is becoming less relevant with time (example: paper to EHR transition)
D. Guidance focuses on a single aspect of an implementation (example: CPOE)
E. Guidance tends to be noncontroversial and described at a high level (example: training is important)
S02-2: Which of the following combinations correctly represents lessons learned in this health system EHR implementation?
A. Given that vendors have extensive experience with implementation, do not deviate from vendor guidance; develop an infrastructure of point people from various domains to support the implementation work; get an early start with change management; forego developing a strategy related to the EHR implementation as the vendor will guide you to a successful outcome.
B. Begin an implementation with internal alignment of practices, policies, equipment so that there is consistency throughout the organization; develop an infrastructure of point people from various domains to support the implementation work; recognize that sometimes vendor guidance will be appropriate and at other times, an organization may need to follow its own plan; wait to initiate change management efforts until 4-6 weeks before go-live so that the rest of the implementation work can be completed first.
C. Get an early start with change management; begin an implementation with internal alignment of practices, policies, equipment so that these are consistent across the organization; develop an infrastructure of point people from various domains to support the implementation work; maintain humility
D. Begin an implementation with internal alignment of practices, policies, equipment so that there is consistency throughout the organization; “no training, no access, no kidding” was an effective training strategy in this implementation; get an early start with change management; recognize that with a thoughtful strategy for an EHR implementation, anyone who reports problems with the system or who participates in workarounds after go-live is likely doing so because they are resisting necessary organizational change.
36 | P a g e
Seize the Moment: Promotion of Lean Principles in Moving from Paper to Electronic Provider Documentation at an Academic Medical Center
--Joel Betesh
S02-3: An Advantage of the APSO (Assessment, Plan, Subjective, Objective) note over a SOAP (Subjective, Objective, Assessment and Plan) note in the EHR is that:
a. It is supported by extensive high quality literature
b. It is easier for the reader of the note to find the key elements of decision making of the note author
c. It protects against notes being bloated with info pulled in from elsewhere in the chart
d. APSO has been used more extensively over the last few decades
S02-4: Moving from paper to electronic format alone is not enough to fully alter ingrained provider practices and perceived documentation needs. Problems with this transition at the health system described include all the choices below except:
a. Providers continue to feel a need to add data to their note for purposes of billing.
b. Manual chart review is still required to assess note quality
c. It is possible for individual providers to deviate significantly from the standard templates offered at a health system level
d. The majority of notes evaluated after the transition from paper to electronic documentation used the APSO format
Maintaining EHR Standards in a Complex Deployment
--Keith Woeltje
S02-5: All of these are important to invite to participate in the change control meetings, except:
A. CMOs/CNOs
B. Local leaders from each hospital
C. All subject matter experts (SMEs) for each specialty
D. CMIO
37 | P a g e
S02-6: Change control is an essential skill for clinical informaticists. Which of the following is TRUE about good change control? Change control…
A. is very technical, and should really only involve trained analysts and informaticians
B. helps prioritize limited resources and ensures system standards are upheld
C. has to be done because it’s part of Meaningful Use
D. is most useful at the start of a project, and is less useful once application teams are more sure about what they are doing.
S03: Panel - Clinical Informatics Fellows Project Ignite Talks
--Christopher Hollweg (Presenter) S03-1: In the FHIR (Fast Healthcare Interoperability Resources) specifications, what is the difference between “Resources” and “Extensions”?
A) Resources manage the incoming FHIR queries and Extensions the outgoing queries
B) Resources map the data back to the database while extensions map the data to SMART applications
C) Resources define the standard data types while extensions the additional data types
D) Resources deal with structured data (i.e., ICD) and extensions with unstructured data (i.e., free text)
S03-2: Which of the following is not a core principle of information system security theory?
A) Availability
B) Vulnerability
C) Confidentiality
D) Integrity
38 | P a g e
S03-3: A swim lane flowchart differs from a simple flowchart in that it:
A. Focuses on the value stream
B. Is a physical map of movements of people in the workflow
C. Visually represents the actions taken by various roles
D. Maps out the steps in the process
S03-4: Why is ELK stack a good tool set to manage healthcare data? Because it is…
A) an SQL database management system and can be easily queried
B) a NoSQL database management system and is highly efficient in querying text rich data
C) a graph database and can be easily queried, especially for imaging data
D) not scalable therefore provides security and stability
E) not open source, with proprietary technology that cannot be breached by outside hacker
S03-5: Which of following is one of the common issues encountered when clinicians are developing SMART-on-FHIR® CDS with developers?
A. Could be hard to have access to a FHIR® database/server. The database might not have the clinical data of your interest (e.g., lab values of the liver function test.)
B. Mock EHR might not be available even when the team has a FHIR® server - building a barrier to the usability testing.
C. Designing UI/UX requires serious feedback from clinicians
D. Medical calculators could be the serious contenders.
E. All of Above
39 | P a g e
S04: Presentations - Evidence-based Care Improvement
Article Relationship Visualization to Support Evidence-based Practice & Knowledge Discovery
--Po-Yin Yen
S04-1: What contributes to the highest quality of evidence for clinical practice?
A. Rapid review
B. Systematic Review
C. Literature Review
D. Scoping Review
S04-2: What is the most commonly reported barrier to evidence-based clinical decision-making? Lack of…
A. time to process new information
B. literature with high level of evidence
C. interprofessional communication
D. departmental leadership
Characterizing Cutaneous Lupus Erythematosus with Meta-Analysis Using STARGEO
--Osama El-Sayed
S04-3: A researcher is interested in performing a meta-analysis with data from the gene expression omnibus. What is a potential barrier to using GEO in one's research?
A. There is no standard way of filling out the metadata for the study
B. There is no information included about how the samples were processed
C. GEO does not contain any data on context of the study design
D. GEO is not accessible to the public and requires a subscription to access its data
40 | P a g e
S04-4: What is one way that generating disease signatures from meta-analyses can improve clinical care?
A. Disease signatures that are generated from meta-analysis are individual specific, which means that we can find unique treatments for each individual
B. There is no way to associate disease signatures with potential treatments
C. Using pathway analysis, we can find treatments that are known to target genes that are upstream in the disease signature
D. Using pathway analysis, we can determine the odds that a potential treatment will work before we know what molecule it interacts with
Digital Health Innovation System Integration Assessment: A methodology for streamlined early-stage evaluation
--Bailey Griffin
S04-5: While there is increased interest and investment in digital tools to improve the outcomes, experience and cost of care, the potential impact of an early stage digital health tool can be difficult to ascertain. Our abstract outlines our process and framework for assessing early stage digital health tools and the results of our retrospective review of recommendations given to companies to better align their product with system needs. The most common recommendation given to companies included:
A) Do better market research and hire consultants to help you
B) Identify individuals who have purchasing power at an organization
C) Properly identify the appropriate level of the health system at which the product is likely to be purchased and increase value to the primary payer
D) Focus only on the end needs of patients. The needs of providers and other users are irrelevant since we’re all most interested in improving the patient experience.
41 | P a g e
S04-6: Our Digital Health Innovation System Integration Assessment framework is focused on providing individuals and organizations with:
A) An early stage assessment/evaluation of digital health tools to better allocate scarce resources.
B) A basis to skip validation of a digital health tool via rigorous scientific inquiry.
C) An alternative to the regulated procurement processes at an organization.
D) A way to feel engaged with the small to medium sized enterprises in digital health.
S05: Presentations - Clinical Decision Support Embedded Predictive Models
Validating Clinic Workflow Models for Prediction
--Rafael Fricks
S05-1: Which of the following statements best describes over-fitting?
a. As model complexity increases, model predictions show decreased error in validation sets while training set error increases.
b. Over-fitting can be determined without the use of independent data sets.
c. An over-fit model provides an optimistic estimate of model quality when evaluated against the training set.
d. Hypothesis testing (i.e. Kolmogorov-Smirnov Test) cannot be used to detect over-fitting.
S05-2: Choose the answer that best completes the statement: “K-fold cross-validation…”
a. reduces the influence of any particular set of data.
b. should divide data into five subsets.
c. requires at least 100 observations.
d. should be used over other model assessment techniques such as bootstrapping.
e. is a feature introduced in MATLAB 2017a.
42 | P a g e
Patient-Specific Explanations from Risk Prediction Models
--Amin Tajgardoon
S05-3: Machine learning models are being increasingly developed to predict clinical outcomes such as mortality, morbidity, and adverse events. Among the following choices, which one enable clinicians to engender trust in a sophisticated machine learning model with excellent prediction power?
A. The model achieves a high area under the ROC curve (AUROC)
B. Satisfy the assumption that training data samples are independent and identically distributed
C. Provide simple and reliable patient-specific explanations for each prediction
D. The model has a very low generalization error
S05-4: Which of the following definitions best describes “prediction explanation” in the context of a machine learning model that predicts a clinical outcome?
A. Present to the user the model’s prediction score for a clinical outcome
B. Present an interpretation of the model to the user in terms of structure and parameters
C. Allow the user to select a model with the best accuracy in predicting the clinical outcome
D. Provide the user with an understandable interpretation of the prediction for an individual to whom the model is applied
Leveraging Electronic Health Record Data to Predict Surgery Cancellation
--Nick Pratap S05-5: In this study we implemented machine learning algorithms to explore variables from patient records to predict day-of-surgery cancellation. Overfitting is one of the common problems in machine learning which will decrease the generalizability of the model. Which machine learning algorithm has the highest risk of overfitting?
A. Gradient boosted LR
B. Random Forest
C. Regularized logistic regression
D. Classification tree
43 | P a g e
S05-6: In this study we utilized a iterative step-forward method to do feature selection. What is the primary objective of doing feature selection?
A. It identifies and selects key predictors to improve the interpretability of the model.
B. It increases the complexity of the model.
C. It creates new combinations of variables in order to reduce the dimensionality of the data.
D. It decreases the prediction performance of the model.
S06: Presentations - Informatics Application and Research to Enhance Surgical Practice
Adapting Technology into the Surgical Environment
-- Alex Langerman
S06-1: Information targeted towards surgical circulating nurses is best presented:
A. for the entire case at the start of a case (e.g., during time out or a preference card)
B. through a sterile interface to prevent surgical site infection (SSI)
C. in a mobile format that can be transported within and outside the operating room
D. with anatomic images to illustrate the choices between surgical techniques
E. for all of the cases of the day, to allow "anticipatory guidance"
S06-2: The most common and financially impactful result of poor communication regarding tools and tasks in the operating room is:
A. tool damage and repair/replacement costs (RRC)
B. over-provisioning of surgical supplies
C. lapses in sterility from improper draping
D. reductions in mean turnover time (TTm)
E. over-staffing of surgical cases
44 | P a g e
EHR Needs of the Surgical Subspecialist: Insights from System Selection
--Saira Haque
S06-3: What functionality should the surgical specialist consider when selecting an EHR?
A. Reconciling long term meds from outside vendors
B. Presenting longitudinal data for chronic disease
C. Targeting information pertinent to surgical problem
D. Graphing of BMI, blood pressure and pulse over time
S06-4: When an EHR vendor demonstrates the system’s functionality, which points should be clarified?
A. the programming and coding employed in the programming software
B. the ability to adapt the system to clinical and administrative workflows
C. the ability of the system to withstand cyber-attacks from external sources
D. workflow of enterprise systems beyond the scope of the current project
Challenges of Electronic Health Record Transition for Surgical Services: A Qualitative Observational Study of “Go Live”
--Megan Shroder S06-5: In our study of surgical practices during an electronic health record transition, one of the most common sources of problems and confusion was related to:
A. Admission orders
B. Phases of care
C. Operative notes
D. Inpatient documentation
45 | P a g e
S06-6: An important recommendation from our study of surgical practices during an electronic health record transition at an academic medical center is dedicated training for:
A. Work shared across teams of residents, staff, and faculty
B. Emergency department to operating room admissions
C. Physician scheduling of procedures from the outpatient setting
D. Documentation for trauma resuscitation
S07: Panel - LEANing into the Quality Payment Program: Using Agile Strategies for Real Practice Improvement
--Julia Skapik, et al S07-1: Which is NOT one of the four reporting categories in the Quality Payment Program (QPP) Merit-based Incentive Payment Program for individual providers and groups?
A. Advancing Care Information
B. Improvement Activities
C. Clinical Decision Support
D. Quality Measures
S07-2: Lean is a tool that has been around the manufacturing industry for a long time and only more recently has it been applied to healthcare organizations. This is partially due to the healthcare reimbursement system changing from fee for service to pay for value incentivizing more and more organizations to undergo Lean transformations. Taking on a Lean transformation can provide an organization significant gains in productivity, quality, and cost reduction but requires a cultural shift. This type of a shift is driven by leadership and needs to have the right principles in place to transform the culture. Which of the following are the most appropriate management principles to create a Lean culture?
A) Value to the customer, open communication, participate and contribute, pursue excellence, share knowledge, keep it simple.
B) Be a leader, exceed customer expectations, continuously improve, keep it simple, seek perfection, respect for every individual.
C) Attitude of continuous improvement, value creation, unity of purpose, respect for front-line workers, visual tracking, flexible regimentation.
D) Create value at every step, quantity over quality, keep it simple, seek relentless perfection, share knowledge, listen and communicate.
46 | P a g e
S07-3: Capturing and extracting valid and reliable data from certified EHRs requires iterative improvements to the certified EHRs, workflow, data entry screens and strategies to capture valid data based on the measures requirements and specifications. Which of the following is most likely the case in terms of iterative effort?
A. 80% of the effort is by the vendor to provide modifications within the certified EHR/20% is with the clinical side to modify workflow to accommodate eCQM data capture
B. 60% of the effort is by the vendor to provide modifications within the certified EHR/40% is with the clinical side to modify workflow to accommodate eCQM data capture
C. 50% of the effort is by the vendor to provide modifications within the certified EHR/50% is with the clinical side to modify workflow to accommodate eCQM data capture
D. 20% of the effort is by the vendor to provide modifications within the certified EHR/80% is with the clinical side to modify workflow to accommodate eCQM data capture
E.10% of the effort is by the vendor to provide modifications within the certified EHR/90% is with the clinical side to modify workflow to accommodate eCQM data capture
S07-4: Which of the following strategies is most likely to improve problems with electronic quality measure data capture and reporting?
A. Using only data from a clinical registry
B. Manual abstraction
C. Natural language processing in writing clinical notes
D. Use of a clinical data model associated with a standardized data warehouse
47 | P a g e
S08: Presentations - Bridging Analytics
POKE-R: Reducing Patient Harm
--James McGlothlin
S08-1: After the analytics have been developed, one of the most important elements to a successful project is a good deployment strategy for user adoption. When attempting to implement POKE-R in a specific unit, what is of the following is a key element which will help us succeed?
A. Utilizing specially trained nurses
B. Having one source of truth for all types of orders
C. Performing structured rounding
D. Meeting regularly with informatics professionals
E. Preventing duplicate orders in the EMR
S08-2: To develop a a project such as POKE-R, it is important to establish specific and attainable goals. Which of the following is a reasonable and significant goal for a POKE-R project.
A. Reduce surgeries
B. Reduce lab draws
C. Reduce medication administrations
D. Change nutritional orders
E. Reduce mortality
48 | P a g e
The Clinical Informatics Fellowship Experience: A Qualitative Analysis (qACIF)
--Benjamin Orwoll S08-3: You are a first-year resident and you find yourself very interested in how the information and computer systems at your institution can be leveraged to improve care. A faculty member mentioned the clinical informatics fellowship, but you are not sure what kinds of qualifications you might need to apply successfully. Of the following, which profile most closely approximates the experiences prior to fellowship from the cohort of fellows surveyed in the qACIF study?
a. Quality improvement scientists with extensive research experience who want to apply informatics principles in their work
b. Physicians with bachelors or graduate degrees in computer science intending to apply machine learning algorithms in health care.
c. Young physicians with interests in technology who are frustrated with the current state of health IT and desire to be leaders toward future system change.
d. Physicians with IT leadership background who need dedicated training in clinical informatics to obtain board certification
e. Entrepreneurial physicians who have previous business experience and need to gain a foothold for their devices in the health information ecosystem.
S08-4: Clinical informatics fellowships, as they are currently structured, do not specify training in a specific medical specialty or specialties as a prerequisite to application. Thus, clinical informatics can be a subspecialty of any medical specialty. Based on the results of the qACIF survey, which of the following medical specialties is most significantly underrepresented among clinical informatics fellows?
a. Pediatrics
b. Emergency Medicine
c. Pathology
d. General Surgery
e. Internal medicine
49 | P a g e
Interdisciplinary Development of Palliative Care EHR Decision Support
--Ruth Bush
S08-5: Palliative care is associated with more efficient resource use and decreased medical costs. The targeted technology available in the EHR, including clinical decision support could facilitate screening for earlier identification of patients in or about to be in distress; to provide more appropriate referrals to specialized care; and to facilitate more rapid communication among patients, caregivers, and clinicians.
Which of the following is a good working definition of palliative care:
a. A short-term approach to providing pain control.
b. Targeted patient education designed to prevent individual chronic disease development.
c. Medical care, which prevents or treats symptoms and side effects of disease and treatment and is designed to be patient and family-centered.
d. Engagement of the patient in their health decision making through technology such as patient portals and tracking devices.
S08-6: Which of the following statements most accurately reflects the results of the palliative care trigger list feasibility study:
a. The final algorithm version had several hundred variables to ensure it captured every possible individual needing palliative care.
b. The final algorithm needed to be validated with manual chart review.
c. The trigger list had poor sensitivity; patients retrospectively identified were not likely to have been seen prospectively by the palliative care team.
d. The algorithm relied primarily on laboratory testing as the basis for the clinical decision support.
S09: Presentations - Clinical Decision Support - Trends in Optimization
Optimize Up: Using Analytics for Clinical Decision Support Optimization
--Anwar Mohammad Sirajuddin
S09-1: Which one of the following is NOT a potential outcome of Clinical Decision Support (CDS) alerts?
A. Creation of a continuity of care document
B. Prevention of adverse drug events
C. Improvement of reimbursement and regulatory compliance
D. Provide treatment recommendations
50 | P a g e
S09-2: In order to optimize Clinical Decision Support (CDS) alerts, Memorial Hermann identified potential areas of inefficient alerting to the clinical users. The main two criteria they reviewed were:
A. Low frequency alerts and high alert override rates
B. Low frequency alerts and low alerts override rates
C. High frequency alerts and low override rates
D. High frequency alerts and high alert override rates
An Evaluation of Clinical Order Patterns Machine-Learned From Clinician Cohorts Stratified by Observed Versus Expected 30-Day Mortality Rates
--Jason Ku Wang
S09-3: With the emergence of data-driven clinical decision support tools, in particular those that aggregate the collective expertise of clinicians and make recommendations based on patterns learned from historical electronic health record data, defining a robust external reference standard is necessary. For the case of promoting compliance with clinical practice guidelines, which of the following proves to be the largest barrier to defining a reference standard, even for common admission diagnoses?
A. There are too few clinical practice guidelines in the literature to assemble a robust reference standard
B. In translating clinical practice guidelines for a given admission diagnosis into a finite reference standard, subjectivity and interpretation are required
C. There is no systematic way for multiple clinicians with different opinions to reach a consensus on a single reference standard
D. Clinical practice guidelines derived from more reputable literature source should be assigned more weight than those derived from less reputable sources
51 | P a g e
S09-4: Clinical order patterns derived from data-mining electronic health records can be a valuable source of decision support content. However, learned patterns can be compromised by underlying provider experience as clinical orders of less experienced providers (trainees) can exhibit greater variability and worsened patient outcomes than those of experienced providers (attendings). Based on the conclusions of this podium abstract, which choice of training data will invariably yield the greatest compliance with clinical practice guidelines?
A. Training indiscriminately on all available data
B. Always training on a cherry-picked subset of the most expert clinicians
C. Always weeding out data generated by less experienced clinicians (e.g. trainees) to improve model accuracy
D. No single approach will invariably yield the best outcomes
Developing a Learning Electronic Medical Record System
--Shyam Visweswaran S09-5: A learning EMR identifies and highlights EMR data that is relevant for a given patient and clinical context. Which of the following can be enabled by such a learning EMR?
A. Faster access to patient data
B. Increased responsiveness of EMR to mouse clicks
C. Reduced time needed to assess the patient’s condition
D. More coherent display of patient data
S09-6: A learning EMR discovers what type of patterns from the EMR database?
A. Medical appointment patterns
B. Patient data access patterns
C. Patterns of comorbidities
D. Medication prescription patterns
52 | P a g e
S10: Presentations - Collaboration, Pathways and Training in EHR Implementations
Using a Cloud-based Collaboration and Dissemination Platform to Support the Implementation of an Evidence-based Clinical Pathway Program Across a Multi-Entity Academic Healthcare System
--Emilia Flores
S10-1: Which of the following best describes a clinical pathway?
A. A graphical overview of a process
B. A document that contains criteria for guiding diagnosis and treatment of diseases or conditions
C. A method for translating evidence into local work structures and processes
D. A document that aims to standardize care for a specific clinical problem in a specific population
E. Structured and multidisciplinary plan of care, used to incorporate evidence into local practice, that has criteria-based progression, and aims to standardize care for a specific clinical problem in a specific population.
S10-2: Research findings have shown that patients do not always receive evidence-based care and treatment, despite the availability of evidence and published guidelines (Agency for Healthcare Research and Quality (2016)). Clinical pathways are one method for integrating evidence into practice; however, the development and implementation approach can have a significant impact on provider adoption. Which of the following approaches for clinical pathway development and implementation is most likely to have a positive impact on provider uptake?
A. Using a rigorous process and framework for clinical pathway development and dissemination, including identification of a clinical owner, basing the pathway on a synthesis of current guidelines, use of a multidisciplinary stakeholder group to review the evidence and adapt based on the local setting, and identification of barriers and facilitators to dissemination and tailoring the plan accordingly
B. Use of technology to provide centralized and easy access to clinical pathways across all practice locations
C. Incorporating access to clinical pathways in provider workflow
D. Use of smart alerts to identify relevant clinical pathways for a given patient
E. Using a rigorous process and framework for pathway development and dissemination, having a centralized and easy to access resource for storing pathways, and use of technology to provide access to relevant pathways in the provider workflow
53 | P a g e
Built-in Care Pathways and Optimization of Electronic Medical Record System for Efficient Throughput at a High-Volume Academic Medical Institution
--Rika Ohkuma
S10-3: A surgical service line A in an academic medical institution investigated their current workflow to find opportunities to reduce clinical variance postoperatively. They found that postoperative orders varied by attending surgeons and ordering residents. A current clinical pathway was developed four years ago, but rotating residents were not aware of the care pathway. Which is the most appropriate next step to reduce variance of clinical treatment by ordering physicians?
A. More education sessions for ordering physicians to think and order more carefully
B. Develop order sets aligned to the care pathway and have them embedded in EMR
C. Develop a more detailed care pathway so that they can follow each process
D. More process alerts in EMR to raise awareness of the care pathway
S10-4: A surgical service line B investigated their current workflow to find opportunities to reduce postoperative length of stay. They found that front line care givers are not aware of benchmark and expected length of stay. Inefficient timely preparation caused a delay in hospital discharge or a transfer. Which is the most appropriate next step to reduce unnecessary delay to discharge patients?
A. Start a discharge plan in EMR when patients become clinically ready
B. Start a discharge plan on the day of surgery and update an estimated discharge date daily in EMR
C. Set a target length of stay for each procedure and show an expected discharge date in a patient chart to set a same expectation in a multidisciplinary team
D. Track overall service line length of stay monthly
54 | P a g e
Evaluation of a Peer Training Model for Electronic Health Record System Implementation
--William Lancaster
S10-5: How does the peer training model (PTM) differ from the traditional training model?
a. With the PTM, providers train other providers in their same or similar specialty as to how to use the new technology.
b. With the PTM, certified trainers train all of the providers in how to use the new technology.
c. The PTM is less costly than the traditional model.
d. The traditional model allows providers to become more engaged earlier on in the implementation process.
e. The PTM does not require additional time away from clinical activities.
S10-6: The majority of the peer trainer survey respondents identified with which of the following statements:
a. I do not feel that my peers learned the system better because it was taught by a peer
b. I felt more prepared for Go-Live after having served as a peer trainer
c. Fewer problems were identified earlier as a result of using the peer training model
d. I sacrificed too much clinical time in order to serve as a peer trainer.
S11: Panel - Quality Measurement: Authoring and Implementing with Clinical Quality Language and FHIR®
--Ben Hamlin, Anne Smith, Bryn Rhodes, Carol Macumber S11-1: Clinical Quality Language (CQL) is an HL7 standard authoring language that:
a. Defines relationships between patients and clinical concepts
b. Uses data from electronic health records or health information technology systems to measure health care quality
c. Allows logic to be shared with both quality measures and decision support
d. Contains quality data for one patient
55 | P a g e
S11-2: Data elements specified for clinical quality measures reference the Quality Data Model (QDM) which defines:
a. The source of the acquired data
b. The relationships between patients and clinical concepts in a standardized format
c. The codes that define each data element concept
d. SNOMED CT categories
S11-3: Clinical quality measures rely on standard terminologies for accuracy and reliability of the process being measured. For patient reported measures and clinical observations, the preferred terminology is:
a. CPT
b. SNOMED CT
c. LOINC
d. ICD
e. HCPCS
S11-4: FHIR® (Fast Health Interoperability Resources) is a standard for exchanging healthcare information electronically. FHIR® is useful for measurement because:
a. FHIR® is a relatively simple exchange model that maintains data integrity
b. FHIR® can be used as a standalone standard or with other existing standards
c. It is organized into modules.
d. It has patient resources.
56 | P a g e
S12: Presentations - mHealth Solutions for Screening, Communication, and Disease Management
Digital Drawing Platform for Early Dementia Screening
--Kelvin Tsoi
S12-1: Assuming that digitization of cognitive screening questionnaires is possible, which of the following is the main benefit of digitized screening tests that can supply the dramatically expanding elderly population?
A. Enhancement of screening accuracy
B. Easy collection of digital data of participants
C. Enabling self-administered assessment
D. Prevention of subjective judgments on the drawing images
S12-2: Which of the following best describes the difference between scoring methods for drawing tests in paper-and-pencil form and scoring based on a digital drawing test?
A. Scoring methods for drawing tests in paper-and-pencil form are well-studied, but scoring based on digital drawing test is not.
B. Scoring based on a digital drawing test considers participants’ drawing behaviors, but scoring methods for drawing tests in paper-and-pencil form can only consider the final drawing image.
C. Scoring based on digital drawing test embraces advanced computing techniques is more comprehensive than the traditional screening tests in paper-and-pencil form.
D. Scoring methods for drawing tests in paper-and-pencil form are more objective than the scoring method on digital drawing test.
57 | P a g e
Increasing Efficiency and Enabling Innovation by Implementing Secure Text Messaging Application for Communication and Coordinating Patient Care
--Neha Patel
S12-3: Which of the following communication tools are Health Insurance Portability and Accountability Act [HIPAA]-complaint?
a. Pagers
b. Hospital-issued Smartphones for text
c. Hospital-issued Smartphone for voice calls
d. Hospital-issued Tablets
S12-4: CMS recently announced that texting patient orders would be allowed but if a hospital ensured that the process is through a secure text messaging platform and complies with which of the following:
a. Secure sign-on process(s)
b. Encrypted messaging
c. Allowed use of text messaging (for example, short message service (SMS) text messaging)
d. Customized message retention time frames
Practical Apps: A Framework to Assess the Quality of Mobile Health Apps for Chronic Disease Management
--Xinjie Yin
S12-5: Which of the following is one of the dimensions of the evaluation framework for mobile health apps for chronic disease management?
A. Accessibility
B. Cost-benefit analysis
C. Sustainability
D. Patient networking
58 | P a g e
S12-6: Which poses the greatest challenge for evaluation development of patient-facing mobile applications?
A. Prediction of user commitment
B. Extreme expense involved
C. Rapid change in mobile platforms
D. Medical society endorsement
S13: Presentations - Healthcare Big Data Analytics
Governing Predictive and Prescriptive Models: Adding the Right People and Processes to Technology
--Farah Ahmed
S13-1: The use of predictive models in healthcare organizations has become more common in the last decade. According to Becker’s Hospital Review, one of the key takeaways from the 2018 Healthcare Information Management Systems Society Conference (HIMSS) was that predictive analytics seems to be driving both operational improvements and clinical decisions (1). However, with the rise of these modeling methods, new challenges have arisen with interoperability and transparency within healthcare systems. While it is clear that predictive analytics has become significant in healthcare, what is the biggest challenge in the deployment of such models?
a. Predictive modeling is not effective when used for small healthcare organizations due to the complexity of the algorithm(s)
b. Predictive model implementation leads to complete replacement of current clinician workflows
c. Lack of oversight in implementation and minimal stakeholder involvement hinders acceptance and use
d. Minimal validation techniques and guidelines are available to ensure scientific validity of the models
59 | P a g e
S13-2: The term “black-box medicine” is one that describes a shortcut to answering complex questions without fully understanding the problems or solutions (1). This shortcut most often occurs to prevent the dissemination of proprietary algorithms in predictive models. With more hospitals utilizing internally and externally developed predictive models that follow “black box medicine,” what guidelines should end-users follow when using these types of prediction models?
a. End-users should not endeavor to understand the algorithm due to clinical liability and patient privacy issues
b. End-users should override or appeal recommendations made by predictive models when they have sound reason
c. End-users should adapt and accept all outcomes made by predictive models that are scientifically valid.
d. End-users do not need to be educated on the possible outcomes presented by the predictive model since they are the subject matter experts.
Using a Validated Predictive Analytics Model for Early Detection of Patients with Sepsis
--Gordon Moyer S13-3: You've created a predictive analytics model for sepsis. In your validation of the model, you calculate a C-statistic of 0.5. How do you interpret this result?
A. The model is no better at predicting which patients will and will not develop sepsis, compared with chance alone.
B. The model is good at predicting which patients will and will not develop sepsis.
C. The model is strong at predicting which patients will and will not develop sepsis.
D. The model always predicts which patients will and will not develop sepsis.
E. Based on this predictive model, 50% of patients will develop sepsis and 50% will not.
60 | P a g e
S13-4: You've created a predictive analytics model for sepsis. In your validation of the model, you calculate a C-statistic of 0.85. How do you interpret this result?
A. The model is no better at predicting which patients will and will not develop sepsis, compared with chance alone.
B. The model is good at predicting which patients will and will not develop sepsis.
C. The model is strong at predicting which patients will and will not develop sepsis.
D. The model always predicts which patients will and will not develop sepsis.
E. Based on this predictive model, 85% of patients will develop sepsis.
Progression of a Quality Scorecard Leveraging Informatics Tools
--Mark Connolly
S13-5: Which of the following is a key quality reporting best practice?
A. Excessive consultation with subject-matter experts
B. Develop a standard validation process to ensure quality and consistency
C. Provide access to underlying data to all report users to promote transparency
D. Do not release measure definitions to minimize the risk of the Hawthorne effect
S13-6: What is the role of subject-matter experts during metric development?
A. Oversee data capture and availability
B. Ensure value of metric to warrant resources for development
B. Taking all validation responsibilities
C. Develop plan that aligns with organizational goals
D. Secure necessary resources for developing metrics
61 | P a g e
S14: Presentations - Infrastructure and Frameworks for Successful Implementations
Application of an Evidence-Based Adoption Framework: Practical Implementation Tools for Clinical Operations
--Blake Lesselroth
S14-1: You have been asked by your leadership to improve and standardize medication reconciliation in an ambulatory clinic. In response, your team has developed electronic health record features to improve medication history collection, documentation of reconciliation, and patient counseling. When rolling out the new technology, which of the following approaches is most important about improving staff adoption?
A. A robust message campaign (e.g., educational materials, posters) is a critical ingredient in an implementation bundle.
B. Software should include patient-facing affordances that improve patient reliability including medication images, plain language, and simple response controls.
C. The implementation team should pay attention to the socio-technical fit of the product, including workflow compatibility and implementation climate.
D. Executive sponsorship and clinical champions are necessary and sufficient for promoting adoption in large practices, managed care organizations, and public systems.
S14-2: You have recently implemented new electronic health record features designed to improve the reliability of medication reconciliation in clinic. Nurses are expected to use new software to collect and document patient history. Physicians are expected to respond to an automated reminder confirming medications have been reconciled. Upon review of the monthly performance dashboards reports, you realize that nurse charting has improved but physician documentation remains unchanged. What would be the most appropriate next step to improve physician documentation?
A. Add a new hard-stop or modal alert that requires physicians to complete documentation before proceeding.
B. Gather observational or time-motion data to better understand physician workflow and identify potential barriers to reminder completion.
C. Consolidate the history collection and reconciliation tasks so that nurses can complete all documentation concurrently.
D. Meet with department leadership and script out physician performance plans that include reconciliation as a measure of effectiveness.
62 | P a g e
Creating the Data Infrastructure to Support Mobile Health and Telehealth Initiatives
--Christopher Hollweg S14-3: Which of the following best describes an error induced by automation bias?
A) Dr. Smith is alerted by an automatic EHR message that his patient may be developing severe sepsis, along with a button to order intravenous (IV) fluids. However, this patient also has heart failure, which may worsen with IV fluids. Dr. Smith assumes that the alert is appropriate and orders the fluids. The patient later develops fluid overload and is transferred to the intensive care unit.
B) Dr. Smith is about to order antibiotics for his patient with a urinary tract infection. He chooses ciprofloxacin because that is what he is used to using for his patients, but did not notice that this patient's previous urine culture grew organisms resistant to ciprofloxacin
C) Dr. Smith needs to discontinue the medication metoprolol for a patient in the EHR. However, he mistakenly discontinues the medication "metoclopramide," which was located right next to metoprolol in the EHR and looked very similar.
D) Dr. Smith is about to admit his 10th patient on a busy call day. When placing his admission orders, he mistakenly signs them in an open chart in the EHR for another patient whom he had seen earlier.
S14-4: Which of the following is one of the largest challenges commonly addressed in clinical risk prediction model studies involving EHR data?
A. Evaluation metrics
B. Informative presence
C. Lead time bias
D. Missing data
S14-5: Which of the following is a recommended goal when developing and implementing an electronic decision support tool for medication reconciliation?
A. Anticipating the users’ needs
B. Conducting usability testing in different settings
C. Focusing on thoroughness rather than efficiency
D. Redesigning the existing clinical workflow to match the tool
63 | P a g e
Use Cases for a WiFi Connected Internet of Things Button in an Academic Medical Center
--Haipeng Zhang
S14-6: The BWH Dash button is an example of using Internet of Things (IoT) devices. IoT is best defined as:
A. A group of related ontologies within a larger database housed on a public facing network.
B. A network of physical objects that contain embedded technology to communicate and sense or interact with their internal states or the external environment.
C. Wearable technologies that have the capability to sense biometric data from the user and communicate with one another and across a larger network.
D. Miniaturized single function computers that communicate via physical connections to one another.
E. Low cost, single board computers that can be programmed to perform various functions.
S14-7: The BWH Dash button uses AWS IoT 1-Click technology with a custom dashboard that allows for easy provisioning and programming of BWH Dash buttons. How many functions can BWH Dash buttons be programmed for?
A. 1 function - single press
B. 2 functions - single press, long press
C. 3 functions - single press, double press, long press
D. 4 functions - single press, double press, triple press, long press
E. 5 functions - single press, double press, triple press, long press, long press and hold
S15: Panel - New Approaches for Improving CPOE Safety: Indications-based Prescribing and CancelRx
--Gordon Schiff, Neri Pamela, Christine Cheng, Samantha Pitts
S15-1: Physicians have long been urged to place the indication for the medication on the prescription for drugs they are ordering. With paper prescriptions, physician acceptance has been poor, but with near universal adoption of electronic ordering (CPOE) there are renewed opportunities to promote utilization of this feature. Integrating indications into prescribing could pave the way for a safer, more complete
64 | P a g e
continuum of care for the patient and save time during prescribing and related tasks. Which of the following is another significant benefit of indications-based prescribing?
A. The indication will always be put on the medication label
B. Drug costs will decrease when indications are included
C. Patients will know the reason why they are taking each of their medications
D. Brand name drugs will be selected more often
S15-2: While including the indication on the prescription is not a new idea, many providers do not use this feature on a consistent basis. It has not become standard practice partially due to poor CPOE design. To successfully incorporate indication into the prescribing workflow, efforts need to focus on reducing mouse clicks, providing help at the point of care, and minimizing alert fatigue while supporting physicians’ autonomy and flexibility. Which of the following design choice best explains how a prototype successfully incorporates indications by allowing for physician autonomy while providing useful guidance?
A. Allow providers to search for a drug only and do not require an indication
B. Allow providers to start with a drug or a problem/indication and see drug suggestions
C. Allow providers to search for a drug only, enter an indication and see alternative drug suggestions
D. Allow providers to search for a problem only and see drug suggestions
S15-3: Associating indications with a drug order may help prevent medication errors associated with look-alike-sound-alike (LASA) drugs. Which of the following best describes the utility of associating high-level indications concepts with drug pairs on the Institute for Safe Medication Practices’ list of commonly confused drug names?
a. The majority of drug pairs that were comprised of antineoplastic drugs could be differentiated using high-level indication concepts.
b. The majority of drug pairs that were comprised of antibacterial drugs could be differentiated using high-level indication concepts.
c. More than half of all the evaluated drug pairs could be differentiated using high-level indication concepts.
d. Only 20% of all evaluated drug pairs could be differentiated using high-level indication concepts.
65 | P a g e
S15-4: The National Council for Prescription Drug Programs’ SCRIPT standard supports electronic functionality for sending prescription cancellations from electronic health records (EHRs) to pharmacies, known as CancelRx. However, adoption of this functionality has been limited. Which of the following presented the greatest challenge to pilot, rather than enterprise-wide,implementation?
A. Impact on pharmacy workflow
B. Impact on prescriber workflow
C. Restricted functionality of prescription cancellation in pilot implementation
D. Buy-in from key stakeholders
S16: Presentations - Care Coordination and Communication
Improving care coordination for children with autism spectrum disorder: defining care team members information needs
--Julia O’Rourke S16-1: Autism spectrum disorder (ASD) is a neurodevelopmental disorder diagnosed in childhood that affects individuals throughout their lifespan. (1). As many as one in 50 children in the U.S. between the ages of six and 17 years have a diagnosis of ASD (2), which reflects a 200-fold increase compared to the estimates of four cases per 10,000 in 1966 (3). Individuals with ASD experience lifelong difficulties in social interaction, communication skills, behavior, and learning. Care for children with ASD is complex, involving on average seven different types of interventions, programs, and therapies (4), including state organizations, local schools, medical providers, in-home behavioral therapy, and rehabilitation services, such as physical and occupational therapies. Children with ASD also have increased medical needs due to comorbid mental and medical conditions including sleep problems, seizures, gastrointestinal problems, and maladaptive behaviors (5, 6). Current treatment delivery model for children with ASD is fragmented with treatment team members delivering care at different entities without any systematic collaboration or communication. To better demonstrate the roles or different care team members working with a child with ASD, we describe treatment of a hypothetical school age boy John, who attends a public elementary school. In school John has several care team members assisting him. His teacher and school aid are working with him daily in the classroom; his occupational therapist focuses on sensory difficulties during a weekly 45- minute session. His physical therapist helps with building strength and endurance during another weekly 45-minute session. His speech therapist sees him three times per week and helps with articulation difficulties. Finally, the school’s behavioral therapist sees John two times per week to teach pragmatic and social skills. At school, John has an Individualized Educational Plan (IEP) plan.
66 | P a g e
John’s parents also bring him to a private speech therapist and occupational therapist for weekly sessions. In addition, John sees a psychologist for weekly sessions outside of school, and another behavioral therapist who helps with some of the troubling behaviors at home. John is also followed by a psychiatrist, who monitors his psychotropic medications. John sees a neurologist for sleep and seizure problems and gastroenterologist for constipation. In this example, all care team members work with John to help him and his family to achieve a better quality of life and help him learn skills to live as an independent and productive adult. John has caregivers at school, at home, in private offices, and in several medical facilities, but they work independently and do not communicate across their practices and locations. These treatments are fragmented and it is not clear whether the therapies being delivered are the optimal choice for John and his family. What is the most likely care coordination scenario?
A. John has a Medical Home at his pediatrician’s office, where a case manager (social worker or nurse) coordinates his medical care and services with other care team members
B. John has an Integrated Medical home at his psychiatrist’s office, where a case manager coordinates his mental health care and services with other care team members
C. John has no care coordination activities
D. John’s mother performs all care coordination activities. To do this, she has had to reduce her hours and now works part-time.
S16-2: Care coordination integrates services across settings and supports interfacing among multiple care systems and care providers, including the following: medical, social and behavioral professionals; the educational system; home care agencies; allied health professionals; and families (1). In a recent opinion piece titled “Health Information Technology and Care Coordination: The Next Big Opportunity for Informatics?”, Dr. David Bates described the care coordination needs for complex patients who are cared for by a group of medical providers: “This entire group needs to be able to communicate better than they do today – to see the care plan, to message each other, and to have synchronous and asynchronous conversations about goals, and progress and lack thereof” (2). Dr. Halamka further highlighted the need for care coordination infrastructure in 2017 NEJM: “If we want patients to be engaged to help reduce the burden of care coordination, care plan tracking, and communication, we need modern tools that enable patients to interact with their providers using devices and workflows that are already part of their daily lives. Many companies are now offering such tools” (3). However, care coordination tools will be useful within a collaborative care model that supports care coordination. The collaborative care model relies on information systems and case managers to link primary care providers, patients, and mental health specialists (4). The collaborative care model has shown to be effective in over 80 studies for in adult population for the treatment of depressive and
67 | P a g e
anxiety disorders (4, 5). Collaborative care also improves outcomes for children and adolescents with mental health problems (6). The collaborative care model is based on the chronic care model developed by Wagner and colleagues nearly two decades ago to guide health services towards better management of chronic diseases (7). According to this model, improved patient outcomes result from the interactions between an activated patient and a proactive healthcare team. Factors that activate patients include support for disease self-management and enabling community resources. Factors that activate healthcare providers include adequate information systems, decision support, and delivery system design to support longitudinal care as opposed to episodic care. What is the greatest barrier in implementing a collaborative care model for children with ASD?
A. Severe shortage of child psychiatrists in the United States
B. No adequate payment mechanism for supporting care coordination
C. Privacy laws (The Health Insurance Portability and Accountability Act of 1996 (HIPAA) and The Family Educational Rights and Privacy Act(FERPA))
D. Difficulties integrating with EHR systems
Challenges to Care Coordination for Children with Special Health Care Needs: Potential Solutions
--Pooja Paode
S16-3: Current efforts to improve care for children with special health care needs (CSHCN) are highly medicalized and designed through the lens of providers, despite the fact that parents are responsible for the vast majority of the care coordination for CSHCN. In response to this issue, we created a multi-sector, parent-centered Stakeholder Coalition dedicated to identifying solutions. The Coalition identified communication, community collaboration, family engagement/advocacy, and systems issues/technology as key areas for improvement for the care coordination of CSHCN. How should we consider the interaction between systems issues/technology and the other three areas? We should…
A. come up with solutions for the four areas separately, then evaluate how they interact.
B. prioritize the development of technological solutions, such as applications, in each of the three other areas.
C. consider solutions that will address gaps in each of the three areas, then identify how technology and systems redesign can enable those solutions.
D. prioritize development on technology and systems changes that simultaneously bring together all three areas: communication, community collaboration, and family engagement.
68 | P a g e
S16-4: Parents of children with special health care needs (CSHCN) often simultaneously operate as care coordinators, caretakers, and key decision-makers for their children. They experience a lack of meaningful information exchange surrounding situation management, planning, monitoring, navigating, and resource usage. When does this problem cause the most adverse impact for parents caring for CSHCN? During…
A. provider-parent encounters
B. a variety of “tension points” throughout the child’s lifetime
C. the time that passes between clinic visits
D. searches for external resources to aid in the care of their CSHCN
Accuracy of Patient Self-Report of 90-Day Post-Discharge Healthcare Utilization and Complications: Relevance to Value Based Care
--Benjamin Rosner
S16-5: University Hospital is a regional center of excellence for total joint arthroplasty (TJA), and as such, it draws patients from a large geographic area. However, post-discharge, most of these patients return to their home area. When complications or readmissions occur, events for which University Hospital bears financial risk under value-based reimbursement models, University Hospital is often unaware. In reading the literature, those at University Hospital leading the post-discharge process improvement efforts have learned that up to 65% of complications and readmissions may occur at other institutions. To get a more complete picture of when and where readmissions are occurring among their patients, the hospital is considering surveying patients directly about readmissions occurring within 90 days of discharge. Assuming an unambiguous readmission question worded in lay language, University Hospital might expect patients to have what degree of accuracy in self-reporting readmission events?
A. Excellent (kappa≥0.8)
B. Very good (kappa 0.60-0.79)
C. Moderate (kappa 0.40-0.59)
D. Fair (kappa 0.20-0.39)
E. Poor (kappa <0.20)
69 | P a g e
S16-6: James is an analyst working for a large healthcare organization trying to understand the incidence of 90-day post-discharge complications. He has done a literature review of papers quantifying the accuracy of patient self-report for the presence of complications using metrics such as agreement and kappa. He believes that he has sufficient information to bring back to leadership about overall accuracy of patient self-report to make an informed decision about the organization's planned outreach to patients on the topic. What should he have done differently to avoid omitting an important element in his assessment?
A. He should have looked for papers using "concordance" as a metric rather than "agreement" and "kappa."
B. He should have limited his search to papers surveying patients at 1 year post-encounter or longer, because recall accuracy improves with time.
C. He failed to consider the importance of also measuring accuracy of self-reporting for the absence of events
D. He should have limited his search to papers in which analysis was limited to patients responding to surveys, and not including analysis of non-responders (non-responder bias).
Thursday, May 10th
S17: Presentations - Decision Support for Clinical Screening
The Power of Two: Rapid Screening and Intervening Combining clinical decision support (CDS) tools and nurse-driven protocols to effectively evaluate patients for high risk conditions and potential interventions at hospital admission
-- John McGreevey
S17-1: All of the following topics would be a good fit for the type of universal screening assessment described in this presentation EXCEPT:
A. Pregnancy status
B. Capturing detailed oncologic history
C. MRSA status
D. Chlorhexidine bathing indications
70 | P a g e
S17-2: Effective screening of patients for MRSA status, pregnancy status and to identify indicators for chlorhexidine can be potentiated when clinical decision support is combined with which one of the following?
A. Problem lists
B. Medication reconciliation
C. Pre-selected orders in order sets
D. Nurse-driven protocols and human assessment
Cancer Patient Trial Matching Assisted by An Efficient Automated Pre-Screening Application
--Hung Le
S17-3: Cancer claims the lives of half a million Americans every year. One of the major bottlenecks in the development of anti-cancer drugs is the lack of sufficient patient enrollment in clinical trials. Which of the following will likely happen if the enrollment goal is not met?
A. Delay or even termination of clinical trials, which, in turn, amounts to substantial loss to the pharmaceutical companies
B. The sponsor will ask the principal investigator to refund the investment to recruit patients
C. The sponsor can extend the time frame for enrollment indefinitely until patient enrollment goal has met
D. The investigator will be able to use synthetic patients to add to the number of actual patients to reach the number needed for enrollment
S17-4: At the department of Investigational Cancer Therapeutics (Clinical Phase I/II program), UT MD Anderson Cancer Center, we have developed an automated patient pre-screening application to significant reduce manual effort to 92% and increase enrollment rate. What eligibility criteria were used in this automated application? (select all that apply)
A. Patient age
B. Tumor type or disease
C. Number of prior therapies
D. Molecular target
E. Performance status
71 | P a g e
StartSmartTM: Decision Support Technology for Prenatal Screening
--Bonnie Gance Cleveland
S17-5: Which of the following best describes the iterative IT development process used in the creation of StartSmart, prenatal screening application, guided by the Davis’s Technology Acceptance model?
a) Review of the literature, focus groups with pregnant women, technology development using Davis Technology Acceptance Model, patient and provider end-user testing, refinement
b) Focus groups with providers & patients guided by the SBIRT Model, prototype development, alpha testing, beta testing
c) Prototype development using guidelines, focus groups, and expert consultation; alpha testing with patients and providers; revisions; beta testing with patient and providers; refinement
S17-6: Which one of the following best describes the process for applying the Davis’s Technology Acceptance Model to the assessment of the StartSmart Prenatal Screening Technology?
a) An iterative development of the decision support/patient education application with end users’ input at each level
b) Assessment of usability and ease of use was included in each step of the development
c) Assessment included user input at each phase/ beta testing with patient interviews that addressed acceptability, ease of use, and intention to change; provider surveys regarding acceptability and ease of use.
S18: Presentations - Supporting Best Practices
Tracking and Managing Abnormal Mammograms with SMART
--Caitlin Cusack
S18-1: At the Veteran’s Health Administration, the SMART project is being implemented to improve the ability to track clinical management against a single index abnormal mammogram via the use of a “Breast Care Cascade.” Which of the following “opens” a cascade?
a. The text of a received mammogram report is entered into the EHR.
b. An order for a mammogram is placed.
c. A woman who is due for a mammogram presents for a preventive care visit.
d. The pathology report for a breast biopsy is entered into the EHR.
72 | P a g e
S18-2: Ms. Jane Doe presents for a routine preventive care visit to her primary care physician (PCP). She is due for a mammogram that is ordered. When completed the mammogram is recorded as a BIRAD 0: Incomplete. The radiologist discusses the findings with the physician covering for Ms. Doe’s PCP, who orders additional views. The followup views are done the following day and reported as BIRAD 4, suspicious abnormality. Both reports generate SMART alerts, one to the PCP, the second to the covering physician. When the PCP views the initial BIRAD 0 alert, which SMART template is shown?
a. BIRAD 0 SMART Template.
b. BIRAD 2 SMART Template.
c. BIRAD 4 SMART Template.
d. Multiple Tests Please Summarize SMART Template.
Integrating Evidence to Inform Lab Test Selection into a Knowledge Management Framework
--Spencer DesAutels
S18-3: An academic medical center makes evidence-based local practice decisions across multiple content domains on a frequent, recurring basis. New external knowledge, e.g., a new guideline or meta-analysis, is shared through various institutional channels and eventually leads to each institutional decision being made or revised. Documenting this underlying knowledge has the potential to promote adherence reuse, while simultaneously establishing a basis for crucial ongoing decision maintenance. To maximize efficiency of documentation, the institution should consider which of the following?
a. Creating a shareable document that provides a running tally of all decisions by a decision-making body.
b. Creating a single record of the underlying knowledge and linking it to a unique record of each decision.
c. Selecting a standardized citation format for each publication type.
d. Requiring all committees to use an approved communication channel for official communications.
S18-4:There are many sources that provide relevant information on lab testing for patient care decisions and analytics. These sources are internal and external, and refer to testing at different scales and for
73 | P a g e
different purposes. Furthermore, sources are constantly emerging, changing, or being deprecated. To make a fully informed decision, e.g., selection of lab tests to perform or lab tests to restrict to certain clinical scenarios, it is often necessary to consult multiple resources concurrently. How can an institution maximize use of these heterogeneous data sources?
a. Publish a list of approved resources.
b. Negotiate contracts with vendors to present data in institutionally-approved formats.
c. Invest in mechanisms to bring informational resources to the point of care.
d. Bring all applicable data together in a consistent internal format that references each data source.
Cleaning House: EHR Order Set Innovation and Transformation at Penn Medicine
--John McGreevey
S18-5: All of the following contributed to the success of the order set transformation process described in this presentation except:
A) Specialty Lead engagement
B) Legacy Order Set analysis
C) Creation of a modular order set framework
D) Development of a standard order set layout and naming convention
E) Replication of all legacy order sets into the new EHR
S18-6: Based on this presentation, each of the following order set sections would be a suitable core module except:
A) Admission
B) Labs
C) Vital Signs
D) Notify Physician
E) Activity
74 | P a g e
S19: Panel - Implementing and Spreading Patient-centered, Evidence-Based Technologies: The Fall TIPS Story
Patricia Dykes, Denise Goldsmith, Ann O’Brien
S19-1: The Fall TIPS (Tailoring Interventions for Patient Safety) program is based on over a decade of research supporting its effectiveness with reducing falls and related injuries in hospitals. Which of the following are most likely to prevent a fall in patients with physiological risks for falling?
A. Universal fall precautions
B. A bed alarm
C. A fall prevention plan tailored to patient-specific risk factors
D. A fall prevention bundle
S19-2: Which of the following are most likely to support evidence-based fall prevention interventions for all patients?
A. Universal fall precautions
B. Clinical decision support
C. A fall prevention bundle
D. Infobuttons in the EHR
S19-3: Which of the following is the strategy most likely to achieve successful implementation of Fall TIPS into an EHR?
A. Hospital leadership support
B. Integration during the design/pre-implementation phase
C. Fall prevention workflow analysis
D. Front line staff involvement
75 | P a g e
S19-4: While the Fall TIPS program is currently used in over 100 hospitals in the US, and around the world, there are challenges to spread that can be addressed by organization leadership. What strategy has been used to improve long term effectiveness of the Fall TIPS program.
A. Regular fall prevention education
B. Regular leadership participation in safety rounds
C. Regular leadership seminars on fall prevention
D. Regular monitoring/feedback related to patient engagement in fall prevention
S20: Presentations - Safety: Root Cause Analysis, Virtual Care, and Medical Reconciliation
Verbal Orders: Root Cause Analysis of Co-signature Authentication Timeframes
--Thomas Powell S20-1: During a visit from the Joint Commission, the inspection team asks for additional information regarding verbal orders (VO) usage for the respiratory therapy department. Aside from the number and percentage of verbal orders, which additional analysis can be performed to present a more detailed overview of order procedures?
A. Perform a dual domain analysis of time/day for threshold based similarity
B. Parse the data to find patterns in VO receiver(s) and/or author(s)
C. Compare the respiratory therapy VO statistics against other departments in the health system
D. Compare the VO records against the health care system policy, describing the data points and timeline to authenticate the validity/authenticity/adherence of the order.
E. Identify grouping/bundling of orders to differentiate clusters from single purpose VO
76 | P a g e
S20-2: When analyzing Verbal Order (VO) records within the EHR, which data element is often overlooked?
A. Staff scheduling
B. Discharge timelines
C. Clinic type
D. Physician inbox time
E. Mode of VO delivery
Quality Standards for Virtual Care: The Roles and Responsibilities of Stakeholders for Advancing Care Quality in an Online Environment
--Kevin Smith
S20-3: A virtual care (telemedicine) provider group has convened a clinical quality council to provide guidance for ongoing clinical quality monitoring and management.
Which of the options below is most appropriate for establishing quality outcome metrics?
A. Start out by creating a new virtual care framework for measuring clinical quality.
B. Partner with a statistician to analyze virtual care claims data.
C. Establish a disciplinary action and remediation program for underperforming doctors.
D. Establish outcome measure targets for the most common virtual care visit types.
77 | P a g e
S20-4: A health care system wishes to improve appropriate antibiotic prescribing practices for the management of acute upper respiratory infections in the virtual care (e-visit) setting by improving clinician adherence to an evidence-based clinical guideline.
Given the above information, which of the following may be the most appropriate intervention to improve adherence to the evidence-based clinical guideline?
A. Add a link to the Infectious Disease Society of America web site on the clinician screen.
B. Incorporate contextual clinical decision support which highlights pertinent patient history of the current illness, a brief summary of the guideline criteria with actionable guidance, all within the clinician workflow.
C. Provide a variety of pop-up reminders and alerts reminding clinicians of the importance of antibiotic stewardship.
D. Disable e-prescribing functionality for antibiotic medications to prevent clinician over-prescribing.
S20-5:Which of the following is most true regarding the current state of clinical quality monitoring and management among virtual care (telemedicine) providers and vendors in the United States?
A. The CDC has been adopted as the source of truth for virtual care quality standards.
B. Any virtual care provider or vendor who can demonstrate that they have a medical reference available for the clinicians, such as UpToDate, is considered to be compliant with quality standards.
C. There are no established standards for measuring clinical quality outcomes in virtual care in the United States.
D. The World Health Organization, in partnership with AHRQ, is considered as the source of truth for virtual care quality
78 | P a g e
S20-6: The quality committee has a debate about how to approach clinical quality as it relates to the relatively new and rapidly growing model of virtual care.
What is the greatest challenge for establishing clinical quality targets for virtual care?
A. The conditions treated via virtual care are vastly different than those treated in a clinic setting.
B. The challenges for establishing clinical quality targets are essentially the same no matter the method of care delivery.
C. The potential limitations of telemedicine equipment and internet connectivity could make it more difficult to deliver quality care.
D. The lack of research in this area makes it impossible to establish quality targets.
Using Qualitative Techniques to Classify Medication Reconciliation Errors
--Rod Tarrago
S20-7: According to the Institute for Healthcare Improvement (IHI), the medication reconciliation process involves three steps. Which of these is NOT one of the steps:
A. Verification
B. Clarification
C. Communication
D. Reconciliation
S20-8: The Joint Commission states that the intention of medication reconciliation is to address all of the following except:
A. Duplications
B. Expirations
C. interactions
D. Omissions
E. Continuations
79 | P a g e
S21: Presentations - Patient Generated Data, Interventions, and Telehealth
Patient Generated Safety Data in the Hospital Setting: Trust, Transparency and Learning
--Sarah Collins
S21-1. Which of the following related to safety reporting in the hospital is true?
A. Most recognized safety concerns and incidents are reported
B. Patients and families are not considered key stakeholders in safety reporting
C. Most recognized safety concerns and incidents are not reported
D. Safety data reported by patients and families provides little insights
S21-2: Which of the following is an identified sociotechnical requirement for patient safety reporting most related to trust from patients?
A. Availability of a real-time clinician dashboard
B. Protected anonymity when submitting
C. Comparing data to other safety data sets
D. Clear responsibility for follow up
Building and Evaluating a Patient-Facing Intervention to Deliver Implantable Cardioverter-Defibrillator Remote Monitoring Data to Patients via MyChart: Patient and Provider Perceptions
--Romisa Rohani Ghahari
S21-3: A healthcare system wishes to send personalized information to patients based on implantable cardiac defibrillator (ICD) remote monitoring data. Survey data from a recent study showed that most of the participants (86%) wanted to receive more information than they currently receive from the clinic about their remote monitoring data through the standard of care letter. Which of the following is most appropriate based on the findings from this study?
A. Patients would be most satisfied with a standard of care letter when compared to the ICD data summary.
B. Patients found the standard of care letter easier to access and use compared to ICD data summary.
C. Participants found the mailed letter with the ICD data summary easier to access compared to the electronic format.
D. Either method (mailed letter vs. electronic format) of providing the ICD data summary works well for the patients.
80 | P a g e
S21-4: A healthcare system wishes to provide patients with their device information without increasing clinical workload and having a negative effect on providers. Prior research reported that providers were concerned that messaging patients with their device data could potentially increase the clinical workload. We evaluated the effect of providing patients with their device information on clinical workload and clinician attitude towards ICD data summary. Which of the following is most appropriate based on the findings from this study?
A. Providing patients with their device information did increase healthcare utilization.
B. Providing patients with their device information did not increase the workload of clinicians from increased patient questions (calls or PHR messaging).
C. Clinicians believed the ICD data summary was NOT easy for patients to understand and use for managing their health.
D. Providing patients with their device information did increase the device-specific workload for clinicians.
Predictors of average adherence to the use of home telehealth devices in Veterans with heart failure
--Jenice Ria Guzman
S21-5: Adherence to the use of home telehealth devices in Veterans with heart failure enrolled in the VA Home Telehealth Program ________ over time.
a. decreased
b. increased
c. reversed
d. stayed the same
S21-6: It is important to understand adherence to the use of home telehealth devices because:
a. Adherence is complicated and needs to be monitored.
b. Home telehealth devices are expensive and should only be given to patients who will use them.
c. Poor adherence to home telehealth devices over time may be a potential reason for negative results in prospective randomized controlled trials.
d. There needs to be a conversation with patients regarding expected adherence to the use of home telehealth devices for outcome effectiveness.
81 | P a g e
S22: Presentations - Identifying Medication Risk
Building a pharmacist prioritization tool using pharmacy-specific data and readmission risk
--Samuel Kabue
S22-1: When building a universal readmission risk prediction model for use in pharmacy, it is important to base the modeling strategy on:
A. Commercially available 'black box' prediction models
B. A single predictor such as age or sex
C. Predictors that are readily available in electronic medical records within many health systems/pharmacy departments
D. Pharmacists have time to intervene on every patient and therefore have little need for risk prediction models
S22-2: A pharmacy department or health system wishes to use our readmission risk prediction model but does not have a COPS2 score, LAPS2 score or RxDxCG score readily available. Which if the following predictors currently available to them can be used to in place of these predictors for the best performing model?
A. Diagnosis groupings
B. Drug subgroups
C. Admission category indicators (i.e. emergency department versus medical)
D. Number of refills on each medicine
82 | P a g e
A data mining approach to detect medication order near misses associated with electronic order sets
--Ron Li
S22-3: Which of the following best describes an error induced by automation bias?
A. Dr. Smith is alerted by an automatic EHR pop-up alert notifying that his patient may be developing severe sepsis and includes a button that places an order for 30cc/kg of normal saline. However, this patient also has congestive heart failure with an ejection fraction of 20%. Dr. Smith, assuming that the alert is appropriate, does not consider this patient's heart failure diagnosis and proceeds to order the 30cc/kg of normal saline. The patient later develops acute pulmonary edema and has to be transferred to the intensive care unit.
B. Dr. Smith is about to order antibiotics for his patient who has urinary tract infection. He chooses ciprofloxacin because that is what he is used to using for his patients, but did not notice that this patient's previous urine culture grew organisms resistant to ciprofloxacin
C. Dr. Smith needs to discontinue the medication metoprolol for a patient in the EHR. However, he mistakenly discontinues the medication "metoclopramide," which was located right next to metoprolol in the EHR and looked very similar.
D. Dr. Smith is about to admit his 10th patient on a busy call day. When placing his admission orders, he mistakenly signs them in an open chart in the EHR for another patient whom he had seen earlier.
S22-4: Which of the following functions would you typically fail to achieve in your design of electronic order sets?
A. Save time for clinicians for placing orders
B. Promote adherence to clinical practice guidelines
C. Provide clinical decision support
D. Increase the number of billable diagnoses for a clinical encounter
Detecting post-approval Adverse Drug Events within EHR Data: A case study correlating Lipophilic Beta-Blockers with the subsequent diagnosis of Parkinson’s Disease.
--Cyril Spiro
83 | P a g e
S22-5: A group of researchers are studying potential adverse drug reactions (ADR) related to prescriptions of medications and subsequent diagnosis of a specific disease. The researchers first select all the ADRs related to the specific disease detected by an ADR detection heuristic. Then the researchers study the ADRs by looking at their odds ratios. The goal of the researchers is to shortlist the potential ADRs for a subsequent detailed study. A few of the potential ADRs are related to a specific class of medications. Previous research has shown that this class of medications, let’s call it “X class,” have a high chance of interacting with other medications and cause complications in patients. The X class of medications have also shown conflicting results in previous research studies which were studying the ADR in question. Assume that a patient can be on X class medication along with other medications in the potential ADR list and the researchers want to shortlist the remaining ADRs in the list.
Which step can the researchers take to most efficiently shortlist the remaining and offset the potential effects of X class medications on other potential ADRs that were detected?
A. Recalculate the odds ratios of only the remaining potential ADRs for the particular disease in question by excluding patients that were on X class of medications.
B. Stop the study until a consensus is reached by clinical trials on the ADR related to X class of medications and the specific disease in question.
C. Continue shortlisting the potential ADRs without any change in the analysis strategy.
D. Start the entire analysis from the beginning by first removing the patients who were on X class of medications from the data and then running the ADR detection heuristic again on entire available data.
S22-6: A group of researchers are studying a particular potential adverse drug reaction (ADRs) related to a particular disease (disease Dx) and a particular medication (medication Rx) found by using data obtained from an EHR system. The design of the study is that of a retrospective cohort study. On investigating the data on the detected potential ADR by using statistical tests and running confounding analysis, the researchers find that all the statistical tests are pointing in the direction of the potential ADR being an actual ADR. What can the researchers conclude from the results of the statistical tests?
A. There is a causal effect of the prescriptions of medication Rx to patients leading to them developing the disease Dx. No more analysis or studies are necessary to confirm the findings
B. There is strong evidence to conclude that there is a correlation between prescription of medication Rx to patients and the subsequent development of disease Dx in those patients. Conducting clinical trials is necessary to establish a causation.
C. No inference can be made from the statistical tests about either causation and correlation between drug Rx and disease Dx.
D. There is strong evidence to conclude that there is a correlation between prescription of medication Rx to patients and the subsequent development of disease Dx in those patients. Running further statistical tests on the EHR data (observational data) will establish causation.
84 | P a g e
S23: Panel - Closing-the-loop: Transforming Medical Information into Action
--Chancey Christenson, Kenan Katranji, Gargi Schneider
S23-1: What do experts identify as the most challenging aspect of the implementation of pharmacogenetics into the clinic?
A. Translation of genetic information into clinical action
B. Test cost, test reimbursement or other economic issues
C. Availability of high quality genotyping test (CLIA approved)
D. Electronic medical record use, such as application of CDS
E. Clinician and/or patient resistance
S23-2: What type of genetic information does the MyCode program at Geisinger use?
A. Whole Gene
B. Proteins
C. Exons
D. Enzymes
E. Introns
S23-3: Of the following which is the most challenging aspect of Natural Language Processing?
A. Tokenization
B. Parts of Speech Tagging
C. Dealing with ambiguity
D. Parsing of a sentence
85 | P a g e
S23-4: Which of the following is most likely to be stored as structured data?
A. Video
B. Images
C. Text
D. Audio
S24: Presentations - Patient Deduplication, Clinical Phenotypes, and Collecting Social Determinants of Health
Developing Methods for Standardized Patient Cohort Deduplication Across the Carolinas Collaborative, A Regional Data Research Network
Robert Bradford
S24-1: A Clinical Data Research Network (CDRN) has decided on a hashing algorithm that utilizes: Last Name, Date of Birth, and Last 4 digits of the Social Security Number (SSN), where Date of Birth and Last 4 digits of the SSN are translated directly into text (with no formatting changes). Site A has the following data for their patient: First Name: John Middle Name: A Last Name: Smith Date of Birth: 01/31/53 SSN: 000-0000-7890 State: NC Which of the following options of data available at Site B would cause the de-duplication process to FAIL (hashes do not align).
A. SSN: 123-456-7890
B. Middle Name: 'null'
C. First Name: "John A"
D. Date of Birth: 31-JAN-1953
E. State: FL
86 | P a g e
S24-2: One of the methods to success in developing the Carolina's Collaborative was the ability to test the de-duplication process internally between two different data sources. If another institution were to join the Collaborative but did not have a legacy EMR to use as an internal gold standard in testing the de-duplication process, what would be the best validation approach to attempt first?
A. Implement the process without additional checks, using previous validation results from other sites as justification
B. Use only dummy data as a validation mechanism
C. Use another internal patient-centered data source, such as claims or registry data
D. Obtain an IRB approval and execute a Data Use Agreement with a collaborating site to share patient data for purposes of testing the de-duplication process
Practical Clinical Phenotype Definitions with SNOMED CT
--DuWayne Willett
S24-3: A health care system wishes to construct diagnosis groupers for a large number of clinical conditions (clinical phenotypes) to support chronic disease registries and population health initiatives. To streamline grouper construction and maintenance, they wish to explore ways to build diagnosis groupers besides a one-dimensional list of terms. Hierarchy-based groupers are possible when using SNOMED CT because:
A. SNOMED has parent-child ("is a") relationships between concepts that relate specific concepts to more general concepts
B. SNOMED has attribute relationships between concepts that specify additional defining characteristics of one of the concepts
C. SNOMED has fewer concepts than ICD
D. Clinical terminologies in EHRs are only mapped to SNOMED, and not to ICD
S24-4: Compared with groupers listing all individual codes, hierarchy-based groupers:
A. require more frequent updating when new codes or concepts are added or deprecated
B. more closely match clinical thinking about disease subtypes to include or exclude from a grouper definition
C. take more time to construct and maintain
D. are more likely to be missing relevant diagnosis clinical terms
87 | P a g e
Supporting Disparities Research by Increasing the Collection of Social Determinants of Health Data through EHRs
--Lauren Hovey
S24-5: To facilitate the standardized collection of REaL data, which of the minimum data sets provides an optimal level of granularity?
A. OMB minimum standard
B. HHS race and ethnicity standards
C. CDC Race and Ethnicity Code Set Version 1.0
D. The optimal data standards are those that are fit for purpose and the context in which the data are collected.
S24-6: Even when providers and/or health systems are motivated to collect and use REaL/SOGI data for clinical purposes, their vendor products may not be optimized for the task. Conversely, vendors may offer robust functionalities that are unintuitive or that providers are not well versed in using. For EHR products designed to facilitate REaL/SOGI collection and use, which of the following is an optimal solution to reducing provider burden during information capture?
A. Pre-populated fields related to gender/sex-specific history
B. Free text fields that allow providers to enter relevant information that may not be captured
C. Capture of the information in a pre-assessment (e.g., via portal or registration questionnaire)
D. Structured fields for REaL/SOGI that occur in a consistent place in the EHR
88 | P a g e
S25: Presentations - Methods for Enhancing EHR Usability
Categorization of EHR Design Problems Using Usability Heuristics
--Benjamin Duncan S25-1: The main goal of the heuristic categorization methodology introduced here is intended to accomplish which of the following?
A) Allows for grouping of the most significant issues in usability and to prioritize them based on their effect on workflow
B) Make suggestions to users to improve the usability of the overall system
C) Have usability experts review a chosen interface and compare it against accepted usability principles
D) Evaluators work through a series of tasks and ask a set of questions from the perspective of the user.
S25-2: Why should workflow be studied in advance of an enterprise-wide EHR conversion?
A) Make improvements to the current clinical systems being used
B) Perform log file analysis on user interactions to develop a standard workflow for EHR-mediated tasks
C) Understand variation across different clinical sites and systems in the healthcare settings
D) Determine how well clinicians can perform their specific tasks through various EHRs based off interactive behavior measures and human-computer interaction frameworks.
Disrupt and Adjust: Insights from a Contextual Inquiry of A Novel Documentation and Workflow Tool for Inpatient Providers
--Subha Airan-Javia
89 | P a g e
S25-3: Your hospital identifies a safety event in which an immunocompromised patient with recent history of extended-spectrum beta-lactamase resistant E. coli bacteremia was not started on sufficiently broad antibiotics when she developed a fever overnight, and effective antibiotics were not instituted until the error was caught 18 hours later. During root cause analysis, your team identifies that the patient’s progress notes made clear that the patient should be started on vancomycin, imipenem, and amikacin in the setting of a new fever, however the written handoff did not include this information.Which of the following systems would be most likely to prevent this error?
A. Instituting a policy in which all febrile, immunocompromised patients are immediately transferred to the intensive care unit.
B. An easily accessible documentation template for fever in immunocompromised patients.
C. Educating providers about the importance of situational awareness in the written handoff.
D. Populating the progress note & written handoff from a single source of truth
S25-4: A healthcare organization aims to increase the number of problems added to the problem list in the inpatient setting. They design a system that forces clinicians to enter each problem on the problem list before being able to document the plan related to that problem. Which of the following unintended consequences is MOST likely?
A. Clinicians will document multiple problems under a single problem heading.
B. Case-mix index will go down, leading to reduced hospital billing.
C. Clinicians will enter fewer problems on the problem list for each patient.
D. The hospital will be at higher risk of Medicare fraud from inaccurate problem lists.
S26: Presentations - Workflow Enhancements for the HER
Improving Point of Care Information Workflow with EHR Embedded Search
--Bhrandon Harris
S26-1: On average, what is the number of unanswered clinical questions that clinicians have after a patient encounter?
A. 0.25 questions per encounter
B. 0.5 questions per encounter
C. 1 question per encounter
D. 2 questions per encounter
90 | P a g e
S26-2: What are the two most commonly cited reasons that clinicians do not pursue answers to clinical questions that have arisen during an encounter?
A. Lack of time and forgetting the question
B. Forgetting the question and a lack of urgency of the question
C. Lack of time and a lack of urgency of the question
D. Lack of time and doubt that an answer exists
E. Forgetting the question and viewing question as unimportant
Integration of Patient Controlled Analgesic Infusion Forms into the Electronic Medical Record (the eAIF Project).
--MaryAnn Connor S26-3: Opioids are a class of medication that confer great and immediate benefit but also carry significant immediate and long-term risks. As pain increases and doses escalate, side effects ensue, and an opioid rotation becomes necessary. What is the best method that ensures opioid conversion and rotation are done safely?
a. Converting between opioids by hand calculation can be done in a standardized manner by clinicians
b. Electronic calculations performed using dedicated smartphone applications, or online resources
c. Use of a consistent conversion algorithm, and automated calculation embedded within the electronic medical record.
d. Conversion first to morphine, a universal opioid, and then secondary conversion to another opioid.
91 | P a g e
S26-4: The Rapid Application Development (RAD) model should be used in software development:
a. To address gaps by disruption to remove the status quo legacy system and build from the ground up focusing only on fundamentals.
b. Only when you are unable to use the preferred approach of Structured Systems Analysis and Design Method (SSADM).
c. When there is limited time and availability for users (e.g., physicians and nurses) to provide input in the design.
d. To allow users to interact with systems analysts and develop models and prototypes that represent all system processes, inputs, and outputs using a multidisciplinary team with the model changing as stakeholders weigh in.
Preoperative Nursing Workflow and Problem Solving
--Lu Zheng S26-5: A health care system wishes to reduce the cognitive burden of clinicians working in preoperative care to ensure the quality of patient-care delivery and enhance the patient safety. However, the clinicians are not typically trained in human factors or cognitive science. They may not be able to pinpoint the sources of burden. Which method is best suited to characterize cognitive burden in clinical work?
A. Semi-structured interview
B. Video-screen capture of the EHR use
C. Ethnographic study of workflow
D. Think-aloud protocol
E. A multi-method approach that triangulates evidence from different sources.
S26-6: The PreOp nurse needs to help coordinate the activities of several surgery team members, such as the anesthesiologist, the resident, the physician assistant or the surgeon, when they stop by and talk to the patient during the PreOp assessment process. The nurse needs to seamlessly and efficiently communicate to clinicians that some action is needed before the patient can be transferred to the OR. Which strategy do you think is the best way to handle this coordination process with minimal disruption given the constraints of the 30 minutes Preop assessment?
92 | P a g e
A. Write it on the blue sheet (checklist) and provide access to all clinicians.
B. Use a symbolic system (e.g., color coded magnetic strips on the exterior of the patient room) that signals to team members that certain actions are needed.
C. Call, email or text the team member before they come to see the patient, so that they will know before they enter the room.
D. Wait until they come and talk to the member directly at the patient bedside.
S27: Panel - Clinical Informatics Efforts to Combat the Opioid Crisis – Experiences from the Front Line
Barry Blumenfeld, Patrick Burns, James Huizenga, Steven Labkoff, Despina-Maria Michaels
S27-1: What major event transpired in 2016 that had a material effect on how primary care doctors think about prescribing opioid analgesics:
A. The Trump Administration published a new law on the limits on daily opioid MMEs
B. The CDC published guidelines to inform primary care and other clinicians on the most appropriate way to prescribe opioids
C. The FDA provided new guidance to prescribe abuse deterrent opioids whenever possible
D. A new formulation of an abuse deterrent opioid was approved for sale and use in the US
S27-2: With regard to the opioid crisis, the use of big data analytics is currently in use to:
A. Identify clinicians who are appropriately writing opioid prescriptions in order to single them out as good practice examples
B. Identify patients at risk for opioid overdose by use of a predictive model
C. Enhance the Prescription Drug Monitoring Programs (PDMP’s), which is a requirement in all states
D. Support integrated systems, which are now wide-spread in Health Information Exchanges across the US to help deter inappropriate opioid prescribing
93 | P a g e
S27-3: With regard to the Opioid Crisis, Clinical Decision Support tools are
A. Currently in wide scale use to deal with the opioid crisis
B. Currently not showing significant impact in cutting down the rate of opioid prescribing
C. Can bring together natural triggers inside the clinical encounter to present in-the-moment guidance
D. Generally easy to implement across a wide swath of the US due, in part, to the adoption of EHRs in the last 5 years.
S27-4: The opioid crisis presents challenges for the entire healthcare ecosystem. Specifically, the use of informatics tools with regards to the role of primary care clinicians should be to:
A. Do primary research to identify informatics tools that can be used to curb the opioid crisis
B. Provide specific data on every patient that has an encounter to ensure that registries and databases have adequate information to draw upon when making policy decisions around opioid use
C. Devote a substantial part of their practice to work with research efforts from organizations like SAMHSA, NIDA, and APA to help curb the opioid crisis
D. Leverage as many existing tools as is practical from their EHR vendors to integrate appropriate screening and documentation during routine clinical care
S28: Presentations - The Learning Health System, Optimization and Leveraging Technology - All to Improve Care
Implementation of a Learning Healthcare System for Sickle Cell Disease
-- Erin Crowgey
S28-1: A division within a health care system wishes to improve the meaningful use of data entered into the electronic healthcare records (EHR). The majority of EHR systems handle coding and liability issues well, but are not robust for clinical research and data visualization at the bedside. Which of the following options is technologically possible to enhance the downstream use and analysis of data entered directly into the EHR?
A. Have providers enter more data via free text in note sections.
B. Encourage providers to work independently on improving data entry into EHR.
C. Create a team consisting of providers and health informaticians to implement a process that supports structured data entry at the bedside.
D. Request that the health informatics department implement solutions per each individual clinician.
E. Enhance the use of IC-9/10 coding for data analysis techniques.
94 | P a g e
S28-2: A common data model (CDM) enables a systematic analysis of disparate databases. The Observational Medical Outcomes Partnership (OMOP) is one type of CDM that transforms data contained within EHR databases into a common format, or data model, with common representation of the data elements, terminologies, vocabularies, and coding schemes. Which of the following options is solution for a department looking to enhance the meaningful use of EHR data?
A. Recommend providers focus on analytical techniques that can only be deployed directly in EHR system and that are independent of any data transformation processes.
B. Create a custom CDM, or use a publicly available model, to map EHR data into a standardized format.
C. Require health informaticians to analyze EHR independently and provide PDF reports to providers.
D. Have providers within a department create their own individual lists of important data elements, and capture them inside excel.
E. Recommend to the clinicians within the department that this is a health informatics issue and does not require their effort or input.
The Veterans Health Administration’s Reproductive Health Project
--Caitlin Cusack
S28-3: Clinical Decision Support Systems (CDS) within Electronic Health Records (EHR) to reduce medication errors are widely used. Medication CDS systems range from basic systems that conduct drug-allergy, drug-interaction, and duplicate drug checking. More advanced systems incorporate data from the EHR such as conditions, laboratory results, and gender. CDS that extends to drug-pregnancy checking have limited clinical usefulness because of the lack of specificity of these alerts. One practice used with these alerts that does not improve their clinical relevancy is:
a. Remove from the cohort for rule-checking those women who have system evidence an inability to conceive, such as those with coding consistent with hysterectomy or menopause.
b. Limit alerts for women using highly effective contraceptive or those who are unable to conceive with their current partner.
c. Build a rules engine that utilizes medications classified with the former FDA categories of C, D, or X.
d. Capture as discrete data pregnancy status including “yes,” “no” or “unknown”.
95 | P a g e
S28-4: At the Veteran’s Health Administration (VHA), the Reproductive Health project uses logic on the backend to classify women as pregnant, medically unable to conceive, low likelihood of becoming pregnant, high likelihood of becoming pregnant, and unknown status. The due date of the VHA’s Reproductive Health (RH) Clinical Reminder Dialog is based on these classifications. Which of the following women is not due now to have the RH dialog completed by her physician?
a. Ms. Smith age 38 is new to the VHA, uses an IUD for contraception, and presents for her first preventive care visit.
b. Ms. Jones has a recorded estimated due date (EDD) in the system from 4 weeks ago, but has not been seen within the VHA during her pregnancy.
c. Ms. Noonan has a history of a hysterectomy that has been recorded in the system.
d. Ms. Johnson has a completed RH dialog from 2 months ago. She is using Depo-Provera for contraception, but is late for her next injection.
Leveraging Technology to Reduce Unnecessary Post Procedure Office Visits for Chemo Port Placement
--Christine VanZandbergen
S28-5: A provider wishes to replace a post-procedure in-person follow-up visit with a virtual visit. Assuming the technology is in place to support virtual workflow, what is the most appropriate next step?
A. Tell all patients to register for a patient portal account and start sending their wound sites photos.
B. Engage the appropriate clinical, regulatory and information systems partners to validate the workflow and ensure appropriate support/monitoring.
C. Tell patients to take a photo of their wound and text it to you
D. Set up the visits as billable encounters
S28-6: Which of the following is a potential benefit of replacing post-procedure follow-up visits with virtual visits?
A. Increased revenue through new billable services
B. Decreased infection rates
C. Improved follow-up with post-procedure wound checks
D. Increased provider satisfaction
96 | P a g e
S29: Presentations - Translating Clinical Decision Support to Practice
Translating evidence into practice: Effect of an Evidence-based Electronic Clinical Decision Support Tool on Provider Ordering for Venous Catheters
--Emilia Flores
S29-1: The 5 Rights for clinical decision support (CDS) interventions (right information, to the right person, in the right format, through the right channel, at the right time in the workflow) are a framework for designing and deploying CDS interventions. Catheter-associated urinary tract infection (CAUTI) is a common device-associated infection in the acute care setting. However, misdiagnosis of catheter-associated asymptomatic bacteriuria as CAUTI can lead to inappropriate use of antibiotics and related consequences, including medication adverse effects and antibiotic resistance. Appropriate urine culturing is dependent on a clinician’s ability to recognize a specific set of symptoms. You notice an increase in inappropriate urine culture ordering in your organization and wish to implement a CDS to ensure that urine culturing is aligned with evidence-based recommendations. Which CDS type would be the most appropriate for this clinical issue?
A. A predictive analytics tool to identify patients at high risk for CAUTI
B. A report containing a summary of patients that meet culturing criteria for CAUTI is available in a reporting module or tab and is accessible to all clinicians
C. An order set that leads the provider through a series of questions based on CAUTI culturing criteria and provides recommendations based on provider responses
D. Order set that defaults to a reflex-urine culture order if the patient has a urinary catheter. A reflex urine culture order is one in which the lab will first conduct a urine analysis and if criteria are met (e.g. urine analysis with >= 10-20 white blood cell count/hpf), the lab will then perform a urine culture to assess for CAUTI.
S29-2: Zhang and Walji (2011) have proposed 14 evidence-based interface design principles that are important to consider for any electronic health record (EHR) embedded clinical decision support (CDS), as they can have a significant impact on user acceptance and satisfaction. (Middleton et al., 2013) Which of the following interface features is an example of a design principle?
A. A CDS tool to promote antibiotic stewardship asks users to manually type in the patient diagnosis associated with a medication order rather than select from a drop-down list
B. A CDS tool for vascular access selection displays relevant patient information and calculations within the CDS interface to aid in decision making, thus avoiding having the user reference and remember information from other screens.
C. A CDS development group organizes and displays data entry fields in all CDS tools alphabetically
D. A CDS tool for vascular access selection utilizes drop down lists rather than check boxes for the five most common medications infused through venous access devices
97 | P a g e
Clinical Pathway Regret Risk Differences Between Affluent and Underserved Populations
--Robert Aarhus
S29-3: Decision Regret is a well-studied patient-centered outcome measuring the quality of the medical decision-making process. Using validated query instruments, researchers have determined key characteristics of decision regret in attempts to understand, retrospectively, the origin and trajectory of patient dissatisfaction with those decisions that have produced disappointing or unexpected healthcare results. In particular, many randomized controlled trials have examined effects of enhanced patient information, or conformity to the level of physician participation in the process expected by patients. According to the literature, which of the following is a characteristic of treatment decision regret?
A. It is negatively correlated with decisional conflict
B. It often increases over time in breast cancer cases
C. It sometimes differs according to the patient's race and insurance status
D. It is uniformly decreased through the use of decision aids
E. It is seen in a majority of complex cancer cases
S29-4: To prospectively assess the likelihood of future regret, a model has been developed to include estimates of regret risk. A novel term in this context, regret risk is a statement of the likelihood of future decision regret driven by both differences between patient expectation and outcome, as well as the curse of the counterfactual narrative -- a seemingly certain and more positive alternative future that might have happened had past choices been different.Key assumptions, based on decision regret research, have driven regret risk formulation and expression. According to the presented model's hypothesis, which of the following could explain differences in long-term cancer regret risk between socioeconomic groups?
A. An emphasis on aleatory rather than epistemic uncertainty
B. A change in action space
C. Linear dependence on measurable decision regret
D. Cultural differences between patients and health care providers
98 | P a g e
S30: Panel - The Use, Usability, and Experience of Implementing a Systemwide Inpatient Portal (IPP)
Jennifer Hefner, Timothy Huerta, Ann McAlearney, Daniel Walker
S30-1: When implementing an inpatient portal that includes secure messaging between care team members and patients, which of the following should be included in front-line staff training?
A. Allow care team members to engage with the tool from the perspective of the patient.
B. Inform care team members that this is the most effective way to communicate with patients now and the health system will be using it exclusively.
C. Provide them with a training manual to read during breaks.
D. Frontline staff will not be using the tool as patients do so they do not need training in this manner.
S30-2: Studying patient and provider use of an inpatient portal requires analysis of log files - server-side records of actions taken by a user - to assess how, when, and which features of the inpatient portal were used. Which of the following is not an issue to consider when analyzing log files?
A. How to find the full-text of the electronic messages sent between patients and the care team.
B. The application may create administrative data (e.g., pushed data to ensure that the tablet does not display stale data) that is unrelated to patient action, resulting in artifacts in the data.
C. The application itself does not offer an indicator of a Session and as such the research team must construct an operational definition to represent a patient’s single continuous use of the tablet for the purpose of using MCB.
D. It may take several days for a patient to get their tablet, and as such length of stay may suffer from distortions associated with provisioning. As a result, patient length of stay (LOS) may not be an appropriate framing for use, because of real-world demand on provisioning practices.
S30-3: What approach(es) to learning do users prefer to familiarize themselves with the inpatient portal?
A. Free exploration
B. Video tutorial
C. In-person training
D. Blended – including free exploration, video tutorial, in-person training
99 | P a g e
S30-4: Which of the following theoretic constructs is not a valid approach to measuring use of patient portal use.
A. Comprehensiveness
B. Coherence
C. Duration
D. Frequency
S31: Presentations - Analysis of Medical Data and Implication for Value-based Care
A clinical informatics consult service to use aggregate patient data at the bedside
--Saurabh Gombar
S31-1: The ideal clinical decision is supported by large multicenter randomized control trials. However, less than 20% of clinical decisions are based on RCT data. Furthermore, an estimated 20% of clinical decisions are not based on published evidence at all. Given this limitation, physicians have long wanted to untap the electronic medical record (EMR) to quickly generate practice-based evidence relevant to the clinical decision they are facing. To date which of the following has NOT been a limitation to rapidly assimilating knowledge from the EMR?
A) Tools to build a cohort of patients who faced a similar clinical scenario are not widely available.
B) The multitude of data types in the EMR (ie ICD, CPT, Notes) require physicians to be proficient with a multitude of medical nomenclature syntaxes
C) Analyzing large datasets is too time intensive for busy physicians
D) Physicians are uncomfortable using evidence generated from EMR data.
100 | P a g e
S31-2: Stanford’s approach to enable precision medicine through aggregate patient data utilizes a consult service. This service is offered by a multidisciplinary team comprised of an informatics physician, an EMR data specialist to create cohorts from the EMR, and a data scientist to perform statistical analysis. Which of the following is NOT a reason utilize a consult model to generate aggregate patient data?
A) The pathway from clinical question to useful information requires a multitude of disparate steps best handled by a team with complimentary skills.
B) The consult model is a model that physicians are comfortable getting subspecialist information from.
C) An asynchronous model allows for quick turnaround time and can be scaled to ever increasing demand.
D) A consult model is already billable as long as a note is generated for the patient.
101 | P a g e
The Winning Health IT Engagement Model of an ACO in a Value-based Setting
--Nathan Riggle
The presenter will provide two multiple choice questions at the end of the presentation.
102 | P a g e
S32: Presentations - Patient Portals: Challenges and Opportunities
Patient Portal Proxy Access for Adolescents: Preserving Confidential Care While Enhancing Parent Proxy Access
--Margaret Punch
S32-1: The parent of a 13-year-old would like full access to their child’s medical information available through an on-line portal at their local health center. What considerations should be given to this request?
a. The parent always has full rights to access the minor’s medical record.
b. Minor patients can’t consent to their own health care under any circumstances.
c. In all 50 states in the United States there are varying statutes that allow minors to seek confidential care and that information can legally be withheld from the parent or legal guardian.
d. In all 50 states in the United States 13-year-olds have the right to confidential care to access contraceptive services.
S32-2: When enabling proxy access to the parent of a minor child which portal feature may need restriction?
a. Demographics
b. Vaccine records
c. Records from birth to age 10
d. Appointment schedules
e. Allergies
103 | P a g e
Do Community Clinic Patients Have Internet Access for Patient Portal Use?
--Ruth Bush
S32-3: Previously identified barriers to patient portal adoption include differences in internet access, computer literacy, and internet proficiency. This digital divide is one explanation for underserved groups being less likely to use portals, with strong correlations to utilization by race and household income.
In the multi-ethnic, lower income community clinic in this study, which of the following was significantly associated with previous patient portal use?
a. Having a “smart” phone
b. Education level
c. Health status
d. Race/ethnicity
S32-4: When reporting past use as well as likely future use of the patient portal, participants reported using all of the following functions of the portal EXCEPT:
a. Contacting provider
b. Renewing prescription
c. Obtaining health information
d. Scheduling appointments
Development of a Taxonomy of Secure Messages from an Inpatient Portal
--Cynthia Sieck
S32-5: On what dimensions can secure messages be classified?
A. Sender, Type and Topic
B. Type, Inpatient and Outpatient
C. Urgent, Non-urgent and Medical
D. Dietary, Medical and Non-medical
104 | P a g e
S32-6: Which of the following would be applications of a secure message taxonomy?
A. To improve the functionality of the secure messaging feature of the portal.
B. To help patients view their lab results.
C. To help care team prioritize responses.
D. A and C.
S33: Panel - The Clinical Informatics Fellowship: A Fellow’s Perspective --Christian Dameff, William Lancaster, Chase Parsons, Melissa Van Cain S33-1: Clinical Informatics Fellowships are accredited by which organization?
A. The National Institute of Health (NIH)
B. The Accreditation Council for Graduate Medical Education (ACGME)
C. The Association of American Medical Colleges (AAMC)
D. The Center for Medicare and Medicaid Services (CMS)
S33-2: Clinical Informatics Fellowship requires the completion of what primary residency?
A. Internal Medicine
B. Emergency Medicine
C. Pediatrics
D. Any ACGME-recognized primary specialty
105 | P a g e
S33-3: The responsibilities of Clinical Informatics fellows include:
A. Research in informatics, patient care, implementing Informatics solutions
B. Patient care, research in informatics, and working in a basic science lab
C. Implementing informatics solutions, research in informatics, and serving as a medical librarian
D. Research in informatics without any dedicated time to patient care
S33-4: The duration of the clinical informatics fellowship is:
A. 1 year
B. 2 years
C. 3 years
D. 4 years
S34: Presentations - Tool Based Analytics
Institution of a Pop Up Reminder Decreases Opioid Prescriptions for Pediatric Patients in the Emergency Department
--Melissa Van Cain S34-1: The opioid epidemic has had a serious effect on the US population in recent years and this is true of adolescent patients as well as adults. What is the approximate number of adolescents who are addicted to prescription pain relievers?
A. 1,500
B. 15,000
C. 150,000
D. 1.5 million
106 | P a g e
S34-2: Massachusetts (MA) enacted a law in 2016 that providers are required to check the MA prescription drug monitoring program before prescribing opioids. We achieved the biggest change in opioid prescriptions with this intervention:
A. Adding a link to the chart
B. Reminder e-mails
C. Physician training
D. Pop-up reminder
E. Hard stop
Radiation Therapy Data Curation: Utilizing Interdisciplinary Expert Knowledge via an Iterative Tool-based Process
--Thilo Schuler
S34-3: Manual data curation is currently the most common approach to retrospectively improving data quality in clinical datasets.
Which of the following options is the greatest challenge with this approach?
A. Unsuitability for inter-institutional data exchange
B. Poor scalability
C. Poor accuracy
D. Lack of contextual knowledge
107 | P a g e
S34-4: Using a fully automated curation approach for large clinical datasets would be ideal, as it is scalable and requires minimal clinician input. However, in most settings this is at the moment not achievable.
Which of the following strategies is most successful in reducing the time imposed on clinical expert curators?
A. Using non-expert curators
B. Focus on subset of data
C. Technology support
D. More training for curators
Developing a visual annotation tool to rapidly collect expert-annotated weight errors in pediatric growth charts
--PJ Van Camp
S34-5: The clinical importance of a child’s weights being entered correctly into the electronic health record is to:
A) correctly classify children as obese or malnourished
B) avoid potential medication dosing errors
C) avoid useless information being stored and taking up valuable disk space
D) ensure correct reimbursement from insurance companies
S34-6: What is a major benefit of creating a software application to let experts annotate a large dataset instead of manual annotation without the use of a specific tool?
A) The process of gathering, annotating and saving the data can be sped up and it allows annotators unfamiliar with data analysis software to interact with the data in an intuitive manner.
B) Data annotated digitally is more credible than data annotated by hand because the latter is more prone to error
C) Once a tool has been developed, it can be used for many other projects that require different dataset annotations
D) Only experts can annotate data manually, while with a tool anyone can
108 | P a g e
Closing Session and Keynote Panel: Building Clinical Decision Support that Works
--Adam Wright, Joan Ash, Dean Sittig, Bill Galanter, Sarah Collins
S35-1: A healthcare system wants to reduce the time it takes to clinical decision support system malfunctions. Because the organization participates in several quality programs, the organization’s leadership is particularly concerned about “false negative” situations, where CDS is supposed to fire, but doesn’t. Which of these strategies is most likely to quickly detect “false negatives”?
A. User reporting through the help desk
B. Ongoing monitoring with anomaly detection
C. Routine clinical use of the system by informatics leaders
D. Sentiment analysis of override reasons
S35-2: A few years after implementation of an Alert type of CDS, or "rule," there are a series of complaints about how it is functioning. The rule was designed by a requesting physician and IS staff. The requesting physician explained what they were looking to have built in a few paragraphs and then it was approved by the CDS committee and built by an IS analyst. A new IS analyst is having a hard time determining if the rule is doing what it was intended to do as the paragraph describing the intent is not clear and does not provide sufficient detail.
What is the best strategy to prevent this type of difficulty?
a) Use clinical informaticists to investigate potential rule malfunctions.
b) Require rule requests to be formatted as flowcharts with sufficient details to allow IS to interpret unambiguously and future clinicians and IS to understand the intent.
c) Require the initial requesting clinician to test all CDS and be responsible for any future queries once the CDS receives approval.
d) Check the override rate of the rule, if greater than 85%, turn the rule off.
109 | P a g e
S35-3: Leaders of a large healthcare system want to optimize their EHR by providing more effective CDS with minimal malfunctions. Which of the following “people” strategies might help the most?
a) Set up a CDS committee of subject matter experts.
b)Involve clinical staff and encourage them to report malfunctions such as alerts not firing when they should.
c) Assure that staff members in informatics, quality improvement, IT, and elsewhere liaise with one another and with clinicians in multiple ways.
d)Hire additional staff to do measurement and monitoring because only they can consistently identify malfunctions.
S35-4: You are leading the governance committee for clinical decision support at your institution. A new request for development of CDS that is nurse-facing was submitted. The request is to implement an evidence-based risk assessment for peripheral nerve damage in the operating room with guidance to support appropriate interventions. Your organization leverages the Data-Information-Knowledge-Wisdom (DIKW) Conceptual Framework to prioritize strategic development of CDS to support nursing practice. Which component of the framework best categorizes this CDS request?
a. Data
b. Information
c. Knowledge
d. Wisdom
110 | P a g e
Tuesday, May 8th
ANSWERS, EXPLANATIONS, REFERENCES WS02: Organizational Issues and Informatics: Translating Theory into Practice
--Kim Unertl, Saira Haque
WS02-1: When implementing health information technology, why is it a good idea to consider organizational theory when developing an implementation plan?
A. Academics love theory
B. To help identify concepts that are relevant for informatics practice
C. It doesn’t make a difference
D. To inform your organization as a whole
Answer: B. To help identify concepts that are relevant for informatics practice
Explanation: Understanding theory is crucial to avoiding past mistakes and developing new insights.
Reference: Workshop presentation materials
WS02-2: What paradox led to the research that established Sociotechnical Systems Theory?
A. The Profit Paradox, that profits were elevated when new technology was implemented
B. The Happiness Paradox, that workers were happier when their work was going poorly
C. The Productivity Paradox, that despite improved equipment and better working conditions, productivity went down
D. The Despair Paradox, that the better technology got, the less happy workers became
Answer: C. The Productivity Paradox, that despite improved equipment and better working conditions, productivity went down
Explanation: The Tavistock Institute was tasked with identifying the reason that productivity and morale decreased in the British coal industry in the 1940s, despite the fact that there was better equipment, higher wages, and amenities like housing provided for workers. They referred to this situation as the Productivity Paradox.
Reference: Fox W, Sociotechnical System Principles and Guidelines: Past and Present, Journal of Applied Behavioral Science 1995;31(1):91-105.
111 | P a g e
WS02-3: Everett Rogers identified four major elements in how innovations are adopted in groups, as part of his Diffusion of Innovations Theory. What were the four elements?
A. The Innovation, Communication Channels, Time, A Social System
B. The Bosses, The Workers, The Tools, The Organizers
C. Advertising, New Technology, Coordinators, End Users
D. Great Ideas, Software Developers, Sales Representatives, Time
Answer: A. The Innovation, Communication Channels, Time, A Social System
Explanation: Rogers analysis of adoption of new farming technologies categorized components of innovation according to the four groups in the answer. Each component was a crucial piece of Diffusions of Innovation Theory.
Reference: Everett Rogers. Diffusion of Innovations. 3rd Edition, The Free Press, 1983
WS02-4: There are five stages in Rogers’ Innovation-Decision Process. What is a major influence on the Persuasion stage?
A. Whether people are really interested in making changes
B. How many people are interested in adopting the innovation
C. How people perceive the innovation
D. How long people have been using their past tools
Answer: C. How people perceive the innovation
Explanation: Although the other answers are potential influences, the major influence is how people understand and interpret what the innovation is and how they can use it in their own work.
Reference: Everett Rogers. Diffusion of Innovations. 3rd Edition, The Free Press, 1983
112 | P a g e
WS02-5: In applying Lewin's Change Model to an organization that is considering a major health information technology change, what strategies might be useful in "unfreezing" an organization?
A. Making sure the opinions of people who oppose the new technology are ignored
B. Avoiding contact with the intended end users of the technology
C. Firing anyone who disagrees with the choice of new technology
D. Involving all levels of intended users, from front-line workers to management, in the decision-making process
Answer: D. Involving all levels of intended users, from front-line workers to management, in the decision-making process to get their buy-in
Explanation: The "unfreezing" stage requires examining the environment and creating the conditions needed for change. By listening to end users at all different levels of the organization in the decision-making process, it builds organizational momentum for change and helps to get people who might be hesitant on board with making changes.
Reference: Burnes B. Kurt Lewin and the Planned Approach to Change: A Re-appraisal. Journal of Management Studies 2004;41:977–1002.
WS02-6: Why is the low point of the Change Curve Model sometimes referred to as the "Valley of Despair"?
A. People like to whine
B. People worry for no reason when productivity goes down
C. When productivity dips because of new technology, people have a hard time understanding how the technology might help them in the future
D. The implementation productivity dip does not actually exist, but is rather just a theoretical model that has been disproven by years of technology implementation
Answer: C. When productivity dips because of new technology, people have a hard time understanding how the technology might help them in the future
Explanation: The implementation productivity dip has actually been proven repeatedly in technology implementations across organizations and fields. Depending on how a new technology is discussed before implementation, end users might not understand that a loss of productivity is an expected part of an organization coming back up to speed after a major change. This causes end users to lose confidence in the technology and to worry that it's not possible to actually get back to where they were before the new technology.
Reference: Elrod PD, Tippett DD, The “death valley” of change, Journal of Organizational Change Management, 2002, 15(3):273 – 291
113 | P a g e
WS02-7: Many organizations refer to the day a new technology system is turned on as a “Go Live” date. Why is this a term that we are better off avoiding?
A. Going “live” makes people think about death
B. Implementing technology occurs over time and “go live” implies it is a single point in time
C. Customers/patients might think we are talking about medical procedures
D. People are unnecessarily picky about language use
Answer: B. Implementing technology occurs over time and “go live” implies it is a single point in time
Explanation: Technology-related change is a process, not an event. By using the term "go live," we imply that a technology change is happening at a point in time. However, the moment a system is turned on is just a fraction of the change new technology creates. It is unnecessary jargon that means something different to technology people than to end users.
Reference: Presentation materials and class discussion
WS02-8: What is one reason that people worry about change?
A. No one worries about change – it’s just a part of life
B. Concerns about wasting time and effort
C. People who worry about change are irrational
D. People enjoy finding new things to worry about
Answer: B. Concerns about wasting time and effort
Explanation: Although sometimes people may appear to be irrational when it comes to change, often these concerns are rooted in basic ideas, like worrying about loss of productivity. Given the large body of evidence supporting the idea that new technology often leads to a productivity dip, people have reasons to be concerned about wasted time and effort.
Reference: Lorenzi NM, Riley RT, Blyth AJC, Southon G, Dixon BJ. Antecedents of the People and Organizational Aspects of Medical Informatics: Review of the Literature. JAMIA 1997;4(2):79-93.
114 | P a g e
WS03: How to Properly Onboard Doctors in the Age of Informatics & Digital Health
--Hamed Abbaszadegan, Nacchal Nachiappan, Shakaib Rehman
WS03-1: As a busy clinician, there is no time to learn a new EHR system in a three-day period. Patients need to be seen. What is your approach to onboarding this busy physician:
A. Cancel Clinic
B. Reschedule patients
C. Find coverage
D. Block time 4 months in advance
Answer: D. Block time 4 months in advance Explanation: Advance preparation is ideal for properly onboarding providers. Additionally, having the providers have some initial workflow experience before attending class makes the experience stronger. References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001. Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249. WS03-2: As a long-time provider at your local facility, you never really learned how to use the EHR nor keep up with new changing widgets and apps. How can you attend New Provider Orientation if you are not new?
A. Displace the new provider
B. Schedule special sessions
C. Advanced planning of participants
D. New Provider Onboarding is only for new providers
Answer: C. Advanced planning of participants Explanation: New Provider Orientation is not just for new providers. As technology changes and the EHR platforms evolve, you can plan in advance and invite any current outstanding provider in your health system. References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001.
115 | P a g e
Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249. WS03-3: Every hospital has a unique set of features and health IT tools that they invested in. How is curriculum established?
A. Given dynamic changes in the health IT world, curriculum can change at any time
B. Teaching is universal and content for onboarding is usually uniform
C. New digital toys should be brought on display for new providers
D. Once you know which providers attend, content should be tailored to fit
Answer: A. Given dynamic changes in the health IT world, curriculum can change at any time Explanation: In the informatics and digital age, content changes as rapidly as the technology advances. This is why you must be flexible in how you prepare onboarding. References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001. Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249. WS03-4: Shorthand and macros can make any process faster. Given EHRs have more than one way to do the same thing, what approach should be taken when onboarding?
A. Show all methods of shortened processes
B. Discuss approaches to macros and let the providers decide what is best
C. Only show what works for the majority
D. Discourage using quick methods for processes
Answer: B. Discuss approaches to macros and let the providers decide what is best Explanation: There are many ways to do the same things in all aspects of life. EHRs and informatics platforms are no different. This is often a robust discussion with showcasing of only a few methods. Each provider can subsequently decide what their preferred process is.
116 | P a g e
References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001. Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249. WS03-5: Health care providers are diverse in their workflows. How can this diversity be a part of your onboarding process?
A. Too much work flow diversity cannot be overcome
B. Customize a trainer depending on who attends class
C. Ensure representation of both in/outpatient settings for instruction
D. Any provider can teach class as long as they have EHR experience
Answer: C. Ensure representation of both in/outpatient settings for instruction Explanation: Healthcare providers live in different domains. It is not common any longer to have both inpatient and outpatient providers covering both domains. Thus, needs and workflow processes are completely different. A tag-team approach will ensure both settings are represented. References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001. Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249. WS03-6: Providing health care now involves utilization of advance messaging techniques such as secure messaging through web portals. How can doctors be more efficient in processing of secure messaging?
A. Show the providers in detail how to send messages
B. Review all bells and whistles with advanced secure messaging
C. Set up teams associated with providers who can help resolve appropriate messages
D. Ensure that rules of timeliness of secure messaging is well explained
Answer: C. Set up teams associated with providers who can help resolve appropriate messages
117 | P a g e
Explanation: Teaching is an important part of onboarding, but also establishing profiles for providers will do wonders for efficiencies. Setting up clinic profiles with additional individuals who can respond to secure messaging will decrease the load of work off of your new provider. References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001. Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249. WS03-7: Navigating a new health system can be daunting with all the rules. What approach will yield a higher likeliness of success?
A. Establish a buddy mentor and organize forms in easy-to-find locations
B. Show which forms matter to physicians and demonstrate how to complete them
C. Invite departmental secretaries to training sessions
D. Teach the providers medial assistant how to obtain forms
Answer: A. Establish a buddy mentor and organize forms in easy-to-find locations Explanation: If professional athletes have numerous mentors and coaches, then your provider can also benefit from mentors and coaches. Partnering providers with mentors who have experience in navigating your health system will help them feel they can reach to resolve problems as they arise. References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001. Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249. WS03-8: Pharmacy prescribing practices such as opiates have recently been highly publicized. Which approach will support providers as they start with your healthcare system?
A. Have a pharmacist team member teach how to write prescriptions in a rapid manner
B. Teach how to build quick order sets
C. Invite your pain management director along with your pharmacy team member to review support roles and processes available
118 | P a g e
D. Show providers how to rapidly access decision support tools via phone apps
Answer: C. Invite your pain management director along with your pharmacy team member to review support roles and processes available Explanation: Involving hot-topic related guests to your onboarding class can help put new providers at ease. This will show your health care system has a strong support structure and that you understand pain points in healthcare systems. References: Kirk VG, Currie G, Dewey D, Howlett A, Lemiux L. Innovation in physician recruitment & wellness: a pilot orientation program. J Pediatr. 2013 Apr 162 (4):664-665.e2. doi: 10.1016/j.peds.2013.01.001. Wright M, Mankey CG, Miller BW. Improving upon the ‘July effect’: a collaborative, interdisciplinary orientation for internal medicine interns. Med Educ Online. 2013 Dec 23; 18:23249. doi: 10.3402/meo.v18i0.23249.
WS04: Hands-On Full Life Cycle Data Science Workshop
--Steven Johnson, Lisiane Pruinelli, Tamara Winden
WS04-1: When using EHR data in an observational study, which of the following is correct? IRB review…
a. Does not apply since this is not a clinical trial.
b. Is not needed since the data is being collected in the EHR to document care
c. Is not needed since we are only accessing the EHR and not interacting directly with patients
d. Is required since we are using patient-specific health data
Answer: d. is required since we are using patient-specific health data
Explanation: IRB review is always required when identifiable patient information is used in a study, even if the patient isn’t directly involved.
Reference: https://aspe.hhs.gov/report/feasibility-using-electronic-health-data-research-small-populations/privacy-and-security-conditions-required-research-using-ehr-and-other-electronic-health-data
119 | P a g e
WS04-2: How do data quality issues impact data science results?
a. Because EHR data is used to document healthcare, there are no data quality issues
b. Data quality can significantly impact results, making them biased or even invalid
c. Data quality issues can be ignored because machine learning and AI will take care of it
d. Poor data quality doesn’t impact results, because the data quality issues disappear with enough data
Answer: b. Data quality can significantly impact results making them biased or even invalid
Explanation: It is critical to understand how complete, consistent and correct your data is for the purpose that you have for the data. Having more data may just give you more errors and using any data science technique on incorrect data will likely produce incorrect results.
Reference: Brown J, Kahn M, Toh S. Data quality assessment for comparative effectiveness research in distributed data networks. Medical care. 2013;51(8 0 3):S22-S29. doi:10.1097/MLR.0b013e31829b1e2c.
WS04-3: A good practice for exploratory data analysis is to use a number of techniques to visualize your dataset. Which of the following is NOT a good reason to visualize your data?
a. Data visualization allows you to better understand and ”get to know” your data
b. Humans can spot visual patterns much better than looking at descriptive statistics
c. Visualization allows you to see large amounts of data summarized and simplified
d. Visualization looks nicer than just having tables of numbers
Answer: d. Visualization looks nicer than just having tables of numbers
Explanation: The reason visualization is so important is that it engages the visual pattern matching center of our brain to allow you to see relationships and trends in data that are hard to discern from tables of numbers.
Reference: Tufte, E., & Graves-Morris, P. (2014). The visual display of quantitative information.; 1983.
120 | P a g e
WS04-4: After receiving a dataset for a data science project, a researcher will spend a significant amount of the total project time performing data preparation and exploratory data analysis. Which of the following is NOT a data preparation task?
a. Assessing missing data and imputing appropriate values or removing records
b. Adding variables, transforming data and joining data tables
c. Ensuring all data is of the appropriate type and is consistent with the data mode
d. Obtaining the appropriate level of IRB approval for the project
Answer: d. Obtaining the appropriate level of IRB approval for the project
Explanation: Obtaining the appropriate level of IRB approval for the project should be done before you request and receive your data. There will be significant time spent in preparing your data for analytics including dealing with missing information, inconsistent formats and creating additional variables from the underlying data. Data preparation, visualization and exploration can consume 80% of the time spent on a data science project.
Reference: Pyle, D. (1999). Data preparation for data mining (Vol. 1). Morgan Kaufmann.
WS04-5: Many datasets have imbalanced data where there is a much smaller amount of data for positive classes than negative ones. For example, the prevalence of a number of diseases is low in the overall population (i.e. Ebola). Which is the best method to evaluate the performance of a classification model for imbalanced data? a. Accuracy
b. F1 Score
c. Precision
d. Recall
Answer: b. F1 Score
Explanation: Imbalanced data has a smaller amount of positive class data. Precision, which is TP / (TP + FP) doesn’t take into account FN and TN and Recall, which is TP / (TP + FN) doesn’t take into account FP and TN which can lead to biased results.
F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. Intuitively it is not as easy to understand as accuracy, but F1 is usually more useful than accuracy, especially if you have an uneven class distribution.
References: https://www.kdnuggets.com/2016/12/best-metric-measure-accuracy-classification-models.html
http://blog.exsilio.com/all/accuracy-precision-recall-f1-score-interpretation-of-performance-measures/
121 | P a g e
WS04-6: Once data are properly prepared and cleaned, researchers should select the most appropriate modeling algorithm depending on the data and the research question. Which modeling technique does NOT work well on categorical variables?
a. Classification
b. Linear Regression
c. Neural Network
d. Random Forest
Answer: b. Linear Regression
Explanation: Linear regression is appropriate for continuous variables. Categorical variables can be directly used in random forest models. Techniques, such as “one hot encoding, ” can be used to incorporate categorical variables into neural networks and other classifiers. Reference: Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J. (2016). Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.
WS04-7: Once you have a good performing model and you have tested it sufficiently on your training data, it is time to implement it in the real world. Which tasks are essential for successful model deployment?
a. Ensure the model is activated at the right time in the workflow, appropriately alerts the clinician and allows specific action
b. It is not necessary to get buy-in from the clinicians, since the model embedded in the EHR will tell them what to do
c. Such models should only be created for research purposes and should not be used to support direct patient care
d. The model should only be implemented in the latest version of the EHR software
Answer: a. Ensure the model is activated at the right time in the workflow, appropriately alerts the clinician and allows specific action
Explanation: Models not only have to be predictive, but the prediction needs to be presented to a clinician at a time in a clinical workflow when they can use the information to perform an action or intervention to potentially improve an outcome. Clinicians must be intimately involved in developing and deploying the model for it to be successful.
122 | P a g e
Reference: Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., ... & Middleton, B. (2003). Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. Journal of the American Medical Informatics Association, 10(6), 523-530.
WS04-8: Data science is fast becoming very useful in healthcare, but success requires close collaboration between people with clinical knowledge and people with data and computer expertise. Which of the following is true about data science in healthcare? Data science…
a. Can be understood sufficiently by everyone in the healthcare organization
b. Is easy since we give raw EHR data to machine learning algorithms to produce highly accurate results c. Is somewhat of a fad and it won’t have a significant impact in improving healthcare d. Only needs to be understood by a few people in the healthcare organization
Answer: a. Can be understood sufficiently by everyone in the healthcare organization
Explanation: If everyone involved in a data science initiative understands the basic concepts of how data science works, the project will have a higher probability of success. Some of the team members will have more clinical knowledge and some will have more technical knowledge, but everyone can speak the same “data science” language and can help reduce the knowledge gap. Having many knowledgeable people participate in date science initiatives improves the chances for success.
Reference: https://www.cio.com/article/3234353/analytics/the-secrets-of-highly-successful-data-analytics-teams.html
WS05: Operational and Practical Aspects of Clinical Knowledge Management
--Dominik Aronsky, Saverio Maviglia, Asli Ozdas Weitkamp, Roberto Rocha, Dirk Wenke
WS05-1: What is a logical first step in starting a clinical knowledge management program within the iterative knowledge management framework?
a) Build standards-based interoperable knowledge assets
b) Centralize the process of creating knowledge assets
c) Create a catalog of existing knowledge assets including relationships with metadata
d) Establish a knowledge asset lifecycle
123 | P a g e
Answer: c) Create a catalog of existing knowledge assets including relationships with metadata
Explanation: An initial assessment of all institutionally available knowledge assets is desirable understand the level of existing knowledge, identify redundancies, identify gaps, and responsibilities.
Reference: Clinical Decision Support - The Road to Broad Adoption. Edited by Robert A. Greenes, Academic Press ISBN 978-0-12-398476-0
WS05-2: Which of the following activities is least likely within the scope of a clinical knowledge management framework?
a) Build a sustainable knowledge asset framework with reusable components
b) Integrate knowledge assets into clinical workflow
c) Maintain the representation of knowledge assets
d) Provide knowledge lifecycle support
Answer: b) Integrate knowledge assets into clinical workflow
Explanation: Knowledge management deals with the creation and maintenance of an institution’s knowledge assets, including sustaining the knowledge assets, maintain the representation, and provide strict lifecycle support. The aspects of (clinical) workflow integration are commonly not represented in a clinical knowledge management environment.
Reference: Clinical Decision Support - The Road to Broad Adoption. Edited by Robert A. Greenes, Academic Press ISBN 978-0-12-398476-0
WS05-3: Which topic is least likely associated with clinical knowledge management?
a) Institutional data governance
b) Lifecycle management of knowledge assets
c) Management of knowledge meta-data
d) Single-login management for clinical workstations
e) InfoButton
Answer: d) Single-login management for clinical workstations
124 | P a g e
Explanation: Single-login management is least likely modeled in a clinical knowledge management environment - the other topics are typical KM activities.
Reference: Clinical Decision Support - The Road to Broad Adoption. Edited by Robert A. Greenes, Academic Press ISBN 978-0-12-398476-0
WS05-4: Which process is not directly supported by a clinical knowledge management system?
a) Centralized prioritization of knowledge management implementation tasks b) Centralized request management for changes in the knowledge base
c) Monitoring clinicians’ usage of clinical decision support systems d) Versioning of knowledge assets
Answer: c) Monitoring clinicians’ usage of clinical decision support systems
Explanation: Once a knowledge asset, such as a rule, an order set, or pathway, is implemented for operational use in the target system, its monitoring is accomplished in other environments, such as the target system itself (e.g. EMR, LIS), a Business Intelligence (BI) or Enterprise Data Warehouse (EDW) environment, or similar.
Reference: Clinical Decision Support - The Road to Broad Adoption. Edited by Robert A. Greenes, Academic Press ISBN 978-0-12-398476-0
WS05-5: Which characteristic is NOT considered best practice for creating knowledge assets?
a) Interoperable
b) Modular
c) Recursive
d) Reusable
Answer: c) recursive
Explanation: Modular, reusable and interoperable are desirable key characteristics for best practices in the creation of knowledge assets. Recursive patterns are unrelated to knowledge asset creation and are best known in programming and describe a function or procedure that can repeat itself.
Reference: Clinical Decision Support - The Road to Broad Adoption. Edited by Robert A. Greenes, Academic Press ISBN 978-0-12-398476-0
125 | P a g e
WS05-6: Which of the following is the least critical method applied in clinical knowledge management?
a) Formal representation of knowledge assets
b) b) Lifecycle management of knowledge assets
c) Management of knowledge asset reference material
d) Versioning of knowledge assets
Answer: c) management of knowledge asset reference material
Explanation: Management of the knowledge asset reference material is an added benefit to document the source, but is not essential as the other methods are which are critical.
Reference: Clinical Decision Support - The Road to Broad Adoption. Edited by Robert A. Greenes, Academic Press ISBN 978-0-12-398476-0
WS05-7: Select the most important role(s) domain experts (or subject matter experts) fulfill within the context of a knowledge management lifecycle?
a) Review quality measures and propose new knowledge assets
b) Create asset specifications based on existing knowledge representation standards
c) Author new knowledge assets using tools available within clinical knowledge management systems
d) Critique asset specifications and share expertise during knowledge engineering sessions
e) Support end-user adoption of new knowledge assets
Answer: d) Critique asset specifications and share expertise during knowledge engineering sessions
Explanation: Domain experts are difficult to find and, once identified, typically have very limited time availability. Considering these restrictions, domain experts are primarily needed during the knowledge engineering phase of the lifecycle, where their expertise is essential to enable proper knowledge acquisition and elucidation. The roles described in the other options can more easily be fulfilled by other professionals.
Reference: Clinical Decision Support - The Road to Broad Adoption. 2nd Edition. Edited by Robert A. Greenes, Academic Press ISBN 978-0-12-398476-0
126 | P a g e
WS05-8: Which of the following is NOT correct when deploying or updating vendor-based knowledge assets in a clinical information system environment?
a) Integration requires workflow adjustments, data mappings, and extensive testing
b) Assets cannot be easily customized to reflect local needs and priorities
c) Deploying updates of vendor-based knowledge assets prevents CDSS malfunctions
d) Local customizations might be lost when subsequent updates are deployed
d) Effective troubleshooting by vendor is associated with security vulnerabilities
Answer: c) Deploying updates of vendor-based knowledge assets prevents CDSS malfunctions.
Explanation: The deployment and maintenance of outside, vendor-based content represents a major threat to CDSS and other malfunctions within an institution as code changes and mappings may not be updated or not updated correctly leading to malfunctions. The deployment of assets created by vendors requires detailed planning, testing, documentation, etc. with dedicated resources.
WS06: CMIO Workshop - Leading Clinical Informatics Initiatives: How to Successfully Tackle Challenges & Build Resilience
--Julie Hollberg, Richard Schreiber
WS06-1: All hospitals with an EHR should follow the principles of EHR governance. A formal, written EHR governance plan is necessary for:
A. Only large university hospitals that have large information technology staffs
B. Hospitals that have a chief medical informatics officer who runs the EHR
C. Hospitals that manage their EHRs, but not those who contract EHR services
D. All hospitals of whatever size, leadership structure, or EHR design
Answer: D. All hospitals of whatever size, leadership structure, or EHR design Explanation: A fundamental learning objective of this workshop is that EHR governance applies to healthcare environments universally.
127 | P a g e
WS06-2: The most critical success factor in EHR governance is:
E. Managing and leading change transformation processes
F. Assuring purchase of the newest technology
G. Establishing best infrastructure prior to and after EHR installation
H. Rewarding staff for meeting or exceeding goals
Answer: A. managing and leading change transformation processes Explanation: The best answer is a) because use of an EHR depends on supporting the workflow requirements of users. The latest and greatest technology (b) is only as good as the ability of personnel to use it. Although solid infrastructure is necessary (c), it is not sufficient. Rewarding staff (d) has merit, but is also not sufficient. WS06-3: Effective governance requires effective communication for EHR optimization and healthcare transformation, as well as implementation. All the following have been shown to be successful forms of communication. The most effective is:
A. Rich channels such as personal interaction and audiovisual aids are better than email alone
B. Encouraging leaders to disseminate information to users by any means
C. Just-in-time training—not too soon, not too late; as well as at-the-elbow support
D. Involvement of clinicians as well as technical experts in creating the communication plans
Answer: D. Involvement of clinicians as well as technical experts in creating the communication plans Explanation: Although most successful programs use all the above, the most effective plan is to involve subject matter experts and the impacted end-users to plan and execute the communication of anticipated changes. Hoping for dissemination of information is not as reliable (a or b). It is critical to loop in clinicians and others who miss meetings. Depending on “top-down” communication is risky. Face-to-face interactive methods are better than email or other indirect means.
128 | P a g e
WS06-4. Lack of available time, lack of political and communication skills, and lack of vision are examples of leadership failures which can damage EHR implementation or migration. How can one avoid such failures?
A. Have a mission statement, a long-term strategic plan, and governance
B. Create a project plan, use Gantt charts to determine time lines, and follow agile processes
C. Hire experienced managers to run the day-to-day operations of the implementation
D. Be sure to hire a chief medical informatics officer to manage the process
Answer: A. Have a mission statement, a long-term strategic plan, and governance Explanation: Answer A clearly addresses the overall structure for a large, long-term effort such as EHR implementation. Project plans (b) are for shorter term, more focused goals. Although having experienced managers is crucial to the success of an implementation, even the best managers cannot overcome leadership failures (answer c). Although having a CMIO is likely to improve implementation, one leader alone is not sufficient to ensure success. EHR implementation requires 100% leadership support. Reference: Lorenzi NM, Riley RT. Managing change: an overview. J Am Med Inform Assoc 2000; 7(2): 116-124. WS06-5. There are many factors which ensure a successful EHR implementation or transformation. Which of the following is most predictive of success?
A. Training for computer-savvy providers or those who have used the EHR before is not necessary
B. Reducing ambulatory productivity permanently will accommodate changed provider workflows
C. Managers supporting implementation should have no other responsibilities at go-live
D. Additional staff is required to execute the new workflows and provide at-the-elbow support.
Answer: C. Managers supporting implementation should have no other responsibilities at go-live Explanation: A best practice for effective go-live support is to ensure that key managers and superusers are able to focus on implementation activities during the go-live. Training is essential for all providers, although one-size-fits-all training programs are not optimal. Residents must also receive training, especially if working across multiple care settings in which other EHR systems are used. Typically, with effective training, additional staff is not required for new workflows; it is when users try to preserve old workflows in parallel with new workflows that there may be increased staffing requests.
129 | P a g e
WS06-6. A recent study showed that different vendor databases vary in their categorization of drug-drug interactions (DDIs). Different EHR vendor and home-grown products are configured differently, and different sites differ as to whether they modify the databases to alter the number of alerts displayed. Other studies show that despite these differences it is possible to decrease the burden of DDI alerting. This data suggests:
A. It is too complicated and fraught with potential error to modify the vendor product’s data base
B. It is best practice to accept the vendor product’s DDI database and alerting configuration
C. Meaningful use, which requires DDI alerting, has set a new standard of care for such alerts
D. It is time for a revamping of the international standards of care for DDI alert strategy Answer: D. It is time for a revamping of the international standards of care for DDI alert strategy. Explanation: Altering the DDI data base is not complicated as it is a function of changing one field in most DDI tables. If informaticians, pharmacists, and technical experts work together, it is possible to minimize errors. Best practice has yet to be determined: the most salient point in the researchers’ results is that enormous variation exists between institutions, including those which use the same vendor product. Meaningful use does require DDI alerting, but this alone does not represent a standard of care. The authors do advocate for “Developing a standard that covers which DDIs to alert on and how to display those alerts [which] will likely require the collaboration of clinical informaticians, pharmacists, and physicians” and “recommend creating an officially approved, standardized DDI knowledge base, and considering a possible safe harbor or other legal protection for sites that implement that KB.” References: McEvoy DS, Sittig DF, Hickman T-T, Aaron K, Ai A, Amato M, Bauer DW, Fraser GM, Harper J, Kennemer A, Krall MA, Lehmann CU, Malhotra S, Murphy DR, O’Kelley B, Samal L, Schreiber R, Singh H, Thomas EJ, Vartian CV, Westmorland J, McCoy AB, Wright A. Variation in High Priority Drug-Drug Interaction Alerts Across Institutions and Electronic Health Records. J Am Med Inform Assoc. 2017;24(2):331-338. DOI: http://dx.doi.org/10.1093/jamia/ocw114 First published online: 28 August 2016. PMID: 27570216. Schreiber R, Shaha J, Gregoire J, Shaha S. Think Time: A Novel Approach to Analysis of Clinicians' Behavior After Reduction of Pharmacologic Interaction Alerts. Int J Med Inform. 2017;97:59-67. Published online September 30, 2016. http://dx.doi.org/10.1016/j.ijmedinf.2016.09.011 PMID: 27919396. WS06-7. The CIO calls you into her office. She tells you the hospital is going to select a new EHR (Electronic Health Record), implement it, and then optimize it. All is to be done within the span of 24
130 | P a g e
months. She wants to hire a Chief Medical Informatics officer (CMIO). You advise her that the best candidate is:
A. Physician with extensive EHR experience but not necessarily any informatics education
B. Clinical Informatician educated in and experienced with Informatics
C. Physician with extensive Informatics education but limited EHR experience
D. Only physicians, and absolutely no PhDs or Masters
Answer: B. Clinical Informatician educated in and experienced with Informatics
Explanation: The correct answer is B. A and C are wrong as a CMIO needs both education and experience to succeed. D is wrong because PhDs with clinical exposure or experience have and can be CMIOs. Only answer B is correct as it includes both education and experience and all types of candidates. WS06-8. Successfully passing the ABPM board exam in applied clinical informatics requires knowledge in a diverse set of disciplines. These include:
A. Workflow analysis, health information technology, and clinical care B. Clinical care, workflow analysis, and information system lifecycles
C. Evidence basis for clinical care, change management, and decision science
D. Health systematics, clinical decision support, and computer programming
Answer: C. Evidence basis for clinical care, change management, and decision science
Explanation: While those who achieve board certification should have clinical care knowledge, it is not a core content area covered on the boards (answers a and b are wrong). Awareness of the evidence basis, including guidelines of care, is in the core curriculum as are change management and decision science. Computer programming is neither a prerequisite for informatics, nor part of the core curriculum (answer d is wrong).
Reference: Gardner R, Overhage JM, Steen EB, Munger BS, Holmes JH, Williamson J, Detmer DE. Core Content for the Subspecialty of Clinical Informatics. J Am Med Inform Assoc. 2009;16:153–157. DOI 10.1197/jamia.M3045.
131 | P a g e
WS06-9. Yogi Berra once said, “When you come to a fork in the road, take it.” Many of us change our career trajectories, and this can be a difficult transition. Some are better at these transitions because:
A. Self-reliant physicians manage change well
B. Career and life coaches can benefit physicians
C. Physicians often change positions without difficulty
D. New positions are a welcome and exciting challenge
Answer B. Career and life coaches can benefit physicians
Explanation: Although many physicians prefer to manage their own affairs they frequently stumble on non-medical decisions (answer a is not always true). Although it is true that physicians change positions frequently, especially in academic settings, not all transitions are successful (answer c is false). Answer d can be true, but physicians need to be aware of the imposter syndrome (“was I really cut out for this job?”), upheavals to one’s personal and family life, and the not uncommon situation that a change was not a willing transition, all of which exact emotional and professional stress. Answer b is the most correct answer—physicians should be willing to ask for and seek help from family, friends, colleagues, and career change professionals.
Reference: Babbott SF. Career Transitions: Traversing the on and off ramps. American College of Physicians, Internal Medicine Meeting 2018. New Orleans, LA.
WS06-10. Book knowledge provides a solid basis for medical and informatics practice; experience provides real-world application of that knowledge. Since young informaticians lack that experience, the single best strategy to increase one’s knowledge is to:
A. Work long hours in the clinic and/or hospital as well as in informatics to augment their learning
B. Seek out a mentor to help with challenging situations and to help guide your career
C. Treat a transition into a new role as a temporary condition as things will work themselves out
D. Avoid revealing one’s difficulties to colleagues as this may jeopardize your job performance
Answer B. Seek out a mentor to help with challenging situations and to help guide your career
Physicians are strongly independent, but asking for help is not only ok, it helps guide us and improve our care as physicians and in the practice of informatics. There may be nothing wrong with putting in long hours (although it may have family and social costs), but it is not a substitute for wise guidance. A transition is not a temporary condition; it is on-going, and needs active intervention to succeed. Avoidance is a common response to difficulties but is rarely successful.
132 | P a g e
References:
Babbott SF. Career Transitions: Traversing the on and off ramps. American College of Physicians, Internal Medicine Meeting 2018. New Orleans, LA.
Gawande A. Personal best. Top athletes and singers have coaches. Should you? The New Yorker. 2011 (Oct 3).
WS06-11. Which of the following is not a key driver or domain of burnout and professional fulfillment?
A. Efficiency of practice
B. Practice productivity
C. A culture of wellness
D. Personal resilience
Answer: B. Practice productivity
Explanation: Professional fulfillment and burnout need to be assessed together as the absence of burnout doesn’t automatically lead to physician fulfillment. While personal resilience is important, it must be connected to larger system focus on institutional and community support for physicians.
Reference: Bryan Bohman, MD, Liselotte Dyrbye, MD, MHPE, Christine A. Sinsky, MD, Mark Linzer, MD, FACP, Kristine Olson, MD, MSc, Stewart Babbott, MD, Mary Lou Murphy, MS, Patty Purpur deVries, MS, Maryam S. Hamidi, PhD & Mickey Trockel, MD, PhD. Physician Well-Being: The Reciprocity of Practice Efficiency, Culture of Wellness, and Person Resilience. https://catalyst.nejm.org/physician-well-being-efficiency-wellness-resilience/
WS06-12. Tuckman’s model of team work posits that groups will develop in a predictable fashion. The usual order of that development is:
A. Norming, storming, forming, performing
B. Storming, forming, norming, performing
C. Forming, performing, storming, norming
D. Forming, storming, norming, performing
Answer D. Forming, storming, norming, performing
Explanation: Tuckman states that groups pull together (form), followed by a period of dispute (storm). Next the group works out the disagreements (normalizes), before being able to perform.
133 | P a g e
Reference: Tuckman’s stages of group development. Found at: https://en.wikipedia.org/wiki/Tuckman%27s_stages_of_group_development Accessed 26 April 2018.
WS06-13. The best statement regarding IT governance is:
A. A single structure for good IT governance can keep pace with all the “wants” and “needs” of an organization
B. IT governance does not need to include research and educational needs in its structure—separate governance provides adequate oversight
C. How an institution makes IT investment decisions is often more important that what specific decisions are made
D. Proper leadership governance in IT is sufficient to overcome the lack of a clinical information systems advisory group
Answer: C. How an institution makes IT investment decisions is often more important that what specific decisions are made
Explanation: No single governance can provide proper structure for disparate functions and departments (answer a is incorrect). That said, research and education has information technology requirements, and IT governance must include those fields, including data governance (answer b is incorrect). Answer d presupposes that IT personnel can function as clinical decision makers; research shows that this often fails. Answer c is the best answer as it is the most flexible, allowing for the rapid change in technology.
References:
Leviss J (ed). HIT or Miss. Chicago: AHIMA press. 2013. See especially chapters 10, 12, and 19.
Shortliffe EH, Cimino JJ. Biomedical informatics. London: Springer-Verlag. Page 459.
WS06-14. When facing conflict over very important issues, and there are no time pressures for resolution, you should first use which of the following leadership/management methods:
A. Accommodation
B. Compromise
C. Collaboration
D. Competition
Answer: C. Collaboration
134 | P a g e
Explanation: There are a few key variables that define conflict management situations and determine which conflict management strategies are likely to be effective. Time pressure is an important variable - if there were never any time pressures, one approach might always be the best to use. When time is not a variable influencing decisions, and the issues are important, collaboration is the best method of conflict management to employ.
Reference: AMIA Clinical Informatics Board Review Course (CIBRC). 2014.
WS06-15. Alert fatigue reduces efficiency of use of electronic health records. Many clinicians gloss over alerts, often without reading them. Research reveals various ranges of override rates, but the most widely quoted and likely most accurate overall rate is:
A. 60%
B. 75%
C. 96%
D. 99%
Answer: C. 96%
Explanation: The most oft-quoted range comes from Van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006 (Mar-Apr);13(2):138-147. It has been widely replicated in numerous subsequent studies as recently as this year. The range has been broad (46 – 98%). 99% is a slight exaggeration.
WS06-16. Which of the following leadership qualities would be most important for a chief medical informatics officer to display when implementing new systems and applications?
A. Charismatic
B. Task-oriented
C. Transformational
D. Supportive
Answer: C. Transformational
135 | P a g e
Explanation: Leadership qualities have the appearance of being related to traits or behaviors, but the key difference is the interpersonal application as leaders interact with various constituencies of the organization. Of the responses, only charismatic and transformational are leadership qualities. Of those two, transformational is the most important for the change management required when implementing new Health IT systems.
Reference: AMIA Clinical Informatics Board Review Course (CIBRC). 2014.
WS07: Get Your Hands on FHIR®
--Laura Heermann, Russ Leftwich
WS07-1. What does the acronym FHIR® stand for?
a. Federal Health Information Record
b. Fast Healthcare Interoperability Resources
c. Fire Hot Information Response
d. Fast Health Information Record
Answer: b. Fast Healthcare Interoperability Resources
Reference: http://hl7.org/fhir/summary.html
WS07-2. FHIR® is designed to
a. Replace enterprise healthcare records
b. Be used exclusively for personal health records
c. Enable the exchange of healthcare-related information
d. Burn up all other attempts at interoperability
Answer: c. Enable the exchange of healthcare-related information
Explanation: FHIR® is designed to enable information exchange to support the provision of healthcare in a wide variety of settings.
Reference: http://hl7.org/fhir/overview-dev.html
136 | P a g e
WS07-3. Why is FHIR® important to clinicians?
a. It leverages the latest web standards and applies a tight focus on interoperability across care settings and disciplines.
b. It is a standard specification specific to the needs of nursing interoperability.
c. FHIR® has specific resources that apply only to hospital care.
d. FHIR®has no implications for clinical interoperability
Answer: a. It leverages the latest web standards and applies a tight focus on interoperability across care settings and disciplines.
Reference: http://hl7.org/fhir/summary.html
WS07-4. How is the Fast Healthcare Interoperability Resources specification accessed?
a. http://hl7.org/fhir
b. www.fhir.com
c. www.FIRE.org
d. http://fasthealthcareinteroperability.org/
Answer: a. http://hl7.org/fhir
WS07-5. What is a FHIR® Resource?
a. A system of interoperability gadgets that hook together
b. A logically discrete data concept that has a defined meaning, a known identity and location, and has meaning in healthcare.
c. Short snippets of reusable clinical themes used to comprise a medical record.
d. Small increments of reusable code.
Answer: b. A logically discrete data concept that has a defined meaning, a known identity and location, and has meaning in healthcare.
137 | P a g e
Explanation. Although option D is close to correct, it is not complete.
Reference: http://hl7.org/fhir/summary.html
WS07-6. What is an extension in FHIR®?
a. A way to limit the use of a specific resource
b. An adornment to a resource to make it more interesting.
c. A reference to an additional resource that makes it more meaningful.
d. A method to implement additional valid requirements not included in the core specification.
Answer: d. A method to implement additional valid requirements not included in the core specification.
Reference: http://hl7.org/fhir/extensibility.html
WS07-7. What does it mean to constrain a FHIR® resource?
a. To limit a resource to a specific definition fitting the intent of the implementation (such as limiting a pick list with the colors red, green, yellow, blue, orange and purple to red, yellow and blue only)
b. To allow for additional items to be added to a resource
c. To compel a user to document using FHIR resources only.
d. To discourage use of specific parts of a FHIR resource but allow them if absolutely necessary.
Answer: a. To limit a resource to a specific definition fitting the intent of the implementation (such as limiting a pick list with the colors red, green, yellow, blue, orange and purple to red, yellow and blue only)
Reference: http://hl7.org/fhir/profiling.html#resources
138 | P a g e
WS07-8. What is a FHIR® Profile?
a. A sketch of a drafted FHIR® resource prior to its being released for use.
b. A specification of a FHIR® resource or resources detailing constraints and extensions necessary for use in a particular use case.
c. A fully instantiated resource
d. A FHIR® resource in action sharing patient data from one clinical site to another.
Answer: b. A specification of a FHIR® resource or resources detailing constraints and extensions necessary for use in a particular use case.
Reference: http://hl7.org/fhir/profilelist.html
WS07-9. Name 3 FHIR® Architectural Principles.
a. Fast, healthcare specific, reusable
b. Fast, easy to learn, scalable
c. Reusable, Scalable, Readily Implementable
d. Implementable, clinically sufficient, highly structured
Answer: c. Reusable, Scalable, Readily Implementable
Reference: http://hl7.org/fhir/overview-arch.html#framework
WS07-10. FHIR® Resources are organized into 5 framework layers. These include:
a. Foundation, Base, Clinical, Financial and Specialized
b. A, B, C, D, E layers
c. Problems, Meds, Allergies, Orders and Labs
d. Structural, Clinical, Observations, Provider, Payer
Answer: a. Foundation, Base, Clinical, Financial and Specialized
Reference: http://hl7.org/fhir/overview-arch.html#framework
139 | P a g e
WS07-11. Resources classified as “workflow” are meant to:
a. Support the healthcare process
b. Conform to a specific institution’s clinical workflow.
c. Be implemented using BPMN
d. Identify the users of a specific workflow
Answer: a. Support the healthcare process
Reference: http://hl7.org/fhir/resourceguide.html
WS07-12. What is a FHIR® Implementation Guide?
a. A listing of FHIR® resources
b. A how-to guide on profiling FHIR® resources
c. A knowledgeable person available for new implementers for assistance as needed.
d. A publication outlining capabilities as defined by national standards, vendor consortiums, clinical societies, etc., regarding how FHIR® specifications are used to solve particular problems.
Answer: d. A publication outlining capabilities as defined by national standards, vendor consortiums, clinical societies, etc., regarding how FHIR® specifications are used to solve particular problems.
Reference: http://www.fhir.org/guides/registry
WS07-13. The Observation Resource should be used for what type of data?
a. Home meds, orders, MAR
b. Vital Signs, physical exam finding, pulmonary artery pressure reading, lab results
c. CT Scan, MRI, Xrays
d. Problem List, Chief Complaint, Past Medical History
Answer: b. Vital Signs, physical exam finding, pulmonary artery pressure reading, lab results
Reference: http://hl7.org/fhir/resourceguide.html#clinical
140 | P a g e
WS07-14. The Condition Resource should be used for what type of data?
a. CT Scan, MRI, Xrays
b. Vital Signs, Physical Exam Findings, Pulmonary Artery Catheter reading
c. Diagnosis, Chief Complaint, Past Medical History
d. Home meds, orders MAR
Answer: c. Diagnosis, Chief Complaint, Past Medical History
Reference: http://hl7.org/fhir/resourceguide.html#clinical
WS07-15. The clinFHIR software tool requires what configuration parameters?
a. A use case type aligned with the categories of FHIR® resources
b. Selection of a set of capability statements from a library
c. Selection of LOINC and SNOMED value sets from the NIH Value Set Access Center
d. Setting compatible data, conformance, and terminology servers
Answer: d. Setting compatible data, conformance, and terminology servers
Reference: www.clinfhir.com
WS07-16. How do I provide feedback to the FHIR development team regarding FHIR resources?
a. Send them an email to [email protected]
b. Call HL7 Headquarters
c. Submit them through the gForge tool
d. Don’t bother – the specification is done and published.
Answer: C. Submit them through the gForge tool
Reference: http://gforge.hl7.org/gf/
141 | P a g e
WS07-17. What is a FHIR® Maturity Level?
a. A method for the FHR specification developers to inform implementers how advanced and stable a FHIR Resource is.
b. How long the FHIR® resource has been in existence
c. How many times the FHIR® resource has been implemented
Answer: a. A method for the FHR specification developers to inform implementers how advanced and stable a FHIR Resource is.
Reference: http://hl7.org/fhir/versions.html#maturity
WS07-18. What is the interoperability paradigm in which FHIR® is operable?
a. The messaging paradigm, like HL7 version 2.
b. The document paradigm, like HL7 Clinical Document Architecture (CDA).
c. As a RESTful API.
d. FHIR® can be used in all of the above interoperability paradigms.
Answer: d. FHIR® can be used in all of the above interoperability paradigms.
References:
https://www.hl7.org/fhir/messaging.html
https://www.hl7.org/fhir/documents.html
WS07-19. The Argonaut project is what type of initiative?
a. An initiative to develop FHIR® certification testing led by the federal government.
b. An initiative initially started to accelerate development of FHIR®profiles for the Common Clinical Data Set sponsored by a vendor collaborative.
c. An initiative to develop FHIR® translations led by the HL7 Greece affiliate.
d. An initiative that is part of SMART on FHIR®.
142 | P a g e
Answer: b. An initiative initially started to accelerate development of FHIR profiles for the Common Clinical Data Set sponsored by a vendor collaborative.
Reference: http://argonautwiki.hl7.org/index.php?title=Main_Page
WS07-20. What is SMART on FHIR®?
a. An ONC project to create FHIR® profiles for CDS.
b. A platform architecture for portable healthcare apps utilizing FHIR® as an API for health data.
c. The organization which has published FHIR® profiles for the Common Clinical Data Set.
d. The organization which first developed the concept of FHIR®.
Answer: b. A platform architecture for portable healthcare apps utilizing FHIR® as an API for health data.
Reference: https://smarthealthit.org
WS07-21. Which of the following HL7 standards include specifications for an application programming interface (API)?
a. HL7 Version 2.x
b. HL7 Version 3
c. HL7 CDA
d. HL7 FHIR®
Answer: d. HL7 FHIR®
References:
http://www.hl7.org/implement/standards/index.cfm?ref=nav
http://www.hl7.org/implement/standards/product_matrix.cfm?ref=nav
http://www.hl7.org/implement/standards/product_brief.cfm?product_id=449
143 | P a g e
WS07-22. FHIR resources can be linked to each other via what type of element in a resource definition?
a. Code
b. Codable Concept
c. Reference
d. Identifier
Answer: c. Reference
Reference: http://hl7.org/fhir/references.html
WS07-23. Which of the following represent types of terminologies that can be linked to a FHIR® Profile?
a. SNOMED and LOINC
b. SnowOwl and RF2
c. C4NNol1 and NaPoLeon
d. Java and Python
Answer: a. SNOMED and LOINC
Reference: http://hl7.org/fhir/terminologies-systems.html
144 | P a g e
WS08: How to Effectively Negotiate the Outcomes You Really Desire?
--Shakaib Rehman, Hamed Abbaszadegan
WS08-1: According to the Thomas-Kilmann Model, if your concern about your own outcome is low and your concern for the other party's outcomes is high; your approach would be:
A. Accommodating
B. Avoiding
C. Compromising
D. Problem-solving
Answer: A. Accommodating
Explanation: According to the Thomas-Kilmann Model if your concern about your own outcome is low and your concern for the other party's outcomes is high, your approach would be considered accommodating.
References: 1. Negotiation and Mediation Spiral-bound – 1991 by Peter Pruitt
2. http://www.kilmanndiagnostics.com/catalog/thomas-kilmann-instrument-one-assessment-person
WS08-2: Psychologist Stephen Covey’s research has shown that a certain percentage of life is under your control and is decided by how you react while the other portion of life is called destiny (no matter what do you do; it will happen because it is inevitable). Which of the following option is correct about his determination of life events decided by you over what happened to you?
A. 60/40
B. 70/30
C. 80/20
D. 90/10
Answer: D. 90/10
Explanation: According to Psychologist Stephen Covey, 90% of life is decided by how you react versus 10% of life is made of what happened to you. it is called 90/10 rule.
Reference: Covey, Stephen R. The 7 Habits of Highly Effective People: Restoring the Character Ethic. New York: Free Press, 2004.
145 | P a g e
WS08-3: Psychologist Abraham Maslow has described the “Human Hierarchy of Needs.” What is the highest need of individuals, according to his model?
A. Esteem
B. Love
C. Security
D. Self-Actualization
Answer: D. Self-Actualization
Explanation: According to psychologist Abraham Maslow's Hierarchy of Needs; Self-Actualization is the highest level of human motivation which cannot be addressed until more fundamental needs are addressed.
Reference: https://www.reference.com/world-view/maslow-s-hierarchy-needs-96f513d6ddfedf6d
WS08-4: What one of the following best describe “Principled Negotiations”?
A. Focus on interests not positions
B. Focus on people not problems
C. Identify subjective criteria to determine success
D. Identify Worst Alternative to a Negotiated Agreement (WATNA)
Answer: A. Focus on interests not positions
Explanation: Five fundamentals of negotiation: 1. Separate people from the problem. 2. Focus on interests, not positions. 3. Invent opportunities together for mutual gain. 4. Identify objective criteria to determine success. 5. Know your BATNA (Best Alternative to a Negotiated Agreement), and calculate the other party's BATNA.
Reference: Fisher, R., Ury, W., & Patton, B. (1991). Getting to yes: Negotiating agreement without giving in. New York, N.Y: Penguin Books.
WS08-5: Which one of the following is not part of the “Principled Negotiation” model?
A. Separate people from problem
B. Invent options for mutual gain
C. Insist on using subjective criteria
D. Focus on interests, not positions
146 | P a g e
Answer: C. Insist on using subjective criteria
Explanation: The model calls for insisting on using objective criteria.
Reference: Fisher, R., Ury, W., & Patton, B. (1991). Getting to yes: Negotiating agreement without giving in. New York, N.Y: Penguin Books.
WS08-6: The “Principled negotiation” model emphasizes the importance of the “BATNA.” What does BATNA stand for?
A. Best Action to a Negotiated Agreement
B. Best Alternative to a Negotiated Agreement
C. Best Alternative to a Nonsense Agreement
D. Best Action to a Nonsense Alternative
Answer: B. Best Alternative to a Negotiated Agreement
Reference: Fisher, R., Ury, W., & Patton, B. (1991). Getting to yes: Negotiating agreement without giving in. New York, N.Y: Penguin Books.
WS08-7: Complete the list of Thomas-Kilmann's five conflict management styles: Competing, accommodating, avoiding, ___ and ___.
A. collaborating, and compromising.
B. colluding, and connecting.
C. communicating, and compromising.
D. cooperating, and connecting
Answer: A. collaborating, and compromising.
Explanation: The five conflict management style orientations are competing, accommodating, avoiding, collaborating, and compromising.
Reference: http://www.kilmanndiagnostics.com/catalog/thomas-kilmann-instrument-one-assessment-person
147 | P a g e
WS08-8: If a person typically tends to delegate controversial decisions and accept others’ decisions, which of the following styles best describes this behavior?
A. Accommodating
B. Avoiding
C. Collaborative
D. Competitive
Answer: B. Avoiding
Explanation: People who tend to use an avoiding style "seek to evade the conflict entirely." They avoid controversial decisions. This style can be appropriate when "victory is impossible, when the controversy is trivial, or when someone else is in a better position to solve the problem."
Reference: http://www.kilmanndiagnostics.com/catalog/thomas-kilmann-instrument-one-assessment-person
WS09: Designing Outpatient Clinical Decision Systems for High Use Rates, Provider Satisfaction, and Improved Care: Lessons from Meta-Analyses and Experience
--Patrick O'Connor, JoAnn Sperl-Hillen
WS09-1: Outpatient Clinical Decision Support (CDS) has been shown to be least effective at improving which of the following clinical measures:
A. Adult Immunization rates
B. Blood pressure control in hypertensives
C. Diabetes eye exam rates
D. Lipid test rates
E. Screening Mammography rates
Answer: B. Blood pressure control in hypertensives Explanation: Most meta-analyses of outpatient CDS indicate a positive impact on testing rates, screening rates, and immunization rates, but much less impact on actual levels of BP, glucose, or lipid control. Reference: Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29-43.
148 | P a g e
WS09-2: Which of the following statements is most true of the impact and cost of Clinical Decision Support and Nurse Case Management to improve the proportion of those with diabetes who reach recommended treatment goals:
A. Clinical Decision Support is as effective as Nurse Case Management
B. Clinical Decision Support and Nurse Case Management can reach similar proportions of adults with uncontrolled diabetes
C. On a population basis, the cost effectiveness of Clinical Decision Support and Nurse Case Management for diabetes care are similar
D. Nurse Case Management can reduce overall health care costs
E. Clinical Decision Support for diabetes care is cost saving to payers
Answer: C. On a population basis, the cost effectiveness of Clinical Decision Support and Nurse Case Management for diabetes care are similar Explanation: Compared to outpatient diabetes CDS, Nurse Case Management (a) reaches a smaller proportion of diabetes patients because many decline to engage with the case manager, (b) costs more per patient than outpatient CDS, and (c) has a bigger impact on measures such as BP level and glycated hemoglobin (A1c) level. Reference: Gilmer TP, O'Connor PJ, Sperl-Hillen JM, et al. Cost-effectiveness of an electronic medical record based clinical decision support system. Health services research. 2012;47(6):2137-2158. WS09-3: Consider a CDS system that extracts clinical information from the EHR, sends it to a web service where it is processed using clinical algorithms, and then transmits CDS back to the clinician and the patient at the point of care. Barriers to broad dissemination this type of CDS system include all of the following except:
A. Getting physician leaders from many medical groups to agree on clinical goals for BP, A1c, lipid management, and aspirin use in various groups of patients.
B. Concerns about data security when transmitting personal health information using the cloud
C. The potential of EHR-derived clinical information to be inaccurate or incomplete
D. Desire to provide evidence-based care personalized to each patient
E. Low use of the CDS system by primary care providers
Answer: D. Desire to provide evidence-based care personalized to each patient Explanation: Most primary care clinicians are trained to personalize care to each patient based on intuition and clinical judgement. Most providers prefer that treatments be evidence-based.
149 | P a g e
Reference: Vermunt NP, Harmsen M, Elwyn G, Westert GP, Burgers JS, Olde Rikkert MG, Faber MJ. A three-goal model for patients with multimorbidity: A qualitative approach. Health Expect. 2018;21(2):528-538. WS09-4: In a classic paper by McGlynn et al published in 2003, the number of evidence-based actions that apply to adults in primary care settings is approximately:
A. 60
B. 300
C. 600
D. 3000
E. 6000
Answer: C. 600 Explanation: McGlynn, et al, measured delivery of 600 evidence-based clinical services to a sample of adult patients, and reported that only 55% were regularly delivered to the patients that they studied. One such patient needed over 100 such services. These data suggest the need to prioritize evidence-based clinical services based on their potential benefit to a given patient. Reference: McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. The New England journal of medicine. 2003;348(26):2635-2645. WS09-5: In a time-motion study of primary care physicians using a popular electronic health record system, the number of clicks and minutes required to get 80% of relevant data on cardiovascular risk to assess control of BP, lipids, glucose, smoking, aspirin use, and weight/BMI during an encounter was:
A. 4 clicks and 20 seconds
B. 12 clicks and 50 seconds
C. 30 clicks and 2 minutes
D. 50 clicks and 4 minutes
E. 70 clicks and 6 minutes
Answer: D. 50 clicks and 4 minutes Explanation: It took clinicians 53 clicks and 4 minutes to get 80% of the information needed to assess control of BP, lipids, glucose, smoking, aspirin use, and weight/BMI in adults when that information was desired. Only about 80% of what was found was accurate and up to date. This suggests the opportunity to have algorithms retrieve these data and present them in an organized format, in order to save clinician time during visits, and likely increase the accuracy of the elicited information.
150 | P a g e
Reference: Koopman RJ, Kochendorfer KM, Moore JL, Mehr DR, Wakefield DS, Yadamsuren B, Coberly JS, Kruse RL, Wakefield BJ, Belden JL. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. Annals of family medicine. 2011;9(5):398-405. WS09-6: The approximate annual payment from a medical group to an EHR vendor for use and maintenance of an EHR software system is approximately:
A. $500 per clinician per year
B. $1,000 per clinician per year
C. $2,000 per clinician per year
D. $5,000 per clinician per year
E. $50,000 per clinician per year
Answer: D. $5,000 per clinician per year Explanation: The actual charges vary from one medical group to another, and are often confidential per contractual agreements. However, many experts estimate that the costs are well in excess of $10,000 per physician per year, and may be up to $20,000 per physician per year in some cases. These figures are useful to contextualize the added cost and potential benefits of add-on clinical decision support systems to improve care. Reference: Adler-Milstein J, Green CE, Bates DW. A survey analysis suggests that electronic health records will yield revenue gains for some practices and losses for many. Health Aff (Millwood). 2013;32(3):562-570. WS09-7: Advantages of web-based, rather than within-EMR clinical decision support systems include all the following except:
A. Many medical groups can use a single CDS installation
B. Updating CDS algorithms when guidelines change is cheaper in a web-based service
C. Updating CDS algorithms when guidelines change is less prone to error if only one installation requires updating
D. Data security is improved when CDS is web-based
E. The cost of shared CDS installations would likely be lower than the aggregate cost of each medial group doing their own CDS installation
Answer: D. Data security is improved when CDS is web-based
151 | P a g e
Explanation: Medical groups have varying programmer capacity and expertise, which is needed to maintain and update CDS algorithms. Maintaining a large CDS installation that can serve many medical groups lowers costs, reduces programming errors, and enables more rapid dissemination of new guidelines. However, transmission of data form the EMR to the web service and back requires additional measures to assure data security. Reference: Schmittdiel JA, Adams SR, Segal J, et al. Novel use and utility of integrated electronic health records to assess rates of prediabetes recognition and treatment: brief report from an integrated electronic health records pilot study. Diabetes care. 2014;37(2):565-568. WS09-8: In many cases, there are conflicting clinical guideline recommendations from different sources. For example, the 2018 American Diabetes Association recommends a BP goal for adults with diabetes of < 140/90 mm Hg, while the 2017 AHA/ACC hypertension guideline recommends a BP goal of < 130/80 mm Hg for adults with diabetes.The best way to handle this type of situation in primary care CDS systems is to:
A. Provide detailed information to clinicians at each encounter, explaining the different recommendations
B. Insert “hard stops” to assure that clinicians must pay attention to BP goals at every patient encounter
C. Provide no CDS on BP management until expert panels resolve their differences
D. Frame treatment recommendations using words like “consider” rather than words like “should,” to accommodate differences in guidelines
E. Give each hypertensive patient a written sheet explaining the situation
Answer: D. Frame treatment recommendations using words like “consider” rather than words like “should,” to accommodate differences in guidelines Explanation: In situations where legitimate and credible guidelines present conflicting recommendations, it is advisable to soften the language and encourage clinicians to individualize care plans to the needs and preferences of each patient. Reference: Ismail-Beigi F, Moghissi E, Tiktin M, Hirsch IB, Inzucchi SE, Genuth S. Individualizing glycemic targets in type 2 diabetes mellitus: implications of recent clinical trials. Annals of Internal Medicine. 2011;154(8):554-559.
152 | P a g e
Wednesday, May 9th
S01: Presentations - Integrating Quality and Workflow
Quality in Action, Adapting Quality Measures for Realtime Guidance
--Andrew Simms
S01-1: When are providers required to report electronic Clinical Quality Measures data to CMS?
A. 2017
B. 2018
C. 2019
D. 2020
Answer: B. 2018 Explanation: The Centers for Medicare and Medicaid Services have mandated this for 2018. Reference: Clinical quality measures basics [Internet].; 2018 [updated Feb 06,; cited Mar 26, 2018]. Available from: https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/ClinicalQualityMeasures.html. S01-2: The Merit-Based Incentive Payment System (MIPS) uses a score system for calculating performance. What percentage of this score is based on quality improvement activities?
A. 10%
B. 15%
C. 25%
D. 60%
Answer: D. 60% Explanation: Although measures vary by specialty, quality improvement is the category with the highest contribution to total score. Reference: Resources by provider type - centers for medicare & medicaid services. Retrieved Mar 26, 2018, from https://www.cms.gov/Medicare/Quality-Payment-Program/Resource-Library/Resources-by-provider-type.html
153 | P a g e
From Pathway to Bedside: Integrating Clinical Informatics in Quality Improvement.
--Eric Shelov
S01-3: The most valuable asset of a Clinical Informatician in a Quality Improvement project centered on Clinical Decision Support is their knowledge of:
A. Data Standards
B. Clinical & IS workflows
C. EHR functionality
D. Database structure
Answer: B. Clinical & IS workflows
Explanation: All of the above are important for a well-rounded Clinical Informatician. The asset of greatest value, however, in the potential success and minimizing unintended consequences of a QI project is their knowledge of workflows. Even if they are intimately familiar with the exact clinical workflow at hand, even a somewhat generic clinical workflow knowledge coupled with an understanding of technical workflows are a critical contribution to a successful implementation.
Reference: Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., et al. (2003). Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. Journal of the American Medical Informatics Association : JAMIA, 10(6), 523–530. http://doi.org/10.1197/jamia.M1370
S01-4: The most fundamental strength of a clinical pathway is its ability to help ________ clinical care and best practices.
A. Improve
B. Disseminate
C. Standardize
D. Accelerate
Answer: C. Standardize
Explanation: All of these are potentially a strength of a clinical pathway, but A, B and D are all dependent on C, the standardization of care that a clinical pathway affords.
154 | P a g e
Reference: Osheroff, J. A., Teich, J., Society, H. I. A. M. S., Levick, D., Saldana, L., Velasco, F., et al. (2012). Improving Outcomes with Clinical Decision Support: An Implementer's Guide. HIMSS.
How am I doing? Online, interactive quality measures for clinicians
--Mark Weiner
S01-5: Patients may see different providers over time, so attributing the quality of a patient’s care to a single provider can be challenging. Which of the following is the most sensible way to ensure that the association between patient and provider is appropriate for attribution of a quality measure?
a. Recent Provider
b. Listed Provider in EHR
c. Listed Provider on the Insurance Card
d. Most frequent Provider
Answer: d. Most frequent Provider
Explanation: The listed provider in the EHR and the Listed Provider on the Insurance Card (b and c) are often administratively the most straightforward, and indeed, insurance carriers will often attribute quality to whomever they think is taking care of the patient. However, in many cases these listed providers are not the actual person caring for the patient. The Recent Provider (a) is a tempting choice since that person may have taken over the care of the patient from providers who have seen the patient in the past. However, the recent provider may not have had the duration of exposure to the patient to impact the patient’s care. The most frequent provider (d) is the person the patient has seen most often in a time period and therefore is likely to have the greatest impact on the quality of the patient’s care.
S01-6: Providers who receive report cards on their quality of care express many concerns about the validity of the measures. Which of the following statements about quality measures is true?
a. If a quality threshold is relaxed, the proportion of a provider’s panel achieving that threshold is likely to increase
b. If a quality threshold is relaxed, a provider’s quality ranking is likely to improve
c. A higher-ranked (“better”) provider will have a greater number of patients meeting a quality threshold then a lower-ranked (“worse”) provider.
d. Stratifying a provider’s panel by the presence of diabetes will decrease the proportion of the cohort receiving a HBA1c test compares with the overall cohort.
Answer: a. If a quality threshold is relaxed, the proportion of a provider’s panel achieving that threshold is likely to increase
155 | P a g e
Explanation: If a threshold for quality is relaxed, it becomes easier for all providers to achieve that threshold (such as changing a HBA1c threshold from 7 to 8). Therefore, A is correct because it is likely that the proportion of patients achieving the “easier” threshold will increase. B is incorrect since the proportion of all providers’ panels achieving the threshold will increase, but some providers may have greater increases than others, so rankings will drop for many providers. C is incorrect because the ranking is based on the proportion of patients in a panel achieving a threshold. Therefore, a lower ranked (worse) provider with a larger panel size may have a larger absolute number of patients achieving a threshold than a higher-ranked (better) provider with a smaller panel. D is incorrect since the proportion of patients receiving an HBA1c test will be more diluted in the overall panel, compared with the smaller set of patients with diabetes on whom the HBA1c test is more likely to be ordered.
S02: Presentations –- Standards and Best Practices for EHR Implementations
Come Together: Lessons Learned from a Health System EHR Migration and Proposal for Sharing Implementation Best Practices
--John McGreevey
S02-1: All of the following are limitations to the broad applicability of existing guidance about how to implement an EHR except:
A. Existing EHR implementation guidance is based on valid experience and research and comes from respected health care, academic, and government institutions
B. Work describing an implementation addresses a narrow population (example: small, rural primary care clinics)
C. Some literature is becoming less relevant with time (example: paper to EHR transition)
D. Guidance focuses on a single aspect of an implementation (example: CPOE)
E. Guidance tends to be noncontroversial and described at a high level (example: training is important)
Answer: A. Existing EHR implementation guidance is based on valid experience and research and comes from respected health care, academic, and government institutions
Explanation: Answers B, C, D, and E are all limitations to the broad applicability of existing guidance about how to implement an EHR. Specifically, limitations include narrow populations or settings, decreasing relevance of paper to electronic implementation literature as more systems have established EHRs in place, focus on implementation of a single aspect of an EHR rather an entire EHR implementation, and high level recommendations that may not instruct readers about the specifics of
156 | P a g e
how to execute such recommendations. Despite these limitations, the existing work is based on valid real-world experiences and research by respected individuals and institutions. Thus the correct answer is A.
S02-2: Which of the following combinations correctly represents lessons learned in this health system EHR implementation?
A. Given that vendors have extensive experience with implementation, do not deviate from vendor guidance; develop an infrastructure of point people from various domains to support the implementation work; get an early start with change management; forego developing a strategy related to the EHR implementation as the vendor will guide you to a successful outcome.
B. Begin an implementation with internal alignment of practices, policies, equipment so that there is consistency throughout the organization; develop an infrastructure of point people from various domains to support the implementation work; recognize that sometimes vendor guidance will be appropriate and at other times, an organization may need to follow its own plan; wait to initiate change management efforts until 4-6 weeks before go-live so that the rest of the implementation work can be completed first.
C. Get an early start with change management; begin an implementation with internal alignment of practices, policies, equipment so that these are consistent across the organization; develop an infrastructure of point people from various domains to support the implementation work; maintain humility
D. Begin an implementation with internal alignment of practices, policies, equipment so that there is consistency throughout the organization; “no training, no access, no kidding” was an effective training strategy in this implementation; get an early start with change management; recognize that with a thoughtful strategy for an EHR implementation, anyone who reports problems with the system or who participates in workarounds after go-live is likely doing so because they are resisting necessary organizational change.
Answer: C. Get an early start with change management; begin an implementation with internal alignment of practices, policies, equipment so that these are consistent across the organization; develop an infrastructure of point people from various domains to support the implementation work; maintain humility
Explanation: Answer C contains correct lessons learned as reported in this health system implementation experience. All other answers have at least one incorrect lesson. Incorrect lessons listed include:
1) given that vendors have extensive experience with implementation, do not deviate from vendor guidance
157 | P a g e
2) forego developing a strategy related to the EHR implementation as the vendor will guide you to a successful outcome
3) wait to initiate change management efforts until 4-6 weeks before go-live so that the rest of the implementation work can be completed first
4) recognize that, with a thoughtful strategy for an EHR implementation, anyone who reports problems with the system or who participates in workarounds after go-live is likely doing so because they are resisting necessary organizational change.
Seize the Moment: Promotion of Lean Principles in Moving from Paper to Electronic Provider Documentation at an Academic Medical Center
--Joel Betesh
S02-3: An Advantage of the APSO (Assessment, Plan, Subjective, Objective) note over a SOAP (Subjective, Objective, Assessment and Plan) note in the EHR is that:
a. It is supported by extensive high quality literature
b. It is easier for the reader of the note to find the key elements of decision making of the note author
c. It protects against notes being bloated with info pulled in from elsewhere in the chart
d. APSO has been used more extensively over the last few decades
Answer: b. It is easier for the reader of the note to find the key elements of decision making of the note author
Explanation: Having the assessment and plan the top of the note brings the thought process of the author directly to the reader's attention.This is true in a paper note and even more so in an electronic note which tends to be longer.
For the other incorrect options:
We did a literature search and found little literature on this topic.
SOAP has been the dominant format of doctors notes over the past few decades.
Although we wanted the notes be lean there is nothing about the APSO format that protects it against note bloat.
Reference: Lin CT, McKenzie M, Pell J, Caplan L. Health care provider satisfaction with a new electronic progress note format: SOAP vs APSO format. JAMA Intern Med. 2013 Jan 28;173(2):160-2.
158 | P a g e
S02-4: Moving from paper to electronic format alone is not enough to fully alter ingrained provider practices and perceived documentation needs. Problems with this transition at the health system described include all the choices below except:
a. Providers continue to feel a need to add data to their note for purposes of billing.
b. Manual chart review is still required to assess note quality
c. It is possible for individual providers to deviate significantly from the standard templates offered at a health system level
d. The majority of notes evaluated after the transition from paper to electronic documentation used the APSO format
Answer: d. The majority of notes evaluated after the transition from paper to electronic documentation used the APSO format
Explanation: D is correct because that is the main positive finding of our study (see table on page 2 of our presentation)
A, B and C are all problems we noted after our go-live with provider documentation. See the first 2 paragraphs of our discussion section.
Maintaining EHR Standards in a Complex Deployment
--Keith Woeltje
S02-5: All of these are important to invite to participate in the change control meetings, except:
A. CMOs/CNOs
B. Local leaders from each hospital
C. All subject matter experts (SMEs) for each specialty
D. CMIO
Answer: C. All subject matter experts (SMEs) for each specialty
Explanation: It is important to have input from leaders across the organization to arrive at consensus decisions. While SME input is essential for developing appropriate content, their participation can be ad hoc when their particular specialty is under discussion.
Reference: Project Management Institute. A Guide to the Project Management Body of Knowledge (PMBOK® Guide)–Sixth Edition. Newtown Square, PA: Project Management Institute; 2017.
159 | P a g e
S02-6: Change control is an essential skill for clinical informaticists. Which of the following is TRUE about good change control? Change control…
A. is very technical, and should really only involve trained analysts and informaticians
B. helps prioritize limited resources and ensures system standards are upheld
C. has to be done because it’s part of Meaningful Use
D. is most useful at the start of a project, and is less useful once application teams are more sure about what they are doing.
Answer: B. [Change control] helps prioritize limited resources and ensures system standards are upheld
Explanation: A good change control process does involve technical staff, but also involves other stakeholders. The process ensures that the highest priorities are addressed first, and that changes meet enterprise operational and technical standards. Change control is not part of the “Meaningful Use” standards. Change control processes are essential at all stages of a project, but become even more essential as a project advances to ensure that previous work is not adversely affected inadvertent
Reference: Project Management Institute. A Guide to the Project Management Body of Knowledge (PMBOK® Guide)–Sixth Edition. Newtown Square, PA: Project Management Institute; 2017.
S03: Panel - Clinical Informatics Fellows Project Ignite Talks
--Christopher Hollweg (Presenter) S03-1: In the FHIR (Fast Healthcare Interoperability Resources) specifications, what is the difference between “Resources” and “Extensions”?
A) Resources manage the incoming FHIR queries and Extensions the outgoing queries
B) Resources map the data back to the database while extensions map the data to SMART applications
C) Resources define the standard data types while extensions the additional data types
D) Resources deal with structured data (i.e., ICD) and extensions with unstructured data (i.e., free text)
Answer: C) Resources define the standard data types while extensions the additional data types Explanation: The FHIR specifications have 'Resources,' which are standard data types that cover about 80% of all data. If additional data types are needed, they can be added on as 'Extensions.' Reference: https://www.hl7.org/fhir/resource.html
160 | P a g e
S03-2: Which of the following is not a core principle of information system security theory?
A) Availability
B) Vulnerability
C) Confidentiality
D) Integrity
Answer: B) Vulnerability Reference: United States Code, Title 44, Chapter 35, Subchapter III, § 3542—Definitions. https://www.law.cornell.edu/uscode/text/44/chapter-35/subchapter-III S03-3: A swim lane flowchart differs from a simple flowchart in that it:
A. Focuses on the value stream
B. Is a physical map of movements of people in the workflow
C. Visually represents the actions taken by various roles
D. Maps out the steps in the process
Answer: C. Visually represents the actions taken by various roles Reference: 3 Types of Flowcharts You Can Use for Workflow Management. July 10, 2017. https://gravityflow.io/3-types-flowcharts-can-use-workflow-management/ S03-4: Why is ELK stack a good tool set to manage healthcare data? Because it is…
A) an SQL database management system and can be easily queried
B) a NoSQL database management system and is highly efficient in querying text rich data
C) a graph database and can be easily queried, especially for imaging data
D) not scalable therefore provides security and stability
E) not open source, with proprietary technology that cannot be breached by outside hacker
Answer: B) a NoSQL database management system and is highly efficient in querying text rich data
161 | P a g e
S03-5: Which of following is one of the common issues encountered when clinicians are developing SMART-on-FHIR® CDS with developers?
A. Could be hard to have access to a FHIR® database/server. The database might not have the clinical data of your interest (e.g., lab values of the liver function test.)
B. Mock EHR might not be available even when the team has a FHIR® server - building a barrier to the usability testing.
C. Designing UI/UX requires serious feedback from clinicians
D. Medical calculators could be the serious contenders.
E. All of Above
Answer: E. All of Above
S04: Presentations - Evidence-based Care Improvement
Article Relationship Visualization to Support Evidence-based Practice & Knowledge Discovery
--Po-Yin Yen
S04-1: What contributes to the highest quality of evidence for clinical practice?
A. Rapid review
B. Systematic Review
C. Literature Review
D. Scoping Review
Answer: B. Systematic Review Explanation: Systematic reviews are carefully synthesized research evidence designed to answer focused clinical questions, and provides the best evidence for Evidence-Based Practice (EBP). Reference: Stevens KR. Systematic reviews: the heart of evidence-based practice. AACN Clin Issues. 2001 Nov;12(4):529-38. Melnyk BM. Achieving a high-reliability organization through implementation of the ARCC model for systemwide sustainability of evidence-based practice. Nursing administration quarterly. 2012;36(2):127-135.
162 | P a g e
S04-2: What is the most commonly reported barrier to evidence-based clinical decision-making? Lack of…
A. time to process new information
B. literature with high level of evidence
C. interprofessional communication
D. departmental leadership
Answer: A. time to process new information Explanation: Although it is well-established that EBP improves healthcare quality and patient outcomes and reduces morbidities, mortality, medical errors, costs, and geographic variation of healthcare services, it is not standard practice by the majority of clinicians across the United States. A major barrier to EBP is that EBP involves time-intensive steps that are burdensome and lead to information overload. Primary resources for identifying evidential studies include health science bibliographic databases such as MEDLINE, CINAHL, and PsycINFO. Over 12,000 studies are published weekly, including more than 300 randomized trials. This current volume of evidence is overwhelming and unmanageable to clinicians. References: Melnyk BM, Fineout-Overholt E. Evidence-based practice in nursing & healthcare: A guide to best practice. Lippincott Williams & Wilkins; 2011. Melnyk BM. The Evidence-Based Practice Mentor: A Promising Strategy for Implementing and Sustaining EBP in Healthcare Systems. Worldviews on Evidence-Based Nursing. 2007;4(3):123-125. Beckett M, Quiter E, Ryan G, et al. Bridging the gap between basic science and clinical practice: the role of organizations in addressing clinician barriers. Implementation Science. 2011;6(1):35. Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol. 2012;65(12):1289-1295.
163 | P a g e
Characterizing Cutaneous Lupus Erythematosus with Meta-Analysis Using STARGEO
--Osama El-Sayed
S04-3: A researcher is interested in performing a meta-analysis with data from the gene expression omnibus. What is a potential barrier to using GEO in one's research?
A. There is no standard way of filling out the metadata for the study
B. There is no information included about how the samples were processed
C. GEO does not contain any data on context of the study design
D. GEO is not accessible to the public and requires a subscription to access its data
Answer: A. There is no standard way of filling out the metadata for the study Explanation: While most studies included the necessary information on sample processing as well as a link to the paper that it is associated with, this data is not necessarily included under the same header for each study. This means that it may take longer to figure out whether studies are relevant or not. GEO is a public repository. Reference: GEO Overview - GEO - NCBI. National Center for Biotechnology Information. https://www.ncbi.nlm.nih.gov/geo/info/overview.html (accessed April 27, 2018). S04-4: What is one way that generating disease signatures from meta-analyses can improve clinical care?
A. Disease signatures that are generated from meta-analysis are individual specific, which means that we can find unique treatments for each individual
B. There is no way to associate disease signatures with potential treatments
C. Using pathway analysis, we can find treatments that are known to target genes that are upstream in the disease signature
D. Using pathway analysis, we can determine the odds that a potential treatment will work before we know what molecule it interacts with
Answer: C. Using pathway analysis, we can find treatments that are known to target genes that are upstream in the disease signature Explanation: Running a pathway analysis on the disease signature allows us to see the upstream regulators that are most likely to be differentially regulated. We can then use this information to find potential treatments based on the information we have on which treatments interact with those upstream regulators.
164 | P a g e
Reference: Ingenuity Pathways Analysis (IPA): Using the Ingenuity Knowledgebase understand genes and disease. Center for Human Health and the Environment. https://chhe.research.ncsu.edu/wordpress/wp-content/uploads/2015/10/IPA-Training-slides-Search-and-Explore-2016_04.pdf (accessed April 26, 2018). Digital Health Innovation System Integration Assessment: A methodology for streamlined early-stage evaluation
--Bailey Griffin
S04-5: While there is increased interest and investment in digital tools to improve the outcomes, experience and cost of care, the potential impact of an early stage digital health tool can be difficult to ascertain. Our abstract outlines our process and framework for assessing early stage digital health tools and the results of our retrospective review of recommendations given to companies to better align their product with system needs. The most common recommendation given to companies included:
A) Do better market research and hire consultants to help you
B) Identify individuals who have purchasing power at an organization
C) Properly identify the appropriate level of the health system at which the product is likely to be purchased and increase value to the primary payer
D) Focus only on the end needs of patients. The needs of providers and other users are irrelevant since we’re all most interested in improving the patient experience.
Answer: C) Properly identify the appropriate level of the health system at which the product is likely to be purchased and increase value to the primary payer
Explanation:
A) Is not correct as our recommendations do not outline the need to hire a consultant to help any company with determining value proposition framing
B) While it is important to identify your target customer, this Answer is incomplete. Additionally, simply identifying someone with purchasing power at the organization level may not be enough, there are many other system stakeholders who have decision making power that will impact your product.
C) This answer is correct as it accurately reflects the most often recommendation given to the companies assessed in our program.
D) Is wrong since the needs of all users of any system need to be considered in the development and implementation of a tool.
Reference: Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, Court CA, et al. Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and chellenges to the scale up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367.
165 | P a g e
S04-6: Our Digital Health Innovation System Integration Assessment framework is focused on providing individuals and organizations with:
A) An early stage assessment/evaluation of digital health tools to better allocate scarce resources.
B) A basis to skip validation of a digital health tool via rigorous scientific inquiry.
C) An alternative to the regulated procurement processes at an organization.
D) A way to feel engaged with the small to medium sized enterprises in digital health.
Answer: A) An early stage assessment/evaluation of digital health tools to better allocate scarce resources.
Explanation:
A) was our intended purpose of the framework.
B) while the assessment using the framework may signal whether or not a product is sufficiently validated or ready to be further validated, the output of the framework will never be sufficient to skip validation of a digital health tool (e.g. a mobile app to manage depression).
C) unfortunately standard procurement processes will still apply regardless of the output of the assessment. However, the framework could be a very complimentary tool to the scoring rubric in procurement activities.
D) feeling engaged with the digital health enterprise community is not the goal of this tool.
Reference: Burke RE, Shojania KG. Rigorous evaluations of evolving interventions: can we have our cake and eat it too? BMJ Qual Saf. 2018;27:251–4.
S05: Presentations - Clinical Decision Support Embedded Predictive Models
Validating Clinic Workflow Models for Prediction
--Rafael Fricks
S05-1: Which of the following statements best describes over-fitting?
a. As model complexity increases, model predictions show decreased error in validation sets while training set error increases.
b. Over-fitting can be determined without the use of independent data sets.
c. An over-fit model provides an optimistic estimate of model quality when evaluated against the training set.
d. Hypothesis testing (i.e. Kolmogorov-Smirnov Test) cannot be used to detect over-fitting.
166 | P a g e
Answer: c. An over-fit model provides an optimistic estimate of model quality when evaluated against the training set.
Explanation: Over-fitting, in plain language, is when a model becomes so well adapted to nuances in a particular data set that it loses accuracy in predicting similar data in a different sample. Over-fitting is characterized by an increase in validation set error even as training error decreases with increasing model complexity, the opposite of choice (a). Since the prediction error tends to decrease in the training set, choice (c) is correct as typically an over-fit model will perform worse than expected against new data. To determine over-fitting, it is necessary to compare performance on an independent data set, usually referred to as a validation or test set in context (choice (b) is incorrect). Finally, while it is more common to use measures such as squared error or absolute error, it is possible to use test statistics from hypothesis testing as a measure of error (choice (d)). For more reading on model testing, see(1).
Reference: Hastie T. The elements of statistical learning : data mining, inference, and prediction. Friedman JH, Tibshirani R, editors. New York: Springer; 2009.
S05-2: Choose the answer that best completes the statement: “K-fold cross-validation…”
a. reduces the influence of any particular set of data.
b. should divide data into five subsets.
c. requires at least 100 observations.
d. should be used over other model assessment techniques such as bootstrapping.
e. is a feature introduced in MATLAB 2017a.
Answer: a. reduces the influence of any particular set of data.
Explanation: K-fold cross-validation subdivides a set into k subsets, where each set is used as the evaluation set once (1). This is an improvement over standard cross-validation that selects one evaluation set, as it puts less emphasis on the selection of the evaluation set (choice (a)). There are however several choices for k, where five is a common choice but not appropriate in all situations (choice (b)). Similarly, while it is desirable to have ‘enough’ data to make subsets, ‘enough’ is determined in context. For instance, few observed subjects (<100) with several fields of data per subject may be suitable for subdivision (choice (c)). Other methods may be appropriate in evaluating certain types of model, and variations such as leave-one-out or bootstrapping exist (choice (d)). Each method provides trade-offs in model evaluation and can be chosen based on the application (1). Finally, cross-validation is not specific to any software package (choice (e)), though several packages may have convenient functions implemented.
Reference: Hastie T. The elements of statistical learning : data mining, inference, and prediction. Friedman JH, Tibshirani R, editors. New York: Springer; 2009.
167 | P a g e
Patient-Specific Explanations from Risk Prediction Models
--Amin Tajgardoon
S05-3: Machine learning models are being increasingly developed to predict clinical outcomes such as mortality, morbidity, and adverse events. Among the following choices, which one enable clinicians to engender trust in a sophisticated machine learning model with excellent prediction power?
A. The model achieves a high area under the ROC curve (AUROC)
B. Satisfy the assumption that training data samples are independent and identically distributed
C. Provide simple and reliable patient-specific explanations for each prediction
D. The model has a very low generalization error
Answer: C. Provide simple and reliable patient-specific explanations for each prediction
Explanation: Sophisticated predictive models with excellent performance are reported in the literature at an increasing pace. These models in most cases are regarded as black boxes that produce a prediction for an outcome, along with a probability reflecting the certainty, from the features of a patient case. However, for such models to be practically useful in prevention, diagnosis, prognosis, treatment and other activities in clinical care, it is critical to provide clinicians with simple and reliable patient-specific explanations for each prediction. Such explanations will enable clinicians to engender trust in the predictions, interpret them in the clinical context, and even influence the clinical decisions they make.
References: 1. Štrumbelj E, Bosnić Z, Kononenko I, Zakotnik B, Kuhar CG. Explanation and reliability of prediction models: the case of breast cancer recurrence. Knowledge and information systems. 2010 Aug 1;24(2):305-24.
2. Luo G. Automatically explaining machine learning prediction results: a demonstration on type 2 diabetes risk prediction. Health information science and systems. 2016 Dec 1;4(1):2.
S05-4: Which of the following definitions best describes “prediction explanation” in the context of a machine learning model that predicts a clinical outcome?
A. Present to the user the model’s prediction score for a clinical outcome
B. Present an interpretation of the model to the user in terms of structure and parameters
C. Allow the user to select a model with the best accuracy in predicting the clinical outcome
D. Provide the user with an understandable interpretation of the prediction for an individual to whom the model is applied
Answer: D. Provide the user with an understandable interpretation of the prediction for an individual to whom the model is applied
168 | P a g e
Explanation: There is a distinction between model explanation and prediction explanation.
Model explanation provides an interpretation of the model to the user in terms of structure and parameters, and is used in discovery. Some predictive models are more easily interpretable such as decision trees and linear models. Often interpretable models have poorer predictive performance than more abstract models such as random forests, support vector machines and neural networks.
Prediction explanation provides an interpretation of the prediction for an individual to whom a model is applied, and will potentially be different from individual to individual. Acceptable prediction explanations possess two properties: 1) explanation use concepts that are understandable to the user such as variables that are not modified or transformed; and 2) the explanation is simple or parsimonious so that it is readily and rapidly grasped by the user.
Prediction explanations tend to be simpler than model explanations.
Reference: Lipton ZC. The mythos of model interpretability. arXiv preprint arXiv:1606.03490. 2016 Jun 10.
Leveraging Electronic Health Record Data to Predict Surgery Cancellation
--Nick Pratap S05-5: In this study we implemented machine learning algorithms to explore variables from patient records to predict day-of-surgery cancellation. Overfitting is one of the common problems in machine learning which will decrease the generalizability of the model. Which machine learning algorithm has the highest risk of overfitting?
A. Gradient boosted LR
B. Random Forest
C. Regularized logistic regression
D. Classification tree
Answer: D. Classification tree Explanation: Classification trees are non-parametric models that have more flexibility and are prone to overfit training data. The other three algorithms use regularization or ensemble techniques to avoid overfitting. Reference: Kuhn M, Johnson K. Applied predictive modeling. New York: Springer; 2013 May 17. S05-6: In this study we utilized a iterative step-forward method to do feature selection. What is the primary objective of doing feature selection?
A. It identifies and selects key predictors to improve the interpretability of the model.
169 | P a g e
B. It increases the complexity of the model.
C. It creates new combinations of variables in order to reduce the dimensionality of the data.
D. It decreases the prediction performance of the model.
Answer: A. It identifies and selects key predictors to improve the interpretability of the model. Explanation: Feature selection methods reduce the complexity of the model and make it easier to understand and explain. It chooses a subset of original variables to reduce the dimensionality of the data without creating or changing them. Moreover, doing feature selection can improve the performance of the model if the optimal subset is chosen. Reference: Kuhn M, Johnson K. Applied predictive modeling. New York: Springer; 2013 May 17.
S06: Presentations - Informatics Application and Research to Enhance Surgical Practice
Adapting Technology into the Surgical Environment
-- Alex Langerman
S06-1: Information targeted towards surgical circulating nurses is best presented:
A. for the entire case at the start of a case (e.g., during time out or a preference card)
B. through a sterile interface to prevent surgical site infection (SSI)
C. in a mobile format that can be transported within and outside the operating room
D. with anatomic images to illustrate the choices between surgical techniques
E. for all of the cases of the day, to allow "anticipatory guidance"
Answer: C. in a mobile format that can be transported within and outside the operating room
Explanation: Circulating nurses are the most mobile team members within the room, and often leave the room to get additional supplies or deliver specimens. They are not scrubbed in or sterile. They can benefit from anatomic knowledge, but do not participate in moment to moment decisions regarding surgical technique (the scrub and particularly the surgeon do this). For situational awareness, information should be given on a step-by-step basis, not all at once at the beginning (non-surgeon team members change throughout a case and therefore not all members were present at the time out); these step data should include guidance on upcoming tasks within a case (not upcoming cases) to support anticipatory guidance.
Reference: Julie M. Mhlaba, Laura W. Christianson, Stuart J. Davidson, et al. Field research in the operating room. Ergonomics Design. 2016;24(4):10-9
170 | P a g e
S06-2: The most common and financially impactful result of poor communication regarding tools and tasks in the operating room is:
A. tool damage and repair/replacement costs (RRC)
B. over-provisioning of surgical supplies
C. lapses in sterility from improper draping
D. reductions in mean turnover time (TTm)
E. over-staffing of surgical cases
Answer: B. over-provisioning of surgical supplies
Explanation: Poor communication around tools and tasks in the operating room has many effects, including slower turnover times, staff member exits out of the room to retrieve missing but necessary items, and surgical delays. One of the most common and impactful problems is over-provisioning of surgical supplies, which results in wasted money on open-but-unused items and cognitive overload on staff. While over-provisioning and opening of unnecessary trays can result in more frequent reprocessing and tool breakdown, the cost of this is much smaller than the other costs associate with over-provisioning. Improper draping sufficient to cause a lapse in sterility can occur but this is not a frequent occurrence.
References: Stockert EW, Langerman A. Assessing the magnitude and costs of intraoperative inefficiencies attributable to surgical instrument trays. J Am Coll Sur. 2014;219(4)646-55
Zygorakis CC, Yoon S, Valencia V, et al. Operating room waste: disposable supply utilization in neurosurgical procedures. J Neurosurg. 2017;126(2):620-5
EHR Needs of the Surgical Subspecialist: Insights from System Selection
--Saira Haque
S06-3: What functionality should the surgical specialist consider when selecting an EHR?
A. Reconciling long term meds from outside vendors
B. Presenting longitudinal data for chronic disease
C. Targeting information pertinent to surgical problem
D. Graphing of BMI, blood pressure and pulse over time
Answer: C. Targeting information pertinent to surgical problem
171 | P a g e
Explanation: The relationship between the surgical subspecialist and the patient is characterized by episodic care that is targeted. The answers A, B and D are more appropriate for primary care providers or specialists whose relationship with the patient is characterized by long-term care management.
Reference: https://www.facs.org/
S06-4: When an EHR vendor demonstrates the system’s functionality, which points should be clarified?
A. the programming and coding employed in the programming software
B. the ability to adapt the system to clinical and administrative workflows
C. the ability of the system to withstand cyber-attacks from external sources
D. workflow of enterprise systems beyond the scope of the current project
Answer: B. the ability to adapt the system to clinical and administrative workflows
Explanation: Clinical and administrative workflows of the system being reviewed are an important piece of the demonstration; the other items are not relevant to a demonstration of system functionality.
References: Sanders DS, Read-Brown S, Tu DC, et al. Impact of an Electronic Health Record Operating Room Management System in Ophthalmology on Documentation Time, Surgical Volume, and Staffing. JAMA Ophthalmol. 2014;132(5):586-592. https://jamanetwork.com/journals/jamaophthalmology/fullarticle/1851727
Duwayri Y et al. The vascular surgeon's roadmap to success in the Quality Payment Program. Journal of Vascular Surgery, Volume 65 , Issue 5 , 1536. http://www.jvascsurg.org/article/S0741-5214(17)30332-4/fulltext
Challenges of Electronic Health Record Transition for Surgical Services: A Qualitative Observational Study of “Go Live”
--Megan Shroder S06-5: In our study of surgical practices during an electronic health record transition, one of the most common sources of problems and confusion was related to:
A. Admission orders
B. Phases of care
C. Operative notes
D. Inpatient documentation
Answer: B. Phases of care
172 | P a g e
Explanation: Phases of Care used in the Epic System were a common point of confusion across surgical faculty, residents, and staff. There were numerous help tickets regarding lost orders with several potential patient safety issues. Activities done in the different Phases of Care in surgery are highly specialty-specific. Workflows varied across surgical specialties and were not amenable to instruction with simple examples. Significant errors related to Phases of Care occurred when the timing of surgery changed, and such changes are the norm rather than the exception in organizations that provide trauma care and emergency surgery services. Innovative solutions to various problems that arose were developed and applied by providers until more formalized solutions could be implemented, as has been seen in some previous studies on electronic health record transitions. Reference: J Tim Scott, Thomas G Rundall, Thomas M Vogt, John Hsu. "Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study". BMJ. 2005; 331:1313-1316. S06-6: An important recommendation from our study of surgical practices during an electronic health record transition at an academic medical center is dedicated training for:
A. Work shared across teams of residents, staff, and faculty
B. Emergency department to operating room admissions
C. Physician scheduling of procedures from the outpatient setting
D. Documentation for trauma resuscitation
Answer: A. Work shared across teams of residents, staff, and faculty Explanation: In our study, the surgical teams were not well prepared for functioning as group and academic practices (i.e., with support of trainees). Provider training focused on individual charting responsibilities, with it largely being assumed that the physician provider would perform all tasks and be consistently available to enter and sign orders. In the emergent setting, surgeons are busy doing procedures or resuscitating patients. Training for faculty was done separately from trainees and staff. The faculty training did not address how work would be shared among team members. Reference: J Tim Scott, Thomas G Rundall, Thomas M Vogt, John Hsu. "Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study". BMJ. 2005; 331:1313-1316.
173 | P a g e
S07: Panel - LEANing into the Quality Payment Program: Using Agile Strategies for Real Practice Improvement
--Julia Skapik, et al S07-1: Which is NOT one of the four reporting categories in the Quality Payment Program (QPP) Merit-based Incentive Payment Program for individual providers and groups?
A. Advancing Care Information
B. Improvement Activities
C. Clinical Decision Support
D. Quality Measures
Answer: C. Clinical Decision Support Explanation: The Quality Payment Program is part of the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA), which eliminated the Sustainable Growth Rate (SGR) and created a broad-based pay for performance program for clinicians (physicians and beyond) and accountable care organizations. Clinicians have two tracks to choose from in the Quality Payment Program based on their practice size, specialty, location, or patient population: Merit-based Incentive Payment System (MIPS) or Advanced Alternative Payment Models. The MIPS arm of the program is targeted at fee-for-service based individual and group physicians and has four components: Advancing Care Information, Improvement Activities, Quality Measures, and Cost. Clinical Decision Support is not included in the reporting requirements for this program. References: https://qpp.cms.gov/ https://www.federalregister.gov/documents/2017/11/16/2017-24067/medicare-program-cy-2018-updates-to-the-quality-payment-program-and-quality-payment-program-extreme S07-2: Lean is a tool that has been around the manufacturing industry for a long time and only more recently has it been applied to healthcare organizations. This is partially due to the healthcare reimbursement system changing from fee for service to pay for value incentivizing more and more organizations to undergo Lean transformations. Taking on a Lean transformation can provide an organization significant gains in productivity, quality, and cost reduction but requires a cultural shift. This type of a shift is driven by leadership and needs to have the right principles in place to transform the culture. Which of the following are the most appropriate management principles to create a Lean culture?
A) Value to the customer, open communication, participate and contribute, pursue excellence, share knowledge, keep it simple.
B) Be a leader, exceed customer expectations, continuously improve, keep it simple, seek perfection, respect for every individual.
C) Attitude of continuous improvement, value creation, unity of purpose, respect for front-line workers, visual tracking, flexible regimentation.
174 | P a g e
D) Create value at every step, quantity over quality, keep it simple, seek relentless perfection, share knowledge, listen and communicate.
Answer: C) Attitude of continuous improvement, value creation, unity of purpose, respect for front-line workers, visual tracking, flexible regimentation. Explanation: To achieve a successful Lean transformation and sustain the gains you must change the culture of the organization. These 6 principles: attitude of continuous improvement, value creation, unity of purpose, respect for front-line workers, visual tracking, and flexible regimentation have been found across multiple successful Lean organizations and provide a starting framework for leaders to instill as they take on their own lean journeys. References: John S. Toussaint, MD, and Leonard L. Berry, PhD.The Promise of Lean in Healthcare. Mayo Clinic Proceedings [Internet] http://www.mayoclinicproceedings.org/article/S0025-6196(12)00938-X/fulltext S07-3: Capturing and extracting valid and reliable data from certified EHRs requires iterative improvements to the certified EHRs, workflow, data entry screens and strategies to capture valid data based on the measures requirements and specifications. Which of the following is most likely the case in terms of iterative effort?
A. 80% of the effort is by the vendor to provide modifications within the certified EHR/20% is with the clinical side to modify workflow to accommodate eCQM data capture
B. 60% of the effort is by the vendor to provide modifications within the certified EHR/40% is with the clinical side to modify workflow to accommodate eCQM data capture
C. 50% of the effort is by the vendor to provide modifications within the certified EHR/50% is with the clinical side to modify workflow to accommodate eCQM data capture
D. 20% of the effort is by the vendor to provide modifications within the certified EHR/80% is with the clinical side to modify workflow to accommodate eCQM data capture
E.10% of the effort is by the vendor to provide modifications within the certified EHR/90% is with the clinical side to modify workflow to accommodate eCQM data capture
Answer: D. 20% of the effort is by the vendor to provide modifications within the certified EHR/80% is with the clinical side to modify workflow to accommodate eCQM data capture Explanation: Eisenberg et al., (ND) in a study done for American Hospital Association on the impact of meaningful use clinical quality indicators found that early adopters of electronic clinical quality measures (eCQMs) had significant challenges getting to valid and reliable data for quality improvement. This study found that iterative modifications were needed to fix the problems with data, with a majority of the effort required by the clinical team. This has continued to be the case with extracting reliable data from certified EHRs. There findings indicate 20% of the effort is by the vendor to provide modifications within the certified EHR/80% is with the clinical side to modify workflow to accommodate valid and reliable eCQM data capture.
175 | P a g e
References: Eisenberg, F., Lasome, C., Advani, A., Martins, R., Craig, P. and Sprenger, S. (N.D.) A study of the impact of Meaningful Us electronic clinical quality measures www.aha.org/content/13/13ehrchallenges-report.pdf S07-4: Which of the following strategies is most likely to improve problems with electronic quality measure data capture and reporting?
A. Using only data from a clinical registry
B. Manual abstraction
C. Natural language processing in writing clinical notes
D. Use of a clinical data model associated with a standardized data warehouse
Answer: D. Use of a clinical data model associated with a standardized data warehouse Explanation: While the use of any of these techniques may be used in the capture or validation of quality measure data, the most direct and immediately impactful is the use of a data model, such as the Quality Data Model (QDM). The data model would need to be mapped to data as it is captured into the system, with that data model information being tagged to it as metadata as it moves into the data warehouse. For maximum effectiveness, the quality measures targeted should be mapped to the data model at the beginning of the period of capture and tested throughout the period of performance. References: Taxiarchis Botsis, Gunnar Hartvigsen, Fei Chen, Chunhua Weng. Secondary Use of EHR: Data Quality Issues and Informatics Opportunities. AMIA Summit on Translational Bioinformatics. 2010 Johnson III RJ. A Comprehensive Review of an Electronic Health Record System Soon to Assume Market Ascendancy. J Healthc Commun. 2016, 1:4. DOI: 10.4172/2472-1654.100036
S08: Presentations - Bridging Analytics
POKE-R: Reducing Patient Harm
--James McGlothlin
S08-1: After the analytics have been developed, one of the most important elements to a successful project is a good deployment strategy for user adoption. When attempting to implement POKE-R in a specific unit, what is of the following is a key element which will help us succeed?
A. Utilizing specially trained nurses
B. Having one source of truth for all types of orders
C. Performing structured rounding
D. Meeting regularly with informatics professionals
E. Preventing duplicate orders in the EMR
176 | P a g e
Answer: C. Performing structured rounding
Explanation: Having a process for all of a patient's providers to meet and discuss the day’s orders and events together is key to reducing POKE-R events. The POKE-R project can be implemented without requiring any special training for nurses and multiple EMRs can be leveraged. It is important to make sure that clinicians have the information to make decisions on their own on a daily basis without relying on IT. It is important, as well, to allow clinicians to write any orders in the EMR which they may deem necessary.
Reference: https://hbr.org/2013/11/a-remedy-for-fragmented-hospital-care
S08-2: To develop a a project such as POKE-R, it is important to establish specific and attainable goals. Which of the following is a reasonable and significant goal for a POKE-R project.
A. Reduce surgeries
B. Reduce lab draws
C. Reduce medication administrations
D. Change nutritional orders
E. Reduce mortality
Answer: B. Reduce lab draws
Explanation: All lab draws are POKE-R events and it is possible to significantly reduce the number of lab draws and the cost of lab draws chosen.
While surgeries are POKE-R events, it is not realistic to reduce surgeries. Instead of reducing medications, the POKE-R project aims to alter the route used for medications.
Nutritional orders are not POKE-R events. Mortality may be reduced but this is not an achievable goal to make as a primary goal of a project.
Reference: http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0006174603620369
177 | P a g e
The Clinical Informatics Fellowship Experience: A Qualitative Analysis (qACIF)
--Benjamin Orwoll S08-3: You are a first-year resident and you find yourself very interested in how the information and computer systems at your institution can be leveraged to improve care. A faculty member mentioned the clinical informatics fellowship, but you are not sure what kinds of qualifications you might need to apply successfully. Of the following, which profile most closely approximates the experiences prior to fellowship from the cohort of fellows surveyed in the qACIF study?
a. Quality improvement scientists with extensive research experience who want to apply informatics principles in their work
b. Physicians with bachelors or graduate degrees in computer science intending to apply machine learning algorithms in health care.
c. Young physicians with interests in technology who are frustrated with the current state of health IT and desire to be leaders toward future system change.
d. Physicians with IT leadership background who need dedicated training in clinical informatics to obtain board certification
e. Entrepreneurial physicians who have previous business experience and need to gain a foothold for their devices in the health information ecosystem.
Answer: c. Young physicians with interests in technology who are frustrated with the current state of health IT and desire to be leaders toward future system change. Explanation: This question really addresses information from the qACIF study related to the demographics and background, as well as motivations, of the first few groups of CI fellows to enter the fellowship. In general, responders were relatively young (low 30's) and did not tend to have a computer science background. They tended to express frustration with the current system as a significant motivator for further work in the area of clinical informatics and expressed the desire to be actors for future change. In general, CI fellows did not have significant prior independent practice experience. Though there were aspirations in some of the fellows toward academic research and entrepreneurship, the most representative answer is C.
S08-4: Clinical informatics fellowships, as they are currently structured, do not specify training in a specific medical specialty or specialties as a prerequisite to application. Thus, clinical informatics can be a subspecialty of any medical specialty. Based on the results of the qACIF survey, which of the following medical specialties is most significantly underrepresented among clinical informatics fellows?
a. Pediatrics
b. Emergency Medicine c. Pathology d. General Surgery e. Internal medicine
178 | P a g e
Answer: d. General Surgery Explanation: The respondents from the qACIF survey included over 70% of the known active CI fellows at the time. Though internal medicine, pediatrics, emergency medicine, and pathology were well-represented, self-reported primary medical specialties in the survey included only one proceduralist: an interventional radiologist. 2.6% (1/38) is a much lower representation of procedural specialties than among active physicians. The reasons for the underrepresentation of procedural/surgical specialties were not explored in this survey. New recruitment strategies and/or curriculum changes may be needed in order to develop a diverse clinical informatics work force. References: AAMC 2016 Physician Specialty Data Report (https://www.aamc.org/data/workforce/reports/457712/2016-specialty-databook.html)
Interdisciplinary Development of Palliative Care EHR Decision Support
--Ruth Bush
S08-5: Palliative care is associated with more efficient resource use and decreased medical costs. The targeted technology available in the EHR, including clinical decision support could facilitate screening for earlier identification of patients in or about to be in distress; to provide more appropriate referrals to specialized care; and to facilitate more rapid communication among patients, caregivers, and clinicians.
Which of the following is a good working definition of palliative care:
a. A short-term approach to providing pain control.
b. Targeted patient education designed to prevent individual chronic disease development.
c. Medical care, which prevents or treats symptoms and side effects of disease and treatment and is designed to be patient and family-centered.
d. Engagement of the patient in their health decision making through technology such as patient portals and tracking devices.
Answer: C. Medical care, which prevents or treats symptoms and side effects of disease and treatment and is designed to be patient and family-centered.
Explanation: The World Health Organization defines palliative care (PC) as, “an approach that improves the quality of life of patients and their families facing the problem associated with life-threatening illness, through the prevention and relief of suffering by means of early identification and impeccable assessment and treatment of pain and other problems, physical, psychosocial and spiritual.” A reasonable working definition of palliative care is medical care, which prevents or treats symptoms and side effects of disease and treatment and is designed to be patient and family-centered. While it contains elements of pain control, patient education, and engagement these choices are too limited and
179 | P a g e
do not recognize the multi-faceted, interdisciplinary care of patient and family, which is expressed in choice “c”.
References: WHO | WHO Definition of Palliative Care. WHO [Internet]. World Health Organization; 2012 [cited 2017 Apr 6]; Available from: http://www.who.int/cancer/palliative/definition/en/
Vorvick LJ. What is palliative care [Internet]. 2016. Available from: https://www.nln.nih.gov/medlineplus/ency/patientinstructions/000536.htm
Wysham NG, Hua M, Hough CL, Gundel S, Docherty SL, Jones DM, et al. Improving ICU-Based Palliative Care Delivery: A Multicenter, Multidisciplinary Survey of Critical Care Clinician Attitudes and Beliefs. Crit Care Med. 2016/09/13. 2016;
S08-6: Which of the following statements most accurately reflects the results of the palliative care trigger list feasibility study:
a. The final algorithm version had several hundred variables to ensure it captured every possible individual needing palliative care.
b. The final algorithm needed to be validated with manual chart review.
c. The trigger list had poor sensitivity; patients retrospectively identified were not likely to have been seen prospectively by the palliative care team.
d. The algorithm relied primarily on laboratory testing as the basis for the clinical decision support.
Answer: B. The final algorithm needed to be validated with manual chart review.
Explanation: The final algorithm was reduced to 49 variable and retrospectively identified almost 98% of patients who were visited prospectively by the palliative care team. The algorithm relied on a number of variables including: race/ethnicity, language, being unresponsive, feeding tube insertions, specialty bed use in patients with difficulty moving, and two-person movement assistance. Accuracy of the electronic data verified using manual chart review to provide verification of electronic clinical decision support algorithm, which is expressed in choice “b”.
Reference: Rhodes RL, Kazi S, Xuan L, Amarasingham R, Halm EA. Initial Development of a Computer Algorithm to Identify Patients With Breast and Lung Cancer Having Poor Prognosis in a Safety Net Hospital. Am J Hosp Palliat Care. 2015/07/05. 2016;33(7):678–83.
180 | P a g e
S09: Presentations - Clinical Decision Support - Trends in Optimization
Optimize Up: Using Analytics for Clinical Decision Support Optimization
--Anwar Mohammad Sirajuddin
S09-1: Which one of the following is NOT a potential outcome of Clinical Decision Support (CDS) alerts?
A. Creation of a continuity of care document
B. Prevention of adverse drug events
C. Improvement of reimbursement and regulatory compliance
D. Provide treatment recommendations
Answer: A. Creation of a continuity of care document
Explanation: A key component of the Electronic Health Record are Clinical Decision Support (CDS) alerts and/or systems that aid the user with clinical decisions. Key outcomes of CDS alerts could be to provide treatment recommendations, prevent adverse drug events, cost containment and improvement of reimbursement and regulatory compliance
CDS alerts are not designed to create a continuity of care document (CCD). Continuity of care documents are structured documents created within the EMR but not within the CDS component.
Reference: https://www.healthit.gov/providers-professionals/faqs/what-clinical-decision-support
S09-2: In order to optimize Clinical Decision Support (CDS) alerts, Memorial Hermann identified potential areas of inefficient alerting to the clinical users. The main two criteria they reviewed were:
A. Low frequency alerts and high alert override rates
B. Low frequency alerts and low alerts override rates
C. High frequency alerts and low override rates
D. High frequency alerts and high alert override rates
Answer: D. High frequency alerts and high alert override rates
Explanation: Data extraction tools and data visualization tools identify potential areas of inefficient alerting to the clinical users. Knowing that CDS alerts need to be actionable and measurable, Memorial Hermann identified these opportunities by looking at two key criteria:
• High frequency alerts (resulting in “alert fatigue”)
• High override rate (suggesting low clinical utility for the alert)
181 | P a g e
Reference: https://www.healthit.gov/sites/.../strategies_for_optimizing_an_ehr_system_0.docx
An Evaluation of Clinical Order Patterns Machine-Learned From Clinician Cohorts Stratified by Observed Versus Expected 30-Day Mortality Rates
--Jason Ku Wang
S09-3: With the emergence of data-driven clinical decision support tools, in particular those that aggregate the collective expertise of clinicians and make recommendations based on patterns learned from historical electronic health record data, defining a robust external reference standard is necessary. For the case of promoting compliance with clinical practice guidelines, which of the following proves to be the largest barrier to defining a reference standard, even for common admission diagnoses?
A. There are too few clinical practice guidelines in the literature to assemble a robust reference standard
B. In translating clinical practice guidelines for a given admission diagnosis into a finite reference standard, subjectivity and interpretation are required
C. There is no systematic way for multiple clinicians with different opinions to reach a consensus on a single reference standard
D. Clinical practice guidelines derived from more reputable literature source should be assigned more weight than those derived from less reputable sources
Answer: B. In translating clinical practice guidelines for a given admission diagnosis into a finite reference standard, subjectivity and interpretation are required
Explanation: With inherent variability in medical practice, clinical practice guidelines for a given admission diagnosis may vary from guideline to guideline. Furthermore, since the guideline may be written in a manner that emphasizes certain procedures, treatments, etc., more than others, it is left up to the clinician reading the guideline to determine whether or not a clinical order should be included in the reference standard. This subjectivity proves to be the largest challenge. Clearly, for rare admission diagnoses, the number of published clinical practice guidelines diminishes substantially, precluding the creation of a robust reference standard. However, the question prompt specifically states “even for common admission diagnoses,” implying that the availability of clinical practice guidelines is not the main issue here. Note also that the process of achieving consensus between multiple clinicians with different opinions can be achieved via adjudication.
Reference: Wang JK, Schuler A, Shah NH, Baiocchi MTM, Chen JH. Inpatient Clinical Order Patterns Machine-Learned from Teaching Versus Attending-Only Medial Services. AMIA Informatics Summit. 2018.
182 | P a g e
S09-4: Clinical order patterns derived from data-mining electronic health records can be a valuable source of decision support content. However, learned patterns can be compromised by underlying provider experience as clinical orders of less experienced providers (trainees) can exhibit greater variability and worsened patient outcomes than those of experienced providers (attendings). Based on the conclusions of this podium abstract, which choice of training data will invariably yield the greatest compliance with clinical practice guidelines?
A. Training indiscriminately on all available data
B. Always training on a cherry-picked subset of the most expert clinicians
C. Always weeding out data generated by less experienced clinicians (e.g. trainees) to improve model accuracy
D. No single approach will invariably yield the best outcomes
Answer: D. No single approach will invariably yield the best outcomes
Explanation: In alignment with clinical practice guidelines, we find that the low-mortality clinician model outperforms the crowd clinician model across 6 common admission diagnoses. However, we cannot conclude that abiding by the same approach (e.g. always training exclusively on low-mortality “experts” or always training on all available data) will invariably result in the greatest compliance with clinical practice guidelines, across all diagnoses.
Reference: ang JK, Schuler A, Shah NH, Baiocchi MTM, Chen JH. Inpatient Clinical Order Patterns Machine-Learned from Teaching Versus Attending-Only Medial Services. AMIA Informatics Summit. 2018.
Developing a Learning Electronic Medical Record System
--Shyam Visweswaran S09-5: A learning EMR identifies and highlights EMR data that is relevant for a given patient and clinical context. Which of the following can be enabled by such a learning EMR?
A. Faster access to patient data
B. Increased responsiveness of EMR to mouse clicks
C. Reduced time needed to assess the patient’s condition
D. More coherent display of patient data
Answer: C. Reduced time needed to assess the patient’s condition Explanation: The learning EMR focuses the physician’s attention on relevant patient data. This could help reduce the time needed to assess the patient’s condition and improve the quality of the resulting judgments, enabling improved decision making, reduced medical errors, and greater efficiency.
183 | P a g e
References: King, AJ, Cooper, GF, Hochheiser, H, Clermont, G, Visweswaran, S. Development and preliminary evaluation of a prototype of a learning electronic medical record system. In: AMIA Annual Symposium Proceedings. 2015 Nov 17; 2015:1967-75. S09-6: A learning EMR discovers what type of patterns from the EMR database?
A. Medical appointment patterns
B. Patient data access patterns
C. Patterns of comorbidities
D. Medication prescription patterns
Answer: B. patient data access patterns Explanation: The learning EMR uses a data-driven approach to predict and highlight data that are most relevant to a patient case. It does so by analyzing patterns of patient data access in the EMR. From these patterns it learns a model of which EMR data are likely to be relevant to highlight for a given patient and clinical context. References: King, AJ, Cooper, GF, Hochheiser, H, Clermont, G, Visweswaran, S. Development and preliminary evaluation of a prototype of a learning electronic medical record system. In: AMIA Annual Symposium Proceedings. 2015 Nov 17; 2015:1967-75.
S10: Presentations - Collaboration, Pathways and Training in EHR Implementations
Using a Cloud-based Collaboration and Dissemination Platform to Support the Implementation of an Evidence-based Clinical Pathway Program Across a Multi-Entity Academic Healthcare System
--Emilia Flores
S10-1: Which of the following best describes a clinical pathway?
A. A graphical overview of a process
B. A document that contains criteria for guiding diagnosis and treatment of diseases or conditions
C. A method for translating evidence into local work structures and processes
D. A document that aims to standardize care for a specific clinical problem in a specific population
E. Structured and multidisciplinary plan of care, used to incorporate evidence into local practice, that has criteria-based progression, and aims to standardize care for a specific clinical problem in a specific population.
184 | P a g e
Answer: E. Structured and multidisciplinary plan of care, used to incorporate evidence into local practice, that has criteria-based progression, and aims to standardize care for a specific clinical problem in a specific population.
Explanation: An analysis of the seminal articles on clinical pathways by Kinsman et at. (2010) found that clinical pathways were characterized by the following descriptions: the intervention is a structured multidisciplinary plan of care and meets three of the following four criteria: (1) the intervention is used to incorporate guidelines or evidence into local practice; (2) the intervention contains detailed steps for the plan of care; (3) the intervention has timeframes or criteria-based progression; and (4) the intervention aims to standardize care for a specific clinical problem, procedure or episode of healthcare in a specific population.
Options A, B, C, and D describe parts of the definition of a clinical pathway, but do not encompass all characteristics.
References:
Kinsman L, Rotter T, James E, Snow P, Willis J. What is a clinical pathway? Development of a definition to inform the debate. BMC Medicine. 2010;8(1):31.
Rotter T, Kinsman L, James E, Machotta A, Willis J, Snow P, Kugler J. The effects of clinical pathways on professional practice, patient outcomes, length of stay, and hospital costs: Cochrane SystematicRreview and Meta-analysis. Evaluation & the Health Professions. 2012;35(1):3-27.
S10-2: Research findings have shown that patients do not always receive evidence-based care and treatment, despite the availability of evidence and published guidelines (Agency for Healthcare Research and Quality (2016)). Clinical pathways are one method for integrating evidence into practice; however, the development and implementation approach can have a significant impact on provider adoption. Which of the following approaches for clinical pathway development and implementation is most likely to have a positive impact on provider uptake?
A. Using a rigorous process and framework for clinical pathway development and dissemination, including identification of a clinical owner, basing the pathway on a synthesis of current guidelines, use of a multidisciplinary stakeholder group to review the evidence and adapt based on the local setting, and identification of barriers and facilitators to dissemination and tailoring the plan accordingly
B. Use of technology to provide centralized and easy access to clinical pathways across all practice locations
C. Incorporating access to clinical pathways in provider workflow
D. Use of smart alerts to identify relevant clinical pathways for a given patient
E. Using a rigorous process and framework for pathway development and dissemination, having a centralized and easy to access resource for storing pathways, and use of technology to provide access to relevant pathways in the provider workflow
185 | P a g e
Answer: E. Using a rigorous process and framework for pathway development and dissemination, having a centralized and easy to access resource for storing pathways, and use of technology to provide access to relevant pathways in the provider workflow
Explanation: Utilizing a systematic approach and framework to identify a clinical owner and recruit a representative stakeholder group, conducting rapid reviews of the evidence, adapting the evidence to the local setting, converting evidence-based guidelines into clinical pathways, and then utilizing technology to facilitate dissemination and thoughtful integration into provider workflow can increase the likelihood of clinical pathway adoption (Davies, Walker, Grimshaw, 2010).
Options A, B, C, and D are components of a systematic approach for pathway development, dissemination, and implementation.
References:
Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map?. Journal of Continuing Education in the Health Professions. 2006;26(1):13-24.
ADAPTE Collaboration (2010) Manual for guideline adaptation – version 2.0. Available at ADAPTE website: http://www.g-i-n.net/document-store/working-groups-documents/adaptation/adapte-resource-toolkit-guideline-adaptation-2-0.pdf (last accessed 20 March 2018).
Built-in Care Pathways and Optimization of Electronic Medical Record System for Efficient Throughput at a High-Volume Academic Medical Institution
--Rika Ohkuma
S10-3: A surgical service line A in an academic medical institution investigated their current workflow to find opportunities to reduce clinical variance postoperatively. They found that postoperative orders varied by attending surgeons and ordering residents. A current clinical pathway was developed four years ago, but rotating residents were not aware of the care pathway. Which is the most appropriate next step to reduce variance of clinical treatment by ordering physicians?
A. More education sessions for ordering physicians to think and order more carefully
B. Develop order sets aligned to the care pathway and have them embedded in EMR
C. Develop a more detailed care pathway so that they can follow each process
D. More process alerts in EMR to raise awareness of the care pathway
Answer: B. Develop order sets aligned to the care pathway and have them embedded in EMR
Explanation: The goal of the service line is to standardize postoperative care among all physicians. Integrating expected orders for each procedure into EMR as order sets makes it easy for ordering
186 | P a g e
physicians to follow the care pathways without building them from the scratch. A brief education session for rotating residents about the order sets would help to be compliant to a new workflow (A). A detailed care pathway might be difficult to follow without a systematic support (C). The more often alerts goes off, the less physicians pay attention to them.
S10-4: A surgical service line B investigated their current workflow to find opportunities to reduce postoperative length of stay. They found that front line care givers are not aware of benchmark and expected length of stay. Inefficient timely preparation caused a delay in hospital discharge or a transfer. Which is the most appropriate next step to reduce unnecessary delay to discharge patients?
A. Start a discharge plan in EMR when patients become clinically ready
B. Start a discharge plan on the day of surgery and update an estimated discharge date daily in EMR
C. Set a target length of stay for each procedure and show an expected discharge date in a patient chart to set a same expectation in a multidisciplinary team
D. Track overall service line length of stay monthly
Answer: C. Set a target length of stay for each procedure and show an expected discharge date in a patient chart to set a same expectation in a multidisciplinary team
Explanation: The goal of the service line is to set a same expectation among front line care givers. Visual cue of target length of stay and discharge dates help them to advance the level of care to meet the discharge goals. It must be too late if you start a discharge plan when patients are clinically ready (A). Early discharge planning and daily updates are critical (B), however, the daily duplicated process should be avoidable by step-wise process guidance for a smooth discharge. Outcome report (D) would help to understand an overall baseline and a longitudinal trend, and procedure specific (patient customized) goal should be considered for more accurate prediction.
Evaluation of a Peer Training Model for Electronic Health Record System Implementation
--William Lancaster
S10-5: How does the peer training model (PTM) differ from the traditional training model?
a. With the PTM, providers train other providers in their same or similar specialty as to how to use the new technology.
b. With the PTM, certified trainers train all of the providers in how to use the new technology.
c. The PTM is less costly than the traditional model.
d. The traditional model allows providers to become more engaged earlier on in the implementation process.
e. The PTM does not require additional time away from clinical activities.
187 | P a g e
Answer: a. With the PTM, providers train other providers in their same or similar specialty as to how to use the new technology.
Explanation: The traditional training model uses certified trainers experienced in an EHR to teach providers how to use a new system. The peer training model coaches providers, who are well known and respected at their institution, to train their peers in the use of the EHR. A purported benefit of this approach learning the system by end-users in a manner that better conforms to their workflow as opposed to learning the various system functions.
Reference: R. A. Orfaly, J. C. Frances, P. Campbell, B. Whittemore, B. Joly, and H. Koh, “Train-the-trainer as an educational model in public health preparedness,” J. Public Health Manag. Pract. JPHMP, vol. Suppl, pp. S123-127, Nov. 2005.
S10-6: The majority of the peer trainer survey respondents identified with which of the following statements:
a. I do not feel that my peers learned the system better because it was taught by a peer
b. I felt more prepared for Go-Live after having served as a peer trainer
c. Fewer problems were identified earlier as a result of using the peer training model
d. I sacrificed too much clinical time in order to serve as a peer trainer.
Answer: b. I felt more prepared for Go-Live after having served as a peer trainer
Explanation: Those that participated as peer trainers overwhelmingly felt more prepared for Go-Live (78.9%). This likely reflects a greater familiarity with the system that was achieved by spending hours teaching the system to their colleagues and answering their questions. Because of this, peer trainers were able to serve as an additional layer of support for their colleagues during Go-Live which may have eased the transition.
Incorrect Answer A: On average, participants felt that their peers learned the system better because of the peer training model (68.0%) This agreement is likely due to the fact that the peer trainers had greater familiarity with their peers’ workflow than a certified trainer.
Incorrect Answer C: The majority of survey respondents (61.6%) also felt that the peer training process allowed for the earlier identification of problems prior to Go-Live. Problems noted during training and described in the comments were largely related to built note templates and order sets that did not match the users’ workflows correctly. Many of these were revised or optimized prior to Go Live.
Incorrect Answer D: 50.1% of trainers did not feel that they sacrificed. too much clinical time in order to be a peer trainer.
188 | P a g e
S11: Panel - Quality Measurement: Authoring and Implementing with Clinical Quality Language and FHIR®
--Ben Hamlin, Anne Smith, Bryn Rhodes, Carol Macumber S11-1: Clinical Quality Language (CQL) is an HL7 standard authoring language that: a. Defines relationships between patients and clinical concepts
b. Uses data from electronic health records or health information technology systems to measure health care quality
c. Allows logic to be shared with both quality measures and decision support
d. Contains quality data for one patient
Answer: c. Allows logic to be shared with both quality measures and decision support
Explanation: CQL is an expression standard that is used for both clinical decision support applications and quality measures and allows logic to be shared with both quality measures and decision support. References: https://ecqi.healthit.gov/cql S11-2: Data elements specified for clinical quality measures reference the Quality Data Model (QDM) which defines:
a. The source of the acquired data
b. The relationships between patients and clinical concepts in a standardized format
c. The codes that define each data element concept
d. SNOMED CT categories
Answer: b. The relationships between patients and clinical concepts in a standardized format Explanation: The QDM is a reference model for the data elements in a clinical quality measure that defines the relationship between patients and clinical concepts in a standardized format. References: https://ecqi.healthit.gov/qdm
189 | P a g e
S11-3: Clinical quality measures rely on standard terminologies for accuracy and reliability of the process being measured. For patient reported measures and clinical observations, the preferred terminology is:
a. CPT
b. SNOMED CT
c. LOINC
d. ICD
e. HCPCS
Answer: c. LOINC Explanation: LOINC is the standard for identifying health measurements, observations, and documents. References: https://loinc.org/ S11-4: FHIR® (Fast Health Interoperability Resources) is a standard for exchanging healthcare information electronically. FHIR® is useful for measurement because:
a. FHIR® is a relatively simple exchange model that maintains data integrity
b. FHIR® can be used as a standalone standard or with other existing standards
c. It is organized into modules.
d. It has patient resources.
Answer: a. FHIR® is a relatively simple exchange model that maintains data integrity. Explanation: While all of the answers are true for FHIR®, the most useful aspect of FHIR® is that it is a simple model for exchanging quality measurement information that ensures consistency of the data being exchanged. Reference: FHIR Overview. https://www.hl7.org/fhir/overview.html
190 | P a g e
S12: Presentations - mHealth Solutions for Screening, Communication, and Disease Management
Digital Drawing Platform for Early Dementia Screening
--Kelvin Tsoi
S12-1: Assuming that digitization of cognitive screening questionnaires is possible, which of the following is the main benefit of digitized screening tests that can supply the dramatically expanding elderly population?
A. Enhancement of screening accuracy
B. Easy collection of digital data of participants
C. Enabling self-administered assessment
D. Prevention of subjective judgments on the drawing images
Answer: C. Enabling self-administered assessment
Explanation: A is incorrect, as we do not have a comprehensive digital screening platform for other cognitive functions, including memory tests. The descriptions on B and D are true, but not the main purpose of using digital screening.
See reference on the presentation.
S12-2: Which of the following best describes the difference between scoring methods for drawing tests in paper-and-pencil form and scoring based on a digital drawing test?
A. Scoring methods for drawing tests in paper-and-pencil form are well-studied, but scoring based on digital drawing test is not.
B. Scoring based on a digital drawing test considers participants’ drawing behaviors, but scoring methods for drawing tests in paper-and-pencil form can only consider the final drawing image.
C. Scoring based on digital drawing test embraces advanced computing techniques is more comprehensive than the traditional screening tests in paper-and-pencil form.
D. Scoring methods for drawing tests in paper-and-pencil form are more objective than the scoring method on digital drawing test.
Answer: B. Scoring based on a digital drawing test considers participants’ drawing behaviors, but scoring methods for drawing tests in paper-and-pencil form can only consider the final drawing image.
Explanation: The key advantage of digital drawing test is the capability of recording the whole drawing process in human-unreachable precision. On the other hand, it is impossible to capture the intermediate
191 | P a g e
steps of the drawing tests in paper-and-pencil tests with a comparable precision. The description in A is incorrect, as there are also other studies on digital clock drawing test. The description in C is vague since the traditional screening tests also include memory tests and other cognitive function, so they are more comprehensive than the digital drawing test. The description in D is reversed. Scoring method on digital drawing methods are objective that human scoring is not involved.
Reference: Souillard-Mandar W, Davis R, Rudin C, Au R, Libon DJ, Swenson R, Price CC, Lamar M, Penney DL. Learning Classification Models of Cognitive Conditions from Subtle Behaviors in the Digital Clock Drawing Test. Mach Learn. 2016;102(3):393-441.
Increasing Efficiency and Enabling Innovation by Implementing Secure Text Messaging Application for Communication and Coordinating Patient Care
--Neha Patel
S12-3: Which of the following communication tools are Health Insurance Portability and Accountability Act [HIPAA]-complaint?
a. Pagers
b. Hospital-issued Smartphones for text
c. Hospital-issued Smartphone for voice calls
d. Hospital-issued Tablets
Answer: c. Hospital-issued Smartphone for voice calls
Explanation: The HIPAA privacy rule provides an individual with the right to access and amend protected health information (PHI) about the individual that is maintained in a designated record set. Accordingly, if text messages are used to make decisions about patient care, then they may be subject to the rights of access and amendment. Text messaging merely represents a set of risks that, like other communication technologies, needs to be managed appropriately to ensure both privacy and security of the information exchanged. Text messages may reside on a mobile device indefinitely, where the information can be exposed to unauthorized third parties due to theft, loss, or recycling of the device. Text messages often can be accessed without any level of authentication, meaning that anyone who has access to the mobile phone may have access to all text messages on the device without the need to enter a password. Texts also are generally not subject to central monitoring by the IT department. Although text messages communicated wirelessly are usually encrypted by the carrier, interception and decryption of such messages can be done with inexpensive equipment and freely available software (although a substantial level of sophistication is needed).
192 | P a g e
Reference: HIPAA Compliance for Clinician Texting. Adam Greene; http://library.ahima.org/doc?oid=105342#.Wqk1PMuWyUk
S12-4: CMS recently announced that texting patient orders would be allowed but if a hospital ensured that the process is through a secure text messaging platform and complies with which of the following:
a. Secure sign-on process(s)
b. Encrypted messaging
c. Allowed use of text messaging (for example, short message service (SMS) text messaging)
d. Customized message retention time frames
Answer: c. Allowed use of text messaging (for example, short message service (SMS) text messaging)
Explanation: On December 28th, 2017, the Center for Clinical Standards and Quality/Survey & Certification Group within the Department of Health and Human Services issued a memorandum that reaffirmed CMS’s stance on texting patient information, specifying that messages sent among clinicians are permissible so long as healthcare teams use a secure platform. Texting patient orders, however, is still prohibited across all platforms.
Reference: https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/Survey-and-Cert-Letter-18-10.pdf
Practical Apps: A Framework to Assess the Quality of Mobile Health Apps for Chronic Disease Management
--Xinjie Yin
S12-5: Which of the following is one of the dimensions of the evaluation framework for mobile health apps for chronic disease management?
A. Accessibility
B. Cost-benefit analysis
C. Sustainability
D. Patient networking
Answer: A. Accessibility
Explanation: Accessibility is part of the evaluation framework.
References:
Monkman H, Griffith J, Kushniruk AW. Evidence-based heuristics for evaluating demands on ehealth literacy and usability in a mobile consumer health application. InMedInfo 2015 Aug 12 (pp. 358-362).
193 | P a g e
Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth. 2015 Jan;3(1).
S12-6: Which poses the greatest challenge for evaluation development of patient-facing mobile applications?
A. Prediction of user commitment
B. Extreme expense involved C. Rapid change in mobile platforms D. Medical society endorsement
Answer. A. Prediction of user commitment
Explanation: Current estimates are that patients abandon up to 80% of apps after two weeks of use. Predicting user commitment is difficult when developing evaluation criteria of patient-facing mobile apps. Mobile health app evaluation framework development does not necessarily need to be a highly expensive endeavor. Change in mobile platforms is not rapid. Medical society endorsement has not been found to be an important evaluation criterion of a patient-facing mobile app.
References: practicalapps.ca
Baldwin JL, Singh H, Sittig DF, Giardina TD. Patient portals and health apps: Pitfalls, promises, and what one might learn from the other. Healthcare. Volume 5, Issue 3, September 2017, Pages 81-85.
S13: Presentations - Healthcare Big Data Analytics
Governing Predictive and Prescriptive Models: Adding the Right People and Processes to Technology
--Farah Ahmed
S13-1: The use of predictive models in healthcare organizations has become more common in the last decade. According to Becker’s Hospital Review, one of the key takeaways from the 2018 Healthcare Information Management Systems Society Conference (HIMSS) was that predictive analytics seems to be driving both operational improvements and clinical decisions (1). However, with the rise of these modeling methods, new challenges have arisen with interoperability and transparency within healthcare systems. While it is clear that predictive analytics has become significant in healthcare, what is the biggest challenge in the deployment of such models?
a. Predictive modeling is not effective when used for small healthcare organizations due to the complexity of the algorithm(s)
b. Predictive model implementation leads to complete replacement of current clinician workflows
c. Lack of oversight in implementation and minimal stakeholder involvement hinders acceptance and use
d. Minimal validation techniques and guidelines are available to ensure scientific validity of the models
194 | P a g e
Answer: c. Lack of oversight in implementation and minimal stakeholder involvement hinders acceptance and use
Explanation: The best response to this question is “c” since it identifies the practical challenges when implementing predictive models in healthcare organizations. When implementing a model for the first time in clinical practice, oversight, stakeholder support, and consensus of model development based on data/assumptions must occur for it to be successful.Stakeholder engagement is vital for the implementation of a predictive model to ensure the appropriate staff has full knowledge of the model deployment and its outcomes (3). The model would not replace the clinical workflow, as indicated in “b”, but rather be integrated into current processes without being a hindrance. Obtaining oversight requires resources and attaining stakeholder support can be time-consuming. Furthermore, the belief that predictive models are more effective in large healthcare organizations is a myth, as indicated in answer “a” (2). The effectiveness of a predictive model is based on the volume of historical data collected by a hospital, not the size of the institution. Answer “d” is also incorrect since validation and guidelines for predictive models have been published in literature and advised by different analytical bodies (4).
References:
1. Michelson D. Becker’s Hospital Review: The Health IT & CIO Report [Internet]. Las Vegas: Dan Michelson. 2018 March -[cited 2018 Mar 25]. Available from: https://www.beckershospitalreview.com/healthcare-information-technology/the-no-1-takeaway-from-himss-2018-amazon-and-apple-and-uber-oh-my.html
2. Amarasingham R, Patzer, RE, Huesch M, Nguyen Q, Xie B, Implementing Electronic Health Care Predictive Analytics: Considerations and Challenges. Health Affairs. 2014; 33(7):1148-1154
3. Bradley P. Becker’s Hospital Review: Health Information Technology [Internet]. ZirMed: Paul Bradley. 2018 January -[cited 2018 Mar 25]. Available from: https://www.beckershospitalreview.com/healthcare-information-technology/4-misconceptions-about-predictive-modeling-in-healthcare.html
4. Collins GS, Reitsma, JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD Statement. 2015; 162(1):55-64
S13-2: The term “black-box medicine” is one that describes a shortcut to answering complex questions without fully understanding the problems or solutions (1). This shortcut most often occurs to prevent the dissemination of proprietary algorithms in predictive models. With more hospitals utilizing internally and externally developed predictive models that follow “black box medicine,” what guidelines should end-users follow when using these types of prediction models?
a. End-users should not endeavor to understand the algorithm due to clinical liability and patient privacy issues
b. End-users should override or appeal recommendations made by predictive models when they have sound reason
195 | P a g e
c. End-users should adapt and accept all outcomes made by predictive models that are scientifically valid.
d. End-users do not need to be educated on the possible outcomes presented by the predictive model since they are the subject matter experts.
Answer: b. End-users should override or appeal recommendations made by predictive models when they have sound reason
Explanation: The best answer is “b” since end-users who are often physicians should feel comfortable in overriding decisions recommended by the model. No model is perfect. Even the best models make inaccurate predictions due to data entry errors, never seen before data patterns, or missing important information not captured in the data (3). Response “a” negates the important concept that comprehension of a model can only come with transparency(2). So although patient data are kept hidden in the “black box” end-users like physicians will need to understand the model to make informed decisions on patient care. Response “c” is untrue since it asks end-users to blindly follow predictive model recommendations, which is unreasonable. While these models are formulated with immense amount of patient data, the number of use cases to test validity are often minimal (4). HealthCatalyst warns healthcare organizations from buying into the “hype” and vendors that oversell their product (4). Response “d” is also incorrect as it is important to equip physicians with the proper training to understand the predictive model so that they may act appropriate when they are faced with incorrect solutions, assumptions, and other issues (4).
References:
1. Ford RA, Price II WN, Privacy and Accountability in Black-Box Medicine. Michigan Telecommunications and Technology Law Review. 2016; 23(1):1-43
2. PR Newswire [Internet]. Tampa: PR Newswire Association, LLC; c2018 [cited 2018 Mar 20]. Providers to Adopt IT Outsourcing Solutions in 2016 as More Hospitals and Physician Practices Slide Deeper Into Financial Uncertainty, Black Book Survey; [1 screen]. Available from: https://www.prnewswire.com/news-releases/providers-to-adopt-it-outsourcing-solutions-in-2016-as-more-hospitals-and-physician-practices-slide-deeper-into-financial-uncertainty-black-book-survey-300183692.html
3. Cohen, IG, Amarasingham R, Shah A, Xie B, Lo B. The Legal and Ethical Concerns that Arise from Using Complex Predictive Analytics in Healthcare. Health Affairs. 2014; 33(7):1139-1147
4. Amarasingham R, Patzer, RE, Huesch M, Nguyen Q, Xie B, Implementing Electronic Health Care Predictive Analytics: Considerations and Challenges. Health Affairs. 2014; 33(7):1148-1154
196 | P a g e
Using a Validated Predictive Analytics Model for Early Detection of Patients with Sepsis
--Gordon Moyer S13-3: You've created a predictive analytics model for sepsis. In your validation of the model, you calculate a C-statistic of 0.5. How do you interpret this result?
A. The model is no better at predicting which patients will and will not develop sepsis, compared with chance alone.
B. The model is good at predicting which patients will and will not develop sepsis.
C. The model is strong at predicting which patients will and will not develop sepsis.
D. The model always predicts which patients will and will not develop sepsis.
E. Based on this predictive model, 50% of patients will develop sepsis and 50% will not.
Answer: A. The model is no better at predicting which patients will and will not develop sepsis, compared with chance alone.
Explanation: C-statistic is a measure of "goodness of fit" for a logistic regression model. It measures the predictive accuracy of such models.
C-statistic is measured on a scale from 0 to 1. A score of 0.5 means that the model is no better at predicting an outcome, compared to random chance. A score of 1 means that the model perfectly predicts patients who will attain a certain outcome from those who will not.
A C-statistic of 0.7 generally indicates a good predictive model. A C-statistic greater than 0.8 generally indicates a strong predictive model.
(A C-statistic less than 0.5 indicates a very poor predictive model, worse than chance)
Reference: http://www.statisticshowto.com/c-statistic/ S13-4: You've created a predictive analytics model for sepsis. In your validation of the model, you calculate a C-statistic of 0.85. How do you interpret this result?
A. The model is no better at predicting which patients will and will not develop sepsis, compared with chance alone.
B. The model is good at predicting which patients will and will not develop sepsis.
C. The model is strong at predicting which patients will and will not develop sepsis.
D. The model always predicts which patients will and will not develop sepsis.
E. Based on this predictive model, 85% of patients will develop sepsis.
197 | P a g e
Answer: C. The model is strong at predicting which patients will and will not develop sepsis..
Explanation: C-statistic is a measure of "goodness of fit" for a logistic regression model. It measures the predictive accuracy of such models.
C-statistic is measured on a scale from 0 to 1. A score of 0.5 means that the model is no better at predicting an outcome, compared to random chance. A score of 1 means that the model perfectly predicts patients who will attain a certain outcome from those who will not.
A C-statistic of 0.7 generally indicates a good predictive model. A C-statistic greater than 0.8 generally indicates a strong predictive model.
(A C-statistic less than 0.5 indicates a very poor predictive model, worse than chance)
References: http://www.statisticshowto.com/c-statistic/
Progression of a Quality Scorecard Leveraging Informatics Tools
--Mark Connolly
S13-5: Which of the following is a key quality reporting best practice?
A. Excessive consultation with subject-matter experts
B. Develop a standard validation process to ensure quality and consistency
C. Provide access to underlying data to all report users to promote transparency
D. Do not release measure definitions to minimize the risk of the Hawthorne effect
Answer: B. Develop a standard validation process to ensure quality and consistency
Explanation: Consultation with subject-matter experts, having a standard validation process, and releasing measure definitions are best practices that provide trust in the quality of data reported. Considerations of HIPAA and patient privacy should be made prior to providing complete access. Providing access to all report users when reporting on healthcare quality and other related data introduces risk.
References: Denney MJ, Long DM, Armistead MG , et al. Validating the extract, transform, load process used to populate a large clinical research database. Int J Med Inform. 2016 Oct;94:271-4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5556907/pdf/nihms808944.pdf
https://hbr.org/2017/03/when-clinicians-know-theyre-being-watched-patients-fare-better
https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/minimum-necessary-requirement/index.html
198 | P a g e
S13-6: What is the role of subject-matter experts during metric development?
A. Oversee data capture and availability
B. Ensure value of metric to warrant resources for development
B. Taking all validation responsibilities
C. Develop plan that aligns with organizational goals
D. Secure necessary resources for developing metrics
Answer: B. Ensure value of metric to warrant resources for development
Explanation: One key role of subject-matter experts (SMEs) is to evaluate the value of metrics and whether resources should be devoted to their developemnt. SMEs should not be used or relied on for complete validation purposes but can be used in addition to other validation processes. They also ensure that the resources required to develop a metric will return value to the organization, that there will be utilization of the metric to drive improvement efforts and can act as a resource to navigate nuances during development. Data custodians oversee data capture and availability, and business sponsors are responsible for developing the plan and securing the necessary resources.
References: Denney MJ, Long DM, Armistead MG , et al. Validating the extract, transform, load process used to populate a large clinical research database. Int J Med Inform. 2016 Oct;94:271-4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5556907/pdf/nihms808944.pdf
https://hbr.org/2017/03/when-clinicians-know-theyre-being-watched-patients-fare-better
https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/minimum-necessary-requirement/index.html
S14: Presentations - Infrastructure and Frameworks for Successful Implementations
Application of an Evidence-Based Adoption Framework: Practical Implementation Tools for Clinical Operations
--Blake Lesselroth
S14-1: You have been asked by your leadership to improve and standardize medication reconciliation in an ambulatory clinic. In response, your team has developed electronic health record features to improve medication history collection, documentation of reconciliation, and patient counseling. When rolling out the new technology, which of the following approaches is most important about improving staff adoption?
199 | P a g e
A. A robust message campaign (e.g., educational materials, posters) is a critical ingredient in an implementation bundle.
B. Software should include patient-facing affordances that improve patient reliability including medication images, plain language, and simple response controls.
C. The implementation team should pay attention to the socio-technical fit of the product, including workflow compatibility and implementation climate.
D. Executive sponsorship and clinical champions are necessary and sufficient for promoting adoption in large practices, managed care organizations, and public systems.
Answer: C. The implementation team should pay attention to the socio-technical fit of the product, including workflow compatibility and implementation climate.
Explanation: The present work and background literature suggest workflow and organizational climate should be addressed during an implementation to improve technology adoption. Although educational materials can be helpful, they are relatively perishable and have a less durable effect on behavior. Other strategies, including at-the-elbow support, tend to have a greater impact. This work did not show that patient-facing technologies either improved the quality of data or influenced staff adoption. Certainly, executive sponsorship and clinical champions are important to an initiative. However, sponsorship alone is rarely sufficient to ensure adoption. This work showed that a practice facilitation strategy can address lack of leadership messaging.
Reference: Holahan PJ, Lesselroth BJ, Adams K, Wang K, Church V. Beyond technology acceptance to effective technology use: a parsimonious and actionable model. Journal of the American Medical Informatics Association. 2015;22(3):718-729.
S14-2: You have recently implemented new electronic health record features designed to improve the reliability of medication reconciliation in clinic. Nurses are expected to use new software to collect and document patient history. Physicians are expected to respond to an automated reminder confirming medications have been reconciled. Upon review of the monthly performance dashboards reports, you realize that nurse charting has improved but physician documentation remains unchanged. What would be the most appropriate next step to improve physician documentation?
A. Add a new hard-stop or modal alert that requires physicians to complete documentation before proceeding.
B. Gather observational or time-motion data to better understand physician workflow and identify potential barriers to reminder completion.
C. Consolidate the history collection and reconciliation tasks so that nurses can complete all documentation concurrently.
D. Meet with department leadership and script out physician performance plans that include reconciliation as a measure of effectiveness.
200 | P a g e
Answer: B. Gather observational or time-motion data to better understand physician workflow and identify potential barriers to reminder completion.
Explanation: It is critical to understand clinician workflow and health record use strategies to understand barriers to adoption and use. Clinics are complex adaptive systems characterized by non-linear workflows, process interruptions, and unanticipated or emergent properties. It is plausible that the interface affordances are either hidden or inaccessible to clinicians at the point-of-care. Adding a hard-stop – particularly if the affordances are problematic – could create an unmanageable bottleneck in throughput. Delegating the tasks and documentation to nurses is not an effective workload leveling strategy and may generate resentment. Aligning quality improvement objectives with clinician performance can be an effective change management strategy, but not before attempting to understand the barriers to technology use.
Reference: Holden RJ, Karsh BT. The Technology Acceptance Model: its past and its future in health care. Journal of Biomedical Informatics. 2010;43(1):159-172.
Creating the Data Infrastructure to Support Mobile Health and Telehealth Initiatives
--Christopher Hollweg S14-3: Which of the following best describes an error induced by automation bias?
A) Dr. Smith is alerted by an automatic EHR message that his patient may be developing severe sepsis, along with a button to order intravenous (IV) fluids. However, this patient also has heart failure, which may worsen with IV fluids. Dr. Smith assumes that the alert is appropriate and orders the fluids. The patient later develops fluid overload and is transferred to the intensive care unit.
B) Dr. Smith is about to order antibiotics for his patient with a urinary tract infection. He chooses ciprofloxacin because that is what he is used to using for his patients, but did not notice that this patient's previous urine culture grew organisms resistant to ciprofloxacin
C) Dr. Smith needs to discontinue the medication metoprolol for a patient in the EHR. However, he mistakenly discontinues the medication "metoclopramide," which was located right next to metoprolol in the EHR and looked very similar.
D) Dr. Smith is about to admit his 10th patient on a busy call day. When placing his admission orders, he mistakenly signs them in an open chart in the EHR for another patient whom he had seen earlier.
Answer: The correct answer is A. Explanation: Automation bias refers to the propensity for humans to be over-reliant on suggestions from automated decision support systems and ignore contradictory evidence. In this case, the patient, who has severe congestive heart failure, should not have been given 30cc/kg of fluid, but Dr. Smith relied on the automated decision support recommendation and gave it anyway, which resulted in an adverse event.
201 | P a g e
Incorrect answers:
B) This error is an example of the effects of availability heuristic.
C) This error resulted from a difficult to read EHR interface.
D) This error is an example of a wrong patient order that can occur when multiple patient charts are open in the EHR.
References: Reference: Goddard, K., Roudsari, A. & Wyatt, J. C. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J. Am. Med. Informatics Assoc. 19, 121–127 (2012). S14-4: Which of the following is one of the largest challenges commonly addressed in clinical risk prediction model studies involving EHR data?
A. Evaluation metrics
B. Informative presence
C. Lead time bias
D. Missing data
Answer: D. Missing data Explanation: One of largest challenges commonly addressed in EHR based studies is missing data. Informative presence, which many studies don’t address, is based off the principle that EHR data is made up of sicker people. Evaluation metrics are useful for comparing risk prediction models across different diseases, i.e. c-statistic. Lead time bias refers to the length of time between the detection of a disease and its clinical presentation and diagnosis. References: Benjamin A Goldstein, Ann Marie Navar, Michael J Pencina, John P A Ioannidis; Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc 2017; 24 (1): 198-208. doi: 10.1093/jamia/ocw042 S14-5: Which of the following is a recommended goal when developing and implementing an electronic decision support tool for medication reconciliation?
A. Anticipating the users’ needs
B. Conducting usability testing in different settings
C. Focusing on thoroughness rather than efficiency
D. Redesigning the existing clinical workflow to match the tool
Answer: B. Conducting usability testing Explanation: Recommendations concerning the implementation of electronic tools to support medication reconciliation (eMedRec) include conducting usability testing and testing the software
202 | P a g e
(reliability + interoperability) in selected and disparate settings, focusing on efficiency in addition to clinical improvements, and adapting the tool workflow to users’ habits. It is recommended to not anticipate the users’ needs during the development of the electronic tools to support medication reconciliation. References: Sophie Marien, Bruno Krug, Anne Spinewine; Electronic tools to support medication reconciliation: a systematic review. J Am Med Inform Assoc 2017; 24 (1): 227-240. doi: 10.1093/jamia/ocw068
Use Cases for a WiFi Connected Internet of Things Button in an Academic Medical Center
--Haipeng Zhang
S14-6: The BWH Dash button is an example of using Internet of Things (IoT) devices. IoT is best defined as: A. A group of related ontologies within a larger database housed on a public facing network. B. A network of physical objects that contain embedded technology to communicate and sense or interact with their internal states or the external environment. C. Wearable technologies that have the capability to sense biometric data from the user and communicate with one another and across a larger network. D. Miniaturized single function computers that communicate via physical connections to one another. E. Low cost, single board computers that can be programmed to perform various functions. Answer: B. A network of physical objects that contain embedded technology to communicate and sense or interact with their internal states or the external environment. Explanation: Answer B best encapsulates the broad definition of IoT devices. Choices C and E are subsegments of IoT technology but not the best definition. Choices A and D are incorrect. Reference: Gartner Internet of Things, defined: https://www.gartner.com/it-glossary/internet-of-things/
203 | P a g e
S14-7: The BWH Dash button uses AWS IoT 1-Click technology with a custom dashboard that allows for easy provisioning and programming of BWH Dash buttons. How many functions can BWH Dash buttons be programmed for?
A. 1 function - single press
B. 2 functions - single press, long press
C. 3 functions - single press, double press, long press
D. 4 functions - single press, double press, triple press, long press
E. 5 functions - single press, double press, triple press, long press, long press and hold
Answer: C. 3 functions - single press, double press, long press Explanation: The AWS IoT Button technology can be programmed to react to three different triggers: a single press, a double press, and a long press. Reference: https://aws.amazon.com/iotbutton/
S15: Panel - New Approaches for Improving CPOE Safety: Indications-based Prescribing and CancelRx
--Gordon Schiff, Neri Pamela, Christine Cheng, Samantha Pitts
S15-1: Physicians have long been urged to place the indication for the medication on the prescription for drugs they are ordering. With paper prescriptions, physician acceptance has been poor, but with near universal adoption of electronic ordering (CPOE) there are renewed opportunities to promote utilization of this feature. Integrating indications into prescribing could pave the way for a safer, more complete continuum of care for the patient and save time during prescribing and related tasks. Which of the following is another significant benefit of indications-based prescribing?
A. The indication will always be put on the medication label
B. Drug costs will decrease when indications are included
C. Patients will know the reason why they are taking each of their medications
D. Brand name drugs will be selected more often
Answer: C. Patients will know the reason why they are taking each of their medications
Explanation: One of the goals of an indications-based prescribing system is to increase patient adherence and education. With an indications-driven process, patients will be able to discuss their indications with their providers and have the indication included on the medication label if they choose. Prescribing by indication will allow patients to experience a safer continuum of care as different
204 | P a g e
members of their care team will understand the reason behind each of their medications and it will allow patient to better discuss their medications.
While including indications on the medication label is important, patients should also have the option to opt-out of including the indication if they would prefer the reason kept private, such as in cases of mental health or STD medications. The goal would not be for a universal inclusion of indications on the label, but instead for a system where indications would be standard with the option for patients to suppress the indication so they have flexibility (A). However, by redesigning the workflow to include indications and making it trivial to include indications on labels, most patients would be able to reference their medications to understand why they are taking each one.
Finally, indications-based prescribing is designed around the idea of safer and more efficient prescribing, not around drug costs and brands. The goal is that a drug would be selected because it treats the indication while incorporating patient information (like allergies), guidelines, insurance and other factors. While insurance and formulary issues are part of the process, the goal is not focused on lowering drug costs or choosing the cheapest drug (B). In addition, the system should not have preference for name brands, but instead choose based on whatever drug is the most appropriate for the patient (D).
Reference: Schiff GD, Seoane-Vazquez E, Wright A. Incorporating indications into medication ordering — time to enter the age of reason. N Engl J Med. 2016 Jul28;375(4):306–9.
S15-2: While including the indication on the prescription is not a new idea, many providers do not use this feature on a consistent basis. It has not become standard practice partially due to poor CPOE design. To successfully incorporate indication into the prescribing workflow, efforts need to focus on reducing mouse clicks, providing help at the point of care, and minimizing alert fatigue while supporting physicians’ autonomy and flexibility. Which of the following design choice best explains how a prototype successfully incorporates indications by allowing for physician autonomy while providing useful guidance?
A. Allow providers to search for a drug only and do not require an indication
B. Allow providers to start with a drug or a problem/indication and see drug suggestions
C. Allow providers to search for a drug only, enter an indication and see alternative drug suggestions
D. Allow providers to search for a problem only and see drug suggestions
Answer: B. Allow providers to start with a drug or a problem/indication and see drug suggestions
Explanation: For recommendations to be most useful, the indication should be required first which allows the drug choices to be tailored to the patient and indication chosen. However, given that many physicians order certain drugs very frequently, allowing the physician to also search for a drug provides them the clinical autonomy and flexibility they might desire. By allowing physicians to only start with a drug and not a problem, we are missing an opportunity to increase provider efficiency and minimize
205 | P a g e
cognitive load during their clinical workflow. If we allow providers to only start with a problem, we have limited the systems flexibility of use and removed some of their clinical autonomy.
Reference: Improving HIT Prescribing Safety by Incorporating Indications Into Computerized Prescriber Order Entry. Content last reviewed March 2017. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/chain/research-tools/featured-certs/improving-hit-prescribing-safety.html
S15-3: Associating indications with a drug order may help prevent medication errors associated with look-alike-sound-alike (LASA) drugs. Which of the following best describes the utility of associating high-level indications concepts with drug pairs on the Institute for Safe Medication Practices’ list of commonly confused drug names?
a. The majority of drug pairs that were comprised of antineoplastic drugs could be differentiated using high-level indication concepts.
b. The majority of drug pairs that were comprised of antibacterial drugs could be differentiated using high-level indication concepts.
c. More than half of all the evaluated drug pairs could be differentiated using high-level indication concepts.
d. Only 20% of all evaluated drug pairs could be differentiated using high-level indication concepts.
Answer: c. More than half of all the evaluated drug pairs could be differentiated using high-level indication concepts.
Explanation: The presented study used high-level indications concepts from a proprietary knowledge base and associated them with drugs on the Institute for Safe Medication Practices’ list of commonly confused drugs.Of the evaluated drug pairs, 60% had no overlap in high-level indications. Twenty percent of the evaluated drug pairs had a complete overlap in high-level indications, and most of these were drug pairs that were comprised of antineoplastic drugs, anti-infective drugs, anti-diabetic drugs or opioid analgesics.
Reference: Seone-Vazquez E, Rodriguez-Monguio R, Alqahtani S, Schiff G. Exploring the potential for using drug indications to prevent look-alike and sound-alike drug errors. Expert Opinion on Drug Safety. 2017;16:10:1103-1009.
S15-4: The National Council for Prescription Drug Programs’ SCRIPT standard supports electronic functionality for sending prescription cancellations from electronic health records (EHRs) to pharmacies, known as CancelRx. However, adoption of this functionality has been limited. Which of the following presented the greatest challenge to pilot, rather than enterprise-wide,implementation?
A. Impact on pharmacy workflow B. Impact on prescriber workflow C. Restricted functionality of prescription cancellation in pilot implementation D. Buy-in from key stakeholders
206 | P a g e
Answer: C. Restricted functionality of prescription cancellation in pilot implementation
Explanation: Pilot implementation at Johns Hopkins was planned to systematically understand the expected performance of the system, the implications for workflows of key stakeholders, and the impact on existing medication safety risks and risks introduced by implementation. However, configuration in the Johns Hopkins EHR for use by a cohort of pilot users restricts the functionality of prescription cancellation, as only prescriptions written by pilot users can be electronically cancelled.
Reference: Fischer S, Rose A. Responsible e-Prescribing Needs e-Discontinuation. JAMA. 2017;317(5):469–470. doi:10.1001/jama.2016.19908
S16: Presentations - Care Coordination and Communication
Improving care coordination for children with autism spectrum disorder: defining care team members information needs
--Julia O’Rourke S16-1: Autism spectrum disorder (ASD) is a neurodevelopmental disorder diagnosed in childhood that affects individuals throughout their lifespan. (1). As many as one in 50 children in the U.S. between the ages of six and 17 years have a diagnosis of ASD (2), which reflects a 200-fold increase compared to the estimates of four cases per 10,000 in 1966 (3). Individuals with ASD experience lifelong difficulties in social interaction, communication skills, behavior, and learning. Care for children with ASD is complex, involving on average seven different types of interventions, programs, and therapies (4), including state organizations, local schools, medical providers, in-home behavioral therapy, and rehabilitation services, such as physical and occupational therapies. Children with ASD also have increased medical needs due to comorbid mental and medical conditions including sleep problems, seizures, gastrointestinal problems, and maladaptive behaviors (5, 6). Current treatment delivery model for children with ASD is fragmented with treatment team members delivering care at different entities without any systematic collaboration or communication. To better demonstrate the roles or different care team members working with a child with ASD, we describe treatment of a hypothetical school age boy John, who attends a public elementary school. In school John has several care team members assisting him. His teacher and school aid are working with him daily in the classroom; his occupational therapist focuses on sensory difficulties during a weekly 45- minute session. His physical therapist helps with building strength and endurance during another weekly 45-minute session. His speech therapist sees him three times per week and helps with articulation difficulties. Finally, the school’s behavioral therapist sees John two times per week to teach pragmatic and social skills. At school, John has an Individualized Educational Plan (IEP) plan. John’s parents also bring him to a private speech therapist and occupational therapist for weekly sessions. In addition, John sees a psychologist for weekly sessions outside of school, and another behavioral therapist who helps with some of the troubling behaviors at home. John is also followed by a psychiatrist, who monitors his psychotropic medications. John sees a neurologist for sleep and seizure
207 | P a g e
problems and gastroenterologist for constipation. In this example, all care team members work with John to help him and his family to achieve a better quality of life and help him learn skills to live as an independent and productive adult. John has caregivers at school, at home, in private offices, and in several medical facilities, but they work independently and do not communicate across their practices and locations. These treatments are fragmented and it is not clear whether the therapies being delivered are the optimal choice for John and his family. What is the most likely care coordination scenario?
A. John has a Medical Home at his pediatrician’s office, where a case manager (social worker or nurse) coordinates his medical care and services with other care team members
B. John has an Integrated Medical home at his psychiatrist’s office, where a case manager coordinates his mental health care and services with other care team members
C. John has no care coordination activities
D. John’s mother performs all care coordination activities. To do this, she has had to reduce her hours and now works part-time. Answer: D. John’s mother performs all care coordination activities. To do this, she has had to reduce her hours and now works part-time. Explanation: As described in the example of John’s fragmented care, in the current care delivery model for ASD, there is no central person with expertise in ASD who could provide oversight of John’s care and tailor the most appropriate treatment strategy for him. A half century ago, the American Academy of Pediatrics developed a medical home care model (7), designed to connect children with special healthcare needs and their families with appropriate resources to improve communication and coordination between care providers, schools and the community for pediatric patients (8). Although medical homes improve many aspects of pediatric clinical care (8), only 27% of children with ASD have a medical home (9). This model does not fit care needs for children with ASD, because pediatricians lack expertise in ASD and the time to coordinate care. Recent research shows that pediatricians lack knowledge about current interventions and community resources for ASD (10). Furthermore, pediatricians report a need for expert guidance in care provision for children with ASD. “I feel like I don’t know if they’re doing the right thing because they’re all so different, and I feel like I’m driving a team of horses and one’s going this way and the others going this way.” Pediatricians found case managers to be useful for care coordination, but because so few ASD patients were in each practice, they suggested moving towards community-based or insurance based case managers (11). We believe that Specialty / Integrated Medical Home is a better model for children with ASD (12, 13). In our work, we interviewed 10 parents and every parent said that they perform duties of care coordinators, and all 8 mothers interviewed either worked part-time or did not work. This is consistent with the national statistics where mothers of children with ASD earn 56% ($14 755) less than the mothers of children with no health limitation. They are 6% less likely to be employed and work 7 hours less per week, on average, than mothers of children with no health limitation (14, 15)
208 | P a g e
Answer A is incorrect because only 27% of children with ASD have a medical home (9). Answer B is incorrect because Integrated Medical Home is a more advanced model of Medical Home which integrates mental health and primary care (16). References: 1. Simonoff E, Pickles A, Charman T, Chandler S, Loucas T, Baird G. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. J Am Acad Child Adolesc Psychiatry 2008;47:921-9. 2. Center for Disease Control. Changes in prevalence of parent-reported autism spectrum disorder in school-aged U.S. children: 2007 to 2011–2012. 2013. 3. Lotter V. Epidimiology of autistic conditions in young children. Social Psychiatry 1966;1:124-35. 4. Green VA, Pituch KA, Itchon J, Choi A, O'Reilly M, Sigafoos J. Internet survey of treatments used by parents of children with autism. Res Dev Disabil 2006;27:70-84. 5. Gurney JG, McPheeters ML, Davis MM. Parental report of health conditions and health care use among children with and without autism: National Survey of Children's Health. Archives of pediatrics & adolescent medicine 2006;160:825-30. 6. Brachlow AE, Ness KK, McPheeters ML, Gurney JG. Comparison of indicators for a primary care medical home between children with autism or asthma and other special health care needs: National Survey of Children's Health. Archives of pediatrics & adolescent medicine 2007;161:399-405. 7. Sia C, Tonniges TF, Osterhus E, Taba S. History of the medical home concept. Pediatrics 2004;113:1473-8. 8. Homer CJ, Klatka K, Romm D, et al. A review of the evidence for the medical home for children with special health care needs. Pediatrics 2008;122:922-37. 9. Kogan MD, Strickland BB, Blumberg SJ, Singh GK, Perrin JM, van Dyck PC. A national profile of the health care experiences and family impact of autism spectrum disorder among children in the United States, 2005-2006. Pediatrics 2008;122:e1149-58. 10. Nolan R, Walker T, Hanson JL, Friedman S. Developmental behavioral pediatrician support of the medical home for children with autism spectrum disorders. J Dev Behav Pediatr 2016;37:687-93. 11. CMS expands Medicare payment for behavioral health services. 2017. (Accessed 3/7/2017, at https://mcdonaldhopkins.com/Insights/Alerts/2016/11/09/CMS-expands-Medicare-payment-for-behavioral-health-services.) 12. Fueyo M, Caldwell T, Mattern SB, Zahid J, Foley T. The Health Home: A Service Delivery Model for Autism and Intellectual Disability. Psychiatr Serv 2015;66:1135-7. 13. Asarnow JR, Rozenman M, Wiblin J, Zeltzer L. Integrated Medical-Behavioral Care Compared With Usual Primary Care for Child and Adolescent Behavioral Health: A Meta-analysis. JAMA Pediatr 2015;169:929-37. 14. Cidav Z, Marcus SC, Mandell DS. Implications of childhood autism for parental employment and earnings. Pediatrics 2012;129:617-23. 15. Hodgetts S, McConnell D, Zwaigenbaum L, Nicholas D. The impact of autism services on mothers' occupational balance and participation. OTJR (Thorofare N J) 2014;34:81-92. 16. Tyler ET, Hulkower RL, Kaminski JW. Behavioral Health Integration in Pediatric Primary Care: Considerations and Opportunities for Policymakers, Planners, and Providers: Milbank Memorial Fund; 2017.
209 | P a g e
S16-2: Care coordination integrates services across settings and supports interfacing among multiple care systems and care providers, including the following: medical, social and behavioral professionals; the educational system; home care agencies; allied health professionals; and families (1). In a recent opinion piece titled “Health Information Technology and Care Coordination: The Next Big Opportunity for Informatics?”, Dr. David Bates described the care coordination needs for complex patients who are cared for by a group of medical providers: “This entire group needs to be able to communicate better than they do today – to see the care plan, to message each other, and to have synchronous and asynchronous conversations about goals, and progress and lack thereof” (2). Dr. Halamka further highlighted the need for care coordination infrastructure in 2017 NEJM: “If we want patients to be engaged to help reduce the burden of care coordination, care plan tracking, and communication, we need modern tools that enable patients to interact with their providers using devices and workflows that are already part of their daily lives. Many companies are now offering such tools” (3). However, care coordination tools will be useful within a collaborative care model that supports care coordination. The collaborative care model relies on information systems and case managers to link primary care providers, patients, and mental health specialists (4). The collaborative care model has shown to be effective in over 80 studies for in adult population for the treatment of depressive and anxiety disorders (4, 5). Collaborative care also improves outcomes for children and adolescents with mental health problems (6). The collaborative care model is based on the chronic care model developed by Wagner and colleagues nearly two decades ago to guide health services towards better management of chronic diseases (7). According to this model, improved patient outcomes result from the interactions between an activated patient and a proactive healthcare team. Factors that activate patients include support for disease self-management and enabling community resources. Factors that activate healthcare providers include adequate information systems, decision support, and delivery system design to support longitudinal care as opposed to episodic care. What is the greatest barrier in implementing a collaborative care model for children with ASD?
A. Severe shortage of child psychiatrists in the United States
B. No adequate payment mechanism for supporting care coordination
C. Privacy laws (The Health Insurance Portability and Accountability Act of 1996 (HIPAA) and The Family Educational Rights and Privacy Act(FERPA))
D. Difficulties integrating with EHR systems
Answer: B. No adequate payment mechanism for supporting care coordination
210 | P a g e
Explanation: Answer A: Although there is severe shortage of child psychiatrists in the United States (8), it is likely that increasing number of psychiatrists will not help with implementing a collaborative care model. Answer B: Currently, there is no payment mechanism (using CPT codes) to support case managers. Although, CPT codes 99487-99489 have recently been published, in our experience very few payers support these codes and the ones that do, require medical care coordination by a nurse. Children with ASD need care coordination across medical, educational and community providers, which is typically provided by a social worker. Answer C: With adequate consent forms, signed by parents and legal guardians, these laws can be followed. Answer D: Although difficulties integrating with EHR systems exist, a qualified case manager can manually extract and integrate information across the care team. References: 1. Turchi RM, Antonelli RC. Patient- and family-centered care coordination: a framework for integrating care for children and youth across multiple systems. Pediatrics 2014;133:e1451-60. 2. Bates DW. Health Information Technology and Care Coordination: The Next Big Opportunity for Informatics? Yearbook of Medical Informatics 2015;10:11-4. 3. Halamka JD, Tripathi M. The HITECH Era in Retrospect. N Engl J Med 2017;377:907-9. 4. Thota AB, Sipe TA, Byard GJ, et al. Collaborative care to improve the management of depressive disorders: a community guide systematic review and meta-analysis. American journal of preventive medicine 2012;42:525-38. 5. Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev 2012;10:CD006525. 6. Asarnow JR, Rozenman M, Wiblin J, Zeltzer L. Integrated Medical-Behavioral Care Compared With Usual Primary Care for Child and Adolescent Behavioral Health: A Meta-analysis. JAMA Pediatr 2015;169:929-37. 7. Wagner EH. Chronic disease management: what will it take to improve care for chronic illness? Effective clinical practice : ECP 1998;1:2-4. 8. Practicing Child and Adolescent Psychiatrists by State 2015. (Accessed 03/26/2018, at https://www.aacap.org/aacap/Advocacy/Federal_and_State_Initiatives/Workforce_Maps/Home.aspx.)
Challenges to Care Coordination for Children with Special Health Care Needs: Potential Solutions
--Pooja Paode
S16-3: Current efforts to improve care for children with special health care needs (CSHCN) are highly medicalized and designed through the lens of providers, despite the fact that parents are responsible for the vast majority of the care coordination for CSHCN. In response to this issue, we created a multi-sector, parent-centered Stakeholder Coalition dedicated to identifying solutions. The Coalition identified communication, community collaboration, family engagement/advocacy, and systems issues/technology as key areas for improvement for the care coordination of CSHCN. How should we consider the interaction between systems issues/technology and the other three areas? We should…
211 | P a g e
A. come up with solutions for the four areas separately, then evaluate how they interact. B. prioritize the development of technological solutions, such as applications, in each of the three other areas. C. consider solutions that will address gaps in each of the three areas, then identify how technology and systems redesign can enable those solutions. D. prioritize development on technology and systems changes that simultaneously bring together all three areas: communication, community collaboration, and family engagement. Answer: C. consider solutions that will address gaps in each of the three areas, then identify how technology and systems redesign can enable those solutions. Explanation: While technology can serve to enhance solutions to care coordination for CSHCN, we should be cautious of excessively fixating on or prioritizing technological solutions to care coordination. It is important to first identify solutions important to children, parents, and providers, and then consider how technology can enhance or enable those solutions to make them more effective or feasible. It would be ineffective to come up with technological solutions or enact systems redesign independently of the other three priority areas. Despite this, many technological solutions to health systems are created in silos and fail to adequately include parent preferences related to communication, collaboration, and engagement. Reference: Turchi RM, Antonelli RC, Norwood KW, Adams RC, Brei TJ, Burke RT, Davis BE, Friedman SL, Houtrow AJ, Kuo DZ, Levy SE. Patient-and family-centered care coordination: A framework for integrating care for children and youth across multiple systems. Pediatrics. 2014 May 1;133(5):e1451-60. S16-4: Parents of children with special health care needs (CSHCN) often simultaneously operate as care coordinators, caretakers, and key decision-makers for their children. They experience a lack of meaningful information exchange surrounding situation management, planning, monitoring, navigating, and resource usage. When does this problem cause the most adverse impact for parents caring for CSHCN? During…
A. provider-parent encounters
B. a variety of “tension points” throughout the child’s lifetime
C. the time that passes between clinic visits
D. searches for external resources to aid in the care of their CSHCN
Answer: B. a variety of “tension points” throughout the child’s lifetime
212 | P a g e
Explanation: Parents of CSHCN have reported feeling vulnerable during specific “tension points” in their children’s lives. Examples of tension points include when parents are first told that their child is likely to have significant developmental problems, when an important person in their child’s care – like a specialist or a case manager leaves, or when a child is transferred from one service or program to another. Parents currently do not have the information or support they need to work through these times of tension. Consequently, parents have reported thinking “I wish someone had told me about this” or “how come no one prepared me for this?” Solutions that promote meaningful information exchange during these key tension points are high-impact target for systems improvement. Reference: Ranade-Kharkar P, Weir C, Norlin C, Collins SA, Scarton LA, Baker GB, Borbolla D, Taliercio V, Del Fiol G. Information needs of physicians, care coordinators, and families to support care coordination of children and youth with special health care needs (CYSHCN). Journal of the American Medical Informatics Association. 2017 Mar 24;24(5):933-41.
Accuracy of Patient Self-Report of 90-Day Post-Discharge Healthcare Utilization and Complications: Relevance to Value Based Care
--Benjamin Rosner
S16-5: University Hospital is a regional center of excellence for total joint arthroplasty (TJA), and as such, it draws patients from a large geographic area. However, post-discharge, most of these patients return to their home area. When complications or readmissions occur, events for which University Hospital bears financial risk under value-based reimbursement models, University Hospital is often unaware. In reading the literature, those at University Hospital leading the post-discharge process improvement efforts have learned that up to 65% of complications and readmissions may occur at other institutions. To get a more complete picture of when and where readmissions are occurring among their patients, the hospital is considering surveying patients directly about readmissions occurring within 90 days of discharge. Assuming an unambiguous readmission question worded in lay language, University Hospital might expect patients to have what degree of accuracy in self-reporting readmission events?
A. Excellent (kappa≥0.8)
B. Very good (kappa 0.60-0.79)
C. Moderate (kappa 0.40-0.59)
D. Fair (kappa 0.20-0.39)
E. Poor (kappa <0.20)
Answer: A. Excellent (kappa≥0.8)
Explanation: The study found that patients were accurate reporters of 90-day hospital readmissions, with a kappa of 0.80, suggesting excellent accuracy.
Reference: Harrold L, Pascal S, Lewis C, O'Keefe R, Pellegrini V, Allison J, Ayers D, Franklin P. Patient Report Improves Posthospital Discharge Event Capture in Total Joint Replacement: A Novel Approach to Capturing All Posthospital Event Data. EGEMS; 2014 Oct 22; 2(1). PMID:25848596
213 | P a g e
S16-6: James is an analyst working for a large healthcare organization trying to understand the incidence of 90-day post-discharge complications. He has done a literature review of papers quantifying the accuracy of patient self-report for the presence of complications using metrics such as agreement and kappa. He believes that he has sufficient information to bring back to leadership about overall accuracy of patient self-report to make an informed decision about the organization's planned outreach to patients on the topic. What should he have done differently to avoid omitting an important element in his assessment?
A. He should have looked for papers using "concordance" as a metric rather than "agreement" and "kappa."
B. He should have limited his search to papers surveying patients at 1 year post-encounter or longer, because recall accuracy improves with time.
C. He failed to consider the importance of also measuring accuracy of self-reporting for the absence of events
D. He should have limited his search to papers in which analysis was limited to patients responding to surveys, and not including analysis of non-responders (non-responder bias).
Answer: C. He failed to consider the importance of also measuring accuracy of self-reporting for the absence of events
Explanation: A is incorrect because agreement and kappa are appropriate, particularly when prevalence event rates are low because kappa does not inflate with low prevalence as does agreement or concordance. B. Is incorrect because shorter times appear to be correlated with more accurate recall. C. is correct because accuracy of self-report should include both the accuracy of the self-report of the presence as well as absence of events. In fact, for low prevalence events, the cohort without an event is much larger than the cohort with an event, and neglecting to capture accuracy of the absence of events neglects the larger cohort of patients. Methodologically, it can be more challenging to determine accuracy around the absence of events, which is why many studies have failed to include it. D. In incorrect. Although non-responder bias analysis can be challenging, it is critical in understanding whether the analysis of the responders can be generalized to the full population receiving surveys.
Reference: Alazzawi S, Bardakos NV, Hadfield SG, Butt U, Beer ZH, Field RE. Patient-reported complications after elective joint replacement surgery: Are they correct? Bone Jt J; 2012 Aug 1;94-B(8):1120-1125. PMID:22844056
214 | P a g e
Thursday, May 10th
S17: Presentations - Decision Support for Clinical Screening
The Power of Two: Rapid Screening and Intervening Combining clinical decision support (CDS) tools and nurse-driven protocols to effectively evaluate patients for high risk conditions and potential interventions at hospital admission
-- John McGreevey
S17-1: All of the following topics would be a good fit for the type of universal screening assessment described in this presentation EXCEPT:
A. Pregnancy status
B. Capturing detailed oncologic history
C. MRSA status
D. Chlorhexidine bathing indications
Answer: B. Capturing detailed oncologic history
Explanation: The criteria for a good candidate for universal screening include applicability to a large population of patients, high impact workflows without a current ideal screening solution, and limited nursing assessment needed. Answers A, C and D meet this threshold while option B does not and would require a provider to gather this history.
S17-2: Effective screening of patients for MRSA status, pregnancy status and to identify indicators for chlorhexidine can be potentiated when clinical decision support is combined with which one of the following?
A. Problem lists
B. Medication reconciliation
C. Pre-selected orders in order sets
D. Nurse-driven protocols and human assessment
Answer: D. Nurse-driven protocols and human assessment
Explanation: By combining clinical decision support tools with nurse-driven protocols and human assessment patients have the potential to be more effectively screened for MRSA status, pregnancy status, and indications for chlorhexidine bathing as described in this presentation.
215 | P a g e
Answer C is incorrect because the pre-selection of orders was described as an ineffective solution in this presentation.
Answers A and B, while important concepts, were not components of the universal screening approach described in this presentation.
Cancer Patient Trial Matching Assisted by An Efficient Automated Pre-Screening Application
--Hung Le
S17-3: Cancer claims the lives of half a million Americans every year. One of the major bottlenecks in the development of anti-cancer drugs is the lack of sufficient patient enrollment in clinical trials. Which of the following will likely happen if the enrollment goal is not met?
A. Delay or even termination of clinical trials, which, in turn, amounts to substantial loss to the pharmaceutical companies
B. The sponsor will ask the principal investigator to refund the investment to recruit patients
C. The sponsor can extend the time frame for enrollment indefinitely until patient enrollment goal has met
D. The investigator will be able to use synthetic patients to add to the number of actual patients to reach the number needed for enrollment
Answer: A. Delay or even termination of clinical trials, which, in turn, amounts to substantial loss to the pharmaceutical companies
Explanation: It is a poorly kept secret in the world of clinical trials that issues with patient recruitment and enrollment are the primary causes for missing clinical trial timelines. For each day a company goes beyond the planned deadline for a clinical trial, it could be lose as much as $600,000 in foregone sales of smaller products and as much as $8 million on blockbuster drugs. Sponsors and CROs are eager to use strategies involving technology to increase the rate of patient enrollment.
The costs associated with patient recruitment delays are not only high, but difficult to anticipate and control. Consistently updated regulations, newly developed media by which to attract patients and investigators, and selecting appropriate service providers for recruitment all play a role in patient recruitment dynamics.
Reference: Industry Standard Research. (2014). The Expanding Web of Clinical Trial Patient Recruitment. Retrieved from http://www.isrreports.com/wp-content/uploads/2014/04/ISR-The-Expanding-Web-of-Clinical-Trial-Patient-Recruitment-Whitepaper.pdf
S17-4: At the department of Investigational Cancer Therapeutics (Clinical Phase I/II program), UT MD Anderson Cancer Center, we have developed an automated patient pre-screening application to
216 | P a g e
significant reduce manual effort to 92% and increase enrollment rate. What eligibility criteria were used in this automated application? (select all that apply)
A. Patient age
B. Tumor type or disease
C. Number of prior therapies
D. Molecular target
E. Performance status
Answer: B. Tumor type or disease D. Molecular target
Explanation: To develop this pre-screening application, we first identified two of the most discerning eligibility criteria in oncology trials: tumor type and molecular target (mutation). Mutation analysis plays an indispensable role in identifying actionable molecular events, predicting responses to targeted therapies and ensuring optimal treatment
Reference: H. Le, J. Zeng, C. Beer, S. Montez, D. Sanders, J. Gong et al ,Cancer Patient Trial Matching Assisted by An Efficient Automated Pre-Screening Application. AMIA 2018 Clinical Informatics Conference, Phoenix, AZ, May 8-10.
StartSmartTM: Decision Support Technology for Prenatal Screening
--Bonnie Gance Cleveland
S17-5: Which of the following best describes the iterative IT development process used in the creation of StartSmart, prenatal screening application, guided by the Davis’s Technology Acceptance model?
a) Review of the literature, focus groups with pregnant women, technology development using Davis Technology Acceptance Model, patient and provider end-user testing, refinement
b) Focus groups with providers & patients guided by the SBIRT Model, prototype development, alpha testing, beta testing
c) Prototype development using guidelines, focus groups, and expert consultation; alpha testing with patients and providers; revisions; beta testing with patient and providers; refinement
Answer: c) Prototype development using guidelines, focus groups, and expert consultation; alpha testing with patients and providers; revisions; beta testing with patient and providers; refinement
Explanation: The iterative development guided by the Davis’s Technology Acceptance Model used in this project included foundational work by our group including reviewing the guidelines, conducting focus groups with providers and patients, interviewing expert clinicians and patients, developing a
217 | P a g e
prototype, alpha testing with patients & provider, refining, beta testing with patient and providers, and refining.
Reference: Belanger E, Bartlett G, Dawes M, Rodriguez C, Hasson-Gidoni I. Examining the evidence of the impact of health information technology in primary care: an argument for participatory research with health professionals and patients. Int J Med Inform. 2012;81(10):654-661.
S17-6: Which one of the following best describes the process for applying the Davis’s Technology Acceptance Model to the assessment of the StartSmart Prenatal Screening Technology?
a) An iterative development of the decision support/patient education application with end users’ input at each level
b) Assessment of usability and ease of use was included in each step of the development
c) Assessment included user input at each phase/ beta testing with patient interviews that addressed acceptability, ease of use, and intention to change; provider surveys regarding acceptability and ease of use.
Answer: c) Assessment included user input at each phase/ beta testing with patient interviews that addressed acceptability, ease of use, and intention to change; provider surveys regarding acceptability and ease of use.
Explanation: The iterative development and assessment guided by the Davis’ Technology Acceptance Model used in this project included expert clinicians and patients, developing a prototype, alpha testing with patients & provider with questions specific to navigation, usability, and ease of use; refining; beta testing with patient and providers regarding usability and ease of use; and further refinement.
Reference: Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 1989;13(3):319-340.
S18: Presentations - Supporting Best Practices
Tracking and Managing Abnormal Mammograms with SMART
--Caitlin Cusack
S18-1: At the Veteran’s Health Administration, the SMART project is being implemented to improve the ability to track clinical management against a single index abnormal mammogram via the use of a “Breast Care Cascade.” Which of the following “opens” a cascade?
218 | P a g e
a. The text of a received mammogram report is entered into the EHR.
b. An order for a mammogram is placed.
c. A woman who is due for a mammogram presents for a preventive care visit.
d. The pathology report for a breast biopsy is entered into the EHR.
Answer: b. An order for a mammogram is placed.
Explanation: A Breast Care Cascade is opened under three circumstances: The BIRAD code of the mammogram is entered into the BIRAD field in the record; the provider enters an outside result into the SMART template; or an order for a mammogram is placed. (a) is incorrect since free-text does not open the cascade. (b) is incorrect since merely being due for a mammogram does not necessarily result in the woman having a mammogram ordered. (d) is incorrect since the Breast Care Cascade tracks care against a single index mammogram. Data regarding a breast biopsy done in the management of an abnormal mammogram would be included in the cascade, but in and of itself would not start a cascade.
Reference: Callen JL, Westbrook JI, Georgiou A, et al. Failure to follow-up test results for ambulatory patients: a systematic review. Journal of general internal medicine. 2012 Oct 1;27(10):1334-48.
S18-2: Ms. Jane Doe presents for a routine preventive care visit to her primary care physician (PCP). She is due for a mammogram that is ordered. When completed the mammogram is recorded as a BIRAD 0: Incomplete. The radiologist discusses the findings with the physician covering for Ms. Doe’s PCP, who orders additional views. The followup views are done the following day and reported as BIRAD 4, suspicious abnormality. Both reports generate SMART alerts, one to the PCP, the second to the covering physician. When the PCP views the initial BIRAD 0 alert, which SMART template is shown?
a. BIRAD 0 SMART Template.
b. BIRAD 2 SMART Template.
c. BIRAD 4 SMART Template.
d. Multiple Tests Please Summarize SMART Template.
Answer: c. BIRAD 4 SMART Template.
Explanation: The linked SMART template defaults to the highest BIRAD result across all alerts and providers. If the follow up images were not completed when the provider opens the alert than (a) the BIRAD 0 SMART Template would be linked. (b) is incorrect since none of the images were coded as a BIRAD 2. If there had been a breast image, such as an ultrasound, without a BIRAD code then (d) Multiple Tests Please Summarize SMART Template would be linked.
219 | P a g e
Murphy DR, Meyer AN, Vaghani V, et.al. Electronic triggers to identify delays in follow-up of mammography: harnessing the power of big data in health care. Journal of the American College of Radiology. 2017 Nov 1.
Integrating Evidence to Inform Lab Test Selection into a Knowledge Management Framework
--Spencer DesAutels
S18-3: An academic medical center makes evidence-based local practice decisions across multiple content domains on a frequent, recurring basis. New external knowledge, e.g., a new guideline or meta-analysis, is shared through various institutional channels and eventually leads to each institutional decision being made or revised. Documenting this underlying knowledge has the potential to promote adherence reuse, while simultaneously establishing a basis for crucial ongoing decision maintenance. To maximize efficiency of documentation, the institution should consider which of the following?
a. Creating a shareable document that provides a running tally of all decisions by a decision-making body.
b. Creating a single record of the underlying knowledge and linking it to a unique record of each decision.
c. Selecting a standardized citation format for each publication type.
d. Requiring all committees to use an approved communication channel for official communications.
Answer: b. Creating a single record of the underlying knowledge and linking it to a unique record of each decision.
Explanation: (b) is the best option because this allows the institution to create and maintain the minimum number of records of the knowledge disseminated across the institution and formally links each decision that draws on the same evidence. While (a) shares many of these benefits, it does not allow the institution to clearly draw links and ensure consistency between the decisions from multiple committees/working groups that draw on the same knowledge. (c) and (d) are unrelated to efficiency of knowledge documentation.
Reference: Shellum JL, Freimuth RR, Peters SG, Nishimura RA, Chaudhry R, Demuth SJ, Knopp AL, Miksch TA, Milliner DS. Knowledge as a Service at the Point of Care. AMIA Annu Symp Proc. 2017 Feb 10;2016:1139-1148. PubMed PMID: 28269911.
S18-4:There are many sources that provide relevant information on lab testing for patient care decisions and analytics. These sources are internal and external, and refer to testing at different scales and for different purposes. Furthermore, sources are constantly emerging, changing, or being deprecated. To make a fully informed decision, e.g., selection of lab tests to perform or lab tests to restrict to certain clinical scenarios, it is often necessary to consult multiple resources concurrently. How can an institution maximize use of these heterogeneous data sources?
220 | P a g e
a. Publish a list of approved resources.
b. Negotiate contracts with vendors to present data in institutionally-approved formats.
c. Invest in mechanisms to bring informational resources to the point of care.
d. Bring all applicable data together in a consistent internal format that references each data source.
Answer: d. Bring all applicable data together in a consistent internal format that references each data source.
Explanation: (d) is the best option because it allows flexibility in integrating new sources as they become available, provides a single access point, and creates a standard representation users can gain familiarity with. (a) allows a user to have confidence in selected sources, but still places a burden on the user to consult each individually. Among other challenges, (b) precludes the existence of emergent knowledge sources, while (c) is not relevant to acknowledging heterogeneity of data sources.
Reference: Lee VS, Kawamoto K, Hess R, Park C, Young J, Hunter C, Johnson S, Gulbransen S, Pelt CE, Horton DJ, Graves KK, Greene TH, Anzai Y, Pendleton RC. Implementation of a Value-Driven Outcomes Program to Identify High Variability in Clinical Costs and Outcomes and Association With Reduced Cost and Improved Quality. JAMA. 2016 Sep 13;316(10):1061-72. doi: 10.1001/jama.2016.12226. PubMed PMID: 27623461.
Cleaning House: EHR Order Set Innovation and Transformation at Penn Medicine
--John McGreevey
S18-5: All of the following contributed to the success of the order set transformation process described in this presentation except:
A) Specialty Lead engagement
B) Legacy Order Set analysis
C) Creation of a modular order set framework
D) Development of a standard order set layout and naming convention
E) Replication of all legacy order sets into the new EHR
Answer: E) Replication of all legacy order sets into the new EHR
Explanation: In the order set transformation process presented, there was a conscious decision to not replicate each and every legacy order set. Rather, there was a decision to carefully assess the need for each legacy order set, in the interest in consolidating order sets and reducing the total number of order sets in circulation. All of the other responses were described as success factors in the order set transformation.
221 | P a g e
S18-6: Based on this presentation, each of the following order set sections would be a suitable core module except:
A) Admission
B) Labs
C) Vital Signs
D) Notify Physician
E) Activity
Answer: B) Labs
Explanation: Based on the experience of the presenters, the optimal core modules were ones where there was broad commonality in the content across order sets. These modules could be built once, include all likely orders within that category, and then be re-used in future order sets. Commonalities were observed in admission orders, vital signs, notify physician, and activity sections of legacy order sets. In contrast, lab orders tended to be variable from one order set to another and each order set tended to have a unique combination of lab orders, thus labs would be unsuitable as a core module.
References:
1. A general reference that discusses broad themes around implementation of HIT:
Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. JAMIA. 2013;20(e1):e9-e13. doi:10.1136/amiajnl-2013-001684.
2. A work describing lessons learned during another institution’s EHR migration:
Rizer MK, Kaufman B, Sieck CJ, Hefner JL, McAlearney AS. Top 10 lessons learned from electronic medical record implementation in a large academic medical center. Perspectives in Health Information Management. 2015;12(Summer):1g.
3. A reference to ONC’s “How to implement EHRs” guide:
How to implement EHRs. Office of the National Coordinator for Health Information Technology. Available from: https://www.healthit.gov/providers-professionals/ehr-implementation-steps
4. A trade press article describing Vanderbilt’s preparations for their recent EHR conversion:
Barkholz D. Vanderbilt is a case study for the dreaded EHR conversion. Modern Healthcare. May 1, 2017. Available from: http://www.modernhealthcare.com/article/20170501/NEWS/170509989
222 | P a g e
S19: Panel - Implementing and Spreading Patient-centered, Evidence-Based Technologies: The Fall TIPS Story
Patricia Dykes, Denise Goldsmith, Ann O’Brien
S19-1: The Fall TIPS (Tailoring Interventions for Patient Safety) program is based on over a decade of research supporting its effectiveness with reducing falls and related injuries in hospitals. Which of the following are most likely to prevent a fall in patients with physiological risks for falling?
A. Universal fall precautions
B. A bed alarm
C. A fall prevention plan tailored to patient-specific risk factors
D. A fall prevention bundle
Answer C. A fall prevention plan tailored to patient-specific risk factors Explanation: Patients at risk for physiological falls require interventions that are tailored to their personal risk factors. Patients at risk due to a gait disturbance require different interventions than patients at risk due to a cognitive deficit or confusion. References: Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, Meltzer S, Tsurikova R, Zuyov L, Middleton B. Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010 Nov 03; 304(17):1912-8. PMID: 21045097; PMCID: PMC3107709. Dykes PC, Duckworth M, Cunningham S, Dubois S, Driscoll M, Feliciano Z, Ferrazzi M, Fevrin FE, Lyons S, Lindros ME, Monahan A, Paley MM, Jean-Pierre S, Scanlan M. Pilot Testing Fall TIPS (Tailoring Interventions for Patient Safety): a Patient-Centered Fall Prevention Toolkit. Jt Comm J Qual Patient Saf. 2017 Aug; 43(8):403-413. PMID: 28738986. S19-2: Which of the following are most likely to support evidence-based fall prevention interventions for all patients?
A. Universal fall precautions
B. Clinical decision support
C. A fall prevention bundle
D. Infobuttons in the EHR
Answer B. Clinical decision support Explanation: There are many factors that place hospitalized patients at risk for falls. Clinical decision support can link patient-specific risk factors to interventions most likely to prevent a fall based on each individual patient’s fall risk profile.
223 | P a g e
References: Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, Meltzer S, Tsurikova R, Zuyov L, Middleton B. Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010 Nov 03; 304(17):1912-8. PMID: 21045097; PMCID: PMC3107709. Dykes PC, Duckworth M, Cunningham S, Dubois S, Driscoll M, Feliciano Z, Ferrazzi M, Fevrin FE, Lyons S, Lindros ME, Monahan A, Paley MM, Jean-Pierre S, Scanlan M. Pilot Testing Fall TIPS (Tailoring Interventions for Patient Safety): a Patient-Centered Fall Prevention Toolkit. Jt Comm J Qual Patient Saf. 2017 Aug; 43(8):403-413. PMID: 28738986. S19-3: Which of the following is the strategy most likely to achieve successful implementation of Fall TIPS into an EHR?
A. Hospital leadership support B. Integration during the design/pre-implementation phase C. Fall prevention workflow analysis
D. Front line staff involvement Answer B. Integration during the design/pre-implementation phase Explanation: There are many factors that lead to a successful implementation of the Fall TIPS program into a vendor EHR product. Integration of Fall TIPS into the nurse’s workflow in advance of “go live” will most likely reduce delays, obstacles and barriers demonstrated when integration occurs during “optimization” phase. References: Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, Meltzer S, Tsurikova R, Zuyov L, Middleton B. Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010 Nov 03; 304(17):1912-8. PMID: 21045097; PMCID: PMC3107709. Dykes PC, Duckworth M, Cunningham S, Dubois S, Driscoll M, Feliciano Z, Ferrazzi M, Fevrin FE, Lyons S, Lindros ME, Monahan A, Paley MM, Jean-Pierre S, Scanlan M. Pilot Testing Fall TIPS (Tailoring Interventions for Patient Safety): a Patient-Centered Fall Prevention Toolkit. Jt Comm J Qual Patient Saf. 2017 Aug; 43(8):403-413. PMID: 28738986. S19-4: While the Fall TIPS program is currently used in over 100 hospitals in the US, and around the world, there are challenges to spread that can be addressed by organization leadership. What strategy has been used to improve long term effectiveness of the Fall TIPS program.
A. Regular fall prevention education B. Regular leadership participation in safety rounds C. Regular leadership seminars on fall prevention
D. Regular monitoring/feedback related to patient engagement in fall prevention
224 | P a g e
Answer D. Regular monitoring/feedback related to patient engagement in fall prevention Explanation: Patient engagement in all three steps of the fall prevention process (risk assessment, developing a tailored plan, and consistently implementing the plan) improves the “dose” of the Fall TIPS intervention and has been shown to decrease both falls and fall-related injuries. References: Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, Meltzer S, Tsurikova R, Zuyov L, Middleton B. Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010 Nov 03; 304(17):1912-8. PMID: 21045097; PMCID: PMC3107709. Dykes PC, Duckworth M, Cunningham S, Dubois S, Driscoll M, Feliciano Z, Ferrazzi M, Fevrin FE, Lyons S, Lindros ME, Monahan A, Paley MM, Jean-Pierre S, Scanlan M. Pilot Testing Fall TIPS (Tailoring Interventions for Patient Safety): a Patient-Centered Fall Prevention Toolkit. Jt Comm J Qual Patient Saf. 2017 Aug; 43(8):403-413. PMID: 28738986.
S20: Presentations - Safety: Root Cause Analysis, Virtual Care, and Medical Reconciliation
Verbal Orders: Root Cause Analysis of Co-signature Authentication Timeframes
--Thomas Powell S20-1: During a visit from the Joint Commission, the inspection team asks for additional information regarding verbal orders (VO) usage for the respiratory therapy department. Aside from the number and percentage of verbal orders, which additional analysis can be performed to present a more detailed overview of order procedures?
A. Perform a dual domain analysis of time/day for threshold based similarity
B. Parse the data to find patterns in VO receiver(s) and/or author(s)
C. Compare the respiratory therapy VO statistics against other departments in the health system
D. Compare the VO records against the health care system policy, describing the data points and timeline to authenticate the validity/authenticity/adherence of the order.
E. Identify grouping/bundling of orders to differentiate clusters from single purpose VO
Answer: D. Compare the VO records against the health care system policy, describing the data points and timeline to authenticate the validity/authenticity/adherence of the order. Explanation: Rationale: Given the current guidance and legislation, the onus regarding VO timelines, rate of issuance and monitoring falls directly on the institution. While pattern analysis of time/frequency, receiver/author, department comparisons and overall granularity analysis might yield vital information regarding VO for health care administrators, the regulatory elements focus more on policy adherence than the constituent data used to create and inform the health care system rules and procedures.
225 | P a g e
References: A review of verbal order policies in acute care hospitals (2012), The Joint Commission Journal on Quality and Patient Safety, Volume 38, Issue 1, Pages 24-AP4. S20-2: When analyzing Verbal Order (VO) records within the EHR, which data element is often overlooked?
A. Staff scheduling
B. Discharge timelines
C. Clinic type
D. Physician inbox time
E. Mode of VO delivery
Answer: D. Physician inbox time Explanation: Rationale: Scheduling, discharge, clinic type and mode (verbal or telephone) have all been addressed in published studies regarding VO analysis within the last twenty years. Given the role of the author is the primary means of VO authentication, the timeline of review is vital towards establishing a root cause analysis. References: Gellert GA, Catzoela L, Patel L, Bruner K, Friedman F, Ramirez R, Saucedo L, Webster SL, Gillean JA. The Impact of Order Source Misattribution on Computerized Provider Order Entry (CPOE) Performance Metrics. Perspectives in health information management. 2017;14(Spring).
Quality Standards for Virtual Care: The Roles and Responsibilities of Stakeholders for Advancing Care Quality in an Online Environment
--Kevin Smith
S20-3: A virtual care (telemedicine) provider group has convened a clinical quality council to provide guidance for ongoing clinical quality monitoring and management.
Which of the options below is most appropriate for establishing quality outcome metrics?
A. Start out by creating a new virtual care framework for measuring clinical quality.
B. Partner with a statistician to analyze virtual care claims data.
C. Establish a disciplinary action and remediation program for underperforming doctors.
D. Establish outcome measure targets for the most common virtual care visit types.
226 | P a g e
Answer: D. Establish outcome measure targets for the most common virtual care visit types.
Explanation: Establishing outcome measure targets provides a standardized framework for evaluating quality performance. These outcomes also relate to clinical effectiveness, or the impact of telemedicine on outcomes, such as adhering to best practice guidelines.
Incorrect options: the standards or framework for virtual care clinical should not be different from care provided in other settings, a statistical analysis is not helpful until the outcome measures have been established, disciplinary action for underperforming doctors does not relate to establishing clinical quality criteria.
Reference: Creating a Framework to Support Measure Development for Telehealth (pp. 9-14, Rep.). (2017). Washington, D.C.: National Quality Forum.
S20-4: A health care system wishes to improve appropriate antibiotic prescribing practices for the management of acute upper respiratory infections in the virtual care (e-visit) setting by improving clinician adherence to an evidence-based clinical guideline.
Given the above information, which of the following may be the most appropriate intervention to improve adherence to the evidence-based clinical guideline?
A. Add a link to the Infectious Disease Society of America web site on the clinician screen.
B. Incorporate contextual clinical decision support which highlights pertinent patient history of the current illness, a brief summary of the guideline criteria with actionable guidance, all within the clinician workflow.
C. Provide a variety of pop-up reminders and alerts reminding clinicians of the importance of antibiotic stewardship.
D. Disable e-prescribing functionality for antibiotic medications to prevent clinician over-prescribing.
Answer: B. Incorporate contextual clinical decision support which highlights pertinent patient history of the current illness, a brief summary of the guideline criteria with actionable guidance, all within the clinician workflow.
Explanation: CDS strategies are most effective when the users are knowledgeable about the clinical guideline and are provided with CDS that is incorporated into their typical workflow - such as highlighting key data elements and including a brief summary of the guideline within the workflow along with actionable guidance.
Individual interventions outside of the clinical workflow, such as provider education, may or may not result in changes in clinician adherence to clinical guidelines.
Incorrect options:
227 | P a g e
A. “Add a link to the Infectious Disease Society of America web site on the clinician screen.” This item is incorrect as it requires the clinician to browse to an external web resource. Although the IDSA web site has excellent evidence based guideline information, the clinician may have to search for the guideline articles making this workflow inconvenient and less effective.
C. “Provide a variety of pop-up reminders and alerts reminding clinicians of the importance of antibiotic stewardship.” This item is incorrect since reminders about antibiotic stewardship are less specific or contextual compared to item b. Pop-up alerts and reminders may lead to "alert fatigue" which can reduce the CDS effectiveness.
D. “Disable e-prescribing functionality for antibiotic medications to prevent clinician over-prescribing.” This item is incorrect because it is extremely impractical. Disabling antibiotic prescribing would certainly prevent over-prescribing of antibiotics but would also prevent the appropriate prescribing of antibiotics.
Reference: Castillo RS, Kelemen A. Considerations for a successful clinical decision support system. Comput Inform Nurs. 2013;31(7):319-26.
S20-5:Which of the following is most true regarding the current state of clinical quality monitoring and management among virtual care (telemedicine) providers and vendors in the United States?
A. The CDC has been adopted as the source of truth for virtual care quality standards.
B. Any virtual care provider or vendor who can demonstrate that they have a medical reference available for the clinicians, such as UpToDate, is considered to be compliant with quality standards.
C. There are no established standards for measuring clinical quality outcomes in virtual care in the United States.
D. The World Health Organization, in partnership with AHRQ, is considered as the source of truth for virtual care quality
Answer: C. There are no established standards for measuring clinical quality outcomes in virtual care in the United States.
Explanation: Although many providers, vendors, and organizations (such as the American Telemedicine Association) have developed and published clinical guideline documents for virtual care, there currently are no agreed-upon outcome measurement standards.
Incorrect options: The CDC has excellent resources for clinical guidance but has not been adopted as the standard, creating guideline documents or making a clinical reference available does not ensure that guidelines are followed. The WHO and AHRQ are also excellent resources related to clinical quality but have not been adopted as the standard.
Reference: Practice Guidelines for Live, On-Demand Primary and Urgent Care [PDF]. (2014, November). Washington, D.C.: American Telemedicine Association.
228 | P a g e
S20-6: The quality committee has a debate about how to approach clinical quality as it relates to the relatively new and rapidly growing model of virtual care.
What is the greatest challenge for establishing clinical quality targets for virtual care?
A. The conditions treated via virtual care are vastly different than those treated in a clinic setting.
B. The challenges for establishing clinical quality targets are essentially the same no matter the method of care delivery.
C. The potential limitations of telemedicine equipment and internet connectivity could make it more difficult to deliver quality care.
D. The lack of research in this area makes it impossible to establish quality targets.
Answer: B. The challenges for establishing clinical quality targets are essentially the same no matter the method of care delivery.
Rationale: Establishing standards for the quality of virtual care based on evidence-based, nationally recognized best practices is an effective way to support quality in the performance of a virtual care program while avoiding unnecessary and redundant measure development.
Incorrect options: Conditions treated via virtual care are no different than those treated in other settings, equipment and internet connectivity should not affect a clinician's ability to provide appropriate care as other options can be provided, and the availability of research in this area has no impact on setting quality standards.
Reference: Creating a Framework to Support Measure Development for Telehealth (pp. 9-14, Rep.). (2017). Washington, D.C.: National Quality Forum.
Using Qualitative Techniques to Classify Medication Reconciliation Errors
--Rod Tarrago
S20-7: According to the Institute for Healthcare Improvement (IHI), the medication reconciliation process involves three steps. Which of these is NOT one of the steps:
A. Verification
B. Clarification
C. Communication
D. Reconciliation
Answer: C. Communication
229 | P a g e
Explanation: According to the IHI, the process involves verification (collection of the medication history), clarification (ensuring that the medications and doses are appropriate) and reconciliation (documentation of the changes in orders). Communication is involved in all three but is not a specific step.
Reference: Institute for Healthcare Improvement. 5 million lives getting started kit: preventing adverse drug events (medication reconciliation), how- to guide. Available at: http://www.ihi.org/IHI/Programs/Campaign/ ADEsMedReconciliation.htm. Published Oct. 1, 2008. Accessed September 2010.
S20-8: The Joint Commission states that the intention of medication reconciliation is to address all of the following except:
A. Duplications
B. Expirations
C. interactions
D. Omissions
E. Continuations
Answer: B. Expirations
Explanation: According to the Joint Commission, there is evidence that medication discrepancies can affect patient outcomes. Medication reconciliation is intended to identify and resolve discrepancies—it is a process of comparing the medications a patient is taking (and should be taking) with newly ordered medications. The comparison addresses duplications, omissions, and interactions, and the need to continue current medications. The types of information that clinicians use to reconcile medications include (among others) medication name, dose, frequency, route, and purpose. Organizations should identify the information that needs to be collected to reconcile current and newly ordered medications and to safely prescribe medications in the future.
Reference: https://www.jointcommission.org/assets/1/6/NPSG_Chapter_OME_Jan2017.pdf
230 | P a g e
S21: Presentations - Patient Generated Data, Interventions, and Telehealth
Patient Generated Safety Data in the Hospital Setting: Trust, Transparency and Learning
--Sarah Collins
S21-1. Which of the following related to safety reporting in the hospital is true?
A. Most recognized safety concerns and incidents are reported
B. Patients and families are not considered key stakeholders in safety reporting
C. Most recognized safety concerns and incidents are not reported
D. Safety data reported by patients and families provides little insights
Answer: C. Most recognized safety concerns and incidents are not reported
Explanation: Only a fraction of safety concerns and safety risks that may lead to adverse events are reported. In addition, hospital safety incidents are underreported and this limits our understanding of safety and ability to learn from unsafe system processes. Patients and families are important stakeholders in hospital safety reporting, and their concerns can provide valuable insights.
References:
Levinson DR. Hospital incident reporting systems do not capture most patient harm. Washington DC: Office of the Inspector General. 2012 Jan.
Weingart SN, Pagovich O, Sands DZ, et al. What can hospitalized patients tell us about adverse events? Learning from patient-reported incidents. J Gen Intern Med. 2005 Sep 1;20(9):830-6.
Couture, B, Lilley, E, Chang, F, et al. User-centered design methods for development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform. In press 2018.
S21-2: Which of the following is an identified sociotechnical requirement for patient safety reporting most related to trust from patients?
A. Availability of a real-time clinician dashboard
B. Protected anonymity when submitting
C. Comparing data to other safety data sets
D. Clear responsibility for follow up
Answer: B. Protected anonymity when submitting
Explanation: The ability to submit anonymously should be an option for patients and families using a safety reporting application in the hospital setting. Users should also feel confident that this anonymity
231 | P a g e
will be protected and preserved. This is important for maintaining trust of patients and families to foster a culture of safety.
Reference: Couture, B, Lilley, E, Chang, F, et al. User-centered design methods for development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform. In press 2018.
Building and Evaluating a Patient-Facing Intervention to Deliver Implantable Cardioverter-Defibrillator Remote Monitoring Data to Patients via MyChart: Patient and Provider Perceptions
--Romisa Rohani Ghahari
S21-3: A healthcare system wishes to send personalized information to patients based on implantable cardiac defibrillator (ICD) remote monitoring data. Survey data from a recent study showed that most of the participants (86%) wanted to receive more information than they currently receive from the clinic about their remote monitoring data through the standard of care letter. Which of the following is most appropriate based on the findings from this study?
A. Patients would be most satisfied with a standard of care letter when compared to the ICD data summary.
B. Patients found the standard of care letter easier to access and use compared to ICD data summary.
C. Participants found the mailed letter with the ICD data summary easier to access compared to the electronic format.
D. Either method (mailed letter vs. electronic format) of providing the ICD data summary works well for the patients.
Answer: D. Either method (mailed letter vs. electronic format) of providing the ICD data summary works well for the patients.
Explanation: The goal of a healthcare system is to personalize device information based on patients’ preferences. The results of this study showed that although electronic format was easier to access, either mailed letter or electronic format worked for the patients. Patients preferred to receive the ICD data summary over the standard of care letter.
Reference: Ghahari RR, Holden RJ, Flanagan ME, Wagner S, Martin E, Ahmed R, Daley CN, Tambe R, Chen E, Allmandinger T, Mirro MJ. Using cardiac implantable electronic device data to facilitate health decision making: A design study. International Journal of Industrial Ergonomics. 2018 Mar 31;64:143-54.
S21-4: A healthcare system wishes to provide patients with their device information without increasing clinical workload and having a negative effect on providers. Prior research reported that providers were
232 | P a g e
concerned that messaging patients with their device data could potentially increase the clinical workload. We evaluated the effect of providing patients with their device information on clinical workload and clinician attitude towards ICD data summary. Which of the following is most appropriate based on the findings from this study?
A. Providing patients with their device information did increase healthcare utilization.
B. Providing patients with their device information did not increase the workload of clinicians from increased patient questions (calls or PHR messaging).
C. Clinicians believed the ICD data summary was NOT easy for patients to understand and use for managing their health.
D. Providing patients with their device information did increase the device-specific workload for clinicians.
Answer: B. Providing patients with their device information did not increase the workload of clinicians from increased patient questions (calls or PHR messaging).
Explanation: The results of this study showed that providing patients with their device information did not increase the device-specific workload for clinicians or healthcare utilization in patients. Further, clinicians who participated in this study agreed that patients would be able to use the ICD data summary for self-management.
Reference: Ghahari RR, Holden RJ, Flanagan ME, Wagner S, Martin E, Ahmed R, Daley CN, Tambe R, Chen E, Allmandinger T, Mirro MJ. Using cardiac implantable electronic device data to facilitate health decision making: A design study. International Journal of Industrial Ergonomics. 2018 Mar 31;64:143-54.
Predictors of average adherence to the use of home telehealth devices in Veterans with heart failure
--Jenice Ria Guzman
S21-5: Adherence to the use of home telehealth devices in Veterans with heart failure enrolled in the VA Home Telehealth Program ________ over time.
a. decreased
b. increased
c. reversed
d. stayed the same
Answer: a. decreased Explanation: Current study that will be presented found that adherence to the use of home telehealth devices decreases over time in Veterans with heart failure.
233 | P a g e
References: Guzman-Clark J, van Servellen G, Chang B, Mentes J, & Hahn T (2013). Predictors and outcomes of early adherence to the use of a home telehealth device by older Veterans with heart failure. Telemedicine & e-Health, 19(3):217-223. Ong M, Romano PS, Edgington S, et al. Effectiveness of remote patient monitoring after discharge of hospitalized patients with heart failure: The Better Effectiveness After Transition-Heart Failure (BEAT-HF) Randomized Clinical Trial. JAMA Intern Med 2016;176(3):310-318. S21-6: It is important to understand adherence to the use of home telehealth devices because:
a. Adherence is complicated and needs to be monitored.
b. Home telehealth devices are expensive and should only be given to patients who will use them.
c. Poor adherence to home telehealth devices over time may be a potential reason for negative results in prospective randomized controlled trials.
d. There needs to be a conversation with patients regarding expected adherence to the use of home telehealth devices for outcome effectiveness.
Answer: c. Poor adherence to home telehealth devices over time may be a potential reason for negative results in prospective randomized controlled trials. Explanation: Although all the answers have some truth in them, c is the best answer and has been previously mentioned in prior articles on the topic. References: Chaudhry SI, Mattera JA, Curtis JP, et al. Telemonitoring in patients with heart failure. N Engl J Med 2010; 363: 2301–09. Guzman-Clark J, van Servellen G, Chang B, Mentes J, & Hahn T (2013). Predictors and outcomes of early adherence to the use of a home telehealth device by older Veterans with heart failure. Telemedicine & e-Health, 19(3):217-223. Ong M, Romano PS, Edgington S, et al. Effectiveness of remote patient monitoring after discharge of hospitalized patients with heart failure: The Better Effectiveness After Transition-Heart Failure (BEAT-HF) Randomized Clinical Trial. JAMA Intern Med 2016;176(3):310-318.
234 | P a g e
S22: Presentations - Identifying Medication Risk
Building a pharmacist prioritization tool using pharmacy-specific data and readmission risk
--Samuel Kabue
S22-1: When building a universal readmission risk prediction model for use in pharmacy, it is important to base the modeling strategy on:
A. Commercially available 'black box' prediction models
B. A single predictor such as age or sex
C. Predictors that are readily available in electronic medical records within many health systems/pharmacy departments
D. Pharmacists have time to intervene on every patient and therefore have little need for risk prediction models
Answer: C. Predictors that are readily available in electronic medical records within many health systems/pharmacy departments
Explanation: One challenge in universal use of pharmacy-specific prediction models is ensuring that all predictors and outcomes used in the model are readily available within a particular health system. When a health system/pharmacy department does not capture a predictor or outcome, the model cannot be used effectively.
Robust prediction models will typically require more than one predictor.
Budgets restrict the purchase of commercial products and even when purchased, coefficients and some predictors remain proprietary limiting interpretation.
Pharmacist-capacity is generally overwhelmed by the low pharmacist-patient ratio.
References: Escobar, G. J., et al. (2015). "Nonelective Readmissions and Postdischarge Mortality: Predictive Models Suitable for Use in Real Time." Med Care 53(11): 916-923.
Steyerberg, E. W., et al. (2010). "Assessing the performance of prediction models: a framework for traditional and novel measures." Epidemiology 21(1): 128-138.
Kansagara, D., et al. (2011). "Risk prediction models for hospital readmission: a systematic review." JAMA 306(15): 1688-1698.
Escobar G, Greene J, Scheirer P, et al. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46:232–239.
S22-2: A pharmacy department or health system wishes to use our readmission risk prediction model but does not have a COPS2 score, LAPS2 score or RxDxCG score readily available. Which if the following
235 | P a g e
predictors currently available to them can be used to in place of these predictors for the best performing model?
A. Diagnosis groupings
B. Drug subgroups
C. Admission category indicators (i.e. emergency department versus medical)
D. Number of refills on each medicine
Answer: B. Drug subgroups
Explanation: Our study shows that addition of COPS2, LAPS2 and RxDxCG scores to the reference model of age, sex, LOS and admission category makes a good performing model. Individually COPS2 is associated with the greatest increase in performance over the reference model. RxDxCG is associated with new identical increase in performance as the drug subgroups, therefore implying that they are interchangeable in our models.
Diagnosis groupings were associated with the smallest increase in model performance.
Admission category was part of the base reference model.
There is no evidence yet linking number of refills to readmission risk.
References:
Escobar, G. J., et al. (2015). "Nonelective Readmissions and Postdischarge Mortality: Predictive Models Suitable for Use in Real Time." Med Care 53(11): 916-923.
Steyerberg, E. W., et al. (2010). "Assessing the performance of prediction models: a framework for traditional and novel measures." Epidemiology 21(1): 128-138.
Kansagara, D., et al. (2011). "Risk prediction models for hospital readmission: a systematic review." JAMA 306(15): 1688-1698.
Escobar G, Greene J, Scheirer P, et al. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46:232–239.
A data mining approach to detect medication order near misses associated with electronic order sets
--Ron Li
S22-3: Which of the following best describes an error induced by automation bias?
A. Dr. Smith is alerted by an automatic EHR pop-up alert notifying that his patient may be developing severe sepsis and includes a button that places an order for 30cc/kg of normal saline. However, this patient also has congestive heart failure with an ejection fraction of 20%. Dr. Smith, assuming that the alert is appropriate, does not consider this patient's heart failure diagnosis and proceeds to order
236 | P a g e
the 30cc/kg of normal saline. The patient later develops acute pulmonary edema and has to be transferred to the intensive care unit.
B. Dr. Smith is about to order antibiotics for his patient who has urinary tract infection. He chooses ciprofloxacin because that is what he is used to using for his patients, but did not notice that this patient's previous urine culture grew organisms resistant to ciprofloxacin
C. Dr. Smith needs to discontinue the medication metoprolol for a patient in the EHR. However, he mistakenly discontinues the medication "metoclopramide," which was located right next to metoprolol in the EHR and looked very similar.
D. Dr. Smith is about to admit his 10th patient on a busy call day. When placing his admission orders, he mistakenly signs them in an open chart in the EHR for another patient whom he had seen earlier.
Answer: A. Dr. Smith is alerted by an automatic EHR pop-up alert notifying that his patient may be developing severe sepsis and includes a button that places an order for 30cc/kg of normal saline. However, this patient also has congestive heart failure with an ejection fraction of 20%. Dr. Smith, assuming that the alert is appropriate, does not consider this patient's heart failure diagnosis and proceeds to order the 30cc/kg of normal saline. The patient later develops acute pulmonary edema and has to be transferred to the intensive care unit.
Explanation: Automation bias refers to the propensity for humans to be over-reliant on suggestions from automated decision support systems and ignore contradictory evidence. In this case, the patient, who has severe congestive heart failure, should not have been given 30cc/kg of fluid, but Dr. Smith relied on the automated decision support recommendation and gave it anyway, which resulted in an adverse event.
The other options are incorrect because:
B. This error is an example of the effects of availability heuristic.
C. This error resulted from a difficult to read EHR interface
D. This error is an example of a wrong patient order that can occur when multiple patient charts are open in the EHR
Reference: Goddard, K., Roudsari, A. & Wyatt, J. C. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J. Am. Med. Informatics Assoc. 19, 121–127 (2012).
237 | P a g e
S22-4: Which of the following functions would you typically fail to achieve in your design of electronic order sets?
A. Save time for clinicians for placing orders
B. Promote adherence to clinical practice guidelines
C. Provide clinical decision support
D. Increase the number of billable diagnoses for a clinical encounter
Answer: D. Increase the number of billable diagnoses for a clinical encounter
Explanation: While the challenge of adequately capturing billable diagnoses for clinical encounters may have informatics solutions, electronic order sets only suggest orders for a given diagnosis or clinical workflow and typically do not include any suggestions for billable diagnoses.
The other options are incorrect because:
A. order sets can save time for placing orders by having relevant orders be located in one place in the EHR
B. order sets can be curated to include orders modeled after clinical practice guidelines
C. By including relevant orders for a diagnosis, order sets are a form of clinical decision support
Reference: McGreevey, J. D. Order sets in electronic health records: Principles of good practice. Chest 143, 228–235 (2013).
Detecting post-approval Adverse Drug Events within EHR Data: A case study correlating Lipophilic Beta-Blockers with the subsequent diagnosis of Parkinson’s Disease.
--Cyril Spiro
S22-5: A group of researchers are studying potential adverse drug reactions (ADR) related to prescriptions of medications and subsequent diagnosis of a specific disease. The researchers first select all the ADRs related to the specific disease detected by an ADR detection heuristic. Then the researchers study the ADRs by looking at their odds ratios. The goal of the researchers is to shortlist the potential ADRs for a subsequent detailed study. A few of the potential ADRs are related to a specific class of medications. Previous research has shown that this class of medications, let’s call it “X class,” have a high chance of interacting with other medications and cause complications in patients. The X class of medications have also shown conflicting results in previous research studies which were studying the ADR in question. Assume that a patient can be on X class medication along with other medications in the potential ADR list and the researchers want to shortlist the remaining ADRs in the list.
238 | P a g e
Which step can the researchers take to most efficiently shortlist the remaining and offset the potential effects of X class medications on other potential ADRs that were detected?
A. Recalculate the odds ratios of only the remaining potential ADRs for the particular disease in question by excluding patients that were on X class of medications.
B. Stop the study until a consensus is reached by clinical trials on the ADR related to X class of medications and the specific disease in question.
C. Continue shortlisting the potential ADRs without any change in the analysis strategy.
D. Start the entire analysis from the beginning by first removing the patients who were on X class of medications from the data and then running the ADR detection heuristic again on entire available data.
Answer: A. Recalculate the odds ratios of only the remaining potential ADRs for the particular disease in question by excluding patients that were on X class of medications.
Explanation: The goal of the researchers is to find the potential ADRs for a specific disease by looking at the data in an EHR database. Considering that the database will have multiple diseases and medications, the ADR detection heuristic will have to scan through the entire EHR database to detect the potential ADRs detected in the database. If the researchers want to negate or offset the potential effects of X class medications on other potential ADRs for a particular disease, it will be inefficient to rescan through the entire database and recalculate the odds ratios for all ADRs (not just the ones related to the disease in question). Nor will it be a good strategy to continue without recalculating the odds ratios or to completely halt the study because of one ADR mixed results. The best strategy here will be to reduce the computing costs and time delays by only recalculating the odds rations for the ADRs related to the diseases in question.
Reference: Szumilas M. Explaining Odds Ratios. Journal of the Canadian Academy of Child and Adolescent Psychiatry. 2010;19(3):227-229.
S22-6: A group of researchers are studying a particular potential adverse drug reaction (ADRs) related to a particular disease (disease Dx) and a particular medication (medication Rx) found by using data obtained from an EHR system. The design of the study is that of a retrospective cohort study. On investigating the data on the detected potential ADR by using statistical tests and running confounding analysis, the researchers find that all the statistical tests are pointing in the direction of the potential ADR being an actual ADR. What can the researchers conclude from the results of the statistical tests?
A. There is a causal effect of the prescriptions of medication Rx to patients leading to them developing the disease Dx. No more analysis or studies are necessary to confirm the findings.
B. There is strong evidence to conclude that there is a correlation between prescription of medication Rx to patients and the subsequent development of disease Dx in those patients. Conducting clinical trials is necessary to establish a causation.
239 | P a g e
C. No inference can be made from the statistical tests about either causation and correlation between drug Rx and disease Dx.
D. There is strong evidence to conclude that there is a correlation between prescription of medication Rx to patients and the subsequent development of disease Dx in those patients. Running further statistical tests on the EHR data (observational data) will establish causation.
Answer: B. There is strong evidence to conclude that there is a correlation between prescription of medication Rx to patients and the subsequent development of disease Dx in those patients. Conducting clinical trials is necessary to establish a causation.
Explanation: The researchers are conducting statistical tests on observational data using a retrospective cohort study design. The statistical tests will not be able to establish causation between the prescription of medication Rx and the subsequent diagnosis of disease Dx because there might be potential confounders in the study that the researchers cannot control for using the observational data alone. To establish causation, the ADR will need be studied in a controlled experiment were the effect of the potential confounders are minimized. The researchers can however check for a significant correlation between medication Rx and disease Dx using the observational data alone. The results from the statistical tests on the observational data can then be used as a hypothesis which can be tested with clinical trials.
Reference: Kawachi I. Injustice at work and health: causation or correlation? Occupational and Environmental Medicine 2006;63:578-579.
S23: Panel - Closing-the-loop: Transforming Medical Information into Action
--Chancey Christenson, Kenan Katranji, Gargi Schneider
S23-1: What do experts identify as the most challenging aspect of the implementation of pharmacogenetics into the clinic?
A. Translation of genetic information into clinical action
B. Test cost, test reimbursement or other economic issues
C. Availability of high quality genotyping test (CLIA approved)
D. Electronic medical record use, such as application of CDS
E. Clinician and/or patient resistance
Answer: A. Translation of genetic information into clinical action Explanation: Survey of pharmacogenomic “experts” (PGRN and ASCPT) on the top 3 challenges to implementing pharmacogenetics in the clinic, 95% of respondents selected “process required to
240 | P a g e
translate genetic information into clinical actions.” The next 2 responses were genotype test interpretation (e.g. using genotype information to assign phenotype), and providing recommendations for selecting the drug/gene pairs to implement. Reference: Relling MV, Klein TE. CPIC: Clinical Pharmacogenetics Implementation Consortium of the Pharmacogenomics Research Network. Clin Pharmacol Ther. 2011 Mar;89(3): 464-7.
S23-2: What type of genetic information does the MyCode program at Geisinger use?
A. Whole Gene
B. Proteins
C. Exons
D. Enzymes
E. Introns Answer: C. Exons Explanation: The exons are the areas containing information for the creation of proteins and they make up about 1% of the DNA. With current technology, exon sequencing is more cost-effective than whole genome sequencing, making it better suited to large scale precision health applications. As of March 1st, 2018, the MyCode program at Geisinger has sequenced 92,816 participants exomes with Regeneron Genetics Center. Reference: https://www.geisinger.org/mycode S23-3: Of the following which is the most challenging aspect of Natural Language Processing?
A. Tokenization
B. Parts of Speech Tagging
C. Dealing with ambiguity
D. Parsing of a sentence
Answer: C. Dealing with ambiguity Explanation: The challenge with NLP is to assign semantic meaning to language. NLP uses different knowledge sources and contextual information to derive meaning; however, there is often ambiguity within sentences that can be difficult to resolve. For example, in an emergency medicine context, "shock" will have completely different meaning than when the term is used to describe an emotional response within the context of a psychiatry visit. Reference: Shortliffe EH. Biomedical Informatics. 4th ed. London: Springer; 2014. 264 p
241 | P a g e
S23-4: Which of the following is most likely to be stored as structured data?
A. Video
B. Images
C. Text
D. Audio
Answer: C. Text Explanation: Although text can be stored as both structured and unstructured data, the other choices are generally stored only as unstructured data References: McCallum A. Information Extraction: Distilling Structured Data from Unstructured Text. Queueing Syst. 2005 Nov;3(9):4:48–4:57.
S24: Presentations - Patient Deduplication, Clinical Phenotypes, and Collecting Social Determinants of Health
Developing Methods for Standardized Patient Cohort Deduplication Across the Carolinas Collaborative, A Regional Data Research Network
--Robert Bradford
S24-1: A Clinical Data Research Network (CDRN) has decided on a hashing algorithm that utilizes: Last Name, Date of Birth, and Last 4 digits of the Social Security Number (SSN), where Date of Birth and Last 4 digits of the SSN are translated directly into text (with no formatting changes). Site A has the following data for their patient: First Name: John Middle Name: A Last Name: Smith Date of Birth: 01/31/53 SSN: 000-0000-7890 State: NC Which of the following options of data available at Site B would cause the de-duplication process to FAIL (hashes do not align).
A. SSN: 123-456-7890 B. Middle Name: 'null'
C. First Name: "John A" D. Date of Birth: 31-JAN-1953 E. State: FL Answer: D. Date of Birth: 31-JAN-1953
242 | P a g e
Explanation: Option D is the correct choice. While the date values both represent the same date within their respective databases, when converted into 'Text' the format difference (Site A: MM/DD/YY, Site B:DD-Mon-YYYY) would produce two different hash values. Option A shows that Site B has extra information about the patient's SSN, however the hash only requires the last 4 to be provided. The last 4 digits of the SSN match between Site A and Site B. Options B, C, and E are not used in the defined hashing method, therefore differences in formatting and the values themselves will not affect the hash. Reference: Christen P. Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. New York: Springer; 2012. doi: 10.1007/978-3-642-31164-2 S24-2: One of the methods to success in developing the Carolina's Collaborative was the ability to test the de-duplication process internally between two different data sources. If another institution were to join the Collaborative but did not have a legacy EMR to use as an internal gold standard in testing the de-duplication process, what would be the best validation approach to attempt first?
A. Implement the process without additional checks, using previous validation results from other sites as justification
B. Use only dummy data as a validation mechanism
C. Use another internal patient-centered data source, such as claims or registry data
D. Obtain an IRB approval and execute a Data Use Agreement with a collaborating site to share patient data for purposes of testing the de-duplication process
Answer: C. Use another internal patient-centered data source available, such as claims or registry data Explanation: Utilizing internal data already available is the easiest, safest, and most thorough course that an institution can pursue when testing data linkage. Most institutions will have an ancillary data source that contains the same identifiers used in the hashing algorithm. Even though it is not EMR data, it allows a new process (such as the Collaborative de-duplication process) to be compared to a previously established and validated linkage to data, such as claims or registry data. This method avoids the complicated processes for sharing fully identified patient data outside of the institution, especially when Social Security numbers are included (Option D). Options A and B would be risky and ill-advised since some level of validation with real data must occur to identify potential conflicts in data formats between institutions. Reference: Kho AN, Cashy JP, Jackson KL, Pah AR, Goel S, Boehnke J, et al. Federated queries of clinical data repositories: the sum of the parts does not equal the whole. J Am Med Inform Assoc. 2015 Sep;22(5):1072-80. doi: 10.1093/jamia/ocv038. Epub 2015 Jun 23.
243 | P a g e
Practical Clinical Phenotype Definitions with SNOMED CT
--DuWayne Willett
S24-3: A health care system wishes to construct diagnosis groupers for a large number of clinical conditions (clinical phenotypes) to support chronic disease registries and population health initiatives. To streamline grouper construction and maintenance, they wish to explore ways to build diagnosis groupers besides a one-dimensional list of terms. Hierarchy-based groupers are possible when using SNOMED CT because:
A. SNOMED has parent-child ("is a") relationships between concepts that relate specific concepts to more general concepts
B. SNOMED has attribute relationships between concepts that specify additional defining characteristics of one of the concepts
C. SNOMED has fewer concepts than ICD
D. Clinical terminologies in EHRs are only mapped to SNOMED, and not to ICD
Answer: A. SNOMED has parent-child ("is a") relationships between concepts that relate specific concepts to more general concepts
Explanation: The correctAnswer is (A). "Is-a" relationships that link specific concepts to more general ones enable succinct definition of groupers by referring to a parent and all its descendants with a single reference to the entire hierarchy.
Attribute relationships that define a SNOMED concept more specifically by referring to other SNOMED concepts enable more precise definitions, but do not create hierarchical relationships. SNOMED contains substantially more concepts than the number of ICD-10 codes. Clinical terminologies (e.g. from IMO or Health Language) used in EHRs are mapped to both ICD codes (for billing) and SNOMED concepts (for interoperability).
References:
Chute CG. Clinical classification and terminology: some history and current observations. J Am Med Inform Assoc. 2000;7(3):298-303.
Benson T, Grieve G. Principles of health interoperability: SNOMED CT, HL7, and FHIR. Third edition. ed. London: Springer; 2016. xxix, 451 pages
S24-4: Compared with groupers listing all individual codes, hierarchy-based groupers:
A. require more frequent updating when new codes or concepts are added or deprecated
B. more closely match clinical thinking about disease subtypes to include or exclude from a grouper definition
244 | P a g e
C. take more time to construct and maintain
D. are more likely to be missing relevant diagnosis clinical terms
Answer: B. more closely match clinical thinking about disease subtypes to include or exclude from a grouper definition
Explanation: The hierarchical "is a" (type of) relationships between SNOMED concepts express type-subtype relationships that match how clinicians think about clinical disorders and their subtypes. This streamlines clinical vetting of groupers to achieve faithful representation of the intended clinical condition. Groupers designed with SNOMED hierarchies are more resilient to additions or deprecations of individual codes/concepts than list-based groupers: addition of new "descendants" within an existing SNOMED hierarchy does not require a change to the grouper definition. SNOMED hierarchy groupers require far fewer concepts/codes to define (median of 3 concepts in this report) than groupers using lists of codes, and thus take less time to construct and maintain. Hierarchy-based groupers automatically include a complete set of relevant descendant concepts.
References:
Chute CG. Clinical classification and terminology: some history and current observations. J Am Med Inform Assoc. 2000;7(3):298-303.
Benson T, Grieve G. Principles of health interoperability: SNOMED CT, HL7, and FHIR. Third edition. ed. London: Springer; 2016. xxix, 451 pages
Supporting Disparities Research by Increasing the Collection of Social Determinants of Health Data through EHRs
--Lauren Hovey
S24-5: To facilitate the standardized collection of REaL data, which of the minimum data sets provides an optimal level of granularity?
A. OMB minimum standard
B. HHS race and ethnicity standards
C. CDC Race and Ethnicity Code Set Version 1.0
D. The optimal data standards are those that are fit for purpose and the context in which the data are collected.
Answer: D. The optimal data standards are those that are fit for purpose and the context in which the data are collected.
245 | P a g e
Explanation: Providers must balance a number of clinical and administrative responsibilities in collecting REaL and SOGI data. This includes managing workflow and time constraints, patient relationships, and navigating the EHR product itself. Most EHRs and health IT products are capable of demographic data collection; therefore, “the best fit” for granularity of demographic data capture involves a triangulation of the providers’ priorities for data capture and use, patient needs, and available standards and products.
Reference: FHIR US Core Implementation Guide. Race and Ethnicity - CDC. http://hl7.org/fhir/us/core/stu1/CodeSystem-cdcrec.html
Current Measures of Sexual Orientation and Gender Identity in Federal Surveys. August 2016. https://s3.amazonaws.com/sitesusa/wp-content/uploads/sites/242/2014/04/WorkingGroupPaper1_CurrentMeasures_08-16.pdf
S24-6: Even when providers and/or health systems are motivated to collect and use REaL/SOGI data for clinical purposes, their vendor products may not be optimized for the task. Conversely, vendors may offer robust functionalities that are unintuitive or that providers are not well versed in using. For EHR products designed to facilitate REaL/SOGI collection and use, which of the following is an optimal solution to reducing provider burden during information capture?
A. Pre-populated fields related to gender/sex-specific history
B. Free text fields that allow providers to enter relevant information that may not be captured
C. Capture of the information in a pre-assessment (e.g., via portal or registration questionnaire)
D. Structured fields for REaL/SOGI that occur in a consistent place in the EHR
Answer: B. Free text fields that allow providers to enter relevant information that may not be captured
Explanation: (B) Ideally, vendors support providers’ in how to use structured REaL/SOGI fields and where they appear within the patient record. (A) Products that restrict or pre-populate sex-specific history, exam, or ordering templates based on a patient’s recorded sex at birth may limit the clinician’s ability to record and update the patient record (e.g., a transgendered patients’ medical transition history and current anatomy). (C) Free text fields are useful for difficult-to-code information, but are difficult to access for future care. (D) The ideal mode and timing of demographic data collection is an open question as far as how best to encourage patient disclosure; however, the utility of this information depends on its electronic captured and how effectively it is incorporated into the EHR.
Reference: U.S. Department Of Health And Human Services Implementation Guidance On Data Collection Standards For Race, Ethnicity, Sex, Primary Language, And Disability Status. Oct 2011. http://aspe.hhs.gov/datacncl/standards/ACA/4302
246 | P a g e
S25: Presentations - Methods for Enhancing EHR Usability
Categorization of EHR Design Problems Using Usability Heuristics
--Benjamin Duncan S25-1: The main goal of the heuristic categorization methodology introduced here is intended to accomplish which of the following?
A) Allows for grouping of the most significant issues in usability and to prioritize them based on their effect on workflow
B) Make suggestions to users to improve the usability of the overall system
C) Have usability experts review a chosen interface and compare it against accepted usability principles
D) Evaluators work through a series of tasks and ask a set of questions from the perspective of the user.
Answer: A) Allows for grouping of the most significant issues in usability and to prioritize them based on their effect on workflow Explanation: In the prior work, a range of user problems was documented and their impact on nursing workflow characterized [1]. However, compared to heuristic evaluation where a usability expert reviews an interface and compares usability problems against usability principles, usability categorization allows for the grouping of usability problems and then categorizes them based off of a set of heuristics. Reference: Duncan, B., Solomon, A.J., Doebbeling, B.N., Furniss, S.K., Grando, A., Poterack, K, Miksch, T., Helmers, R., &Kaufman, D.R. “A Comparative In-Situ Study of Two Medical Reconciliation Interfaces in Pre-Operative Nurse Assessments. WISH 2017, AMIA 2017. S25-2: Why should workflow be studied in advance of an enterprise-wide EHR conversion?
A) Make improvements to the current clinical systems being used
B) Perform log file analysis on user interactions to develop a standard workflow for EHR-mediated tasks
C) Understand variation across different clinical sites and systems in the healthcare settings
D) Determine how well clinicians can perform their specific tasks through various EHRs based off interactive behavior measures and human-computer interaction frameworks.
Answer: C) Understand variation across different clinical sites and systems in the healthcare settings Explanation: In order to understand workflow in advance of an enterprise-wide workflow, the variation across not only various clinical sites but the variation across clinicians needs to be understood to understand bottlenecks in workflow along with sources of variation. This allows for the harmonization of workflows and the establishment of a unified workflow that considers all clinical sites and their independent workflows.
247 | P a g e
References: Doebbeling B, Burton M, Kaufman D, Poterack K, McCullough M, Grando A. Integrated Workflow Capture in an EHR Conversion: Standardizing on Best Practice Methods. American Medical Informatics Association. 2016;
Disrupt and Adjust: Insights from a Contextual Inquiry of A Novel Documentation and Workflow Tool for Inpatient Providers
--Subha Airan-Javia
S25-3: Your hospital identifies a safety event in which an immunocompromised patient with recent history of extended-spectrum beta-lactamase resistant E. coli bacteremia was not started on sufficiently broad antibiotics when she developed a fever overnight, and effective antibiotics were not instituted until the error was caught 18 hours later. During root cause analysis, your team identifies that the patient’s progress notes made clear that the patient should be started on vancomycin, imipenem, and amikacin in the setting of a new fever, however the written handoff did not include this information.Which of the following systems would be most likely to prevent this error?
A. Instituting a policy in which all febrile, immunocompromised patients are immediately transferred to the intensive care unit.
B. An easily accessible documentation template for fever in immunocompromised patients.
C. Educating providers about the importance of situational awareness in the written handoff.
D. Populating the progress note & written handoff from a single source of truth
Answer: D. Populating the progress note & written handoff from a single source of truth
Explanation: In addition to writing and updating admission notes, progress notes, and discharge summaries, front line clinicians (FLCs) must also update plans of care and problem lists in their handoffs. Often, these documents do not communicate with each other, forcing FLCs to maintain each document independently. As length of stay for an individual patient and team census/workload increases, progress notes and handoffs are known to drift away from each other in content and decrease in overall quality (1). In the absence of links from information in the electronic health record (EHR) to elements of the written handoff, inconsistencies are known to occur for the list of problems, code status, medications, and allergies (2-4). In this scenario, the key piece of information that was lost between documents (a contingency plan in case this patient developed a fever) is unlikely to be captured discretely in the EHR. Thus, a unified documentation system that populates both the progress note and the written handoff from a single source of truth would most likely have avoided this error since the FLC would only have had to make an update in one location instead of two.
Instituting a policy requiring transfer of febrile, immunocompromised patients to the intensive care unit would have introduced another handoff and required substantial time to accomplish. The intensive care unit clinicians would still have needed to have read the most recent progress note in addition to any transfer notes or the handoff in order to have caught this mistake, which may not be part of their
248 | P a g e
workflow. Thus, this approach is not likely to prevent this error in addition to leading to many potentially unnecessary intensive care unit admissions.
While a documentation template for fever in immunocompromised patients can provide guidance to FLCs to include explicit fever plans, the problem in this case is not that the patient had no fever plan. Rather this patient’s individualized, unusual fever plan was not copied over from the progress note to the handoff. Thus, a documentation template alone would probably not have prevented this error.
While it is certainly important to educate providers about the importance of situational awareness in the written handoff, high census, individual patient complexity, and increased length of stay make it increasingly difficult for FLCs to keep all inpatient documents updated and high quality. Thus, education alone is unlikely to prevent the error seen in this patient without technological solutions that streamline workflow and decrease the documentation burden (2).
References:
1. Miller DM, Schapira MM, Visotcky AM, et al. Changes in written sign-out composition across hospitalization: Handoffs and Written Sign-out Quality. J Hosp Med 2015;10:534–6. doi:10.1002/jhm.2390
2. Airan-Javia SL, Kogan JR, Smith M, et al. Effects of Education on Interns’ Verbal and Electronic Handoff Documentation Skills. J Grad Med Educ 2012;4:209–14. doi:10.4300/JGME-D-11-00017.1
3. Aylward MJ, Rogers T, Duane PG. Inaccuracy in Patient Handoffs: Discrepancies between Resident-Generated Reports and the Medical Record. Minn Med 2011;94:38–41.
4. Derienzo C, Lenfestey R, Horvath M, Goldberg R, Ferranti J. Neonatal intensive care unit handoffs: a pilot study on core elements and epidemiology of errors. J Perinatol. 2014;34(2):149–152.
S25-4: A healthcare organization aims to increase the number of problems added to the problem list in the inpatient setting. They design a system that forces clinicians to enter each problem on the problem list before being able to document the plan related to that problem. Which of the following unintended consequences is MOST likely?
A. Clinicians will document multiple problems under a single problem heading.
B. Case-mix index will go down, leading to reduced hospital billing.
C. Clinicians will enter fewer problems on the problem list for each patient.
D. The hospital will be at higher risk of Medicare fraud from inaccurate problem lists.
Answer: A. Clinicians will document multiple problems under a single problem heading.
Explanation: In addition to simplifying administrative efforts, quality improvement, and billing, accurate problem list documentation improves patient outcomes (1). Forcing clinicians to enter a problem on the problem list before being able to document a plan for that problem, mandatory problem-oriented
249 | P a g e
charting, is the most effective method for encouraging problem list utilization (2). While this approach is highly effective for this purpose, clinical informaticists should be aware that such rigid structures and forcing functions may aggravate clinicians due to limited expressivity (3–5). This frustration often leads clinicians to document multiple problems under a single problem heading in order to gain efficiency from free-text data entry (6), which can reduce overall data quality.
Mandatory problem-oriented charting increases problem list completeness [Wright, Stanford], which is unlikely to reduce the number of problems entered per patient or reduce the case-mix index. There is no evidence that clinicians enter problems that do not exist on the problem list that could lead to Medicare fraud.
References:
1. Hartung DM, Hunt J, Siemienczuk J, Miller H, Touchette DR. Clinical implications of an accurate problem list on heart failure treatment. J Gen Intern Med. 2005 Feb;20(2):143–7.
2. Wright A, McCoy AB, Hickman T-TT, Hilaire DS, Borbolla D, Bowes WA, et al. Problem list completeness in electronic health records: A multi-site study and assessment of success factors. Int J Med Inf. 2015 Oct;84(10):784–90.
3. Rosenbloom ST, Denny JC, Xu H, Lorenzi N, Stead WW, Johnson KB. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc. 2011 Mar 1;18(2):181–6.
4. Wright A, Maloney FL, Feblowitz JC. Clinician attitudes toward and use of electronic problem lists: a thematic analysis. BMC Med Inform Decis Mak. 2011;11(1):1.
5. Hodge CM, Narus SP, Hodge C. Electronic problem lists: a thematic analysis of a systematic literature review to identify aspects critical to success. J Am Med Inform Assoc. 2018.
6. Li RC, Garg T, Cun T, Shieh L, Krishnan G, Fang D, et al. Impact of problem-based charting on the utilization and accuracy of the electronic problem list. J Am Med Inform Assoc. 2018.
250 | P a g e
S26: Presentations - Workflow Enhancements for the HER
Improving Point of Care Information Workflow with EHR Embedded Search
--Bhrandon Harris
S26-1: On average, what is the number of unanswered clinical questions that clinicians have after a patient encounter?
A. 0.25 questions per encounter
B. 0.5 questions per encounter
C. 1 question per encounter
D. 2 questions per encounter
Answer: B. 0.5 questions per encounter Explanation: According to Del Fiol, et al, 2014, clinicians typically have an average of 0.57 (95% CI, 0.38-0.77) unanswered clinical questions per patient seen. This is based on a meta-analysis of 72 articles investigating information seeking practices of clinicians with regards to clinical encounters. Reference: Del Fiol G, Workman TE, Gorman PN. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA internal medicine. 2014 May 1;174(5):710-8. S26-2: What are the two most commonly cited reasons that clinicians do not pursue answers to clinical questions that have arisen during an encounter?
A. Lack of time and forgetting the question
B. Forgetting the question and a lack of urgency of the question
C. Lack of time and a lack of urgency of the question
D. Lack of time and doubt that an answer exists
E. Forgetting the question and viewing question as unimportant
Answer: D. Lack of time and doubt that an answer exists Explanation: While clinicians typically have an average of 0.57 (95% CI, 0.38-0.77) unanswered clinical questions per patient seen, about 50% of those questions are never pursued. The most common reason cited in the literature is lack of time followed by doubt that an answer exists. Of the 50% of clinical questions that are pursued, clinicians are able to find an answer in 78% (67%-88%). However, this means that a majority of clinical questions that exist after patient encounters are not pursued or resolved. Reference: Del Fiol G, Workman TE, Gorman PN. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA internal medicine. 2014 May 1;174(5):710-8.
251 | P a g e
Integration of Patient Controlled Analgesic Infusion Forms into the Electronic Medical Record (the eAIF Project).
--MaryAnn Connor S26-3: Opioids are a class of medication that confer great and immediate benefit but also carry significant immediate and long-term risks. As pain increases and doses escalate, side effects ensue, and an opioid rotation becomes necessary. What is the best method that ensures opioid conversion and rotation are done safely?
a. Converting between opioids by hand calculation can be done in a standardized manner by clinicians b. Electronic calculations performed using dedicated smartphone applications, or online resources
c. Use of a consistent conversion algorithm, and automated calculation embedded within the electronic medical record. d. Conversion first to morphine, a universal opioid, and then secondary conversion to another opioid. Answer: c. Use of a consistent conversion algorithm, and automated calculation embedded within the electronic medical record. Explanation: While converting between opioids by hand calculation can be done in a standardized manner by clinicians, it may cause errors, potentially leading to under- or over- dosing of patients. Electronic calculations performed using dedicated smartphone applications, or online resources are more accurate, but each clinician may use a different resource, all of which may employ formulas or conversion ratios Linking opioid conversion calculations directly to the EMR permits greater standardization, accuracy, and control over opioid rotations. All doses administered are accurately counted, and the algorithm used throughout the hospital can be consistently applied. References: Durham. M. et al (2017). Addressing safe opioid monitoring practices using an interprofessional approach. JONA, 47(11), 537-544. Reis-Pina, P., Lawlor, P., & Barbosa, A. (2017). Adequacy of cancer-related pain management and predictors of undertreatment at referral to a pain clinic. Journal of Pain Research, 10, 2017-2107. S26-4: The Rapid Application Development (RAD) model should be used in software development: a. To address gaps by disruption to remove the status quo legacy system and build from the ground up focusing only on fundamentals. b. Only when you are unable to use the preferred approach of Structured Systems Analysis and Design Method (SSADM). c. When there is limited time and availability for users (e.g., physicians and nurses) to provide input in the design.
d. To allow users to interact with systems analysts and develop models and prototypes that represent all system processes, inputs, and outputs using a multidisciplinary team with the model changing as stakeholders weigh in.
252 | P a g e
Answer: d. To allow users to interact with systems analysts and develop models and prototypes that represent all system processes, inputs, and outputs using a multidisciplinary team with the model changing as stakeholders weigh in. Explanation: SSADM covers those aspects of the life-cycle of a system from the feasibility study stage to the production of a physical design; it is generally used in conjunction with other methods, such as PRINCE, which is concerned with the broader aspects of project management. SSADM sets out a cascade or waterfall view of systems development, in which there are a series of steps, each of which leads to the next step. The Rapid Application Development (or RAD) model is based on prototyping and iterative model with no (or less) specific planning. In general, RAD approach to software development means putting lesser emphasis on planning tasks and more emphasis on development and coming up with a prototype. In disparity to the waterfall model, which emphasizes meticulous specification and planning, the RAD approach means building on continuously evolving requirements, as more and more learnings are drawn as the development progresses. Reference: McGonigle, D and Mastrian, K. (2012). Nursing Informatics and the Foundation of Knowledge. Burlington, MA. Jones and Bartlett Learning Witten, J. L., & Bentley, L. D. (2007). Systems Analysis & Design Methods. New York, NY: McGraw-Hill Companies, Inc. Preoperative Nursing Workflow and Problem Solving
--Lu Zheng S26-5: A health care system wishes to reduce the cognitive burden of clinicians working in preoperative care to ensure the quality of patient-care delivery and enhance the patient safety. However, the clinicians are not typically trained in human factors or cognitive science. They may not be able to pinpoint the sources of burden. Which method is best suited to characterize cognitive burden in clinical work?
A. Semi-structured interview
B. Video-screen capture of the EHR use
C. Ethnographic study of workflow
D. Think-aloud protocol
E. A multi-method approach that triangulates evidence from different sources.
Answer: E. A multi-method approach that triangulates evidence from different sources. Explanation: Cognitive informatics looks at the mental process and decision-making method when a clinician uses the electronic health record system. The cognitive burden appears when the workflow or environment necessitates that clinician has to commit information to memory, perform duplicate work, and engage in frequent multitasking. It is difficult to capture that information in a survey. The screen capture of the EHR use can partially reflect the complexity of interacting with a difficult system, but may
253 | P a g e
not fully reflect the clinicians’ personal mental burden. The Think-Aloud protocol inspires the individual to express what’s on their mind as they encounter particular difficulties. Ethnography of workflow allows researchers to trace the trajectory of problems, for example, the bottleneck in communications, difficult interfaces and unmet information needs. The combination of methods can best identify and characterize the sources of cognitive burden and can be used to fashion solutions. Reference: Grando A, Groat D, Furniss SK, Nowak J, Gaines R, Kaufman, DR, Poterack K, Miksch T, Helmers R. Using Process Mining Techniques to Study Workflows in a Pre-operative Setting. AMIA Annual Symp 2017, November 4-8, 2017. S26-6: The PreOp nurse needs to help coordinate the activities of several surgery team members, such as the anesthesiologist, the resident, the physician assistant or the surgeon, when they stop by and talk to the patient during the PreOp assessment process. The nurse needs to seamlessly and efficiently communicate to clinicians that some action is needed before the patient can be transferred to the OR. Which strategy do you think is the best way to handle this coordination process with minimal disruption given the constraints of the 30 minutes Preop assessment?
A. Write it on the blue sheet (checklist) and provide access to all clinicians.
B. Use a symbolic system (e.g., color coded magnetic strips on the exterior of the patient room) that signals to team members that certain actions are needed.
C. Call, email or text the team member before they come to see the patient, so that they will know before they enter the room.
D. Wait until they come and talk to the member directly at the patient bedside.
Answer: B. Use a symbolic system (e.g., color coded magnetic strips on the exterior of the patient room) that signals to team members that certain actions are needed. Explanation: Coordination in a short period of time given a multiplicity of patient needs is challenging. Reaching the right person without extraneous interruptions and actions needs a coordinating strategy. Incorporating the artifacts and tools in the room environment to remind people is straight-forward, requires minimal effort and is less disruptive than the other methods. Reference: Furniss SK, Burton MM, Larson DW, Kaufman DR. Modeling Patient-Centered Cognitive Work for High-Value Care Goals. Proceedings International Symposium on Human Factors and Ergonomics in Health Care 2016 Jun (Vol. 5, No. 1, pp. 112-119). Sage India: New Delhi, India: SAGE Publications.
S27: Panel - Clinical Informatics Efforts to Combat the Opioid Crisis – Experiences from the Front Line
Barry Blumenfeld, Patrick Burns, James Huizenga, Steven Labkoff, Despina-Maria Michaels
S27-1: What major event transpired in 2016 that had a material effect on how primary care doctors think about prescribing opioid analgesics:
254 | P a g e
A. The Trump Administration published a new law on the limits on daily opioid MMEs
B. The CDC published guidelines to inform primary care and other clinicians on the most appropriate way to prescribe opioids
C. The FDA provided new guidance to prescribe abuse deterrent opioids whenever possible
D. A new formulation of an abuse deterrent opioid was approved for sale and use in the US
Answer: B. The CDC published guidelines to inform primary care and other clinicians on the most appropriate way to prescribe opioids
Explanation: In March 2016, the CDC announced is Guideline for Opioid use for primary care clinicians. This provided 12 specific action points within the guideline for best practices in writing for opioid analgesics.
A is incorrect. While the Trump Administration is attempting to deal with the opioid epidemic in a number of ways, no new laws were passed in 2016 concerning the opioid epidemic. The Christi Commission was empaneled in late 2017 to come up with recommendations concerning ways that the US government can help with the opioid epidemic.
C is incorrect. The first abuse deterrent opioid was approved for use in the US in 2010. However, the FDA has never issued guidance requiring the use of such formulations. That choice remains with the clinician.
D is incorrect. While the first abuse deterrent opioid was approved in 2010, others have followed since. The latest abuse deterrent opioid analgesic was approved in early 2018.
Reference: https://www.cdc.gov/drugoverdose/prescribing/guideline.html
S27-2: With regard to the opioid crisis, the use of big data analytics is currently in use to:
A. Identify clinicians who are appropriately writing opioid prescriptions in order to single them out as good practice examples
B. Identify patients at risk for opioid overdose by use of a predictive model
C. Enhance the Prescription Drug Monitoring Programs (PDMP’s), which is a requirement in all states
D. Support integrated systems, which are now wide-spread in Health Information Exchanges across the US to help deter inappropriate opioid prescribing
Answer: B. Identify patients at risk for opioid overdose by use of a predictive model
Explanation: A is incorrect: There are predictive models currently in use today to identify doctors (and patients) who are inappropriately prescribing opioids in a way that could put the patient at risk. Some of those issues are concomitant use with benzodiazepines or muscle relaxants.
255 | P a g e
B is correct: There are at least two companies who are marketing and implementing solutions that identify patients at risk for opioid overdose during the prescribing moment inside the EHR.
C is incorrect: The advent of PDMPs started about 10-15 years ago. While they are tools in 49 of 50 states, they are not legally required to be used in all jurisdictions.
D is incorrect: Only a handful of IDNs and ACOs currently are leveraging predictive analytics for dealing with the opioid crisis, but the number is growing.
References: www.ApprissHealth.com & www.PrincipledStrategies.com
S27-3: With regard to the Opioid Crisis, Clinical Decision Support tools are
A. Currently in wide scale use to deal with the opioid crisis
B. Currently not showing significant impact in cutting down the rate of opioid prescribing
C. Can bring together natural triggers inside the clinical encounter to present in-the-moment guidance
D. Generally easy to implement across a wide swath of the US due, in part, to the adoption of EHRs in the last 5 years.
Answer: C. Can bring together natural triggers inside the clinical encounter to present in-the-moment guidance on everything from prescribing guidelines to dosage recommendations
Explanation: A is incorrect: Though CDS has been shown to have a positive effect on things like opioid prescribing, these tools are not yet deployed at large scale across the US at this time, though that is changing.
B is incorrect: There are at least two recent studies, Univ of Penn and Purdue Pharma/Practice Fusion that have shown good results of CDS implementations on curbing the use of opioids in Chronic and Acute pain situations.
C is correct: CDS tools are beginning to be deployed in EHRs across the US. They are being shown to have a significant impact on how doctors prescribe opioids
D is incorrect, though EHR adoption is now over 95% hospitals and over 85% in the outpatient setting, implementing CDS in large health systems remains difficult because systems vary so greatly. As there is currently limited uptake of CDS standards in EHRs, a write once/use many approach to CDS remains elusive in most of the US. Though EHRs that rely on individual implementations can and do implement CDS, getting the same tools uniformly deployed remains a challenge.
256 | P a g e
S27-4: The opioid crisis presents challenges for the entire healthcare ecosystem. Specifically, the use of informatics tools with regards to the role of primary care clinicians should be to:
A. Do primary research to identify informatics tools that can be used to curb the opioid crisis
B. Provide specific data on every patient that has an encounter to ensure that registries and databases have adequate information to draw upon when making policy decisions around opioid use
C. Devote a substantial part of their practice to work with research efforts from organizations like SAMHSA, NIDA, and APA to help curb the opioid crisis
D. Leverage as many existing tools as is practical from their HER vendors to integrate appropriate screening and documentation during routine clinical care
Answer: D. Leverage as many existing tools as is practical from their HER vendors to integrate appropriate screening and documentation during routine clinical care
Explanation:
A is incorrect: While it would be wonderful if every clinician were to take part in clinical research, it is impractical for most PCPs to work on research when seeing their patient panels. An occasional practice might want to do some research, but it is unrealistic for most PCPs to create and research new informatics tools.
B is incorrect: While the accumulation of specific data for research purposes is critical to successfully fighting the opioid crisis, specifically using the concept of a registry in most practices remains enticing, but not yet practical.
C is incorrect: While working with the likes of NIDA and SAMHSA is appealing, most primary care clinicians simply do not have the time to devote to this research.
D is correct: From a practical perspective, leveraging a practices EHR infrastructure through the use of order sets, guidelines, and clinical decision support, a local practice can and should leverage all the tools that it has at its disposal – such as PDMP integration, predictive models, and EHR-created order sets in the course of their clinical practice to ensure safe and effective use of opioids.
257 | P a g e
S28: Presentations - The Learning Health System, Optimization and Leveraging Technology - All to Improve Care
Implementation of a Learning Healthcare System for Sickle Cell Disease
-- Erin Crowgey
S28-1: A division within a health care system wishes to improve the meaningful use of data entered into the electronic healthcare records (EHR). The majority of EHR systems handle coding and liability issues well, but are not robust for clinical research and data visualization at the bedside. Which of the following options is technologically possible to enhance the downstream use and analysis of data entered directly into the EHR?
A. Have providers enter more data via free text in note sections.
B. Encourage providers to work independently on improving data entry into EHR.
C. Create a team consisting of providers and health informaticians to implement a process that supports structured data entry at the bedside.
D. Request that the health informatics department implement solutions per each individual clinician.
E. Enhance the use of IC-9/10 coding for data analysis techniques.
Answer: C. Create a team consisting of providers and health informaticians to implement a process that supports structured data entry at the bedside.
Explanation: A multi-disciplinary team that bridges the gaps between clinical data entry and downstream use of EHR data, is required to improve the meaningful use of EHR data. Clinicians and health informaticians need to work in collaboration to improve clinical workflows. Structured data entry at the bedside is required for robust downstream analysis.
Reference:. Bush RA, Kuelbs C, Ryu J, Jiang W, Chiang G. Structured Data Entry in the Electronic Medical Record: Perspectives of Pediatric Specialty Physicians and Surgeons. J Med Syst. 2017 May;41(5):75. doi: 10.1007/s10916-017-0716-5. Epub 2017 Mar 21
S28-2: A common data model (CDM) enables a systematic analysis of disparate databases. The Observational Medical Outcomes Partnership (OMOP) is one type of CDM that transforms data contained within EHR databases into a common format, or data model, with common representation of the data elements, terminologies, vocabularies, and coding schemes. Which of the following options is solution for a department looking to enhance the meaningful use of EHR data?
A. Recommend providers focus on analytical techniques that can only be deployed directly in EHR system and that are independent of any data transformation processes.
B. Create a custom CDM, or use a publicly available model, to map EHR data into a standardized format.
258 | P a g e
C. Require health informaticians to analyze EHR independently and provide PDF reports to providers.
D. Have providers within a department create their own individual lists of important data elements, and capture them inside excel.
E. Recommend to the clinicians within the department that this is a health informatics issue and does not require their effort or input.
Answer: B. Create a custom CDM, or use a publicly available model, to map EHR data into a standardized format.
Explanation: Common data models provide a mechanism of standardizing data entry at the bedside. Expert clinicians across institutions should decide on key data elements required to improve patient care, and work closely with informaticians for transforming their data dictionary into a CDM.
Reference: Maryam Garza, Guilherme Del Fiol, Jessica Tenenbaum, Anita Walden, Meredith Nahm Zozus. Evaluating common data models for use with a longitudinal community registry. Journal of Biomedical Informatics. Volume 64, December 2016, Pages 333-341. https://doi.org/10.1016/j.jbi.2016.10.016
The Veterans Health Administration’s Reproductive Health Project
--Caitlin Cusack
S28-3: Clinical Decision Support Systems (CDS) within Electronic Health Records (EHR) to reduce medication errors are widely used. Medication CDS systems range from basic systems that conduct drug-allergy, drug-interaction, and duplicate drug checking. More advanced systems incorporate data from the EHR such as conditions, laboratory results, and gender. CDS that extends to drug-pregnancy checking have limited clinical usefulness because of the lack of specificity of these alerts. One practice used with these alerts that does not improve their clinical relevancy is:
a. Remove from the cohort for rule-checking those women who have system evidence an inability to conceive, such as those with coding consistent with hysterectomy or menopause.
b. Limit alerts for women using highly effective contraceptive or those who are unable to conceive with their current partner.
c. Build a rules engine that utilizes medications classified with the former FDA categories of C, D, or X.
d. Capture as discrete data pregnancy status including “yes,” “no” or “unknown”.
Answer: c. Build a rules engine that utilizes medications classified with the former FDA categories of C, D, or X.
Explanation: The FDA moved away from pregnancy categories due to the failure of the older lettering system to correlate with available evidence regarding the safety of these medications in pregnancy.
259 | P a g e
With many medications classified as being less safe in pregnancy than research has shown, relying on the old classification system results in numerous clinically irrelevant alerts. (a), (b) and (d) are incorrect because all help to narrow the cohort of women these rules are run against. Many EHRs do not capture pregnancy status as discrete data and thus most systems that conduct drug-pregnancy testing do so against the cohort of women of reproductive age. By capturing pregnancy status and ability to conceive, along with identifying those women using highly effective forms of contraceptive, the cohort is narrowed allowing greater clinical relevance of alerts.
References: Schwarz EB, Parisi SM, Handler SM, et al. Clinical decision support to promote safe prescribing to women of reproductive age: a cluster-randomized trial. Journal of general internal medicine. 2012 Jul 1;27(7):831-8.
Kuperman, Gilad J., et al. "Medication-related clinical decision support in computerized provider order entry systems: a review." Journal of the American Medical Informatics Association 14.1 (2007): 29-40.
S28-4: At the Veteran’s Health Administration (VHA), the Reproductive Health project uses logic on the backend to classify women as pregnant, medically unable to conceive, low likelihood of becoming pregnant, high likelihood of becoming pregnant, and unknown status. The due date of the VHA’s Reproductive Health (RH) Clinical Reminder Dialog is based on these classifications. Which of the following women is not due now to have the RH dialog completed by her physician?
a. Ms. Smith age 38 is new to the VHA, uses an IUD for contraception, and presents for her first preventive care visit.
b. Ms. Jones has a recorded estimated due date (EDD) in the system from 4 weeks ago, but has not been seen within the VHA during her pregnancy.
c. Ms. Noonan has a history of a hysterectomy that has been recorded in the system.
d. Ms. Johnson has a completed RH dialog from 2 months ago. She is using Depo-Provera for contraception, but is late for her next injection.
Answer: c. Ms. Noonan has a history of a hysterectomy that has been recorded in the system
References:
Schwarz EB, Postlethwaite DA, Hung Y-Y et al. Documentation of contraception and pregnancy when prescribing potentially teratogenic medications for reproductive-age women. Ann Intern Med. 2007;147(6):370-376.
Schwarz EB, Longo LS, Zhao X, etal. Provision of potentially teratogenic medications to female veterans of childbearing age. Medical care. 2010 Sep 1:834-42.
260 | P a g e
Leveraging Technology to Reduce Unnecessary Post Procedure Office Visits for Chemo Port Placement
--Christine VanZandbergen
S28-5: A provider wishes to replace a post-procedure in-person follow-up visit with a virtual visit. Assuming the technology is in place to support virtual workflow, what is the most appropriate next step? A. Tell all patients to register for a patient portal account and start sending their wound sites photos. B. Engage the appropriate clinical, regulatory and information systems partners to validate the workflow and ensure appropriate support/monitoring. C. Tell patients to take a photo of their wound and text it to you D. Set up the visits as billable encounters Answer: B. Engage the appropriate clinical, regulatory and information systems partners to validate the workflow and ensure appropriate support/monitoring. Explanation: Post-procedure in-person follow-up visits may be able to be replaced by virtual visits provided the right infrastructure is in place. A critical success factor is ensuring the workflow has been fully defined and vetted with appropriate stakeholders to support these initiatives. Reference: Watkins, M. Making Virtual Teams Work: Ten Basic Principals. Harvard Business Review, June 27, 2013 S28-6: Which of the following is a potential benefit of replacing post-procedure follow-up visits with virtual visits? A. Increased revenue through new billable services B. Decreased infection rates C. Improved follow-up with post-procedure wound checks D. Increased provider satisfaction Answer: C. Improved follow-up with post-procedure wound checks Explanation: There are several benefits of using virtual visits to evaluate patients post port insertion. These benefits include increased compliance with follow up visits, patient time savings and increased availability of clinic appointments for other patients.
261 | P a g e
Reference: Palen et al. Comparing virtual consults to traditional consults using an electronic health record: an observational case-control study. BMC Medical Informatics and Decision Making, 2012, 12:65
S29: Presentations - Translating Clinical Decision Support to Practice
Translating evidence into practice: Effect of an Evidence-based Electronic Clinical Decision Support Tool on Provider Ordering for Venous Catheters
--Emilia Flores
S29-1: The 5 Rights for clinical decision support (CDS) interventions (right information, to the right person, in the right format, through the right channel, at the right time in the workflow) are a framework for designing and deploying CDS interventions. Catheter-associated urinary tract infection (CAUTI) is a common device-associated infection in the acute care setting. However, misdiagnosis of catheter-associated asymptomatic bacteriuria as CAUTI can lead to inappropriate use of antibiotics and related consequences, including medication adverse effects and antibiotic resistance. Appropriate urine culturing is dependent on a clinician’s ability to recognize a specific set of symptoms. You notice an increase in inappropriate urine culture ordering in your organization and wish to implement a CDS to ensure that urine culturing is aligned with evidence-based recommendations. Which CDS type would be the most appropriate for this clinical issue?
A. A predictive analytics tool to identify patients at high risk for CAUTI
B. A report containing a summary of patients that meet culturing criteria for CAUTI is available in a reporting module or tab and is accessible to all clinicians
C. An order set that leads the provider through a series of questions based on CAUTI culturing criteria and provides recommendations based on provider responses
D. Order set that defaults to a reflex-urine culture order if the patient has a urinary catheter. A reflex urine culture order is one in which the lab will first conduct a urine analysis and if criteria are met (e.g. urine analysis with >= 10-20 white blood cell count/hpf), the lab will then perform a urine culture to assess for CAUTI.
Answer: D. Order set that defaults to a reflex-urine culture order if the patient has a urinary catheter. A reflex urine culture order is one in which the lab will first conduct a urine analysis and if criteria are met (e.g. urine analysis with >= 10-20 white blood cell count/hpf), the lab will then perform a urine culture to assess for CAUTI.
Explanation: In this case, the decision support is presented to the right person (ordering provider), is embedded within provider workflow (point of order entry) and uses objective criteria to assess whether a urine culture should be performed. By using patient data (presence of a urinary catheter) to default to a reflex urine culture order, this design removes the cognitive burden from the provider in having to remember appropriate culturing criteria. By moving the culturing decision to laboratory services, this further reduces the possibility of inappropriate culturing.
262 | P a g e
Option A is a possible option, but this information doesn’t necessarily use evidenced-based information for culturing. Risk information for CAUTI might also be difficult for providers to interpret.
Option B is a possible option, but this report isn’t embedded in a provider workflow for assessing this patient.
Option C is a possible option, but this relies on a provider to enter patient data into the order entry screen.
References:
1) Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, Khorasani R, Tanasijevic M, Middleton B. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. Journal of the American Medical Informatics Association. 2003;10(6):523-30.
2) Osheroff JA. Improving outcomes with clinical decision support: an implementer's guide. HIMSS.
S29-2: Zhang and Walji (2011) have proposed 14 evidence-based interface design principles that are important to consider for any electronic health record (EHR) embedded clinical decision support (CDS), as they can have a significant impact on user acceptance and satisfaction. (Middleton et al., 2013) Which of the following interface features is an example of a design principle?
A. A CDS tool to promote antibiotic stewardship asks users to manually type in the patient diagnosis associated with a medication order rather than select from a drop-down list
B. A CDS tool for vascular access selection displays relevant patient information and calculations within the CDS interface to aid in decision making, thus avoiding having the user reference and remember information from other screens.
C. A CDS development group organizes and displays data entry fields in all CDS tools alphabetically
D. A CDS tool for vascular access selection utilizes drop down lists rather than check boxes for the five most common medications infused through venous access devices
Answer: B. A CDS tool for vascular access selection displays relevant patient information and calculations within the CDS interface to aid in decision making, thus avoiding having the user reference and remember information from other screens.
Explanation: Research has shown that short-term memory is limited (Miller, 1956). Reduce potential errors, completion time, and improve efficiency by avoiding having users remember information from other screens or having them perform calculations. Design Principle #5 [Memory]
Option A: this design relies on users having to remember diagnosis. Design principle #5 [Memory] states that designs should minimize the amount of information that users have to recall from memory. A better design would be to list potential patient diagnoses in a drop-down list.
263 | P a g e
Option C: Design principle # 3 [Match] states that the system should match user workflow in the real-world setting and displaying information in an alphabetical order does not necessarily match the real-world cognitive and decision-making process. A better design approach would be to understand user workflow and the decision-making process and order fields (whether data entry or displayed patient information) using this structure.
Option D: Design principle # 7 [Flexibility and efficiency] states that designs should use the fewest number of steps to complete the task. In this case, since there are few predefined medications, using checkboxes verses drop-down lists results in fewer clicks for the user.
References:
1) Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, Payne TH, Rosenbloom ST, Weaver C, Zhang J. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. Journal of the American Medical Informatics Association. 2013;20(e1):e2-8.
2) Zhang J, Walji MF. TURF: Toward a unified framework of EHR usability. Journal of Biomedical Informatics. 2011;44(6):1056-67.
Clinical Pathway Regret Risk Differences Between Affluent and Underserved Populations
--Robert Aarhus
S29-3: Decision Regret is a well-studied patient-centered outcome measuring the quality of the medical decision-making process. Using validated query instruments, researchers have determined key characteristics of decision regret in attempts to understand, retrospectively, the origin and trajectory of patient dissatisfaction with those decisions that have produced disappointing or unexpected healthcare results. In particular, many randomized controlled trials have examined effects of enhanced patient information, or conformity to the level of physician participation in the process expected by patients. According to the literature, which of the following is a characteristic of treatment decision regret?
A. It is negatively correlated with decisional conflict
B. It often increases over time in breast cancer cases
C. It sometimes differs according to the patient's race and insurance status
D. It is uniformly decreased through the use of decision aids
E. It is seen in a majority of complex cancer cases
Answer: C. It sometimes differs according to the patient's race and insurance status
Explanation: Decision regret is a complex, post-decision cognitive process with many facets that must be taken into consideration if a predictive model of regret is to be constructed. Decisional conflict -- a pre-decisional state of lack of decision clarity because of too little, too much, or improperly presented
264 | P a g e
alternatives, to name a just few causes -- is positively correlated with decision regret, and is largely a predictor of the degree to which patients are dissatisfied with outcomes. The stability of decision regret over time has been examined and, in the case of breast cancer specifically, determined not to change longitudinally for most patients. This stands in contrast to disease processes such as prostate cancer, where various studies have indicated that decision regret increases over time, an important factor which must be accounted for in the model. Decision regret has been shown to differ according to race and insurance status, suggesting the need for a deeper look into the institutional systems to identify the origin of these effects. Many decision aids have been evaluated in randomized controlled trials, and while some have demonstrated a reduction in decision regret, other studies have been equivocal, or actually show an increase in patient dissatisfaction. While decision regret can be pronounced in certain disease processes like prostate and head and neck cancers, overall the frequency of decision regret is fairly low, often not exceeding a quarter of all patients evaluated.
References:
1. Hu JC, Kwan L, Krupski TL, Anger JT, Maliski SL, Connor S, et al. Determinants of Treatment Regret in Low-Income, Uninsured Men With Prostate Cancer. Urology. 2008;72(6):1274–9.
2. Collingwood SA, McBride RB, Leapman M, Hobbs AR, Kwon YS, Stensland KD, et al. Decisional regret after robotic-assisted laparoscopic prostatectomy is higher in African American men. Urol Oncol Semin Orig Investig. 2014;32(4):419–25.
3. Brehaut JC, O’Connor AM, Wood TJ, Hack TF, Siminoff L, Gordon E, et al. Validation of a decision regret scale. Med Decis Mak. 2003;23(4):281–92.
4. Martinez KA, Li Y, Resnicow K, Graff JJ, Hamilton AS, Hawley ST. Decision Regret following Treatment for Localized Breast Cancer: Is Regret Stable Over Time? Med Decis Mak. 2015;35(4):446–57.
S29-4: To prospectively assess the likelihood of future regret, a model has been developed to include estimates of regret risk. A novel term in this context, regret risk is a statement of the likelihood of future decision regret driven by both differences between patient expectation and outcome, as well as the curse of the counterfactual narrative -- a seemingly certain and more positive alternative future that might have happened had past choices been different.Key assumptions, based on decision regret research, have driven regret risk formulation and expression. According to the presented model's hypothesis, which of the following could explain differences in long-term cancer regret risk between socioeconomic groups?
A. An emphasis on aleatory rather than epistemic uncertainty
B. A change in action space
C. Linear dependence on measurable decision regret
D. Cultural differences between patients and health care providers
Answer: B. A change in action space
265 | P a g e
Explanation: Keeping in mind the aphorism "all models are wrong; some models are useful," the art and science of model building is largely an exercise in deciding what and how to represent certain phenomena, while deliberately choosing to exclude others. The roles of aleatory (random) and epistemic (knowledge-based) uncertainty are crucial to auguring future outcomes and anticipating potential decision regret; the model includes both factors, but makes no claim as to which predominates. The hypothesis that is to be explored by the model is the change in action space; when certain treatment pathways (such as robotic surgery) are not available to populations because of cost or geographic location, alternatives may not yield results that are as consistent, and therefore increase the likelihood of regret. Few real-world patient utility models are linear, and no such assumption is required (or needed) by the model. The model does not attempt to evaluate cultural factors of any stakeholder; if such differences do exist, they would have to be ascertained by a much different approach.
References:
1. Engelhardt EG, Pieterse AH, Han PKJ, van Duijn-Bakker N, Cluitmans F, Maartense E, et al. Disclosing the Uncertainty Associated with Prognostic Estimates in Breast Cancer: Current Practices and Patients’ Perceptions of Uncertainty. Med Decis Mak 2017;37(3):179-192.
2. Brehaut JC, O’Connor AM, Wood TJ, Hack TF, Siminoff L, Gordon E, et al. Validation of a decision regret scale. Med Decis Mak. 2003;23(4):281–92.
S30: Panel - The Use, Usability, and Experience of Implementing a Systemwide Inpatient Portal (IPP)
--Jennifer Hefner, Timothy Huerta, Ann McAlearney, Daniel Walker
S30-1: When implementing an inpatient portal that includes secure messaging between care team members and patients, which of the following should be included in front-line staff training?
A. Allow care team members to engage with the tool from the perspective of the patient.
B. Inform care team members that this is the most effective way to communicate with patients now and the health system will be using it exclusively.
C. Provide them with a training manual to read during breaks.
D. Frontline staff will not be using the tool as patients do so they do not need training in this manner.
Answer: A. Allow care team members to engage with the tool from the perspective of the patient.
Explanation: Using a qualitative approach, we found evidence that successful implementation of an inpatient portal requires training of front-line staff and nurse managers in a manner that is different from other HIT implementations. Specifically, this finds that expanded training should allow providers to engage with the tool from the perspective of patients, in addition to teaching both care team members and patients about the provider-facing, components in order to fully support optimal collaborative use. These training components are not included in the current best practices for training in EHRs and are a new consideration for this type of collaborative tool.
266 | P a g e
Reference: McAlearney, AS, Sieck, CJ, Hefner, JL, Aldrich, AM, Walker, DM, Rize,r MK, Moffatt-Bruce, SD, Huerta, TR. 2016. “High Touch and High Tech (HT2): Transforming Patient Engagement Throughout the Continuum of Care by Engaging Patients with Portal Technology at the Bedside.” JMIR Research Protocols. 2016; 5(4):e221 http://www.researchprotocols.org/2016/4/e221 doi: 10.2196/resprot.6355
S30-2: Studying patient and provider use of an inpatient portal requires analysis of log files - server-side records of actions taken by a user - to assess how, when, and which features of the inpatient portal were used. Which of the following is not an issue to consider when analyzing log files?
A. How to find the full-text of the electronic messages sent between patients and the care team.
B. The application may create administrative data (e.g., pushed data to ensure that the tablet does not display stale data) that is unrelated to patient action, resulting in artifacts in the data.
C. The application itself does not offer an indicator of a Session and as such the research team must construct an operational definition to represent a patient’s single continuous use of the tablet for the purpose of using MCB.
D. It may take several days for a patient to get their tablet, and as such length of stay may suffer from distortions associated with provisioning. As a result, patient length of stay (LOS) may not be an appropriate framing for use, because of real-world demand on provisioning practices.
Answer: A. How to find the full-text of the electronic messages sent between patients and the care team.
Explanation: The full-text of messages are recorded in the audit log files. While this is a variable within the log files, there are IRB considerations. A full IRB application will be necessary to view the full-text as these messages often contain PHI. Answer choices B-D however are issues that need to be considered and will be discussed during the panel presentation.
Reference: McAlearney, AS, Sieck, CJ, Hefner, JL, Aldrich, AM, Walker, DM, Rize,r MK, Moffatt-Bruce, SD, Huerta, TR. 2016. “High Touch and High Tech (HT2): Transforming Patient Engagement Throughout the Continuum of Care by Engaging Patients with Portal Technology at the Bedside.” JMIR Research Protocols. 2016; 5(4):e221 http://www.researchprotocols.org/2016/4/e221 doi: 10.2196/resprot.6355
267 | P a g e
S30-3: What approach(es) to learning do users prefer to familiarize themselves with the inpatient portal?
A. Free exploration
B. Video tutorial
C. In-person training
D. Blended – including free exploration, video tutorial, in-person training
Answer: D. Blended – including free exploration, video tutorial, in-person training
Explanation: Based on our formal usability study, different users expressed a preference for different approaches that would help them learn to use the inpatient portal. Some users preferred to freely explore the application, while others watched the video tutorial. While not offered in our study, some participants voiced a preference for in-person training. To accommodate these different preferences, hospitals implementing this technology may offer a blend of these approaches. This blended approach would most effectively address different user needs.
Reference:
McAlearney, AS, Sieck, CJ, Hefner, JL, Aldrich, AM, Walker, DM, Rize,r MK, Moffatt-Bruce, SD, Huerta, TR. 2016. “High Touch and High Tech (HT2): Transforming Patient Engagement Throughout the Continuum of Care by Engaging Patients with Portal Technology at the Bedside.” JMIR Research Protocols. 2016; 5(4):e221 http://www.researchprotocols.org/2016/4/e221 doi: 10.2196/resprot.6355
S30-4: Which of the following theoretic constructs is not a valid approach to measuring use of patient portal use.
A. Comprehensiveness
B. Coherence
C. Duration
D. Frequency
Answer: B. Coherence
Explanation: While there are problems measuring duration in the context of the OSU study, they may be idiosyncratic to OSUs implementation. In a system that used the standard implementation, there would be an expectation that comprehensiveness, duration and frequency would be able to be robustly measured, coherence wasn’t a measure that was discussed.
Reference: McAlearney, AS, Sieck, CJ, Hefner, JL, Aldrich, AM, Walker, DM, Rize,r MK, Moffatt-Bruce, SD, Huerta, TR. 2016. “High Touch and High Tech (HT2): Transforming Patient Engagement Throughout the
268 | P a g e
Continuum of Care by Engaging Patients with Portal Technology at the Bedside.” JMIR Research Protocols. 2016; 5(4):e221 http://www.researchprotocols.org/2016/4/e221 doi: 10.2196/resprot.6355
S31: Presentations - Analysis of Medical Data and Implication for Value-based Care
A clinical informatics consult service to use aggregate patient data at the bedside
--Saurabh Gombar
S31-1: The ideal clinical decision is supported by large multicenter randomized control trials. However, less than 20% of clinical decisions are based on RCT data. Furthermore, an estimated 20% of clinical decisions are not based on published evidence at all. Given this limitation, physicians have long wanted to untap the electronic medical record (EMR) to quickly generate practice-based evidence relevant to the clinical decision they are facing. To date which of the following has NOT been a limitation to rapidly assimilating knowledge from the EMR?
A) Tools to build a cohort of patients who faced a similar clinical scenario are not widely available.
B) The multitude of data types in the EMR (ie ICD, CPT, Notes) require physicians to be proficient with a multitude of medical nomenclature syntaxes
C) Analyzing large datasets is too time intensive for busy physicians
D) Physicians are uncomfortable using evidence generated from EMR data.
Answer: D) Physicians are uncomfortable using evidence generated from EMR data.
Explanation: The barriers to successfully utilizing the EMR to inform clinical decisions are numerous. First, EMR data for large cohort analysis is not available to the vast majority of physicians. Second, the tools to generate a cohort out of an EMR is extremely difficult and requires knowledge of database querying, esoteric medical nomenclatures, and advanced statistics in addition to clinical knowledge. Third, the time to do such an analysis is typically in the order of months, which is outside the available time for busy physicians and outside the length of time to be useful to a patient. However, physician comfort with trusting evidence generated from EMRs is not a barrier to adoption. Most large retrospective studies published in the current literature rely on database querying of the EMR for at least one step in their methodology.
Reference: Clemens SK. Kristof C. Jones B. Mitchell E. Martinez A. Barriers to electronic health record adoption: a systematic literature review. J Med Syst. 2016; 40(12) 252
S31-2: Stanford’s approach to enable precision medicine through aggregate patient data utilizes a consult service. This service is offered by a multidisciplinary team comprised of an informatics physician,
269 | P a g e
an EMR data specialist to create cohorts from the EMR, and a data scientist to perform statistical analysis. Which of the following is NOT a reason utilize a consult model to generate aggregate patient data?
A) The pathway from clinical question to useful information requires a multitude of disparate steps best handled by a team with complimentary skills.
B) The consult model is a model that physicians are comfortable getting subspecialist information from.
C) An asynchronous model allows for quick turnaround time and can be scaled to ever increasing demand.
D) A consult model is already billable as long as a note is generated for the patient.
Answer: D) A consult model is already billable as long as a note is generated for the patient.
Explanation: The consult model was chosen to deliver the aggregate patient data service for a multitude of reason. Chief among them is that the journey from clinical question to knowledge requires an array of steps in which no single party is an expert. Other reasons to use this approach include the familiarity of physicians with a consult model and its ability to scale infinitely as each element can be expanded as necessary. Although billing of a service will one day be important for widespread adoption it was not a factor when deciding which model to implement.
Reference: Longhurst, CA. Harrington RA. Shah NH. A ‘Green Button’ for using aggregate patient data at the point of care. Health Affairs. 2014 Jul; 33(7).
270 | P a g e
The Winning Health IT Engagement Model of an ACO in a Value-based Setting
--Nathan Riggle
The presenter will provide feedback on two multiple choice questions at the conclusion of the presentation.
271 | P a g e
S32: Presentations - Patient Portals: Challenges and Opportunities
Patient Portal Proxy Access for Adolescents: Preserving Confidential Care While Enhancing Parent Proxy Access
--Margaret Punch
S32-1: The parent of a 13-year-old would like full access to their child’s medical information available through an on-line portal at their local health center. What considerations should be given to this request?
a. The parent always has full rights to access the minor’s medical record.
b. Minor patients can’t consent to their own health care under any circumstances.
c. In all 50 states in the United States there are varying statutes that allow minors to seek confidential care and that information can legally be withheld from the parent or legal guardian.
d. In all 50 states in the United States 13-year-olds have the right to confidential care to access contraceptive services.
Answer: c. In all 50 states in the United States there are varying statutes that allow minors to seek confidential care and that information can legally be withheld from the parent or legal guardian.
Explanation: The legal ability of minors to access confidential care varies from state to state but all 50 states and the District of Columbia have some consent laws or statues that allow minors to seek confidential care. This is generally related to contraceptive services, STI services, prenatal care and abortion. In some states there are restrictions based on specific age, or other conditions. All medical care not covered under specific laws, statues or case law does require consent of the parent or legal guardian.
Reference: https://www.guttmacher.org/state-policy/explore/overview-minors-consent-law
S32-2: When enabling proxy access to the parent of a minor child which portal feature may need restriction?
a. Demographics
b. Vaccine records
c. Records from birth to age 10
d. Appointment schedules
e. Allergies
Answer: d. Appointment schedules
272 | P a g e
Explanation: In no states can a minor age 11 or younger consent to confidential care. Release of demographics, allergies or vaccine records would not disclose that a minor has sought confidential care, but knowledge of appointment schedules could, so appointments in areas where a minor can seek confidential care should not be displayed to a proxy-holding parent.
Reference: https://www.guttmacher.org/state-policy/explore/overview-minors-consent-law
Do Community Clinic Patients Have Internet Access for Patient Portal Use?
--Ruth Bush
S32-3: Previously identified barriers to patient portal adoption include differences in internet access, computer literacy, and internet proficiency. This digital divide is one explanation for underserved groups being less likely to use portals, with strong correlations to utilization by race and household income.
In the multi-ethnic, lower income community clinic in this study, which of the following was significantly associated with previous patient portal use?
a. Having a “smart” phone
b. Education level
c. Health status
d. Race/ethnicity
Answer: A. Having a “smart” phone
Explanation: Within this sample, the variables of race/ethnicity, gender, education level, and health status were not statistically associated with previous patient portal use. Having a smart phone was associated (p=.02) with previous use.
Reference: Gordon NP, Hornbrook MC. Differences in access to and preferences for using patient portals and other eHealth technologies based on race, ethnicity, and age: a database and survey study of seniors in a large health plan. J Med Internet Res. 2016;18(3):e50. Available at: http://www.jmir.org/2016/3/e50/. Accessed January 16, 2018
273 | P a g e
S32-4: When reporting past use as well as likely future use of the patient portal, participants reported using all of the following functions of the portal EXCEPT:
a. Contacting provider
b. Renewing prescription
c. Obtaining health information
d. Scheduling appointments
Answer: c. Obtaining health information
Explanation: Reported use varied, from looking at test results (31%); communicating securely with providers (20%); scheduling appointments (18%); and requesting prescription refills (9%). They did not report obtaining additional health information about their condition(s) using the portal. Respondents did express that they would like to do so in the future.
Reference: Griffin A, Skinner A, Thornhill J, Weinberger M. Patient portals: who uses them? what features do they use? and do they reduce hospital readmissions? Appl Clin Inf. 2016;7(2):489-501.
Development of a Taxonomy of Secure Messages from an Inpatient Portal
--Cynthia Sieck
S32-5: On what dimensions can secure messages be classified?
A. Sender, Type and Topic
B. Type, Inpatient and Outpatient
C. Urgent, Non-urgent and Medical
D. Dietary, Medical and Non-medical
Answer: A. Sender, Type and Topic
Explanations: These represent the most inclusive initial dimensions. The other responses all fit within these.
Reference: McAlearney, Ann Scheck, et al. "High touch and high tech (HT2) proposal: transforming patient engagement throughout the continuum of care by engaging patients with portal technology at the bedside." JMIR research protocols 5.4 (2016).
274 | P a g e
S32-6: Which of the following would be applications of a secure message taxonomy?
A. To improve the functionality of the secure messaging feature of the portal.
B. To help patients view their lab results.
C. To help care team prioritize responses.
D. A and C.
Answer: D. A and C
Explanation: Development of a taxonomy of secure messages does not assist patients in using other features of the tool such as viewing lab results.
Reference: McAlearney, Ann Scheck, et al. "High touch and high tech (HT2) proposal: transforming patient engagement throughout the continuum of care by engaging patients with portal technology at the bedside." JMIR research protocols 5.4 (2016).
S33: Panel - The Clinical Informatics Fellowship: A Fellow’s Perspective --Christian Dameff, William Lancaster, Chase Parsons, Melissa Van Cain S33-1: Clinical Informatics Fellowships are accredited by which organization?
A. The National Institute of Health (NIH)
B. The Accreditation Council for Graduate Medical Education (ACGME)
C. The Association of American Medical Colleges (AAMC)
D. The Center for Medicare and Medicaid Services (CMS)
Answer: B. The Accreditation Council for Graduate Medical Education (ACGME)
S33-2: Clinical Informatics Fellowship requires the completion of what primary residency?
A. Internal Medicine
B. Emergency Medicine
C. Pediatrics
D. Any ACGME-recognized primary specialty
Answer: D. Any ACGME-recognized primary specialty
275 | P a g e
S33-3: The responsibilities of Clinical Informatics fellows include:
A. Research in informatics, patient care, implementing Informatics solutions
B. Patient care, research in informatics, and working in a basic science lab
C. Implementing informatics solutions, research in informatics, and serving as a medical librarian
D. Research in informatics without any dedicated time to patient care
Answer: A. Research in informatics, patient care, implementing Informatics solutions
Explanation: A fellow in an ACGME-accredited fellowship is trained to “transform[s] health care by analyzing, designing, implementing, and evaluating information and communication systems to improve patient care, enhance access to care, advance individual and population health outcomes, and strengthen the clinician-patient relationship."
References: AMIA Clinical Informatics Fellows. https://www.amia.org/membership/acif
Shortliffe E.H., Detmer D.E., Munger B.S. (2016) Clinical Informatics: Emergence of a New Profession. In: Finnell J., Dixon B. (eds) Clinical Informatics Study Guide. Springer, Cham
S33-4: The duration of the clinical informatics fellowship is:
A. 1 year
B. 2 years
C. 3 years
D. 4 years
Answer: 2 years
Explanation: The educational program in ACGME-accredited clinical informatics (CI) fellowships must be 24 months in length.
Reference: Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Clinical Informatics (Review Committees for Anesthesiology, Diagnostic Radiology, Emergency Medicine, Family Medicine, Internal Medicine, Medical Genetics and Genomics, Pathology, Pediatrics, or Preventive Medicine). https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/381_clinical_informatics_2017-07-01.pdf
276 | P a g e
Additional details on Program Requirements for Graduate Medical Education in Clinical Informatics are located at: https://www.acgme.org/Portals/0/PDFs/FAQ/381_clinical_informatics_2017-07-01.pdf https://www.theabpm.org/become-certified/subspecialties/clinical-informatics/ http://www.abpath.org/index.php/to-become-certified/requirements-for-certification?id=40
S34: Presentations - Tool Based Analytics
Institution of a Pop Up Reminder Decreases Opioid Prescriptions for Pediatric Patients in the Emergency Department
--Melissa Van Cain S34-1: The opioid epidemic has had a serious effect on the US population in recent years and this is true of adolescent patients as well as adults. What is the approximate number of adolescents who are addicted to prescription pain relievers?
A. 1,500
B. 15,000
C. 150,000
D. 1.5 million
Answer: C. 150,000 Explanation: According to the Center for Behavioral Health Statistics and Quality, in 2014 approximately 168,000 adolescents in the US had an addiction to prescription pain relievers based on the 2014 National Survey on Drug Use and Health. That same study showed that there were more than 460,000 adolescents who were current nonmedical users of pain relievers and that most adolescents misusing opioids receive them free from a friend or relative. Reference: Center for Behavioral Health Statistics and Quality. Behavioral health trends in the United States: results from the 2014 National Survey on Drug Use and Health (HHS Publication No. SMA 15-4927, NSDUH Series H-50). http://www.samhsa.gov/data/sites/default/files/NSDUH-FRR1-2014/NSDUH-FRR1-2014.pdf. Published September 2015. Accessed March 25, 2018. S34-2: Massachusetts (MA) enacted a law in 2016 that providers are required to check the MA prescription drug monitoring program before prescribing opioids. We achieved the biggest change in opioid prescriptions with this intervention:
277 | P a g e
A. Adding a link to the chart
B. Reminder e-mails
C. Physician training
D. Pop-up reminder
E. Hard stop
Answer: D. Pop-up reminder Explanation: This project used a pop-up window to remind providers to check the Massachusetts Prescription Monitoring Program web page before prescribing opioid prescriptions to a patient. There was a significant decrease in the number of opioid prescriptions ordered after the pop-up reminder was implemented. Adding a link to the chart did not have a significant impact because it was hard to remember to click it. Reminder e-mails are not presented when the prescriber is actually ordering the medication. Physician training is important, but is also not presented at the time of the order. Hard stop was not instituted because they are very disruptive to workflow. Reference: Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005 Apr 2;330(7494):765. Epub 2005 Mar 14. Review. PubMed PMID: 15767266; PubMed Central PMCID: PMC555881.
Radiation Therapy Data Curation: Utilizing Interdisciplinary Expert Knowledge via an Iterative Tool-based Process
--Thilo Schuler
S34-3: Manual data curation is currently the most common approach to retrospectively improving data quality in clinical datasets.
Which of the following options is the greatest challenge with this approach?
A. Unsuitability for inter-institutional data exchange
B. Poor scalability
C. Poor accuracy
D. Lack of contextual knowledge
Answer: B. Poor scalability
Explanation: The biggest reason why manual curation doesn’t scale well is the need for clinical expert involvement, which is a scarce resource. The accuracy of manual approaches is typically higher as
278 | P a g e
experts use implicit contextual knowledge. Manual curation can enable inter-institutional data exchange if the curation strategy is designed for it (e.g. mapping to global standards).
Reference: Freitas A, Curry E. Big Data Curation. In: New Horizons for a Data-Driven Economy [Internet]. Springer, Cham; 2016 [cited 2018 Mar 25]. p. 87–118. Available from: https://link.springer.com/chapter/10.1007/978-3-319-21569-3_6
S34-4: Using a fully automated curation approach for large clinical datasets would be ideal, as it is scalable and requires minimal clinician input. However, in most settings this is at the moment not achievable.
Which of the following strategies is most successful in reducing the time imposed on clinical expert curators?
A. Using non-expert curators
B. Focus on subset of data
C. Technology support
D. More training for curators
Answer: C. Technology support
Explanation: Technology can increase the efficiency of clinical expert curators. Often a hybrid strategy is used: A large proportion of data is curated without human intervention. Where the algorithm can’t reach a decision with sufficient certainty, the human expert helps to make the decision. Another technology-enabled strategy is to use the human expert to verify the automated curation by scanning for abnormalities (e.g. via a visualisation tool) that can arise from process errors. Non-expert curators don’t have sufficient knowledge to do the curation task. Sampling a subset of data to verify data quality might be an option, but if manual curation is needed, this would have to be done for the full dataset. Training can increase curation efficiency, but this is modest compared to the scale of large datasets.
Reference: The Deeply Human Core Of Roche’s $2.1 Billion Tech Acquisition -- And Why It Made It [Internet]. [cited 2018 Mar 25]. Available from: https://www.forbes.com/sites/davidshaywitz/2018/02/18/the-deeply-human-core-of-roches-2-1b-tech-acquisition-and-why-they-did-it
279 | P a g e
Developing a visual annotation tool to rapidly collect expert-annotated weight errors in pediatric growth charts
--PJ Van Camp
S34-5: The clinical importance of a child’s weights being entered correctly into the electronic health record is to:
A) correctly classify children as obese or malnourished
B) avoid potential medication dosing errors
C) avoid useless information being stored and taking up valuable disk space
D) ensure correct reimbursement from insurance companies
Answer: B) avoid potential medication dosing errors Explanation: Weight is often used to calculate the correct dose of medications in children. Using a wrong weight could lead to under- or over- dosing the child and potentially result in treatment failure or increased adverse effects, respectively. Weight alone is not enough to classify a child as obese (one needs height as well) or malnourished (one needs intake). The storage of extra useless data is not of clinical importance and insignificant compared to other data volume. Reimbursement by insurance is again not of clinical importance and uses other sources of information (e.g. ICD codes) Reference: Kaushal, R., et al., Medication errors and adverse drug events in pediatric inpatients. JAMA, 2001. 285(16): p. 2114-20. S34-6: What is a major benefit of creating a software application to let experts annotate a large dataset instead of manual annotation without the use of a specific tool? A) The process of gathering, annotating and saving the data can be sped up and it allows annotators unfamiliar with data analysis software to interact with the data in an intuitive manner. B) Data annotated digitally is more credible than data annotated by hand because the latter is more prone to error C) Once a tool has been developed, it can be used for many other projects that require different dataset annotations D) Only experts can annotate data manually, while with a tool anyone can Answer: A) The process of gathering, annotating and saving the data can be sped up and it allows annotators unfamiliar with data analysis software to interact with the data in an intuitive manner.
280 | P a g e
Explanation: The point of developing a tool is to automate certain processes and facilitate interaction with data for users. Depending on the problem, tools often reduce repetition or enhance visualization of the task at hand. Digitally annotated data is not per se of better quality, especially if a software error messes up the data without the knowledge of the annotator. Vigorous testing is needed to make sure everything is as good as if it were done manually. Tools are developed for a specific task, meaning often it needs considerable modifications if wanting to be used for other projects. Finally, annotation itself depends on the expertise of the annotator, not the tool used to do the task, though it may facilitate it. References: https://www.usability.gov/what-and-why/user-interface-design.html https://shiny.rstudio.com/ https://material.io/
Closing Session and Keynote Panel: Building Clinical Decision Support that Works
--Adam Wright, Joan Ash, Dean Sittig, Bill Galanter, Sarah Collins
S35-1: A healthcare system wants to reduce the time it takes to clinical decision support system malfunctions. Because the organization participates in several quality programs, the organization’s leadership is particularly concerned about “false negative” situations, where CDS is supposed to fire, but doesn’t. Which of these strategies is most likely to quickly detect “false negatives”?
A. User reporting through the help desk
B. Ongoing monitoring with anomaly detection
C. Routine clinical use of the system by informatics leaders
D. Sentiment analysis of override reasons
Answer: B. Ongoing monitoring with anomaly detection
Explanation: The approach most likely to detect false negatives quickly is ongoing monitoring with anomaly detection. This approach is suitable for detecting both false positives and false negatives, and can work quickly. Research suggests that user reporting through the help desk is helpful for detecting false positives (situations where alerts fire when they shouldn't), but isn't effective for detecting false positives. Similarly, routine clinical use of the system by informatics leaders is more likely to detect false positives - false negatives are observed less commonly with this approach, and often after significant delay. Sentiment analysis of override reasons can only be applied when alerts fire and users comment
281 | P a g e
on them -- users can't comment on missed firings, and are very unlikely to make a note about a missed firing if they later see an appropriate firing.
Reference: Wright A, Hickman TT, McEvoy D, Aaron S, Ai A, Andersen JM, Hussain S, Ramoni R, Fiskio J, Sittig DF, Bates DW. Analysis of clinical decision support system malfunctions: a case series and survey. Journal of the American Medical Informatics Association. 2016 Mar 28;23(6):1068-76.
S35-2: A few years after implementation of an Alert type of CDS, or "rule," there are a series of complaints about how it is functioning. The rule was designed by a requesting physician and IS staff. The requesting physician explained what they were looking to have built in a few paragraphs and then it was approved by the CDS committee and built by an IS analyst. A new IS analyst is having a hard time determining if the rule is doing what it was intended to do as the paragraph describing the intent is not clear and does not provide sufficient detail.
What is the best strategy to prevent this type of difficulty?
a) Use clinical informaticists to investigate potential rule malfunctions.
b) Require rule requests to be formatted as flowcharts with sufficient details to allow IS to interpret unambiguously and future clinicians and IS to understand the intent.
c) Require the initial requesting clinician to test all CDS and be responsible for any future queries once the CDS receives approval.
d) Check the override rate of the rule, if greater than 85%, turn the rule off.
Answer: b) Require rule requests to be formatted as flowcharts with sufficient details to allow IS to interpret unambiguously and future clinicians and IS to understand the intent.
Explanation: When CDS is conceived, designed and implemented, documentation needs to occur in a manner that will most likely produce proper functioning CDS that matches the request. In addition, studies show that despite the best intentions and efforts, CDS can malfunction in the future. The documentation needs to support future efforts to diagnose and fix errors or outdated design assumptions. Flowcharts are a long-used standard method for representing processes, initially in industrial engineering almost 100 years ago and then transitioning to use in electronics and finally computer programming. Flowcharts allow a visual representation of logic and workflow which is not possible with prose. Thus, the use of flowcharts in rule design and implementation can help improve initial success as well as future troubleshooting.
Reference: Osheroff JA. Improving outcomes with clinical decision support: an implementer's guide. HIMSS.
282 | P a g e
S35-3: Leaders of a large healthcare system want to optimize their EHR by providing more effective CDS with minimal malfunctions. Which of the following “people” strategies might help the most?
a) Set up a CDS committee of subject matter experts.
b)Involve clinical staff and encourage them to report malfunctions such as alerts not firing when they should.
c) Assure that staff members in informatics, quality improvement, IT, and elsewhere liaise with one another and with clinicians in multiple ways.
d)Hire additional staff to do measurement and monitoring because only they can consistently identify malfunctions.
Answer: c) Assure that staff members in informatics, quality improvement, IT, and elsewhere liaise with one another and with clinicians in multiple ways.
Rationale: The most useful strategy would be a multiple perspectives approach that would provide resources for inclusion of leadership and personnel from different areas to liaise with one another and with clinicians about CDS. The option of setting up a CDS committee of subject matter experts is definitely a good and necessary strategy, but it does not go far enough. Involving clinical staff and encouraging their reporting of malfunctions is also an excellent strategy, but malfunctions like alerts failing to fire are hard for them to identify. Finally, hiring staff members to do additional monitoring would be an excellent idea if resources were available, but this move should be part of a larger, well planned strategy.
Reference: Wright A, Ash JS, Erickson JL, Wasserman J, Bunce A, Stanescu A, St Hilaire D, Panzenhagen M, Gebhardt E, McMullen C, Middleton B. A qualitative study of the activities performed by people involved in clinical decision support: recommended practices for success. Journal of the American Medical Informatics Association. 2013 Sep 2;21(3):464-72.
S35-4: You are leading the governance committee for clinical decision support at your institution. A new request for development of CDS that is nurse-facing was submitted. The request is to implement an evidence-based risk assessment for peripheral nerve damage in the operating room with guidance to support appropriate interventions. Your organization leverages the Data-Information-Knowledge-Wisdom (DIKW) Conceptual Framework to prioritize strategic development of CDS to support nursing practice. Which component of the framework best categorizes this CDS request?
a. Data
b. Information
c. Knowledge
d. Wisdom
283 | P a g e
Answer: c. Knowledge
Explanation: The data category pertains to facilitating data capture, such as ‘does patient smoke?’. The nurse documents a yes/ no value. At this level the nurse just enters data to a question. There is no nursing judgment or knowledge needed to document this data. Information category pertains to CDS that provide nurses with information about their patient at the right time. Knowledge category refers to any CDS that provides an alert according to the information the nurse enters into the system based on nursing knowledge. Another example in the knowledge category is: CDS that promotes accurate documentation of pressure ulcer stage based on nursing knowledge and recommends an appropriate plan of care. The wisdom category includes exposing analytics for wisdom-based clinical interpretation by the nurse, such as a sepsis alert.In a ‘wisdom’ categorized CDS, the nurse leverages his or her wisdom when deciding if the evidence-based recommendations provided by the CDS intervention are appropriate for that patient. To effectively distribute limited resources in meeting the highest priority needs, the Data-Information-Knowledge-Wisdom (DIKW) Conceptual Framework can guide selection of CDS aligned with an organization’s strategic plan for how CDS should support nurses practicing at the top of their professional license.
References:
Whalen, K., Bavuso, K., Bouyer-Ferullo, S., Goldsmith, D., Fairbanks, A., Gesner, E., lagor, C., Collins, S. (2016). Analysis of Nursing Clinical Decision Support Requests and Strategic Plan in a Large Academic Health System. Applied Clinical Informatics, 7(2), 227–237.
Matney S, Brewster PJ, Sward KA, Cloyes KG, Staggers N. Philosophical approaches to the nursing informatics data-information-knowledge-wisdom framework. Adv Nurs Sci 2011; 34(1):6–18. doi:10.1097/ANS.0b013e3182071813
Nursing Informatics. Scope and Standards of Practice. Silver Spring, MD: American Nurses Association. 2008; 1: 177.