12/12/16

1

AN OUTLINE OF WEB-BASED TOPIC DISCUSSIONS

Pharmacy Student’s Retention of Knowledge Following Web-based Topic Discussions as Part of an Advanced Pharmacy Practice Experience in Ambulatory Care Veronica M. Nieto, Pharm.D., BCPS Texas A&M University Health Science Center – College of Medicine Scholarly Project Presentation December 12, 2016

Presentation Objectives • Describe the objective and goals of the project • Assess the literature supporting the use of online content

to supplement clinical practice experiences • Review the Three Levels of Planned E-learning

Interactions • Discuss procedures and methodology leading to the

develop the online modules • Review the outline of proposed topic discussions • Demonstrate a web-based topic discussion prototype

2

Project Introduction • Advanced Pharmacy Practice Experiences (APPE)

•  Occur during the 4th year of pharmacy education •  Provide application of learned material from didactic phase •  An extension of Introductory Pharmacy Practice Experiences

(IPPE)

•  Identified problem •  Loss of basic pharmacology and therapeutic knowledge limits

students’ ability to effectively participate in patient care activities

• Challenge for educators •  How to best enhance student learning and knowledge retention

3

Educational Standards •  Accreditation Council for Pharmacy Education – Standards 2016

•  Standard 4 •  4.1: Self-awareness and reflection on personal knowledge, skills, and abilities

•  Standard 10 •  10.12: Teaching and learning methods actively engage learners and promote student

responsibility for self-directed learning •  10.15: Experiential quality assurance… standardize key components of experiences

and promote consistent assessment of student performance •  Standard 24

•  24.3. Student achievement and readiness – develop and implement a plan to assess attainment of education outcomes to ensure that graduates are prepared to enter practice

•  2013 Center for Advancement of Pharmacy Education (CAPE) Outcomes •  Domain 4 – “Personal and Professional Development”

•  Self-awareness the use of metacognition to regulate one’s own thinking and learning

ACPE, Standards 2016 2013 CAPE Outcomes

4

Project Objective •  To outline the content of a web-based learning module

series for use during an ambulatory care APPE rotation

•  Five core disease states •  Anticoagulation •  Asthma/COPD •  Diabetes •  Hypertension •  Hyperlipidemia

5

Project Goals • Promote the transfer of pharmacology and therapeutics

from the classroom to patient care

• Provide an effective and efficient mechanism to deliver standardized content

• Enhance knowledge retention of commonly encountered disease states in the ambulatory care setting through the use of a multimedia-centered, active learning environment

• Obtain an objective assessment of students’ knowledge

6

12/12/16

2

PICOT Question • Does the incorporation of web-based training modules

affect 4th year pharmacy students’ knowledge retention of primary care disease states and facilitate their ability to provide pharmaceutical care during a 6-week APPE in ambulatory care?

7

LITERATURE EVALUATION Pharmacy and Medical Education

8

Examples in Pharmacy and Medial Education Study Objective Study Design Results/Conclusions

Flowers SK, et al. Am J Pharm Educ. 2010; 74(3).

Evaluate the effect of web-based vignettes on complex drug administration techniques in Community Pharmacy

-  Retrospective review -  n = 79 P4 students -  5 vignettes -  ½ assigned to study group -  12Q pre/post assessment

-  Improved results in study group

-  Vignettes effective in increasing knowledge and provided standard experience

Morris JL & Knoderer CA. J Pediatr Pharmacolo Ther. 2011;16(3): 210-215.

Assess effectiveness of web-based training modules to enhance/facilitate learning and ability to provide patient care to children

-  Retrospective review -  n = 20 P4 students -  4 modules -  All students reviewed the

modules -  10 question pre/post-modular

assessments

-  Post-modular assessment scores improved for all but one area

-  Web-based modules may expand/improve understanding of pediatric pharmaceutical care

Isaacs AN, et al. Am J Pharm Educ. 2015; 79(3).

Implement and evaluate web-based learning modules prior to APPEs on inpatient general medicine

-  Retrospective review -  n = 40 P4 students; at four

health care systems -  3 modules

-  Web-based learning enables standardization and assessment of baseline knowledge

-  Positive student perception

Rebel A, et al. J Clin Ane. 2011;23:469-74.

Develop and evaluate new curriculum in transfusion medicine for anesthesiology resident rotation

-  Retrospective, quasi-experimental study

-  TM (n = 9) non-TM (n=21) -  Performance on pre/post- test

and on the ABA-ITE

-  Increase of 25 ± 14% of correctly answered questions on posttest (p < 0.05)

-  No statistical difference between TM and non-TM on ABA-ITE 2009

Kumar AB, et al. J Surg. 2013; 70:109-16.

Determine if traditional/web-based curriculum improves test scores in elective, 4 week, MS4 critical care rotation (n = 121)

-  Retrospective review of hybird curriculum

-  Pre/post-test, random selection of 20 MCQs

-  Each group improved post-test scores by ~ 17%

Benefits of Web-based Instruction • Optimizes APPE education time

• Allows for standardization of educational content

•  Leads to higher level of comprehension

• Content can be easily updated and readily accessible

• Allows for real time feedback and objective assessment

10

Flowers et al., 2010 Isaacs et al., 2015

Morris & Knoderer, 2011 Reuhter et al., 2012

Salter et al., 2014 Seybert & Kane-Gill, 2011

THEORETICAL FRAMEWORK Educational Theory

11

Adult Learners • Knowles’ Adult Learning Theory

•  Adult leaners seek practical information •  Necessary information to accomplish task, be successful

•  Five Assumptions of Adult Learners 1.  Independent self-concept, able to direct their own learning 2.  Life experiences serve as resource of information 3.  Needs are related to changing social roles 4.  Interest lies in immediate application of knowledge 5.  Internally motivated

12

Merriam, 2001 Pappas, 2013

12/12/16

3

Planned E-learning Interactions

13

Learner – Instruction Interactions

Learner-Human Interactions Learner-Non-human Interactions

Learner-Instructor

Learner-Learner

Learner-Other

Learner-Content

Learner-Interface

Learner-Environment

Learner-Self Interactions

Level III

Level II

Level I

Hirumi, 2002

PROCEDURES & METHODS

14

Module Development

Step 1: Determine core disease states for areas of clinical focus

Step 2: Establish learning objectives •  Module series and individual disease states

Step 3: Select modular content material •  Review articles, guidelines, standards of care

Step 4: Develop cases, assessments, and surveys

15

Step 1: Core Disease State Selection • Anticoagulation • Asthma/COPD • Diabetes • Hypertension • Hyperlipidemia

• Benefits of pharmacist participation in these areas: • Cost-effective •  Improvement in surrogate

endpoints •  Decreased blood pressures,

HgbA1c, cholesterol, and risk factors for coronary heart disease

• Reduced adverse events

16

ASHP, 1999 Carter et al., 2008

Cone, Brown & Starmbaug, 2008 Devine et al., 2009

Kislan, Bernstein, Fearrington, & Ives, 2016 Pett & Nye, 2016

Zhong, Ni, Cui, & Liu, 2014

Step 2: Establishing Learning Objectives • Module series objectives

•  Demonstrate knowledge and comprehension of pharmacotherapy of core primary disease states •  Areas of focus:

•  Pathophysiology •  Diagnostic studies •  Clinical laboratory monitoring •  Non-pharmacologic and pharmacologic treatment modalities

•  Apply therapeutic management strategies, clinical guidelines and clinical trial data to make clinical decisions in simple and complex patients

•  Develop treatment plan for core primary diseases given case-based scenarios

17

Step 2: Establishing Learning Objectives Anticoagulation Asthma/COPD Diabetes Hypertension Hyperlipidemia 1)  Identify risk factors

and s/sx of DVT and PE

2)  Calculate CHA2DS2-VASc score

3)  Review the clotting cascade and identify targets of therapy

4)  Identify risk factors for bleeding

5)  Select and interpret laboratory tests

6)  Manage an elevated INR

7)  Identify warfarin drug-drug and drug-food interactions

8)  Formulate a treatment plan for a patient receiving oral anticoagulation therapy in the outpatient setting

9)  Develop a plan for perioperative management of anticoagulation for a procedure requiring therapy interruption

1)  Compare/contrast pathogenesis, presentation, diagnosis, staging

2)  Define spirometric measurements

3)  Review current guidelines

4)  Identify drug classes of oral and systemic medications used for asthma/COPD

5)  Review counseling points for inhalation devices

6)  List a stepwise approach for managing asthma

7)  Counsel on the management of exercise induced asthma

8)  Review recommended vaccinations

9)  Design a therapeutic plan for the management of COPD

1)  Outline pathogenesis and targets of therapy

2)  Compare/contrast treatment guidelines

3)  List screening and diagnostic criteria

4)  Differentiate between s/sx of hypo- and hyperglycemia

5)  Identify screening criteria for micro- and macrovascular complications

6)  Review goals for comorbidities

7)  Provide education on glucometer and insulin use

8)  Review nonpharmacologic recommendations

9)  Outline the steps of a diabetic foot exam

10) Compare/contrast diabetes agents

11) Design a therapeutic plan for the management of type 2 diabetes

1)  Outline the pathogenesis and targets of therapy

2)  Compare/contrast treatment guidelines

3)  Identify barriers to successful management

4)  Demonstrate blood pressure monitoring technique

5)  Review lifestyle modifications

6)  Review therapeutic drug classes and respective medications

7)  Select a medication based on comorbid conditions

8)  Develop a therapeutic plan for the management of hypertension

9)  Differentiate between hypertensive urgency and emergency

1)  List the major lipoproteins and described their role in atherosclerosis

2)  Identify statin-benefit groups and statin intensities

3)  Review current guidelines for the management of hyperlipidemia

4)  Identify the key components of the ASCVD risk calculation and its limitations

5)  Determine a patients ASCVD risk and treatment goals

6)  Recommend TLCs for patient with elevated lipid levels

7)  Design a monitoring plan to assess effectiveness and adverse effects of pharmacotherapy for hyperlipidemia

18

12/12/16

4

Steps 3 & 4: Content Selection and Development of Assessments • Selection of content material

•  Current pharmacy resources •  Disease state review articles •  Organizational guidelines •  Standards of care

• Development of patient cases and pre- and post- assessments •  Peer review by panel of faculty

• Development of module blueprints •  Faculty review of blueprints

•  Determine if module and disease state objectives are met •  Provide feedback with regard to corrections, revisions, or clarifications

19

Vision for Implementation •  Ambulatory care faculty from Texas A&M Rangel College of

Pharmacy (RCOP) •  Responsible for development of modules, assessments, and updates •  Distributed according to geographic region to promote collaborations amongst

faculty •  Corpus Christi, Houston, Temple/Round Rock, McAllen, Bryan/College Station

•  Students will be required to complete pre-and post-assessments for each module •  10-12 multiple-choice questions (randomized sequence) •  Scored on a scale of 0-100% (80% or greater = passing)

•  If students score 100% on the pre-assessment, module encouraged not required •  No feedback provided on incorrect answers until initial post-assessment attempt

•  Up to three opportunities for successful completion of the post-assessment •  Order of completion of the modules intentionally not prescribed

•  Allow for introspective reflection of the adult learner

20

Navigation & Outline • Each module to take

~2hrs •  Broken down into smaller

sections, shorter duration •  Selection of specific area of

need

• Each module will contain: •  Learning objectives •  Pre- and post- assessment •  Review of disease specific

pathophysiology, guidelines, and medications

•  Clinical application exercises •  Links to additional resources

Anticoagulation Module Learning Objectives

Pre-assessment

Pathophysiology review

Review of CHEST Guidelines

Medications

Clinical application

Post-Assessment

Helpful Links

21

Vision for Implementation

• Modules and assessments administered via the university’s web-based learning management system •  eCampus

• Post-rotation survey of the learning experience •  Administered via Qualtrics

22

Next Steps • Pilot the program

•  To students nearing completion of the PharmD Program •  Obtain feedback on feasibility, accessibility, ease of use, and

students’ perception of improved understanding of pharmaceutical care in the ambulatory care setting •  Survey questions

• Resultant pilot data to guide changes to content prior to implementation

23

Future Implications •  Implement into ambulatory care APPE curriculum • Collect data

•  Pre- and post-assessment results •  Student perceptions of material – survey data

•  Feasibility, accessibility, ease of use, and improved learning •  Preceptor perceptions of material

•  Feasibility, accessibility, ease of use

• Develop written report summarizing the findings • Disseminate findings to professional audiences of medical

educators •  Posters •  Platform presentations •  Journal articles

24

12/12/16

5

Future Implications • Determination of Success

•  Improvement in knowledge of chronic disease state management •  Improved post-assessment scores, benchmark exams, and domain

specific areas on national licensure exams •  Improvement in effectiveness of patient care

•  Increasing number of pharmacy interventions •  Satisfaction with overall learning experience

•  Results of post-rotation survey

•  If effective others could utilize this approach as a model •  Other areas of pharmacy •  Other health professions

25

Modular Demonstration Ambulatory Care Online Module Series

Anticoagulation Asthma/COPD Diabetes Hypertension Hyperlipidemia

Module Introduction •  Series consists of five disease state modules

•  Each module made up of shorter subsections focusing on specific areas of disease state management

•  Should take ~1.5-2hrs to complete each module •  Can be completed in any order

•  Open access to review any completed modules/subsections

•  Each module will contain: •  Pre- and post- assessment

•  If score 100% on pre-test; module optional •  Must score 80% or greater on post-test to demonstrate competency

•  Three attempts allowed •  Feedback provided after first post-test attempt

•  Clinical application exercises •  Knowledge checks of recently reviewed information •  Built in patient cases, simulations

Module Introduction • Module series objectives

•  Demonstrate knowledge and comprehension and pharmacotherapy of core primary disease states •  Areas of focus:

•  Pathophysiology •  Diagnosis and diagnostic studies •  Clinical laboratory monitoring •  Non-pharmacologic and pharmacologic treatment modalities

•  Apply therapeutic management strategies, clinical guidelines and clinical trial data to make clinical decisions in simple and complex patients

•  Develop treatment plan for core primary diseases given case-based scenarios

Home

DIABETES APPE Ambulatory Care Online Module

Diabetes Learning Objectives 1) Outline pathogenesis and targets of therapy 2) Compare/contrast treatment guidelines 3) List screening and diagnostic criteria 4) Differentiate between s/sx of hypo- and

hyperglycemia 5)  Identify screening criteria for micro- and

macrovascular complications 6) Review goals for comorbidities 7) Provide education on glucometer and

insulin use 8) Review nonpharmacologic

recommendations 9) Outline the steps of a diabetic foot exam 10) Compare/contrast diabetes agents 11) Design a therapeutic plan for the

management of type 2 diabetes

Pre-Assessment

Pathophysiology Review

Diagnostic and Screening Criteria

Guideline Review

Medications

Goals of Therapy

Clinical Application

Post-Assessment

Helpful Links

12/12/16

6

Diabetes Pre-Assessment

You’ve successfully answered 7 out of 10 questions correctly

Score = 70%

Please proceed through the

remainder of the module

1) Outline pathogenesis and targets of therapy 2) Compare/contrast treatment guidelines 3) List screening and diagnostic criteria 4) Differentiate between s/sx of hypo- and

hyperglycemia 5)  Identify screening criteria for micro- and

macrovascular complications 6) Review goals for comorbidities 7) Provide education on glucometer and

insulin use 8) Review nonpharmacologic

recommendations 9) Outline the steps of a diabetic foot exam 10) Compare/contrast diabetes agents 11) Design a therapeutic plan for the

management of type 2 diabetes

Pre-Assessment

Pathophysiology Review

Diagnostic and Screening Criteria

Guideline Review

Medications

Goals of Therapy

Clinical Application

Post-Assessment

Helpful Links

Diabetes Learning Objectives

Pathophysiology Review

Introduction •  Type 1 vs. Type 2 Diabetes •  Clinical Presentation •  Long-term complications

Understanding Type 2 Diabetes

Home

Diagnostic & Screening Criteria

Diagnosis of Type 2 Diabetes Diabetic Complications

Test your knowledge

Home

Guideline Review

American Diabetes Association – Standards of Care 2016

AACE/ACE Developing a Diabetes Comprehensive Care Plan

Standards of Care 2016: Summary of Revisions

AACE/ACE Type 2 Diabetes Management Algorithm

Home

Medications

Treatment and Management of Type 2

Diabetes

•  Drug Class Review •  Oral Medications

•  Biguanides •  Sulfonylureas •  Glitinides •  Thiazolidinediones (TZD) •  Alpha-glucosidase inhibitors (AGI) •  Dipeptidyl peptidase-4 (DPP4)

inhibitors •  Sodium-glucose transporter 2

(SGLT2) inhibitors •  Other (dopamine receptor agonist,

bile acid sequestrant)

•  Injectable medications •  GLP-1 agonists •  Insulin (rapid, short, intermediate,

long-acting & mixed preparations)

Home

Test your knowledge

12/12/16

7

Goals of Therapy (ADA)

Home

AACE/ACE Goals of Therapy

Clinical Application 56 y/o male diagnosed with T2DM one month ago (A1c = 8.6%). He was initially started on metformin 500mg once daily for one week, then 500mg twice daily. His fasting blood glucose values over the past week have been 148, 152, 144, 198, 178, and 164. No other blood glucose values are available for assessment. Which of the following is the most appropriate recommendation?

a.  Increase metformin to 1000mg BID and add detemir 10 units QHS

b.  Continue metformin 500mg BID c.  Increase metformin to 1000mg BID d.  Continue metformin 500mg BID and add glipizide 10mg BID-AC

Home

Answer C: Increase metformin to 1000mg BID; optimize current therapy. Request additional BG checks at other times during the day to aid in future assessment and need for glycemic control.

1) Outline pathogenesis and targets of therapy 2) Compare/contrast treatment guidelines 3) List screening and diagnostic criteria 4) Differentiate between s/sx of hypo- and

hyperglycemia 5)  Identify screening criteria for micro- and

macrovascular complications 6) Review goals for comorbidities 7) Provide education on glucometer and

insulin use 8) Review nonpharmacologic

recommendations 9) Outline the steps of a diabetic foot exam 10) Compare/contrast diabetes agents 11) Design a therapeutic plan for the

management of type 2 diabetes

Pre-Assessment

Pathophysiology Review

Diagnostic and Screening Criteria

Guideline Review

Medications

Goals of Therapy

Clinical Application

Post-Assessment

Helpful Links

Diabetes Learning Objectives Diabetes Post-Assessment

You successfully answered 9 out of 10 questions correctly

Score = 90%

You have completed this

module successfully

Helpful Links & Additional Resources •  ADA – Diabetic Foot Exams •  ADA – Living with Diabetes •  New Evidence Worth Reading:

•  Marso, S. P., Daniels, G. H., Brown-Frandsen, K., Kristensen, P., Mann, J. F. E., Nauck, M. A., . . . Buse, J. B. (2016). Liraglutide and Cardiovascular Outcomes in Type 2 Diabetes. New England Journal of Medicine, 375(4), 311-322. doi:doi:10.1056/NEJMoa1603827

•  Pfeffer, M. A., Claggett, B., Diaz, R., Dickstein, K., Gerstein, H. C., Køber, L. V., . . . Tardif, J.-C. (2015). Lixisenatide in Patients with Type 2 Diabetes and Acute Coronary Syndrome. New England Journal of Medicine, 373(23), 2247-2257. doi:doi:10.1056/NEJMoa1509225

•  Marso, S. P., Daniels, G. H., Brown-Frandsen, K., Kristensen, P., Mann, J. F. E., Nauck, M. A., . . . Buse, J. B. (2016). Liraglutide and Cardiovascular Outcomes in Type 2 Diabetes. New England Journal of Medicine, 375(4), 311-322. doi:doi:10.1056/NEJMoa1603827

•  Vora, J., Cohen, N., Evans, M., Hockey, A., Speight, J., & Whately-Smith, C. (2015). Intensifying insulin regimen after basal insulin optimization in adults with type 2 diabetes: a 24-week, randomized, open-label trial comparing insulin glargine plus insulin glulisine with biphasic insulin aspart (LanScape). Diabetes Obes Metab, 17(12), 1133-1141. doi:10.1111/dom.12528

•  Maruthur, N. M., Tseng, E., Hutfless, S., & et al. (2016). Diabetes medications as monotherapy or metformin-based combination therapy for type 2 diabetes: A systematic review and meta-analysis. Annals of Internal Medicine, 164(11), 740-751. doi:10.7326/M15-2650

•  Schwartz, S. S., Epstein, S., Corkey, B. E., Grant, S. F. A., Gavin, J. R., & Aguilar, R. B. (2016). The Time Is Right for a New Classification System for Diabetes: Rationale and Implications of the β-Cell–Centric Classification Schema. Diabetes Care, 39(2), 179-186. doi:10.2337/dc15-1585

Home

References: •  Accreditation Council for Pharmacy Education. (2015). Accreditation standards and key elements for the professional program in pharmacy

leading to the doctor of pharmacy degree. "Standards 2016". Chicago, Illinois •  American Diabetes Association. (2016). Standards of medical care in diabetes - 2016. Diabetes Care, 39((Suppl. 1)), S1-S106. •  American Society of Health-System Pharmacists. (1999). ASHP statement on the pharmacist's role in primary care. . American Journal of

Health-System Pharmacy, 56, 1665-1667. •  Flowers, S. K., Vanderbrush, R. E., K, H. J., & West, D. (2010). Web-based Multimedia Vignettes in Advanced Community Pharmacy

practice Experiences. American Journal of Pharmaceutical Education, 74(3). •  Carter, B. L., Bergus, G. R., Dawson, J. D., Farris, K. B., Doucette, W. R., Chrischilles, E. A., & Hartz, A. J. (2008). A cluster-randomized trial

to evaluate physicain/pharmacist collaboration to improve blood pressure control. Journal of Clinical Hypertension, 10(4), 260-271. •  Cone, S. M., Brown, M. C., & Stambaug, R. L. (2008). Characteristics of ambulatory care clinics and pharmacists in Veterans Affairs

medical centers: An update. American Journal of Health-System Pharmacy, 65, 631-635. •  Devine, E. B., Hoang, S., Fisk, A. W., Wilson-Norton, J. L., Lawless, N. M., & Louie, C. (2009). Strategies to optimize mediation use in the

physican group practice: The role of the clinical pharmacist. Journal of The American Pharmacists Association, 49, 181-191. •  Flowers, S. K., Vanderbrush, R. E., K, H. J., & West, D. (2010). Web-based Multimedia Vignettes in Advanced Community Pharmacy

practice Experiences. American Journal of Pharmaceutical Education, 74(3). •  Hirumi, A. (2002). A Framework for Analyzing, Designing, and Sequencing Planned Elearning Interactions. The Quarterly Review of

Distance Education, 3(2), 141-160. •  Isaacs, A. N., Walton, A. M., & Nisly, S. A. (2015). Interactive web-based learning modules prior to general medicine advanced pharmacy

practice expereinces. American Journal of Pharmaceutical Education, 79(3). •  Kislan, M. M., Bernstein, A. T., Fearrington, L. R., & Ives, T. J. (2016). Advanced Practice Pharmacists: a retrospective evaluation of the

efficacy and cost of Clinical Pharmacists PractitionErs managing ambulatory Medicare patients in North Carolina (APPLE-NC). BMC Health Services Research, 16, 607.

•  Kumar, A. B., Hata, J. S., Bayman, E. O., & Krishnan, S. (2013). Implementing a hybrid web-based curriculum for an elective medical student clerkship in a busy surgical intensive care unit (ICU): effect on test and satisfaction scores. Journal of Surgical Education, 70(1), 109-116.

•  Merriam, S. B. (2001). Andragogy and Self-Directed Learning: Pillars of Adult Learning Theory. New Directions for Adult and Continuing Education, 2001(89), 3-14.

•  Morris, J. L., & Knoderer, C. A. (2011). Assessment of Web-Based Training Modules on Learning Facilitation for Advanced Pharmacy Practice Experiences in Pediatrics. The Journal of Pediatric Pharmacology and Therapeutics, 16(3), 210-215.

•  Pappas, C. (2013). The Adult Learning Theory - Andragogy. Retrieved from https://elearningindustry.com/the-adult-learning-theory-andragogy-of-malcolm-knowles

•  Rebel, A., Hassan, Z.-U., Boral, L., Lin, Y., DiLorenze, A., & Schell, R. M. (2011). Initial results of a structured rotation in hematology and transfusion medicine for anesthesiology residents. Journal of Clinical Anesthesia, 23, 469-474.

•  Reuhter, V., Lindsey, C., Graham, M., & Garavalia, L. (2012). Use of Online Modules to Enhance knowledge and skills application during an introductory pharmacy practice experience. American Journal of Pharmaceutical Education, 76(4).

•  Salter, S. M., Karia, A., Sanfilippo, F. M., & Clifford, R. (2014). Effectiveness of E-learning in Pharmacy Education. American Journal of Pharmaceutical Education, 78(4).

•  Schneider, E. F., Castleberry, A. N., Vuk, J., & Stowe, C. (2014). Pharmacy Students' Ability to Think About Thinking. American Journal of Pharmaceutical Education, 78(8).

•  Zhong, H., Ni, X.-J., Cui, M., & Liu, X.-Y. (2014). Evaluation of pharmacist care for patients with chronic obstructive pulmonary disease: a systematic review and meta-analysis. International Journal of Clinical Pharmacy, 36, 1230-1240.

12/12/16

8

AN OUTLINE OF WEB-BASED TOPIC DISCUSSIONS

Pharmacy Student’s Retention of Knowledge Following Web-based Topic Discussions as Part of an Advanced Pharmacy Practice Experience in Ambulatory Care Veronica M. Nieto, Pharm.D., BCPS Texas A&M University Health Science Center – College of Medicine Scholarly Project Presentation December 12, 2016