1. Using Rubrics to Enhance ExtensionProgram Planning and Evaluation: A Health Insurance Literacy Example Bonnie Braun, PhDProfessor, Department of Family ScienceSpecialist, University of Maryland Extension Faculty Scholar,Horowitz Center for Health LiteracyUniversity of Marylandbbraun@umd.edu

2. Rubrics 3. Why Rubrics? Encourages consistent and thorough evaluation Assists in evaluation of existing and newprograms, curricula and educational materials Permits multiple individuals to make judgmentsusing common definitions 4. University of Maryland Health SmartHealth Insurance Literacy Initiative(HILI) Emergent area for education Strives to enhance the confidence and competence of consumers in making health insurance purchasing decisions Targets both those who already purchase insurance and those newly eligible for health insurance under the Affordable Care Act From Maryland-only to multi-state initiative Pilot testing launched April 1, 2013 5. Design Specifications &ProductsComprehensive, systematic process*Health literacy principlesScienceTheory *Coleman, et al, 2011 6. What did we create?1. Program Assessment Tool2. Curriculum Assessment Tool3. Materials Assessment Tool 7. Curriculum Assessment Tool Curriculum Assessment Tool -- providesa standardized set of criteria to evaluateexisting educational curricula and to use increating new educational curricula. Uses a four-point scale(Effective, Good, Fair, and Ineffective) Based on recommendations of Extensioneducators who provided a rationale forcurricula review**Coleman et al, (2011) 8. Educational Materials Tool Educational Materials Assessment Tool --provides a standardized set of criteria to judgeeducational materials used in programs and incurricula. Can be used both for critiquing existingeducational materials and creating newmaterials. Based on Doak et al (2007) which provides astandardized way to analyze materials forlearners with varying levels of prose, documentand quantitative literacy. 9. How did we create them? The roleof theory Both rubrics grounded in theoretical principles of adultlearning, health literacy, health behavior WHY? Application of theory to educational interventions increasesthe likelihood that intended outcomes will be achieved; Theories provide the rationale for how the intervention isstrategically structured and delivered; and Theories offer the basis for assessment of the programsdegree of success in achieving intended outcomes.Braun, et al (forthcoming) 10. How did we create them? The roleof theory Depends on situation, targeted population, understanding of behavior change and determination of outcomes that are strategic, measurable, achievable, relevant and timely* Level of interventionindividual, family or other group, community or policywill guide the appropriate programming theories. For the Health Insurance Literacy Initiative curriculum, we used individual-level theories: social cognitive theory, stages of change or readiness, theory of planned behavior, communications, adult or youth development, empowerment, and evaluation and action research*Doran, 1981 11. How did we create them? The role of theory Key theoretical concepts can be combined into amodel that frames the design and measurement ofeducational programming impact. The theories we selected directed us to: Involve the targeted population Identify the level of confidence, competence andsense of control before and after interventions. Create messages and deliver via appropriate channels Design learning experiences to increase criticalthinking and reflection. Assure that evaluation of need, process and outcomeis effectively conducted and reported. 12. How can you use these tools? Language of the rubrics could be substituted withappropriate vocabulary for the curricula contentunder review. May need revise based on the level of intervention(family, individual, community) and the intent or goalsof the program. May need other theories to guide the development ofprograms and assessment tools if not focused onchange-making. 13. Lets practice1. Find your copy of the Educational Materials Assessment Tool and the Scoring Sheet2. Review the provided document3. Complete the scoring sheet as it relates to the material 14. Questions for discussion: What are your thoughts about thematerial after using the rubricguidelines to assess it? In what ways did you find therubric useful in guiding yourevaluation of the material? 15. Coming Soon 16. References Braun, B., McCoy, T., & Finkbeiner, N. (In press).Extension Education Theoretical Guide with Criterion-Referenced Assessment Tools. College Park:University of Maryland Extension Coleman, G., Byrd-Bredbenner, C., Baker, S., &Bowen, E. (2011). Best practices for extensioncurricula review. Journal of Extension, 49, Article2T0T1. Available at:http://www.joe.org/joe/2011april/tt1.php Doak, C.C., Doak, L.G., & Root, J.H. (2007). Teachingpatients with low literacy skills. (2nd ed.). Philadelphia:J.B. Lippincott Company. Doran, G. T. (1981). Theres a S.M.A.R.T. way to writemanagements goals and objectives. ManagementReview, 70, 35-36. 17. AcknowledgementThis presentation and the rubrics were co-createdwith Nicole Finkbeiner, MS, GraduateResearch Assistant, University of MarylandDepartment of Family Sciences.