What? What is Reflective Practice (RP)? RP PerspectivesSo What? A Need to Reconceptualize A Theoretical Framework for RP in EvaluationNow What? Conclusions and Implications for Professional

Practice Application of the Theoretical Framework

WHAT?: WHAT IS REFLECTIVE PRACTICE (RP)?Stevahn, King, Ghere, and Minnema (2005) identify (RP) as one of six essential competency domains in program evaluation.

“One’s awareness of evaluation expertise and needs for growth, including knowing oneself as an evaluator, assessing personal needs for enhanced practice, and engaging in professional development toward that goal.” (p. 52)

There is a clear expectation that we perform RP in our evaluation work.WHAT?: RP PERSPECTIVES

Stevahn and colleagues emphasize the self-directed element of reflective practiceo Self-improvement, self-awareness

Other literature views RP as a more collaborative effort.o What are the potential benefits of the inclusion of others in the reflection process?

The RP literature suggests that evaluators are utilizing reflection from a much broader perspective, facilitating improvement of the evaluation to include further description of the targeted program and the sponsoring organization, increased evaluative capacity, strengthened evaluation findings, and as a vessel for organizational learning.

o These broader RP perspectives suggest a substantial amount of stakeholder involvement and communication may be needed in support of reflection.

SO WHAT?: A NEED TO RECONCEPTUALIZE There is a need to reconceptualize RP as more than just systematic introspection. Provide an encompassing and realistic theoretical framework for RP in evaluation

grounded in the literature, and Introduce associated implications for the field regarding this essential competency. Fournier (2005) states that, “Evaluation is an applied inquiry process for collecting and

synthesizing evidence that culminates in conclusions about the state of affairs, value, merit, worth, significance, or quality of a program, product, person, policy, proposal, or plan” (p. 140).

PAGE 1 OF 6

TO BE A REFLECTIVE EVALUATION PRACTITIONER: FRAMING REFLECTION AS A SELF-

ORIENTED AND COLLABORATIVE PRACTICETiffany Smith, Gary Skolits, Patrick Barlow, and Karl Lipovsek

University of Wisconsin – StoutUniversity of Tennessee, Knoxville

University of Iowa Carver College of Medicine

This requires evaluators to assess the merit and value of the program or organization that they are evaluating.

The culmination of this process is both empirical and judgmental

AS EVALUATORS, THERE ARE A NUMBER OF DIFFERENT REASONS TO BE REFLECTIVE PRACTITIONERS… Awareness of ourselves, our personal beliefs/biases, past experiences, and how we

operate in our professional worlds Awareness and continued use of the guidelines in our profession Awareness of the evaluand(s) in which we operate Better communication Assistance in promoting organizational learning as well as evaluation capacity building. Professional growth AND profession growth RP is a multifaceted phenomenon, and it isn’t JUST about YOUR reflection.

SO WHAT?: A THEORETICAL FRAMEWORK FOR RP IN EVALUATION

REFLECTIVE PRACTICE REDEFINEDCritical and deliberate inquiry into professional practice in order to gain a deeper understanding of oneself, others, and the meaning that is shared among individuals. This can happen during practice and after the fact, and can either be done alone or with others.

Self-Oriented / Collaborative Elements are not mutually exclusive Self-oriented reflection can sometimes involve reflection with others Reflection with others can help to benefit self-oriented reflection

SELF-ORIENTED REFLECTIVE PRACTICE Self-oriented reflective practice is critical and deliberate inquiry into one’s own practice; this aspect of reflective practice is a process that focuses on an understanding of the lenses through which one views the world.

(1) SELF-AWARENESS Mindfulness of one’s knowledge, skills, perceptions, and dispositions and how those affect

professional practice, especially regarding responsiveness to one’s own views as well as the views of those involved in the evaluation context

Derives from the Essential Competencies o “Aware of self as an evaluator (knowledge, skills, dispositions)”o “Reflects on personal evaluation practice (competencies and areas for growth)”

(Stevahn et al., 2005, p. 51). A way to gain a better understanding of who we are as both professionals and individuals. Evaluators bring their own cultural, political, socioeconomic, and behavioral worldviews to

the table, and being aware of one’s own lenses is essential to professional practice.

PAGE 2 OF 6

PAGE 3 OF 6

(2) PROFESSIONAL GROWTH Rigorous and critical reflection on one’s own practice in order to improve future actions Competencies highlight professional growth:

o “Pursues professional development in evaluation”, o “Pursues professional development in relevant content areas” (p. 51).

Attending conferences/workshops, reading about new techniques and skills “The most personalized and individualized professional development comes from rigorous

and systematic reflection on your own practice” (Patton, 2012, p. 401). Metaevaluation: What worked well?, what didn’t work well?, what was useful and not so

useful? Decision-making: “Through reflection, practitioners build up a collection of images, ideas,

examples, and actions that they can draw upon when making practice decisions” (Kundin, 2010, p. 354)

(3) ETHICAL AWARENESS Awareness of professional ethical standards, values, and practice as they apply to each

evaluation situation. Questioning assumptions and values in order to promote an ethically sound evaluation. Face ethically charged tasks that oftentimes do not have clear solutions. “Moral courage requires doing the right thing simply because it is the right thing to do.

Reflection must be transformed into action” (Morris, 2008, p. 202)CONCLUSIONS ABOUT SELF-ORIENTED REFLECTIVE PRACTICEThe purpose of this component:

Reflection is a means of learning more about oneself including assumptions and worldviews, improving professional practice, and becoming more aware of ethical issues, moral ideas and their implications.

COLLABORATIVE REFLECTIVE PRACTICE Collaborative reflective practice is critical and deliberate inquiry into a community of practice which involves critical thinking with others to understand oneself, others, and the meanings that are jointly constructed.

(1) DIALOGUE A process of thinking together in order in a group in order for all parties gain deeper

understandings and enhanced perspectives of the evaluation and organizational processes.

“To reach new understanding and, in doing so, to form a totally new basis from which to think and act. In dialogue, one not only solves problems, one dissolves them” (Isaacs, 1999, p. 19).

“Facilitates the evaluative inquiry learning process of reflection, asking questions, and identifying and clarifying values, beliefs, assumptions, and knowledge” (Preskill & Torres 1999, p. 53).

A way to build relationships of trust, respect, caring, and openness, and to understand others’ perspectives (Greene, 2001)

o Understanding ≠ Agreement “Enables people to develop their own insights, rather than leading (no matter how

cleverly) people to a predetermined set of answers” (Senge, 1989, p. 44).(2) STAKEHOLDER INVOLVEMENT

The participation of stakeholders in evaluation processes, decisions, and reporting, so as to increase the use of the evaluation and to give stakeholders a voice in the evaluation as a whole.

Stakeholder involvement has been shown to improve the effectiveness of evaluation efforts.

PAGE 4 OF 6

o Stakeholder involvement has been a focus of evaluation practice for over 40 years There are a number of different evaluation approaches that promote stakeholder

involvement. o Utilization-Focused Evaluation (UFE)o Collaborative/participatoryo Dialogic evaluation

(3) ORGANIZATIONAL LEARNING Facilitation of the evaluation process with stakeholders in order to promote organizational

growth and capacity for evaluation. “When reflection becomes habit we see evidence of a learning organization; a critical

hallmark that proves an organization can stand the test of time” (Forrester 2011, p. 139). Evaluation as a venue for the learning process. Learning from others in the organization is claimed to produce greater insights into

evaluative issues. “Within organizations, individuals are always learning. It is becoming increasingly clear

that this learning is maximized through opportunities to share individual knowledge experiences with others” (Preskill & Torres, 1999, p. 23).

“Learning how to think evaluatively is learning how to learn” (Patton, 1998, p. 226).CONCLUSIONS ABOUT COLLABORATIVE REFLECTIVE PRACTICEThe purpose of this component:

Collaborative RP has as its base the idea that reflection is a means of learning together and thinking together about evaluation and organizational processes.

o The stakeholders AND the evaluators reciprocally reflect and learn through this process.

NOW WHAT?: CONCLUSIONS AND IMPLICATIONS FOR PROFESSIONAL PRACTICE

A theoretical framework has been introduced that addresses the major aspects of the use of RP in evaluation.

Essential organization of RP in evaluation and its associated elements in an effort to

o Operationalize the concept for practice purposes and make it easier to both conceptualize and study empirically

There is the need for more research following the creation of this theoretical framework. NOW WHAT?: APPLICATION OF THE THEORETICAL FRAMEWORK

How might this theoretical framework be applied by evaluators and evaluation researchers?

Evaluators can use this framework as a guide for thinking about, and reflection upon, their evaluation practice.

Evaluation researchers can use this framework as a guide to study several aspects of evaluation.

SUGGESTED READING:Brønn, P., & Brønn, C. (2000). A reflective stakeholder approach: Co-orientation as a basis for communication and learning. Paper Presented at the 7th International Public Relations Research Symposium. Bled, Slovenia. Retrieved from http://www.emeraldinsight.com/journals.htm?articleid=1524239&show=abstractForrester, D. P. (2011). Consider: Harnessing the Power of Reflective Thinking in Your Organization. New York, NY: Palgrave Macmillan.Fournier, D. M. (2005). Evaluation. In S. Mathison (Ed.), Encyclopedia of Evaluation (pp. 139-140). Thousand Oaks, CA: SAGE Publications. Gergen, K. J. (2009). Relational Being: Beyond Self and Community. New York, NY: Oxford University Press.

PAGE 5 OF 6

THANK YOU!Tiffany Smith, smithtif@uwstout.edu

Gary Skolits, gskolits@utk.eduPatrick Barlow, patrick-

barlow@uiowa.eduKarl Lipovsek,

Greene, J. C. (2001). Dialogue in evaluation: A relational perspective. Evaluation, 7(2), 181-187.Isaacs, W. (1999). Dialogue and the art of thinking together. New York, NY: Random House.Jones, D., & Stubbe, M. (2004). Communication and the reflective practitioner: A shared perspective from sociolinguistics and organisational communication. International Journal of Applied Linguistics, 14(2), 185-211. doi:10.1111/j.1473-4192.2004.00059.xKundin, D. M. (2010). A conceptual framework for how evaluators make everyday practice decisions. American Journal of Evaluation, 31(3), 347-362. Morris, M. (2008). Evaluation Ethics for Best Practice. New York, NY: The Guillford Press.Patton, M. Q. (1998). Discovering process use. Evaluation, 4(2), 225-233. Patton, M. Q. (2012). Essentials of Utilization-Focused Evaluation (3rd ed.). Thousand Oaks, CA: SAGE Publications.Preskill, H. S., & Torres, R. T. (1999). Evaluative Inquiry for Learning in Organizations. Thousand Oaks, CA: SAGE Publications.Rallis, S. F., & Rossman, G. B. (2000). Dialogue for learning: Evaluator as critical friend. New Directions for Evaluation, 86, 81-92.Rossman, G. B., & Rallis, S. F. (2000). Critical inquiry and use as action. New Directions for Evaluation, 88, 55–69. Schön, D. A. (1983). The Reflective Practitioner: How Professionals Think in Action. New York, NY: Basic Books.Schwandt, T. A. (1997). Evaluation as practical hermeneutics. Evaluation, 3(1), 69-83.Senge, P.M. (1989). Organizational Learning: A New Challenge for Systems Dynamics. System Dynamics Group Report D4023, Sloan School of Management MIT.Smith, T. L., Barlow, P. B., Peters, J. M., & Skolits, G. J. (2015). Demystifying reflective practice: Using the DATA model to enhance evaluators’ professional activities. Evaluation and Program Planning, 52, 142-147.Stevahn, L., King, J. A., Ghere, G., & Minnema, J. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26(1), 43-59. Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). Program Evaluation Standards. Thousand Oaks, CA: SAGE Publications.

PAGE 6 OF 6