branding educational data use through professional learning

Download Branding Educational Data Use through Professional Learning

Post on 02-Jan-2017




0 download

Embed Size (px)


  • Branding Educational Data Use through Professional Learning:

    Findings from a Study in Three School Districts

    Jo Beth Jimerson, Ph.D.

    Texas Christian University

    Jeffrey C. Wayman, Ph.D.

    The University of Texas at Austin

    This study was made possible through funding provided by The Spencer Foundation.

    Paper presented at the 2012 Annual Meeting of American Educational Research Association,

    Vancouver, British Columbia.

    We welcome feedback. Please send questions, comments, and requests to

    Jo Beth Jimerson:

  • Branding Educational Data Use 2


    In order to learn more about how school districts support educator data use, we examined the

    intersection of data use and professional learning in three school districts. We conducted a

    qualitative study, relying on interview data from n=110 individuals across the three districts, as

    well as documents from those districts, to inform our analysis. We found that a chasm exists in

    how educators frame data use, with some framing data use as a student-oriented improvement

    process, and others framing it as a mere exercise in the accountability ratings chase. Further,

    these perceptions seemed to impact educators willingness to invest time and effort in data-

    informed practice. District leaders often spoke of data use as improvement-oriented; however,

    participants descriptions of data-related professional learning opportunities consistently

    underscored a focus on accountability system concerns and an overall accountability orientation.

  • Branding Educational Data Use 3

    Branding Educational Data Use through Professional Learning:

    Findings from a Study in Three School Districts

    In one way or another, educators have always been expected to use data, and they always

    have used data: Even in the one-room schoolhouses of the past, teachers took data by

    providing assignments and issuing grades. In the last few decades, however, an increasing focus

    on school accountability at the state and federal levels has escalated the expectations on

    educators to attend to particular types of data in addition to the gamut of information already in

    play. In the accountability era, educators are expected to use a range of local and broad-scale

    standardized data to inform what happens in classrooms and schools on a daily basis (Anderson,

    Leithwood, & Strauss, 2010; Means, Padilla, DeBarger, & Bakia, 2009; Park & Datnow, 2009).

    Despite increasing expectations to engage in data-informed practice, many educators

    struggle with aspects of data use (Goertz, Olah, & Riggin, 2010; Means et al., 2009; Wayman,

    Jimerson, & Cho, in press). A variety of factors contribute to this difficulty, from user-unfriendly

    data systems (Wayman & Cho, 2008), to the lack of a clear vision for the role of data use data

    use (Louis, Leithwood, Wahlstrom, & Anderson, 2010), to mistrust among teachers related to

    past abuses and misuses of data (Earl & Fullan, 2003; Louis et al, 2010). Also, obstacles to

    effective data use can be linked to the scarcity of data use-related knowledge or supports in some

    educational contexts. These supports include time dedicated to learning about and practicing

    collaborative data use (e.g., Ikemoto & Marsh, 2007) and professional learning that aims at

    improving data use capacity of teachers and school leaders (Jimerson & Wayman, 2011).

    Initially, we set out to learn more about how districts could support improved data use

    among teachers. We were particularly interested in how professional learning might serve as a

    catalyst for improved data use capacity among educators. In early explorations of this issue, we

  • Branding Educational Data Use 4

    noted that: (1) teachers and administrators (including district leaders) seemed to articulate a

    range of rationales for data use; (2) some of these rationales seemed in conflict; and (3) in

    several instances, the descriptions provided by educators of data-related professional learning

    experiences seemed related to these rationales (Wayman, Cho, Jimerson, & Snodgrass Rangel,

    2010; Wayman, Jimerson, & Cho, 2010). We decided to press more on this notion of whether

    districts intentionally or inadvertently brand data use through leadership and professional

    learning structures. Accordingly, the purpose of the present study was to examine the

    intersection of data use and professional learning with particular attention to the role that

    rationales for data use play in garnering educator commitment to engage in data-informed

    practice. Our research was guided by two questions:

    (1) How do educators conceptualize the purpose or rationale for data use?

    (2) Does the structure of data-related professional learning influence these conceptions and,

    if so, how?

    Data Use, Educators, & the Accountability-Improvement Divide

    In this section, we describe the context of the research in which this study is situated. We

    first discuss the research that focuses on the value of data use itself. Then, we examine factors

    that facilitate educator data use, and which might be well-addressed via professional learning

    structures. Finally, we discuss two rationales toward which educator data use might be oriented.

    Data Use & School/District Effectiveness

    Prior to delving into the supports that facilitate or hinder data use, or the perspectives that

    inform how data use is shaped in schools or districts, we think it important to consider whether

    data use matters to improving student outcomes.

  • Branding Educational Data Use 5

    Is data use by any other name still data use? A challenge in reviewing the research

    context for the efficacy of data use is that similar structures exist under various names. In line

    with much of the research, we assert that data use is at the core a social venture through which

    educators interact with a variety of data, engage in collaborative meaning-making, and adjust

    practice accordingly (e.g., Coburn & Turner, 2012; Coburn, Honig, & Stein, 2009; Datnow, Park

    & Wohlstetter, 2007; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Knapp, Copland, &

    Swinnerton, 2007; Supovitz, 2010; Wayman et al., in press). With this definition of data use,

    we note that several other decision-making models make similar use of data to inform practice.

    First, teams of educators who engage in action research reflect on and examine data to

    address problems of practice in very context-specific ways (Ferrance, 2000; Mills, 2007).

    Second, continuous improvement models (e.g., Langley et al., 2009) make use of work teams

    which collect and analyzing information in the hopes of improving situation-specific practice.

    Third, professional learning communities depend on collaborative inquiry processes (DuFour &

    Marzano, 2011). Because these processes parallel data informed decision making, or data use,

    we think it important that questions of effectiveness and impact not be limited to studies that

    explicitly examine connections between the use of broad-scale or standardized data and

    outcomes, but should include studies of how continuous improvement teams, professional

    learning communities, and school-based action research teams make use of data and the

    outcomes associated with these highly contextualized processes.

    Does data use matter? If we understand data use as a concept at the core of data-

    informed decision making, action research, continuous improvement models, and professional

    learning communities in that these all rely on similar data-informed inquiry processes, we must

    still contend with the question, Does data use matter?

  • Branding Educational Data Use 6

    To this question, evidence is mixed. In terms of classroom-level use of formative

    assessment data, Black and Wiliam (1998) reviewed over 250 studies and concluded that the use

    of formative assessment by classroom teachers had a strong and statistically significant and

    positive impact on student achievement. Using assessment to inform changes in practice is at the

    heart of data use, and Black and Wiliams metaanalysis points out that attending to the

    information gleaned from classroom-based formative assessment plays a significant role in

    addressing student needs.

    Other studies have examined schoolwide effects of data use. Marsh, McCombs, and

    Martorell (2010) examined the effects of instructional coaches on data-driven decision-making.

    While data analysis support was but one of many supports provided to teachers by the coaches,

    the authors noted that the majority of coaches focused considerable attention on data use, and

    that data analysis support was associated with higher student achievement outcomes on the

    Florida Comprehensive Achievement Test (FCAT) at the middle school level and with perceived

    improvements in teaching.

    In a six-year study that drew from data collected in nine states, 43 schools, and 180

    campuses, Louis et al. (2010) looked at breadth and patterns of data used by principals. The

    authors concluded:

    When schools are considered in the aggregate, typical approaches to data use by districts

    and principals have no measurable influence on student achievement.