EBP for school libraries Building evidence of your impact

Lyn Hay Charles Sturt University

your day

our profession our evidence

your day

our profession our evidence

Program 9.00am Registration / Welcome 9.30am The What and Why of EBP 10.30am Morning Tea 11.00am How to Build Your Evidence Base 12.00noon The Who and When of EBP 12.30pm Lunch 1.30pm Developing your EBP portfolio 3.00 pm Close

What did we learn?

The challenge: articulating how school libraries impact on student learning Assumption that by

virtue of actions, outcomes happen

What did we learn? Move from TL ‘actions’

to student outcomes Lack of systematically

gathered, empirical evidence

Barriers to evidence building

Evidence begins with TL action

Developing capacity as evidence-based practitioners

“We recommend that the profession as a whole needs to develop the capacity to articulate needs from research-based evidence and local evidence collected in the school.”

The verdict Acknowledged

international research evidence as useful A lot of anecdotal

evidence presented EBP takes time Documenting

evidence is critical

IT’S TIME!! building an EBP movement in Australia

https://www.facebook.com/groups/EBPforSL/

What is evidence-based practice? What is evidence?

Our 3 pronged approach

• Evidence-Based Practice: concepts, research findings, issues and enablers

• Evidence-Based Approaches: strategies for action

• Evidence-Based Action: your commitment and action plan

Evidence-Based practice • Evolving concept in many professions as new approach to

professional practice

• 1990s: Medicine & Health Care professions; gradual take up by many service professions: duty of care & optimum outcomes

• 2000+ Intersection of evidence-based education & evidence-based librarianship - international movement - sense of urgency that teacher librarians engage with data to underpin & inform practice and measure outcomes more rigorously

Evidence-Based Education

Practices

Programs

Scientifically Based Research

Benchmarks

Local Data

Objective Measures

Empirical Evidence

Personal Experience

Consensus Views

Professional Wisdom

Evidence-based Education

(Whitehurst, 2002)

Evidence-Based School Librarianship Professional practice that systematically uses: - research-derived evidence - teacher / TL - observed evidence meshed with experience and wisdom - learner-reported evidence

decision making, development & continuous improvement of school libraries program processes and impact on student

achievement, reading / literacy outcomes advocacy / promotion informs ongoing research agenda Central focus: student’s information-to-knowledge experience enabled though school library initiatives, and adding value to the school’s goals LOCAL FOCUS

Holistic Model of Evidence-Based Practice for School Libraries

Evidence FOR Practice

FOUNDATION INFORMATIONAL

Existing formal research provides the essential building blocks for professional practice

Evidence IN Practice Applications /

Actions

PROCESS TRANSFORMATIONAL

Locally produced evidence Data generated by practice is meshed with

research-based evidence to provide a dynamic decision-making and learning environment:

TL’s evidence / thinking / wisdom

Evidence OF Practice Results – impacts &

outcomes

OUTCOMES FORMATIONAL

User-reported evidence where learners change as result of inputs, interventions,

activities, processes

Todd, R. (2008) The Evidence-Based Manifesto for School Librarians www.schoollibraryjournal.com/article/CA6545434.html

Student Learning Through Ohio School Libraries 13,000 students

Evidence-Based Practice

“Without quality we are condemned to be ordinary. With it, we have a shot at being part of the future” (Koechlin & Zwaan)

Ross Todd’s Variation “Without evidence we are condemned to be oblivious. With it,

we have a shot at being part of the future”

Gary Hartzell “Occupational Invisibility”

Others often do not see depth, breadth and importance of what teacher librarians contribute to learning in schools

Evidence-Based Practice Gathering evidence in YOUR local school

You are able to provide convincing evidence that answers

these questions:

“What differences do my school library and its learning initiatives make to student learning outcomes?

“What are the differences, the tangible learning outcomes

and learning benefits of my school library”?

EVIDENCE = ADDING VALUE: Learning, Living, Life

EBP – Issues and Concerns

• Accountability: Threat to professional authority and autonomy; “proving our worth”

• I have to be a researcher: information literacy competencies • Our goal is lifelong learning, so how can we identify outcomes?

Evidence enables feedback for tomorrow • EBP detracts from the job! What then is your job? • Time: It is about priority • Professional Development: examples models, templates

Emphasis On Information • Number of classes in the library • Number of library items borrowed • Number of students using the library at lunch times • Number of items purchased annually • Number of web searches • Number of books lost

And who can do this?

Emphasis on Knowledge Understanding how school libraries help kids learn: Learning outcomes in terms of

– Knowledge outcomes – deep mastery of content through inquiry

– Information processes: transformation of information into new knowledge through inquiry

– Information technology: using tools to construct representations of knowledge and to present new knowledge

– Reading for learning in digital and print formats – Independent learning: knowledge transfer – Attitudes and values of information, learning – Self concept and personal agency

And who can do this?

Focus on curriculum

standards, national library and state library standards and benchmarks, research evidence

Focus on formational

Focus on informational and transformational

SOURCES OF EVIDENCE

Mission Statements & Policy Documents Contexts for Evidence

School Library Policy and Mission Statements: • Redefine school libraries from Place to Experience • Shift in focus from School Library as collections,

access, information literacy, to inquiry, thinking, knowledge development and learning outcomes

Does your mission and policy statements do this?

School Libraries as Verbs "Libraries are the verbs in the content standards. Wherever verbs such as read, research, analyze, explore, examine, compare, contrast, understand, interpret, investigate, and find appear in the standards, Teacher Librarians and library resources are involved." (Oxnard Union High School District)

http://www.ouhsd.k12.ca.us/lmc/ohs/stronglib/StrongSLMP.ppt

Your Role Statement • Positioning of the Teacher Librarian as the

Information-Learning / Inquiry Learning Specialist

• Implement inquiry-based instruction in the curriculum to enable curriculum standards – team based

• An instructional designer for integrating

information, technical and critical literacies into the curriculum – professional development role

• Reading for learning and living in the digital

age • Continuous improvement of learning

outcomes through resource-based learning

Did they learn anything? What knowledge of the

curriculum did they come away with?

QUALITY TEACHING = MOST POWERFUL EFFECT ON

STUDENT ACHIEVEMENT

EBP Knowledge Outcomes

Intellectual Quality Intellectual Agency Deep knowledge

Deep understanding Problematic knowledge Higher order thinking

Meta-language Substantive communication

Personal Agency Self Confidence

Willingness to take risks Trying new ideas and practices

Independence Autonomy

Social and Cultural Agency Respect for different values, cultural knowledges and viewpoints

Team building, collaboration, negotiation and decision making: inclusivity Knowledge integration: conceptual coherence and integration

Connect with current and future lives Social and ethical values

Pedagogy: Dimensions of Evidence

Deep Knowledge Knowledge is deep when focus is sustained on key concepts and ideas

Evidence: • Sustained focus on specific topic • Explanatory detail • Knowledge of hierarchical and associative relationships

Sources of Evidence • Research products • Oral presentations • Sustained communication

Deep Understanding • Students are able to demonstrate meaningful understanding

of the central ideas and the relationships between them Evidence: • Appropriate recall of complex ideas • Organized, structured sequence of ideas: knowledge

connections • Presentations and discussions around topic development • Focus questions go beyond fact finding • Formulation of conclusions, implications • Students use complex terms relevant to their subject • Students learn to search relevant databases using complex

language

Problematic knowledge • Students are encouraged to address multiple perspectives

and/or solutions and to recognise that knowledge is constructed though diverse ideas

Evidence: • Students understand the various arguments, evidences,

counterarguments • Acknowledge conflicts in information

Sources of evidence? Assessment rubrics

Higher-order thinking

• Students are engaged in thinking that requires them to organise, reorganise, apply, analyse, synthesise and evaluate knowledge and information

Evidence: • Students use information from more than one website,

compare it, select the best to use for the purpose, decide what is relevant for each aspect of task

• Students accurately interpret evidence • Students construct sound conclusions

Sources of evidence?

Substantive communication • Students regularly engage in sustained conversations about

the concepts and ideas – can manifest in oral, written, artistic forms

Evidence: • Increased confidence in presentations • Increased quality of classroom discussion after library visits • Increase in quality and frequency of questions about

research topics • Increased retention of content which is utilized in later

discussions

Sources of evidence?

Examples of claims • Following an inquiry-based unit of work, 70% students showed

improved ability in formulating essential questions that directed their inquiry as compared to their previous research task

• As a result of instructional interventions focusing on the development of arguments, students showed stronger ability to identify claims, provide supporting evidence and to identify rebuttals

• Students’ final products showed improved ability to analyze and synthesize information

• Students’ research reports showed improved ability to draw conclusions and state implications of their findings

• Students’ presentations showed ability to present different viewpoints and to provide a strong and supported case for their own position

Example of Claims • 93% of the students showed mastery of strategies for

evaluating websites for misinformation and bias • Following instructional interventions that focused on

establishing the quality of websites, 100% of the students’ bibliographies showed use of high quality websites

• Based on a pre-test of initial and final knowledge of the Year 8 science curriculum, students’ knowledge of their topics changed from descriptive and random listing of facts to statements that showed explanations, conflicting knowledge and making predictions

• Students’ products showed increasing complexity of language used to describe their knowledge, and the ordering of this knowledge into conceptually coherent units

• 83 % of the class show improved ability in thoughtfully analysing and evaluating major alternative points of view

your day

our profession our evidence

Program 9.00am Registration / Welcome 9.30am The What and Why of EBP 10.30am Morning Tea 11.00am How to Build Your Evidence Base 12.00noon The Who and When of EBP 12.30pm Lunch 1.30pm Developing your EBP portfolio 3.00 pm Close

How to Build Your Evidence Base

Tools and measures for charting & documenting evidence

Focus of Instructional Interventions

• Resource-based capabilities: These are abilities and dispositions related to seeking, accessing and evaluating resources in a variety of formats, including people and cultural artefacts as sources. They also include using information technology tools to seek out, access and evaluate these sources, and the development of digital and print-based literacies.

• Thinking-based capabilities: These are abilities and dispositions that focus on substantive engagement with data and information, the processes of higher order thinking and critical analysis that lead to the creation of representations/products that demonstrate deep knowledge and deep understanding.

• Knowledge-based capabilities: These are the abilities and dispositions that focus on the creation, construction and sharing the products of knowledge that demonstrate deep knowledge and understanding.

GIVING FOCUS TO OUTCOMES

• Reading to learn capabilities: These are the abilities and dispositions related to the transformation, communication and dissemination of text in its multiple forms and modes to enable the development of meaning and understanding.

• Personal and interpersonal capabilities: These are the abilities and dispositions related to the social and personal aspects of leaning about self as a learner, and the social and cultural participation in inquiry learning.

• Learning management capabilities: These are the abilities and dispositions that enable students to prepare for, plan and successfully undertake a curriculum-based inquiry unit.

• Developed from Ohio, Delaware and Australia studies (25,000 students)

GIVING FOCUS TO OUTCOMES

Kuhlthau’s Information Search Process

Tasks Initiation Selection Exploration Formulation Collection Presentation Evaluation ---------------------------------------------------------------------------------------------------------------------------------------------------------------

Feelings uncertainly optimism confusion clarity sense of satisfaction or (affective) frustration direction/ disappointment doubt confidence Thoughts vague----------------------------------------→focused (cognitive) ----------------------------------------------→ increased interest Actions seeking relevant information-------------------------------→seeking pertinent information (physical) exploring documenting

Zone of Intervention: the critical point / need for instruction

GUIDED INQUIRY (Kuhlthau, 2004, p. 82)

CISSL WEB SITE

• Student Learning through Inquiry Measure (SLIM)

• SLIM Handbook • SLIM Reflection Instruments and Scoring Guidelines

• SLIM Scoring Sheet

http://cissl.rutgers.edu/images/stories/docs/slimreflectionsheet.pdf

Inquiry Reflection Tasks Tasks 1, 2 & 3 consisted of the following questions 1. Write the title that best describes your research project at this time. 2. Take some time to think about your research topic. Now write down

what you know about this topic. 3. What interests you about this topic? 4. How much do you know about this topic? Check () one box that best

matches how much you know. Nothing, Not much, Some, Quite a bit and A great deal

5. Write down what you think is EASY about researching your topic. 6. Write down what you think is DIFFICULT about researching your topic. 7. Write down how you are FEELING now about your project. Check ()

only the boxes that apply to you. Confident, Disappointed, Relieved, Frustrated, Confused, Optimistic, Uncertain, Satisfied, Anxious or Other.

Hay, PhD research

Analysis of student bibliographies

• Diversity of choice of sources

• Depth / levels of knowledge

• Accuracy of citations

• Relevance to learning task

• Use of multiple formats

• Engaging with state-of-the art knowledge – recency / accuracy

• Reasons for choice of source

Rubric Strategies • Students’ performance in final products are scaled according to a

set of criteria that clearly define what is the range of acceptable to unacceptable performances and/or information products look like

• Compare with previous assignments where no instructional intervention took place

http://www.schrockguide.net/assessment-and-rubrics.html

Criti

cal T

hink

ing:

Mak

e Th

inki

ng V

isibl

e www.in

sightassessmen

t.com

/pdf_files/Ru

bric%20

HCTSR.pd

f

Critical Thinking Rubric http://www.winona.edu/AIR/resourcelinks/critical%20thinking%203.pdf

History Essay: Information Literacy in Action (Gordon)

Criteria Level 1 Level 2 Level 3 Level 4 Mks

Introduction / Thesis

-weak introduction of topic, thesis & subtopics -thesis is weak and lacks an arguable position

-adequate introduction that states topic , thesis and some of the subtopics - thesis is somewhat clear and arguable

-proficient introduction that states topic, thesis, and all subtopics in proper order - thesis is a clear and arguable statement of position

-exceptional introduction that grabs interest of reader and states topic, thesis, and all subtopics in proper order - thesis is exceptionally clear, arguable, well developed, and a definitive statement

/8

Quality of Information / Evidence

-limited information on topic with lack of research, details or historically accurate evidence

-some aspects of paper is researched with some accurate evidence from limited sources

-paper is well researched in detail with accurate & critical evidence from a variety of sources

-paper is exceptionally researched, extremely detailed and historically accurate with critical evidence from a wide variety of sources

/12

Support of Ideas / Analysis

-limited connections made between evidence, subtopics, counterarguments & thesis / topic -lack of analysis

-some connections made between evidence, subtopics, counterarguments & thesis / topic showing analysis

-consistent connections made between evidence, subtopics, counterarguments & thesis / topic showing good analysis

-exceptionally critical, relevant and consistent connections made between evidence, subtopics, counter-arguments & thesis / topic showing excellent analysis

/10

History Essay Organization / Development of Ideas

-paper lacks clear and logical development of ideas with weak transition b/w ideas and paragraphs

-somewhat clear and logical development of subtopics with adequate transitions b/w paragraphs

-clear and logical subtopic order that supports thesis with good transitions b/w paragraphs

-exceptionally clear, logical, mature, and thorough development of subtopics that support thesis with excellent transition b/w paragraphs

/10

Conclusion -lack of summary of topic, thesis & subtopics with weak concluding ideas

-adequate summary of topic, thesis and some subtopics with some final concluding ideas

-good summary of topic, thesis and all subtopics with clear concluding ideas

-excellent summary of topic (with no new information), thesis & all subtopics in proper order with concluding ideas that leave an impact on reader

/5

Language Conventions

- inconsistent grammar, spelling and paragraphing throughout paper

-paper has some errors in grammar, spelling and paragraphing

-paper is clear, with mostly proper grammar, spelling and paragraphing

-paper is very concise, clear, with consistently proper grammar, spelling and paragraphing

/5

Footnotes -inconsistent use of footnotes with limited details and improper format

- sometimes inconsistent use of footnotes with limited details

-consistent & correct format inserted to validate evidence

-proper detailed format always used consistently & correctly to validate evidence in paper

/5

Bibliography -lack of proper format and limited details with many sources missing or incomplete

-some errors in MLA format with most sources shown and a variety of sources

-mostly proper MLA format used in alphabetical order with all sources shown and a variety of sources

-proper, detailed MLA format always used in alphabetical order with all sources shown and a wide variety of sources

/5

Evidence through formal feedback strategies

Examples • Simple feedback – survey every term on what the library

does “best” and “least” to help students with their school work

• Feedback at end of instructional unit: what helped and did not help

• Feedback on quality of resources • Feedback on what students could do better at • Use of “How School Libraries Help” Survey instrument

(Ohio / Australia) • Judicious use of evidence: How learning is enabled and

hindered

Use Student Voice statements • (Respondent 777) When I was working on a project about

science I had no idea what I was doing I asked my library teachers for help they helped and by the end of the day I felt so much better!!! And from that day on I knew what I was doing on that project and I got a A I was so proud of myself and my confidence went up a whole lot and now when ever I do a project I know I have a lot of power now to do well on projects!!!

• (Respondent 1532) The school librarians don’t help me at all like they make me do all the stuff myself and wont tell me where the things are even when I already looked – they show me and make me learn how to find the stuff myself and its hard work!!!! You gotta use your brain, they say

Personal Reflection

• What worked well? • What pleased me? • How do I know that it was successful? • Who can/did I share my success with? • What did I do that helped me to: - prepare for the task - create new ideas - practise new skills - improve existing skills - modify my learning habits - find relevant information or materials - organize information or materials - correctly summarise information - understand unfamiliar ideas - take relevant notes - use my existing knowledge or skills - represent information in meaningful ways? • What could I do differently next time? • What factors influenced my ability to learn? • What might help me learn more about this?

Reflections: Students • Question formulating was difficult and required a lot of thinking and deliberating. • I was very interested in my topic, it made me think what life was like back then. I want

to read more about my topic. • These skills have come in handy for some of my other projects. • I saw the value of multiple sources to cross check information for reliability • I loved making up my own questions. (This) made me more comfortable with what I

was researching. • The thing you have to know before you start researching is if you are interested in the

topic of not. And if you are not, you will not make a big effort to find the information you want. So, always choose a topic that you are interested in!...

• Research takes time and patience • You don’t have to use all the information you collect. • Instead of just writing down the facts you also need to elaborate and interpret it. • I learnt that having a choice of topic meant that I had freedom and individuality. I

enjoyed this because I was able to work on something no-one else did. More of this type of work would be good.

• I learnt the process of doing the assignment is just as important as the final product. • I now understand that I need to search for very specific information which answers my

question rather than finding general information like we usually do

Analysis of standardized test score data • Matches between scores and high-use

library groups • Matches between scores and reading

enrichment programs • Improvement in critical thinking on

test score measures after inquiry-based interventions

• Target gaps in scores for Library

Improvement Planning

Reading Initiatives • Motivation to read: pre / post

reading program • Access to reading materials • Readers’ Advisory logs • Reports of reading celebrations,

events, initiatives, projects + student voice

• Impact of SSR programs • Logs of reading amounts: free

voluntary reading and topical reading

• Classroom reading audit

Teacher- Related

Strategies

• Number of collaborations • Instructional focus of

collaborations (curriculum standards and inquiry-based interventions)

• Subject / grade levels • Teacher observations of

student engagement • Teacher summary of

outcomes • Teacher summary of benefits

ACTION RESEARCH A tool of evidence-based practice

What is action research

• Action research is both an orientation to inquiry and with an obligation to action

• Seeks to illuminate ‘the local’ • Provide practitioners with insight and

understanding through forms of systematic inquiry that address issues and questions that are of significance to those concerned

(Davis, Fletcher & Groundwater-Smith, 2009)

It requires a capacity for the systematic gathering of evidence related to a specific problem or challenge and made available for public scrutiny and debate that will allow the ways in which practitioners think and act to be more transparent than is the case when they work in the isolation of their classrooms or offices.

(Davis, Fletcher & Groundwater-Smith, 2009)

What is action research

Is conducted in order that those who participate may engage in becoming critical of the conditions of their practice

• CAR employs qualitative interpretive modes of inquiry and data collection by teachers with a view to teachers making judgments about how to improve their own practices It can be transformational (Davis, Fletcher & Groundwater-Smith, 2009;

Kemmis & McTaggart, 2005)

Planning a change Acting & observing the

process & consequences of the change

Reflecting on these processes & consequences

Replanning Acting and observing again Reflecting again

Replanning...

Spiral of self-reflective cycles

(Kemmis & McTaggart, 2005)

(Gordon, 2006)

• Main bullet point slide

(Gordon, 2006)

Benefits of action research Directly addresses the problem of the division

between theory & practice Integrates the development of practice with the

construction of research knowledge in a cyclical process

Excellent approach to support practitioner research

Integral to day-to-day practice of teachers can have immediate impact

Benefits of action research A method that provides for both individual

professional development as well as collaborative institutional change (Herr & Anderson, 2005)

Reflection helps teachers understand what they are currently doing, why they are doing it, whether it is what they want to do, and what they should do in the future (Patterson, 1996; Gordon, 2006)

Increasing confidence of teachers in their ability to reflect on their standard practice and the potential to improve their future instructional role (Gordon, 2006)

AR, EBP & the TL Practitioner-researchers engaged in research that

modeled methodology for students doing inquiry learning and for teachers who collaborated with school librarians to conduct action research

Action research emerged as a professional development tool that used the school library as a laboratory where educators, as well as students, learned to be reflective and self-critical through systematic collection and analysis of evidence

(Gordon, 2009)

AR, EBP & the TL AR project had a positive effect on TLs’ practice,

their confidence in managing the collaboration process, and their perceptions of themselves as leaders and researchers.

Replication of the AR study has shown a transformative effect on the school district

Decision-making in schools, especially in the context of instruction, is driven by a culture of evidence based practice, inquiry, collaboration, and action research (Gordon, 2009)

4 ways of approaching AR Lead an action research project – seek AR

partnerships in school & beyond. Start small!! Seek out proposed or existing AR projects in

your school with teachers Seek out proposed or existing AR projects

with a TL community of practice As a newbie AR, seek out a mentor or CoP for

support

Reading preparation Callison, D. (2007). Action research. School Library Media Activities Monthly, 23(10). Available

http://www.schoollibrarymonthly.com/articles/Callison2007-v23n10p40.html Farmer, L.S.J. (2003). How to conduct action research: A guide for library media specialists.

Chicago: American Association of School Librarians. Gordon, C.A. (2006). A study of a three‐dimensional action research training program for

school library media programs. School Library Media Research , 9(1), 1‐21. Available http://www.ala.org/aasl/aaslpubsandjournals/slmrb/slmrcontents/volume09/gordon_study3daction

Gordon, C.A. (2009). An emerging theory for evidence based information literacy instruction in school libraries, Part 2: Building a culture of enquiry. Evidence Based Library and Information Practice, 4(3). Available http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/6449

Harada, V. H. (2004). Action research: How teacher-librarians can build evidence of student learning. Scan, 23 (1), pp. 27-33.

Joyce. M.Z. & Martin, B.S. (2005). Implementing and promoting library programs through action research. AASL 12th National Conference, Pittsburgh, Pennsylvania. Available http://www.thinkingfoundation.org/research/action_research/pdf/2005oct-AASL-actionresearch.pdf

Padak, N. & Padak, G. (2012). Research to practice: Guidelines for planning action research projects. Ohio Literacy Resource Center. Available http://literacy.kent.edu/Oasis/Pubs/0200-08.htm

Sykes, J.A. (2002). Action research: A practical guide for transforming your school library. Libraries Unlimited.

your day

our profession our evidence

Program 9.00am Registration / Welcome 9.30am The What and Why of EBP 10.30am Morning Tea 11.00am How to Build Your Evidence Base 12.00noon The Who and When of EBP 12.30pm Lunch 1.30pm Developing your EBP portfolio 3.00 pm Close