national accessible reading assessment projects goals of project narap collaboration general...

National Accessible Reading Assessment Projects

Goals of Project

NARAP Collaboration

General Advisory Committee

Project Details (ETS and PARA)

Plans for Year 1

Overview


Goals for Project1. Develop a definition of reading proficiency 2. Research the assessment of reading

proficiency 3. Develop research-based principles and

guidelines making large-scale reading assessments more accessible for students who have disabilities that affect reading

4. Develop and field trial a prototype reading assessment


NARAP CollaborationProposal hinted at collaboration and we decided to bring it alive – in several ways:

• Name for Collaborative Entity

• Shared Web site

• Shared Committees (GAC, Definition Panel)

• Collaboration on Activities



RFP Language: Form a general advisory committee on which representation is invited from the National Center for Education Statistics; the National Assessment Governing Board; the National Center on Educational Outcomes; the National Center for Research on Evaluation, Standards, and Student Testing; organizations representing test developers, reading educators, and researchers; disability groups; and other appropriate projects and organizations.



RFP Language: The primary purposes of this advisory committee are to review and advise on the general plans of the project and to provide liaison with significant stakeholder groups.


QuestionDoes the Committee have any ideas about how to maximize collaboration between the projects? Or, does the committee have any concerns about the collaboration of the two projects, and suggestions for things to avoid?


Project Details (ETS)• Focus on learning disabilities• Component approach to assessing

reading– Isolate individual components of reading

(e.g., comprehension or word recognition) into subtests

– Allows for accommodations (e.g. audio presentation) to be specific to subtest without impact validity of entire test


Research Agenda (ETS)

• 2005– Focus Groups– Differential Boost– Factor Analysis – Differential Item Functioning


Research Agenda (ETS)

• 2006– Focus Groups– Cognitive Labs– Usability Studies– Diagnostic Score Reporting


Project Details (PARA)Partnership for Accessible Reading Assessment

Partners = National Center on Educational Outcomes (NCEO)

Center for Research on Evaluation, Standards, and Student Testing (CRESST)

Westat, Inc.

Disability Focus = All disabilities that affect reading (for field test, will have sufficient samples of students with Learning Disabilities, Speech Language Impairments, Mental Retardation, and Deaf/Hearing Impairments)


PARA – Seizing the Opportunity to Build on the Expertise of Others!

Reading proficiency is being approached from the lens of students with vastly diverse needs

There is a need for expertise and input of reasoned scholars, key national stakeholders, expert advisors, and end users of assessments.



Definition Panel

Technical Advisory Committee

Principles Committee

GOAL 1

Formulate a definition of reading proficiency, analyze the definition in relation to state standards, obtain input, and refine the definition

GOAL 3

Develop research based principles and guidelines for making large-scale assessment of reading proficiency more accessible

GOAL 4

Develop instruments and methods for assessing reading proficiency that are suitable for large scale administration

Review and advise the project on general plans and to liaison with significant stakeholder groups

Formulate working definitions of key terms for use in collaborative effort with other funded projects

GOAL 2Conduct a program of research on the assessment of reading proficiency to determine the effects of various factors of test development, design and administration

Formulate working papers on research based principles and guidelines for collaborative effort

Review and provide technical input on the specific research and development plans and activities

Col

labo

ratio

nProject Details (PARA)


Research Agenda (PARA)General Questions:

1. Sampling Issues: Which students need which kinds of assistance to access the types of reading assessment that states currently administer?

2. Validity of Accessible Assessments: How can a valid assessment be developed that will permit students with disabilities that affect reading to demonstrate their reading proficiency?


Research Agenda (PARA)General Questions:

3. Universal Design and Individual Student Characteristics: What common forms of assessment are appropriate for group testing that are accessible to students with disabilities that affect reading?

4. Instructional Sensitivity/Opportunity for Use of Access Tools: What opportunities do students with disabilities that affect reading have to access reading through alternatives to print in the classroom?


Plans for Year 1 Develop “reading proficiency” definition

Projects hold meetings with advisory groups (GAC, TACs)


Work of the Definition Panel

(DP)

Leadership Roles, Work Group, Membership of the DP, Tasks and Timeline


Work Group

• Group co-leaders: Deborah Dillon, University of Minnesota and John Sabatini, ETS

• Members: Ann Clapper, NCEO; Laurie Cutting, ETS and Kennedy Krieger Institute; Lee Galda and David O’Brien, University of Minnesota


The Role of the Work Group

• Determine membership of the DP

• Develop agendas for the DP meetings

• Outline tasks for the DP

• Identify the products that will emerge from the work of the DP


Membership of the DP

• Reviewed the current proposed membership ( 9 members plus 2 from ETS and 3 from the U of MN)

• Considered new panel members and a rationale for their inclusion

• Struggled with the need for expertise (re: the task at hand) and the potential for unwieldiness with a large panel


Agendas for DP Meetings

• Teleconference Meeting—overview tasks and timeline; begin to identify research that we need to review

• Document Reviews—request that panel members review selected reports and drafts of the “Synthesis Document”

• Face-to-Face Panel Meeting/s—seek panel members’ consensus on the working definitions of key terms and the definition of “reading proficiency”


DP Tasks

• Identify critical issues and terms in addition to “reading proficiency” needed for the collaborative efforts

• Identify relevant research literature for the comprehensive review to generate a working definition of reading proficiency and other related terms


DP Tasks—cont.

• Draft a “Synthesis Document”—this document is comprised of key pieces of research, panel meeting minutes, and a review of current literature and research reports (e.g., RAND; NAEP Reading Framework for 2009); a list of key terms (glossary) will also be documented

• Identify issues and points in the document that need clarification


Products from the DP and Continued Work

• Synthesis Document; formulation of a definition of Reading Proficiency

• Supporting documentation for the definition

• Focus Groups—individuals will respond to the work of the DP and offer feedback

• Reconvene the DP to finalize the definition based on external feedback

national accessible reading assessment projects goals of project narap collaboration general...

Documents

types of reading assessment

assessment slide

national center

valid assessment

accessible goal

appropriate projects

key national stakeholders

funded projects goal