how was laa 2 developed?

How was LAA 2 developed?

Committee of Louisiana educators (general ed and special ed)

•Two meetings (July and August 2005)

•Facilitated by contractor West Ed

LAA 2 Development Content and grade-level committees identified:

•Range of abilities/capabilities of LAA 2 population•Range of appropriate sample items taken from LEAP

and GEE item banks•Appropriate grade level expectations (GLEs)

regarding breadth and depth

Reviewed proposed assessment design and made recommendations regarding:•Content depth, breadth, and difficulty• Item types (MC and CR)•Test length•Test accommodations•Test format

LAA 2 Development LAA 2 Items:

•Selected from LEAP and GEE item banks

•Reviewed by LDE staff: assessment development and special populations

•Prior review of items by Content and Bias Committees

How does LAA 2 differ from LEAP, GEE, and iLEAP?

LAA 2 has• fewer questions• less reading• less writing • a modified format

fewer questions per page more white space larger font

Most Significant Format Change English Language Arts

•Using Information Resources (UIR) The test question follows the resource it references.

•Proofreading Each of the 8 items is a complete sentence with an underlined part to edit.(no intact passage for students to read)

How is LAA 2 like LEAP, GEE, and iLEAP?

LAA 2•Based on Louisiana content standards

(modified content standards)•Includes both multiple-choice and

constructed-response questions•Requires the same procedure for test

administration and test security

Who took LAA 2 in 2006?

Student whose IEP reflected a functioning grade level in ELA and/or Mathematics at least three (3) grade levels below the grade in which he/she is enrolled

Student who scored Unsatisfactory in ELA and/or math on the previous year’s LEAP/GEE or one who participated in LAA 1

Student whose instructional program is predominantly academic in nature

Student who met the criteria listed on the LAA 2 Participation Criteria Form (www.louisianaschools.net)

LAA 2 in 2005–2006

Grades 4, 8, and 10• English Language Arts• Mathematics

Grade 11

• Science• Social Studies

LAA 2 in 2006–2007 Grades 4, 5, 6, 7, 8, 9, and 10

• English Language Arts• Mathematics

Grade 11• Science• Social Studies

LAA 2 in 2007–2008 Grades 4, 5, 6, 7, 8, 9, and 10

• English Language Arts• Mathematics

Grades 4, 5, 6, 7, 8, and 11• Science• Social Studies

Standard Setting for LAA 2

Convened Standard Setting Committee • June 8-9, 2006 in Lafayette•Educators representing general and special

education •Content and grade-level expertise•Parent and community representatives•Selection based on: ethnicity, geographic location,

and past experience in development activities related to standards and assessments

Facilitated by testing contractors:•Data Recognition Corp. and West Ed content and

psychometric staff •LDE staff present in each group

Standard Setting for LAA 2

Purpose•To establish the level of ability students

would be required to have to be placed into any one of the four achievement categories, which are:

Basic Approaching Basic Foundational Pre-Foundational

LAA 2 Achievement Levels

Top two LAA 2 achievement levels:•Basic•Approaching Basic (AB)

Align with the Basic and AB levels for LEAP and GEE

•Students scoring at these 2 levels on LAA 2 have similar performance to students scoring at Basic and Approaching Basic on LEAP or GEE.

LAA 2 Achievement Levels

Lower two levels of LAA 2:•Foundational•Pre-Foundational

•Intent is to differentiate the performance of LAA 2 students who would fall into the “Unsatisfactory” level on LEAP and GEE

Alignment of Achievement Levels: LEAP/GEE and LAA

2

LAA 2 Achievement Level Policy Definitions

Basic: A student at this level has demonstrated only the fundamental knowledge and skills needed for the next level of schooling (same as for LEAP, GEE, and iLEAP)

Approaching Basic: A student at this level has only partially demonstrated the fundamental knowledge and skills needed for the next level of schooling (same as for LEAP, GEE, and iLEAP)

LAA 2 Achievement Level Policy Definitions

Foundational: A student at this level has not demonstrated the fundamental knowledge and skills needed for the next level of schooling but has demonstrated the foundational knowledge and skills that can be built upon to access the grade-level curriculum.

Pre-Foundational: A student at this level has not demonstrated the fundamental knowledge and skills needed for the next level of schooling. However, the student may be developing the foundational knowledge and skills that can be built upon to access the grade-level curriculum.

LAA 2 Accountability Points

LAA 2 Achievement Level SPS Points Subgroup

Basic 100 Proficient

Approaching Basic 75

Non-proficient Foundational 50

Pre-Foundational 0

Standard Setting Process

Panelists were trained extensively in large group setting: “Bookmark” Method• The method used to set the standards (cut points).• A tried, true, and defensible standard setting

methodology• Method used to set standards for LEAP, GEE, and iLEAP

(Preliminary) Achievement Level Descriptors (ALDs)• ALDs describe the content and skills students taking

LAA 2 should know and be able to do at each achievement level.

• They guide the bookmark process.

Small group setting (content and grade level)• Took the LAA 2 assessment

Bookmark Procedure

Panelists were given a LAA 2 Ordered Item Booklet (OIB).•LAA 2 test items, ordered from

easiest to hardest, based on 2006 scale score data

Bookmark Procedure Using the Preliminary ALDs to guide their

decisions, panelists placed a bookmark (post-it) in a location where they thought there was a separation between achievement levels.

•Ex: To place a cut between the Approaching Basic and Basic level, panelists reviewed the items in the OIB, starting at the beginning, and asked themselves, “Would I expect a student at the top of the AB level to answer questions like this accurately most of the time?”

•Reminder: Focus is what a student “should be able to do,” not what he/she “can” do.

Bookmark Procedure Panelists had at least 2 rounds to place their

bookmarks at each cut.•Round 1: Panelists made judgments

individually, then discussed their cuts as a small group (approx. 5) and as a large group (approx. 10).

After round 1, panelists were presented with percentages of students that would fall within in each achievement level, based on group recommended cuts. Further discussion ensued.

Bookmark Procedure Round 2: Panelists had another

opportunity to place bookmarks; the same procedures were followed.

Round 3: If necessary, a 3rd round was conducted. (seldom occurred)

Group consensus was not required.

Standard Setting Results

Psychometric staff (contractor and LDE) reviewed the final recommended cut points.

Final cuts were converted into scale score ranges.

SBESE Approval of LAA 2 Achievement Levels

August 2006, SBESE approved:•Recommended cut points for

the four LAA 2 Achievement Levels

•Final Achievement Level Descriptors

????? Questions ????? Claudia.Davis@la.gov

•Phone: 225-342-3393

Jeanne.Johnson@la.gov•Phone: 225-342-1722