measuring inter-personal and intra- personal...

MEASURING INTER-PERSONAL AND INTRA-

PERSONAL COMPETENCIES -

Laura Hamilton, Jim Soland, Brian Stecher, Kun Yuan

• Guidance for Selecting Assessments

• Promoting a High-Quality Research and

Development Agenda

• Developing an Assessment Repository

Stecher, NRC, July 30, 2015, 1






Development Agenda



Guide to Educators for Selecting

Assessments

• In conjunction with the

Asia Society

• What are 21st century

competencies?

• How are they assessed?

• What to consider when

choosing an assessment?


Theory of Mind

Global Empathy Scale

Watson-Glaser

Formulating Hypotheses

SimScientists

Graduation Portfolio System

World Savvy Challenge

College and Career Ready School Diagnostic

Queensland Performance Assessment

Work Extrinsic Intrinsic Motivation Scale

Advanced Placement

Grit Scale

PARCC & Smarter Balanced

Alelo Language & Culture Simulator

Singapore Elementary Portfolios

EcoMUVE

Mission Skills Assessment

PISA

There Are Many Assessment Tools

from Which to Choose


Published on PARCC (http://www.parcconline.org)

Home > Grade 3 > Grade 3 Mathematics (The Field)

Grade 3 Mathematics (The Field)Sample Item

[1] [1]

Part AA farmer plants 3/4 of the field with soybeans.

Drag the soybean to the field as many times

as needed to show the fraction of the field

that is planted with soybeans.

Farmer's

Field

Soybean

Reset [1]

.For More Item Specific Information

PARCC Math Sample Problems_GR3_The Field_PartAV2.pdf [2]

Grade3MathTypeIISampleItem-TheFieldPartB.pdf [3]

..Source URL: http://www.parcconline.org/samples/mathematics/grade-3-mathematics-field

Links:

[1] http://www.parcconline.org/javascript%3Avoid%280%29

[2]

http://www.parcconline.org/sites/parcc/files/PARCC%20Math%20Sample%20Problems_GR3_The%20Field_PartAV2.pdf

[3] http://www.parcconline.org/sites/parcc/files/Grade3MathTypeIISampleItem-TheFieldPartB.pdf


56

CPS skill(s) assessed

(A1) Discovering perspectives and abilities of team members; (A2) Discovering the type of collaborative

interaction to solve the problem, along with goals; (C1) Communicating with team members about the

actions to be/being performed; (B1) Building a shared representation and negotiating the meaning of the

problem (common ground)

The following figure illustrates Task 2:

PISA 2015

Task 3 of 7

CHAT

?Unit name: The Aquarium Time remaining: 13 minutes

Tryout conditions

Bad GreatOK

Results: these conditions are suitable, but they can be better.

Results

Abby

Its not great. What should we do

now?

Let’s change the temperature

You

You and Abby have 5 trials to find the best

conditions for the fish to live in the aquarium.

Abby

Wait. I’m not sure that this is the

right strategy

Water type:

Scenery:

Lightning:

Fresh Sea

Rocky Plants

Low High

Control panel Abby’s control panel

Food type:

Fish:

Temperature:

Dry Food blocks

Few Many

Low High

o Why do you think that?

o Let’s change the

o I know that this is the right thing to do

You

scenery

Task 3: Monitoring and repairing the shared understanding

Activity

Item 1: TT implements the plan as discussed with Abby. TT monitors if Abby followed the plan as

discussed. Abby’s controls show that she is following the plan.

Item 2: TT shares his/her understanding of the result (fish conditions).

Item 3: TT repairs Abby’s misunderstanding of the result.

Item 4: TT has to offer a plan of how to proceed (e.g. "let’s change this variable to start"). If the TT doesn’t

offer an idea then Abby can prompt. If still no idea is offered then Abby will suggest an idea herself.

Item 5: TT asks Abby for her point of view before implementing the plan. If the TT doesn’t then Abby

shares her perspective with the TT.

Convergence

There is a change in the aquarium variables. The results of the trial are presented.


doi: 10.1037/t07896-000

Pathway

Academic behaviors

Goal-driven behaviors

Persistence

Self-monitoring

Study skills

Goal-driven behaviors

i17. I take complete, organized, and accurate notes during class

i23. I am able to identify resources needed to complete a task or project

i32. I have researched educational option(s) that match my strengths or interests

i9. I participate in study groups outside of class

i3. I discussed with my guidance counselor the plans I have for after graduation

i1. I can identify steps I need to take to reach my career goals

i2. I can identify steps I need to take to reach my educational goals

Persistence

i42. I allow enough time to complete multiple drafts of an assignment if necessary

i56. I refer to the syllabus or class website to prepare for and complete course assignments

i40. I complete assignments even when they aren’t collected or graded

i29. I talk to adults when I have concerns about school

i43. I balance schoolwork with my other activities and commitments

Study skills

i10. I work in small groups during class time

i33. I have identified specific classes in high school that are a good match to my strengths

i34. I journal or blog about what I learn

i39. I analyze the questions on a test to plan my approach prior to answering

i37. I spend enough time studying for exams to learn the material well

i36. I try to predict the possible test questions in preparation for exams

i41. I use unstructured time during the school day to complete assignments

Self-monitoring

i53. I know the best environment and time of day for me to study

i51. I complete my projects and assignments without being reminded

i6. I figure out what score I need on tests or assignments in order to get the grade I want in a class

i18. I enjoy challenging myself in school

i28. I review materials ahead of time that will be discussed in class

i30. I find out what I missed on exams to make improvements

College & Career Ready School Diagnostic

CCRSD

PsycTESTS™ is a database of the American Psychological Association


The Choice of Tool Should Be

Based on Purpose



Based on Purpose


System

Monitoring


Based on Purpose


System

Monitoring

Setting

Priorities


Based on Purpose


System

Monitoring

Accountability

Setting

Priorities


Based on Purpose


System

Monitoring

Accountability

Setting

Priorities

Improving

Instruction

TECHNICAL

PRACTICAL INSTRUCTIONAL

reliability

validity

fairness

training

scoring

cost

actionable information to teachers

feedback to students

formative or summative

appropriate

authentic

encourage effective teaching and learning technology

administration

Other Factors Should Also Be

Considered


TECHNICAL


reliability

validity

fairness

training

scoring

cost




appropriate

authentic


administration


Considered


Lack high-quality, practical,

accessible assessments for many

intra-personal and intra-personal

competencies


Inadequate Choice of

Assessments

Developing an R&D Agenda

• Draws on expertise from

75 researchers,

policymakers, practitioners

and funders

• Based on formal meetings,

individual interviews, and

literature review


Defining and selecting constructs

Identifying the intended use(s)

of the measures

Developing practical measures

Evaluating their technical quality

Documenting consequences of use

Five Broad R&D Tasks


Which Competencies Should Be

Addressed First?

Adequacy of existing measures

Educational efficacy

• Competencies are malleable in school

• Important to stakeholders

• Technical quality

• Number

• Practicality

Difficulty of developing new ones

• Depth of understanding

• Familiarity with measurement

strategies


Which R&D Goals Should Receive

Priority?

Goal: A balanced portfolio with some investment

in each type of research.

Basic research

(especially intrapersonal)

New measures for well-

understood construct

Assess or improve an

existing measure

Investigate the consequences


Practical Questions that Must Be

Answered

How long will R&D take?

How much will R&D cost?

How should R&D be organized

and managed?


Managing R&D: Research

Coordinating Boards

• Independent

• Diverse expertise

• Coordinate but not conduct the research

• Assemble, allocate and monitor the use

of philanthropic and public resources

• Revisit plans regularly


Challenges

• Reaching consensus

• Maintaining standards

• Generating public support;

maintaining interest

• Staying the course

• Sustaining a collaborative

culture


Building a Repository of

Assessments

• Feasibility Study

• Build a database

• Find, Assess, Catalog,

Arrange, etc.

• Develop online portal

• Search, Compare, Share,

Administer, etc.


measuring inter-personal and intra- personal...

Documents