computerized adaptive testing and multistage testing: in
Post on 29-Oct-2021
5 Views
Preview:
TRANSCRIPT
智慧评测与智慧学习
张华华 (hhchang@Illinois.edu)
美国伊利诺伊大学教育心理兼心理和统计学教授
华东师范大学长江学者讲座教授
全通教育特邀嘉宾
2015-10-25 中国杭州
内容:“互联网+评测” 1. 智慧评测? 2. 大数据带来哪些挑战? 3. 自适应技术
1. 统计模型 2. 智慧题库 3. 自适应测验 4. 认知诊断
4. 智慧评测帮助智慧学习 – Making Teaching/Testing Adaptive – Paper/pencil Test Adaptive
5. 总结
2
从智慧评测到智慧学习---如何使用 评测技术帮助新一代的教育
张苏苏、张华华 (2015)
3
4
5
Knewton:世界最大的自适应学习平台
“A is for Adaptive-- Personalized learning is
poised to transform education. Can it enrich students
and investors at the same time?” (Time, June 17, 2013)
7
Bradley: “Knewton is based on what you can do, not what the class can do” Clenna: “It adapts to you, so it starts easy and then gets harder”
可汗学院(KHAN Academy)
8
SMART EDUCATION – INNOVATIONS AND ATTEMPTS Personalized learning routes • Knewton system Blended learning • Khan Academy • Intelligent Tutoring Systems • e-Schoolbags/student portfolios Open Educational Resources and Massive Online Open Courseware • Coursera • EdX • International expansions and personal device use
9
MOOCS
MIT OCW and the OER initiative Massive Online Open Courseware (interactive learning): • Instruction • Resource provision • Peer grading • Forum discussion • Certification edX, Udacity, Coursera (>10 mil users, 839
courses, 114 institutes as of 2014)
Personal device and mobile interfaces • Self-sufficient
10
智慧教育当前面临的最大挑战 • MOOCs退学率居高不下
• 可能原因: – 成年人时间宝贵
– 无因人而异的教学起点
– 与基于实际表现的人员选拔冲突
• 有那么一点“放任自流”
挑战: 如何可靠地确定个人的需求来筛选教学资源、并“量身定制”学习内容?
如何在MOOC’s中加入“自适应”
– 基于序列设计的自动化控制
11
12
个体化学习---因材施教 祝智庭提出的模型:
“智慧教育”需要“智慧评测” 以促成个体化的 “自适应”学习
13
智慧评测包括以下几个方面
• 计算机自适应考试 (CAT)
• 认知诊断模型 (CD)
• CAT + CD
• 在线估计(Online calibration)
• 客观题阅卷的自动化 (Performance based assessment)
• 无偏化评测、考试公平性 (DIF and test equity)
• 纸笔考试自适应化 (Paper pencil based adaptive testing)
14
15
是“题库”还是“题堆”? 题库自动化组卷实例
16
自动化试卷生成
原始卷
卷 1 卷 2 … 卷 N
题库
Parallel Parallel
Parallel
Select items from the item pool
Parallel Assembly
• Bottom-up strategy
• Parallel Assembly for subtest
Priority index: based on classical item characteristic(item difficulty and
discrimination parameters)
Item Pool Analysis
Sub pool Listening
Sub Pool Reading
Sub Pool Writing
Form 1
…
Item Pool Analysis
Sub pool Listening
Sub Pool Reading
Sub Pool Writing
Form 1
Form 2
Item Pool Analysis
Sub pool Listening
Sub Pool Reading
Sub Pool Writing
Form 1
Form 2
Form N
用CAT 帮助学习
• The idea is not new. But before technology is ready it is impossible to provide 1-to-1 teaching on a large scale.
• CAT can help! – Selecting items sequentially helps students better understand the
concepts being taught
– CAT provides more flexibility in a CAT + Learning setting
• Examples in China: – As the government increasingly invest in schools’ computers and
Internet infrastructure, some schools are using CAT during on-line teaching to offer individualized assessment and get immediate feedback to students
22
CAT can be utilized to get Cognitive Diagnostic
Information: CD-CAT
• A-matrix
• Q-matrix
ik
jkq
Item
1
2
3
4
0 1 1 0 0
0 0 1 1 1
1 0 0 0 1
0 0 0 0 1
Attributes 1 2 3 4 5
0 1 1 0 0
0 0 1 1 1
1 0 0 0 1
0 0 0 0 1
Attributes 1 2 3 4 5
Exam
inee
1
2
3
4
23
24
智慧学习: (1) 电脑教学教室;(2) 老师教学教室
In Zhang and Chang (2015)
25
COMBINING CD-CAT AND SMART LEARNING --COMPUTER-BASED CLASSROOM
26
COMBINING CD-CAT AND SMART LEARNING --TEACHER-BASED CLASSROOM
27
如何用 “INTERNET +”来帮助一线教师?
我们的目的不是取代课堂教学和一线教师
28
How to make P&P Test Adaptive ?
z
zzv
来自中国大连的实例-- 纸笔考试“自适应化”
美国国家科学基金会研究项目
CD-CAT 帮助 STEM 教育
• 解决科学、技术、工程及数学学科中退课率居高不下的状况
• 考试成绩不好导致很多学生放弃STEM课程
• 这些低分学生 (包括尚未准备充分) 不善于预测自己考试前后的学习能力,为了避免因此失败并退学,通常最终放弃STEM专业并转向其他专业。
• 我们使用 CD-CAT 来帮助低表现学生在物理课程中的表现。
• 物理211 是伊利诺伊大学第一门用到微积分的物理课
30
伊利诺伊大学香槟分校(UIUC)
• 成立于1867;公立大学中的常青藤
• 图书馆藏书量全世界大学第三
• 学校SCI论文总数全美名列前5
• 有24位教授/校友荣获诺贝尔奖
• 著名的华人校友包括:
– 姚期智,华罗庚,黄万里,邢其毅,竺可桢,李安等。
• 该校还是许多心理测量专家的母校
31
CD-CAT in STEM Fields
• Research Design in the UIUC Physics Course
Hourly Exam 1 Mid-term P&P
Hourly Exam 2 Mid-term P&P
Hourly Exam 3 Mid-term P&P
Hourly Exam 4 Final P&P
Identify students scoring
below 70%
Three CD-CATs
Three CD-CATs
Intrv 1: CD-CAT Intrv 2: CD-CAT + Worked Examples Intrv 3: CD-CAT + Interactive problems Intrv 4: CD-CAT + Human tutoring Control group
Pass
Fail
Ex1. CD-CAT in STEM Fields
• Project Stages
Collect & Analyze data
Administer high-stakes
in-class tests
Provide remedial
interventions based on CD-
CAT results
Recruit students and administer
CD-CAT
Develop a web-based
CD-CAT platform
Build item pools by analyzing
legacy items
Ex1. CD-CAT in STEM Fields
• Example Data Coding
Ex1. CD-CAT in STEM Fields
• Snapshot of CD-CAT Web-Delivery
Login page via university course website
Ex1. CD-CAT in STEM Fields
• Snapshot of CD-CAT Web-Delivery
Student’s mode of taking CD-CAT
Ex1. CD-CAT in STEM Fields
• Snapshot of CD-CAT Web-Delivery
Administrator’s mode of maintaining CD-CAT
CD-CAT 在中国大规模试验实例
39
40
In December 2011, 30,000 Grade 5 Students in Dalian China were taking a cognitive diagnostic CAT for their English proficiency assessment.
A Large Scale CAT with 2000 PC’s in Dalian, China
“圆的面积” 课例展示(北京市海淀区西颐小学六年级二班)
Utilizing CAT in Classroom Teaching, Students
are learning “Area of a Circle”
图片说明:1.集体学习系统中“圆的面积”的视频内容; 41
CAT Is Revolutionarily Changing the Way We Address
Challenges in Learning
█ Students really enjoy the new mode of testing, which makes learning more enjoyable comparing with regular teaching and P&P testing
42
Web-delivered CD-CAT Brings Equality
█Thanks to the Internet, whether schools are equipped with fancy LEDs or bulky CRTs, students receive the same quality of assessment.
43
Most Students said the CD-CAT is helpful!
greatly,
113, 58%
yes, 65,
33%
no, 17, 9%
greatly yes no
Is CD-CAT helpful to your learning?
Teachers (郑州金水实验区) :Assigning different items to each student, CAT encourages critical thinking, and makes students more independent in problem solving, and offers remedy according to their individual needs, which makes learning more interesting.
44
never, 47,
26%
once, 40,
22% 2 times,
39, 21%
3 and
above, 58,
31%
never
once
2 times
3 and above
How many times/week you use CD-CAT without teacher’s assignment
< 20
minutes,
49, 24%
20-40
minutes, 112,
55%
40-60
minutes, 27,
13%
> 60minutes,
15, 8%
< 20 minutes
20-40 minutes
40-60 minutes
> 60minutes
How many minutes each time
45
Help Teachers Know Their Students Better. According to
the diagnostic report, remedial planning is on the way
█ The in-class CAT provides more information to teachers, which facilitates research and career development
【郑州市金水区纬一路小学】的老师在实验中,借助易学通系统对习题进行钻研,通过对学生学习情况的不断分析总结,促进教师在反思中提高自身的教学技能,在提高教学质量的同时,也使自身的专业素养得到提升。
图片说明: 三位实验教师在讨论学习内容
46
总结
• 智慧学习是离不开智慧评测的
• 智慧评测的关键技术是自适应算法,应该也是“联网+ 教育”的关键技术
• 感谢“全通教育”的邀请,使我有机会与大家共享我团队的工作
“自适应评测正在实质性地影响社会的日常运转,通过影响人们如何选拔、分配
归类、和诊断决策; 自适应方法的研究将引导致更有效的评估,并因此造福社
会。”
摘自张华华的2013世界心理测量学会主席报告 联系方式: hhchang@Illinois.edu
47
top related