techniques for data-driven curriculum analysis
DESCRIPTION
Five techniques to understand the data that could help to re-design CurriculumTRANSCRIPT
Techniques for Data-Driven Curriculum Analysis
Gonzalo Mendez, Xavier Ochoa & Katherine Chiluiza
Siemens, George, and Phil Long. "Penetrating the fog: Analytics in learning and education." Educause Review 46.5 (2011): 30-32.
Siemens, George, and Phil Long. "Penetrating the fog: Analytics in learning and education." Educause Review 46.5 (2011): 30-32.
Which are the hardest/more difficult courses?
What lead our students to success/failure?
How courses are related?
Are there courses that could be eliminated?
Is the work-load adequate for our students? ??
How can Learning Analytics help?
Which tools could it provide to curriculum-designers?
Our goals
Use readily available data
Grades are always collected and historically stored
Create discussion starters
Metrics for evaluation are evil, butmetrics for insight could be useful
Easy to apply and understand
Could be integrated into a Learning Analytics toolbox
Eat your own dog-food
Apply them to our own data to obtain insight
(12-year historical data on CS program)
Let’s start
(1) Difficulty Estimation
How difficult a course is, not how good the students are
Technique
Difficulty metrics
Two estimation metrics
GPA - Course grade
Course grade > GPA
Course grade < GPA
0
Course grade = GPA
Three scenarios:
Differences betweenGPA and course grade
> 0< 0
Real examples
But…
They are not normal!
Three Two estimation metrics
Difficult Classes (Top 10)
Perceived
Estimated (first 5)Algorithms Analysis
Operating Systems
Physics A
Differential Equations
Linear Algebra
Programming Fundamentals
Object-Oriented Programming
Differential Calculus
Data Structures
Statistics
Operating Systems
Statistics
Differential Equations
Linear Algebra
Programming Languages
Electrical Networks I
Artificial Intelligence
Programming Fundamentals
Data Structures
Hardware Architecture and Organization
Perception != Estimation
What makes a course difficult then?
(2) Dependance Estimation
How well I do a student does in a course affects how well he/she does
in another
CORE - CS CURRICULUMBasic Physics
Integral Calculus
Multivariate Calculus
Electrical Networks
Digital Systems I
Hardware Architectures
Operative Systems
General Chemistry
ProgrammingFundamentals
Object-orientedProgramming
Data Structures
ProgrammingLanguages
Database Systems I
Software Engineering I
Software Engineering II
Oral and WrittenCommunication Techniques
Computing and Society
Discrete Mathematics
Algorithms Analysis
Human-computerInteraction
Differential Calculus
Linear Algebra
Differential Equations
Ecology andEnvironmental Education
Statistics
Economic Engineering I
Artificial Intelligence
PROFESSIONAL TRAINING HUMANITIES BASIC SCIENCE
Technique
Pearson product-moment correlation coefficient
(A lot of it)
DEPENDANCE ESTIMATIONProgrammingFundamentals
Data Structures(0.321)
Object Oriented Programming
(0.309)
DEPENDANCE ESTIMATION
Computingand Society
Operating Systems(0.582)
Discrete Mathematics(0.614)
Human-Computer Interaction(0.6226)
Maybe we should rethink our prerequisites
Why Programming Fundamentals does not correlates?Why Computers and Society correlates with a lot of
courses?
(3) Curriculum Coherence
How courses group together
CORE - CS CURRICULUMBasic Physics
Integral Calculus
Multivariate Calculus
Electrical Networks
Digital Systems I
Hardware Architectures
Operative Systems
General Chemistry
ProgrammingFundamentals
Object-orientedProgramming
Data Structures
ProgrammingLanguages
Database Systems I
Software Engineering I
Software Engineering II
Oral and WrittenCommunication
Techniques
Computing and Society
Discrete Mathematics
Algorithms Analysis
Human-computerInteraction
Differential Calculus
Linear Algebra
Differential Equations
Ecology andEnvironmental Education
Statistics
Economic Engineering I
Artificial Intelligence
PROFESSIONAL TRAINING HUMANITIES BASIC SCIENCE
Technique
Exploratory Factor Analysis
(EFA)
31
UNDERLYING STRUCTURE
Electrical Networks
Differential Equations
Software Engineering II
Software Engineering I
HCI
Oral and Written
Communication
Techniques
General Chemistry
Programming Languages
Object-Oriented Programming
Data Structures
Artificial Intelligence
Operative Systems
Software Engineering
Object-Oriented Programming
Economic Engineering
Hardware Architectures
Database Systems
Digital Systems I
HCI
Differential and Integral CalculusLinear Algebra
Multivariate CalculusDigital Systems I
Basic PhysicsProgramming Fundamentals
Discrete MathematicsGeneral Chemistry
StatisticsData Structures
Computing and SocietyAlgorithms Analysis
Differential EquationsEcology and Environmental Education
Object-Oriented Programming
FACTOR 1: The basic training factor
FACTOR 2: The advanced CS topics factor
FACTOR 3: The client interaction factor
FACTOR 4: The programming
factor
FACTOR 5: The ? factor
Grouping is also off
Fundamental Programming is not in the Programming factor?What to do with Electrical Networks and Differential Equations?
(4) Drop-out Paths
What courses lead the students to drop-out
DROPOUT AND ENROLLING PATHSTime
(semesters)
0
1
2
3
4
Dropout
They are all happy, but as time goes by…
Technique
Sequence Mining (Sequential PAttern Discovery using
Equivalence classes - SPADE)
DROPOUT PATHS
Sequence Support<Physics A, Dropout> 0.6081967
21
<Differential Calculus , Dropout> 0.570491803
<Programming Fundamentals , Dropout> 0.532786885
<Integral Calculus , Dropout> 0.496721311
<Physics A, Differential Calculus , Dropout> 0.43442623
<Linear Algebra , Dropout> 0.432786885
<Differential Calculus, Integral Calculus , Dropout>
0.385245902
<Physics C , Dropout> 0.347540984
<Physics A, Integral Calculus , Dropout> 0.327868852
<General Chemistry , Dropout> 0.319672131
<Differential Equations , Dropout> 0.31147541
Most drop-outs fail basic courses
Should students start with CS topics?Too much pressure in engineering courses?
(5) Load/Performance Graph
What students think they can manage vs. what they can actually manage
Technique
Simple Visualisation:Density Plot of
Difficulty taken vs. Difficulty approved
LOAD/PERFORMANCE GRAPH
LOAD/PERFORMANCE GRAPH
LOAD/PERFORMANCE GRAPH
Unrealistic Suggested Load
How to the present the Curriculum in a better way?How we can recommend students the right load?
Our goals?
Which are the hardest/more difficult courses?
What lead our students to success/failure?
How courses are related?
Are there courses that could be eliminated?
Is the work-load adequate for our students? ??
??What makes a course difficult then?
Why Programming Fundamentals does not correlates?
Why Computers and Society correlates with a lot of courses?Fundamental Programming is not in the Programming
factor?
Should students start with CS topics?Too much pressure in engineering
courses?How to the present the Curriculum in a better way?How we can recommend students the right
load?
What to do with Electrical Networks and Differential Equations?
Our ambitious goal?
Apply these techniques at your own data in your own institution
Our more ambitious goal?
Make you think about LA techniques that can be easily transferred to
practitioners
