aspiring minds | outcomes using test scores
TRANSCRIPT
Aspiring Minds
www.aspiringminds.com
Outcome predictions using test scores
Who can learn programming through a short 3-month training?
Varun Aggarwal
Presented at Education Workshop at ICML, 2015
Case of a large IT Services company
• Companies wants to hire candidates who when trained for 3-months will learn programming to be deployed as software engineers.
• 1371 candidates tested on AMCAT at the time of joining a large IT company. Modules tested:• English Comprehension• Quantitative Ability• Logical Reasoning• Computer Programming
• Company shares who succeeded and who did not. Candidates classified as High Performers, Average Performers and Mid-Performers.
The Model Type• Should be theoretically plausible
• Models need to be interpretable
• Models need to be simple
• Trade-off models between type 1 and type 2 error required
Class 1
Class 0
Not allowed
L > 400E + 0.4*L > 870E + 0.7*Q > 980
English
Logi
cal
Designed a genetic algorithm based system to search the model-space
HP 76.66
AP 15.36
PP 7.98
Test Shortlist rate40%
AP Rejected 26.88%
PP Rejected32.88%
HP 82.49
AP 11.24
PP 6.27
Test Shortlist rate29.31 %
AP Rejected 39.17%
PP Rejected47.26%
HP 86.09
AP 9.1
PP 4.8
Test Shortlist rate21.72%
AP Rejected 50.75%
PP Rejected59.62%
Test shortlist Rate: 31%
Poor Performers11.89%
Average Performer18.38%
High Performers69.73%
CURRENT ORGANIZATION
We can help the organization…
15 20 25 30 35 40 45 500
10
20
30
40
50
60
ELELC
Shortlist Rate
% lo
w p
erfo
rmer
s rem
oved
COUNTER-INTUITIVE TO BUSINESS PEOPLE!!!
CP model gets you lower type 1 and higher SR
Knowledge Discovery: Exposure to programming reduces need of logical ability
1• Computer > 360
2• 0.47*English + 0.98*Computer + Logical > 1196
• The first filter eliminates bottom 28% candidates in Programming. •Above a minimum benchmark, lack of Programming knowledge can be offset by high Logical Ability and English skills.
1• 0.42* English + Logical > 722
2• Logical > 464
OR
• Logical ability needs to above the mean.• Above a minimum benchmark, lack of logical Ability can be offset by high English comprehension skills