mcat update 2011 aamc annual meeting final
TRANSCRIPT
MCAT UpdateJudith ByrneDirector, MCAT Administration & Reporting
Jen PageDirector, MCAT Preparation Products
Scott OpplerDirector, MCAT Development & Psychometrics
November 5, 2011
MCAT UpdateMCAT Update• Test Administration 2011 Review 2012 Preview
• Resources for Examinees• Test Scores and Research 2011 Test Scores Using MCAT Data in Student Selection
• Upcoming MR5 Sessions
2
Test Administration:2011 in Review
3
We saw overall increasesWe saw overall increases
2009 2010 20112009 2010 2011
Scores Released 79,344 82,033 85,754
Unique Examinees 68,823 70,941 75,313
1 Attempt 58 768 60 297 64 8691 Attempt 58,768 60,297 64,869
2 Attempts 9,686 10,225 10,018
3 Attempts 369 419 426
4
Number of unique examinees increased by 6% in 2011 increased by 6% in 2011
78,000
80,000
75,313
74,000
76,000
78,000
68,823
70,941
70,000
72,000
66,000
68,000
62,000
64,000
2009 2010 2011
5
Number of scores increased by 5%
85,75486,000
88,000
82 033
84,000
79,344
82,033
80,000
82,000
79,344
78,000
76,0002009 2010 2011
Number of voids increased by 5% in 2011
4,8975,120
5 000
6,000
4,2184,000
5,000
2,000
3,000
0
1,000
2009 2010 2011
7
14% of examinees retested within 2011
64,869
60 000
70,000
50,000
60,000
30,000
40,000
10,01810,000
20,000
4260
1 Attempt 2 Attempts 3 Attempts
Shaking things up
• 28 administrations in 2011 (Jan – Sept)th• PLUS one additional exam on October 7th
to accommodate earthquake examinees from August 23rdfrom August 23rd
• 112 examinees tested on October 7th
• Scores for October 7th released on October 27thOctober 27
Numbers of 2011 examinees varied by race/ethnicityrace/ethnicity
34,02535000
40000
25000
30000
17,760
15000
20000
897
6,654 6,4465000
10000
8970
American Indian and Alaskan Native
Asian Black or African American
Hispanic or Latino White
10
Numbers of examinees in each group increased, some more than othersincreased, some more than others
Percent Group 2010 2011 Change
American Indian and Alaskan Native 857 897 5%
Asian 16,574 17,760 7%
Black or African American 6,185 6,654 8%
Hispanic or Latino 5,632 6,446 14%
White 32,640 34,025 4%
11
35% increase in accommodations applications approved in 2011applications approved in 2011
399400
450
292 295300
350
400
200
250
50
100
150
02009 2010 2011
12
Still a very small percentage of examinees tested with accommodationsexaminees tested with accommodations• < .5% testers with accommodations• Most commonly provided accommodations include:
Extra testing timeTime and a quarter• Time-and-a-quarter
• Time-and-a-half• Double-time
E t B k• Extra Breaks
Separate testing rooms• Specifically for disabilitySpecifically for disability • To allow other accommodations, such as food and
drink
Additional accommodations offered• Food and drink, medication, other personal medical items
• Zoom Text or other magnification• Adjustable workstations and other seating adaptations
• Extra breaks• Paper exams• Readers and scribes• Readers and scribes
FAP approvals have gone up by approximately 8% in 2011pp y
6 6967 000
8,000
4,827
6,2036,696
5,000
6,000
7,000
3,000
4,000
0
1,000
2,000
02009 2010 2011
15
Benefits to FAP Recipients in 2011• Reduced MCAT registration fee of $85 (from $235))• $500 to help defray costs of re-evaluations for those seeking accommodationsg• Free copy of The Official Guide to the MCAT ®
Exam • Free copy of MSAR ®
•14 free AMCAS applications•14 free AMCAS applications
Test-Day Survey Results
84% 84% 86%
80%
90%
100%
60%
70%
80%
40%
50% Satisfied
Dissatisfied
10%
20%
30%
3% 3% 4%0%
10%
2009 2010 2011
Test Administration:2012 Preview
18
No major changes for 2012• Planning for analogous volume increases
S&R d O t b 12th f th 2012 t ti• S&R opened October 12th for the 2012 testing season
O 4k i t t i d• Over 4k registrants on opening day• Same distribution of test dates• Base registration fee has increased by $5• No major policy changes, but have updated webNo major policy changes, but have updated web and print material to communicate better
19
New AdditionsNew roles, but not new faces:Jeremy Manus, M.A.Supervisor, MCAT Examinee Servicesp(202)862-6095
Oscar WhitemanSupervisor, Accommodated Services(202)862-6004
New to AAMC:Kathryn Bugbee, Ph.D.Director Accommodated TestingDirector, Accommodated Testing(202)862-6073
Resources for ExamineesUpdates to existing resources &Updates to existing resources & new resources planned
What do we know about student preparation? p pHow did you prepare for the examination you took today (select all that apply)?took today (select all that apply)?
• Used an MCAT preparation book 77%
• Took one or more MCAT practice tests online 69%
• Took an MCAT preparation course provided by a private company 44%company 44%
• Took one or more paper-based MCAT practice tests 25%
T k MCAT ti id d b (• Took an MCAT preparation course provided by your (or another) university 8%
• No Response 3%• No Response 3%
Preparing for the MCATResources from AAMC:• The Official Guide to the MCAT® Exam• MCAT Practice Tests• New Resources
The Official Guide to the MCAT® Exam
Updated content includes:
• Current data on MCAT scores & GPA• Updated info on changes in MCAT
scores after retestingscores after retesting• 2 new writing samples, with sample
essays y
Buy online for $30 at: www.aamc.org/officialmcatguide
Official Guide: Two SectionsNarrative: 7 chapters Practice: new questions
with detailed analysis
Part I: Narrative Overview of the MCAT Exam MCAT Fundamentals Test-Day Process and Test Day Process and
Procedures
All About MCAT Scores
Your MCAT Score as Part of the Admissions Decision
To Retake or Not Retake…THAT is the Question
Th St t M ti The Strategy Mystique: Debunking the Myths
Practice Effectively Practice Effectively
Part II: Practice
Content and Skill Content and Skill ClassificationsClassificationsClassificationsClassifications
TipsTips
Diffi lt L lDiffi lt L lDifficulty Level:Difficulty Level:Medium (45%)Medium (45%)Explanation of correct Explanation of correct
& incorrect answers& incorrect answers
The Official Guide to the MCAT® Exam
Great for getting familiar with the exam!
• 140 previously administered items, none found on practice tests
• Overview of “everything MCAT”• Overview of everything MCAT
Available online for $30 at: www.aamc.org/officialmcatguideg g
*free to FAP recipients*free to FAP recipients
MCAT Practice Tests• Updated look, same tests• Free practice test
available to allavailable to all• 7 additional tests for sale• $35 each$35 each
www.e-mcat.com
MCAT Practice Tests• Simulates actual testing
environmentwww.ewww.e--mcat.commcat.com
• Real MCAT questionsC t i bl• Customizable practice modesS l ti b t t• Solutions by test developers
• Detailed score• Detailed score reports
Detailed Score Reports
What’s next? ®The Official MCAT® Self-Assessment Package
•Coming in Spring 2012•3 Self-Assessment Tests
• Physical Sciences Bi l i l S i• Biological Sciences
• Verbal Reasoning R lt i di id l f t th•Results compare individual performance to the average performance on the same items
•Provides guidance on where to focus test•Provides guidance on where to focus test preparation efforts
•Help students use practice data to guide studyHelp students use practice data to guide study
Data to Support Individualize Prep®The Official MCAT® Self-Assessment Package
•Detailed analysis by:• Exam section • Content category within a section• Self rated confidence pre & post test• Self-rated confidence pre & post test• Items self-identified as “guessed”• Cognitive skills testedg• Item difficulty and type (discrete or passage-based)
Practice Test Data to Track Progressg•Baseline test, before preparation begins•Follow-up tests, to track improvementsFollow up tests, to track improvements
New AdditionsCreating more resources
Abby Thomsen, M.A.Abby Thomsen, M.A.Instructional Designer, MCAT Preparation [email protected](202) 828-0664 ( )
Test Scores and Research
Test Scores and Research
• Comparison of 2010 and 2011 test scores“Using MCAT Data in Medical Student• “Using MCAT Data in Medical Student Selection”
• Looking to the future: MR5 and the new MCAT• Looking to the future: MR5 and the new MCAT
T S d R hTest Scores and ResearchComparison of 2010 and 2011Comparison of 2010 and 2011 scores
How did examinees score on the MCAT in 2011?• Results in 2011 were very similar to thoseResults in 2011 were very similar to those
reported in 2010 Total scores Section scores
38
2010 vs. 2011 Total scores
7
5
6
ge
2010 (Mean=25.0)2011 (Mean=25.1)
2
3
4
Perc
enta
g
0
1
3 6 9 12 15 18 21 24 27 30 33 36 39 42 45
Total Score
39
2010 vs. 2011 MC Section Scores2010 vs. 2011 MC Section Scores
MCAT Section 2010 2011MCAT Section 2010 2011
Physical Sciences 8 3 8 4Physical Sciences 8.3 8.4
Biological Sciences 8.8 8.8g
Verbal Reasoning 7.9 7.9
2010 vs. 2011 Writing Sample Scores2010 vs. 2011 Writing Sample Scores
Percentile 2010 2011Percentile 2010 2011
25th percentile M M25 percentile M M
50th percentile O Op
75th percentile Q Q
Test Scores and Research“Using MCAT Data in MedicalUsing MCAT Data in Medical Student Selection”
Using MCAT Data in Medical Student SelectionSelection• Expanded from version of document sent to
admissions offices last yearadmissions offices last year• Purpose is to provide admissions committees
with information relevant to the use of MCATwith information relevant to the use of MCAT scores and other applicant information in the selection of this year’s medical school
liapplicants• Please take one with you (and keep it handy)!• We want to hear your feedback…
Questions addressed:Questions addressed:• What does the MCAT exam measure?• How is the MCAT exam scored?• How well do examinees score on the MCAT
?exam?• How accurate are examinees’ MCAT scores
and how should they be interpreted?and how should they be interpreted?• Why are some MCAT scores marked as non-
standard?standard?
44
Questions addressed (cont’d):Questions addressed (cont d):• How do examinees’ scores change when they
retake the MCAT exam and how do admissionsretake the MCAT exam and how do admissions officers use scores for applicants who test more than once? [EXPANDED][ ]
• How do admissions officers use MCAT scores and other application data in the holistic review
f li t ’ lifi ti ? [EXPANDED]of applicants’ qualifications? [EXPANDED]• How well do UGPAs and MCAT scores predict
students’ performance in medical school?students performance in medical school? [EXPANDED]
45
How do examinees’ scores change when they retake the MCAT exam?• MCAT examinees can test up to three times inMCAT examinees can test up to three times in
one calendar year, with no lifetime limit• In 2010, approximately 47% of applicants , pp y pp
reported scores from multiple test administrations in their applicationsO th l th i iti l MCAT• On average, the lower the initial MCAT score, the greater the amount of score improvement
• However the degree (and direction) of score• However, the degree (and direction) of score change varies considerably across individuals
46
Changes in MCAT Total Scores for First-time Examinees Retesting in Same Year (2008-2010)
47
How well do UGPAs and MCAT scores predict students’ scores predict students performance in medical school?
I thi ’ d t h id d d d• In this year’s document, we have provided an expanded set of “grids” depicting the relationships between UGPA, MCAT Total scores, and a variety of medical education
t i bl i l dioutcome variables, including: 4- and 5-year graduation rates
Withd i b i di i d f d i Withdrawing or being dismissed for academic reasons
First attempt and eventual pass rates for Step 1,First attempt and eventual pass rates for Step 1, Step 2-CK, and Step 2-CS exams (see Appendix)
48
Percentage of 2003-2005 Students who Graduated from Medical School in 4 Years, by MCAT and UGPA Ranges
6
7
4
5Series 1Series 2
3
4 Series 2Series 3Series 4Series 5
1
2Series 5Series 6
0Category 1 Category 2 Category 3 Category 4
Percentage of 2003-2005 Students who Graduated from Medical School in 5 Years, by MCAT and UGPA Ranges
6
7
4
5Series 1Series 2
3
4 Series 2Series 3Series 4Series 5
1
2Series 5Series 6
0Category 1 Category 2 Category 3 Category 4
Percentage of 2003-2005 Students Passing Step 1 Exam on the First Attempt, by MCAT and UGPA Ranges
6
7
4
5Series 1Series 2
3
4 Series 2Series 3Series 4Series 5
1
2Series 5Series 6
0Category 1 Category 2 Category 3 Category 4
Percentage of 2003-2005 Students Eventually Passing Step 1 Exam, by MCAT and UGPA Ranges
6
7
4
5Series 1Series 2
3
4 Series 2Series 3Series 4Series 5
1
2Series 5Series 6
0Category 1 Category 2 Category 3 Category 4
Test Scores and ResearchLooking to the Future: MR5 and theLooking to the Future: MR5 and the New MCAT
The 5th Comprehensive Review of the MCAT• After 3 years of dedicated effort, the MR5After 3 years of dedicated effort, the MR5
committee has made its final recommendations regarding changes to the MCAT exam
• Among these recommendations are: The addition of a new section that will include
content from psychology, sociology, and biology The addition of content typically taught in first
semester introductory biochemistry coursessemester introductory biochemistry courses The elimination of the Writing Sample section
54
The 5th Comprehensive Review of the MCAT• The committee has recommended that the newThe committee has recommended that the new
MCAT be introduced in the Spring of 2015
• To support the development of new testTo support the development of new test content, the AAMC plans to begin administering an optional un-scored section at the end of the current exam beginning in January 2013current exam, beginning in January 2013
• To prevent an increase in the length of the testing day the decision has been made to stoptesting day, the decision has been made to stop administering the Writing Sample at that time
55
The 5th Comprehensive Review of the MCAT• For more information about theFor more information about the
recommendations of the MR5 committee, please join us at the following sessions: MR5 Update (Today,12:00 – 1:00 p.m., Convention
Center, Rooms 406/407)AAMC Focus Session: “Rethinking Admissions: AAMC Focus Session: “Rethinking Admissions: Gateway to the Profession of Medicine” (Tuesday, 8:00 – 9:30 a.m., Convention Center, Rooms 205/207)205/207)
56
And last but not least…..
• Please join us in welcoming the most recent addition to the MCAT team:addition to the MCAT team: Dr. Cindy Searcy, Director of MCAT
Research Phone number: (202) 862-6105
57
Questions?
58
©