equitable assessment of the practical component of scientific modules
DESCRIPTION
TRANSCRIPT
Equitable assessment of the practical component
of Scientific Modules
Darvree Downey and Maeve ScottScience Department
Institute of Technology Tallaght
1
2
Students value laboratory
practicals as much for their role in
illuminating and reinforcing
theory as for the opportunity to
learn the ‘tools of the trade’
(Willmott, 2005).
• Background• Objectives
Study
• Descriptions• Delivery of practical element
Module
• Statistical methodsMethod
Results
Conclusions
Contents
3
Laboratory Assessment ~ 35% of module
marks
Laboratory assessment aims to assess
achievement of the practical-component
learning outcomes.
Changes have been introduced in laboratory
assessment practices in ITT in recent years.
Educational Situation
4
Since the last programmatic review (in 2006) there
is no failed element in any of the laboratory
subjects in ITT’S Science Department.
Anecdotal belief that students are getting sufficient
marks to pass without achieving core learning
outcomes
Educational Situation
5
How does laboratory assessment equate with
students achievements in other summative
assessments within the same module?
Is correlation with other forms of assessment
indicative that the laboratory assessment is
equitable?
Study Background
6
Practical Component Learning Outcomes can include:
Interpretation of legal requirements
or guidelines
Use of analytical equipment
Application of calculations and
statistics
Analysis, interpretation and
application of published resources
Critical evaluation of results Organisational skills
Time management Meeting industry needs
7
The practical component of laboratory
exams are designed to assess laboratory
technique and equipment usage skills.
Learning outcomes common to both theory
(final module exam) and laboratory exams
include organisational and time
management skills and data analysis skills.
8
Learning Outcomes
To determine if current laboratory
assessment methods used for a range of
scientific modules in ITT Dublin are
1. Reliable indicators of a student’s ability to
meet the overall module learning
outcomes.
2. Equitable for all learning styles
Objectives
9
Students have diverse learning styles so
assessment methods should vary for an
equitable approach.
Summative assessment methods should not
disadvantage any group of students.
Inclusive and Diverse Assessment
10
The practical element of a course can be
assessed a number of ways to determine
the same learning outcomes. All
assessment methods have benefits and
limitations.
Assessment methods
11
Suits students with good language and planning skills
Can disadvantage some learners, in particular
Students with Dyslexia
Students for whom English is a second language
Auditory learners
Students with organisational difficulties
Too much time spent writing reports at the expense
of getting to grips with the subject matter (Race,
2006)
Laboratory Reports
12
1. Originality?
◦ Anti-plagiarism software
2. High marks not necessarily reflective of
laboratory competence
◦ Revision of marking schemes
◦ Introduction of practical lab exams
◦ In-class questions & observation
Laboratory reports
13
Helping the students to think about the
quality of their own work (Rust, 2005)
Generation of internal feedback
Self Assessment of Laboratory Reports
14
Formal oral laboratory assessment
exams can disadvantage students with
poor auditory and language skills
Challenging for students who suffer
from anxiety
Preferred by auditory learners and
students who have poor written skills.
Oral Assessment
15
Questioning of students during laboratory
sessions can assess the preparation,
depth of learning and organisational
skills.
difficult to assess through other methods?
Oral Questions in Laboratory Practicals
16
◦ Students with poor language skills can have
difficulties with providing articulate responses
◦ Students with auditory problems can have
comprehension difficulties.
◦ Questions during laboratory work can add
additional stress to students with poor
organisational skills such as those with specific
learning difficulties
Oral Questions in Laboratory Practicals: Disadvantages
17
A chance to assess students’ laboratory technique and data
analysis skills
Should suit sensing learners
Reflective learners can find practical skills assessment
stressful
Dyslexic students and students with other learning difficulties
may not be given extra time/special consideration
18
Laboratory Exams
Method
19
4 modules:
◦ Systems Validation (Pharmaceutical Science, Semester 7)
◦ Scientific Analysis (Pharmaceutical Science, Semester 4)
◦ Biochemistry (Bioanalysis & DNA&Forensic Science, Semester 4)
◦ Principles of Biochemistry (Pharmaceutical Science, Semester 3)
Final exam as marker of module competence
Lab assessments methods as predictors
Statistical analysis: Regression
Method
20
Analysis
21
Laboratory Report Marks as a predictor of Final Exam Marks: Systems Validation Semester 5
22
23
Comparison of Laboratory Report Marks and Laboratory Exam Marks as a predictor of Final Exam Marks: Principles of Biochemistry, Semester 3
Suggests need for failed element?
24
Laboratory Report Marks as a predictor of Final Exam Marks: Biochemistry, Semester 4
25
Laboratory Exam Marks as a predictor of Final Exam Marks: Biochemistry, Semester 4
26
Why did the lab assessments in
2011 correlate with the theory
exam to a greater extent than in
previous years?
More in-class time spent on laboratory preparation & feedback? YES
Academically more capable students? NO
Suggestion of a ‘better teaching’ effect?
Expectations vs. Actual
27
28
Laboratory assessment and module exams measure different learning outcomes but there is crossover
Very high correlation not expected
Where there is very little correlation, it does suggest a disconnect
Intuitive learners struggling with practical skills?
Sensing learners struggling to relate the theory to the practical?
Are global learners losing out in the laboratory?
29
RecommendationsFor ITT
Re-instatement of failed element where applicable to
ensure theory and practical learning outcomes are met
Use of this type of analysis in all laboratory-based modules
to highlight inconsistencies and measure the effects of
year-on-year assessment changes
General
Use the wealth of data available to highlight potential areas
for improvement of summative assessment practices
Measure how learning style affects summative assessment outcomes for laboratory-based modules
Monitoring effects of new laboratory interventions on summative assessment outcomes
What next?: Future analysis
30
Broaden the range of lab assessment
methods…..
VLEs; Mindmaps; Reflective journals;
Posters,
Broaden use of questions in labs & oral
assessments
What next?: Changes to Summative Assessment Practices
31
32
Acknowledgement:Many thanks to James Reilly, ITT Dublin for his help and advice on statistical methodologies