de assessment results1014 - university of...

1

DE Course Assessment Strategies and Results – Spring 2013 Courses (updated October 14, 2013) Listed in alphabetical order based on discipline These are summaries of individual course assessment reports, and in some cases distillation of departmental analyses. A few sample departmental repots are linked; hard copies are available upon request. Course Instructor Assessment

Strategy Findings Responses

Implemented Comparison assessment

ACC 201 Mei Wang Spring 2013 Assessment Embedded Assessment. Mapping Final Exam questions to SLOs. Will cycle assessment of specific SLOs over semesters.

Spring 2013 results: For some SLOs, desired percentage of students achieving standard score level set was not met.

Spring 2013 responses For SLOs where less than 70% achieved desired scores, assignments reviewed and adjusted.

N/A Only taught online

ACC 202 Mei Wang Spring 2013 Assessment Embedded Assessment. Mapping of Final Exam questions to SLOs. Will cycle assessment of specific SLOs over semesters.

Spring 2013 results: For some SLOs, desired percentage of students achieving standard score level set was not met..

Spring 2013 responses For SLOs where less than 70% achieved desired scores, assignments will be adjusted.


AEC 118

Doug Madden

Past Semesters/Fall 2012 Assessment Knowledge Survey results analyzed. Grade Analysis. Analysis of discussion postings.

Past Semesters/Fall 2012 Results Grade analysis indicated actions needed to improve student performance. Analysis of discussion postings indicated changes needed based on results.

Past Semesters/ Fall 2012 Responses Revised structure of course and due dates. Modified course to focus on improving rate of student retention and completion


Spring 2013 Assessment Four SLOs evaluated using: *20 post test items embedded in final exam (same topics/ different phrasing from pretest items); *embedded final exam questions related to definitions/con

Spring 2013 results: 1st SLO evaluated, students did not achieve desired performance level; 2nd SLO – while not meeting desired goal, showed marked improvement from past semesters reflects impact of changes implemented. For final two SLOs, desired performance level

Spring 2013 responses Continued revising pre and post exam questions. Instituted mandatory initial individual meetings with instructor. Addition of more YouTube videos.

2

cepts; *scoring on rubric of construction project and sketching projects; *scoring on rubric grading of weekly discussion performance.

achieved though improvements still possible.

Revised exam format. New student tutoring program. More feedback on exams. More individual contact with students who do not regularly participate in class discussions, etc.. Clarifying time frame regarding posting deadlines.

Am St 202

Ralph Kam Spring 2013 Assessment Embedded questions mapped to course work and final exam

Spring 2013 results Insufficient data from this class

Spring 2013 responses Insufficient data from this class

Instructors teaching Am St working on revising SLOs and creation of aligned assessment strategy

Anth 135 Rob Edmondson

Spring 2013 Assessment Comparison of Spring 2013 & Fall 2012 student work on research reports

Spring 2013 results Student results indicate areas where improvements can be made in achieving desired performance levels; not all students submitted revised, improved essays

Spring 2013 responses Some changes to be implemented include: clarification of Laulima directions, better example given for Report #1; earlier notification of assignment expectations; weekly communication with DE students to provide help with Report #1; more instructor feedback on Report # 1 to encourage editing and corrections.



Spring 2013 Assessment Embedded questions on first exam mapped to SLOs.

Spring 2013 results Students did not meet desired performance level on all exam questions.

Spring 2013 responses Some changes to be implemented include: clarification

FtF version offered for the first time in Fall 2013. Will have embedded assessment

3

Embedded assessment of student work on field research report, mapped to SLOs

Related to research reports, some areas where further improvement was needed.

of instructions on Laulima, better feedback on weekly assignments leading to Test #1; clarification of instructions on reports; earlier instructions on report assignment; weekly communication to students to provide help with Report #1; more feedback given on each report to encourage editing and corrections.

and align embedded questions for FtF and DE sections.


Spring 2013 Assessment Embedded assessment of student work on field research report, mapped to SLOs

Spring 2013 results Due to low enrollment during spring semester, data on ANTH 200 classes compiled from additional semesters. In comparison to FtF students, cable students did not meet desired performance level of work on field research report as compared to FtF students, in large part due to lack of follow through in improving drafts.

Spring 2013 responses Limited ability to modify since taped cable class but will implement clarification of written directions via Laulima; students given sample Report #2; instructor will make weekly calls &/or emails to provide help with Report #2.

Embedded assessment with aligned questions used for FtF and DE sections.

Art 101 Moana (Marty) Nikou

Spring 2013 Assessment Embedded Assessment: Questions on Final Exam mapped to SLOs. Students required to: 1) analyze specific work of art and 2)

Spring 2013 results: FtF: Acceptable percentage of students achieving desired performance on mapped questions, though improvement can be made.

Spring 2013 responses Reevaluating exam design to assess merit of experimental emphasis in DE course of more explicit links between

Use of common embedded question(s) on the final exams of FtF and DE courses mapped to specific SLOs.

4

analyze historical context of work and thematic emphasis.

DE: Students achieved desired results for both SLOs assessed and did better overall than students in the FtF class. Improved student performance linked to inclusion of new module quizzes in DE section, focused more heavily on visual analyses of subjects.

material coverage and exam.

Students in DE course did better in performance measures -attributed to course design of DE course with more explicit links to exam.

Astr 110 Shidong Kan, Kerry Tanimoto, Paul Sherard

Spring 2013 Assessment Mapping questions from homework assignments, chapter exams and comprehensive final exam to SLOs

Spring 2013 results Students in general achieved desired performance levels – both in DE and FtF classes. Students did not achieve desired performance on two questions assessed; questions are intended to be more difficult. (see report)

Spring 2013 responses No significant changes identified; more assessment planned to refine understanding.

Identical embedded questions on final exams used to compare student performance in DE and face-to-face courses. Spring 2013 results Analysis of scores indicates no significant difference between DE and FtF performance; changes noted are due to different testing conditions.

Chem 105

Michael Ferguson

Fall 2013 (not taught Spring 2013) Use of embedded questions on exam mapped to SLO on scientific method

Awaiting results from Fall 2013

Will use same embedded exam questions for DE and FtF sections and compare cumulative student results.

CMGT 100

Norman Takeya

Spring 2013 Assessment Analysis of Knowledge Survey results. Embedded Assessment: specific questions on exams/quizzes mapped to SLOs. Numerical

Spring 2013 results: Most measures indicate adequate student achievement of SLOs. In few areas, need improvement achieve desired statistical measures.

Spring 2013 responses Clarification of SLO. Modifying quiz questions for greater clarity

Only offered online. When FtF version offered in future, will compare student results on common embedded questions in DE and FtF classes mapped to

5

analysis of grades earned.

SLOs.

CMGT 220

Norman Takeya

Spring 2013 Assessment Analysis of Knowledge Survey results. Embedded Assessment - SLOs, linked to specific questions on exams and quizzes. Numerical analysis of grades earned carried out.

Spring 2013 results: (only KS analysis carried out this semester; embedded analysis will occur in future semesters): Most measures indicate adequate student achievement of SLOs. In a few areas, need improvement to achieve desired level of student confidence.

Spring 2013 responses Review of curriculum materials. Will include embedded assessment for enhanced understanding of student learning.


Econ 130 Vicky Chiu-Iron

Spring 2013 Assessment Embedded Assessment: Final Exam questions mapped to SLOs.

Spring 2013 results Students met desired performance levels for some SLOs, did not for other SLOs.

Spring 2013 responses Will assign more exercises emphasizing use of graphs and applying in real-world economic situations

Assessment strategies aligned for courses taught by instructor to enable comparison. Different instructor teaches FtF version so working on strategy for comparing student performance in FtF versus DE, with different instructors.


Embedded Assessment: Final Exam questions directly linked to SLOs.

Not offered Spring 2013.

Different instructor teaches FtF version. Need to develop strategy for comparing student performance in FtF versus DE with different instructors.


Embedded Assessment: Final Exam questions directly linked to SLOs.

Not offered Spring 2013; awaiting Fall 2013 results

Different instructor teaches FtF version. Need to develop strategy for comparing student performance in FtF versus DE with different

6

instructors. ED 105 Lisa Yogi,

Linda Buck

Fall 2012 Assessment Embedded Assessment: SLOs mapped to course assignments: Child Observation Report, Family Resource file, Portfolio. Plan to expand to include embedded question on midterm exam.

Fall 2012 results Student responses met desired statistical outcomes.

Fall 2012 responses Continue improving by including short answer question in the Midterm to address three SLOs.

Taught online and FtF – by different instructors; department developing assessment strategies to ensure alignment of assessment and evaluation of results.

Spring 2013 Assessment Embedded Assessment: SLOs mapped to course assignments: Child Observation Report, Family Resource file, Portfolio.

Spring 2013 results Students achieved desired measures in all but one SLO.

Spring 2013 responses Revision of curriculum with more focus on SLO 4

ED 110 Lisa Yogi, Gaynel Buxton

Fall 2012 assessment Embedded Assessment: SLOs mapped to midterm / final exam questions, or focused class projects.

Fall 2012 results Student responses met desired statistical outcomes.

Fall 2012 responses No significant changes required.

Taught online and FtF – by different instructors; department developing assessment strategies to ensure alignment of assessment and evaluation of results.

Spring 2013 Assessment Embedded Assessment: SLOs mapped to midterm / final exam questions, or focused class projects. Numerical targets established for student performance (percentage achieving acceptable percentage of 70 or 80%)

Spring 2013 results Student responses met most desired statistical outcomes; there was one area where student performance could be improved

Spring 2013 responses Greater emphasis on student understanding technical requirements of the class; consider allowing students to revise first assignment/ or extended time.

7

ED 131 Miles Nakanishi, Gaynel Buxton, Linda Buck

Fall 2012 Assessment Embedded Assessment: * SLOs mapped to class journal and observation assignments as well as milestone project; rubric break down of learning outcomes enables mapping to SLOs. *For the future - embedded questions in final exam mapped to some SLOs. Numerical targets established.

Fall 2012 results Most SLOs assessed and responses indicated students met desired performance measures.

Fall 2012 responses No changes needed

Taught online and FtF – by different instructors; department developing assessment strategies to ensure alignment of assessment and evaluation of results. Spring 2013 Numerical comparison carried out between DE and FtF. All scores exceeded desired performance SLOs were met. Results were comparable between sections. DE course compares favorably with FtF course.

Spring 2013 Assessment Three SLOs evaluated via KS results

Spring 2013 Results Students met desired performance goals set for all SLOs

Spring 2013 Responses No changes needed; will expand assessment to include embedded assessment in the future.

ED 170 Gaynel Buxton

Fall 2013 (not taught DE Spring 2013) Embedded Assessment: SLOs mapped to Family Interview assignment; class assignments; quizzes and final exams, and final project


Assessment methods the same between DE and FtF to enable comparison between DE and FtF beginning with Fall 2013 results.

ED 274 Gaynel Buxton

Fall 2013 (not taught DE Spring 2013) Embedded Assessment: required course papers; targeted course assignments


Assessment methods the same between DE and FtF to enable comparison between DE and FtF.

8

Eng 22 Chris McKinney

Fall 2012 Assessment Embedded Assessment: My Writing Lab quiz questions mapped to specific SLOs. For some SLOs, linked to specific writing assignments and rubric measures. All SLOs evaluated.

Fall 2012 results Only a few SLOs where acceptable numbers not reached.

Fall 2012 responses Revisions to improve student performance include: more due date notifications sent to students; revision of assignments; revision of grading rubric; additional outside readings.

Fall 2012 Instructor carried out comparisons of DE and FtF student performance for all SLOs using same measures. Major area of difference in retention and completion, with DE numbers lower. Full report includes analysis and responses.

Spring 2013 Assessment Embedded Assessment: My Writing Lab quiz questions mapped to specific SLOs. For some SLOs, linked to specific writing assignments. All SLOs evaluated.

Spring 2013 results: Students achievement of SLOs dropped in terms of percentages – possibly due to greater attrition during Spring semester; also only one class and low number of students so not representative.

Spring 2013 responses Continue to revise and fine-tune assessment strategies and monitor effects of changes implemented this year.

Spring 2013 Conclusion: Appeared to be changes between Spring and Fall semester students overall. More productive to compare Fall semesters and Spring semesters to each other after implementations of changes. Will gather data for Fall 2013 and compare it to Fall 2012. Will then do the same for Spring 2013 and Spring 2014.

Eng 100 Chris McKinney

Fall 2012 Assessment Embedded Assessment: My Writing Lab quiz questions mapped to specific SLOs. For some SLOs, linked to specific writing assignments. All SLOs evaluated.

Fall 2012 results Most SLO measures met; in a few areas, need for improvements noted.

Fall 2012 responses Revisions to improve student performance include: review and revision of submission policies; clarification in assignment expectations.

Instructor only teaches online. Department working on strategy to comparison between DE and FtF versions, since multiple instructors teach this course.

9

Spring 2013 Assessment Embedded Assessment: My Writing Lab quiz questions mapped to specific SLOs. For some SLOs, linked to specific writing assignments. All SLOs evaluated.

Spring 2013 results: Students achievement of SLOs dropped in terms of percentages – possibly due to greater attrition effects of Spring semester – also only one class and low number of students so not fully representative.

Spring 2013 responses Will revise and fine-tune assessment strategies and continue to monitor effects of changes being implemented this year.

Eng 100 Brenda Kwan

Fall 2012 Assessment Embedded Assessment: SLOs mapped to several class assignments including MyWriting Lab, components of required writing assignments and online assignments.

Fall 2012 results Students largely met desired performance levels; a few areas where changes needed.

Fall 2012 responses Revisions to improve student performance include: additional targeted assignments; additional announcements and explanation messages; more rigorous policy regarding assignment completion dates.

Instructor only teaches online. Department working on strategy to comparison between DE and FtF versions, since multiple instructors teach this course.

Spring 2013 Assessment Embedded Assessment: SLOs mapped to several class assignments including MyWriting Lab, components of required writing assignments and online assignments.

Spring 2013 results: Desired level of student performance met. Instructor contact and quick response time via email, chat and phone calls proving very effective.

Spring 2013 responses Students felt frequent contact instrumental to success so will continue. Students unfamiliar with online format needed more help. Early semester check-in/conference will help identify students needing more assistance.

FSHN 185

Karen Hastings

Spring 2013 Assessment Analyzed SLO #3 using pre test and post test to measure student confidence. Analyzed success in learning via embedded question on

Spring 2013 results: Cable Students demonstrated dramatic increase in confidence. Success in student learning also reflected in performance on embedded exam question.

Spring 2013 responses Student performance acceptable in both DE and FtF classes, Will work to further improve by adding in-class exercises to reinforce

Same embedded questions used to measure FtF and De students. Comparisons indicate students achieving desired learning of

10

final exam. Compared performance of cable students and FtF students.

FtF Students: demonstrated dramatic increase in confidence. Success also in student learning reflected in performance on embedded question on exam. Room for improvement in FtF class compared to DE section

SLO #3 in FtF class.

SLO in both FtF and DE but will implement changes to improve FtF class.

FIRE 100

Stacy Rogers

Spring 2013 Assessment Questions on exams and quizzes mapped directly to course SLOs; demonstration of skills linked to course and program SLOs. SLOs derived from National Fire Protection Associations (NFPA) professional firefighter standards and National Professional Qualifications Board. Department coordinating assessment and analysis of results

Spring 2013 results Awaiting October departmental meeting and report

Spring 2013 responses Awaiting October departmental meeting and report

Departmental coordination of assessment enables comparison of student performance across classes and instructors

FIRE 102

Stacy Rogers

Spring 2013 Assessment Questions on exams and quizzes mapped directly to course SLOs; demonstration of skills linked to course and program SLOs. SLOs derived from National Fire Protection Associations (NFPA) professional




11

firefighter standards and National Professional Qualifications Board. Department coordinating assessment and analysis of results

FIRE 107

Stacy Rogers

Questions on exams and quizzes mapped directly to course SLOs; demonstration of skills linked to course and program SLOs. SLOs derived from National Fire Protection Associations (NFPA) professional firefighter standards and National Professional Qualifications Board. Department coordinating assessment and analysis of results




FIRE 202

Scotty Rhode

Questions on exams and quizzes mapped directly to course SLOs; demonstration of skills linked to course and program SLOs. SLOs derived from National Fire Protection Associations (NFPA) professional firefighter standards and National Professional Qualifications Board. Department coordinating assessment and analysis of results




12

FIRE 111

Stacy Rogers


Spring 2013 results: Awaiting October departmental meeting and report



FIRE 206

Jeremy Rhode





FIRE 280A

Scotty Rhode

Spring 2013 Assessment Questions on exams and quizzes mapped directly to course SLOs;


Spring 2013 response Awaiting October departmental meeting and report

Departmental coordination of assessment enables comparison of student performance across classes

13

demonstration of skills linked to course and program SLOs. SLOs derived from National Fire Protection Associations (NFPA) professional firefighter standards and National Professional Qualifications Board. Department coordinating assessment and analysis of results

and instructors

Geog 101

John Delay Fall 21012 Assessment Five SLOs assessed through five separate embedded questions on exam.

Fall 2012Results: For all SLOs measured, desired threshold for student performance met or exceeded.

Fall 2012 responses No changes needed

Same assessment methods carried out in DE and FtF sections. Fall 2012 For four SLOs measured, DE students did same or better than FtF courses. For fifth measure, results very close and exceeded threshold.

Spring 2013 Assessment Five SLOs assessed through five separate embedded questions on exam.

Spring 2013 results: For four SLOs, student performance results exceeded threshold, though in some cases bit lower than previous semester. For one SLO –significant drop in student performance and desired threshold not met.

Spring 2013 responses: Will review exams for needed rewording of questions, clarification of study guide, and/or need for targeted supplemental materials.

Spring 2013 Higher correct response rate for all but one individual question and higher correct response rate overall

Geog 122

John DeLay Spring 2013 Assessment Evaluated all SLOs through exam questions directly mapped to SLOs.

Spring 2013 results: For most SLOs, students met or exceeded set standard for passing scores. For a few, fell a bit short of desired percentages.

Spring 2013 responses Strategies to improve student performance include: revising and clarifying study guide items;

Compared student performance on common exam questions in DE vs FtF classes. Comparison of DE vs FtF

14

providing supplemental materials in topic areas.

classes revealed no substantive differences in student performance, with DE students consistently performing the same or better.

Geol 101 Rick Brill Spring 2013 Assessment Embedded Assessment: Each SLO mapped to specific question on exams. Identification of target goals set for every measure.

Spring 2013 results Awaiting data analysis

Spring 2013 responses Awaiting data analysis

Need to develop departmental strategies to compare FtF and online across different instructors.

Geol 101 L

Rick Brill Spring 2013 Assessment Embedded Assessment: Each SLO mapped to specific question on exams. Identification of target goals set for every measure.

Spring 2013 results: Awaiting data analysis


Need to develop departmental strategies to compare FtF and online across different instructors.

Geol 103 Nasir Gazdar

Spring 2013 Assessment Embedded Assessment: All SLOs measured, mapped to exam questions, virtual field trips, research reports, and student presentations

Spring 2013 results For some SLOs, desired student performance level met. For others, areas of improvement identified.

Spring 2013 responses Changes identified to improve student learning include: use of more interactive web-based activities including use of Google Earth; greater emphasis on principles of science; more emphasis on morphology /dimensions of structures; assignments related to topic of beaches and shorelines;

Working on departmental strategies to compare FtF and online, across different instructors.

15

inclusion of capstone project and service learning possibilities

Hist 151 Pat Patterson Cynthia Smith Keith Knuuti Deborah Forbis

Fall 2012 assessment Knowledge Survey Analysis of group data and individual performance on Knowledge Survey questions mapped to course SLOs. Individual analyses written and shared in department meetings as basis for departmental level interpretations.

Fall 2012 results Cumulative department report: In general results similar and no areas where important material is not being sufficiently understood by a majority of the students. Generally weaker student confidence related to topics Renaissance and Reformation – likely due to fact these topics fall in the chronological break point between 151 and 152 – instructors cover differently or run out of time to do adequately. Students generally did better in topics they are familiar with and have existing interest. Clearly instructors emphasize different topics in classes and differences in KS results reflect instructors make different decisions regarding necessary trade off– since to focus on one topic means less coverage of another.

Fall 2012 responses Instructors should further tap into existing areas of interest and supplement awareness by discussing topics that complement existing interest/knowledge base e.g. Roman Empire. Regarding KS result on Reformation and Renaissance - instructors should explicitly ensure topics are addressed to some degree in both the end of 151 and start of 152. See Dept. meeting Minutes: http://programs.honolulu.hawaii.edu/intranet/sites/programs.honolulu.hawaii.edu.intranet/files/history%20department%20mt%20april513.docx

Full time instructors teaching both FtF and online compare student performance on embedded questions as well as response to KS questions. Results comparable between DE and FtF classes – greater attention to be paid to standardizing assessment enable clearer comparisons in future semesters.

Spring 2013 assessment Knowledge Survey Analysis Faculty summarized analyses of individual KS responses and comparisons of individual sections to department numbers.

Spring 2013 results Cumulative department report: Knowledge Survey Analysis, For some topics, students registered less than desired confidence (environmental impacts on civilization, and Renaissance and Reformation).

Spring 2013 responses Assign more in-class assignments and/or homework requiring students to compare different topics/civilizations/belief systems; address

Spring 2013 Comparison – DE vs FtF The rigor of assessment methods is equivalent and in general, grade distribution and quality of answers showed parity

16

Those teaching FtF and DE sections compared student performance Embedded Assessment Department chose a general topic area mapped to specific SLOs and all included question related to this on final exams.

Linked to inadequate attention in class consistently across topics, as well as running out of time by end of semester. Students indicated less than desired confidence in ability to compare and contrast between civilizations, experiences and time periods. Linked to need to better demonstrate and walk students through cross topic comparisons explicitly in class. Embedded analysis Based on individual comments input following conclusions were drawn regarding student performance. Content: *Points scored on question related to Islam demonstrates need for improvement. *Students did not answer all aspects of essay questions – incomplete work *Some explanations too simplistic and superficial and contained problematic statements. *Lack of grounding in chronology when doing compare and contrast question; some lacked understanding of evolution of religions. Skills: *Need work tracing changes over time related to growth / evolution of religions. *Compare and contrast abilities needs work

climate change/environmental influences on societies consistently throughout semester. Cumulative list of actions from embedded results: *Reevaluate/ revise essay question; proactively address areas of common misunderstanding; more time spent outlining fundamental cause and effect relations; more homework and class activities to evaluate cause and effect topics; more class time to emphasize change over time of world religions.

between students in DE and FtF classes

17

Embedded analysis Fall 2013. Evolving system of embedded exam questions – moving from general theme question to common question included in all exams. In Fall 2013 all sections will use common exam question mapped to SLO.

Awaiting Fall 2013 results

Beginning in Fall 2013 Qualitative comparison assessment with cross-instructor reading and evaluation of students comparing DE and online performance

Hist 152 Pat Patterson Cynthia Smith Keith Knuuti

Fall 2012: Analysis of group data and individual performance on Knowledge Survey questions mapped to course SLOs. Individual analyses written and shared in department meetings as basis for departmental level interpretations.

Fall 2012 results Cumulative department report: All instructors had areas that were strong and others weak – reflecting ongoing challenge of covering so much material. To focus on one topic leads to less coverage of other topics. Also, for those who emphasize class work related to development skills (e.g. working with primary documents), there is a trade off in less treatment and less student confidence in other content areas.

Fall 2012 responses Instructors should maximize existing student interest and supplement awareness by complementing existing interest/ knowledge base e.g. World War II. Will continue to have departmental wide discussions of solutions to balance in topic coverage.

Full time instructors teaching both FtF and online compare student performance on imbedded questions as well as response to KS questions.

Spring 2013 Assessment Knowledge Survey Analysis, Faculty summarized analyses of individual KS responses and comparisons of individual sections to department numbers. Those teaching FtF and DE sections compared student performance Embedded Assessment

Spring 2013 results Knowledge Survey Analysis, Less retention and confidence regarding topics covered at beginning of semester; less confidence about topics covered at end of semester when instructors can run out of time. Strengths –students scored high in degree of confidence in areas emphasized by instructors and where there is

Spring 2013 responses Knowledge Survey Stay on schedule to adequately address final key topics; use more media / readings to link earlier topics to later contemporary concerns; increase sharing between instructors of materials related to current issues such as decolonization

Comparison; Knowledge Survey Responses, Results were comparable. But will increase number of media links for DE class. Embedded Assessment Spring 2013: DE students showed better understanding of impact of WWII – likely

18

Department chose general topic area mapped to specific SLOs and all included question related to this on final exams.

existing interest, and recent topics where more media or visual supplements available. Embedded analysis Results – Spring 2013 Content: Some content areas need improvement in scope of understanding (e.g. all areas of WWII, not just Europe), more understanding of causes and effects of key terms (e.g. appeasement); clearer understanding of chronology of events leading to outbreak of WWII. In terms of skills: improve ability to provide more specific examples in support of statements

Embedded Analysis Responses - Fall 2013 Need clearer instructions on study guides regarding need to support with specific examples; need better use of visual aids in class to clarify early stages of WWII in Asia/Pacific and impact on civilians.

because more visuals (pictures, linked media). DE students answers were briefer (possibly due to online exam format?) but comparable in effective answers given; scores were comparable.

Embedded analysis Fall 2013. Evolving system of embedded exam questions – moving from shared general theme to common question included in all exams. In Fall 2013 all sections will use common exam question mapped to SLO.

Awaiting results Beginning Fall 2013 Qualitative comparison assessment with cross-instructor reading and evaluation of students comparing DE and online performance

Hist 231 Cynthia Smith

Fall 2012 Assessment Embedded Assessment: Specific terms on final exam, and some formal essay topics (for WI class) mapped to course SLOs. Analyzing student performance in first and final

Fall 2012 data analyzed Assessed student performance related to 6 out of 7 SLOs related to written skills, and mastery of historical content. Student performance adequate in some areas; for other SLOs, areas of improvement identified.

Responses – Fall 2012 results More explicit instructions on expectations for written assignments. Finding more accessible primary source documents for analysis.


19

draft of required essay. Assessment of written skills, mastery as well as of historical content.

Spring 2013 results: Not taught in Spring 2013

Reviewing and changing as needed homework questions to ensure alignment with core concepts appearing on exam.

Hist 232 Cynthia Smith

Spring 2013 Assessment Embedded Assessment: Specific terms on final exam, and some essay topics (for WI class) mapped to course SLOs. Analyzing of student performance in first and final draft of required essay. Key terms identified on Final Exam; goal set for 70% of students to get 3 or higher points on historical identifications.

Spring 2013 results: Insufficient number of students demonstrating mastery of citation formats on research writing. While most students scored 3 or higher on the embedded terms, there were a few terms where many students provided inadequate answers.

Spring 2013 responses Will review /revise explanation sheet for citation styles; will include as explicit aspect of grading rubric accuracy of citation methods. Will create and share explicit grading rubric to better emphasize key aspects of research, supporting with direct quotations and citing sources. Will review and revise webtext and assignments related to terms where student performance was weaker.


Hist 246 James McKeon

Spring 2013 Assessment Embedded Assessment: Each SLO mapped to specific question on final exam, or mapped to specific essay topic (WI class)

Spring 2013 results: Knowledge of specific terms/content demonstrated on exam met desired performance levels. Work on exam essays need improvement in ability to reinforce themes with supporting examples and details. Work on formal essays did show ability to integrate information from readings but use of

Spring 2013 responses Emphasize need to support exam responses with specific examples. Provide more feedback on how to improve organization, style and grammar. More references for students to writing resources. Content delivery


20

citations and supporting evidence did not meet desired levels.

focused more on need to integrate synthesis of themes with supporting details.

IS 106 Norman Takeya

Fall 2012 Assessment Embedded Assessment: For some SLOs, components of final project mapped to SLOs. Other SLOs linked to exam questions. Numerical analysis of grades earned carried out.

Fall 2012 results Sufficient number of students achieved desired performance levels for all SLOs

Fall 2012 responses No changes needed.


Spring 2013 Assessment Embedded Assessment: For some SLOs, components of final project mapped to SLOs. Other SLOs linked to exam questions. Numerical analysis of grades earned carried out.

Spring 2013 results: Sufficient number of students achieved desired performance levels for all SLOs

Spring 2013 responses To improve assessment, adding embedded assessment in future semesters.

Math 24 Steven Mandraccia

Spring 2013 Assessment Embedded assessment



Met 101

Rick Brill Fall 2012 Assessment Each SLO mapped to specific question on exams. Identification of target goals set for every measure.

Fall 2012 analysis Responses to findings indicated in inventory report. Carried out statistical analysis of student performance on exam questions, and compared to student performance in Fall 2012. A few areas where improvement was identified as needed or desirable. Also analyzed areas where student performance

Fall 2012 results Strategies to improve include: greater use of student interactions; instructor online focus; increased attention given to concepts through discussion and student interaction. Spring 2013 responses


21

improved.

Spring 2013 Assessment Embedded Assessment: Each SLO mapped to specific question on exams. Identification of target goals set for every measure.



Met 101 L

Rick Brill Spring 2013 Assessment Each SLO mapped to specific question on exams. Identification of target goals set for every measure.




Micr 130 John Shen Spring 2013 Assessment All SLOs measured, mapped to specific exam questions.

Spring 2013 results Student results indicate meeting the desired performance standard

Spring 2013 responses No identified changes

Same embedded questions used in DE and FtF sections of the class to enable comparison of student learning across sections.

Phil 110 Ron Pine

Fall 2012 Assessment Analysis of Knowledge Surveys done since 2005: Pre and Post test and analysis done since 2010. Inclusion of DE and FtF courses done since 2009. Embedded Assessment: Test instrument mapped to SLOs. Instructors cross analyze student responses to final exam questions linked directly to one SLO – norming grading and discussing student work

Fall 2012/earlier semesters analysis Reflection and responses to assessment findings on Knowledge Surveys indicated areas of improvement, did identify a few differences between FtF and online courses.

Fall 2012/earlier semester responses Identified departmental strategies: Need to provide increased tutoring support and revision of class materials including increased emphasis on value of abstraction and recognizing patterns of reasoning; need for more seamless integration of pattern recognition for informal

Fall 2012 System for comparison in place. Courses in alignment and learning outcomes comparable. All instructors discuss results of assessment findings.

22

and responses needed if applicable.

and formal logic; instructor written textbook revised to further emphasize applying symbolic and pattern recognition techniques; FtF and DE instructors focused more on early diagnosis techniques and getting students to see tutors since recognized need for students t realize level of mastery and encourage early tutoring help when needed.

Spring 2013 Assessment Analysis of Knowledge Surveys done since 2005: Pre and Post test and analysis done since 2010. Inclusion of DE and FtF courses done since 2009. Embedded Assessment: Test instrument mapped to SLOs. Instructors cross analyze student responses to final exam questions linked directly to one SLO – norming grading and discussing student work and responses needed if applicable.

Spring 2013 results: Some differences noted between results of students in FtF classes and online classes. Noted improvement in curricular areas revised from past semester; strategies of increased attention appear successful.

Spring 2013 responses Continue to fine-tune intervention strategies; explore online tutoring options; expand on created online tutorials for additional help.

Spring 2013 System for comparison in place. Courses in alignment and learning outcomes comparable. All instructors discuss results of assessment findings. It was noted that those in DE courses indicated greater confidence in learning of SLOs than those in FtF sections. See sample report: http://programs.honolulu.hawaii.edu/intranet/sites/programs.honolulu.hawaii.edu.intranet/files/PHIL110_

23

Assessment_Summary_SPR13.docx

Phil 120 Ron Pine Assessment Exam questions mapped to SLOs. Essay assignment evaluated via rubric of assignment components – mapped to SLOs

Couse not taught since Fall 2010.

N/A Only taught online; has not been taught for several semesters.

Phil 100 Ron Pine Fall 2012 Assessment Exam questions mapped to SLOs.

Fall 2012/earlier semester analysis (not taught Spring 2013) Students met desired performance levels.

Fall 2012/earlier semester responses Revision and expansion of rubric used to grade.

Faculty teaching DE and FtF versions of the class will align assessment methods to enable comparison of student learning.

Fall 2013 Exam questions mapped to SLOs.

Fall 2013 results (awaiting Fall 2013 results

Phil 255 Ron Pine Embedded Assessment: Embedded assessment of exam questions and written assignments mapped to SLOs

Not offered since 2010


Poli Sci 120

Larry Meacham

Spring 2013 Assessment Embedded questions on final exam mapped to SLOs

Spring 2013 results Students met desired performance levels set. Learning of students in the DE section on these mapped questions was comparable to FtF class.

Spring 2013 responses No identified changes.

Same embedded questions used in FtF and DE courses to enable comparison across sections and modes of delivery

Poli Sci 130

Larry Meacham

Spring 2013 Assessment Embedded questions on final exam mapped to SLOs

Spring 2013 results Student performance in DE and FtF fell a bit short of desired performance levels for first SLOs assessed. Online students fell a bit short of desired performance level for second SLO assessed.

Spring 2013 responses Revision of lectures given in class, and revision of online written materials to address areas addressed in exam questions.

Same embedded questions used in FtF and DE courses to enable comparison across sections and modes of delivery

Poli Sci 250

Larry Meacham

Spring 2013 Assessment Embedded questions on

Spring 2013 results Insufficient data

Spring 2013 responses Insufficient

Same embedded questions used in FtF

24

final exam mapped to SLOs

(class size too small)

data (class size too small)

and DE courses to enable comparison across sections/modes of delivery

Sci 122/L Rick Brill Spring 2013 Assessment Embedded Assessment: Each SLO mapped to specific question on exams. Identification of target goals set for every measure.




Soc 100 Krista Hodges, Matthew O’Neal

Spring 2013 Assessment Combination of methods including: analysis of student performance on embedded questions on end of semester exam mapped to SLOs; performance on discussion questions mapped to SLOs; skills performance on project mapped to SLOs. All SLOs evaluated.

Spring 2013 results: All SLOs evaluated; for most, students met or exceeded established performance levels. In a few cases students fell bit short of performance levels set by department

Spring 2013 responses Department identified specific strategies for improvement including SLOs where students met desired goals but performance could be improved. Strategies include: more assignments building research skills, more assignments to foster application of concepts.

Beginning Fall 2013, all sections of Soc 100 (DE and FtF) will conduct same embedded assessment and compare results.

Speech 151

Doug Raphael, Barbara Dias, Julie Lau

Fall 2012 assessment SLOs evaluated each semester via embedded questions on final exam. Cross-instructor peer evals conducted - viewing Informative speeches done by each other’s students, with observations linked to SLOs.

Fall 2012 results Students met desired performance levels in DE courses with one exception where DE section scored a bit lower than desired level.

Fall 2012 responses Instructor emphasized review of material with students.

Comparative assessment carried out comparing numbers of DE vs FtF students. Differences noted, analyzed and strategies to address differences clearly identified.

25

Spring 2013 Assessment SLOs evaluated each semester via embedded questions on final exam. Cross-instructor peer evals conducted - viewing Informative speeches done by each other’s students, with observations linked to SLOs.

Spring 2013 results: Department meetings conducted to discuss results of all assessment outcomes. (full report posted) Comparison carried out of FtF versus online classes: in two areas, DE students indicated less confidence in skills gained: less improvement in small group discussion and lower ability to effectively use visual aids. Changes implemented in some DE classes in Spring 2013 improved student performance including more learning from group interactions; improvement in student use of visual aids for support; all DE instructors will implement these pedagogical changes in Fall 2013.

Spring 2013 responses Response to comparison findings: new methods for and projects requiring collaborative small group work in DE courses including debate; increasing student viewing work of others and learning from them; students working together to create PowerPoint work.

Sp 251 Julie Lau Fall 2013 (not taught Spring 2013) SLOs mapped to course assignments and final exam


de assessment results1014 - university of...

Documents