assessment and student learning: direct and indirect measures of assessment

Engaging Online Faculty and Administrators in the Assessment Process at the American Public University SystemAssessment and Student Learning: Direct and Indirect Measures of Assessment

Fundamental Questions

AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE

•How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students? •What evidence do you have that students achieve your stated learning outcomes? •In what ways do you analyze and use evidence of student learning? •How do you ensure shared responsibility for student learning and for assessment of student learning? •How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning? •In what ways do you inform the public and other stakeholders about what students are learning---and how well?


How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students?


Week Topics Course Objectives Readings Assignments

1

International Relations: General Theory Part 1

CO-1: Assess and critique the validity of basic theories that explain the international system, power politics, war, and diplomacy.

CO-2: Appraise the realist, liberal, and constructivist, perspectives of international relations.

Stephen M. Walt, “International Relations: One World, Many Theories” Foreign Policy (Spring 1998)

Ole R.holsti, “Theories of International Relations” from Explaining the History of American Foreign relations, 2nd ed. 2004 Stanley Hoffman, “An American Social Science” Daedalus (1977). Carol Cohn, “Slick’ems, Glick’ems, Christmas Trees, and Cookie Cutters: Nuclear Language and How We Learned to Pat the Bomb,” Bulletin of the Atomic Scientists (1987). J. Ann Tickner, “You Just Don’t Understand: Troubled Engagements between Feminists and IR Theorists” International Studies Quarterly

Discussion Board: Virtual Introduction Web Resources: Access www.irtheory.com/index.htm

Course Materials: Lesson 1 Lecture Notes

Course Syllabus Outline

By explicitly stating these expectations up front, you have defined what your students will learn and how you will assess their learning.


Curriculum Mapping

(I)ntroduces – Student is not familiar with content or skill. Course instruction concentrates on introducing students to the content area or skill. (R)einforces – Student brings reasonable knowledge/content/skill/competency to the course. Course instructional activity continues to teach and build upon previous competency and reinforces content or skill competency. (A)pplies – Student brings reasonable knowledge/content/skill/competency to the course, and the course instruction applies the content or skill. Indicate whether each institutional outcome is introduced, reinforced, or/and applied in each of the required and core courses for your degree program. Please denote this by placing an I, R, or A in the appropriate box.

Academic Skill Graduates of APUS will possess academic skill related to their specific discipline. The graduate will master the generally accepted theories, concepts, principles, and/or practices associated with their discipline.

Communication Graduates of APUS will be able to clearly communicate ideas in written form.

Critical Thinking Graduates of APUS will be able to analytically identify issues with the ability to evaluate problems, formulate solutions, and evaluate the consequences. Graduates will be able to express a variety of other viewpoints and methods of inquiry.

Information Literacy Graduates of APUS will be technologically proficient in accessing and utilizing information. The graduate will be able to gather information from a variety of sources, utilize information in an appropriate manner to address issues and inform action.

Lifelong Learning Graduates of APUS will have the ability to identify, pursue and acquire specific and new knowledge after the end of formal schooling. They will be able to apply this knowledge in real world settings.

Required and Core Courses


What evidence do you have that students achieve your stated learning outcomes?


Direct and Indirect Measures


Evidence

In what ways do you analyze and use evidence of student learning?

• What are the essential skills and knowledge I am trying to teach? • How can I find out whether students are learning them? • How can I help students learn better?


Example

•Choose a student learning outcome to assess

•Choose an assessment technique – direct or indirect measure

•Administer the assessment to the student

•Analyze the data

•Use data for continuous improvement


Example

•In a particular course, a faculty member chose to evaluate a student learning outcome relevant to information literacy.

•The faculty member chose the quiz for week two which assesses student’s proficiency in information literacy.

•After a thorough analysis of the data over a six month period, data demonstrated that students often missed a specific question on how to conduct basis research in an electronic environment.

•Lead faculty members of the course met to discuss the data and how the curriculum could be reinforced to ensure that students gain proficient knowledge on how to conduct research in an electronic environment.

•A new lesson (e.g., interactive simulation) that reinforces the content was added to the course.

•Student quiz results were then monitored to evaluate whether the reinforced curriculum increased the quiz scores of the students, thus enhancing their learning.


How do you ensure shared responsibility for student learning and for assessment of student learning?

ASSESSMENT AT APUS


How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?

APUS Program Review Process


External Review

Analysis

Program Review

Meeting

Three Year Plan

Follow-Up

Data Collection

Data Collection and Analysis


• Course books

• Electronic resources

• Learning strategies

Library and Learning Resources

• Student learning outcomes

• Instructional strategies

• Evaluation procedures

• Academic rigor

Curriculum Assessment

• Expert reviewer report

• Industry Advisory Council report

External Reviewer Feedback



• Analysis of faculty credentials and expertise to ensure breadth and diversity

Faculty

• Student demographic information

• Enrollment History

• Growth trends

Students

• Curricular Mapping

• Assessment measures

• Fact book

Learning Outcomes Assessment



• Program benchmarking with similar programs and institutions

Program Benchmarking

• Evaluation of findings

• Program recommendations

• Three year proposed strategic plan

Program Directory Summary

• Dean’s observations

• Meeting minutes

Review and Findings


• One of fourteen charter member institutions

• College Choices for Adults website

http://www.collegechoicesforadults.org/

Transparency by Design Initiative

http://www.apus.edu/community-scholars/learning-outcomes-assessment/index.htm

APUS Learning Outcomes Assessment Website

• Meetings

• Faculty, staff, and student newsletters

• Faculty and student lounges

Internal Communications

In what ways do you inform the public and other stakeholders about what students are learning---and how well?







Things you do to contribute to assessment at APUS:

•Be aware of the program and institutional level outcomes and how your course objectives contribute to them

•If an assignment in your class measures learning in those areas particularly well, let the PD know

•Use internal assessments (class assignments) to ensure your students are learning, and review them over time

•Be aware of how students in your areas are performing on external assessments (MFT and EPP tests)

•Participate in program review processes when they occur (every 3 years)

assessment and student learning: direct and indirect measures of assessment

Documents

assessment of student

evidence of student

stated learning outcomes

students proficiency

student quiz results

data use data

assessment process

assessment technique