Structuring and Scaling Up Embedded Assessment of

General Education Outcomes atSt. Louis Community College

Thomas M. Dieckmann, English FacultyJoyce Starr Johnson, Ph.D., Director of Curriculum and Assessment

St. Louis Community College expands minds and changes lives every day.

We create accessible, dynamic learning environments

focused on the needs of our diverse communities.

Florissant Valley campus

South County Education and University Center

About the College

Forest Park campus

Meramec campusWildwood

campus

Florissant Valley’s Emerson Center for Engineering and Manufacturing

Enrollment Profile (Fall 2014)

• 21, 218 students enrolled in credit courses

• Median Age = 23• Mean Age = 27• 59% Female• Race and culture

• Caucasian = 55%• African-American = 33%• Asian = 3%• Hispanic/Latino = 3%

• Represent nearly 100 countries

• Curricula• College Transfer = 69%• Occupational = 29%• Self-Improvement = 1%• Undecided = 1%

Students in the Florissant Valley cafeteria

Where are you in your assessment journey?

What are your challenges with assessment of student learning?

General Education Re-Design

• 2011-2013 Developed Student Learning Outcomes and rubrics to assess the outcomes

• Disciplinary “champions” identified in 2011 to lead revision of course profiles for all courses designated as “General Education”

• Intensive revision process throughout 2013-2014 academic year resulted in more than 120 courses being revised

General Education Redesign

Skill Areas• Communicating*• Higher Order Thinking*• Valuing*• Managing Information*

Knowledge Areas• Social & Behavioral

Sciences*• Life & Physical Sciences*• Mathematics*• Humanities and Fine Arts*• Global/Intercultural Studies• Interdisciplinary Studies

*Missouri State General Education Learning Outcomes

Beginning Assessment Planning

• STLCC joined HLC Academy for Assessment of Student Learning

• 4-year program, beginning Fall 2013

HLC Academy for Assessment of Student Learning

• Short-term Project Goal: – Assessment of Student Learning Outcomes (SLOs) in

General Education – focus of today’s presentation

• Long-term Project Goals (in progress):– Articulation and assessment of SLOs for all transfer programs – Articulation and assessment of SLOs for all CTE and co-

curricular programs

Project Outline:1. Sharing and gathering information2. Mapping curriculum for Gen Ed SLOs3. Developing a system for collecting artifacts4. Implementing system for analysis 5. Sharing data to affect change

General Education SLO Assessment

1. Sharing and gathering information

• Information campaign during 2013-2014 academic year stressed importance of faculty participation

2. Mapping Gen Ed SLOs

Piloted a process in

Spring 2014

3. Developing a system for collecting artifacts

Spring 2014 Pilot Results – Rating Session Planning for Spring 2015

Category Target # of artifacts

Artifacts x2 for Inter-Rater Rel.

Hours needed to review/rate (div. by 8)

Number of Faculty reviewers (div. by 2)

Estimated number of sessions (div. by 15 per session)

ENG:101 150 300 38 19 2COM:101 & COM:107

150 300 38 19 2

ENG:102 300 600 76 38 3G/I Valuing 300 600 76 38 3IDS Higher Order Thinking

300 600 76 38 3

Social and Behavioral Sciences

330 660 82 41 3

Humanities and Fine Arts

330 660 82 41 3

Life and Physical Sciences

250 520 65 33 3

Mathematics 140 288 36 18 2IDS 120 240 30 15 2G/I 120 240 30 15 2 2490 629 315 28

Spring 2014 Pilot Results – Artifact Collection Strategies

• Each of the four skill areas will strive to collect and assess artifacts from 85 sections:

– 13 sections of ENG:101

(Communicating), approx. 320 – 6 sections of COM:101

(Communicating), approx. 140– 6 sections of

COM:107(Communicating), approx. 140

– 20 sections of ENG:102 (Managing Information), approx. 400

– 20 sections of G/I designated courses (Valuing), approx. 400

– 20 sections of IDS courses (Higher Order Thinking), approx. 400

• Each of the five knowledge areas will strive to collect and assess a proportional number of artifacts based upon the number of credits they represent within Gen Ed. from the remaining 85 sections.

– 24 sections, approx. 480 from Social & Behavioral Sciences (9 out of 32 credit hours, 28%)

– 24 sections, approx. 480 from Humanities & Fine Arts (9 out of 32 credit hours, 28%)

– 18 sections, approx. 360 from Life & Physical Sciences (7 out of 32 credit hours, 22%)

– 10 sections, approx. 300 from Mathematics (4 out of 32 credit hours, 12%)

– 8 sections, approx. 160 from Interdisciplinary Studies (3 out of 32, 10%)

– 8 sections, approx. 160 from Global Intercultural (3 out of 32, 10%)

Spring 2014 Pilot Results – Creation of GEAF Position

• Respected faculty leaders were needed to vet faculty artifact submission for applicability to chosen SLOs

• Creation of General Education Assessment Facilitator (GEAF) position for each Gen Ed SLO category, to be filled by faculty members

• Work would include small stipend

GEAF Job Description

• The facilitator for each general education category coordinates the assessment processes of a category of general education and makes recommendations to the General Education Council on the basis of the assessment activities and findings.

• Duties beginning Fall 2015: – Review General Education assignments as they are

submitted; remove those that don’t address the designated student learning outcomes (1-2 hours per semester)

– Participate in orientation meeting for the reading/rating sessions (2 hours per year)

– Serve as a liaison with faculty in the collection of artifacts

– Hold 2-3 reading-rating sessions (4-6 hours per semester starting in Spring 2015)

– Review findings and make recommendations to Gen Ed Council (2 hours per semester)

– Attend occasional meetings, including those of the General Education Council as needed, and participate in discussions related to the general education categories

Current STLCC Assessment Team Personnel• Director of Curriculum and Assessment (DCA)• Director of Institutional Research and Planning• General Education Council

– Includes • Vice Presidents for Academic Affairs from each of 4 campuses• Faculty General Education Program Coordinator from 3 campuses

• HLC Academy for the Assessment of Student Learning Team• In addition to VPAAs, also includes a faculty representative from 3 campuses

• 10 Faculty GEAFs (one for each Gen Ed SLO Area)• Ad hoc members of Institutional Research and Planning and professional

staff as necessary

Collection Activities

Fall 2014 Status Report

All faculty received this document during

Service Week presentation before the start of the Fall

2014 term

Instructions

All faculty whose Gen Ed section was chosen for

assessment received this document as Part 1 (of 2) of

participation packet at beginning of Fall 2014 term

Cover LetterAll faculty whose Gen Ed section was chosen for

assessment received this document (with appropriate Gen Ed SLO listed) as Part 2

(of 2) of participation packet at beginning of Fall 2014

term

Fall 2014 Artifact Collection Process

Fall 2014 Artifact Collection Process:Challenges

• Some confusion amongst faculty that all student submissions for chosen assignment must be collected

• Some faculty failed to eliminate all identifying faculty or student information from artifacts

• Some chosen assignments unsuitable for assessment of selected SLOs

• Difficulties of collecting/submitting large and/or electronic artifacts, including videos oral presentations

Early Spring 2015 Work

• Faculty given reminder and update of assessment work during Service Week meetings

• Faculty provided with rating session schedule at beginning of semester and all Gen Ed faculty asked to participate in one session

• Courses randomly selected for second round of artifact collection during Spring semester and faculty of these courses notified

4. Implementing system for analysis

Spring 2015 Rating Sessions • Each session focused on one particular knowledge or skill goal

area (e.g. “Communicating”, “Valuing”, “Higher-Order Thinking,” or “Mathematics”) and its defined SLOs

• Faculty allowed to self-select the session he/she would attend

• Sessions led by faculty GEAF

• Sessions held in computer labs on various campuses

Spring 2015 Rating Sessions (continued)

• Faculty participants provided with brief discussion of Gen Ed assessment history, plan, and rationale

• Participants introduced to appropriate rubric for area• Participants given a sample assignment and student artifact to

rate using the rubric• Discussion (sometimes robust) followed of application of the

rubric to the assignment to ensure inter-rater reliability• Sample activity also provided participants practice with survey

instrument

Spring 2015 Assessment Survey InstrumentOpening Page screenshot

Spring 2015 Assessment Survey InstrumentGen Ed SLO category selection screenshot

Spring 2015 Assessment Survey Instrument“Communicating” Rubric screenshot

Spring 2015 Rating Sessions: Challenges• Faculty participation for some sessions– For example, 3 “Communicating” sessions had a total of 11

faculty participants– Our planning based upon Spring 2014 pilot was based up 38

attendees

• Because of this, not all submitted artifacts were read/reviewed twice, or, in some cases, at all

Category Target # of artifacts

Artifacts x2 for Inter-Rater Rel.

Hours needed to review/rate (div. by 8)

Number of Faculty reviewers (div. by 2)

Estimated number of sessions (div. by 15 per session)

ENG:101 150 300 38 19 2COM:101 & COM:107

150 300 38 19 2

Rating Session Participant Feedback: Areas of success

• Faculty reported 2-hour time block for rating sessions worked well and was appropriate

• Faculty reported appreciation for the assistance and leadership of the faculty GEAFs during the sessions

• Benchmark data was generated for future reference

Rating Session Participant Feedback: Areas for Improvement (continued)

• Faculty need to be provided with professional development regarding providing assignment instructions/documentation and selecting SLOs intended to be elicited by assignments

• Faculty should provide answer keys to assignments when appropriate

• Faculty with specific discipline expertise should rate artifacts coming out of their disciplines when disciplinary knowledge or vocabulary is being assessed

Rating Session Participant Feedback: Areas for Improvement (continued)

• Survey instrument needs some refinement to make assessment more efficient and less cumbersome

• Knowledge and Skill Area rubrics need some refinement in some categories to more clearly distinguish between levels of achievement

• Rating session scheduling should take into account particular departmental/disciplinary needs to ensure appropriate faculty can attend relevant sessions

Preliminary Findings

• Positive growth in four skill areas between introduction and reinforcement

• Baseline performance indicators for all 10 General Education categories

Current Status• Collecting second round of artifacts for assessment in the Fall

semester

• Next is step 5 (the most important) of our Assessment plan:

5) Sharing data to affect change– The long term goal is to ensure that students develop skills

and knowledge that will improve their lives– Planning is underway for faculty development activities in

the fall

Questions?Thomas DieckmannInstructor II, English

St. Louis Community College, Florissant Valleytdieckmann5@stlcc.edu

Joyce Starr Johnson, Ph.D.Director of Curriculum and Assessment

St. Louis Community Collegejjohnson789@stlcc.edu