table of - moberly area community college...continuous improvement report 5 services. select course...

TABLE OF CONTENTS Table of Contents 1

Purpose of Assessment 2

History of Assessment at MACC 2

Relationship to Mission and Purposes 6

Assessment Activities 2013-2015 10

Professional Development 10

Strategic Planning and Accreditation 11

Assessment of General Education 11

Assessment of Career and Technical Education Programs 11

Assessment of Support Services 12

Other Assessment Activities 12

Budget Summary 13

Entry-Level Assessment of Student Learning 14

Placement Testing 14 Student Advisement 15

Class Profile Report 16

Student Profile 17

Mid-Level Assessment of Student Learning 19

Course Grades 19

Academic Probation 20

Classroom Assessments 20

Comprehensive Final Exams 20

Developmental Education 20

Course Evaluations 22

Assessment of Support Services 22

Student Retention 23

Exit-Level Assessment of Student Learning 25

CAAP Test 25

Capstone Courses 28

Essential Skills 28

Licensure, Certification, and Exit Exams 29

Degrees Conferred 30

Follow-Up Assessment 31

180-Day Placement Survey 31

Student Transfer 33

Other Assessment Efforts 33

Program Review 33

Employee Satisfaction 34

Conclusion 35

Continuous Improvement Report 2

2013-2015 Continuous Improvement Report

PURPOSE OF ASSESSMENT Accountability is at the forefront of higher education, and assessment is an integral component of

demonstrating institutional effectiveness. Postsecondary institutions are expected to provide a framework to

encourage, cultivate, and nurture student success.

Assessment provides an opportunity for feedback, reflection, and improvement of teaching and learning

across multiple levels, including at the classroom, program, and institution level. The Missouri Assessment

Consortium identifies three major purposes of assessment:

Improvement of student learning and instruction

Achievement of institutional mission

Accountability for achievement of educational goals

Although assessment should be fully integrated into the institution’s entire operational system, including

academic services, administration, student affairs, institutional planning, and resource allocation, the focus of

institutional assessment must be on student learning (Missouri Assessment Consortium, Handbook of

Assessment).

The Higher Learning Commission of the North Central Association of Colleges and Schools has also heralded

the importance of assessment. Assessment is embedded within the criteria for regional accreditation. The

Commission’s Statement on Assessment of Student Learning states that “assessment of student achievement

is essential for each higher learning organization that values its effect on the learning of its students.” The

Higher Learning Commission expects organizations to have structured assessment processes that are

continuous, that involve a variety of institutional constituencies, and that provide meaningful and useful

information to the planning processes as well as to students, faculty, staff and administration. Organizations

should be able to document the relationship between its assessment processes and its improvement of

institutional effectiveness.

HISTORY OF ASSESSMENT AT MACC An Assessment Advisory Committee was named in fall 1997 for the purpose of developing an Institutional

Effectiveness Plan (i.e., to develop processes and procedures to monitor factors affecting how well MACC is

accomplishing its mission). The goal of the committee was to produce a written, comprehensive plan

containing two primary components: an assessment of student learning and an assessment of support

services. In its initial year, the committee published its purpose, goal, guiding principles, and other

information.

In 1998-1999, the committee continued building on the work of year one, specifically increasing awareness

campus-wide about assessment, furthering the work of the general education subcommittee, increasing


committee members’ knowledge of assessment, and drafting assessment plans for each degree and

certificate program offered by MACC.

In 1999-2000, the committee focused on identifying strategies and measures to determine accomplishment

of learning outcomes; evaluating, revising, and finalizing assessment plans; developing an administrative

evaluation process; collecting and evaluating data according to assessment plans already adopted;

developing a systematic review of educational programs; and providing conference and workshop

opportunities for faculty and staff.

In 2000-2001, the committee concentrated on assessing general education; collecting and analyzing data;

educating faculty and staff about assessment issues; and using the results of assessment to improve student

learning.

Assessment efforts begun in previous years and continued in 2001-2002 culminated with the re-accreditation

visit by the Higher Learning Commission in March 2002. Staff development opportunities in assessment

practices were offered, and a group of general education faculty studied and experimented with classroom

assessment techniques. The student newspaper featured several articles on assessment and accreditation,

and a student was appointed to the Assessment Advisory Committee. Additionally, the first assessment

stipend was offered to assist faculty with assessment efforts. Course syllabi were revised to include

assessment practices, and the Administrative Evaluation Process was piloted.

In 2002-2003, following a study by two MACC faculty, the Assessment Advisory Committee recommended

that the CAAP (Collegiate Assessment of Academic Proficiency) exam continue as the exam of choice to

assess general education knowledge and skills, but that the administration of the exam be restructured.

Additionally, the Assessment Advisory Committee recommended that the WorkKeys assessment replace the

CAAP exam for career and technical graduates. President’s Council adopted these recommendations for

implementation in 2003-2004. An Institutional Effectiveness website was developed, faculty-authored

articles were included in the assessment newsletter, and two student positions were permanently added to

the Assessment Advisory Committee. In addition, the Assessment of Support Services committee

implemented an annual faculty/staff satisfaction survey.

In 2003-2004, the first annual student satisfaction survey was developed and administered, revealing that

students are generally satisfied with the quality of service MACC provides. The faculty evaluation process

was revised and MACC joined the Missouri Consortium for Value-Added Learning. The CAAP general

education Writing and Critical Thinking subtests were embedded into related subjects. MACC students

scored above the national norms on both tests. Students completing AAS degrees participated in WorkKeys

assessments. In fall 2003, some career and technical programs began pre-testing student samples using the

WorkKeys test. Four nursing instructors completed a critical thinking project supported by an assessment

stipend.

While many assessment activities were continued in 2004-2005, there were new projects as well. MACC

participated in a pilot program to measure value-added learning by giving the Collegiate Learning Assessment

(CLA) to a sampling of students in spring 2005. The Director of Institutional Effectiveness and Planning wrote

and received a grant to support a statewide consortium of community college faculty and staff to study and


make recommendations about developmental education in Missouri. A Language & Literature Professor

received an assessment stipend to purchase software to analyze the relationship between students’ self-

reported information about attendance and study habits to their responses about teaching effectiveness on

faculty evaluations. The 2005-2010 Master Plan was completed and approved by the Board of Trustees.

During the 2005-2006 academic year, technology was utilized to make assessment activities simpler and

more effective. The fourth annual Faculty/Staff Satisfaction Survey was administered during spring 2006 in

an online format. Employees received a link to the survey via an email message. Responses were

automatically entered into a downloadable spreadsheet, which saved hours of staff time in data entry. The

paper and pencil ASSET test was phased out and the College began using Compass, an Internet-based

placement exam. With Compass, students can be placed in courses the same day because results are

received immediately. MACC also began administering the CBASE exam as an exit assessment for all students

completing the Associate of Arts in Teaching (AAT) degree. Traditional assessment activities such as the

Assessment in Action newsletter and the assessment stipend were continued.

In 2006-2007, it was decided that comparing entrance and exit WorkKeys scores did not prove to be an

effective method of measuring value-added learning in the career and technical programs. Pre-testing of

career and technical students was discontinued, and the Dean of Career and Technical Education began

research into other assessment options. The program review processes for general education and career and

technical programs was revised, and an assessment plan was developed for the Medical Laboratory

Technician program. Administration began researching the PEAQ and AQIP accreditation processes in

preparation for the College’s pending accreditation visit in 2012. The 2007-2012 Master Plan was completed

and approved by the Board of Trustees with a renewed focus on assessment activities at the College.

In 2007-2008, examination and revision of the career and technical education plans began in an effort to

bring current departmental practices in line with the goals and objectives of the plans and to determine best

practices for each department. The Coordinator of Early Childhood Education completed an Early Childhood

capstone portfolio project funded by the assessment stipend. In spring 2008, it was decided after campus-

wide discussion through the Assessment Advisory Committee that MACC would continue using the PEAQ

(traditional) accreditation process for the next accreditation cycle. Traditional assessment activities such as

the Assessment in Action newsletter were continued.

In fall 2008, MACC began administering the Noel-Levitz Student Satisfaction Inventory to solicit student

feedback about the quality of instruction provided by the College. Revision of the career and technical

education program assessment plans continued throughout 2008-2009 year. In January 2009, MACC joined

the National Community College Benchmarking Project (NCCBP) in order to better gauge College

performance among peer institutions and determine future standards of achievement. The 2009-2014

Master Plan was completed and approved by the Board of Trustees. The plan contains the new mission and

vision documentation adopted by the Board in November 2008, and the entire plan was reformatted and

rewritten to better reflect College practices and future plans.

In 2009-2010, an increased focus on College-wide assessment practices led to several changes in traditional

assessment instruments and methods. The Student Satisfaction Survey was completely updated and

redesigned to better reflect current practices and to include the assessment of additional college support


services. Select course sections test-piloted an electronic version of the survey. Additional communications

questions were added to the Staff Satisfaction Survey to gauge the effectiveness of current communications

practices at MACC. Feedback from these questions was distributed to administration and was incorporated

into the College’s Master Planning efforts. The Office of Academic Affairs, in conjunction with MACC English

faculty, reviewed and revised the current developmental English placement score cutoffs.

During 2010-2011, the 2011-2016 Master Plan was completed and approved by the Board of Trustees. For

the first time, stakeholder feedback for the plan was collected online via Survey Monkey. The change in data

collection methods was successful, increasing the number of responses received to over 800 (compared to

354 responses collected for the 2009-2014 Master Plan). Also in 2010-2011, the Missouri Department of

Higher Education (MDHE) began actively developing a plan to implement a Performance-Based funding

model for all Missouri colleges based on requests by the legislature for increased data and accountability.

In fall 2011, the College underwent a comprehensive accreditation review and site visit by the Higher

Learning Commission (HLC). MACC received the Commission’s top rating - full ten-year accreditation with no

additional follow-up visits or reports. Also in 2011-2012, the annual satisfaction survey was delivered entirely

online for the first time, and the employee satisfaction survey was rewritten and reorganized to gather more

relevant data about employee concerns. Better methods were developed to more accurately track retention

and graduation rates, as well as distance education enrollment patterns. Enhanced datasets were distributed

college-wide through appropriate committees for additional discussion and review. The Missouri

Department of Higher Education began implementation of state-wide Performance Funding measures,

leading to significantly increased reporting burdens for the College.

In 2012-2013, an increased call for transparency and accountability at the national and state levels led to

significantly increased reporting requirements for the College. Additional sections on consumer information

were added to the college catalog and the website, and a college-wide compliance committee was formed to

ensure the College is fulfilling its compliance obligations. The student satisfaction survey was updated to

include questions about student activity preferences, and a question asking students to direct their

comments towards a specific site was added to give data recipients more site-specific feedback. Faculty

groups met throughout the year to continue ongoing assessment projects, and several MACC administrators

participated in state-wide assessment initiatives and groups. The 2011-2016 Master Plan was updated and

presented to the Board of Trustees in summer 2013.

In 2013, MACC joined the Complete College America initiative in Missouri. MACC’s Completion Team focused

on identifying strategies that would improve student retention and completion. The committee outlined

initiatives that included multiple measures of placement, academic maps, and redesigned developmental

education courses. It piloted its first initiative, a writing appeal, in spring 2014. Participation in the Complete

College America initiative also included the collection of additional data. In anticipation of the Year 4 HLC

assurance review scheduled in fall 2015, administrators began familiarizing themselves with HLC’s Open

Pathway model for continued accreditation.

Nationally, college completion became one of the top higher education issues in 2014-2015. MACC

continued its completion efforts by piloting four additional Completion Team initiatives (i.e., Math Exit

Competency Assessments, a College Algebra math pilot, Compass workshops, and an English co-req class).


MACC ramped up accreditation efforts by forming a HLC Leadership Team in fall 2014. The team researched,

gathered evidence, and drafted MACC’s Assurance Argument for reaccreditation during 2014-2015. A newly-

formed Strategic Planning Committee spent 2014-2015 redesigning the College’s planning process and

drafting the MACC 2015-2018 Strategic Plan. The scope of the plan was expanded to include a departmental

component. The planning process also involved revising MACC’s mission statement. Missouri Department of

Higher Education requested additional data collection on remediation, specifically considering multiple

measures and alternate pathways. MDHE also explored the possibility of adding a sixth performance funding

measure on graduate outcomes (i.e., employment and transfer). Data collection of a pilot cohort may be

required in 2015-2016.

RELATIONSHIP TO MISSION AND PURPOSES

MACC is dedicated to accomplishing its mission statement. To gauge MACC’s progress, measures of

institutional effectiveness are in place and are reviewed regularly. The tables on the following pages describe

assessment processes and tools that have been implemented to evaluate specific components of MACC’s

mission statement and institutional purposes. The first column lists key areas of MACC’s mission statement

and institutional purposes. The second column identifies specific components of the mission statement and

purposes. The remaining columns list the assessment measure and tool, the responsible person or

department, and the primary use of the results.

7

Measures of Institutional Effectiveness 2013-2015

Mission and Purposes

(What areas are critical to mission fulfillment?)

Specific Components

(What specific components of these areas will be evaluated?)

Indicator

(What will be measured?)

Assessment or Measurement Tool

(How will it be measured?)

Responsibility

(Who directs or Implements the process?)

Action

(How are results reviewed and used?)

Quality of Educational Programs and Services

Career and Technical Programs (AAS, ADN, and Certificates)

Occupational Skills Essential Skills Attainment

Capstone Courses

Certification and Licensure Exams

Post Testing

Student Portfolios

TSA Exam

National Career Readiness Workforce Certificate

Faculty

Program Coordinators

Director of Career and Technical Programs

Testing Services Personnel

Program Advisory Committee Review

Program Reviews

Curriculum Revision

National, State, and Local Benchmarking

General Education Competencies

WorkKeys Assessment (AAS, Cert)

CAAP Exam

Capstone Courses

Compass/CAAP linkage report

ACT/Compass linkage report


Faculty

Office of Academic Affairs

Director of Institutional Effectiveness and Planning

Program Improvement

Curriculum Revision

Teaching/Learning Improvement

Program Reviews

Job Placement Follow-Up Surveys

Placement Rates

Placement Office Program Advisory Committee Review

Program Improvement

Curriculum Revision

State and Local Benchmarking

Program Reviews

Student Retention Retention Rates

Graduation Rates

Completion Team

Retention Committee

Director of Institutional Effectiveness & Planning

Focus Retention Efforts

State and National Benchmarking

Retention Manager

Student Satisfaction Course Evaluations

Follow-Up Surveys

Noel-Levitz Student Satisfaction Inventory

MACC Student Satisfaction Survey

Graduate Satisfaction Survey

Dean of Career and Technical Education

Placement Coordinator

Faculty




Curriculum Revision

Program Reviews

President’s Council

Assessment Committees

Student Success Post-Testing

Grades

Transfer follow-up studies


Faculty

Program & Department Coordinators

Curriculum Revision


Program Value Program Review Process Dean of Career and Technical Education

Program Improvement

Resource Allocation

Program Certification State Agency Review

State and National Certification Exams

Program Directors/Coordinators


Program Improvement

National and State Benchmarking

8



Specific Components

(What specific components of these areas will be evaluated?)

Indicator




Responsibility


Action


Quality of Educational Programs and Services

General Education (AA, AAT, and AS)

General Education Competencies

CAAP Exam

Student Artifacts/Institutional Portfolio

General Education Rubrics

ACT/CAAP Linkage reports

Compass/CAAP Linkage report

MoGEA Exam (AAT)


General Education Assessment Panel

Faculty



Program Improvement

Curriculum Revision


Program Reviews

Transfer Success Transfer Studies

Student Tracking

Transfer Rates

State Reports

Articulation Agreements

Dean of Student Services

Vice President for Instruction

Dean of Academic Affairs


Program Improvement

Curriculum Revision

Student Support Services Improvement

Program Reviews



Student Satisfaction Survey



Faculty

Dean of Student Services



Curriculum Revision

National and State Benchmarking

College Assessment Committees


Graduation Rates

Completion Team

Retention Committee

Director of Institutional Effectiveness & Planning

Retention Manager

Program Reviews

Student Academic Gains

Pre/Post Testing

Final Exams

Grades/Grade Distribution

ACT Linkage Reports

Follow-Up Studies

Faculty



Curriculum Revision

Program Value Program Review Process Dean of Academic Affairs

Faculty

Program Improvement

Resource Allocation/Planning

Developmental Education Student Academic Gains

Pre/Post Testing

Grades

Journals

Follow-Up Studies


Faculty

Director of LARC

Developmental Education Coordinators


Curriculum Revision




Student Survey/Comments



Student Satisfaction Survey


Academic Department


Curriculum Revision



Program Reviews


Graduation Rates

Completion Team

Developmental Education Coordinators


Retention Committee

Retention Manager

Program Review

9



Specific Components

(What specific components of these areas will be

evaluated?)

Indicator




Responsibility


Action


Quality of Support Services for Students

Student Support Services (departments closely aligned with student learning)

User Satisfaction Student Satisfaction Survey

Faculty/Staff Satisfaction Survey

Departmental Self-Studies

Assessment of Support Services Committee

Department

Service Improvement

Resource Allocation

Quality of College Personnel Faculty, Staff, and Administrators Professional Competencies

Faculty

Faculty Portfolios

Administrative Observation and Evaluation

Course Evaluations

Staff Development Participation Supervisors

Supervisor Evaluation

Staff Development Participation Administrators

Administrative Evaluation Surveys

Supervisor Evaluation

Staff Development Participation

President

Vice-President for Instruction



Department Supervisors

Administrative Evaluation Committee


Departmental Improvement

Administrative Improvement

Quality of Collegiate Environment

Student Support Services User Satisfaction Student Satisfaction Survey

Faculty/Staff Satisfaction Survey

Service Area Self-Study

Course Evaluations


Assessment of Support Services Committee

Department Heads

Service Improvement


Quality of Community Partnerships and Cooperative Efforts

Workforce Education Client Satisfaction Certification Rates

Course Evaluation

Entrepreneurship and Business Development Center

Program Improvement

Accessibility Numbers Served

Fiscal Year Report

Entrepreneurship and Business Development Center

Advisory Committee Review

Program Improvement

Fiscal Responsibility Fiscal Year Activity Report Entrepreneurship and Business Development Center

Finance Office

Program Improvement

Resource Allocation

Educational Partnerships Partnerships Articulation Agreements

Dual-Credit Agreements

Clinical Agreements

On-Site 4-year Programs (CMU, Columbia College)

Co-Enrollment Models (MU, TSU)

Vice-President for Instruction



Program Improvement

Curriculum Revision


ASSESSMENT ACTIVITIES 2013-2015 The following narrative highlights assessment accomplishments and activities during the 2013-2014 and the

2014-2015 academic years.

Professional Development

Three faculty professional development days were held each year.

The following general sessions were connected to teaching and learning improvement in 2013-2014:

Cengage TeamUp, Variations on Writing Assignments, Amending the Universal Rubric (John

Manccuso)

Writing Across the Curriculum (WAC) Initiatives (Dr. Patti Cole and Cindy Droege)

Multiple Pathways Model for Using Brain Research to Enhance and Energize Learning

High-Challenge, High-Support Pedagogy for Underprepared Students Across the Curriculum

and Rethinking Remediation to Increase Student Completion of College-Level English and

Math (Dr. Katie Hern)

During the 2014-2015 academic year, faculty professional development day general sessions

included the following:

Critical Thinking (and Its Assessment) Is at the Heart of a Higher Education (Dr. Scott Alberts

and Dr. Carol Marshall)

What’s Working in Improving Student Completion (Dr. Kent Farnsworth)

Office of Civil Rights 101 and Civil Rights Law Compliance within the Classroom (Joshua

Douglass)

LOGOS Project: The One-Read Pilot (Jill Mackey and Michelle Scanavino)

Introduction to Competency-Based Education (Becky Lopanec)

The Vice President for Instruction continued to serve on the statewide Taskforce on College & Career

Readiness created by Missouri Department of Higher Education (MDHE). Among other things, this

group worked on developing statewide placement and threshold scores and alternate math

pathways for students that do not include College Algebra and bridge courses for high school

students to reduce the need for remediation post-high school graduation.

Dean of Off-Campus Programs & Instructional Technology served on a MDHE Dual Credit Workgroup

that reviewed the Dual Credit Policy point-by-point checking for alignment with Higher Learning

Commission (HLC) and National Alliance of Concurrent Enrollment Partnerships (NACEP)

accreditation requirements. Their review was also conducted to ensure statewide consistency in dual

credit programs. As a result, changes were made to the eligibility requirements for high school

students and the required credentials for faculty.

The Director of Instructional Technology was elected to serve a 2-year term on the Missouri Distance

Learning Association (MoDLA) the Board of Directors beginning in July 2015.

Members of the Distance Learning Subcommittee that were tasked with making a recommendation

to President’s Council regarding which learning management system the College should adopt

attended several LMS conferences (e.g., Desire2Learn, Jenzabar, Canvas). Evaluating the assessment

tools and features within each LMS was an important part of their review process.


Two full-time English faculty members served on the Midwest Regional Association for

Developmental Education’s (MRADE) Executive Board, on as Secretary and one as the Public

Relations/Marketing contact.

Several math and English faculty were active in Missouri Developmental Education Consortium

(MoDEC).

The Developmental Math Coordinator served on the Missouri Mathematics Pathways Task Force, a

group charged with exploring alternative math pathways for students.

The Coordinator of Teacher Education was active on Missouri Association for Teacher Education

(MACTE).

The Director of Institutional Effectiveness & Planning served on the Missouri Community College

Association (MCCA) Research Council. The group met regularly to discuss federal and state reporting

requirements and ensure consistent reporting at the state and federal levels.

Eight employees attended the Higher Learning Commission’s Annual Meeting in April in 2014 to

become more familiar with the Open Pathways system.

In April 2015, ten employees, primarily co-chairs of the criterion teams, attended the HLC Annual

Meeting to gather information and guidance that would help them draft the Assurance Argument.

The President of MACC’s Board of Trustees also attended.

Strategic Planning and Accreditation

The College’s 2015-2018 Strategic Plan was developed during the 2014-2015 academic year by a 15-

person committee comprised of faculty, staff, and administrators across locations. It was presented

to the Board in fall 2015 and will be reviewed and updated annually.

The Vice President for Instruction led the efforts to form a 15-person Higher Learning Commission

Leadership Team in fall 2014 and was responsible for the general oversight of reaccreditation efforts.

The Leadership Team was tasked with developing outlines and drafts of the Assurance Argument.

Almost 50 additional employees across departments and locations served on criterion,

communications, and evidence teams that were led by leadership co-chairs. The final argument will

be submitted in November 2015.

President’s Council began exploring HLC’s Persistence and Completion Academy for MACC’s Quality

Initiative, a requirement in the Open Pathway model.

A Multi-Site Location Report was submitted to the HLC in preparation for a Multi-Location Evaluation

Visit on June 1, 2, and 3, 2015, with a peer reviewer. (The peer reviewer found that the operations

at locations were adequate and required no further review or monitoring by the HLC.)

The Director of Library Services continued to oversee the expansion of the Resource Library. The

Resource Library is accessible to all employees and includes institutional data, statistics, reports,

meeting minutes, etc. It is relied on heavily by employees at the College who conduct research, track

progress on initiatives, and prepare reports for internal and external purposes.

Assessment of General Education

Each year two sections of the Collegiate Assessment of Academic Proficiency (CAAP) are embedded

in selected general education classes. The critical thinking subtest is administered each year, and the

other four subtests are administered in alternating years. The math subtest was the second section

administered in 2014 and the reading subtest was given in 2015.


Associate of Arts (AA) degree program review was presented to the Board in fall 2014.

In fall 2015, program reviews the Developmental Education and Dual Credit programs were

presented to the Board.

The General Education Assessment Panel met regularly in 2013-2014 and 2014-2015 to evaluate the

assessment of student outcomes within courses. An assessment cycle aligns courses and outcomes

and ensures that all outcomes are assessed on a regular basis.

Beginning in spring 2014, each discipline began developing responses together (rather than

independently) to the first four questions on the “Closing the Loop” document. Department

Coordinators shared the documents with all faculty members (full- and part-time) beginning in fall

2014 to guide faculty in fine-tuning their teaching. Part-time faculty began completing the remaining

two questions of the “Closing the Loop” document in fall 2014 and submitting it to the Dean of

Academic Affairs.

An Adjunct Faculty Pre-Observation Form was developed in spring 2015 to be piloted in fall 2015.

The form asks faculty to articulate classroom objectives and methods of assessment in advance of a

classroom observation.

Assessment of Career and Technical Education Programs

WorkKeys was given in Career and Technical Education (CTE) graduates in the summer, fall, and

spring.

CTE faculty groups met to redevelop program outcomes and revise program assessment plans.

A part-time CTE faculty observation cycle was implemented in fall 2014. All new part-time CTE

faculty are now observed during their first semester of teaching by lead instructors in the discipline.

Returning part-time CTE faculty are observed every two years.

Program reviews were conducted for Occupational Therapy Assistant (OTA), Welding and Metals

Technology, and the Law Enforcement Training Center (LETC) and presented to the Board in fall

2014.

In fall 2015, program reviews for the Graphic Arts and Associate Degree in Nursing (ADN) programs

were presented to the Board.

Assessment of Support Services

The annual Noel-Levitz Student Satisfaction Inventory and Noel-Levitz Priorities Survey for Online

Learners were distributed in 2014 and 2015. In anticipation of the HLC Multi-Location Evaluation

Visit in June 2015, the College purchased site-specific results. The results were shared in President’s

Council and shared with site directors.

The Assessment of Support Services Committee reviewed and revised the student and employee

surveys during 2014-2015.

The annual Student Assessment of Support Services Survey was administered during the spring of

2014 and 2015. Almost 500 responses were collected in 2015. Results were shared with President’s

Council and distributed to appropriate offices.

The annual Faculty/Staff Assessment of Support Services Survey was administered in the spring of

2014 and 2015. Over 100 responses were received in 2015. Results were shared with President’s

Council and distributed to appropriate offices.


Dual credit program follow-up surveys were administered (i.e., 1-year and 4-year alumni, faculty,

principals, and guidance counselors). Feedback was used to make changes in services and processes.

Other Assessment Activities

MACC applied for and was accepted into the state’s Complete College America initiative in 2013.

Since then, the College has implemented several completion strategies (e.g., multiple measures of

placement, academic maps, redesigned developmental courses, accelerated developmental

education routes).

In spring 2015, a new requirement was established that degree- and certificate seeking students

required to complete developmental coursework must enroll in the required developmental

course(s) prior to or immediately following their first six hours of enrollment at MACC.

The LOGOS Project, a faculty-initiated committee focused on improving critical thinking, reading, and

writing skills of students, piloted a One-Read in spring 2015. Faculty representing fourteen

disciplines integrated the use of the Immortal Life of Henrietta Lacks into their classes. The LOGOS

Project identified the development of a critical thinking rubric as their next initiative.

Administrative evaluations were conducted in the fall in 2014 and 2015. Half of the members of

President’s Council are evaluated each year.

“Pizza with the President” sessions were held at each location during 2014-2015 to provide

opportunities for open dialogue between the President and students, faculty, and staff.

BUDGET SUMMARY 2014-2015 The College’s assessment budget and a portion of the budget for the Office of Institutional Effectiveness and

Planning support assessment activities that are not already embedded within other departments’ budgets

and expenses. Faculty and staff attend assessment-related conferences, but these expenses are generally

deducted from individual travel budgets rather than the assessment budget. The assessment budget for

2014-2015 was $25,000. Below is a summary of expenditures supported by the College’s assessment budget

for the fiscal year 2015.

Scantron $ 2,641.71 External Data (CAAP & ACT/Compass Linkage Reports $ 864.00 National Community College Benchmarking Project $ 1,250.00 SurveyMonkey $ 378.00 Jenzabar, Inc. for Congnos $ 3,609.75 Meetings $ 14.82 Printing $ 92.66 Travel Costs for Faculty Observations $ 1,480.58 Total $10,331.52

Additional assessment activities/expenses were supported through the budget for the Office of Institutional Effectiveness and Planning. In 2014-1015 these expenses included the following:

Professional Development $ 800.00 Assessment meetings $ 87.75 Conferences/Travel $ 134.16 Copies/Supplies $ 215.11 Total $ 1,237.02


The assessment budget has been adjusted in recent years to reflect additional expenses incurred by the

College as it continues to place more emphasis on assessment and to respond to significantly increased

reporting burdens at the state and federal levels. The assessment budget will continue to be closely

monitored and adjustments in funding made as needed.

ENTRY-LEVEL ASSESSMENT OF STUDENT LEARNING As an open admission institution, MACC often finds students enrolling at the College without the necessary

skills to succeed in college-level courses. Through appropriate and effective placement, MACC students are

given the opportunity to enhance their skills and to be challenged at a level equitable to their current ability.

Placement Testing

MACC uses the Compass and ACT exams to place new students into English and mathematics classes. These

tests assess language usage, math, algebra, and reading skills. Placement assessments enable MACC to place

students appropriately and are required of all students who

Are certificate- or degree-seeking, or

Wish to enroll in any math or composition/technical writing course.

Approximately two-thirds of MACC students who take a placement exam are required to take one or more

developmental courses. Figure 1 shows the total enrollment in developmental courses for the past five years

and the percentage of College’s overall total enrollment.

Figure 1: Enrollment in Developmental Courses (Fall 2010-Fall 2014) Fall 2010 Fall 2011 Fall 2012 Fall 2013 Fall 2014 Enrollment in Developmental Courses

1,192

1,241

1,261

1,210

1,049

Percentage of Total Enrollment

22% 22% 22% 21% 19%

As the above table reflects, there has been a slight decrease in the percentage of total students who enroll in

developmental courses. Because a delay in students completing their required developmental coursework

means that students were not gaining the skills they need to be successful in other courses, and, in some

cases, delaying their time to completion, Curriculum Committee approved a new requirement in spring 2015

that will ensure that more students will complete the required developmental coursework early in their

college career. The new requirement stipulates that degree- and certificate-seeking students required to

complete developmental coursework must enroll in the required developmental courses prior to or

immediately following their first six credit hours of enrollment at MACC. The requirement went into effect in

summer 2015.

In fall 2009, the Office of Academic Affairs in conjunction with MACC English faculty developed new cutoff

score levels for placement in developmental English courses. The new cutoff scores are more consistent with

current ACT/Compass guidelines and will increase student success rates by ensuring that students are better


prepared for the rigors of college work. All students taking a placement test after September 1, 2009, were

evaluated and placed using the updated scoring guidelines.

In fall 2010, the Missouri Community College Association Presidents and Chancellors group charged

community college Chief Academic Officers to determine a set of common placement scores to better

streamline student articulation and to allow for easier data comparison at the state level. These revised

scores were adopted by the Presidents and Chancellors group in spring 2011 and were fully implemented by

all Missouri community colleges in fall 2012.

In spring 2014, MACC began implementing multiple measures to more accurately place students into

coursework. Developmental English faculty created writing prompts for a Writing Appeal. The appeal

progress gives students who believe that they have the necessary writing skills to take the college-level

Composition I course another opportunity to demonstrate their skills. In fall 2014, a multiple measures pilot

was also developed to place students into College Algebra if they scored into a range below the cut core but

met a combination of other measures (e.g., high school GPA). During 2014-2015, developmental math

faculty created Math Exit Competency Assessments (MECA tests) as an additional option beyond the

Compass/ACT exams.

In spring 2015, Missouri Department of Higher Education adopted statewide placement scores for college-

level English and math courses. MACC’s reading and writing scores were already in alignment with the

scores set by the state, but the new cut scores for math allowed the College to lower the math ACT score by 1

point for each math class in the sequence. Later in the spring, ACT notified institutions of its plans to phase

out all Compass products (old version and newer web-based version) by the end of 2016. MACC began

reviewing options for a Compass replacement (e.g., Accuplacer).

Student Advisement Students are provided with advisement that focuses on academic achievement upon entry to MACC and

throughout their educational experience. The following advising tools and interventions have been put into

place to ensure consistent and accurate advising:

An advising checklist that is updated each semester

A nursing faculty advising handbook

Degree audit forms posted on the website for all programs

Purchased Academic Planner software from Jenzabar to allow students to register for courses online

after a plan has been approved by an advisor

Development of Academic Maps which serve as course sequence guides for students

Use of RegisterBlast to allow students to make advising appointments through an online scheduling

system

In spring 2015, non-degree seeking students were given access to enroll themselves in courses through the

online portal without being required to meet with an advisor.


Class Profile Report

ACT’s Class Profile Service Report provides a comprehensive summary of the College’s entering freshman

class and a parallel description of ACT-tested students who were admitted but did not enroll. The fall 2014

Class Profile Report included 711 ACT-tested students who enrolled at MACC full-time. In all, 1,309 students

who took the ACT requested that their scores be sent to MACC. Figure 2 below outlines descriptors of the

students who ultimately enrolled at the College in fall 2014.

Figure 2: ACT Fall 2014 Class Profile Report

Students Who Enrolled at MACC Fall 2014

Average ACT Score of MACC incoming full-time freshman 19.8

Most popular planned educational major Health Science & Technologies

Percent of students completing core high school curriculum 65%

Percent of students ranking in top quarter of high school class 30%

Percent reporting minority racial/ethnic background 14%

Percent planning to apply for financial aid 81%

Percent planning to work while attending classes 80%

Percent needing math skills 38% (self-reported)

Percent needing study skills 47% (self-reported)

Percent needing assistance with educational/occupational plans 47% (self-reported)

Percent needing reading skills 22% (self-reported)

Percent needing assistance expressing ideas in writing 19% (self-reported)

Figures 3-6 illustrate a five-year trend in the characteristics of MACC’s ACT-tested freshmen.

Figure 3: Student Reported Deficiencies (2010-2014) Figure 4: College Preparation-Core Curriculum & Top Quarter of Class (2010-2014)

Figure 5: Financial Need--Plan to Apply for Figure 6: Minority Racial/Ethnic Background for Financial Aid and to Work (2010-2014) (2010-2014)

0

10

20

30

40

50

2010 2011 2012 2013 2014

Math

Study

Reading

010203040506070

2010 2011 2012 2013 2014

CoreCurr

TopQtr

0

20

40

60

80

100

2010 2011 2012 2013 2014

FinAid

Work

0

5

10

15

2010 2011 2012 2013 2014


As Figure 7 below indicates, the national average ACT score for all high school graduates who took the exam has stayed relatively consistent. Figure 7: National Average ACT Score for All High School Graduates

2009-2010 2010-2011 2011-2012 2012-1013 2013-2014

21.0

21.1

21.1

20.9

21.0

Figure 8 illustrates by year the average ACT score of MACC students who submit scores through the ACT Class

Profile Service and the national average ACT score of freshmen enrolling in two-year colleges. The national

data is one year behind the institution-specific data.

Figure 8: Average ACT Composite Score for MACC Students versus National Average

Student Profile

Each fall, a Student Profile Report is distributed to administration for their review and posted in the Resource

Library that is accessible to all employees. The ethnic/racial composition of MACC students tends to be

rather homogeneous, although the percentage of minority students has increased over the past several

years. (Between 2006 and 2014, the total percentage of White students decreased by 7%). According to fall

2014 census enrollment data, 82% of MACC students are White, 8% are African-American, 3% are Multi-Race,

3% are Hispanic, 2% are Asian, 2% are Unknown/Other, and less than 1% are Native Americans and Non-

Resident Alien.

Figure 9: Fall 2014 Students by Classification

18.4

18.9

19.4

19.9

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

AVERAGE ACT COMPOSITE SCORE OF STUDENTS ENROLLED AT MACC

AVERAGE NATIONAL ACT COMPOSITE SCORE OF TWO-YEAR COLLEGES

First-Time Freshman

24%

Freshman 23%

Sophomore 28%

Non-Degree Seeking

5%

Special With Degree

1%

Dual Credit/Enrolled

16%

Visitor 3%

As Figure 9 shows, in fall 2014, 47% of MACC students had accumulated less than thirty hours. MACC’s dual-credit and dual-enrolled students made up 16% the student population.


The distribution of full-time and part-time students has remained relatively level over the past decade. In fall

2005, 53% of MACC students attended part-time. Figure 10 below shows that there was just a 1% increase in

part-time enrollment by fall 2014. Three-fifths (60%) of MACC students are female (Figure 11).

Figure 10: Students by Enrollment Status (Fall 2014) Figure 11: Students by Gender (Fall 2014)

The average age of all fall 2014 students was 22.9. Without dual-credit students, who are usually in the 18

and under age range, the average age was 24. In fall 2014, the largest student categories were those

students aged 24 -18, as shown below in Figure 12. A little less than one-fourth (24%) of MACC’s students

are non-traditional students (aged 25 and above). The average age of MACC college-level students (excluding

dual-credit) steadily decreased from 2000-2005, then began increasing slightly as more non-traditional

students enrolled in classes due to negative economic conditions, and then began decreasing again as the

economy improved. This trend is illustrated in Figure 13.

Figure 12: Students by Age (Fall 2014) Figure 13: Average Age Trend 1999-2014

Under 18 9%

18-19 30%

20-24 31%

25-29 11%

30-39 11%

40-49 6%

50-64 2%

Over 65/Unknown

<1%

202122232425262728

Age (all)Age (less dual credit)

Full-Time 46%

Part-Time 54%

Male 40% Female

60%


The most popular areas of study in fall 2014 were Liberal/General Studies (1,741 students), Business and Management (822 students), Education (298 students), Psychology (160 students), and Nursing (157).

MID-LEVEL ASSESSMENT OF STUDENT LEARNING Formative evaluation plays an important role at MACC in determining ways to improve the teaching/learning process. Mid-level assessment of student learning occurs at the course level through course grades, individual classroom assessments, final examinations, pre- and post-testing, retention studies, and student evaluations of teaching.

Course Grades Faculty issue final course grades at the end of each semester. These grades are used to indicate student mastery of the objectives of the course and are used to determine grade points students receive for individual courses that are then used in computation of cumulative grade point averages. Faculty are expected to evaluate students frequently enough to get a sound measure of the quality of their work and to give them ample opportunity to improve their grades. Grades are issued at the close of each semester on a letter basis indicating quality of academic work as follows: A Superior Work: excellence in originality of work; accuracy and grasp of content; independence in

completing work; and literate expression of ideas, imagination, and initiative

B Above Average Work: high achievement in knowledge and grasp of work, effective use of knowledge, independence of work, and competence in expression

C Average Work: average attainment in familiarity with course content, participation in class work, evidence of intellectual growth and thinking, and intelligible expression of ideas

D Below Average Work: meager achievement in both quantity and quality of work, difficulty in understanding principles, slow progress, and instructor direction needed to meet requirements

F Failing Work: failure to achieve the minimum requirements for passing, inaccurate results, faulty reasoning, irresponsible, vague, disorganized, and lack of initiative

I Incomplete Work: unable to complete the required work by the close of the semester; approved only when student has maintained a passing grade prior to the incompletion. No incompletes will be approved for students who have not completed work through three-fourths of a session; all incomplete grades must be completed within the next semester of the regular academic year.

W Withdraw

P Pass: Pass/Fail; the grading policy for some courses Letter grades earned in developmental courses are by designation only and are not computed when figuring

grade point average.

In summer 2008, the grading scale policy was amended to eliminate the Withdraw – Satisfactory (WS) and

Withdraw – Unsatisfactory (WU) options. The single designation of Withdraw (W) for students who drop a

course is preferred by faculty and is consistent with the practices of other postsecondary institutions.


Academic Probation Prior to fall 2010, students who attempted six or more credit hours and fell below a 1.60 grade point average

were placed on academic probation for one semester. Effective fall 2010, the GPA cutoff for academic

probation was raised to 1.75. This change was suggested by MACC’s Retention Committee and allows

administration to identify a greater number of at-risk students and begin intervention measures earlier to

increase student success rates.

Students not showing improvement during a probationary semester may request permission to continue

from the Dean of Academic Affairs or the Dean of Career and Technical Education, as determined by the

student’s major. No student placed on academic probation will be allowed to carry over sixteen hours.

Tutoring and mentoring services are available for students who need help in achieving academic success.

Classroom Assessments Assessment involves both instructors and students in the continuous monitoring of student learning.

Individual classroom assessments are conducted by faculty through various means at the course level. These

assessments provide faculty with useful information on what, how much, and how well students are learning

throughout the semester. Instructors are encouraged to allow for daily or weekly input from students

concerning their learning progress. Weekly participation points for learning progress reports, one-minute

assessments, concept maps, and journaling are some examples of the ways MACC instructors have

encouraged feedback from students. Course portfolios and student exhibits and demonstrations are being

utilized in several classes to provide instructors with an additional assessment of student progress.

Comprehensive Final Examinations Students are given comprehensive course finals at the end of each semester. The course final enables faculty

to assess how students have progressed toward meeting course goals.

Developmental Education MACC’s commitment to open admission is made possible by a strong developmental education program.

The College has established the following developmental courses which concentrate on student writing skills,

reading skills, math skills, and algebra skills:

Language & Literature

LAL020-Transition to Composition I

LAL022-Fundamentals of Writing

LAL023/024-Tech Enhanced Fundamentals of Writing I & II

LAL031-Fundamentals of Critical Reading

LAL032-Reading in Content Area

LAL090-Composition I Lab

Mathematics

MTH010-Fundamentals of Math

MTH011-Fundamentals of Algebra

MTH015-Developmental Technical Math

MTH016-018-Computer Assisted Pre-Algebra I, II, II

MTH019-Computer Assisted Pre-Algebra Supplement

MTH090-Support for College Algebra


All students scoring into developmental courses are given a pre- and post-test to determine progress in the

course. Student success is also assessed through journals and surveys. Results of the pre- and post-tests and

surveys are reviewed by administration, faculty, and staff to develop recommendations for curricular

improvement.

In 2011, MACC math faculty reviewed and redesigned the College’s developmental math coursework based

on the model provided by the National Center for Academic Transformation. Studies have shown that this

model, which allows students greater freedom to complete developmental coursework at their own pace,

greatly increases student success rates. The newly redesigned math courses were offered in fall 2011. The

College’s developmental English course was redesigned during the 2011-2012 academic year.

In spring 2012, three faculty members won the College’s Innovation Award for their work in redesigning the

College’s developmental math and English courses. The data that has been collected to determine student

success rates in the redesigned courses as compared to the College’s traditionally structured courses shows

encouraging results. In fall 2012, the first sections of MACC’s redesigned developmental English course were

offered to students.

As discussed previously, during 2013-2014 MACC’s Completion Team and English and math faculty explored

the development of completion strategies which included multiple measures and alternate pathways. Figure

14 outlines the developmental education initiatives that were implemented in 2013-2014 and 2014-2015.

Figure 14: Developmental Education Completion Initiatives

Year Initiative Implemented Description Results

2013-2014 Writing Appeal Students can opt for a one-time writing appeal to demonstrate the writing skills necessary for Composition I. The appeal is a proctored writing assignment. English faculty score the writing responses using a rubric. If the writing appeal is granted, students test out the developmental writing course.

58 students have completed writing appeals

38 (60%) have passed the appeal 14 of the 35 (40%) successfully

completed LAL101-Comp I

2014-2015

Math Multiple Measure Alternative (College Algebra Pilot)

Students without the necessary placement score for College Algebra can enroll in the course if they meet other criteria (e.g., recent graduation from high school, GPA, and ACT/Compass score).

73 students participated

75% passed with a C or higher (compared to 64% of all students in course)

Compass Prep Workshop Students can enroll in a free workshop that provides test-taking tips and strategies for the Compass Reading and Writing tests. After they complete the workshop, students are given the opportunity to retake the Compass at no cost.

25 students participated Avg. gain of 8.5 points on reading Avg. gain of 28 points on writing 11 tested out of LAL031

7 tested out of LAL022 8 enrolled in LAL101 (4 passed and

4 withdrew) Composition I Co-Req Class (LAL090)

In addition to enrolling in Composition I, eligible students participate in a co-requisite lab where foundational reading and writing skills are supported.

14 students participated

100% pass rate in LAL090 & LAL101

Math Exit Competency Assessments (MECAs)

MECAs are internally-developed competency exams that give students another option beyond the Compass and ACT to determine which math course they can take.

TBD (MECAs were finalized in spring 2015 for use in summer 2015)

The results of these initiatives are being carefully monitored.


Course Evaluations The primary purpose of course evaluations by students is to improve teaching and learning. These

evaluations provide feedback to instructors and administrators concerning student perceptions of the

effectiveness of instructional methods and procedures used in the course. The course evaluation form was

revised during 2003-2004, and a new form was implemented in fall 2004. The form collects demographic

information on each student, so that evaluation results can be interpreted more meaningfully.

All full-time and part-time instructors administer course evaluations each semester. Typically, a student

representative distributes the survey in class, collects the responses, and returns them to the appropriate

dean’s office. Survey responses are tabulated and a summary report is prepared for each course section.

Faculty members receive a typed list of the written comments and the summary form. The deans monitor

survey results to detect any potential issues that should be addressed. Results are also included in the faculty

member’s teaching portfolio.

Assessment of Support Services Surveys

Student Survey The Assessment of Support Services Committee conducts a comprehensive annual Student Assessment of

Support Services Survey. The survey is delivered completely online. Students access the survey through a link

sent to student email accounts, posted in myMACC, within eCollege, and from a banner announcement on the

website.

In spring 2015, 479 students completed the surveyed at Moberly (123), Mexico (32), Columbia (192), Kirksville

(35), and Hannibal (31), as well as (66) online students. The following departments were evaluated: Activity

Center, Area 27, Bookstore, Business Office, Career and Placement Services/New Traditions, Computer

Services, Financial Aid, Instructional Technology, Learning Center/Resource Centers, Library, Plant Operations,

Student Services and Vending Services.

Survey items cover professionalism and knowledge of staff, convenience and timeliness of services, and

availability and usefulness of information and resources among other topics that are specific to certain areas

(e.g., eBooks and electronic course resources). Students were also asked to give an overall satisfaction rating to

each department, which ranged from a low of 3.03 to a high of 3.51 on a 4-point scale. Three questions were

added to the survey this year. One question asked students whether they would recommend MACC to friends

and family (94% responded that they would). The two other new questions were open-ended and asked

students what they like most about MACC and what they would change to make MACC better. Statistical

results and student comments were distributed to area supervisors for use in a variety of quality improvement

efforts. Survey results were also discussed during President’s Council Retreat in July. Figure 15 on the next

page shows student satisfaction survey trends from 2008 to 2015.


Student Retention MACC defines retention as the return of full-time, certificate/degree-seeking freshmen from fall to fall as

measured on census date. Students are considered retained even if they attend on a part-time basis. ACT

reports the national retention rate for public two-year community colleges with open admission policies to

be 55% for fall 2013 to fall 2014. MACC’s retention rate for the same period was just slightly higher at 56%.

Figure 16 below shows a five-year trend of MACC’s retention as compared to the national retention rate for

two-year colleges with open admission. MACC’s retention rate has improved considerably since 2009.

Figure 16: Fall-to-Fall Retention Rates of First-time, Full-time Degree-Certificate Seeking Students

(2009-2010 to 2013-2014)

0

0.5

1

1.5

2

2.5

3

3.5

4

Low

High

0%

10%

20%

30%

40%

50%

60%

2009-10 2010-11 2011-12 2012-13 2013-14

MACC

National

Figure 15: Student Satisfaction Survey Trends (2008-2015)


The College also tracks fall-to-fall retention of all full- and part-time students (excluding high school

students), which gives a much broader perspective than the first-time, full-time cohort addressed above.

Figure 17 shows MACC’s retention rate of all students compared to the national 50th percentile, as reported

by National Community College Benchmarking Project. Although there has been a downward trend, the

biggest spread occurred in the FA13 to FA14 rate, when the College also experienced a 5% enrollment

decline.

Figure 17: Fall-to-Fall Retention Rates of Total Full- and Part-Time Credit Students

(Fall 2010-Fall 2011 to Fall 2013-Fall 2014)

Evaluating next-term retention (fall to spring) trends helps the College determine not just the most

appropriate type of retention strategies to implement, but also the best timing for them. Because a

downward trend is also occurring with next-term retention, as Figure 18 indicates, MACC has been discussing

more effective ways to specifically support student success during the fall term.

Figure 18: Next-Term Retention Rates of Total Full- and Part-time Credit Students

(Fall 2010-Spring 2011 to Fall 2013 to Spring 2014)

25%

30%

35%

40%

45%

50%

55%

FA10-FA11 FA11-FA12 FA12-FA13 FA13-FA14

MACC

National 50th Percentile

50%

55%

60%

65%

70%

75%

FA10-SP11 FA11-SP12 FA12-SP13 FA13-SP14

MACC

National 50th Percentile


MACC is committed to student retention and to providing the support students need to achieve their goals.

To this end, the College has implemented a combination of retention initiatives in recent years which include

the following:

Early-alert programs (Students Monitored by Academic Retention Teams (SMART) and Retention

Manager)

Intrusive advising

Academic maps (degree program sequence/guide)

Academic Planner (establishes academic plan and allows online enrollment)

SmartThinking online tutoring service

Expanded Library and Academic Resource Center (LARC) services and hours

Attendance policy

Requirement to complete developmental hours prior to or following first six credit hours

Default prevention programs (supported by Default Prevention Grant)

Student Assistance Program

Adjustment to Allied Health admission requirements (required GPA from 2.5 to 2.7)

Multiple measures of placement (Writing Appeal, MECAs, College Algebra math pilot)

Redesigned Math and English Courses (modular and mastery based)

Co-requisite courses

Blended developmental reading and writing instruction

Boot camps

Compass Prep Workshops

Establishment of Department Coordinators (faculty leaders)

Increased role of Lead Instructors (for courses and to observe part-time faculty)

Professional development workshops focused on retention

Setting measurable retention goals in 2015-2018 Strategic Plan

The College monitors the effectiveness of individual measures. Retention has been and will continue to be an institutional priority.

EXIT-LEVEL ASSESSMENT OF STUDENT LEARNING The ability to measure student achievement contributes to improving programs and services. Assessing

outcomes will also help the College respond to accountability requirements from its constituents and other

external agencies.

CAAP Test The CAAP (Collegiate Assessment of Academic Proficiency) test is designed to measure general education

competencies typically attained in the first two years of college. MACC used to require all graduates to take

the CAAP exam; thus, a significant population of students, such as certificate and AAS completers, was being

tested for content which was not the focus of their programs. Thus, MACC graduates typically scored below

the national mean on most subtests.


Beginning in spring 2004, subtests of the CAAP exam were embedded into related general education courses.

The Critical Thinking subtest is administered every year, while the other four subtests – Math, Writing,

Reading, and Science – are used alternately. This change in the administration of the exam more accurately

assessed general education competencies and provided more useful data. Figures 19-23 present the CAAP

performance of MACC students as compared to students nationally.

As shown in Figure 19, student performance initially improved after the test was embedded in courses that

encourage critical thinking in 2004. Since 2004, MACC scores have been very close to the national scores

(within 1%).

As Figure 20 indicates, MACC students have consistently scored very close to the national norm in the writing

section of the CAAP. This test is generally administered in Composition I and II classes.

61.1 61.1 61 60.7

60.5 60.5 60.7

60.9 60.9 60.8 60.7 60.7 60.6 60.8

60.6

59.2 59.3

58.8

59.4

61 60.9

61.4

60.7 60.3 60.3

61

60.3 59.9

60.1

60.7

58

58.5

59

59.5

60

60.5

61

61.5

62

62.5

63

200

0 (

22

7)

200

1 (

29

2)

200

2 (

30

8)

200

3 (

32

7)

200

4*

(310

)

200

5 (

31

3)

200

6 (

30

9)

200

7 (

25

9)

200

8 (

26

3)

200

9 (

29

1)

201

0 (

25

5)

201

1 (

32

3)

201

3 (

27

8)

201

4 (

33

1)

201

5 (

31

3)

National MACC

62.7 62.7

62.7

62.6

62.6 62.6

62.5 62.5

62.1

61.5

64.1

62.3 62.9

62.6

62.2 62.4 63.2

62.6 61.7 61.5

61

62

63

64

65

66

199

7 (

19

2)

199

8 (

23

2)

199

9 (

16

8)

200

0 (

22

7)

200

1 (

29

2)

200

2 (

30

8)

200

3 (

32

7)

200

4*

(345

)

200

8 (

28

0)

201

3 (

32

2)

National MACC

* Indicates the first year the subtest was embedded. The Writing and Critical Thinking Subtests were given that year.

Figure 19: CAAP Critical Thinking Scores (2000-2015)

Figure 20: CAAP Writing Scores (1997-2013)


As illustrated in Figure 21, since the CAAP math subtest was embedded in math-only courses, MACC students

have scored above the national average, with the largest difference (2.4%) occurring in the most recent

administration in 2014.

MACC students scored below the national average on the past two administrations (2010 and 2015) of the

CAAP reading test (Figure 22). The reading test is administered in classes that have a focus on reading

comprehension skills. However, it should be noted that the only course that requires a prerequisite reading

score is Composition I. Faculty members have expressed concern about students’ reading skills.

56.1 56.2 56.1 56.2 56.3 56.3 56

56.1 56.2 56

55.7 55.5 55.5

54.8 55

55.7 55.6

58 57.9 58.4

54

55

56

57

58

59

199

7 (

19

2)

199

8 (

23

2)

199

9 (

16

8)

200

0 (

22

7)

200

1 (

29

2)

200

2 (

30

8)

200

3 (

32

7)

200

5 (

25

3)

200

9 (

30

1)

201

4 (

34

4)

National MACC

61.1 61.3

61.1

61

61 61.1

60.9 60.4 60.4

60.1

61.4

59.6

61.5

59.6 58.6

59.9 60.1 60.3

59.2

58.7

58

59

60

61

62

63

19

97

(19

2)

19

98

(23

2)

19

99

(16

8)

20

00

(22

7)

20

01

(29

2)

20

02

(30

8)

20

03

(32

7)

20

06

(28

5)

20

10

(26

7)

20

15

(29

6)

National MACC

Figure 21: CAAP Math Scores (1997-2014)

Figure 22: CAAP Reading Scores (1997-2015)


As illustrated in Figure 23, MACC student scores siginificantly improved their performance on the science test

beginning in 2002 when the order the subtests was changed and the science test was given first. In the last

two admnistrations of the test, MACC students scored within 0.5% of the national norm.

Capstone Courses MACC has added capstone courses to many of the Associate of Applied Science degree programs, including

Business Accounting Technology, Architectural & Mechanical Design, Business & Office Administration,

Computer Information Technology, Early Childhood Education, and Graphic Arts Technology.

Each capstone course requires a culminating project in which the student demonstrates mastery of essential

skills within the program. Results of each capstone course are used in the program’s assessment plan and

are reviewed by faculty, division chairs, the Dean of Career and Technical Education, the program advisory

committee, and various other college-wide assessment committees.

Essential Skills Each Career and Technical Education Program has its own assessment plan, and one portion of that plan is

mastery of Essential Skills. The goal for this assessment component is that 80% of AAS program completers

achieve 80% mastery of all Essential Skills in that program area. Previously, under the Federal Perkins III

legislation, results of this assessment were reported annually to the Missouri Department of Elementary and

Secondary Education (DESE). Since the implementation of Perkins IV in 2006, this data is no longer reported

to the state but is still used for internal program assessment. Results are used in each program’s assessment

plan and are reviewed by faculty, division chairs, the Dean of Career and Technical Education, each program’s

advisory committee, and the Assessment Advisory Committee. Figure 24 reflects Essential Skill attainment of

2014-2015 Associate of Applied Science graduates.

58.8 58.9 58.8

59 58.9 59 58.9

59 59.1

59.1 58.9 58.8

56.9

57.1

55.2 55.7

59.5

59.6

58.9

58.9

55

56

57

58

59

60

61

19

96

(18

7)

19

97

(19

2)

19

98

(23

2)

19

99

(16

8)

20

00

(22

7)

2001

(2

91)

20

02

(30

8)

20

03

(32

7)

20

07

(27

9)

20

11

(31

4)

National MACC

Figure 23: CAAP Science Scores (1996-2011)


Figure 24: Essential Skills Attainment (2014-2015 AAS Graduates)

Essential Skills Attainment

Program Number of Graduates

Assessed Percent Meeting Essential

Skills Business Accounting Technology 1 0%

Business Administration-Accounting 1 100%

Business Administration-Marketing 1 100%

Business Office Administration 4 100%

Business and Office Technology-Executive 1 100%

Business and Office Technology-Medical 4 75%

Computer Information Technology-Networking 9 100%

Computer Information Technology-Programming 3 100%

Drafting Design Technology 4 100%

Early Childhood Education 10 90%

Graphic Arts Technology 10 100%

Industrial Technology-Machine Tool 1 100%

Industrial Technology-Maintenance 5 100%

Industrial Technology-Mechatronics NA NA

Marketing/Management 1 100%

Medical Laboratory Technician 9 100%

Nursing – Associate Degree 32 100%

Nursing- Accelerated ADN 25 100%

Nursing – Practical (Moberly) 26 100%

Licensure, Certification, and Exit Exams

Additional measures of student outcomes in place at MACC include licensure, certification, and exit

examinations. Currently, completers of five career/technical programs take either national or state

competency examinations. Associate Degree of Nursing students take the NCLEX-RN exam and Practical

Nursing students take the NCLEX-PN. Business and Office Technology completers take the Office Proficiency

Assessment and Certification (OPAC) exam. Occupational Therapy Assistant completers take the National

Board for Certification in Occupational Therapy (NCBOT) exam. Completers of the Law Enforcement Training

Certificate take the Missouri State Certification Exam for Basic Peace Officers.

Results of these exams are reported in each program’s assessment plan, reviewed by faculty and program

advisory committees, and are used in curriculum revision. Results of these outcome exams are cited in Figure

25 for the 2014-2015 academic year.

Figure 25: Licensure, Certification, Exit Exams (2014-2015) Program Outcomes Assessment Number Tested Pass Rate

Associate Degree in Nursing (Fall 2014 and Spring 2015 Grads)

NCLEX-RN Exam 57 96%

Certificate in Practical Nursing (Summer 2014 Grads)

NCLEX-PN Exam 49 90%

Business and Office Technology OPAC Exam 9 89%

Law Enforcement Basic Peace Officers (POST)

Exam 20 100%

Occupational Therapy Assistant NBCOT Exam 12

100%


Degrees Conferred The number of degrees and certificates conferred is one indicator of successful student academic

achievement and goal attainment. Figure 26 lists the number of graduates by program over the last five

years (2010-2011 to 2014-2015).

Figure 26: Degrees Conferred (2010-2011 to 2014-2015)

2010-2011 2011-2012 2012-2013 2013-2014 2014-2015

Associate Degrees 627 681 659 750 711

Associate of Arts 467 507 481 549 543

Associate of Arts in Teaching 27 34 41 62 24

Associate of Science 9 8 5 8 10

Associate Degree in Nursing 19 24 36 36 32

Accelerated Associate Degree Nursing 9 8 10 14 25

Associate of Applied Science 96 100 86 81 77

Business Accounting Technology 9 7 7 10 1

Business Administration, Accounting 0 0 0 0 1

Business Administration, Marketing 0 0 0 0 1

Business Office Administration 0 0 0 0 4

Business & Office Tech, Executive 6 1 8 3 1

Business & Office Tech, Legal 1 0 3 0 0

Business & Office Tech, Medical 6 19 10 5 4

Computer Information Systems 3 0 0 0 0

Computer Information Tech, Programming 5 5 2 6 3

Computer Information Tech, Networking 7 8 8 6 9

Drafting Design Technology, General 6 2 3 4 4

Early Childhood 4 8 8 11 10

Graphic Arts Technology 13 11 6 3 10

Ind. Electronics/Electricity Technology 2 1 0 0 0

Industrial Technology - Machine Tool 1 1 1 0 1

Industrial Technology – Industrial Maintenance 0 1 2 2 5

Industrial Technology - Mechatronics 0 0 0 0 1

Industrial Tech, Power Plant Operations 13 11 2 0 0

Marketing/Management Technology 2 4 3 10 1

Medical Laboratory Technician 5 7 5 5 9

Occupational Therapy Assistant 13 11 17 16 12

Welding & Metals Technology 0 3 1 0 0

Certificates 87 103 84 94 76

Accounting 0 0 1 0 0

Business & Office Administration 0 0 0 0 1

Business & Office Technology, Medical 1 1 0 0 1

Computer Information Tech – Networking 3 1 2 2 0

Early Childhood 0 0 0 1 1

Graphic Arts Technology 2 1 1 0 2

Industrial Technology 0 0 2 1 0

Law Enforcement 42 46 32 39 22

Marketing/Management 1 1 0 0 0

Practical Nursing 38 52 45 49 49

Welding & Metals Technology 0 0 1 0 0

Certificates of Specialization 0 1 0 5 3

Computer Support Specialist 0 0 0 1 0

Early Learning Assistant 0 1 0 2 1

Networking 0 0 0 0 1

Phlebotomy 0 0 0 0 1

Programming 0 0 0 1 0

Systems Operations 0 0 0 1 0


According to IPEDS provisional data on College Navigator, the statewide graduation rate for public

community colleges in Missouri is 21.6%. MACC’s graduation rate for the fall 2011 cohort was 22.9%. The

graduation rate reflects the number of students who completed their program of study within three years.

The rate includes full-time certificate-seeking or degree-seeking students who were not previously enrolled in

any institution. Figure 27 shows the total number of students (by race and gender) who enrolled at MACC in

fall 2011 and completed their program by August 2014.

Figure 27: Graduation Rate (Fall 2011 Cohort--Full-time Certificate or Degree-seeking Students) Year 2011

Non-Resident

Alien

Black or African-

American

American Indian or

Alaska Native

Hispanic White Asian Two or

More Races

Native Hawaiian or Other Pacific

Islander

Race/Ethnicity Unknown

Total Students

Initial Cohort Men Women

1 0

45 44

1 3

8 7

304 404

5 6

8 15

1 0

0 0

373 479

Total Students

1 89 4 15 708 11 23 1 0 852

Completers of Programs

Men Women

0 0

1 4

0 0

1 2

76 103

1 1

1 5

0 0

0 0

80 115

Total Students

0 5 0 3 179 2 6 0 0 195

FOLLOW-UP ASSESSMENT

180-Day Placement Survey

MACC’s Job Placement Office conducts an annual 180-day follow-up study – as required by DESE – for all

career program graduates each year. The basic intent is to report the employment status, continued

education patterns, and salaries of the graduates by program.

Figure 28 indicates the results of the follow-up study for 2013-2014 MACC Career and Technical Education

program graduates. As shown, 87% of MACC graduates were positively placed (ER, CER, and MR). MACC had

an overall placement rate (includes positive placement plus ENR and CENR) of 96%.

Figure 28: Results from 180-Day Follow-up Study for 2013-2014 CTE Graduates

Program Grads ER ENR CER CENR MR NP UNK NAP Avg. Wage Bus Accounting Tech 10 5 3 1 0 0 0 1 0 $9.00

Bus Office Tech 8 7 0 0 0 0 1 0 0 $9.87

Comp Info Tech 14 11 1 0 0 0 0 0 2 $11.20

Drafting Design Tech 4 3 0 0 0 0 0 0 1 NA

Early Childhood Ed 14 10 0 3 0 0 0 1 0 $9.58

Graphic Arts Tech 3 1 2 0 0 0 0 0 0 NA

Industrial Tech 3 1 1 0 0 0 1 0 0 NA

Law Enforcement 39 27 6 4 0 1 0 1 0 $12.28

Mkt/Mgmt 10 6 1 3 0 0 0 0 0 $9.71

Med Lab Tech 5 4 0 1 0 0 0 0 0 $14.75

Occup Therapy Asst. 16 14 1 0 0 0 1 0 0 $25.14

ADN 50 46 0 4 0 0 0 0 0 $19.54

PN – Mexico 25 14 4 6 0 0 0 1 0 $15.26

PN – Moberly 24 21 0 3 0 0 0 0 0 $14.85

TOTALS 225 170 19 25 0 1 3 4 3 $15.89

ER Employed, Related ENR Employed, Not Related NAP Not Available for Placement CER Continuing Education, Related CENR Continuing Education, Not Related MR Military, Related NP Not Placed/Unemployed UNK Situation Unknown NA Wage not shared


MACC’s graduation application includes a satisfaction survey section with questions that ask graduates to rate 1) the quality of their educational experience, 2) the quality of instruction, and 3) the quality of academic advising. Graduates are also invited to share general comments and are asked the open-ended question, “What would have improved your student experience at MACC?” Figure 29 illustrates 2014-2015 graduates’ average ratings by program on a scale of 1 to 6, with 1 being Very Satisfied and 6 being Very Dissatisfied.

Figure 29: Results from Graduation Satisfaction Survey by Program (2014-2015 Graduates)

PROGRAM Quality of

Educational Experience

Quality of Instruction

Quality of Academic Advising

Number of Respondents

Transfer Associate Degrees

Associate of Arts 1.81 1.92 1.99 432

Associate of Arts in Teaching 1.50 1.61 2.11 18

Associate of Science 2.13 2.50 3.25 8

CTE Associate Degrees

Associate Degree in Nursing 2.00 1.97 2.00 30

Accelerated Associate Degree in Nursing

2.00 1.82 2.00 22

Bus Accounting Tech 2.00 2.00 2.00 1

Business Administration-Accounting

1.00 1.00 1.00 1

Business Administration- Marketing

--- --- --- 0

Business Office Administration 1.25 1.25 1.00 4

Business & Office Tech, Exec --- --- --- 0

Business & Office Tech, Medical

1.67 1.67 1.67 3

Computer Information Technology, Programming

1.00 1.00 1.33 3

Computer Information Technology, Networking

1.50 1.50 2.25 8

Drafting Design Technology 2.00 1.75 1.50 4

Early Childhood 1.56 1.89 1.67 9

Graphic Arts Tech 2.78 2.33 2.67 9

Ind Tech, Ind Maint Tech 1.60 1.60 1.40 5

Ind Tech, Mechatronics 2.00 2.00 2.00 1

Ind Tech, Machine Tool 1.00 1.00 1.00 1

Marketing/Management 1.00 1.00 1.00 1

Medical Laboratory Technology (MLT)

1.22 1.33 1.22 9

Occupational Therapy Assistant (OTA)

2.17 2.33 2.67 12

CTE Certificates

Business Office Technology-Medical

2.00 2.00 2.00 1

Business Office Administration 3.00 4.00 2.00 1

Early Childhood --- --- --- 0

Graphic Arts Technology 2.00 1.00 1.50 2

Law Enforcement 1.61 1.67 1.72 18

Practical Nursing 2.35 2.35 2.25 20

CTE Certificates of Specialization* (Registrar awards certificates. Students do not apply for graduation for certificates of specialization.)

Early Learning Assistant --- --- --- ---

Networking --- --- --- ---

Phlebotomy --- --- --- ---


The survey results indicate that the vast majority of graduates were satisfied with their educational

experience, the quality of instruction, and the quality of advising at MACC. Results of the survey are shared

with President’s Council and will be included in each program’s assessment plan.

Student Transfer There are two sources of student transfer data available to MACC – the Statistical Summary published by the

Missouri Department of Higher Education (MDHE), which presents the collective data from all of the state’s

public colleges and universities, and the National Student Clearinghouse service, which is used by the

Financial Aid Office to track student financial aid records. MACC normally uses National Student

Clearinghouse data for all internal and external reporting and comparison purposes as National Student

Clearinghouse captures transfer data on students who transfer to public and private schools across the US

whereas MDHE only captures transfer data on students who transfer to public institutions within the state of

Missouri.

Over 80% of MACC students intend to transfer. According to the National Student Clearinghouse data, in fall

2015 MACC’s top two transfer institutions accounted for 50% of the 192 student transfers to four-year

schools from the fall 2013 first-time full-time freshman cohort: University of Missouri-Columbia (70) and

Columbia College (26). With 34% of the students transferring to other four-year institutions in the state of

Missouri, just 16% transferred to out-of-state schools. Fifty-two students transferred to other two-year

institutions, with Ozarks Technical College (6), State Fair Community College (5), St. Charles Community

College (5), and St. Louis Community College (5) being the most popular transfer schools.

MACC encourages student transfer by having the following elements in place:

Having accredited programs

Developing and updating articulation agreements with area schools

Participating in Missouri Transfer Course Library

Developing new partnership agreements with 4-year institutions (e.g., Columbia College and Central

Methodist University will offer courses and programs at an MACC site beginning in spring 2016)

Inviting other institutions to present and provide one-on-one transfer counseling to MACC students

Evaluating the transferability of programs and courses in regular academic program reviews

Realigning programs to improve transfer (e.g., Associate Degree of Science in 2013)

Tracking student transfers

Setting measurable transfer goals in 2015-2018 Strategic Plan

OTHER ASSESSMENT EFFORTS

Program Reviews In summer 2013, program reviews were conducted for the following programs: Associate of Arts in Teaching

(AAT), Law Enforcement Training Center (LETC), Medial Laboratory Technology (MLT), and the

Entrepreneurship & Business Development Center (EBDC). Results of the program reviews were presented at

Board of Trustees work sessions in September and October 2013.


In summer 2014, program reviews were conducted for the Associate of Arts (AA), Occupational Therapy

Assistant (OTA), and Welding and Metals Technology (WLD). Results were shared during Board of Trustees

work sessions in September and October 2014.

Employee Satisfaction

The Assessment of Support Services Committee distributed its fourteenth Faculty/Staff Assessment of Support

Services Survey during the spring 2015 semester regarding the quality of support services across locations. The

survey was offered online. Employees could access the survey from any computer and complete it in one

session or multiple sessions as their schedules allowed.

In 2012, the Faculty/Staff Assessment of Support Services Survey was split into two parts to prevent survey

fatigue. Departments are surveyed every other year. Seventeen departments were evaluated in spring 2015

and included the following:

A total of 104 faculty and staff completed the Faculty/Staff Assessment of Support Services Survey. This

represented a 40% response rate.

Employees were asked to respond to four statements about each support area by choosing one of the following

options: 1-Strongly Disagree, 2-Disagree, 3-Agree, 4-Strongly Agree. The college-wide average and low-high

ranges for each statement are shown below in Figure 30 that includes four years of comparative data.

Figure 30: Results from Faculty/Staff Assessment of Support Services Survey (2012-2015)

#1 Staff are approachable and

friendly.

#2 Business is generally conducted in a

timely fashion.

#3 Current and accurate information is

provided.

#4 Overall, I am generally satisfied with the

service this department provides.

2012 2013 2014 2015 2012 2013 2014 2015 2012 2013 2014 2015 2012 2013 2014 2015

College-Wide

Average 3.40 3.44 3.43 3.47 3.26 3.40 3.35 3.40 3.28 3.43 3.34 3.44 3.27 3.40 3.35 3.43

Range Low - High

3.11-3.61

3.24-3.76

3.24-3.61

3.21-3.74

2.85-3.55

3.06-3.75

3.10-3.56

3.02-3.71

2.92-3.58

3.20-3.73

3.07-3.53

3.22-3.71

2.94-3.55

3.15-3.71

3.12-3.50

3.20-3.72

The supervisors of each office received the following survey data: individual office results, college-wide

results, an average score for each question, a low/high range for each question, and the typed responses to

two open-ended questions about their departments. Supervisors were encouraged to share the results

within their departments as appropriate and to use the results to develop goals for improvement.

1. Academic Affairs 2. Access & ADA Services 3. Bookstore 4. Business Office 5. Career & Placement Services 6. Career & Technical Dean’s Office 7. Entrepreneurship & Business Development Center 8. Finance Office

9. Food Services 10. Human Resources 11. Law Enforcement Training Center 12. Library Services 13. Marketing & Public Relations 14. Mail Room/Copier Services 15. Vice President for Instruction’s Office 16. Plant Operations 17. President’s Office


Due to the quality of feedback received on previous surveys, in spring 2008 the “Comments” section was

replaced with two questions of a more focused nature: 1) “What are the strengths of this department?”, and

2) “What changes could this department make to improve services?” It was found that these questions

helped elicit more constructive feedback from survey participants, making it easier for positive departmental

change.

CONCLUSION In gauging its progress, Moberly Area Community College demonstrates success in continually moving

assessment forward. The focus for the future is on ensuring completion of the feedback loop for all

assessment processes, creating clear structures for decision-making processes as related to assessment, and

using assessment results to drive improvement in student learning. MACC’s 2015-2018 Strategic Plan

emphasizes the importance the College places on teaching, learning, and continuous improvement.

Operationalizing the new plan will result in growth and will keep MACC’s focus squarely on its mission.

table of - moberly area community college...continuous improvement report 5 services. select course...

Documents