WASC ARC 2008 CBU 1
Assessment at California Baptist University
The goal of assessment is the continual improvement of the learning environment. Student Outcomes Assessment
Components of the Cycle Defining Collecting Analyzing Reporting Improving
WASC ARC 2008 CBU 2
Schools and Departments develop PSOs for each major or program with “critical assignments” and evaluation rubrics
DEFINING
Students produce “critical assignments” in noted courses using pictured rubrics as guide as well as suggested formats, etc. Assignments are shared with professors for evaluation
COLLECTING
Course / Program & University Student Out Matrix
I = Introduced; P = Practiced; D = Demonstrated
Schools and Departments complete the CPUSO matrices to link USOs & PSOs with particular courses in which specific critical assignments will be evaluated
DEFINING
Professors provide guidance in syllabi/rubrics and direct critical assignments based on School/Dept planning
DEFINING
Profs evaluate student’s critical assignments using rubrics, etc. In LiveText, the rubric results are gathered for ALL that are evaluated using that rubric
ANALYZING
Results are analyzed & will
help identify strengths and weaknesses in curriculum, pedagogy, methods, etc.
ANALYZING -REPORTING
Plans for improvement are implemented and assessment continues. External constituencies will be canvassed to validate outcomes & PSO listings.
IMPROVING
Schools and Departments develop plans for curricular improvement.
IMPROVING
Annual Student Outcomes Assessment Process, page 2
Goal: Continuous Improvement of the Learning Environment
asmtz/asoprocess.ppt
Initial Report ID’s “AYs PSO Focus”
LiveText Rubrics serve to provide for formative assessment for students and summative assessment for critical assignmnts/PSOs
Collected data is arranged in various charts for analysis
Final Reports explain analyses from the standpoint of intended improvements
Annual Recap Reports and FEEE display process progress and intended adjustments
WASC ARC 2008 CBU 3
DEFINING
Fall 2008 Fall 2007
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING RESULTS
ACTION TAKEN
CONTINUED IMPROVEMENT Faculty Workshop Theme
Writing Across the Curriculum
USO 3 Critical Thinking and Literacy Skills
Faculty implement plans and report results and plans for follow-up at Fall Assessment Colloquium.
Academic units develop a plan for enhancing writing instruction and assignments requiring critical thinking.
Faculty across campus report decline in student written and oral skills when expressing ideas
Identify points at which writing and expressing ideas are introduced, practiced, evaluated
WASC ARC 2008 CBU 4
USOs, PSOs and Discipline Rubric Relationships
University Student Outcomes (6)
Discipline X PSOs w/PSO Rubric
Course X1 w/ Disp Rubric incl PSO
Course X2 w/ Disp Rubric incl PSO
Course X3 w/ Disp Rubric incl PSO
Discipline Y PSOs w/PSO Rubric
Course Y1 w/ Disp Rubric incl PSO
Course Y2 w/ Disp Rubric incl PSO
Course Y3 w/ Disp Rubric incl PSO
Discipline Z PSOs w/PSO Rubric
Course Z1 w/ Disp Rubric incl PSO
Course Z2 w/ Disp Rubric incl PSO
Course Z3 w/ Disp Rubric incl PSO
1. “Disciplines” represent the Schools, Departments and their individual Majors or Progams. Individual Majors or Programs have distinct PSO sets that Identify a graduate in that discipline. The PSO Rubric is developed to evaluate the critical elements of any student’s “outcomes” with the ultimate goal of determining efficacy of the program.
2. The subordinate courses use the same rubric, but can have distinctive elements of the discipline evaluated as a subordinate component of the rubric by using “milestones.” Milestones can be time-related such as entry level, mid-range and capstone. Other milestone options could be differing components of a single discipline, etc.
3. Using these LiveText rubrics and the milestone characteristic allows the various disciplines to develop a variety of reporting levels that support the course, PSO, discipline and university evaluation of student outcomes.
WASC ARC 2008 CBU 5
Step 1, Defining
In this stage the development of the Program Student Outcomes (PSO) occur.
In one major or program, the list of PSOs DEFINE what knowledge, skills, abilities and behaviors a graduate in that discipline should possess.
This stage also is where the Course and Program/University Student Outcomes matrix is developed (and reviewed and revised)
WASC ARC 2008 CBU 6
Step 2, Collecting
Stage one merges into this Collecting stage as the courses and critical assignments are identified for data collection planning
In each annual cycle approximately one-fifth of the PSOs are identified for evaluation
The specific means of assessment are decided upon and plans are put in place for collecting the data
WASC ARC 2008 CBU 7
Step 3, Analyzing
Stage two merges into this Analysis stage as the noted means of assessment occur and data is collected.
Faculty and assessment coordinators evaluate the collected data, develop means for analysis and conduct the analysis
Results of the analyses are prepared for reporting.
WASC ARC 2008 CBU 8
Step 4, Reporting
Stage two merges into this Reporting stage as the data and analyses are placed in the Annual Final Report and Recap Report
A VERY key step in this stage is arriving at the “so what” of the analysis in terms of what changes will be made to improve
In addition to the two reports noted above, each unit will also complete a Self-Evaluation on their assessment Process (the FEEE)
WASC ARC 2008 CBU 9
Step 5, Improving
This final step is the culmination of all of the assessment effort Improving the learning environment as a result of acting on what is discovered in the annual cycle.
Units will take action such as modifying pedagogy or curriculum, etc.
This step merges back into the first step of Defining as the cycle repeats with a review of the PSOs, etc.
WASC ARC 2008 CBU 10
Examples of “Improving”, p.1
Reviewing the evaluation process for standardization Review the use and integration of writing instruction Enhance the development of the “voice” concept in
writing instruction Better incorporate the involvement of faculty in
assessment development Increase coverage of effective modes and media for
diverse population communication Develop a rubric for better evaluation of PSO Develop a new required course to provide for missing
skills and knowledge
WASC ARC 2008 CBU 11
Examples of “Improving”, p.2
Review standards Review exit review process Enhance curricular and assessment processes
Note: These examples are extracted from the CBU “Assessment Activities Review ‘Sampler’ for the 2006-2007 Academic Year.
WASC ARC 2008 CBU 12
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATA
ACTION TAKEN
CONTINUED IMPROVMENT
School of Behavioral Sciences Masters in Family Therapy
PSO 1
Dr. Collins and Dr. Stokes will continue to collect exam pass rate data for the next academic year
CA State Comp Exam writing pass rates = 14% at 80 – 90%; 82% at 90% or above; 4% incomplete.
Most instruction is adequate or better and not indicative of any significant shortcomings.
This early evaluation needs to be reviewed for standardization.
WASC ARC 2008 CBU 13
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATA
ACTION TAKEN
CONTINUED IMPROVEMENT
School of Behavioral Science BA in Sociology
PSO 3 = Written Communications
Dr. Minton and Professor Moore will continue the development of process and procedures.
Continue to use and review use of writing/integration instruction.
90% of students scored at 78% or better on writing rubric.
Standard of success was met (80% at 75th percentile).
WASC ARC 2008 CBU 14
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATAACTION TAKEN
CONTINUED IMPROVEMENT
School of Business BA/S in Business
PSO 5 = Integration of Faith
Dr. Herrity and Dr. Chapman will collect and analyze rubric data and peer evaluation of faculty using new methods.
Training of faculty on the use of Power Posters and other assessment methods. Rubrics will be developed.
30% exemplary, 20% proficient, 9% adequate, 9% inadequate, 32% not evaluated.
Generally acceptable instruction, but process of evaluation needs improvement.
WASC ARC 2008 CBU 15
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATA
ACTION TAKEN
CONTINUED IMPROVEMENT
School of Education Graduate Programs
PSO 4 = Research skills
Dr. Crist, Dr. Timmons, and all program coordinators will continue the development of process and procedures
MS/SS Rubric- 43% exemplary; 55% proficient. SOE Portfl. Rubric- 69% exemplary; 30% N/A
Curriculum is clear and students have good opportunities; some weakness remains in establishing their “voice”.
Continue/Improve focus on writing; Enhance developing “voice” concept; Continue review and development of PSO.
WASC ARC 2008 CBU 16
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATA
ACTION TAKEN
CONTINUED IMPROVEMENT
School of Education Graduate Programs
PSO 6 = Cross Cultural Communication
Professor Cabral will coordinate integration of new curricular elements in targeted classes in each grad program.
Increase coverage of effective modes and media for diverse population communication.
Class average 80.7% on critical assignments evaluated.
Instruction on working with diverse populations is adequate.
WASC ARC 2008 CBU 17
DEFINING
Faculty Self-Assessment Cycle
COLLECTING DATA
ANALYZING DATAACTION TAKEN
CONTINUED IMPROVEMENT
Personal Development Plan
Faculty
Increased Scholarly Activity
Faculty Self Studyreports results of efforts and personal level of satisfaction with growth.
Faculty Self Study identifies number of conferences, papers, and presentations for previous year
Faculty member implements improvement plan (answers additional calls for papers, collaborates with colleagues, takes advantage of internal opportunities.)
In conference with supervisor, faculty member formulates an improvement plan.
WASC ARC 2008 CBU 18
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATAACTION TAKEN
CONTINUED IMPROVEMENT
History, Political Science, Criminal Justice BAS
PSO 3 = History and Political Science
Result for neither major were acceptable: History only 24% exemplary or proficient; Poly Sci only 50% were exemplary or proficient.
Instruction, curriculum and/or evaluation and MOAs need to be improved/revised.
Dr. McHorney and Political Science and History faculty will continue the development of process and procedures.
Poly Sci faculty will develop a new required course to provide for the missing skills and knowledge. History faculty will consider this as well.
WASC ARC 2008 CBU 19
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATAACTION TAKEN
CONTINUED IMPROVEMENT
History, Political Science, Criminal Justice BAS
PSO 3 = History and Political Science
Result for neither major were acceptable: History only 24% exemplary or proficient; Poly Sci only 50% were exemplary or proficient.
Instruction, curriculum and/or evaluation and MOAs need to be improved/revised.
Dr. McHorney and Political Science and History faculty will continue the development of process and procedures.
Poly Sci faculty will develop a new required course to provide for the missing skills and knowledge. History faculty will consider this as well.
WASC ARC 2008 CBU 20
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATAACTION TAKEN
CONTINUED IMPROVEMENT
Modern Languages and Literature MA in English
PSO 1 Integration of FaithPSO 4 Original Scholarship
Dr. Lu and Graduate English faculty will pilot a new Thesis Defense rubric in 2008-09.
Success was achieved with all students meeting the standards at some level
Evaluation rubric for thesis proposals will be designed; further refinement will continue.
Thesis proposal process is developing properly to include these elements.
WASC ARC 2008 CBU 21
DEFINING
Student Outcomes Assessment Cycle
COLLECTING DATA
ANALYZING DATA
ACTION TAKEN
CONTINUED IMPROVEMENT
Modern Languages and Literature BA in Spanish
PSO 2 Respect DiversityPSO 6 Literature Knowledge
Professor Flores and Spanish faculty will monitor newly implemented curriculum in SPA 125.
Success was generally achieved in these evaluations with small exceptions in each PSO.
Literature will be introduced in SPA 125 allowing SPA 115 focus on vocabulary, language, and culture.
As a new program this evaluation is good news . The noted shortcomings are the lack of time in SPA 115 for literature.