academic program: assessment report 2011-2012od 2.11.10 1 component ... direct and indirect...

Academic Program: Assessment Report 2011-2012

Unit/Department: Spanish / LLC Division: College of Arts and Sciences

P&OD 2.11.10 1

Component Description

Program Mission Statement

From your Program Assessment Plan (Statement should articulate the unit/ program mission in support of the institutional mission and include a clearly defined purpose appropriate to collegiate education.)

The Department of Languages, Literature, and Composition provides a significant component of the liberal arts curriculum at USC Upstate. Programs and curricula are designed to foster personal and intellectual growth, to enhance awareness of cultural diversity, and to enrich the cultural life of USC Upstate students, the USC Upstate community, and the greater metropolitan area. The Department is committed to the highest level of quality in the performance of this mission.

The USC Upstate Spanish program aims to teach and facilitate communication in Spanish, provide knowledge and understanding of the Hispanic cultures, build multilingual communities, draw comparisons between various cultural and linguistic systems, and expand our students’ educational experience by connecting Spanish with other disciplines in the University. Our Spanish curriculum is designed to prepare students for a competitive global marketplace, particularly in view of the international nature of the Upstate community and the growing Hispanic population in the United States.

Goal 1

From your Program Assessment Plan (Describe broad learning outcomes and concepts (what you want students to learn) expressed in general terms (clear communication, problem-solving skills, etc). Goals should focus on discipline-specific outcomes relevant to the program.)

Students graduating in Spanish at USC Upstate will be able to demonstrate speaking and listening proficiency in Spanish.

Objectives SLO’s (student learning outcomes)

From your Program Assessment Plan (Describes the specific skills, values and attitudes students should be able to exhibit that reflect the broader goals. Objectives (student learning outcomes) transform the general program goals into specific student performance/behaviors that demonstrate student learning and skill development along these goals.

Students will demonstrate competence in syntax and rules of discourse in varying linguistic tasks as measured by the ACTFL Proficiency Guidelines—Speaking.

Assessment Methods

From your Program Assessment Plan (Describes the measure(s) by which the department will know the students are meeting the departmental learning objectives. Includes both direct and indirect assessment. Each SLO should have at least one assessment method.)

Students will take the ACTFL Oral Proficiency Interview.



P&OD 2.11.10 2

Assessment Criteria

Level of achievement you are targeting (Indicate benchmarks, scores on assessment instruments, etc… that would indicate acceptable achievement under your plan)

Students must pass the ACTFL Oral Proficiency Interview with a score of Advanced Low or above.

Assessment Results

Actual results and data collected (Make sure to break down data by subgroups (e.g. other campuses or emphases). As appropriate, also include item or category analysis.)

During the 2011-2012 academic year, a total of nine students took the ACTFL Oral Proficiency Interview (OPI) via telephone. All nine students were enrolled in Spanish 490: Senior Seminar during Fall 2011 and took the exam between October 10, 2011 and October 19, 2011. Of the nine, two scored at Superior (the highest possible score), three scored at Advanced High, one at Advanced Low, one at Intermediate High and two at Intermediate Mid. The three students (33%) at Intermediate High and Intermediate Mid did not reach our benchmark. The one student (11%) at Advanced Low met our benchmark and the remaining five (56%) surpassed our expectations. In total, 67% of the students met or exceeded our expectations. If we were to separate these results into Spanish and Spanish education majors, of the four Spanish majors three (75%) scored at or above the Advanced Low level thereby meeting our benchmark. Of the five Spanish education majors, only three (60%) scored at or above the Advanced Low level thereby meeting our benchmark. Although only 67% of our students met the benchmark that is 17% higher than the 2010-2011 assessment period when only 50% of the students met it. This is only our second year with the higher benchmark of Advanced Low.



P&OD 2.11.10 3

Action Plan What actions or modifications have been or will be made based on this assessment?

We will continue discussions about implementing an introductory OPI earlier in our students’ academic careers. The idea is that if we evaluate the students’ oral proficiency earlier in their studies and establish a baseline, we can better assist them in achieving an Advanced Low level by the time they are officially tested in senior seminar. This testing is absolutely necessary to determine just how much our students have improved their oral communication skills while in our program. We know where our students are at when they leave, but we never know where they begin with these language skills. We are confident that if we administer this interview earlier, our students will realize just how much they need to work in order to achieve the Advanced Low benchmark in senior seminar. Therefore, all efforts will be made this year to ensure the administration of the mock OPI during the next offering of Spanish 310 in Fall 2013.

In the meantime, we will continue to encourage our students to immerse themselves in the language, whether that be by becoming more involved with the Hispanic community in the Upstate area or immersing themselves in the Hispanic culture through study abroad opportunities.



P&OD 2.11.10 4

Implementation of Previous Years’ Action Plan

Which of the modifications indicated in the previous years’ reports were implemented this year and what was the impact?

As explained above, we are continuing our discussion about the implementation of an introductory OPI before students are enrolled in Spanish 490: Senior Seminar. We want to administer this earlier OPI in Spanish 310: Advanced Oral Communication since this is a required course for both Spanish and Spanish education majors, one that is typically taken during the junior year.

In our discussions, we realized that before this can take place, we needed to change our rotation of upper division classes. Since the creation of Spanish 300: Topics in Conversation, we have offered Spanish 300 in the fall and Spanish 310 in the spring. With this rotation, students enrolled in Spanish 310 in the spring continue on into senior seminar in the fall. This does not leave much time for students to work on their oral communication skills. Therefore, we switched our rotation for the 2012-2013 academic year. Spanish 310 is now offered in the fall and Spanish 300 is offered in the spring. Now a student in Spanish 310 will have an entire year to work on oral communication before having to take the OPI. To do this, we will encourage students to use their Spanish outside of the classroom via study abroad and internships in the community.

Another factor in implementing a mock OPI in Spanish 310 is cost. The OPI administered in senior seminar costs $134 per student. This test is necessary for the School of Education’s Spanish education program to remain nationally accredited through NCATE / ACTFL. The mock OPI we have found through Versant is more reasonable, only $25 per student. However, the total cost of both exams might be cost prohibitive, especially during the 2012-2013 year with 15 students enrolled in Spanish 490 and 19 in Spanish 310. However, we feel that the increased enrollment numbers in Spanish 490 will drop off in our next offering in Fall 2013. Between now and Fall 2013, we will look into less expensive administration of the OPI in hope that we can include the mock OPI into Spanish 310 with the savings.

Once we are able to establish our students’ baseline, we will be better able to see how Spanish 300 has affected our students’ oral communication skills. The students from our first offering of Spanish 300 in Fall 2010 finally reached senior seminar in Fall 2011. However, it is difficult to say just how this class assisted with their performance on the OPI with no baseline, especially with our lower-performing students.



P&OD 2.11.10 5


Goal 2


Students graduating in Spanish at USC Upstate will be able to demonstrate the ability to write in different styles.



At various points within the course of study of Spanish at USC Upstate, students will be able to:

2a. Write with clarity and precision on assigned texts. 2b. Demonstrate an ability to write for various audiences.

Assessment Methods


1. SSPN 309 will require students to write two documents showing different

writing styles/purposes. These papers will be included in the students’ portfolios.

2. A paper from one upper-division literature course other than SSPN 301 will become a part of the students’ portfolios.

Assessment Criteria


The Spanish Program Assessment Committee used the USC Upstate Writing Assessment Rubric consistent with ACTFL standards to assess research papers (See Attachment 1). The benchmarks for literary analysis papers are: approaches standard; meets standard; exceeds standard in ACTFL terminology. These correspond to weak (high/low), average, and excellent (high/low) respectively on the USC Upstate Writing Assessment Rubric. Under our current plan, an acceptable level of achievement would be meets standards or above.



P&OD 2.11.10 6

Assessment Results


In the Fall 2011 offering of Spanish 309: Advanced Spanish Language I, we collected essays in the first days of the semester. These papers will give us a baseline of where our students are upon entering upper division classes. These students will be tracked as they move through the program. These initial scores will allow us to compare the students’ skills at the beginning and end of the major. I have only included papers from Spanish and Spanish education majors. Student 4781 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 3 / 5 4 Clarity of Purpose 3 / 4 3.5 Organization of Content 2 / 4 3 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 3 3 Total Average 3.4 (meets standard) Student 6632 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 3 / 4 3.5 Clarity of Purpose 3 / 4 3.5 Organization of Content 3 / 4 3.5 Language and Style/Voice 4 / 5 4.5 Mechanics 4 / 4 4 Total Average 3.8 (meets standard) Student 4966 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 4 4.5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 4 / 4 4 Total Average 4.7 (exceeds standard) Student 0220 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 3 / 4 3.5 Clarity of Purpose 3 / 3 3 Organization of Content 3 / 4 3.5



P&OD 2.11.10 7

Language and Style/Voice 2 / 3 2.5 Mechanics 2 / 4 3 Total Average 3.1 (meets standard) Student 9353 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 3 / 4 3.5 Clarity of Purpose 3 / 4 3.5 Organization of Content 3 / 5 4 Language and Style/Voice 3 / 4 3.5 Mechanics 2 / 3 2.5 Total Average 3.4 (meets standard) Student 9759 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought NA / 5 5 Clarity of Purpose NA / 4 4 Organization of Content 5 / 5 5 Language and Style/Voice 4.5 / 4 4.25 Mechanics 5 / 4 4.5 Total Average 4.55 (exceeds standard) Student 9567 (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 3 / 4 3.5 Clarity of Purpose 3 / 4 3.5 Organization of Content 3 / 3 3 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 3 3 Total Average 3.3 (meets standard) Of the seven students, all of them currently meet the standards. The two highest scores are from heritage/native speakers. The actual composition scores tend to be higher than the language scores. Of the two language scores, our students need assistance with mechanics (grammar and usage, punctuation, and spelling).



P&OD 2.11.10 8

Of the eight students that completed Spanish 490: Senior Seminar, only six students submitted papers from Spanish 309: Advanced Composition and Grammar. Of the two other students, one transferred this class in from another university and did not retain any of his papers from this class. No students submitted two papers. The students’ papers were rated as follows:

Student 1126 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5 Clarity of Purpose 4 / 4 4 Organization of Content 4 / 4 4 Language and Style/Voice 5 / 5 5 Mechanics 4 / 4 4 Total Average 4.3 (exceeds standard) Student 1268 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 3 / 3 3 Clarity of Purpose 3 / 3 3 Organization of Content 3 / 3 3 Language and Style/Voice 3 / 3 3 Mechanics 3 / 3 3 Total Average 3.0 (meets standard) Student 2374 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 4 4.5 Total Average 4.9 (exceeds standard) Student 6018 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5



P&OD 2.11.10 9

Total Average 5.0 (exceeds standard) Student 5591 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 4.5 / 4 4.25 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Total Average 4.85 (exceeds standard) Student 3516 (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 3 / 5 4 Clarity of Purpose 3 / 5 4 Organization of Content 3 / 5 4 Language and Style/Voice 3 / 5 4 Mechanics 3 / 4 3.5 Total Average 4.7 (exceeds standard) With regard to these papers produced in Spanish 309: Advanced Spanish Language I, all met or exceeded our benchmark. One (17%) met the benchmark while five (83%) exceeded the benchmark. As for the literature papers to assess SLO 2b, all students submitted at least one paper. Assessment criteria depended on the type of assignment given by each professor. Specifically, some assignments had a research component and necessitated the Research and Research Mechanics sections found below. Students were allowed to select the papers that went into their portfolios, so the papers are from a variety of literature courses. Student 1126 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 3 / 3 3 Clarity of Purpose 3 / 3 3 Organization of Content 3 / 4 3.5 Language and Style/Voice 3 / 3 3 Mechanics 3 / 3 3 Research 3 / 3 3 Research Methods 3 / 3 3



P&OD 2.11.10 10

Total Average 3.1 (meets standard) Reader 1 / Reader 2 Average Quality of Thought 4 / 4 4 Clarity of Purpose 4 / 3 3.5 Organization of Content 4 / 4 4 Language and Style/Voice 3 / 3 3 Mechanics 3 / 3 3 Research 4 / 4 4 Research Methods 4 / 4 4 Total Average 3.6 (meets standard) Student 1268 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 3 / 4 3.5 Clarity of Purpose 3 / 4 3.5 Organization of Content 3 / 5 4 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 3 3 Research 3 / 4 3.5 Research Methods 3 / 4 3.5 Total Average 3.5 (meets standard) Reader 1 / Reader 2 Average Quality of Thought 4 / 5 4.5 Clarity of Purpose 4 / 5 4.5 Organization of Content 4 / 5 4.5 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 4 3.5 Research 4 / 5 4.5 Research Methods 4 / 5 4.5 Total Average 4.2 (exceeds standard) Student 2374 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5



P&OD 2.11.10 11

Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Total Average 5.0 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Total Average 5.0 (exceeds standard) Student 6018 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 4 / 5 4.5 Mechanics 3 / 3 3 Research 5 / 5 5 Research Mechanics 5 / 5 5 Total Average 4.6 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 3 3 Research 5 / 5 5 Research Mechanics 5 / 5 5 Total average 4.5 (exceeds standard) Student 5011 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5



P&OD 2.11.10 12

Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Research 5 / 5 5 Research Mechanics 5 / 5 5 Total Average 5 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Total Average 5 (exceeds standard) Student 2822 (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5 Clarity of Purpose 5 / 4 4.5 Organization of Content 4 / 4 4 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 4 3.5 Total Average 4.0 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 4 4.5 Organization of Content 4 / 4 4 Language and Style/Voice 5 / 4 4.5 Mechanics 3 / 4 3.5 Total Average 4.3 (exceeds standard) Student 5591 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 4 4.5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 4 4.5



P&OD 2.11.10 13

Research 5 / 5 5 Research Mechanics 5 / 5 5 Total Average 4.9 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5 Clarity of Purpose 5 / 5 5 Organization of Content 4 / 5 4.5 Language and Style/Voice 5 / 5 5 Mechanics 4 / 5 4.5 Total Average 4.8 (exceeds standard) Student 3516 (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 4 4.5 Mechanics 4 / 3 3.5 Total Average 4.6 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 4 / 4 4 Total Average 4.83 (exceeds standard) In total, sixteen essays were evaluated. Of the sixteen, three (19%) received meets standards. Thirteen (81%) received exceeds standards. All sixteen essays (100%) met our benchmark.



P&OD 2.11.10 14


Looking at these results, the results for the initial papers produced in Spanish 309 have me worried. According to the results, all of our students met the benchmark expected of them coming out of senior seminar. This has the preparer of the report concerned about our inter-rater reliability. This will be an issue that we explore as we move forward in future assessment cycles.

We will continue to collect an essay from students in the initial days of Spanish 309 to compare the results with papers produced in Spanish 490. We will also require students to include a paper from the end of Spanish 314 to chart the progress made by the students during our year-long writing sequence, Spanish 309 and Spanish 314. Collecting papers from these two courses will better enable us to monitor student success.



According to last year’s assessment report, we stated that we would continue to monitor students’ writing skills in literature and composition courses. This assessment period, every student met or exceeded the benchmarks set for written communication. With this being the first cohort of students to be able to submit papers for both Spanish 314 and Spanish 490, the results indicate that the students are benefitting from the additional composition course.

To better monitor our students’ writing skills, we needed to establish a baseline of our students’ writing skills upon entering into the upper division classes. In previous assessment reports, we knew where our students were with regard to their oral and written communication skills upon graduating from our program, but we never had a baseline to show how our program assisted them in reaching their level. Therefore in Spanish 309: Advanced Spanish Language I, all students completed a composition during the first week of school. We selected this course because it is typically one of the first courses that a student takes upon entering upper division classes. We will use the data we collected from the course offered in Fall 2011 and compare it to the students’ senior seminar projects to see we have on our students’ writing skills.



P&OD 2.11.10 15


Goal 3


Students graduating in Spanish at USC Upstate will be able to demonstrate an adequate command of grammar and linguistics.



At various points within the course of study of Spanish at USC Upstate, students should be able to:

3a. Demonstrate competence in syntax and rules of discourse, as measured by the ACTFL Written Proficiency Scale. 3b. Demonstrate understanding of the principles of Spanish phonetics and how it compares with English phonetics.

Assessment Methods


1. The paper completed for senior seminar will be evaluated according to the ACTFL

Written Proficiency Scale. This will serve as the basis for assessing student learning objective 3a.

2. Students are required to take either SSPN 453 or SSPN 454. Both courses will

require students to write a paper comparing the Spanish and English phonetic systems. The paper will be maintained in the student portfolios. A checklist will be used to determine adequate understanding. This will serve as the basis for assessing student learning objective 3b.



P&OD 2.11.10 16

Assessment Criteria


To assess the senior seminar paper, the Spanish Program Assessment Committee used the USC Upstate Writing Assessment Rubric consistent with ACTFL standards to assess research papers (See Attachment 1). The benchmarks for literary analysis papers are: approaches standard; meets standard; exceeds standard in ACTFL terminology. These correspond to weak (high/low), average, and excellent (high/low) respectively on the USC Upstate Writing Assessment Rubric. Under our current plan, an acceptable level of achievement would be meets standards or above. To assess student learning outcome 3b, students must produce a table illustrating: a) the letters of the Spanish alphabet, including the “non-letters” rr and ll, b) the phoneme(s) produced by that orthographic representation in all relevant environments, and c) illustrative examples of those phonemes from real words in Spanish and English. Please see Attachment 2 for a representation of this assessment method. Seventy-five percent of the entries must be essentially correct.

Assessment Results


To assess SLO 3a, the Spanish Program Assessment Committee scored eight papers generated from Spanish 490: Senior Seminar. Using the USC Upstate Writing Assessment Rubric, each paper was scored as a whole. To analyze the students’ competence in syntax and rules of discourse, one needs to look at the score for “Language and Style/Voice” and “Mechanics” categories on the rubric since they are the only items on the rubric that deal specifically with syntax and rules of discourse. The scores in these two areas are as follow: Student 1126 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Language and Style/Voice 3 / 3 3 Mechanics 3 / 3 3 Total Average 3 (meets standard) Student 1268 (Spanish major) Reader 1 / Reader 2 Average Language and Style/Voice 4 / 4 4 Mechanics 4 / 4 4 Total Average 4 (exceeds standard) Student 2374 (Spanish major) Reader 1 / Reader 2 Average Language and Style/Voice 5 / 5 5



P&OD 2.11.10 17

Mechanics 5 / 5 5 Total Average 5 (exceeds standard) Student 6018 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Language and Style/Voice 4 / 5 4.5 Mechanics 4 / 4 4 Total Average 4.25 (exceeds standard) Student 5011 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Language and Style/Voice 4 / 5 4.5 Mechanics 3 / 5 4 Total Average 4.25 (exceeds standard) Student 2822 (Spanish education major) Reader 1 / Reader 2 Average Language and Style/Voice 5 / 4 4.5 Mechanics 4 / 4 4 Total Average 4.25 (exceeds standard) Student 5591 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Language and Style/Voice 5 / 5 5 Mechanics 5 / 4 4.5 Total Average 4.75 (exceeds standard) Student 3516 (Spanish education major) Reader 1 / Reader 2 Average Language and Style/Voice 4 / 4 4 Mechanics 4 / 3 3.5 Total Average 3.75 (meets standard) Of the eight papers, all students received either meets standard or exceeds standard. Two students (25%) met the standard and six students (75%) exceeded our benchmark. Heritage/native speakers received the highest scores, and there seems to be no major differences between scores from Spanish majors and those from Spanish education majors. However, when we compare the language scores from the senior seminar papers with those scores produced on papers from Spanish 309, we do see an increase in the language scores of our non-native speakers.



P&OD 2.11.10 18

For SLO 3b, seventeen students were enrolled in SSPN 454: Spanish Linguistics during spring semester of 2012. Fifteen of the seventeen students (88%) produced an acceptable representation of the comparison between Spanish and English phonemes. The two students (12%) who did not meet the benchmark of 75% did not miss it by much (1% and 2%, respectively). There were no significant differences between the Spanish and Spanish education majors.

Student Number % score 1 (Spanish) 90 2 (Spanish) 86 3 (Spanish) 85 4 (Spanish min.) 85 5 (Spanish) 89 6 (Spanish) 74 7 (Spanish) 90 8 (Spanish ed.) 95 9 (Spanish) 73 10 (Spanish) 79 11 (Spanish ed.) 88 12 (Spanish) 81 13 (Spanish ed.) 93 14 (Spanish ed.) 83 15 (Spanish) 85 16 (Spanish ed.) 79 17 (Spanish ed.) 76

In the past five assessment cycles, since 2007-2008, our students have continually achieved the benchmark. While 100% of the students enrolled in Spanish 453 and 454 were able to meet the benchmark, and most were able to surpass our expectations for the past 3 assessment cycles, the students from Spring 2012 had more difficulty. Yes, 88% of the students reached the benchmark. However, the scores are lower when compared to the last few assessment cycles. While this trend data indicates that SLO 3b needs to be re-evaluated, the Spanish faculty did discuss this, but determined that the SLO needed to be left as is since it is an essential component for the School of Education’s NCATE / ACTFL reaccreditation process for their Spanish education program.



P&OD 2.11.10 19


With regard to SLO 3a, not modifications will be made at this time. With the curriculum changes that have been made to the Spanish major, all students need to take an additional writing course (Spanish 314: Advanced Spanish Language II). This is the first graduating class to take Spanish 314 in Spring 2011 and then Spanish 490 in the fall. With all students meeting and exceeding our benchmark, we will continue to monitor student success.

With regard to SLO 3b, the trend data indicates that it is time to either re-evaluate our benchmark and raise our standards or we need to find a new, more difficult assessment to measure the SLO. However, the Spanish faculty did discuss this in October 2011 and determined that the SLO needs to be left as it is since this exercise is an essential component in the School of Education’s NCATE / ACTFL reaccreditation process for the Spanish education program. Therefore, we will continue to monitor our students’ success.



According to the previous year’s report, we noted that it was time to re-evaluate SLO 3b. In a faculty meeting in October 2011, we discussed other measures that we could implement since our students are continually meeting the benchmark. However, this assessment was originally selected due to the School of Education’s NCATE / ACTFL accreditation process for their Spanish education program. Students need to demonstrate linguistic competency and this is the best way to do measure their competency. Therefore, we have decided to keep this assessment and continue to monitor our students’ performance.



P&OD 2.11.10 20


Goal 4


Students graduating in Spanish at USC Upstate will be able to read, understand and analyze popular and literary texts in Spanish.



At various points within the course of study of Spanish at USC Upstate, students should be able to:

4a. Demonstrate an ability to situate and interpret texts in their historical and cultural contexts. 4b. Make connections between texts written in Spanish.

Assessment Methods


Students will write papers in their upper-division literature courses, two of which must be maintained for inclusion in the portfolio. These papers may satisfy both SLOs listed above, but between the two papers, both must be addressed.

Assessment Criteria


The Spanish Program Assessment Committee used the USC Upstate Writing Assessment Rubric consistent with ACTFL standards to assess literary papers. (See Attachment 1)

The benchmarks for literary analysis papers are: approaches standard; meets standard; exceeds standard in ACTFL terminology. These correspond to weak (high/low), average, and excellent (high/low) respectively on the USC Upstate Writing Assessment Rubric. Under our current plan, an acceptable level of achievement would be meets standards or above.



P&OD 2.11.10 21

Assessment Results


All students submitted at least one literature paper in their portfolio. Assessment criteria depended on the type of assignment given by each professor. Specifically, some assignments had a research component and necessitated the Research and Research Mechanics sections found below. Students were allowed to select the papers that went into their portfolios, so the papers are from a variety of literature courses. Student 1126 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 3 / 3 3 Clarity of Purpose 3 / 3 3 Organization of Content 3 / 4 3.5 Language and Style/Voice 3 / 3 3 Mechanics 3 / 3 3 Research 3 / 3 3 Research Methods 3 / 3 3 Total Average 3.1 (meets standard) Reader 1 / Reader 2 Average Quality of Thought 4 / 4 4 Clarity of Purpose 4 / 3 3.5 Organization of Content 4 / 4 4 Language and Style/Voice 3 / 3 3 Mechanics 3 / 3 3 Research 4 / 4 4 Research Methods 4 / 4 4 Total Average 3.6 (meets standard) Student 1268 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 3 / 4 3.5 Clarity of Purpose 3 / 4 3.5 Organization of Content 3 / 5 4 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 3 3 Research 3 / 4 3.5 Research Methods 3 / 4 3.5 Total Average 3.5 (meets standard)



P&OD 2.11.10 22

Reader 1 / Reader 2 Average Quality of Thought 4 / 5 4.5 Clarity of Purpose 4 / 5 4.5 Organization of Content 4 / 5 4.5 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 4 3.5 Research 4 / 5 4.5 Research Methods 4 / 5 4.5 Total Average 4.2 (exceeds standard) Student 2374 (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Total Average 5.0 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Total Average 5.0 (exceeds standard) Student 6018 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 4 / 5 4.5 Mechanics 3 / 3 3 Research 5 / 5 5 Research Mechanics 5 / 5 5 Total Average 4.6 (exceeds standard)



P&OD 2.11.10 23

Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 3 3 Research 5 / 5 5 Research Mechanics 5 / 5 5 4.5 (exceeds standard) Student 5011 (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Research 5 / 5 5 Research Mechanics 5 / 5 5 Total Average 5 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 5 5 Total Average 5 (exceeds standard) Student 2822 (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5 Clarity of Purpose 5 / 4 4.5 Organization of Content 4 / 4 4 Language and Style/Voice 3 / 4 3.5 Mechanics 3 / 4 3.5 Total Average 4.0 (exceeds standard) Reader 1 / Reader 2 Average



P&OD 2.11.10 24

Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 4 4.5 Organization of Content 4 / 4 4 Language and Style/Voice 5 / 4 4.5 Mechanics 3 / 4 3.5 Total Average 4.3 (exceeds standard) Student 5591 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 4 4.5 Language and Style/Voice 5 / 5 5 Mechanics 5 / 4 4.5 Research 5 / 5 5 Research Mechanics 5 / 5 5 Total Average 4.9 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5 Clarity of Purpose 5 / 5 5 Organization of Content 4 / 5 4.5 Language and Style/Voice 5 / 5 5 Mechanics 4 / 5 4.5 Total Average 4.8 (exceeds standard) Student 3516 (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Language and Style/Voice 5 / 4 4.5 Mechanics 4 / 3 3.5 Total Average 4.6 (exceeds standard) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5



P&OD 2.11.10 25

Organization of Content 5 / 5 5 Language and Style/Voice 5 / 5 5 Mechanics 4 / 4 4 Total Average 4.83 (exceeds standard) In total, sixteen essays were evaluated. Of the sixteen, three (19%) received meets standards. Thirteen (81%) received exceeds standards. All sixteen essays (100%) met our benchmark.


While we have satisfactory scores in this section, there is one error that was pointed out by the Assessment Committee during the 2009-2010 cycle: How does the USC Upstate Writing Rubric assist us in evaluation 4b? Looking at the papers being turned in with the portfolios, there is still one glaring omission: papers that make connections between texts in Spanish, or SLO 4b. The essays collected either situate a work within its historical and cultural contexts or analyze a salient characteristic, but none compare works. Students enrolled in Spanish 330: Survey of Spanish Literature I were asked to write a paper comparing works, but this is not an assignment that is given in other literature class.

This SLO continues to be one in which we need to re-evaulate. Our program has changed dramatically over the years, but the ways in which we assess our students have not. Therefore, we need to reassess our assessment methods for each SLO this year and create assessments that better reflect what we are asking from our students.



With the major changes that we made to the Spanish curriculum, we have yet to implement any modifications to this SLO. We focused more on establishing baselines for our students’ oral and written communication. With regard to writing, we did this by assigning an initial writing exercise at the beginning of Spanish 309. The addition of writing classes such as Spanish 314: Advanced Spanish Language II should strengthen our students’ abilities to write in different styles. We will need to spend this year modifying SLO, looking at the assignments we give in our literature classes and the rubric we use to assess them.



P&OD 2.11.10 26


Goal 5


Students graduating in Spanish at USC Upstate will be able to demonstrate an ability to compare the products, practices and perspectives of the cultures of Spanish speakers with others.



At various points within the course of study of Spanish at USC Upstate, students should be able to compare the culture of a particular Hispanic group with the students’ own culture.

Assessment Methods


Students will write papers in their upper-division culture courses, one of which must be maintained for inclusion in the portfolio.

Assessment Criteria


The Spanish Program Assessment Committee used the USC Upstate Writing Assessment Rubric consistent with ACTFL standards to assess research papers (See Attachment 1). The benchmarks for cultural analysis papers are: approaches standard; meets standard; exceeds standard in ACTFL terminology. These correspond to weak (high/low), average, and excellent (high/low) respectively on the USC Upstate Writing Assessment Rubric. Under our current plan, an acceptable level of achievement would be meets standards or above.

Assessment Results


Of the eight students who submitted portfolios in Spanish 490: Senior Seminar, all eight included a culture paper. This is an increase from past assessment periods. Of the eight portfolios, six students included the culture paper written in Spanish 320: Spanish Civilization. One student submitted a paper written while studying abroad. The final student submitted a comparative culture paper in Spanish 314: Advanced Spanish Language II that was assessed.



P&OD 2.11.10 27

In assessing the paper, the Spanish Program Assessment Committee (PAC) focused specifically on the following three sections of the USC Upstate Writing Assessment Rubric: 1) quality of thought; 2) clarity of purpose; and 3) organization of content. The PAC assessed how these three sections tied into culture. The papers were assessed by two PAC members who rated the papers as follows:

Student 1126 (submitted actual comparative paper) (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5 Clarity of Purpose 5 / 4 4.5 Organization of Content 5 / 5 5 Total Average 4.67 (exceeds standard) Student 1268 (submitted actual comparative paper) (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5 Clarity of Purpose 5 / 4 4.5 Organization of Content 5 / 4 4.5 Total Average 4.5 (exceeds standard) Student 2374 (submitted actual comparative paper) (Spanish major) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Total Average 5 (exceeds standard) Student 6018 (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Total Average 5 (exceeds standard) Student 5011 (submitted actual comparative paper, but SSPN 314, not a culture class) (Spanish education major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 4 4.5



P&OD 2.11.10 28

Clarity of Purpose 5 / 5 5 Organization of Content 4 / 4 4 Total Average 4.5 (exceeds standard) Student 2822 (submitted actual comparative paper) (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 3 / 4 3.5 Clarity of Purpose 4 / 5 4.5 Organization of Content 4 / 5 4.5 Total Average 4.12 (exceeds standard) Student 5591 (submitted actual comparative paper) (Spanish major; heritage/native speaker) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Total Average 5 (exceeds standard) Student 3516 (submitted actual comparative paper) (Spanish education major) Reader 1 / Reader 2 Average Quality of Thought 5 / 5 5 Clarity of Purpose 5 / 5 5 Organization of Content 5 / 5 5 Total Average 5 (exceeds standard) According to the data, our students have made significant strides in this SLO since the last assessment period. All eight students submitted an actual comparative paper. In addition, all eight students exceeded our expectations overall. With each student at “exceeds standard,” there is no real difference between the scores received from Spanish majors and those from Spanish education majors. There is only a slight difference between the papers produced by heritage/native speakers and non-native speakers. The lowest scores did come from non-native speakers, but the differences were minimal.



P&OD 2.11.10 29


With the chages that have been made to the Spanish program, students have more opportunities to take culture classes with at least one culture class being offered each semester. However, as we continue to build on our offerings, the foreign language faculty needs to continue reconsidering the writing assignments being given in these courses. Not all writing assignments submitted align with this SLO which is problematic. Since students now have a more open curriculum, one could theoretically take only one culture class during their academic careers here at USC Upstate. If the student is enrolled in a class that does not have an assignment aligned with this SLO, we will have no assessment data for that particular student. Therefore, we need to either create a common assessment that can be used across culture classes or reevaluate the way we assess culture.



Many students in Spanish 490: Senior Seminar also completed the new writing course we incorporated into the Spanish core curriculum in Spring 2011. The course is not a requirement for the Spanish education majors, but it is a requirement for Spanish majors. This is an additional class where students focus on oral and written communication skills. This is the first assessment period where students took Spanish 314 before Spanish 490. With all of our students exceeding the standard this assessment cycle, it appears that the additional writing practice is benefitting our students. The Spanish faculty made only one modification this past year, and it was motivated by the changes that we have made to the Spanish program. The Spanish major has made substantial changes to its core curriculum. Students have a set of core classes that they need to take, but they have more variety with their remaining major courses. They are allowed to create their own specialty (language, literature, or culture) within the discipline. Students are no longer required to take two culture classes as before. Since not all students will take two culture classes, they will not necessarily have two culture papers to include in their portfolios. Therefore, we decreased the number of papers to be included in the portfolio from two to one. With this change, the Spanish faculty needs to develop a common assessment to incorporate across all culture classes.



P&OD 2.11.10 30

Attachment 1--USC Upstate Writing Assessment Rubric Student Name: ________________________________________

Writing Skills Evaluation Excellent Average Weak High Low High Low Quality of Thought 5 4 3 2 1 Recognition/examination of complex ideas Credible and logical support

Unified and strong control of content. Opinions and claims are well-supported. Ample evidence. Logic is maintained. Paper is ambitious and insightful.

Generally unified. Ideas are balanced. Opinions and claims are adequately supported and usually logical. Paper is ambitious but quality is inconsistent.

Little evidence of control of ideas. Ideas are not supported, or support is clichéd or illogical. Paper is not ambitious or insightful.

Clarity of Purpose 5 4 3 2 1 Clear purpose Audience awareness

Positions are insightful and clear. Thesis is focused; intent is obvious. Clear sense of audience.

Positions are sound and understandable. Thesis is general; intent is evident. Somewhat aware of audience.

Lacks position on topics. Thesis may be absent or intent is unclear. Superficial. Little to no recognition of audience.

Organization of Content 5 4 3 2 1 Arrangement Unity Coherence

Ideas are well connected through structural and linguistic transitions. Structure complements and completes content.

Some evidence of structural and linguistic transitions. Structure is generally adequate for the content.

Overall unity and coherence are flawed. Parts are poorly connected, and there is little evidence of planning or organization.

Language and Style / Voice 5 4 3 2 1 Vocabulary Register Tone

Exhibits confident, skillful use of language. Uses a varied, accurate, and appropriate vocabulary. Sentences are purposefully designed and matched to content.

Exhibits adequate but inconsistent facility in the use of language. Uses generally appropriate vocabulary and varied sentences.

Displays little facility in the use of language. Uses limited vocabulary or inappropriate word choice. Sentences are basic. Paper may be repetitive or wordy.

Mechanics 5 4 3 2 1 Grammar and usage Punctuation Spelling

Grammatical structures are well-chosen. No errors detract from meaning. Understood by those unaccustomed to the writing of non-natives.

Grammatical structures carry the meaning forward, although readers notice occasional error(s). Generally understood by those accustomed to the writing of non-natives.

Grammar errors are so obtrusive that readers are seriously distracted by them.

Research 5 4 3 2 1 Quality of sources Integration of sources

Sources are of high quality and are credible. They are ample and appropriate. Source materials are integrated smoothly. Sources are clearly identified.

Sources are of reasonable quality and are credible. Some source materials may be included awkwardly. Some attempt at synthesizing source material.

Sources are of low quality or are not credible. Little to no citation of source material in text or works cited. Possible plagiarism due to lack of awareness or skill.

Research Mechanics 5 4 3 2 1 MLA formatting Number of sources

Clear attribution and accurate documentation. In-text citations and works cited page are correct. Proper use of signal phrases. Cites required number of sources.

Some evidence of citations in text and on works cited page. Inconsistent use of MLA formatting. Lacks required number of sources.

Little or no use of signal phrases. Significant errors in formatting in-text and works cited page citations. Cites few or no sources.



P&OD 2.11.10 31

Essays plagiarized or not written on the essay assignment will receive a score of zero.

TOTAL SCORE: _______

COMMENTS:



P&OD 2.11.10 32

Attachment 2

Consonants English Spanish

Example Phoneme Letter Letter Phoneme Example boat /b/ B B /b/

/β/ burro nube

camp city

/k/ /s/

C C /k/ /θ/ or /s/

campo cielo

church /ʧ/ Ch Ch /ʧ/ chico dad /d/ D D /d/

/ ð / donde usted

feather /f/ F F /f/ fonetica goat /g/ G G /g/

/x/ gato

general happy /h/ H H silent hacer jump /ʤ/ J J /x/ juego key /k/ K K /k/ kiosko

letter /l/ L L /l/ hielo Ll /ʎ/ or

/ʝ/ /ʤ/

llorar

mom /m/ M M /m/ madre note sing

/n/ /η/

N N /n/

noche

Ñ /ɲ/ niño perfect /p/ P P /p/ perro

Q /k/ queso real /ɹ/ R R /ɾ/

/ r/ cantar rojo

RR /r/ carro supper

pleasure /s/ /ʒ/

S S /s/ si

telephone /t/ T T /t/ tomate value /v/ V V /b/

/v/ vaca obvio

water /w/ W yellow /j/ Y Y /j/ ya

zoo /z/ Z Z /θ/ or /s/ Nariz



P&OD 2.11.10 33

Other Other she /ʃ/ /gw/ güiro think /θ/

mother /ð/

Vowels & Dipthongs

English Spanish

Phoneme Example Phoneme Example /ʌ/ cup /a/ andar

/ɑ:/ arm /e/ elephante

/æ/ cat /i/ si

/ǝ/ away, cinema /o/ todo

/e/ met

/ɜ:/ turn, learn /u/ perú

/I/ hit /i/ y

/i:/ see, heat, happy

/ɒ/ hot

/ɔ:/ call, four

/ʊ/ put, could

/u:/ blue, food

/aI/ five, eye

/aʊ/ now, out

/oʊ/ or /ǝʊ/ go

/eǝ/ where, air

/eI/ say, eight

/Iǝ/ near, here

/ǝI/ boy, join

/ʊǝ/ pure, tourist

academic program: assessment report 2011-2012od 2.11.10 1 component ... direct and indirect...

Documents