ommuniation studies program evaluationmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013...

49
COMMUNICATION STUDIES PROGRAM EVALUATION May 2 nd , 2013 PREPARED FOR: DR. RICK OLSEN COMMUNICATION STUDIES DEPARTMENT CHAIR PRESENTED BY: MEGHAN BENNETT, CHIP BOBBERT, MICHAEL WEBB MIT GRADUATE CANDIDATES

Upload: others

Post on 28-May-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

COMMUNICATION STUDIES PROGRAM EVALUATION

May 2nd, 2013

PREPARED FOR:

DR. RICK OLSEN

COMMUNICATION STUDIES DEPARTMENT CHAIR

PRESENTED BY:

MEGHAN BENNETT, CHIP BOBBERT, MICHAEL WEBB

MIT GRADUATE CANDIDATES

Page 2: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 2

Table of Contents

Communication Studies Evaluation ......................................................................................... 4

Executive Summary..................................................................................................................................... 4

Background and rationale ........................................................................................................................... 4

Client ...................................................................................................................................................... 4

Rationale ................................................................................................................................................ 5

Purposes ..................................................................................................................................................... 5

Audiences / Stakeholders ........................................................................................................................... 6

Primary Stakeholders .............................................................................................................................. 7

Secondary Stakeholders .......................................................................................................................... 7

Tertiary Stakeholders .............................................................................................................................. 7

Logic Model ................................................................................................................................................ 8

Evaluation Model / Framework .................................................................................................................. 9

Reactions ................................................................................................................................................ 9

Learning .................................................................................................................................................. 9

Key Questions ............................................................................................................................................. 9

Evaluation Design: One-Shot Design with Attitudes Survey ..................................................................... 10

Sample ...................................................................................................................................................... 11

Survey Sample: Purposive Sampling ..................................................................................................... 11

Limitations ................................................................................................................................................ 11

Time ...................................................................................................................................................... 11

Sensitivity ............................................................................................................................................. 11

Sample Size ........................................................................................................................................... 12

Inexperience ......................................................................................................................................... 12

Instrumentation ........................................................................................................................................ 12

Demographics Survey ........................................................................................................................... 12

Knowledge Mastery Test ...................................................................................................................... 12

Attitudes Survey.................................................................................................................................... 13

Logistics .................................................................................................................................................... 13

Time Line .................................................................................................................................................. 14

Budget ...................................................................................................................................................... 15

Analysis and results .................................................................................................................................. 15

Demogrpahic Information .................................................................................................................... 15

Knowledge Mastery Test ...................................................................................................................... 16

Attitudes Survey.................................................................................................................................... 17

Discussion and Reccomendations ............................................................................................................. 18

Knowledge Mastery Test ...................................................................................................................... 18

Attitudes Survey.................................................................................................................................... 20

Key Questions: Revisited ........................................................................................................................... 22

Appendices ........................................................................................................................... 23

Page 3: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 3

Appendix A: Bibliography...................................................................................................................... 24

Appendix B: Sub Discipline Course List .................................................................................................. 25

Appendix C: BA Requirements............................................................................................................... 28

Appendix D: Client Stated Learning Outcomes ..................................................................................... 30

Appendix E: Demographics Survey ........................................................................................................ 32

Appendix F: Knowledge Mastery Test ................................................................................................... 34

Appendix G: Attitudes Survey ............................................................................................................... 41

Appendix H: Most useful part of the program question results ............................................................ 42

Appendix I: Part of program to change Question results ...................................................................... 46

Page 4: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 4

COMMUNICATION STUDIES EVALUATION

EXECUTIVE SUMMARY

This plan has been developed as a systematic method of evaluating the effectiveness of the

Communication Studies program at the University of North Carolina Wilmington. Department

heads and faculty have expressed a concern about the content knowledge mastery gained by

students who are expected to graduate the program. This plan includes the background

information necessary to understand the plan’s context as well as a clear rationale for

evaluation. The primary and secondary stakeholders will be identified to ensure that all

applicable parties are informed and involved in the process. The logic model and evaluation

framework chosen to evaluate the program will be described. The key questions that the

evaluation is intended to answer are presented and the design methods chosen to answer

these questions are provided. The sample of students chosen to participate and rationales for

these choices are given. The plan also illustrates any potential threats or limitations to the

evaluation procedures recognized by the evaluation team.

The evaluation team will consist of Master of Science in Instructional Technology candidates

Meghan Bennett, Chip Bobbert, and Michael Webb. The students will develop and implement

the evaluation plan for the department of Communication Studies under the guidance of Dr.

Florence Martin and with the support of Dr. Richard Olsen, Communication Studies

Department Chair, and other departmental faculty.

BACKGROUND AND RATIONALE

CLIENT

The client is UNC Wilmington Communication Studies, an arts and sciences program housed

within a liberal arts college located in the Southeastern United States. They offer a BA in

Communication Studies, which includes ten different sub discipline categories. (See Appendix

B) The department has approximately five hundred and fifty students, this number includes

students who are already declared as Communication Studies majors and those who are still

completing their courses required to enter the program. The program graduates roughly two

hundred and forty students per year. The undergraduate degree requires a minimum of 124

semester credit hours, which includes 21 hours of core course work and 21 hours of elective

courses related to the students’ selected sub discipline. (See Appendix C) Twenty full time

professors and fifteen part time lectures comprise the faculty. Their mission statement states

“Above all, our primary purpose is to provide our majors with a degree in which the

philosophy, practice, criticism, and study of communication are emphasized and balanced.

Page 5: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 5

Specifically, we seek to integrate application with theory.” While they describe their

department as in the same way they define their discipline:

A broad discipline that covers rhetorical theory from the ancient

Greeks to the latest in digital media. It explores the complexity of

interpersonal relationships and the challenges of effective public

relations campaigns. It includes courses in small group communication,

intercultural communication and so much more. Our graduates have

gone on to careers in sales, entrepreneurship, PR/IMC, human

resources, fund raising, graduate programs in law, divinity, MBA and

the list goes on. Versatility is a great way to describe the COM degree

at UNCW.

RATIONALE

Periodically, university programs must assess whether or not their graduates are prepared to

enter the workforce in their field of training. The project has come to fruition as part of a

routine evaluation of program outcomes coupled with the desire to add additional learning

outcomes to their mission.

PURPOSES

The UNC Wilmington Department of Communications Studies is seeking to evaluate the

learning retention of their graduating students. The goal of this evaluation is to provide an

analysis of student knowledge mastery as it relates to the core courses in the program: COM

101, COM 105, COM 200, COM 301, and COM 305. The information will be used by the

faculty in that unit to gauge the preparedness of graduating students and in determining

whether or not it is feasible to add a fifth learning outcome. The client has stated four specific

learning outcomes as part of their educational mission along with the desire to possibly add

additional outcomes. The current outcomes are as follows (cited in Appendix D):

Capacity to construct and deliver an effective oral argument or performance of

text, based on thorough audience analysis and clear rhetorical objectives.

Capacity to construct effective written argument or media product based on

thorough audience analysis and clear rhetorical objectives.

Capacities to conduct original primary research, locate and critically evaluate

secondary research, and integrate such information into new communication

products.

Capacity to analyze and critique messages in all forms and across objectives

from an informed, critical perspective.

Page 6: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 6

Figure 1: Client Provided Outcome Sample

Additional outcomes being considered by the client:

Ethics and ethical considerations in communication

Knowledge mastery

The ultimate outcome of this evaluation will be to provide a guiding document that can be

used by program leadership to make informed decisions and adjustments.

AUDIENCES / STAKEHOLDERS

A stakeholder is anyone who has interest in the evaluation of the organization and/or can use

the results. There are three levels of stakeholders for the evaluation of the core courses in

the Communication Studies department at UNCW. The major stakeholder of this project is

the contracting client and is identified as the UNC Wilmington Communication Studies

Department. The students currently enrolled in the programs will benefit by the results that

either confirm program efficacy or compel a course correction. Other stakeholders include

the institution at large as they are actively invested in this program as the overseer and

responsible party.

Page 7: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 7

PRIMARY STAKEHOLDERS

UNC Wilmington Department of Communication Studies Faculty

o Faculty members are heavily invested in their programs. Successes and failures

reflect directly upon them. Evaluating the program for the purposes of

providing strategic guidance will directly impact them.

UNC Wilmington Communication Studies Students

o Communication Studies Students have invested considerable time and money

into their education. The better the training offered by the program, the more

equity they will have in their degree. Participating in improvements will directly

benefit them and future classes.

SECONDARY STAKEHOLDERS

UNC Wilmington (University of)

o An effective communications program is an essential component of any liberal

arts college and education. As UNC Wilmington’s largest major, Communication

Studies represents a significant part of UNCW’s College of Arts and Sciences.

Successes and failures will impact the university as a whole.

Communication Studies Alumni

o Alumni of the program have invested several years of their lives as tens of

thousands of dollars pursing training offered by this program. Improvements or

degradation of the program will directly impact the equity held in their degree.

TERTIARY STAKEHOLDERS

Industry employers of UNCW Communication Studies Graduates

o Employers base hiring decisions on the experiences and education of their

prospects. Degrees represent

Page 8: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

LOGIC MODEL

For this evaluation, the designers chose to focus on the program action logic model. The logic model will give the evaluators and

the stakeholders a cleared picture of the evaluand’s background and the purpose for the evaluation. The visual representation

provides a more focused view on the resources, activities, outputs and outcomes for the project, and shows the optimal operations

of the organization.

Page 9: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

EVALUATION MODEL / FRAMEWORK

The Kirkpatrick 4 level training model will guide our evaluation. This model works under the

assumption that evaluation should be conducted with four outcomes in mind: reactions,

behavior, learning and results. However, the constraints of this project will only allow the

evaluation team to look at the first two levels.

REACTIONS

Level one will focus on the reactions of the learners during their time enrolled in the

Communication Studies program. We will focus on the learner’s reaction of their core

coursework.At this level, we will focus on asking questions such as: how do you think and feel

about your core courses in the Communication Studies program? Since this evaluation is

being completed at the end of the participants’ time in the program, it will be assessed in a

retrospective survey. While this may not be ideal and validity of the data may suffer due to

the length of time passed, the reactions measured may prove to provide important data for

the organization and influence their decisions in the future to complete these types of

assessments throughout the program.

LEARNING

Level two focuses on learner’s learning and retention. During this stage we will measure how

much knowledge about the core content in their coursework the students have gained over

the course of the program. The focus of questions inthis level being, “What knowledge do

participants have about the field of Communication Studies?” In this stage, we will work under

the assumption that all results from the survey will be attributed to the program, since there

is no way to go back and give a survey to the students before entering the program. Again, at

the conclusion of this evaluation the organization may see the value in performing such an

entry level test at the beginning of their core courses.

KEY QUESTIONS

1. Do graduating communication studies students possess the necessary knowledge in the

four core subject areas to be contributing members of the field?

2. What are graduating seniors’ attitudes about the communication studies program?

Page 10: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 10

Key Questions Data Collection Method

Data Sources Performance Indicators

Do graduating communication studies students possess the necessary knowledge in the four core subject areas?

Interviews with faculty

Surveys of graduating seniors

Archival Data of student’s prior grades and performance tasks

Communication Studies faculty

Graduating students

Survey results

Interview summary

What are graduating seniors’ attitudes about the communication studies program?

Surveys of graduating seniors

Graduating students

Survey results

EVALUATION DESIGN: ONE-SHOT DESIGN WITH ATTITUDES SURVEY

This evaluation seeks to measure the effectiveness of the core classes in the Communication

Studies curriculum to provide content knowledge to their students. In selecting an evaluation

design, the evaluation team was limited in their ability to choose a design model. The

circumstances of this evaluation included a sample that was already selected and an

intervention that had already been completed. The client’s concern is the effectiveness of the

core classes that the students have already completed. With these factors in mind, we felt

that the one shot design was the best model to utilize. Creating a test that addresses the

knowledge mastery, achieved in the core classes, the team is able to analyze results and

report its findings to the client. This test will be the primary method of measuring the

learning of the participants. An attitudessurvey has also been added to the design in order to

analyze the attitudes of the students as they entered the program compared to the time of

graduation. This survey will be used as a qualitative tool for looking at the reactions of the

students in the program. This evaluation will mostly consist of quantitative data collected

from a test developed by the evaluation team and a group of subject matter experts, the

professors. The retrospective survey will add some qualitative information to the analysis, as

well as interviews with faculty involved in teaching the core courses.

Page 11: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 11

SAMPLE

SURVEY SAMPLE: PURPOSIVE SAMPLING

The sample group for this evaluation was predetermined by the client. Purposive sampling will

be employed by testing each student that is enrolled in the Communication Studies program

and has completed each of the four courses from which the evaluation stems. These sixty-six

students were selected based on the experience in the program and their assumed knowledge

of the core learning outcomes. We have no access to a control group since this evaluation is

being completed after the intervention has already taken place. Each of the students in the

selected sample will participate in a survey and the results will be tabulated and analyzed.

LIMITATIONS

The evaluators identified four major limitations to the evaluation. These limitations have

been taken into consideration as it relates to the analysis of data and preparation of an

evaluation report.

TIME

The short period of time available to complete this evaluation may have some negative effects

on the outcomes and the quality of the reporting. The evaluation team will need to meet with

the faculty to create the assessment items and receive information on the specific core classes.

Understanding that the faculty is busy with classes and other responsibilities, we hope to keep

their commitment to attend meetings to a minimum. Other forms of communication (email,

telephone, videoconferencing) are available, but we do not wish to burden them with

numerous face-to-face meetings. Similarly, college students are busy and it will prove difficult

to schedule required meetings or have them complete surveys in a timely manner. While we

have the full support of the faculty and will have their “encouragement” when it comes to

students, we recognize that this access may be a limitation. After creating a plan and

developing instruments, the evaluation team will also have limited time to collect and analyze

the data. While we will do our best to analyze and report meaningful results, ideally there

would be more time before the deadline of the end of the semester.

SENSITIVITY

Any time student grades and records are being examined, sensitivity can play a major role.

The team understands the private nature of these documents and may not have complete

access to information that could be helpful in analysis. While we have support of the program

and have been granted to archival student data, the team will need to uphold responsible

standards when dealing with sensitive student data.

Page 12: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 12

SAMPLE SIZE

Recognizing that our sample was preselected by the evaluand, sample size and selection will

be a limitation. In our initial interviews with the department chair, they mentioned that they

wanted to use the current graduating seniors for this evaluation andthey were going to have

certain classes pilot the survey. This factor limited the team’s ability to select a completely

random sample, which may affect the reliability of the data collected. We will evaluate a large

group of students, but not a true random sample.

INEXPERIENCE

The inexperience of the members of the evaluation team may also be a limitation to the

outcomes of the project. As graduate students and not professional evaluators, there will

most likely be a number of oversights and inaccuracies in our planning, data collection, and

analysis.

INSTRUMENTATION

This section of the evaluation plan describes the instruments that were developed by the

evaluation team. Each instrument was developed for a specific purpose as it relates to the

key questions addressed in the plan.

DEMOGRAPHICS SURVEY (APPENDIX E)

This survey is included in the evaluation plan at the request of the client. They are interested

matching the results of the learning test with specific demographic information. The faculty of

the Communication Studies program feels that it is important to look for correlations between

these variables and the results of the test. This survey not only includes the usual

demographic information (sex, gender, etc.), but also includes questions related to students’

extracurricular activities, choice of sub discipline, and online course enrollment. Looking at

these factors and the results of the corresponding learning test will provide valuable insight

into the results of the evaluation.

KNOWLEDGE MASTERY TEST (APPENDIX F)

The core purpose of this evaluation is to look at the level of learning of the graduating seniors

in the Communication Studies program. With the assistance of the faculty, a pool of

questions has been developed to measure the knowledge that has been retained by the

participants. The faculty will use the analysis of this data to assess their own teaching

practices and make a decision about adding a new “core competency” to their program goals.

This test will point out specific areas of weakness and strength in the content knowledge that

is considered crucial to the program. The development of this test took dedicated

collaboration of the evaluation team and departmental faculty to create questions that

Page 13: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 13

reflected the outcomes that were seen as vital. It was important to use the proper language

and content specific “jargon” that the students had been exposed to in their course work. The

length of the test was also carefully considered in order to account for the attention span of

the participants.

ATTITUDES SURVEY (APPENDIX G)

In Kilpatrick’s training model the first level of evaluation looks at the reactions of the learners

as they are involved with the program being evaluated. The constraints of this project has

not allowed information to be collected at the most effective intervals, but after discussing

this with the client, it has been decided to conduct a attitudes survey with each of the

participants. This will allow the evaluation team to provide some qualitative analysis to the

Communication Studies faculty and show the opinions and reaction from the participants’

experiences as they have moved through the program. This survey includes questions about

the learners’ perceptions about their level of mastery and preparedness for the workforce.

The survey was designed using a five point likert scale to for simplicity of administration and

analysis.

LOGISTICS

This evaluation plan is being designed and implemented by MIT students Meghan Bennett,

Chip Bobbert, and Michael Webb. This team will be responsible for the completion of each

phase of the project. The final report and analysis will be provided to Dr. Rick Olsen and the

members of the Communication Studies Department at UNCW. The faculty in this department

will be utilized as subject matter experts and Dr. Florence Martin, MIT professor, will guide

and mentor the team throughout the process.

Phase Members Responsible Major Tasks

Planning Evaluation Team

Department Faculty

Initial planning meetings

Evaluation Proposal

Revise Proposal and present to client

Implementation Evaluation Team

Department Faculty

Write instrument questions

Review/Revise Instruments

Create online versions of instruments

Scheduling of tests/ surveys

Page 14: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 14

Analysis Evaluation Team

Collect and review data

Organize data into appropriate database platforms

Analyze and interpret data

Reporting Evaluation Team

Prepare final report based on findings and results

Present report and results to client

TIME LINE

The allotted time for this evaluation is January 23rd through May 2nd. Within this

timeline the evaluators will plan, implement analyze and report their findings to Dr.

Olsen while maintaining appropriate time per task. Listed in the table below are each

major task with appropriate start and ending date.

Phase Major Tasks Start Date End Date

Planning

Initial planning meetings January 23 January 29

Evaluation Proposal January 29 February 14

Revise Proposal and present to client

February 15 March 4

Implementation

Write instrument questions March 5 March 21

Review/Revise Instruments March 22 April 5

Create online versions of instruments

April 8 April 11

Scheduling of tests/ surveys April 12 April 22

Analysis

Collect and review data April 23 April 25

Organize data into appropriate database platforms

April 25 April 25

Analyze and interpret data April 25 April 29

Reporting

Prepare final report based on findings and results

April 30 May 2

Present report and results to client

-- May 3

Page 15: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 15

BUDGET

The costs associated with the program evaluation are listed below. These fees include all

hours for staff that will be planning, implementing, analyzing and reporting the evaluation

results. The budget includes limited outside costs because the survey software is already

included within the Communication Studies program budget and there is no need for

incentive because the students’ are taking the survey as part of their Capstone final grade.

The total estimated costs for this evaluation are $7,875.

Member Role Cost per Hour

Total Hours

Total Cost

Meghan Bennett Evaluation Team $35 70 $2,450

Chip Bobbert Evaluation Team $35 70 $2,450

Michael Webb Evaluation Team $35 70 $2,450

Total $7,350

Cost Description Price Per Total Cost

Mileage .75/mile $225

Supplies Computer Equipment

$300 $300

Total $525

ANALYSIS AND RESULTS

The analysis of the project was performed using the results of the three instruments:

demographic survey, knowledge mastery test, and the attitudes survey. Each of these

analyses was utilized to create a picture of the overall result of the information that was

gathered.

DEMOGRPAHIC INFORMATION

The demographics information was collected at the request of the client and used to correlate

test scores with different categories of students. Each participant completed the survey

before beginning the knowledge mastery test within the same select survey instrument. The

demographic information is summarized in the charts below. However, the correlation results

Page 16: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 16

as each category relates to the knowledge mastery test can be found in the Knowledge

Mastery section of the results.

KNOWLEDGE MASTERY TEST

After completing the demographic survey, each participant was then required to complete a

multiple choice test, which consisted of forty questions. The questions consisted of eight

from each of the core classes determined by the department. (COM 101, 105, 200, 301 & 305)

These questions were pulled from question banks, created by professors, which are used on

final exams. The results from this test were calculated and categorized by the evaluation

team to ascertain the competency of the graduating students.

Overall Scores:

This initial report looks at the overall scores of the sixty-six students who participated in the

knowledge mastery test, without comparison to other variables collected. The table illustrates

the overall score and the breakdown of student performance on each of the five core courses.

The scores collected from this instrument do not show a great deal of success on the

knowledge mastery test as a whole. The evaluation team was able to determine that the

COM 200 score was the strongest out of the group (72%) by a reasonable margin. The

remainder of the scores hovered around the 50% area, with the COM 305 score being the

lowest at 48%.

Knowledge Test

Median Mean Median % Mean %

Overall Score 23 22.9393939 57.50% 57.35%

COM101 Score 4 4.3030303 50.00% 53.79%

COM105 Score 5 4.72727273 62.50% 59.09%

COM200 Score 6 5.75757576 75.00% 71.97%

COM301 Score 4 4.24242424 50.00% 53.03%

COM305 Score 4 3.90909091 50.00% 48.86%

Itemized Scores by Course and Questions

This data table shows the breakdown of each of the questions on the test and the course to

which they are related. The percent correct is included for each question and each question

that has a correct percentage below 30% is highlighted. Looking at this data will allow the

evaluation team to recognize items that may have been written poorly or make

recommendation about information that may have been inadequately covered throughout

the courses and the program.

Page 17: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 17

COM 101 COM 105 COM 200 COM 301 COM 305

1 86% 92% 74% 70% 61%

2 79% 100% 79% 91% 24%

3 8% 15% 41% 47% 38%

4 70% 62% 88% 44% 8%

5 89% 3% 88% 70% 65%

6 17% 92% 67% 70% 77%

7 71% 44% 44% 8% 48%

8 86% 64% 95% 26% 70%

ATTITUDES SURVEY

The attitudes survey was completed at the end of the knowledge mastery test. Each student

answered questions about their experience both with the testing process itself and the

Communication Studies program as a whole. This survey was also requested by the client to

gatherinformation about how they might improve their program moving forward.

This data table illustrates the mean response average of the students on the likert scale

portion of the attitudes survey.

Question Mean Response

Did you feel you knew most of the answers? 3.35

Do you think this capstone helped you to better retain the

information presented in the program?

2.86

Has feedback the instructors have given been adequate? 2.64

Do you feel that your experience in Communication Studies

adequately prepared you for this test?

2.88

Have the classes in Communication Studies been interesting and

clear to you?

2.87

Generally, did you enjoy this program? 2.85

Did you learn to do things that you did not know before? 2.84

Page 18: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 18

DISCUSSION AND RECCOMENDATIONS

The overall results of our evaluation activities were interesting and can prove to be valuable

to the client. Looking through the data we recognized several trends and some correlations

that exist between the mastery test data and the demographic information provided by the

students who completed the evaluation instruments.

KNOWLEDGE MASTERY TEST

Looking first at the overall average score of the students, 57.35%, it is the determination of

this evaluation team that the graduating students are not shown to have a complete mastery

of the knowledge areas addressed in this test. This score shows a need for improvement in the

program focus if these are key concepts that are to be retained when receiving a degree. The

program faculty needs to spend some time reviewing the material in the test and ensuring

that it is highlighted in the course objectives and outcomes. In this practice, it would be

beneficial for the faculty to review of the data from each of the individual core courses. This

data shows that the program’s strength was in COM 200 at 71.97%, while the rest of the core

courses scored in a lower range between 48-58% correct. These results will allow the faculty

to focus on the classes where the results are lowest and where they desire their students’

scores to increase.

Further analysis of the test results caused the evaluation team to closely review the individual

test items. Reviewing a question by question breakdown, there were some questions that

showed a large number of incorrect responses. Each of the questions highlighted in the table

below should be removed from the question bank and reviewed for accuracy. The faculty

should determine whether they are valid assessment items for the specific content and decide

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

COM101 Score COM105 Score COM200 Score COM301 Score COM305 Score

Average Correct by Course

Page 19: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 19

whether they should be kept in the question bank or removed for further graduating group

evaluations.

Ques. # COM 101 COM 105 COM 200 COM 301 COM 305

1 86% 92% 74% 70% 61%

2 79% 100% 79% 91% 24%

3 8% 15% 41% 47% 38%

4 70% 62% 88% 44% 8%

5 89% 3% 88% 70% 65%

6 17% 92% 67% 70% 77%

7 71% 44% 44% 8% 48%

8 86% 64% 95% 26% 70%

Correlations

Test Score

GPA2 Pearson Correlation .229

Sig. (2-tailed) .065

N 66

Transfer Pearson Correlation -.154

Sig. (2-tailed) .217

N 66

Employment Pearson Correlation -.089

Sig. (2-tailed) .477

N 66

The evaluation team was also able to run some correlations with the scores received on the

test and the demographic information collected. The results showed no correlation with

respect to students’ transfer status (p=-.154, r=.217) or their employment hours worked per

week (p=-.089, r=.477). However, the results did show a positive correlation between overall

GPA and test score (p=.229, r=.065) Students with higher GPAs were more likely to receive a

higher test score than students with lower grades. These results show that there is also a

Page 20: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 20

degree of responsibility for the students in the learning process. Students, who are

completing their assignments with a high degree of success, are obviously retaining the

information more readily than others.

ATTITUDES SURVEY

Likert Scale

In this section of the evaluation the students overall reactions to both the knowledge mastery

test itself and the program in general were analyzed. The initial data on the scale questions

were inconclusive. Each of the question’s mean scores hovered around the neutral area of

the likert scale, with the students’ confidence in knowing the answers being the highest rated

at 3.35. These results show that the students’ responses are varied and make it difficult for

the evaluation team or the client to make solid recommendations or reach conclusions.

Open-Ended Responses

The second section of the attitude survey asked two open-ended response questions of the

participants. These questions asked students what they felt were the most useful parts of the

Communication Studies program and what parts might they change. In order to analyze this

qualitative data, the evaluation team was able to group the participants’’ answers into

categories to help better understand and react to the student responses. (See Appendix H and

Appendix I) Looking at this information carefully could help the faculty understand what

adjustments might be beneficial and what components of the program are already strong in

the eyes of their students. Reading through the comments of the students can give some

0.000.501.001.502.002.503.003.504.00

Attitudes

Page 21: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 21

insight into the perceived strengths and weaknesses of the program. Pie charts were also

created to help analyze the frequency of responses in some of the most popular categories.

WHAT WAS THE MOST USEFUL PART OF THE PROGRAM?

IS THERE A PART OF THE PROGRAM YOU WOULD CHANGE?

Compiling and analyzing all of the data from this evaluation, this team is able to make three

firm recommendations for the faculty of the Communication Studies Department at UNCW:

1. As a department, it will be crucial in moving forward to review the objectives and learning

outcomes for each of the core courses of the program. It is important for the department

to have a set of terminal outcomes in mind for their students, and then to apply these

outcomes in the instructional interactions they have with their students. These goals must

be consistent from professor to professor if the department wishes to add knowledge

mastery as a departmental learning outcome.

16%

12%

9%

5%

34%

11%

13%

Capstone/COM400

PR Classes

Specific Skills

COM 200

Speciality AreaCourses

19%

12%

12%

8% 6% 7%

36%

No Changes

Capstone

Gateway Classes

Minor/Concentration

Class Activities

Page 22: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 22

2. If there is a plan moving forward to continue to use a similar assessment instrument for

graduating students, the faculty needs to review each question carefully and make sure

that they address the specific content as intended. There are several questions which need

to be removed or reworded in order for the test to maintain its validity.

3. As a faculty, review the results of the attitudes survey carefully. Heed the comments of

your students, positive and negative, and keep their reactions in mind when making further

decisions about the direction of the program and the coursework within it.

KEY QUESTIONS: REVIS ITED

At the outset of this project, the intention was to use this evaluation to answer two key

questions for the UNCW Communication Studies Department.

“Do graduating communication studies students possess the necessary knowledge in the four core subject areas to be contributing members of the field?”

With the results of this evaluation it was determined the graduating seniors are in need of

some refreshing of their knowledge about many of the core courses. Careful adherence to

learning outcomes and the communication of objectives among faculty could help to increase

student performance in the future.

What are graduating seniors’ attitudes about the communication studies program?

This evaluation program collected a great deal of information about students’ attitudes. The

faculty now has data to review about the participants perceived strengths and weaknesses of

the program. While this information is difficult to quantify, it is important that it be utilized

when making future decisions.

Page 23: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

COMMUNICATION STUDIES PROGRAM EVALUATION

APPENDICES

Page 24: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 24

APPENDIX A: BIBLIOGRAPHY

Larson, M (2013) Communication Studies: UNCW. Retrieved February 17, 2013, from

http://uncw.edu/com/

Page 25: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 25

APPENDIX B: SUB DISCIPLINE COURSE LIST

Page 26: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 26

Page 27: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 27

Page 28: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 28

APPENDIX C: BA REQUIREMENTS

A student may apply to the Communication Studies program and become a Pre-Communication Studies (PCOM) major after earning a minimum 24 semester credit hours and a grade of "B" or above in COM 105: Introduction to Communication Studies. A PCOM major becomes a Communication Studies (COM) major by earning a grade of "B" or above in COM 200: Research Methods. All UNCW continuing and transfer students are subject to this gateway at the time of application.

Core Courses

Flexibility is the key to a communication studies degree at UNCW. The major is designed to afford each student the opportunity to create a program of study around a core of required courses.

To earn a B.A. Degree in Communication Studies, the following courses must be completed. A minimum 2.00 GPA is required for these core courses.

COM 105: Introduction to Communication Studies: For potential communication studies majors. Overview of the discipline and department with emphasis on foundational concepts for describing, evaluating, and practicing communication; and skills necessary to understand, evaluate, and apply research in the discipline.

CSC 105: Introduction to Computing and Computer Applications COM 101: Public Speaking:Preparation and presentation of public speeches, with

emphasis on reasoning, evidence, organization, delivery, and analysis of speeches by self and others.

COM 200: Research Methods:Overview of research methods and how research is used to answer questions and support claims. Included secondary quantitative, qualitative, and critical research. Partially satisfies University Studies IV: Building Competencies/Information Literacy.

COM 301: Rhetorical Theory: Prerequisites: A grade of ‘B’ or above in COM 200. Restricted to COM majors. Introduction to classical and contemporary theories of rhetoric from Plato to the present. Emphasis on applying such theories for greater description, analysis, and evaluation of rhetorical process and outcomes.

COM 305: Communication Theory: Prerequisite: A grade of ‘B’ or above in COM 200. Restricted to COM majors. Theoretical perspectives concerning how and why people communicate. Emphasis on applying theories for greater description, analysis, and evaluation of the human communication processes and outcomes. Partially satisfies University Studies IV: Building Competencies/Writing Intensive.

COM 400: Discipline Capstone

Page 29: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 29

Additional COM Course Work

Majors must also complete 21 hours of additional COM coursework following these stipulations:

A minimum of 12 credit hours (out of the 21 additional hours) must be at the 300-400 level.

A minimum of 3 credit hours (out of the 12 hours at the 300-400 level) must include a 400 level course other than COM 400 and COM 498. (this stipulation does not apply to those registered prior to the 2010 Academic Catalogue)

A minimum 2.00 GPA is required for these additional 21 hours of COM courses. Differently titled courses with the numbers 295 or 495 may be repeated for additional

credit.

With respect to these 21 hours, note the following restrictions on the maximum number of credit hours, towards the COM major, for these courses:

COM 104: Concepts in Communication Studies ( 0 hours ) COM 160: Engaging in Contemporary Media ( 0 or 3 hours ) COM 212: Storytelling in the Community ( 3 Hours ) COM 295: Seminar in Communication Studies ( 6 hours ) COM 491: Directed Individual Study ( 0 hours ) COM 495: Advanced Seminar in Communication Studies ( 6 hours ) COM 498: Internship in Communication Studies ( 3 hours )

A maximum number of credit hours may be counted toward graduation from these courses as follows;*

COM 212: Storytelling in the Community ( 8 hours ) COM 491: Directed Individual Study ( 9 hours ) COM 498: Internship in Communication Studies ( 12 hours )

COM 160: Engaging in Contemporary Media may be used to fulfill university studies or be counted toward the COM major but not both.

COM 104: Concepts in Communication Studies should not be taken if you plan on majoring in Communication Studies.

*A minimum of 124 semester hours of credit and a minimum GPA of 2.00 are required by the University for graduation.

Page 30: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 30

APPENDIX D: CLIENT STATED LEARNING OUTCOMES

(Provided client documentation)

Department of Communication Studies

The University of North Carolina Wilmington

Learning Outcomes

Revised: Fall 2011

The Department of Communication Studies has been working since Fall 2005 on a document

that articulates what the key objectives and ideal outcomes are for students who complete

the major. Coordinating with that internal effort is the larger university initiative to enhance

outcomes assessment efforts on campus. It is our understanding that department strategies

are to be an incremental and sustainable aspect of unit planning and performance review.

We have identified an initial set of key objectives that are common to all communication

studies majors, regardless of sub-discipline emphasis or career objectives. Our approach is an

abbreviated but explicit effort to connect these four departmental objectives with broader

university goals expressed in the UNCW Mission Statement. Particularly salient is the

statement, “Our powerful academic experience stimulates creative inquiry, critical thinking,

thoughtful expression, and responsible citizenship in an array of high-quality programs at the

baccalaureate and master’s levels ….” These four categories of student learning gave rise to

eight learning goals for every baccalaureate graduate of UNCW—foundational knowledge,

inquiry, information literacy, critical thinking, thoughtful expression, second language,

diversity, and global citizenship (http://uncw.edu/assessment/goals.html). These

characteristics clearly work together synergistically within any well-crafted university

curriculum, and we have sought to indicate key connections between that and a department’s

programming. We also noted in the mission statement that UNCW is “committed to diversity”

and striving to be “technologically progressive.” The four learning objectives below also seek

to produce these aptitudes in our students.

Capacity to construct and deliver an effective oral argument or performance of text, based on

thorough audience analysis and clear rhetorical objectives.

Page 31: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 31

This objective directly focuses on a consciousness of diversity in the broadest sense of that

term (e.g., to take into account diversity of identities and values among audiences and

listeners) and for thoughtful expression articulated in the mission statement. Equally

important are the skills associated with information literacy that will prepare the

speaker/performer to be effective.

Capacity to construct effective written argument or media product based on thorough

audience analysis and clear rhetorical objectives.

The change in media fosters change in the process of creation and cognitive processes.

However, this objective also directly focuses on a consciousness of diversity in the broadest

sense of that term (e.g., to take into account diversity of identities and values among

audiences and listeners) and for thoughtful expression articulated in the mission statement.

Equally important are the skills associated with information literacy that will prepare the

speaker/performer to be effective.

Capacity to conduct original primary research, locate and critically evaluate secondary

research, and integrate such information into new communication products.

The ability to carry out meaningful primary research certainly requires each of the attributes

mentioned in the mission statement. In particular, we find vital connections here with the

outcomes of inquiry, information literacy and foundational knowledge, since conducting

primary research implicitly involves observation that leads to raising good questions. It’s also

vital that students identify and critically assess secondary data and have the ability to apply

research findings.

Capacity to analyze and critique messages in all forms and across objectives from an informed,

critical perspective.

Our mission statement reads, in part, “We consider our students potentially responsible

consumers of communication who can use, analyze, and create it effectively.” This learning

outcome was identified to reflect our mission and UNCW’s learning goals of critical thinking

and global citizenship.

Page 32: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 32

APPENDIX E: DEMOGRAPHICS SURVEY

The information below is distilled from the actual instrument and can be found here:

http://appserv01.uncw.edu/selectsurveynet/TakeSurvey.aspx?PageNumber=1&SurveyID=9l2

0no83

Demographic Questions:

1. What is your name?*

2. What is your age?*

3. What is your GPA?

a. 3.6 – 4.0

b. 3.1 – 3.5

c. 2.5 – 3.0

d. 2.0 – 2.5

e. Less than 2.0

4. What core classes have you successfully completed? (Check all that apply)*

a. 101

b. 105

c. 200

d. 301

e. 305

5. Did you transfer to UNCW from another university or community college?*

6. Will you be graduating with a minor?*

7. How are you involved on campus outside your academic coursework and the

Communication Studies program? (check all that apply)*

a. Student organization or committee

b. Student government

c. Greek life

d. Leadership development program

e. Community service

Page 33: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 33

f. Internship (on or off campus)

g. Religious organization

h. I am not involved outside my academic coursework and Communication Studies

8. Have you been employed while attending UNCW and if so, how many hours on average

have you worked over the last school year?*

a. I am not employed

b. 0-5 hours per week

c. 5-10 hours per week

d. 10-15 hours per week

e. 15-20 hours per week

f. 20-25 hours per week

g. 25-30 hours per week

h. 30-35 hours per week

i. 35-40 hours per week

j. Over 40 hours per week

9. What calendar year did you enter UNCW?*

10. What is the average number of credit hours you have taken per semester?*

Page 34: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 34

APPENDIX F: KNOWLEDGE MASTERY TEST

Questions for COM 101

1. Which of the following is characteristic of a speech thesis? a. A declarative sentence b. Not too simple c. Two or more sentences d. Slightly vague for interest appeal

2. Which of the following questions is important to ask when testing the quality of an example that supports an assertion in a speech?

a. Is the example famous b. Is the example researchable c. Is the example typical d. Is the example implied

3. Statistics may be used in a speech to:

a. Pad the speech with information b. Point out trends over time c. Impress an audience with the speaker’s knowledge d. Support an ambiguous assertion

4. Of the following, which one is the single most important part of your speech?

a. The conclusion b. The strongest piece of evidence in the speech c. The thesis d. Either the first or last main point in the body

5. Which of the following most probably gives an audience the most adequate basis for

accepting a beginning speaker as a qualified person on the subject he or she is discussing?

a. A technical rather than a simplified treatment of the subject b. The use of eloquent, beautiful language c. A lot of humor d. Reference in the speech to a wide variety of support materials

6. Which of the following should a speaker not do when either writing a speech or

delivering a speech? a. Search for audience values

Page 35: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 35

b. Use strong emotion when addressing an audience c. Adopt his or her audience’s belief system d. Talk “to” and not “at” an audience

7. Arguing for the merits of honesty is suitable for a persuasive speech of:

a. Fact b. Value c. Policy d. Opinion

8. At the next staff meeting, you must present a detailed 20-minute report on the market

gains made by your company. Stockholders will be present at the meeting, so it is important that you be conversational, easy to understand, and able to respond to listeners’ questions during the speech. What type of delivery style would be most useful to you in promoting this flexibility in your presentational manner?

a. Memorized b. Extemporaneous c. Manuscript d. Impromptu

Questions for COM 105

1. At UNCW, you can major in ____ and for short can refer to your major as ____. a. communications … communication b. communications studies … communications c. communication studies … communication d. communication studies … rhetoric

2. The three appeals are ___.

a. mindfulness, cognitive complexity and rhetorical flexibility b. motivation, knowledge and skill c. value, policy, fact d. logos, pathos, ethos

3. The “official COM 105” definition of communication is _____

a. managing mutual responding b. applying knowledge at the recall and comprehension levels c. shaping ideas to people, and people to ideas d. habits of heart, mind and behavior favored by insiders

4. The habits of justice, transparency, search and respect for dissent are related to which

of the following communication concepts? a. cosmopolitan communication

Page 36: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 36

b. communicating ethically c. constructing relationships d. deconstructing media messages

5. In one of your communication courses, on the first day of class, the professor says the

following: “Communication is simple to understand: You have a sender, you have a receiver. The sender formulates a message and sends it. The receiver receives it and feeds back a response to the sender. That’s the way I learned what communication is, and that’s the way we are going to think of communication all semester.” This professor is a proponent of the ____ paradigm of communication.

a. Psychological b. Social constructionist c. Pragmatic d. Rhetorical

6. Which of the following is NOT one of the five canons?

a. Invention b. Rhetoric c. Arrangement d. Memory

7. Bloom’s Taxonomy of Learning Objectives in the Cognitive Domain illustrates the idea

that ____. a. Application and analysis cannot be performed effectively without relevant

knowledge at the recall and comprehension levels b. Recall and comprehension are the first two levels of what we often refer to

generically as “knowledge” c. As a process of working with knowledge, application is less complex than analysis

is d. [All of the above are correct.] e. [None of the above are correct.]

8. Which of the following statements describe scholarly ways of knowing?

a. They are “natural,” i.e., you need not take any special steps to learn them b. Solipsistic, emotivistic c. Systematic, ordered, question-oriented d. [A and B each describe scholarly ways of knowing.] e. [A and C each describe scholarly ways of knowing.]

Questions for COM 200

1. Making sense of subjective experience is the primary purpose of quantitative

research.

a. True

Page 37: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 37

b. False

2. Content analysis involves analyzing a large sample of texts using a coding sheet that

allows the researcher to measure and count quantitative features of the texts in the

sample.

a. True

b. False

3. Statistics can be used to describe but not identify differences or correlations; that is

up to the interpretation of the researcher.

a. True

b. False

4. Quantitative research focuses on reliable and valid measurement of variables of

interest to the researcher.

a. True

b. False

5. While it is important to plan questions for a qualitative interview, the key

information will likely emerge from follow up questions and dialogue with the

participant.

a. True

b. False

6. The proper sequence for the steps to any primary research project is preparation,

engagement analysis, and expression.

a. True

b. False

7. The four levels of measurement are nominal, ordinal, equalized, and rational.

a. True

b. False

8. If Chris is asked by his boss to conduct a focus group with members of a new market

the company is thinking of targeting he should begin the project by doing some

secondary research about the market and key variables.

a. True

b. False

Page 38: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 38

Questions for COM 301

1. The study of human character, one of Aristotle’s three artistic proofs, is called:

a. Pathos

b. Ethos

c. Logos

d. Epideictic

2. Cicero’s canon that deals with how to organize a speech and the order in which it is

presented is:

a. Invention

b. Arrangement

c. Style

d. Memory

e. Delivery

3. Which of Aristotle’s rhetorical settings deals with the present, virtue, and vice?

a. Epideictic

b. Judicial

c. Forensic

d. Deliberative

4. The three types of medieval rhetoric are letter writing, sermons, and:

a. Novels

b. Poetry

c. Music

d. Speeches

5. The art of letter writing, often used to petition a person of higher authority, is one of

the three types of __________ rhetoric.

a. Ancient

b. Medieval

c. Christian

d. Renaissance

6. Wearing the appropriate clothes to a job interview is an example of what Cicero calls:

a. Invention

b. Decorum

c. Pathos

Page 39: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 39

d. Stasis

7. According to Burke, the elements of the pentad most helpful when they are combined

in ________ to demonstrate the dynamics of a particular rhetorical act.

a. Ratios

b. Groups

c. Dialogues

d. Constraints

8. In Kenneth Burke’s dramatistic pentad, what term describes the means by which the

speaker performs the act?

a. Agency

b. Act

c. Agent

d. Purpose

e. Scene

Questions for COM 305

1. Knowledge gained through subjective means as opposed to objective means is most consistent with:

a. Scientific scholarship b. Humanistic scholarship c. Social scientific scholarship d. Critical scholarship

2. The theory of XYZ has been criticized on various grounds except on the ground that it

has failed to generate considerable research. In short, one could say that it cannot be legitimately criticized for failing to perform which function?

a. Focus b. Observational c. Heuristic d. Control

3. One-semester researchers gave students in four different speech classes the same pre-

test to measure their level of speaking anxiety. Afterwards, three classes were administered three different semester-long programs designed to help each class reduce their level of anxiety. A fourth class was given nothing. At the end of the semester, all four classes were given a post-test to measure students’ level of anxiety. In this study, what was the independent variable?

a. The I.Q. of each student

Page 40: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 40

b. The level of anxiety of each student c. The post-test given to each student d. The program employed to reduce anxiety

4. Language theorists who believe that persons-in-conversation co-construct their own

social realities and are simultaneously shaped by the worlds they create are known as: a. Social constructivists b. Social constructionists c. Social interactionists d. Social observationists

5. The idea that persons-in-conversation co-construct their own social realities and are

simultaneously shaped by the worlds they create most closely describes which theory? a. Uses and Gratification Theory b. Coordinated Management of Meaning Theory c. Communication Accommodation Theory d. Agenda-Setting Theory

6. This theory assumes that when strangers meet, their primary concern is one of

increasing predictability about the behavior of both themselves and others: a. Coordinated Management of Meaning Theory b. Uncertainty Reduction Theory c. Expectancy Violations Theory d. Interaction Appearance Theory

7. Whenever a theory causes an individual to notice something about the world that he or

she had not previously noticed prior to reading the theory, we say that the theory has

served which function?

a. To summarize

b. To focus

c. To predict

d. To control behavior

8. Joe Researcher recently took an undercover job on the assembly line at a Mitsubishi

automobile plant in Japan in order to learn more about Japanese communication styles and strategies. What type of research was Joe engaged in?

a. Classic b. Problematic c. Experimental d. Ethnographic

Page 41: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 41

APPENDIX G: ATTITUDES SURVEY

Using a Likert scale, 1 being strongly disagree, 5 being strongly agree, tell us how you feel about the following statements

1. Were the questions on this test clear? 2. Did you feel you knew most of the answers? 3. Do you think this capstone helped you to better retain the information presented in the

program? 4. Has feedback the instructors have given been adequate? 5. Do you feel that your experience in Communication Studies adequately prepared you for this

test? 6. Have the classes in Communication Studies been interesting and clear to you? 7. Generally, did you enjoy this program? 8. Did you learn to do things that you did not know before?

Short Answer 9. What was the most useful part of the program? 10. Is there a part of the program that you recommend to be changed? What and how?

Page 42: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 42

APPENDIX H: MOST USEFUL PART OF THE PROGRAM QUESTION RESULTS

Capstone/COM 400

1. I am unsure what program this survey is talking about. I'm assuming it's the capstone

course. I think the most useful part of the program has been incorporating concepts

from past courses into our projects.

2. I belive that the capstone course for our major is very unique and is a good way to

prepare us for our next step after graduation. The capstone has allowed me to apply

everything I have learned in the major and learn how to use the information I have

learned and skills I have aquired to my advantage.

3. Probably the assembly of the portfolio in COM 400

4. COM 400

5. Com 400

6. Public Relations 2 and Capstone

7. Capstone probably had the most concrete and real-life information that I will be using

once I graduate.

8. COM 400

9. The capstone course

10. COM 400

PR Classes

1. Hands on projects in PR and IMC that are applicable to real life experiences.

2. Jennifer Chin's PR classes.

3. The focused classes such as Integrated Marketing Communication I and II as well as both

levels of Public Relations. These courses provided us with the knowledge to move

forward with our clients while also leaving us enough room to be creative and hone in

on our own unique skill sets.

4. To me the most useful part of UNCW's Communication Studies programs were the

higher level PR and IMC courses I took. It was really rewarding for me to be able to take

the theories and concepts I learned in the classes prior and be able to apply that

information to help satisfy and actual client with their organization's needs. COM 200

was definitely helpful as well and I returned back to my texts form that class many times

to help me figure out and decipher various types of research.

5. My integrated marketing communication course, my public relations course, and my

capstone course that prepared me for my all of my job interviews.

6. Public Relations 2 and Capstone

7. Honestly, I can think of one. I like Public Relations, learned a little in there, but it doesn't

have too much to do with COM. Might as well only be taught in the business school. In

fact after taking many of these classes I wonder why UNCW keeps this department.

Page 43: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 43

Most of them are tame off shoots of other departments. Had I had a good adviser, I

would have tried harder and gone into the business school.

8. Public Relations II and IMC II are the best classes I have taken in the Communication

Studies Department. Dr. Persuit and Chin provide the best opportunities to learn and

grow as students. I think that it is important to get hands-on experience and practice

with what you really want to do. I want to go in to PR so these classes were most

beneficial. The professors also seem to actually care about what they are teaching you.

They give you great feedback so that you know what to do in the future, after

graduation. I really like the fact that you have freedom and that you get to work with

real-life clients. I also really enjoyed writing as strategic com.

Test Related Answers

1. It made me think back to my classes when I was younger

2. The true and false questions

3. The most useful part of this program was the fact that it presented students with

information from all COM core classes. I have not seen some of those questions in

quite some time so it was good to get the mind going again.

4. Helped me think more about past courses to try to recall information.

Specific Skills/Class Activities

1. Classes that emphasize applied learning.

2. Teamwork promoted in all class levels, Applied learning

3. Hands on projects in PR and IMC that are applicable to real life experiences.

4. Learning about the communication process.

5. Public Speaking because it really gets you confidence.

6. Application of theories

7. Learning and understanding different theories and applying them to real life scenarios.

In a way it allows us to see why things happen and how the concepts came to be.

8. The interpersonal classes and learning abut real life things that could help me when I

look to apply to a job.

Misc

1. Not applicable

2. Everything

COM 200

1. To me the most useful part of UNCW's Communication Studies programs were the

higher level PR and IMC courses I took. It was really rewarding for me to be able to

take the theories and concepts I learned in the classes prior and be able to apply that

information to help satisfy and actual client with their organization's needs. COM 200

Page 44: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 44

was definitely helpful as well and I returned back to my texts form that class many

times to help me figure out and decipher various types of research.

2. COM 200, learning how to do research.

3. Com 200, Public speaking

4. I truly did like the fact that we were expected to take both communication and

rhetorical theory, as I felt both taught me a lot in different aspects of the

communication studies department and overall understanding communication as a

whole. Also I like how COM 200 is the second class requirement as opposed to using it

as a senior level class.

General Program Statements

1. The variety of classes we are able to take within this major has given me an extremely

well rounded base. The professors are excellent and passionate.

2. The ease with which we're allowed to construct our own programs and disciplines

within the major

3. The resources and professors here are great. They really enriched my experience here

at UNCW.

4. The professors's attitudes and the fact that students are treated as family. I never felt

that the professors were just trying to pass the students along they really helped make

the material interesting and wanted the students to learn.

5. Developing stronger interpersonal relationships

6. The ability to become comfortable talking I front of groups of people and expressing

my ideas and thoughts adequately.

The future

1. Providing the tools for a successful launch into the work force...and life in general.

2. Preparing for graduation and future careers

3. I gained a lot of knowledge through classes, my internship, and COM studies day that I

know will be beneficial when I graduate, as I am already seeing their relevancy to my

life.

4. The interpersonal classes and learning abut real life things that could help me when I

look to apply to a job.

Other classes

1. The focused classes such as Integrated Marketing Communication I and II as well as

both levels of Public Relations. These courses provided us with the knowledge to move

forward with our clients while also leaving us enough room to be creative and hone in

on our own unique skill sets.

Page 45: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 45

2. I found things in each class that I've taken to be useful in some capacity. The most

helpful class though has been IMC 1 and 2. I've learned more in those two classes than I

have in any other.

3. The various course subjects offered to students.

4. My internship, COM 498. It helped better prepare me for the real world corporate life

and to connect in class objectives with hands on experience. It also help me realized

my strengths and weaknesses.

5. My integrated marketing communication course, my public relations course, and my

capstone course that prepared me for my all of my job interviews.

6. The com theory class was one of the most interesting and helpful classes in the

program. Aside from that, I think the flexibility and variety of classes offered make the

com department quality.

7. Increased awareness of the world around me and the vast variety of communication

styles one may encounter.

8. learning how to become an effective communicator and my capstone course has been

very helpful because it has prepared me for entering the real world

9. Getting comfortable publicly speaking and also getting comfortable working with

groups in the professional sense in order to prepare for similar situations in the work

world after graduation

10. I enjoyed the specialty classes more than anything, including On Camera Performance,

PR1, IMC1 and TV Asthetics. Those classes I felt like taught me more than anything in

the core classes because they were skills to obtain.

11. The most useful part of the program I think are the class selections. There are a lot of

different classes which allows us students to find our passion within Communication

Studies.

12. I felt that the most useful part of the Communication Studies program in all honesty

were the core classes. In hindsight I remember being intimidated by the tough classes

that everyone always talked about but after experiencing them first hand I believe that

I have been exposed to many thought altering theories that I find myself using on a

daily basis. I think of the program as an eye opening experience that broadened my

view of the world. The biggest change I have noticed in myself now that I am near

graduation is that regardless of the situation, I always think of what the other person

may be experiencing and I take that into consideration before generating a response;

positive or negative.

13. The gateway classes. This program makes it so helpful for students to gain the proper

information they need to move forward. As a student in 400, I feel as if this whole

major has prepared me for my next step.

Page 46: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 46

APPENDIX I: PART OF PROGRAM TO CHANGE QUESTION RESULTS

No

1. NO

2. I cannot think of anything I would change.

3. No

4. No.

5. No

6. I wouldn't change anything about the program

7. no.

8. No

9. Not applicable

10. No

11. I don't think I would change anything

12. No

13. No

Capstone

1. Capstone classes should have a unique course description for each section. Also

capstone should be more structured across the board. There is too much variety in each

section and some sections are much harder and time consuming than others.

2. Capstone would be more helpful the semester before the semester we are scheduled to

graduate. Additionally, the capstone class seems to be a lot of bending the individual

professors requests, and not actually preparing us for what we will face in the so called

real world. I don't think capstone should be set up that way. A few people in my class

are having to build their portfolio for a job they don't want because the job they are

going for isn't approved. This was really upsetting to find out because this is such a

versatile major. It is ridiculous to limit the students who have a different life plan than

the mainstream crowd. We should be encouraging the differences, not just trying to fit a

mold. We learn how to fit the mold of society the other 16 years we are in school, it

doesn't need to be reinforced. Creativity and ambition should be encouraged.

3. COM 400, require students to take the course the semester prior to their last so that it

actually helps them prepare. Those who are attending graduate school are at a huge

disadvantage because their applications are already submitted. Nothing done in class

assists them in selling themselves through the application when the course is taken in

the final semester. I wish my advisors had been able to share this information with me.

4. I would improve the consistincy of the capstone course. I think the material and

information given can sometimes be subjective to who is teaching the course. For

Page 47: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 47

example, formal busniess attire should be consistent thoughout the capstone course;

not based on personal opinion of the professor.

5. Capstone! I think that the capstone courses should all either be the same syllabus, or

that they should be advertised what the focus of each course is. My capstone is focused

on leadership and we are even reading a leadership textbook, but I have already gained

my leadership experience through multiple student organizations and course work. Plus,

not everyone wants to end up being a manager or a professor. I wanted a practical

capstone course. I wanted to practice interviewing more, to hear and learn more about

salaries and how to negotiate them, to learn everything about preparing for graduation.

I think my professor is great and I have taken her for other courses, but I am unhappy

with what I have gained from this course. I also think there should be time near the

beginning to go through resumes individually with the professor.

6. There should be a set of requirements that professors MUST cover in COM 400. Guest

speakers and meetings with career center reps are done by many professors but not all.

I believe some people get more out of the Capstone based on who their professor is.

7. I would change the structure of COM 400 because I think it is good to know interview

skills etc but I know one day Dr. Olsen came in to teach our class and I loved it because

he overviewed things we had learned before. After 4 years there is a lot that is forgotten

and it was nice to be refreshed about that information before I go out to talk about

potential employers.

8. The Capstone and make it a little more useful and helpful

Professors

1. The teachers. Sorry, but a lot of you don't need to be there. Now don't get me wrong, all

of you are intelligent. It's just some of you are either way to busy with your own projects

to care, don't teach at a college level because you're too busy being trying to be our rad

old teacher friend, or you need an assistant to help you teach.

2. Ironically, I would change the communication. All teachers in the program think they are

such great communicators, when in reality, they cannot clearly communicate

assignments, a specific syllabus, or even concepts in a way that is sometimes even

understandable. I also think the department could do a better job promoting their

special events and promoting their clubs and involvements. These are literally the only

two complains, because everything else has been quality.

Gateway Classes

1. COM 200 needs to be smaller, separate classes.

Page 48: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 48

2. The interaction and involvement amongst students in 105 and 200 to make it more

enjoyable for students.

3. I did not like waiting to complete gateway courses before entering courses I wanted to

take before I graduated. I felt it put me behind

4. CSC 105 shouldn't be necessary to obtain a degree in Communication Studies.

5. COM 200: it needs to be more of a review of COM 105 and then start teaching us

research methods, but we nee to continue to learn and use them throughout the rest of

the program.

6. I would make a slight change to the must make a B in COM 105 and 200 gateway.

Overall I believe that the students should achieve an average of B between the two

courses. I feel like this would show you which students truly care about making it

through the gateway. I also believe that the pre-com advising system needs to be

completely revamped. The information provided by advisors is always extremely helpful

and accurate, yet I couldn't help but find myself extremely annoyed by the two+ hour

long wait. Several times I had to leave after waiting an hour because I would have been

late to work. I also was less motivated to ask questions and seek clarification from the

advisors because of the incredibly long wait.

7. I'm not really sure I got a lot out of COM 105. The actual COM classes were far more

interesting and applicable than the information in 105.

8. I didn't like waiting to pass certain classes in order to take other classes in the major. I

felt it made me behind and created stress.

Test related answers

1. I can't remember a lot of the concepts from before

2. I was completely unfamiliar with the content of some of the questions in this test, which

makes me think that what is taught in core classes is sometimes inconsistent. I've found

that because I took some core classes earlier or with a different instructor than my

peers I did not learn all of the same things. Having more consistent class content among

professors for each course may be helpful.

3. I would change the wording of some of the questions. Some of them were lengthy and

worded in a confusing manner, resulting in confusion on my part.

4. Should get feedback on correct or incorrect questions so we can see how well we did.

5. I did not enjoy the questions on com 101 because it's been a while since i took that class

Minor/Concentration

1. I wish that some fields of study were more emphasized i.e. event planning, additional PR

courses, advertising & marketing, etc.

2. Probably the fact that you don't have to declare a concentration, I think it should be

required to have one.

Page 49: OMMUNIATION STUDIES PROGRAM EVALUATIONmeghanbennett725.weebly.com/uploads/5/3/2/5/... · 4/25/2013 ommunication Studies Program Evaluation 5 Specifically, we seek to integrate application

4/25/2013 Communication Studies Program Evaluation 49

3. I would make sure that all communication theories studied throughout the major are

emphasized in every course.

4. More classes to concentrate in rather than just an overall Communication Studies

Degree.

5. I wish there were more distinct pathways or concentrations you had to choose similar to

how the Business School works. I felt like I came out of the major with a wide variety of

information but because of that I wasn't as sure on what to focus on and had to look

outside of school to figure that out.

Class activities

1. Often concepts are not reviewed. This is difficult because the material from past

courses have sometimes not been discussed in over a year.

2. Less group work

3. Less emphasis on details - yes, they're important, but are they really THAT important?

Keep it simple, stupid.

4. What material we present during presentations

Miscellaneous

1. I wouldn't change any part of the program per se. I would however offer some of the

popular classes more often so more people have the option of taking it.

2. I think an internship experience should be a senior requirement.

3. Rhetorical theory was a course that I thought could have been improved if the concepts

taught had been better linked to modern day examples. The terms, naturally, were most

foreign and were harder to understand easily.

4. I would just like to see the COM dept. expand. I love our department and would love to

see it as big as other schools and make it a more credible major to those not in the COM

dept.

5. The amount of mandatory classes.