Illinois State University Writing ProgramProgram Wide Assessment: ENG 145 (fall 2013) and ENG 101 (spring 2014)March 2013

General Description: The ISU Writing Program is designed to teach students concepts and techniques for writing in different kinds of situations and genres. Thus, our curriculum focuses on the learning and adaptation of these concepts and techniques, along with efforts to help students understand exactly how these skills can apply to their work as writers in different genres. Students in Writing Program courses investigate (in great detail) their own productions, learning to observe and assess their writing as it transforms in different settings at the same time as they learn techniques for producing effective writing in these different settings. While we do not expect that students will be able to employ these skills to create “perfect” (or nearly-perfect) productions by the end of the semester, we do expect that students will be able to make choices that are intelligent and that use observational and background research data to adapt their writing to new situations. We also expect that they will be able to assess, to some degree, how their productions do (or don’t) conform to the requirements of particular writing situations.

The ISU program-wide assessment is designed to allow all members of the Writing Program (including students) to assess how primary concepts are being taken up and used in our ENG 101 and ENG 145 classrooms. It is also designed to allow the Writing Program to review and assess (1) students’ ability to think critically about writing and how it works, (2) their ability to engage in rigorous exploration of different kinds of literate situations, and (3) their ability to assess their own productions for correctness and appropriateness for the genre/situations in which they are writing. Finally, it allows us to highlight concepts and skills that we may want to focus on more specifically in future semesters of the course.

The general goal of our program-wide assessment is then to observe and assess both the research and critical thinking skills that students use as they confront new writing situations, the choices these students make vis-à-vis the genres they are working to investigate.

Assessment Design: The assessment is designed as an exercise in “problem-solving.” Students will participate in groups to respond to writing scenarios1 that require certain kinds of skills at researching and making decisions about content and structure of particular genres, in response to particular literate situations. We are interested in the entire process of how students respond to these scenarios, so students will be asked to show their work as they solve the scenario.

Logistics for Completing Responses:

1 At the beginning of the semester, teachers and students will be given materials that explain the assessment, its goals, and generally describe how the assessment will work.

2 Instructors will be given a list of possible scenarios. They will select the groups and the scenarios that each group will work on. [We decided on this process because it allows instructors the

1 Most of the scenarios are designed for group work and ask the students to solve particular problems in writing and then produce some kind of text in response to the prompt. However, instructors will have the option to allow students who request it to work individually. To accommodate this possibility we’ve designed some assessment scenarios that can be completed individually.

most flexibility to use their knowledge of the class and students to create groups that will be effective -- our goal is to create groups that can complete the assessment successfully].

3 Instructors can create the number of groups they feel are appropriate, and also decide on numbers of students in each group. We’re estimating it might about 5 groups per ENG145 class, but the final decision on these issues is up to the instructors

4 Students will spend approximately 1-3 weeks completing the scenarios, and this work can happen at any time during the final three weeks for the semester. (NOTE: See the handout on class planning for the assessment for more information). The groups (or individual students) will be asked to include a specific kind of written production in response to the scenario, but they will also be expected to include additional information, and they will fill out a “genre research” analysis that asks them to answer specific questions about their work process and their final product. [Instructors who already use a very specific, repeated analysis process for genre research in their classes will need to review our form to figure out how your process will map onto the form we’ve provided. You should be able to ask students to use the same analysis process to answer the questions on our form – we’ve left it pretty open-ended on purpose, specifically so that we can get at the different activities that instructors may be using.

5 Instructors will also be asked to provide some additional data in an online form. In this form they’ll be asked some very basic questions about the groups (so instructors can let us know if they feel the groups are outliers in any ways). Also, they’ll be asked a few short questions about their teaching practices. The form shouldn’t take more than 1 hour to complete (at most) and won’t require any long narratives about specific students in the class (see Appendix C for the information that we’ll be asking instructors to provide).

6 Both student materials and instructor assessments will be uploaded through a web-based database. However, we will also provide instructors with large envelopes, so that if students have any kind of printed materials that they wish to include (e.g., handwritten notes, printed copies of the production, or other artifacts) they can do so.

7 Instructors will also distribute and collect a “permission to use excerpts” form that explicitly allows (or doesn’t) the Writing Program to use excerpts from the scenarios in published materials. Students and instructors will fill out these forms, sign them, and use their numbered code as identification. This will allow the Writing Program Office Manager to determine which coded responses can be used for excerpts.

Student and Instructor Anonymity:Since our goal, as a program, is to understand what we are teaching, and what students are learning, rather than evaluating the specific work of any one student or instructors, we have decided on an assessment process that allows the responses of students and instructors to remain anonymous. The reason for this procedure is to protect students and instructors from any fears of grading or employment penalties resulting from responses to the scenarios. In addition, it will allow us to provide students and instructors with the results of the reviews, but without linking these results to any one instructor or single group of students. This anonymity thus protects students and teachers, while at the same time leaves the Writing Program free to investigate a range of different aspects of student work, and to make decisions about the future direction of the program based on analysis of the assessment data. For example:

1. We’ll be able to capture an overall view of student knowledge-making.2. We’ll be able to determine which areas of the program pedoagogy might need additional review

or new/different resources.3. We’ll be able to determine which concepts and strategies we might need to focus on for

additional professional development.

2

4. We’ll even be able to run different kinds of data analysis -- perhaps comparing the ways that students are accomplishing different kinds of tasks; or the error patterns that are more frequently occurring in student writing.

However, because we won’t have access to any instructors’ names, the decisions about specific classroom practices (i.e., how instructors might change any personal teaching practices) are left where we think they belong – in the hands of the instructors. In order to make sure that this anonymity is preserved, we’ve developed the following plan:

1 The instructors will be given randomly assigned code numbers.2 Instructors will need to create code numbers for their students that match the group numbers.

For example, if your number is 063, you’d create group numbers that matched your code: 063-1, 063-2, etc. You will need to make sure students put their code numbers in the database when they upload their assessments (it will just be an entry in the online form where they’ll enter their number). This way we can match up students with instructors without access to anyone’s name.

3 We also want to give both instructors and students the ability to limit how we can se the data we collect. On the online submission form, each student group will be asked to decide whether they are willing to allow the writing program to use excerpts from their assessment document in our reports (generally no more than a couple of sentences or a small portion of a piece of writing). If any member of the group is uncomfortable with allowing us to use excerpts, they can choose the other option, which allows us to use their materials to create aggregate data, but not to quote specifically from any of their materials. Instructors will also have the opportunity to limit our use of their statements when they fill out their instructor form.

4 All of the uploaded responses (and any printed materials) will be identified ONLY by number (instructor number and student group number).

5 As the reviews are completed (see more information on the review process below), reviewers will see only the code numbers, with no access to individual identities.

6 Members of the Writing Program Administration Team will also see only coded numbers and not individual names.

7 Since there will be at least two reviews of each student group, these individual reviews will be collected into a “report,” which will be printed out and placed into envelopes by the instructors’ codes. These reports will be given to the Writing Program Office Manager. Instructors can pick up their envelopes from the Writing Program office (for the ENG145 assessment, reports will be available in spring 2014. For the spring 2014 assessment of ENG101, the reports will be available at the beginning of the following fall semester).

8 The responsibility for reviewing the assessment and making decisions about teaching – what’s working (and not working) and how to improve the overall class design, or whether to share successful designs with the program -- will be left to the individual instructors.

9 Group reports (by coded number only) can also be made available for students to review (in print form only, in the writing program). An Inforgraphic report that includes information about the entire assessment will be posted in the Writing Program (in a poster form) so that interested students can review information. A written report will also be posted to the Writing Program website.

Assessment CriteriaThere will be a range of areas for review, which will include both program Learning Outcomes and other program goals that we want to assess. For the ENG 145 assessment, these areas will include the following:

3

● Identifying Genres: This generally includes the ability to figure out the “requirements” or “boundaries” of particular kinds of writing one has been asked (or has decided to) produce. It includes the ability to find examples of the genre (when possible) and to do close analysis of the features of the genre vis-à-vis other genres (similar and dissimilar) and in relation to the antecedent genres that an author might be able to use to understand the new situation.

● Creating Content (within a genre or for a setting): This generally includes the ability to fit one’s content into the setting/genre. What have the authors left out? What have they included? How does one’s research about the topic support and shape the content one produces, and how is one’s research shaped by the requirements of the writing genre or setting? Does the group’s ability to create and shape their content reflect a clear understanding of the requirements of the genre and setting?

● Awareness of Language Conventions: This will include analysis of grammar and syntax patterns students use. Matches in grammar and syntax to the conventions of the genre will be reviewed as well a potential mis-matches.

● Flexible Research Skills: This skill is generally about knowing where to look and how to look for information, as well as how to document what one finds when researching. We’ve divided this category more fully into (a) genre research – or the tactics for finding out about and investigating genres and settings; and (b) content research – or the ability to find content materials that support a particular writing task.

● Citation and Citing: This includes not only general knowledge of academic citation practices but knowledge of the different ways that citation can work in different genres and settings: making choices about how to cite, knowledge about how to cite “correctly” in different settings, and knowledge about how to analyze genres and settings to determine appropriate citation practices.

● Discourse Communities: Generally, this knowledge includes the ability to examine how groups of people who produce and use texts in particular settings understand and enact those productions and uses. Understanding discourse communities is not simply about understanding the “audience” for one’s text. Instead, it’s a way of investigating how institutions and genres can be formed and supported by groups of people all doing things together in certain ways.

● Trajectory: This term is related to temporally rich understandings of the different ways that texts “make meaning in time.” It means a having a nuanced understanding of the ways that texts come into being and the ways that their production and distribution can diverge (sometimes dramatically) among different instances of genres or over time.

● Technologies and Modalities: This term relates to the ability to consider how different technologies for production and different modes of production (graphic, alphabetic, aural, oral, etc.) impact literate productions.

● Globalization, Cultural Contexts & Diversity: This term relates to the ability to consider complex contingencies that impact particular genres and settings of writing with regard specifically to cultural/geographical/social/technological differences.

● Ethics: The ability to consider possible ethical complications of producing writing in various literate settings – and the ability to make decisions about how to deal with these issues.

● Terms and Concepts: This involves the use of specific program terms and concepts (we’ll create a specific list) related to our Writing Program pedagogy. It is not that we are looking for students to be able to use all of the terms, but that we are interested in examining what terms they do use, and how terms are used. We’ll also be looking for the use of terms not on the list, but which relate to concepts we are interested in.

4

● Disposition: This will be a review that involves the reviewer developing an overall sense of how students are taking up skills, terms and concepts in terms of their personal identities. Some markers that we might be looking for could include:

○ Language that illustrates how students are considering themselves as researchers (both for genre analysis and for content)

○ Language that illustrates student attitudes towards the activities of research (esp. genre research)

○ Language that illustrates how/if students are making connections between the research they are doing and “ownership” of their own productions in different settings

○ Anything that might point to “intent” or “ability” to use learned skills in a range of literate settings beyond the classroom.

The Scenarios:There will be four general categories of scenarios:

● Academic Writing Settings● Social Writing Settings● Workplace Writing Settings● Topical Research Settings (these are settings where the primary focus is on “finding

information,” but where the setting in which that information is used will be less defined).

Different Scenarios may highlight different assessment criteria and may include only some (and not all) of the assessment criteria. Each scenario will include a description of the particular assessment criteria that it focuses on and includes. When instructors are given the Scenario list, they will be asked to make sure that their selections for all of the groups combined end up representing the full range of assessment criteria.

Examples:● Some scenarios may include a primary task such as deciding what genre is the best

genre to use for the production of particular content – given certain situation constraints. So the FOCUS of this kind of scenario is “identifying genres” and specifically thinking flexibly about how different genres might work to present content in different ways. Other assessment criteria would also be included in these scenarios; however, this kind of scenario might not focus specifically on ethical considerations, or global/cultural contexts, or citation (although it could).

● Other scenarios might focus primarily on “research” – that is, researching a topic and then considering how to present the topic to different audiences. These scenarios would overlap in many ways with “genre research” scenarios, but they would have a different focus (on content research) that could be expected to shape student responses.

● Finally, although not all of the scenarios will focus on criteria like understanding citation, or considering global contexts, we will have some scenarios that do focus specifically on these criteria.

The Review Process: Logistically, the review process will be completed in the following way:

5

1 Reviewers will access scenarios online – each reviewer will be assigned multiple scenarios (we’re not sure yet how many) and each will be asked to review all of the responses to the assigned scenarios, so he/she will be looking at responses from groups in different sections of ENG 101.

2 Each scenario will have at least 2 reviewers. 3 Reviewers will be using a form that asks them to respond to the scenarios using the assessment

criteria (see below).4 The reviews (2) for each groups’ responses can then be collected and compiled into an envelope

(we’ll be using printed materials for this to help support instructor and student anonymity).5 Instructors will be able to physically pick up a packet that includes reviews of all of their student

groups, along with overall information for the writing program as a whole. The overall program review will include reviews by scenarios (so review totals for each scenario, across sections) as well as reviews organized by learning outcomes (sub-divided by scenario). The reviews will also include excerpts from different responses to serve as examples of what reviewers were looking for, what they found, and how they ranked the different responses.

6 If possible, we’d like to “publish” results of the program-wide assessment, as well as information by coded group, so that individual students can discover how their groups performed in the review process. Since the rankings are not linked to grades, we felt that making this information available might be useful for students to self-assess their learning in the course. Thus, we will invite students to request printed reports of their groups assessment, which they will be able to pick up in the Writing Program office.

Scenario Description:

Student Group Number:

Instructor Number:

N/A for scenario

Expected but not found

Evidence of requirement

Exemplary response

Please include example that supports your assessment and explain in general your reasons for your ranking

Identifying Genres

Creating Content

Flexible Research Skills

Citation and

6

citing

Discourse Communities

Trajectory

Technology and Modality

Globalization, Cultural Contexts

Ethics

Terms & Concepts

Disposition (affect)[Note: This criteria will not be ranked but evidence of disposition should be considered]

Additional Data Analysis:Because we’re collecting this assessment data in online forms, we’ll actually be able to do many different kinds of analysis on the different scenarios and different learning outcomes. The following list includes some (but definitely not all) of the kinds of “searches” we might engage in using the data:

Running “error” analyses that look at which kinds of scenarios or genres seem to generate more (or less) different kinds of sentence-level errors.

Looking at students’ ability to “match up” their understanding of their production (it’s strengths, weaknesses, or “genre requirements”) with reviewers assessments of the same products.

Making numerical counts of which terms students use most (or least) from our list of important terms and concepts. Then perhaps even connecting those counts with which groups seemed most able to complete the scenarios successfully, thus establishing (or not) a correlation between the ability to use and understand terms and concepts and the ability to more generally investigate literate situations (this kind of analysis would be connected to Meyer and Land’s work on “threshold concepts”).

Looking at which scenarios (if any) seemed to end up with higher “scores” than others…trying to answer questions such as, “are some kinds of writing harder to investigate than others?”

Analyzing the comments students make about antecedent genres (stuff they already know how to do) and how they applied that knowledge to the scenarios, and using that information to try to understand whether an ability to connect accurately to appropriate antecedent genres affects either success in understanding the new genre, or perhaps simply “perceived success.”

Obviously, the aspects of the data we decide to analyze will be based on what we (as a program) think is most important, or perhaps what we feel we need to analyze in order to defend our program to outside institutions or to administrative authorities within ISU. However, once the data is collected, we would

7

also be able to give access to researchers who want to use the data for their own research projects – or just for their learning about assessment or different ways to use data for researcher purposes. The Writing Program is definitely interested in hearing from instructors who would be interested in using the data for their own work, and we plan to support this kind of work financially, perhaps with assistance for conference presentations, among other things.

Permissions:As noted above, signed permissions forms will be needed for us to use any excerpts from student or instructor responses in our printed materials and the Anonymity section explains how we’ve planned for the collection of these permission while preserving anonymity. However, students can still participate in the assessment, even if they elect to say “no” to our use of excerpts of their writing. We can still use aggregate data about all of the students work in our report.

Additional Surveys: We will also be asking students to complete a survey that asks them some questions about their learning in the course. This survey, coupled with the data from assessment, will help us gain a preliminary understanding of how students respond to the course, what they learn, and areas of the course that may need improvement. We’ll be using Survey Monkey to administer the survey, and students can complete the survey on the day they finish uploading their assessment materials. (Note: We will NOT be asking students any information about specific instructors. In fact, we’ll remind students that they shouldn’t identify their instructor in their responses). This survey will include the following questions (though we may add more):

● Students may be asked to share some demographic information, as well as information about their status (what year and are they transfer students).

● In what ways did this course meet your expectations? In what ways was it different from what you expected?

● What activities/assignments did you find the most useful in this class? Why?● What activities did you find the least useful? Why?● Do you feel this course had a direct impact on your writing skills, particularly as you might need

them in your future major or career? How?● Do you have any suggestions about how we might organize the course in future semesters?● How would you describe this course to another student who was going to be taking it?

Once instructors have received the reviewer evaluations for the groups in their sections, we’ll also be asking them to complete surveys regarding their experience with the assessment (including advice for future assessments) and their sense of where the Writing Program might want to focus resources for professional development in subsequent semesters.

Technology & Submission:We’ll be using an online submission process hosted by a online provider (http://www.wufoo.com/). We’ll be providing instructors with explicit instructions (and more personal, face-to-face assistance if needed) for the submission process.

8

APPENDICES

APPENDIX A: Sample Scenarios (with matched assessment criteria – using the form from above)

SAMPLE SCENARIO 1 [Workplace Writing, (Topical Research)]

Education – “Permission Slip”You are a high school teacher for a junior level class (English, Social Studies, Spanish, Science, etc.). This semester you are showing two rated R movies. However, in order for your students to watch the films, you need to get permission from all of the students’ parents. Additionally, you need to be prepared for the fact that not all students will be able to obtain permission to watch the movies. It is your responsibility to create a permission slip explaining the necessity of the films and how they fall in line with the course learning goals for the unit and/or the semester. You also need to provide an alternative assignment for those students who do not obtain permission to watch the films. You are tasks include:

1. Determine the class for which the permission slip is required2. Determine the two rated R films to be shown in the course3. Determine how those films are related to the overall course goals and/or the goal of the

unit for which the films are being shown. 4. Create the permission slips5. Create the alternative assignment and assignment sheet6. Find at least one example of the genres that you are creating

[This scenario was created with the expectation that it would be completed by education majors, but the information provided is vague enough that it could be completed by anyone. For this scenario, the students are already given the genres that they are to produce; however, much of the scenario is left undefined so that they can have room to participate in creating the scenario]

Assessment CriteriaIdentifying genres: There are at least two genres that the students must identify, the permission slip and the assignment sheet. The permission slip will most likely only require genre research whereas the assignment sheet may require content (topical) research along with genre research.

Creating Content: Once the students identify the genres that they are creating and have provided examples, then they should be able to produce the genres for the needs created by the scenario.

Flexible Research Skills: For this scenario students are expected to do genre research for both genres as well as content research for the assignment sheet. The content research should inform the decisions that they make for the alternative assignment.

9

Trajectory: Students should consider the trajectory of their decisions. More importantly, the trajectory of the permission slip has an immediate trajectory to the imaginary students’ parents whereas the alternative assignment will have an immediate trajectory to the imaginary students (and potentially their parents). This can also be a way for students to think about the trajectory of the course and how the two rated R movies fit into that overarching trajectory.

Ethics: Students should consider the ethical dilemma in which an instructor could find herself in regard to showing two rated R movies to students who are under the age of 17. Even though this is a junior level class and some students may be 17, because they are still minors, it is thus necessary to obtain parental permission for them to view a rated R film in class.

SAMPLE SCENARIO 2 [Social Writing, Topical Research]

General – “Op-Ed” (Variation 1)Write an Op-ed about a currently relevant issue in a relevant venue. Your tasks include:

1. Determine the issue and the venue in which the op-ed would be published or to which it could be submitted.

2. Find multiple examples of Op-Ed pieces for that venue3. Find multiple examples of Op-Ed for a different venue4. Do a complete genre analysis that examines the “genre” of the Op-Ed for the selected

venue (explaining what features are critical, and where there are differences or areas that aren’t clearly defined).

5. Write the Op-Ed6. Note: In responding to the assessment, the group will also need to analyze their

production vis-a-vis their genre research.

Keep in mind that the issue should be currently relevant. The venue to which you choose to submit your Op-ed should care specifically about that issue. Your Op-ed piece should coincide with other Op-eds for that venue.

[This scenario is designed to be completed by an individual rather than in a team. There is a variation to this scenario where it asks students to write three Op-Ed pieces on the same topic for three different venues, that variation allows for this same scenario to be completed in a team. This scenario could also be used in ENG 145 or ENG 145.13 since it is general.]

Assessment CriteriaIdentifying Genres: The scenario already provides the genre for the students, but their task is to identify what an Op-Ed piece looks like in a particular venue. Though there are particular conventions for the Op-Ed genre, the idea is that the students conform to the conventions of a particular venue by examining Op-Eds in different venues.

Creating Content: Once the students have identified what an Op-Ed looks like for a particular venue, relative to other venues, then they should be able to create an Op-Ed that could be submitted to the

10

venue of their choosing.

Flexible Research Skills: This scenario requires both content research and genre research. The students must research what an Op-Ed is, where it would be published and they must also research the conventions for the Op-Ed for the different venues. Because the scenario is open ended, they must determine what a currently relevant topic is and where said topic would be discussed and presented in an Op-Ed. Students would, in essence, have to present themselves as an expert on the issue for whatever they are advocating.

Citation and Citing: Since students will be presenting themselves an expert, then they will have to determine (from their previous genre research) how to cite (or attribute credit to) their sources within the venue. They should recognize that the citation necessary for the Op-Ed will be different than the citation for academic writing.

Discourse Communities: Students should recognize that as the write their Op-Eds, that they are participating in a discourse community which is why the issue must be relevant within a particular venue. They would be expected to use a particular language/jargon, provide an opinion from a particular perspective, and respond effectively to the needs of their audience. Students should recognize their Op-Eds as existing within the realm of a discourse community rather than as only satisfying the needs a particular audience.

Trajectory: Students should recognize the different ways that their text will move in the world. They should recognize a trajectory beyond the scope of the audience, outside of a discourse community and into the world. They should recognize that discussion of the issue exists outside of the manner in which they will discuss the issue within the Op-Ed.

Technologies and Modalities: Keeping in mind trajectory, discourse communities and the venues for which students will choose to write their Op-Eds, the students should be aware of the manner in which technologies shape and filter what they write. For instance, students may elect to write an Op-Ed for an online venue versus a print venue, and the medium/mode of distribution largely determines the manner in which the information is presented in the Op-Ed.

Globalization, Cultural Contexts & Diverse Settings: Again, keeping in mind discourse communities, trajectories and technologies, students should be acutely aware of the manner in which their Op-Ed will be received culturally, socially, politically, and historically. Since students are to write about a currently relevant issue, they should be aware of how this issue may be interpreted differently in diverse, multicultural, or global contexts. They should also consider the bearing that those different contexts have on the production of the Op-Ed.

Ethics: Since students are considering a currently relevant issue, there are also ethical dimension to the argument that they should consider. This may be addressed directly in the Op-Ed or within the supporting materials. Nonetheless, the ethical dimension is relevant when discourse communities, trajectories, technologies, and multicultural/global elements are considered.

SAMPLE SCENARIO 3 [Workplace writing, (Social Writing), Topical Research]

11

“Product Recall” (Variation 2)You work for a large international automotive company, and on various makes and models of cars in your company’s fleet, there is a defect in the accelerator that causes it to stick. As a result of this defect, there have been numerous car accidents, and even more reports of runaway cars. This defect has already been fixed on new models that have not been sold yet; however, there have already been millions of cars sold around the world, over the span of four years, that still require the fix. Your team needs to encourage consumers to get the problem fixed before anyone else gets injured or any more property gets damaged. It is your team’s responsibility to determine the genre(s) necessary to inform consumers, distributors, and news media of the situation as well as your company’s solution to this problem.Your tasks include:

1. Identify and produce the genre(s) necessary to inform the public of this recall.i. Provide examples of the genre that you will be creating

2. Identify your company, the year(s), model(s), and make(s) of the cars included in the recall.

3. Identify the cause of the defect and the solution to the problem.4. Identify best case, worst case, and middle of the road outcomes to the recall

[This scenario was designed specifically for ENG 145.13. It is based on the Toyota recall a few years back. There are two other variations on this scenario that require different tasks. The idea is to allow students to research real life events in order to determine a plan of action through written documents.]

Assessment CriteriaIdentifying Genres: Students will identify the “best” genre to relay information to consumers, distributors and consumers. It is understandable that there is more than one genre that would work, but students need to identify the genre that they believe will best solve the problem with which they are presented in the scenario. They will provide an example as a model from which to work.

Creating Content: Once the “best” genre has been identified and a sample is provided, then students will produce that genre to fit the needs of the scenario.

Flexible Research Skills: Students are expected to conduct both genre research and content research. Genre research will allow them to identify the “best” genre for the task at hand, and content research is necessary for them to create a plan of action that will be outlined in the document that they create.

Citing and Citation: Though it might not be necessary to cite outside researched material in the document that they produce, they may have to cite/attribute credit to consumers who were “injured” as a result of the defect, or they may have to cite/attribute credit to high level executives within the company speaking about how to resolve the issue. They should recognize this form of citation as different from academic citation styles.

Trajectory: Students need to consider the possible trajectories of the recall by imagining a best case, worst case and middle of the road outcome to the recall effort. They must also consider the trajectories of the document that they produce within the grander scope of the overall trajectory of the product recall.

12

Technologies & Modalities: The genre that the students choose as the “best” option for solving the problem presented by the scenario will largely determine the technologies and media that will be used to distribute the information. The students must also notify news media, and they must determine which news media to notify.

Globalization, Cultural Contexts & Diversity: As suggested by the scenario, this is an international company (like Toyota), and so students must recognize that their consumers, distributors and news media are not all in the United States. With that in mind, they should consider these different and diverse contexts.

Ethics: As this defect poses a threat to the safety and well being of the general public and not just the owners and potential owners of these cars, it is an ethical decision that the company has to make to inform the public of the defect. It would be unethical for the company not to inform the public in addition to their customers.

APPENDIX B: Sample Response Form

As students complete their assigned scenarios, they will also complete a “Response Form” that asks them to include particular information. The response form will require several different “reports” on various areas of their work. Report 1: Genre Research, Analysis and ProductionPlease report on the work that your group did to select the genre you are using for your production. Please be as detailed as possible in your descriptions, and be sure to note where different members of the group did different work (but don’t use names!). Your report should be in paragraph form (with headers), and you should include the following sections: (1) Researching & Analyzing Genres: In this report your group will need to explain in detail the activities you engaged in as you worked to research and analyze the writing that you did. Your description should include – as much as possible – all of the activities you engaged in when trying to make decisions about what genres to use, what those genres might look like, and how you learned to understand the genre well enough to produce it. (2) Using Research Genres: Please describe as specifically as you can how you used the genre research you did to help you produce the response text. This means we want you to make very specific and detailed explanations of how you used your research and analysis in specific aspects of the composing process. Citing from your research and analysis report and then referring to specific parts of your actual production would be an ideal way to create a more complete report. (3) Antecedent Genres: Please list and discuss any kinds of writing that your group members already knew about that came into play as you worked on this scenario. What did you know how to do that you could use? Did you find that any of the writing you already know how to do “got in the way” when you were creating this production? Explain in as much detail as you can. One important aspect of this report will be to note any negotiations the group had to engage in to make decisions about whose knowledge to use.(4) Bibliography of Source Material with Notes: Please include a bibliography – a list of all the sources you used to do your genre research). For each source, please include a short explanation that explains how you used the various sources you found to better understand the genre. Report 2: Content Research (note: Some scenarios might not require this report)

13

If this scenario required content research (that is, looking up information about a topic that you didn’t already know about), please submit a report explaining how you went about doing that research, and be as detailed as possible in your explanations, and be sure to note places where different members of the group had different opinions. Your report should have the following sections: (1) Finding Sources: Please explain the methods you used to find the sources you found. How did you go about searching? (2) Useful Sources: How did you figure out which sources would be useful? Please give at least one example with significant details – why was this a useful source? How did you go about deciding it was useful?(3) Credible/Authoritative: How did you decide which sources were credible? Please give at least one example with significant details – what information did you use to decide if the source was credible. Was the(4) List of Sources With Notes: Please include a bibliography – a list of all the sources you used to do content research) with short explanations for each source that explain how you used the various sources you found in the final product you produced. Report 3: Considering the complex setting – or activity system – in which your scenario is embedded

Many instructors use terms from Cultural-Historical Activity Theory to help students think about how complex writing situations can be (how all kinds of factors can influence what and how texts are produced and received). However, even if your instructor doesn’t use these terms -- or even if you felt like you didn’t fully understand them -- you can still discuss your understanding of this scenario as a complex, complicated activity that included a lot of decision-making. Your Report should include the following sections:

1. Understanding the Scenarios: What were some of the most complicated aspects of understanding how to respond to this scenario?

2. Questions you needed to answer: What were some of the questions your group asked and answered? Which questions were the most difficult, and how did you resolve them (as a group)?

3. Knowledge, Writing Tools, and Planning: What knowledge or writing tools did you draw on to complete the response? Did you have to learn how to do anything new? Explain how you first envisioned this text, and whether that representation changed (and how it changed) as you worked.

4. Ethical Considerations: What kinds of ethical considerations did you make as you worked through this scenario?

5. Ecology and Trajectory: Explain your understanding of the environment in which this text would/might exist. Who would use it? How would it be used? Could it be used in any ways you didn’t envision?

6. How Could this Text be Received: What are some of the unexpected ways that your texts could, hypothetically, be responded to? How might it cause trouble or solve problems?

Part 4: The Production: Please upload a copy of your production (cannot exceed 2 mb. If the file is larger, you’ll need to print it out and submit it in the envelopes provided). Part 5: Group Evaluation of the Production

14

This part of the Assessment includes your honest analysis of how well your group answered the problem posed by the scenario. Remember that no grade is assessed for how “well” you did the production, but we will be closely examining how well you can describe and defend the choices that you made. Your report should be organized (generally) in paragraph form, using the following headers:

(1) Group Evaluation of the Production: Please provide an evaluation of how well your production matches up with what the Assessment Scenario asked you to produce. You’ll need to consider the overall effectiveness of the text in relation to the scenarios and the the sample/master texts that were use to produce it. You are essentially be asked to make an argument for the effectiveness of this text: How would you rank your production? (excellent, very good, less than successful, or unsuccessful). Please use examples from the text to support your ranking. We’d also really like to know if the group is in agreement about this ranking. Were there any dissenting opinions about the ranking, and why?

(2) Genre Knowledge and Antecedent Genres: Please discuss the kinds of writing that you “already know about” that came into play when you were working on the scenario. Discuss ways in which that knowledge helped the group and also any times when that knowledge caused problems. Particularly discuss moments where the group didn’t (at least initially) agree about your different knowledge of different genres. Describe any moments when one or more group members were able to help the others by explaining knowledge about genres? Give as many examples as you can.

(3) Grammar and style, Citation and Formatting: Please discuss specific examples of choices you made to make sure that what you produced for production genre would be appropriate. For example, if you ended up having to cite information, how did you do that, and why did you do it that way? How careful were you to make your style and grammar match the genre, and what did you have to do to do that? Were there visual design or formatting choices you had to make, and how did you make them? Did you feel like you needed to carefully proofread to make sure that the sentence structures were in line with “standard grammar and syntax”? If not, why not, and how was the language that you used different. Please provide as many examples as you can.

Additional Documents: In addition to uploading your actual textual production, please upload any additional documents that you feel would help us to understand how your group went about solving the scenario you were given. In order to upload material, you’ll need to save it into a single document and upload that document here. These documents can be no more than 2 MB in size.

15

APPENDIX C: Questions for the Instructor Form

We may revise these based on instructor input, but we believe that the form that instructors are asked to fill out will include the following elements:

(1) Please tell us about how you organized the groups for your class. Things we might find important or interesting include:

Did you assign them randomly, or based on your knowledge of the students? If you used your knowledge of the students, what might you want to share with us

about the personality/psychological make up of any of the groups? [Note: you aren’t required to give us these details, but if you know something about the groups that you feel will affect their performance (positively or negatively) you might want to share that (but use the group numbers WITHOUT ANY STUDENT NAMES!)

If you used groups that have worked together before, that might affect their performance, so please let us know about that.

(2) Please tell us about the process through which you selected the scenarios. Things that might be interesting or relevant to us include:

If you selected them specifically to make then connect to kinds of writing you already do in your class

Or if you selected scenarios that you thought would work well with the interests of the groups. [Note: basically, you are free to make whatever choices you want with the scenarios and matching them to groups, but we just want to know about how you went about making the decision].

(3) Please give us a general sense of how your class takes up the program learning outcomes. Do you talk about them in class – in what ways and how often? Do you feel that there are some learning outcomes you are able to connect to more

effectively than others? Which are these and why? Do you have learning outcomes you aren’t sure if you are covering? Which ones? Do you feel that you do a particularly good job at teaching any of the learning

outcomes? Which ones?(4) Please give us a general sense of your use of specific terms and concepts, based on our program

list Are there any terms that you really like and use a lot? Which ones? Are there any terms you don’t really feel like you use? Which terms do you feel like your students really learn to use during the semester? Which terms do you feel like students have a really hard time grasping (or using

effectively). Are there any terms that the program uses that you’d like to learn more about, or

that you’d like us to provide resources for using?

(5) Please let us know if there are specific terms or concepts that you feel like you cover in your class(es), but which aren’t addressed in our learning outcomes or terms and concepts list. Briefly describe how you use these terms and if you can, explain how they are different from WP terms (i.e., they cover concepts we don’t cover, or they cover the same concepts, but you like to use a different term, etc.).

16