Slide 1

2014 Self-Study Workshop Characteristics of Good Self-Study Reports [NOTE: Updated January 2015 to reflect changes in Handbook] Tom Tanner ATS Commission Staff

Slide 2

Before we begin, a caution ATS Commission on Accrediting: 2014 Self-Study Workshop If youre going to work that hard, do the right car

Slide 3

Characteristics of Good Self-Study Reports Five things to remember to do the right report: 1.The 4 purposes of the report 2.The 3 qualities of the report 3.The 5 features of good/weak reports 4.The 4 parts of the report 5.The 3 last things to remember ATS Commission on Accrediting: 2014 Self-Study Workshop 1 2 3 4 5

Slide 4

Characteristics of Good Self-Study Reports Five things to remember to do the right report: 1.The 4 purposes of the report 2.The 3 qualities of the report 3.The 5 features of good/weak reports 4.The 4 parts of the report 5.The 3 last things to remember ATS Commission on Accrediting: 2014 Self-Study Workshop 1

Slide 5

Characteristics of Good Self-Study Reports 1.The 4 purposes of the report (www.ats.edu) ATS Commission on Accrediting: 2014 Self-Study Workshop [formerly Handbook of Accreditation]

Slide 6

Characteristics of Good Self-Study Reports 1.The 4 purposes of the report 1)Demonstrate ability to analyze institutions own effectiveness 2)Develop plans for improvement 3)Provide evidence of meeting standards 4)Form basis for external review ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 7

Characteristics of Good Self-Study Reports 1.The 4 purposes of the report 1)Demonstrate ability to analyze institutions own effectiveness Good long look in the mirror The mirror has two faces: Schools own mission ATS Commission standards ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 8

Characteristics of Good Self-Study Reports 1.The 4 purposes of the report 2)Develop plans for improvement Report must not only look around but look ahead Assessment (self-study) should show ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 9

Characteristics of Good Self-Study Reports 1.The 4 purposes of the report 3)Provide evidence of meeting standards Weak report:Our seminary clearly meets this standard. Good report: Evidence of how our seminary meets this standard may be summarized in these three ways. First, ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 10

Characteristics of Good Self-Study Reports 1.The 4 purposes of the report 4)Form basis for external review The self-study report is your story but told to strangers (so dont assume) who are sympathetic, but objective so you must present and persuade ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 11

Characteristics of Good Self-Study Reports Five things to remember to do the right report: 1.The 4 purposes of the report 2.The 3 qualities of the report 3.The 5 features of good/weak reports 4.The 4 parts of the report 5.The 3 last things to remember ATS Commission on Accrediting: 2014 Self-Study Workshop 2

Slide 12

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report* 1)Provide sufficient description 2)Document appropriate evaluation 3)Identify strategic recommendations * Page 11 of Chapter Two of the Handbook NOTE: Under heading General Characteristics ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 13

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 1)Provide sufficient description Weak report: We have sufficient faculty, and they are all sufficiently qualified. Good report:We have seven full-time faculty; six have PhD degrees and one has a DMin (see this link for detailed list). ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 14

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 1)Provide sufficient description Weak report: Our SLAP efforts have produced significant improvements. Good report:Our Student Learning Assessment Program (affectionately called SLAP by students) has produced these three improvements in student learning this past year. First,... ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 15

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 1)Provide sufficient description 2)Document appropriate evaluation 3)Identify strategic recommendations ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 16

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 2)Document appropriate evaluation Weak report: Our MDiv program addresses the cultural context requirement (Degree Program Standard A, section A.2.3) fairly well, but could use improvement. Good report: We evaluate the cultural context component of our MDiv program in two ways: (1) a cultural awareness survey (an indirect measure) and (2) a cultural engagement project (a direct measure). The results for these two assessments are provided in Table A.2 below, followed by a discussion of how those results have been used to improve our MDiv students learning in this critical area. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 17

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 2)Document appropriate evaluation Weak report: We try hard not to spend more than we take in. Good report:The seminary has balanced its budget four of the last five years, as Table 8.6 below demonstrates. The deficit of $74,892 in FY12 represented only a 2.5% overage due to an unexpected need to replace a furnace. We have since established a contingency fund for such emergencies, with a line-item budget of $50,000 per year. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 18

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 1)Provide sufficient description 2)Document appropriate evaluation 3)Identify strategic recommendations ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 19

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 3)Identify strategic recommendations Weak report: We know more should be done to address our lack of board diversity, and we are working on that. Good report: Given our global mission and given that one fourth of our students and staff are ethnic minorities, while no one on the board is, our trustees have set a goal of recruiting one new ethnic trustee each year for the next five years until our board more fairly represents our constituency and our mission. Our first African American trustee will begin her term this fall. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 20

Characteristics of Good Self-Study Reports 2.The 3 qualities of the report 3)Identify strategic recommendations Weak report: As you will see scattered throughout this report, we have identified dozens and dozens of different areas to improve. Good report:We have organized each chapter of the report (one per standard) into four parts: (1) a brief description of that area, (2) a critical evaluation of that area, (3) a list of strengths in that area, and (4) a set of two or three strategic recommendations per area to address key concerns identified. All recommendations are summarized and prioritized in the last chapter. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 21

Characteristics of Good Self-Study Reports Five things to remember to do the right report: 1.The 4 purposes of the report 2.The 3 qualities of the report 3.The 5 features of good/weak reports 4.The 4 parts of the report 5.The 3 last things to remember ATS Commission on Accrediting: 2014 Self-Study Workshop 3

Slide 22

Characteristics of Good Self-Study Reports 3.The 5 features of good [vs. weak] reports 1)Are organized coherently 2)Describe the self-study process 3)Present the data crucial to understanding 4)Comprehensive as necessary; brief as possible 5)Demonstrate objectivity and honesty ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 23

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 1)Are organized coherently Must be organized by the standards If joint visit with one report, must have detailed cross walk [Consult with Commission staff about joint visits] Must also have Introduction and Conclusion Will discuss later under The 4 parts of the report ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 24

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 2)Describe the self-study process Introduction describes process Process engages the community Reflected in report and interviews Caution: Summarize process in 1-2 pages If use peoples names, use titles too ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 25

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 3)Present the data crucial to understanding Be clear and concise with data Use qualitative and quantitative data Use tables and charts to support narrative Caution: Put extensive data in Documents Room and/or hyperlink in narrative But include summary data in text When using links, make easy to use ATS Commission on Accrediting: 2014 Self-Study Workshop Clearly label each item

Slide 26

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 3)Present the data crucial to understanding ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 27

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 3)Present the data crucial to understanding Weak report: Our students are very diverse. Good report:Our 150 students, whose ages range from 24 to 64 (with most in their 30s), are 45% female, and 30% ethnic minorities, with a dozen different denominations represented (see Table 6.3 below for demographic details). This is what one recent graduate said about her experience here regarding diversity: [quote]. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 28

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 3)Present the data crucial to understanding Weak report: Most of our MDiv students are placed. Good report:Over the last five years our MDiv graduates have averaged a 90% placement rate, with a steady increase each year, as the table below shows. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 29

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 3)Present the data crucial to understanding Weak report: We evaluated our student services through a survey last year; see data in Appendix 6. Good report:Faculty and staff reviewed the results of a recent student survey (found in Documents Room; File 6.1 Student Survey for 2014; see also this link) and drew these four conclusions regarding student services ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 30

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 4)Comprehensive as necessary; brief as possible Weak reports: Missing standards (e.g., Degree Program Standards) Too few pages: 50 pages wont cover the standards Too many pages: 175+ pages overwhelm reviewers Printed incorrectly: Single-sided/double-spaced reports double the length and the weight (remember your traveling readers) ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 31

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 4)Comprehensive as necessary; brief as possible Good reports: 100-135 pages is ideal range 5-10 page Introduction 75ish pages for General Institutional Standards (1-8) 15-40 pages for Educational and Degree Program Standards 5-10 page Conclusion Dont overwhelm text with supplemental information Put lengthy information in Documents Room and/or in hyperlinks in report narrative (see later slide for appendices) ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 32

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 5)Demonstrate objectivity and honesty Weak report: Morale on campus is much better. Good report: Three years ago faculty morale hit a low point when a doctrinal controversy led to the dismissal of a popular professor. The seminary followed its procedures (documented in our personnel files), but it was a very difficult time. Since then, the board and administration have worked hard to rebuild unity, with significant successas noted in the recent faculty survey (see Documents Room, File 5.4: Faculty Survey, item #8; see also this link). ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 33

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 5)Demonstrate objectivity and honesty Weak report: The seminary has experienced some administrative turnover since the last visit. Good report: We have had three presidents and four deans in the last decade. These turnovers have occurred for various reasons and with varying results. We will summarize those reasons and results here, but the evaluation committee is also encouraged to ask about this during their on-site interviews to have a more nuanced understanding of this issue. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 34

Characteristics of Good Self-Study Reports 3.The 5 features of good/weak reports 5)Demonstrate objectivity and honesty Weak report: [no mention of recent public controversy] Good report: Just prior to submitting this self-study report to our trustees for their review, five students filed a complaint against a faculty member alleging sexual harassment. Unfortunately, portions of that complaint have surfaced recently in social media. We have initiated our formal complaint process, and we will inform the evaluation committee of any significant developments that we are permitted legally and ethically to share. NOTE: Boards are to receive self-study reports (Chapter Two, p. 10) ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 35

Characteristics of Good Self-Study Reports Five things to remember to do the right report: 1.The 4 purposes of the report 2.The 3 qualities of the report 3.The 5 features of good/weak reports 4.The 4 parts of the report 5.The 3 last things to remember ATS Commission on Accrediting: 2014 Self-Study Workshop 4

Slide 36

Characteristics of Good Self-Study Reports 4.The 4 parts of the report* 1)Introduction 2)Main Narrative 3)Conclusion 4)Appendices (not printed; on flash drive only) * Pages 13-14 of Chapter Two of the Handbook [NOTE: These 4 parts were discussed, in part, earlier] Reminder: For how to copy and distribute the report and the appendices, see Chapter Three, p. 12 and p. 16) ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 37

Characteristics of Good Self-Study Reports 4.The 4 parts of the report 1)Introduction 2)Main Narrative 3)Conclusion 4)Appendices ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 38

Characteristics of Good Self-Study Reports 4.The 4 parts of the report 1)Introduction (5-10 pages) Introduce the school (mission/history) Review accreditation history (since last visit) Give overview of the self-study process Identify any special features of the report Summarize any major changes (since last visit) Orient readers to any unique school issues ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 39

Characteristics of Good Self-Study Reports 4.The 4 parts of the report 2)Main Narrative (90-115 pages) Part 1: General Institutional Standards (75 pp.) Standards 1-8must cover each one Part 2: Educational and Degree Program Standards ( 15-40 pp.) Educational Standardcover only applicable sections Degree Programs Standardscover only applicable standards NOTE: Can cover assessment under ES.6 or under section 1 of each DPS ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 40

Characteristics of Good Self-Study Reports 4.The 4 parts of the report 2)Main Narrative (90-115 pages) One suggested structure per chapter (per earlier slide): brief description of schools resources for that standard critical evaluation of schools alignment with standard list of strengths of the school in terms of that standard strategic recommendations to improve in that standard NOTE: If not meeting standard, must describe plans to do so. * Do not need to cover every subsection in every standard: group things. ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 41

Characteristics of Good Self-Study Reports ATS Commission on Accrediting: 2014 Self-Study Workshop Weak Report: Discuss each of 10 subsections Good Report: Discuss each of 3 sections Standard 4 Library (as example)

Slide 42

Characteristics of Good Self-Study Reports 4.The 4 parts of the report 3)Conclusion (5-10 pages) Summarize overall findings of self- study Organize and prioritize strengths Organize and prioritize concerns Organize and prioritize recommendations Conclude with desired action (optional) ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 43

Characteristics of Good Self-Study Reports 4.The 4 parts of the report 4)Appendices (lengths vary, but try to keep under 250 pages) Put only these items in the appendices (PDF on USB); put everything else in Documents Room and/or hyperlink 1) Current strategic plan4) Handbooks* 2) Current budget5) Academic Catalog* 3) Most recent audit6) Targeted Issues Checklist * Trustee, faculty, staff, and student handbooks ** Also provide Word version of TIC for committee to complete Appendices must be combined into one PDF and bookmarked ATS Commission on Accrediting: 2014 Self-Study Workshop **

Slide 44

Characteristics of Good Self-Study Reports 4.The 4 parts of the report Documents Room (pp. 15-16 of Chapter Three of Handbook) Historically, schools have been required to provide paper copies of various documents in a Documents Room Increasingly, schools are asking if some or all of those documents could be provided instead electronically A Virtual Documents Room is acceptable, provided these five conditions are met [listed on pp. 15-16 of Chapter Three] KEY: Easily accessible and clearly marked ATS Commission on Accrediting: 2014 Self-Study Workshop

Slide 45

Characteristics of Good Self-Study Reports Five things to remember to do the right report: 1.The 4 purposes of the report 2.The 3 qualities of the report 3.The 5 features of good/weak reports 4.The 4 parts of the report 5.The 3 last things to remember ATS Commission on Accrediting: 2014 Self-Study Workshop 5

Slide 46

Characteristics of Good Self-Study Reports 5. The 3 last things to remember Tell your story in light of the standards to improve your students and school ATS Commission on Accrediting: 2014 Self-Study Workshop Where will this journey take you?