diagnose

AdvancED Surveys. Copyright 2011 AdvancED®

ASSIST Technical Guide

Administering Diagnostics

Questions 888.413.3669

[email protected]

1

AdvancED Diagnostics Copyright 2012 AdvancED®

About Diagnostics

The Adaptive System of School Improvement Support Tools (ASSIST™) offers a suite of diagnostic tools, including a self-assessment, executive summary, student performance, stakeholder feedback and more. These tools are designed to offer a comprehensive, yet simplistic way of identifying an organization’s strengths and challenges. Internal and external diagnostics results are an important part of the AdvancED accreditation process, but the intent is for education institutions to use diagnostic tools as part of their ongoing data collection and improvement process.

Table of Contents Accessing Diagnostics in ASSIST ................................................................................................................2 Administering Diagnostics .........................................................................................................................3 Completing Diagnostics Executive Summary ..........................................................................................................................4 Self-Assessment ...............................................................................................................................6 Student Performance Diagnostic .....................................................................................................9 Stakeholder Feedback Diagnostic ..................................................................................................13 Additional Unique Diagnostics .......................................................................................................16 Submitting Diagnostics ............................................................................................................................17 Frequently Asked Questions ...................................................................................................................19

You can now administer the following diagnostic tools in ASSIST!

• Executive Summary • Self-Assessment • Student Performance • Stakeholder Feedback • and more…

Learn more at: www.advanc-ed.org/assistresources

http://www.advanc-ed.org/assistresources

2


Accessing Diagnostics in ASSIST

Diagnostics can be accessed by logging into the Adaptive System of School Improvement Support Tools (ASSIST) at www.advanc-ed.org/assist.

To access ASSIST, enter your username and password on the login page. Once successfully logged in, click the ASSIST link.

If you need additional assistance with the login process, refer to the ASSIST Quick Start Guide at www.advanc-ed.org/assistresources.

Once you are in ASSIST, click the Diagnostic & Surveys tab at the top of the screen.

Any diagnostic that has already been started for your institution will show in a table within the Diagnostic section. Clicking the diagnostic name will take

you to that specific diagnostic.

http://www.advanc-ed.org/assist

http://www.advanc-ed.org/assist


3


Administering Diagnostics

Diagnostics can be self-administered at any time by clicking on the Start a Diagnostic button. Schools and school systems are encouraged to use these diagnostic tools as part of an ongoing Self Assessment process.

Upon clicking the Start a Diagnostic button, you will be taken to the Start a Diagnostic page, where you can select the diagnostic you would like to administer from a drop-down list.

Once you have selected a diagnostic and provided a description, click Start.

All diagnostics are self-administered in the same fashion. If you need a diagnostic that is not provided in your institution’s drop-down list, please contact your AdvancED State Office or [email protected].

Note: The only way to preview a diagnostic is to start it. You can delete the diagnostic if you decide it is not needed.

Be sure to provide a brief description that will help

to differentiate one diagnostic from another.

You may decide to administer multiple

diagnostics over time.

mailto:[email protected]

4


Completing Diagnostics: Executive Summary

The Executive Summary is a narrative diagnostic tool that captures your school or school system’s story. It contains the vision and cultural context of the institution, as well as key information about how the school provides teaching and learning. Challenges and opportunities are highlighted in the document.

The diagnostic content is tailored for schools, school systems, corporations, ESAs and early learning institutions. ASSIST provides the appropriate content based on institution type.

The Executive Summary is a required component of the Accreditation Report that must be submitted in preparation for an External Review; however, institutions are encouraged to use it as an ongoing support tool.

Clicking a section title on the Diagnostic Summary will open the Section Summary screen, where you can view the item and respond by clicking the Respond link.

Respond to each section of the

Executive Summary by clicking the section title.

The number of items within a section is

provided below the section title. The

progress provides a visual of section completeness.

Note: You can return to this screen at any time to change an item response.

5


Clicking the Respond link will open a narrative box where you can type your response to the item. You can copy and paste text from a text editor or Microsoft Word® document. ASSIST does not support special formatting, so do not include bulleted lists, tables or images.

Click Save and Continue to move forward to the next item.

Once you have completed each section of the Executive Summary, a Publish to the AdvancED Registry button will be added at the bottom of the Diagnostic Summary screen. Upon clicking this button, the Executive Summary will be available to the public on the AdvancED Institution Locator on the AdvancED website, if you are an accredited institution.

Note: Even after publishing, you can come back to this diagnostic at any time to make updates and republish.

Once the Executive Summary has been reviewed and approved, click the Complete button to mark the diagnostic as complete.

There is a 6,000 character limit to

each section of the Executive Summary.

Note: There is not a submit button on this screen. All documents required for Accreditation are submitted in the Portfolio tab of ASSIST. The submission process is covered at the end of this document.

View, save or print a PDF of the Executive Summary at any time during the process.

6


Completing Diagnostics: Self Assessment

The Self Assessment diagnostic is based on the AdvancED Standards of Quality, which serve as the foundation of the accreditation and continuous improvement process. The Self Assessment is a valuable tool that should be used to engage staff members and stakeholders in purposeful and honest dialogue and reflection in order to evaluate the institution's adherence to the Standards and guide continuous improvement efforts. The External Review Team uses the results of the Self Assessment to learn about the institution in preparation for the External Review. The Self Assessment includes the institution's self-ratings and the evidence cited for each of the indicators for the Standards. At the conclusion of the Self Assessment ,an overall assessment summary is provided in graphic and text forms.

The diagnostic content is tailored for schools, school systems, corporations, ESAs and early learning institutions. ASSIST provides the appropriate content based on institution type.

The Self-Assessment is a required component of the Accreditation Report that must be submitted in preparation for an External Review; however, institutions are encouraged to use it as an ongoing support tool.

Respond to each section of the Self

Assessment by clicking the

Standard name.

The number of items within each section is provided below the

section title. The items within the Self Assessment

represent the Indicators within each Standard, plus a Standard Narrative. The progress provides a visual of section completeness.

7


Upon entering each Standard, you will see a table that provides each Indicator within the Standard.

Select an Indicator by clicking the Respond link.

The last item in each Standard is a Narrative where you will be asked to reflect on your responses to the Standard and highlight strengths and areas for improvement.

Respond to each Indicator by selecting the performance level that most accurately represents your institution. Then identify the evidence your institution has to substantiate the selection of that performance level.

Clicking Save and Continue will automatically take you to the next item in the diagnostic.

At any time you can return to the

Diagnostic Summary to view

a different Standard or

overall diagnostic.

Note: You can return to this screen at any time to change an item response.

Additional evidence can be typed into the Other box. You are not required to upload or

attach evidence.

8


Once all questions in the Self-Assessment have a response and you are ready to finalize the Self Assessment, click Complete.

Once the Self Assessment has been completed, the View Results button is available.

Viewing diagnostic results provides the overall ratings for your institution at the Standard level. Standard level ratings are calculated by an algorithm in ASSIST that looks at your performance level responses to each Indicator.

Note: There is not a submit button on this screen. All documents required for Accreditation are submitted in the Portfolio tab of ASSIST. The submission process is covered at the end of this document.

View, save, or print a PDF of the Self-

Assessment at any time during the process.

9


Completing Diagnostics: Student Performance Diagnostic

The Student Performance Diagnostic facilitates the process of collecting and analyzing student performance data in order to identify areas of achievement and in need of improvement. Completion of this diagnostic requires the Student Performance Worksheet, which is located at www.advanc-ed.org/assistresources.

This diagnostic will be a required component of the Accreditation Report submitted in preparation for External Reviews beginning in spring 2013. However, institutions are encouraged to use it as an ongoing support tool.

The Areas of Notable Achievement and Areas in Need of Improvement sections of the diagnostic are narrative in format. Each section contains six reflection questions that guide a thoughtful analysis of student performance data.

Note: Before you begin this diagnostic in ASSIST, download and complete the Student Performance Worksheet located at www.advanc-ed.org/assistresources. This Microsoft Excel® worksheet will allow you to record various assessments you use to measure student performance. The worksheet includes the rubrics you will need to evaluate the quality and results of the assessments chosen to represent student performance at your institution.


diagnostic by clicking the section title.

The number of items within each section of the

Student Performance Diagnostic is provided

below the section title. The progress provides a visual of section completeness.



10


The Assessment Scores section of the diagnostic is where you will capture the overall scores from your Student Performance Worksheet into ASSIST.

Each overall score from the Worksheet will translate to an item on the Student Performance Diagnostic Section Summary screen.

Clicking Respond for each item will allow you to enter the value from the Worksheet into ASSIST.

The overall scores in the Worksheet are

automatically calculated based on the combination

of data provided for all assessments in the

Worksheet. The overall scores will recalculate each

time an additional assessment is added.

11


Entering information into the Worksheet may seem a little overwhelming at first, so let’s walk through it. Each step of the process will refer to the sample worksheet below.

1. Assessment Name: Select an assessment and type the name into the first available cell. Selection of assessments is completely up to you; however, below are selection guidelines to ensure your complete “Assessment Package” paints a clear picture of all student performance. Assessments should be summative. Assessment should be administered as required by the governing authority (i.e., national, state or local)

for all students, with accommodations as appropriate, in tested grades or for all students at the end of a particular course.

Include at least one assessment for any content area required by a governing authority (e.g. national, state or local).

Include at least one assessment for core academic areas (language arts, mathematics, science, social studies).

Include results from multiple administrations (usually annual) of the assessment. Include multiple assessments (two or more) for each area in need of improvement (goal area).

2. Quality/Analysis Rubric: For each assessment, determine a Quality, Alignment and Disaggregation score using the corresponding rubric.

1 2 4 3

To view the rubric text, hover over the column heading. Any cell that has a rubric or hidden

text will have a small red triangle in the upper right corner.

12


3. Most Recent Assessment Results: For each assessment, take the total number of students assessed and divide them into four categories based on performance. You should end with a count of students that fell into the top quarter of performance results (Category 4), a count of students that fell into the bottom quarter of performance results (Category 1), etc. Below is an example using an assessment with mean scale score results with a total of 206 students tested.

Category Scale Score

Range Student Count

1 0 175 14 2 175.01 350 45 3 350.01 525 106 4 525.01 700 41

4. Baseline Assessment Results: Following the process outlined in Step 3 above, record student counts for a prior administration of the same assessment. This data will be used to calculate an Improvement Score.

Upon the completion of all four steps, the Worksheet should provide the overall scores (in green) to be transposed into ASSIST.

Once you are ready to finalize the Student Performance Diagnostic, click Complete.

If additional guidance is needed on the completion of the Worksheet or Student Performance Diagnostic, please download the Student Performance Diagnostic Workbook, located at www.advanc-ed.org/assistresources.

Record the student counts in the Most Recent Administration section of the Worksheet.

View, save or print a PDF of the Student Performance

Diagnostic at any time during the process.


13


Completing Diagnostics: Stakeholder Feedback Diagnostic

The Stakeholder Feedback Diagnostic facilitates the process of analyzing stakeholder feedback data (survey data) in order to identify areas of achievement as well as areas in need of improvement. Completion of this diagnostic requires the Stakeholder Feedback Worksheet, which is located at www.advanc-ed.org/assistresources.

This diagnostic will be a required component of the Accreditation Report submitted in preparation for External Reviews beginning in spring 2013. However, institutions are encouraged to use it as an ongoing support tool.

The Areas of Notable Achievement and Areas in Need of Improvement sections of the diagnostic are narrative in format. Each section contains reflection questions that guide a thoughtful analysis of stakeholder feedback results.

Note: Before you begin this diagnostic, you will need to administer AdvancED stakeholder perception surveys, which are located in the Diagnostic and Surveys tab of ASSIST. You will also need to download and complete the Stakeholder Feedback Worksheet, located at www.advanc-ed.org/assistresources using data from your stakeholder survey reports.


diagnostic by clicking the section title.

The number of items within each section of the

Stakeholder Feedback Diagnostic is provided

below the section title. The progress provides a visual of section completeness.




14


The Criteria for Assessing Stakeholder Feedback section of the diagnostic is where you will capture your overall rubric level obtained in the Stakeholder Feedback Worksheet into ASSIST.

Note: If you administered more than one type of Student Survey (e.g. Elementary and Middle/High), you will record the Average Score in the corresponding cells and the Worksheet will calculate a Combined Student Survey Score.

At any time you can return to the

Diagnostic Summary to view a different section or overall

diagnostic progress.

You will need to enter the Average Score from your

most recent administration of the

AdvancED Stakeholder Surveys into the

Worksheet. If you have not administered these

surveys, you will need to do so before completing

the Worksheet.

15


You will only need to enter one number per survey (e.g., Parent, Staff, etc.) into the Student Feedback Worksheet. The number you should enter is the Average Score, which can be found in the Survey Scoring Report for the identified survey.

View your institution’s survey results under the Reports tab.

Click the link for the Survey Scoring Summary. This will open the report where you will find the Average Score and Number of Respondents. Transcribe both numbers from the Survey Scoring Report into the appropriate line of the Stakeholder Feedback Worksheet.

You may want to make sure that the survey is closed prior to

recording the Survey Score in your Worksheet. This will ensure

that all results are reflected in the recorded score.

16


17


Completing Diagnostics: Additional Unique Diagnostics

In addition to the AdvancED Self-Assessment, Executive Summary, Student Performance and Stakeholder Feedback Diagnsotics, ASSIST supports content from several AdvancED partner agencies. Some partner agency content is provided in the form of unique diagnostic tools.

Partner agency diagnostics are made available to schools and school systems affiliated with the partner agency.

All affiliations for your institution can be viewed by clicking the Affiliations tab in your ASSIST Profile.

If you are affiliated with a partner agency that has unique diagnostic content, a partner agnecy diagnosic will be displayed in the Start a Diagnostic drop-down menu. Below is an example of the ACSI Diagnostic.

AdvancED also partners with some State Departments of Education that provide unique diagnostic tools for their schools and school systems in ASSIST.

If you think there is a discrepancy with the diagnostics you see in your drop-down list, please contact your AdvancED State Office or [email protected].

The Association of Christian Schools International (ACSI) is

one of AdvancED’s partner agencies. Schools affiliated

with this agency can engage in a single accreditation process and receive dual

accreditation by NCA/SACS and ACSI.



18


Submitting Diagnostics

We strongly encourage the use of all diagnostic tools available in ASSIST as part of a healthy continuous improvement process. However, you are only required to submit diagnostic results if you have an upcoming or past due reporting task listed in your institution’s ASSIST Overview tab.

The submission of an Accreditation Report is required four to six weeks prior to an External Review.

The Document Details page provides a list of all components to be included in the submitted document.

Note: The document details identify what needs to be submitted, but we can tell that nothing has been attached because the Components section is blank.

Click on the task name to view the document details required for submission.

Note: Clicking the task name takes you to the

Portfolio tab.

The section identifies which components are

Diagnostics.

If a component of the document is not required

for submission, there will be a checkmark in the Optional

field.

19


Click the Add/Remove button next to the diagnostic name. This will open a window that allows you to select the diagnostic you would like to attach from a list of active diagnostics. Select a diagnostic by checking the box and Save your selection.

Note: If you click the Add/Remove button prior to starting that diagnostic under the Diagnostic & Surveys tab, you will not see anything to attach. If you have created multiple diagnostics (e.g., two Self Assessments), you will see them both in the list and select only the one you want to attach for submission.

Once all required components of the document have a completed checkmark, the document can be submitted by clicking the Submit button at the bottom of the page.

Note: You can view a PDF of the document at any time during the process by clicking the View PDF button at the bottom of the page.

Saving a selected diagnostic will make it appear within the

Components section of the report table. You can click the name to

view the diagnostic results.

If all questions in a diagnostic have been answered, a

checkmark with appear in the completed column. Responses

can be edited on any diagnostic until the document is submitted.

20


Frequently Asked Questions

How often can diagnostics be administered? Diagnostics can be self-administered as often as a school/school system would like. ASSIST does not impose any limitations or restrictions on the frequency of administration.

Are diagnostics required? Yes, institutions engaged in the accreditation process are required to complete the AdvancED Self Assessment, Executive Summary, Stakeholder Feedback (optional for Fall 2012) and Student Performance Diagnostics (optional for Fall 2012) in ASSIST during the 12 months prior to an External Review. Is there a cost to administer diagnostics? No. All diagnostics are available through ASSIST to institutions engaged in the AdvancED accreditation process at no additional cost. Can additional questions be added to a diagnostic? Additional questions cannot be added to diagnostic tools in ASSIST. Can a School System view a summary report to see which schools have completed diagnostics? There is not a summary report available for school systems at this time. This is a feature that will be coming very soon. School Systems can view schools individually to see their progress on diagnostics. Can a School System review and approve school Accreditation Reports before they are submitted? ASSIST does not currently support an online review and approval workflow between schools and districts; however, this has been identified as a need and will be included in a future release of ASSIST. Districts can view all work completed by schools in ASSIST at any time and are encouraged to establish a working timeline that allows for district review and approval prior to school submission. Are diagnostics available in languages other than English? At this time, diagnostics are only available in English. How are diagnostics submitted? Diagnostics can only be submitted in the form of an Accreditation Report in the Portfolio tab of ASSIST. Can diagnostics be downloaded and/or printed? Yes. All diagnostics in ASSIST can be downloaded, saved and printed at any time during the process by clicking the “View PDF” button at the bottom of the diagnostic screen. Can diagnostic responses be changed? All diagnostic responses can be changed until the diagnostic is submitted as part of a report in the Portfolio tab of ASSIST.

21


Can a diagnostic be deleted? Yes, diagnostics can be deleted by clicking the “Delete” button at the bottom of the diagnostic screen unless the diagnostic has been submitted as part of a report in the Portfolio tab. Submitted diagnostics cannot be deleted. Does the Executive Summary have to be published? Institutions engaged in the AdvancED Accreditation process are required to publish an Executive Summary prior to an External Review. Even a published Executive Summary can be updated at any time.

diagnose

Documents

assist diagnostics

advanced diagnostics

advanced accessing diagnostics

diagnostics questions

diagnostics executive

suite of diagnostic

following diagnostic

assist link