instructional effectiveness: proficiency rating scales...educator proficiency ratings popup window...

Instructional Effectiveness: Proficiency Rating Scales

Kerridan A. Kawecki, B.A.

Assessment Technology, Incorporated

6700 E. Speedway Boulevard Tucson, AZ 85710

800.367.4762 www.ati-online.com

[email protected]

©Assessment Technology, Incorporated 2016 “Galileo” and the Galileo logos are trademarks or registered trademarks of Assessment Technology Incorporated.

All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without

permission from the publisher. Printed in the U.S.A

http://www.ati-online.com/

mailto:[email protected]

Galileo® K-12 Online

©2016 Assessment Technology, Incorporated Revised as of 11.30.16


©2016 Assessment Technology, Incorporated

Revised as of 11.30.16 i

Instructional Effectiveness: Proficiency Rating Scales

Table of Contents

About Proficiency Rating Scales .................................................................. 1

Getting Started ........................................................................................... 1 Number of rating scales ............................................................................... 1 Standards .................................................................................................. 2 Tallied and Reporting ................................................................................... 2

Process ....................................................................................................... 4

Accessing Proficieny Rating Scales ............................................................. 5 Class-level User .......................................................................................... 5 School-level user ......................................................................................... 6 District-level user ........................................................................................ 8

Administer Proficiency Rating Scales ........................................................ 10 Tips ......................................................................................................... 10 Steps ....................................................................................................... 10

Print Proficiency Rating Scale ................................................................... 14 Steps ....................................................................................................... 14

Adding Comments ..................................................................................... 17 Steps ....................................................................................................... 18

Import and View Staff Files....................................................................... 20 Purpose ................................................................................................... 20 Tips ......................................................................................................... 20 Steps ....................................................................................................... 20

Post Results .............................................................................................. 23 Steps ....................................................................................................... 24

Monitor Progress ....................................................................................... 26 Steps ....................................................................................................... 26

Aggregate Analysis ................................................................................... 29 Steps ....................................................................................................... 29

Individual Results ..................................................................................... 32 Steps ....................................................................................................... 32 Proficiency Rating Results ........................................................................... 34 Scoring Details .......................................................................................... 35 Proficiency Rating Details ........................................................................... 36 Staff Response .......................................................................................... 38


©2016 Assessment Technology, Incorporated Revised as of 11.30.16 1

ABOUT PROFICIENCY RATING SCALES

WHAT IS A PROFICIENCY RATING SCALE? A Proficiency Rating Scale is a Galileo tool that allows districts and charter schools

to document staff members’ teaching performance, professional practice and leadership performance. Most states require that this component of a staff member’s evaluation be based upon observations and scored based on a rubric.

Galileo’s Proficiency Rating Scales may be used to create rubric-based forms to document both teacher and principal observations.

Additionally, Galileo’s performance rating scales are easily aligned to standards such as the Interstate New Teachers Assessment and Support Consortium (InTASC)

Professional Teaching Standards and the Professional Administrative Standards from the Interstate School Leaders Licensure Consortium (ISLLC).

OVERVIEW OF THE PROFICIENCY RATING SCALE TOOLS School districts and charter schools have expressed the need to develop, customize, administer and manage the implementation of their own teacher and principal evaluation tools. Galileo addresses this need by giving school districts and

charter schools online access to an Instructional Effectiveness Dashboard. Among other tools, the dashboard includes:

A Proficiency Rating Scale Administration tool

An Educator Proficiency Ratings Progress report

A Proficiency Rating Results report and

Staff File Import and View functionality.

GETTING STARTED

Assessment Technology, Incorporated (ATI) provides districts/charter schools with

the opportunity to work with our Education Management Services (EMS) department in building the Proficiency Rating Scales that will be utilized by the district/charter school.

Please contact ATI’s Educational Management Services ([email protected])

or your ATI Field Services Coordinator (800.367.4762) to discuss the number of Proficiency Rating Scales you need, to which, (if any) standards your Proficiency Rating Scales will be linked, and how the results should be tallied and reported.

NUMBER OF RATING SCALES It is possible to have more than one Proficiency Rating Scale created. For example, many districts/charter schools like to have their informal or “spot” observations

placed in Galileo, as well as the summative. Typically it is the summative that is tied to the final evaluation.

mailto:[email protected]


©2016 Assessment Technology, Incorporated 2 Revised as of 11.30.16

STANDARDS Each Proficiency Rating Scale in Galileo will be linked to standards. The standards may take many forms. They may be the InTASC, ISLLC, a propriety set of

standards, or custom standards. In the event your tool is not linked to standards, ATI will place the items to be observed in as standards.

TALLIED AND REPORTING The Galileo system will tally, present, and report out rating scale results through proficiency categories determined by the district/charter school. The naming and cut scores associated with each proficiency category are established when the

Proficiency Rating Scales are entered into Galileo. For example they might be:

Highly Effective, Effective, Partially Effective, and Ineffective

Distinguished, Proficient, Basic, and Unsatisfactory

The Proficiency Rating Results widget lists all of the domains and standards on a given rating scale and displays the proficiency category for each, based on how

staff members were rated on all the items linked to the standards.

Example 1:



Example 2:

Example 3:



PROCESS

To administer a Proficiency Rating Scale the following steps must be taken:

Step Information found in

1. Confirm with ATI’s EMS department that the proficiency categories for each Proficiency Rating Scale have

been set up.

Getting Started section, page 1

2. Determine what sources of

Evidence you would like to be available when scoring each item on

the proficiency rating scale. For example, Classroom Environment, Informal Observation, Student Work,

etc.

Getting Started section,

page 1

3. Determine if you will record the

Proficiency Rating Scale online or offline.

4. If administering offline, print the Proficiency Rating Scale so the

observation may be documented and later transcribed online.

Print Proficiency Rating Scale section, page 14

5. Administer the Proficiency Rating Scale according to your

district/charter school evaluation protocol.

a. Manually score items. b. Select the appropriate Evidence

for each item.

c. Record anecdotal notes. d. Add Evaluator comments.

Administer Proficiency Rating Scales section,

page 10

6. Import files, as needed. Import and View Staff Files section, page 20

7. Post Results to staff members. Post Results section, page 23

8. Monitor the continued

administration of Proficiency Rating Scales.

Monitor Progress section,

page 26

9. Analyze Aggregate Results. Aggregate Analysis

section, page 29



Step Information found in

10.Access Individual Results. Individual Results section, page 32

ACCESSING PROFICIENY RATING SCALES

Which Proficiency Rating Scale tools a user may access will depend on what level user an individual is and what Instructional Effectiveness (IE) permissions the individual user has been given. Below is a summary of each level and the access to

the various IE tools:

CLASS-LEVEL USER User may see the Proficiency Rating Results widget once the results are posted.

These are accessed from the Results tab on the Instructional Effectiveness Dashboard.

and



SCHOOL-LEVEL USER User may access the Instructional Effectiveness Dashboard’s Results tab, to

see their own evaluation results.

If provided the User Has Access to Instructional Effectiveness Tools permission, the user can administer and import files for Class-level users at his or her school and can see the Class-level users Proficiency Rating Results.

and



If provided the User Can Administer and View Instructional

Effectiveness Results for Staff of the Same User Level permission, the user can administer and import files for Class-level users at his or her school

and other School-level users at their school. This user can also see the Class- and School-level users’ Proficiency Rating Scale Results.

and



DISTRICT-LEVEL USER User may access the Instructional Effectiveness Dashboard page, to see their own evaluation results.

If provided the User Has Access to Instructional Effectiveness Tools permission, the user can administer and import files for Class-level and

School-level users and can see the Class-level and School-level users Proficiency Results.

If provided the User Can Administer and View Instructional Effectiveness Results for Staff of the Same User Level permission, the

user can administer and import files for Class-level, School-level users, and other District-level users. This user can also see the Class-, School-, and District-level users’ Proficiency Results.

(Image on next page.)



and

Refer to “A Guide for Galileo K-12: Configure Instructional

Effectiveness Security Settings” for permission options and how to establish such permission. Document is found in the Galileo Help Files.



ADMINISTER PROFICIENCY RATING SCALES

TIPS

The link is available even if no ratings have been

administered. The same rating scale cannot be administered to the

same educator twice.

STEPS School- and District-level users with the appropriate IE permissions may administer Proficiency Rating Scales by accessing two different links.

1. Click the Dashboard icon.

2. Hover over the Dashboard menu option and select Instructional Effectiveness.

3. Use the down arrow to select the desired School or to select All Schools

option.

4. Click the Administration tab.



5. Select the Filter by User Level option – School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets.

6. In the Administer Proficiency Ratings widget, click the Administer

Proficiency Rating Scales link.

6. In the Administer Proficiency Ratings widget, click the View all link or click

the bar column.

7. Proceed to the next step.

7. Click the scale icon to administer a

Proficiency Rating Scale.

8. On the Educator Proficiency Ratings popup window page, select the School.

9. Use the down arrow on Staff drop-down menu to select the educator.

10.Use the down arrow on the Library drop-down menu to select rating scale’s library.

11.Use the down arrow on Rating Scale drop-down menu to select the scale.



12. Enter the Observation Date or use the calendar to select the date.

13. The Proficiency Rating Scale will appear.

14. Rate each item that appears. Each item will contain the following elements:

a. The domain to which the evaluation item is aligned. This will show in

burgundy and in all capital letters.

b. The standard to which the evaluation item is aligned.

c. The evaluation item that is being scored.

d. The level of proficiency as determined by the evaluation rubric. The

level of proficiency selected determines the number of “points” earned

on this evaluation item.

e. Link to the scoring rules. Typically, an explanation of the rubric is

displayed. The Scoring Rules and Examples appear in a new window.

f. Expands to open a Notes box, where the observer may document

anecdotal information.

g. Allows the observer to document the Evidence that supports the

selected proficiency category.



15.For each item, click the appropriate proficiency radio button (e.g., Evident, Not Evident).

16.Use the down arrow on the Evidence field to select the evidence.

17.(optional) Click the Show notes link to open the notes box. Type in notes regarding that objective. Users typically note anecdotal information related to

what was observed or document why a particular proficiency category was selected.



18.Click the Hide notes link to close the notes box.

19.Repeat steps 15-18 for each item on the scale.

20.Click the Save Changes button located at the top or bottom of the page to

save the changes.

21. A pop-up will appear informing you of the items that have not been

evaluated or confirming that all items have been scored.

It is possible for an evaluator and the staff member to record comments on the Proficiency Rating Scale. To learn how to record and utilize this feature please see Adding

Comments section, page 17.

PRINT PROFICIENCY RATING SCALE

It is possible to print the Proficiency Rating Scale for a staff member. A Proficiency

Rating Scale may be printed as a worksheet or printed to serve as a record.

WORKSHEET: If the Proficiency Rating Scale has not yet been completed, it

can be printed as a worksheet to be completed by the evaluator offline. If used as a worksheet, the evaluator will need to enter this information into

Galileo.

RECORD: Once a Proficiency Rating Scale has been completed by an

evaluator, it can printed, and signed by the staff member and evaluator and kept as a record.

STEPS School- and District-level users with the appropriate IE permissions may print Proficiency Rating Scales by accessing two different links.

1. Click the Dashboard icon.




3. Use the down arrow to select the desired School or to select All Schools option.





6. In the Administer Proficiency Ratings widget, click the

Administer Proficiency Rating Scales link.


the bar column.


7. Click the scale icon to administer a Proficiency Rating Scale.

8. On the Educator Proficiency Ratings popup window page, select the School.

9. Use the down arrow on the Staff drop-down menu to select the educator.

10.Use the down arrow on the Library drop-down menu to select the rating

scale’s library.


12.To include notes when printing, place a checkmark in the Display note in

report box.




13.Click the printer icon.

14.A printable version of the document is generated.

Notice that signature and date lines are available on the printed copy enabling the copy to be signed and kept for

record-keeping.

ADDING COMMENTS

Once an evaluation has been conducted, the evaluator may add general comments.

These comments may be saved and made visible to the staff member when the Allow selected staff to view rating scale results permission is extended.

There is also a place for the staff member to enter a response to evaluator comments. Once a staff member has viewed his or her results and saved his or her

response, those comments will appear to the evaluator on the Proficiency Rating Scale.

Both evaluator comments and staff responses will also show at the bottom of the page and will appear on any printed copies of the Proficiency Rating Scale.



STEPS 1. Click the Dashboard icon.



option.





6. In the Administer Proficiency Ratings widget, click the

Administer Proficiency Rating Scales link.


the bar column.


7. Click the scale icon to administer a Proficiency Rating Scale.

8. On the Educator Proficiency Ratings page or popup window, select the

School.


10.Use the down arrow on the Library drop-down menu to select the rating

scale’s library.


12.Scroll to the bottom of the page to see two comment boxes.



13.The Evaluator Comments box allows the administrator to add general comments about the proficiency scale or other evaluation topics.

14.Once comments are added, click the Save Changes button.

While you may see the Staff Response, only the selected

staff member may add to this box. Once the selected staff member has added his or her response and saved it, it will appear here. Staff response instructions may be found on

Staff Response section, page 38.

IMPORT AND VIEW STAFF FILES

PURPOSE This feature allows evaluators to link important documents, to each staff member’s Proficiency Rating Scale. Many districts/charter schools, for example, like to build

and maintain portfolios for staff members. The import and view staff file capability allows for these documents to be housed and managed online.

TIPS

All file types are permitted.

File size is limited to 20 MB.







5. Select the Filter by User Level option – School, Class, or All. Filtering by

user-level will only display data for the selected user-levels in the widgets.



6. In the Administer Proficiency Ratings widget, click the Import and View Staff Files link.

7. The Import and View Staff Files popup window opens.

8. Use the down arrow on the School drop-down menu to select the school.

9. Determine if you’d like to filter using the Filter by user level (school, class, or all) before selecting a staff member.

10.Use the down arrow on the Staff drop-down menu to select the educator.

ATTACH A FILE 11.Type a File description in the file description field.

12.Click the Browse … button and locate the relevant staff file to upload.

13.Click the Add button. The file is now attached to this educator’s file.

14.For each attached file determine if the file should be Read Only. Checking the Read Only box will allow a staff member to only view the file. Unchecking the file will allow a staff member to remove the attached file.




15.For each attached file determine if the file should be posted for staff to see. If the file should be posted, place a checkmark in the Post for Staff checkbox.

VIEW A FILE 16.Click the binoculars icon to view the desired file.

DELETE A FILE 17.Click the trash can icon for the desired file.

POST RESULTS

Once a Proficiency Rating Scale has been completed, and appropriate notes and comments recorded, evaluators then post the results for staff members. Posting the results for staff members will allow staff members to access and see the Proficiency

Rating Scale results from the Galileo’s Instructional Effectiveness Dashboard’s Results tab page.

and



A detailed explanation of the Proficiency Rating Scale Results accessed from the Results tab may be found in the Proficiency Rating Results section, page 34.


2. Hover over the Dashboard menu option and select Instructional

Effectiveness.


option.






6. Click the Administer Proficiency Rating Scales link.

1.

6. Click the View all link or click the bar column.


7. Click the scale icon to administer a

Proficiency Rating Scale.

8. On the Educator Proficiency Ratings page or popup window, select the School.




10.Use the down arrow on the Library drop-down menu to select the rating scale’s library.

11. Place a checkmark in the Allow selected staff to view rating scale

results field.

12. Click the Save Changes button. The staff’s rating scale results will be

displayed on the staff’s Dashboard page.

Refer to the Proficiency Rating Results section, page 34, for directions on how a

staff member views their Proficiency Rating Scale results.

MONITOR PROGRESS

School- and District-level users with the appopriate IE permissions can monitor the progress of the administration of proficiency ratings scales. The Administer

Proficiency Rating Scales widget tracks completed proficeny rating scales in the bar graph. Intially, all staff memebers list in the “0” column. As administraors complete proficiency ratings scales on staff members, the staff members will be move to the

subsequent column(s). This allows administrators to know and easily access the proficiency rating scales of staff members who still need to be evulated.






option.




6. An explanation of the monitoring tools found in the Administer Proficiency Ratings widget is described below:



a. The total number of staff members represented in the graph.

b. Graphic representation of completed Educator Proficiency Rating Scales. There will always be a “0” column representing those staff

members who have no Proficiency Rating Scales completed. A column will be present in the bar graph for each Proficiency Rating Scale that needs to be completed on a staff member. As Proficiency Rating Scales are

completed staff members are automatically moved into the completed columns.

c. The column visually indicates the total number of staff members in a particular column. The number of staff members represented in that

column appears at the top of the column. Each column provides a drill down to a list of staff members in the column and provides direct access to the Proficiency Rating Scale(s).

d. This number indicates the number of Proficiency Rating Scales

completed.

e. Clicking the View All link displays a list off all staff members,

alphabetically who are represented in the widget.

7. To complete or view a Proficiency Rating Scale for a staff member click the

column, or the View All link.



8. Click the staff member’s name to access the most recent Proficiency Rating Scale or click the desired Proficiency Rating Scale.

9. Proceed to the Administer Proficiency Rating Scales section, step #8, page

11, for further instructions.

AGGREGATE ANALYSIS

The Proficiency Rating Scale Results by School widget summarizes the overall Proficiency Rating Scale scores at the selected schools. District- and School-level

users with the appropriate permission may drilldown within the selected school, to view a staff member’s Proficiency Rating Scale Score.


2. Hover over the Dashboard menu option and select Instructional

Effectiveness.






6. On the Proficiency Rating Scale Results by School widget, select the Rating Scale for which you wish to see results.

7. An explanation of the monitoring tools found in the Proficiency Rating Scale Results by School widget is described below:

a. The Rating Scales drop-down menu.



b. Displays the name of the Proficiency Rating Scale, the number of staff represented, the total number of items scored on the selected Proficiency

Rating Scale, and the total possible points for the rating scale.

c. Each column on this table can be sorted using the green down/up arrows.

d. Displays the average points earned out of total points possible.

e. Displays the average percent.

f. Displays the number of staff members included in the results.

g. The school is a hyperlink to drill down to view staff rating scales and scores.

8. Click the School name to drill down to see individual staff scores.

9. To sort the list, click the headings.

10.To print the list click the printer icon.

11.Click the Staff Name to see individual Proficiency Rating Scale Results in the Rating Scale Results window.




For information on how to interact with the individual Proficiency Rating Scale Results see the Individual Results section, page 32.

INDIVIDUAL RESULTS

Staff members may view their individual Proficiency Rating Scale Results from the

Results tab on the Instructional Effectiveness Dashboard page.





3. If a School- or District-level user, select the School. (The School defaults based on your Settings page.)

4. If a School- or District-level user with appropriate Instructional Effectiveness

(IE) permissions, select the Results tab.

IMPORT STAFF FILES 1. If an administrator has opted to “Post” file(s) for staff, a user may access

these by clicking on the folder icon.

2. The Staff Portfolio: Import and View Staff Files window will appear. The user will see any files the administrator/evaluator has imported and opted to

post.

3. By default each “posted” file provides “Read Access” only. Clicking on the

binoculars icon will allow the user to view the file.

4. If a trash can icon appears by the file, then the “Read Only” permission has been removed from the file, allowing the user to click the trash can and remove the imported file.



PROFICIENCY RATING RESULTS If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the results by navigating to the Results tab and

selecting the Rating Scale in the Proficiency Rating Scale widget.

The results of the Proficiency Rating Scale will appear:

a. Total points earned on the rating scale out of total points possible.

b. Date that the Proficiency Rating Scale was administered.

c. The proficiency category designated for the domain as set up by the district/charter school.

d. The domain that is being tallied and reported on.

e. The proficiency category of the individual standards associated with each item.

f. The standards that make up the domain (d). The standards are directly aligned to the items scored on the Proficiency Rating Scale. The actual items

on the scale may be viewed by clicking the Proficiency Rating Scale details.



SCORING DETAILS If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the scoring details by navigating to the Results tab

and selecting the Rating Scale in the Proficiency Rating Scale widget. Then they may click the Scoring Detail link.

The scoring details will appear:

a. The staff member’s name.

b. The Proficiency Rating Scale selected.

c. The total points earned out of the total points possible.

d. Domains and the total points earned out of the total points possible for a domain. Please note, it is possible to have more than one goal make up a

domain.




e. The standard and the total points earned out of the total points possible for a particular standard.

f. The item scored and the total points earned out of the total points possible for an item.

g. Printer icon that allows the scoring details to be printed.

PROFICIENCY RATING DETAILS If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the Proficiency Rating Scale Results Details by navigating to the Results tab and selecting the Rating Scale in the Proficiency

Rating Scale widget. Then they may click the Proficiency Rating Details link. This interface allows the user to see the interface as evaluators see it, including how

each item was scored and which proficiency category was marked for each item.



The proficiency rating details will appear:

a. The date the Proficiency Rating Scale was administered.

b. Provides the ability to print the Proficiency Rating Scale. The printed version

of the scale has signature and date lines for record keeping.

c. Checking the Display note in report check box includes notes in the printed

version of the report.

d. The domain of the standard and item being assessed.

e. The rubric used to score the Proficiency Rating Scale.

f. The standard being assessed.

g. The item being evaluated and scored.

h. The proficiency category documented for the item. This determines the number of points earned for this item.

i. Clicking Show Notes will allow the user to see any notes recorded by the evaluator.

j. Shows the Evidence used to document the assigned proficiency category for the item.



Scrolling down to the bottom of the page will show general Evaluator Comments, and a place for Staff to respond. For instructions on responding, see the Staff

Response section, page 38.

STAFF RESPONSE If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the Proficiency Rating Scale Results details and record a response. This is done by:

1. Navigating to the Results tab and selecting the Rating Scale in the

Proficiency Rating Scale widget.

2. Click the Proficiency Rating Details link.

3. Scroll to the bottom of the page.

4. Simply type in your response

5. Click the Save button.

instructional effectiveness: proficiency rating scales...educator proficiency ratings popup window...

Documents