administrator manual - cae.org · this commitment helps cae and its member institutions maximize...

ADMINISTRATOR MANUAL

Please do not bookmark the Proctor Interface or the Student Interface.

Copyright © 2018 Council for Aid to Education

WebsitesProctor Interface: http://admin-cla.cae.orgStudent Interface: http://cla.cae.orgUsers Portal: http://www.cae.org/CLAusersportal

Contact Information

ITS Technical SupportMonday-Friday, 8 AM-8 PM ET(800) 514-8494Outside of business hours and on weekends, press 1 to be taken to after-hours support or 2 to leave a voicemail. (you will be contacted the next business day)

General [email protected](212) 217-0700

1

CLA+ Administrator Manual

CLA+ represents the next evolution of the Collegiate Learning Assessment: a progressive, problem-based assessment that has been utilized by institutions of higher education for the past 10 years. Member institutions have been using CLA+ to determine if their graduating students garner the higher-order skills needed to succeed beyond college. Long-term CLA+ member institutions know that CAE is committed to regular product enhancements. This commitment helps CAE and its member institutions maximize the utility of CLA+ results for institutional improvement. Along with that commitment is CAE’s dedication to ensuring that the data provided are comparable over time. This commitment remains true as CAE moves into the next decade of our work.

CLA+ introduced additional subscores (scientific and quantitative reasoning, critical reading and evaluation, and the ability to critique an argument) to complement the subscores we’ve provided in the past (analysis and problem solving, writing effectiveness, and writing mechanics). All of these data points together enable greater opportunities for faculty and administrators to investigate or diagnose specific areas of student, cohort, programmatic, or institutional achievement.

We thank the over 600 institutions of higher education that have participated in CLA+ to date. Were it not for regular feedback, we may not have realized the communal need for these enhancements. Our fundamental goal at CAE has been and remains the better alignment of assessment with teaching and learning.

Executive Summary

CLA CLA+The Instrument Performance Task (90 minutes) Analytic Writing Tasks (75 minutes) Performance Tasks (60 minutes) Selected Response-Questions (30 minutes)

••

••

Skills Assessed Analysis and Problem Solving Writing Effectiveness Writing Mechanics Critical Reading and Evaluation Scientific and Quantitative Reasoning Critiquing an Argument

•••

•

••••••

Results Summary Results: Freshmen & Seniors Summary Results: Sophomores & Juniors Reliable for Interpretation at Institution Level Reliable for Interpretation at Student Level Norm-based Results Mastery Level Results Data on Student Motivation

•

•

•

•••••••

2


Table Of Contents3 General Contact Information

29 Appendix A: Sample Custom Questions

4 Quick Reference for Proctors

30 Appendix B: Glossary of CLA+ Terms

5 In Advance of Testing5 Overview5 Room and Staffing Arrangements5 Check Readiness7 Contacting Technical Support7 Accommodations for Students

Requiring Additional Time7 Proctor Demonstration Capabilities7 Consent Forms7 The Proctor Interface7 Entering the Proctor Interface and

Setting Up Users8 Home Page9 Creating New Users10 Custom Questions

26 Configuration Requirements26 Operating System & Browser Requirements26 Recommended Connection Speeds and Software26 Visually Impaired Student Accommodations26 Please Contact If...27 Common Testing Issues & Troubleshooting27 Scenarios28 Communication Problems

12 On the Day of Testing12 Proctor Perspective12 Checking Materials12 Creating a Session13 Proctor Session Details14 Student Roster15 Importing Students16 Standardizing Testing16 Proctor Script17 Student Perspective17 Student Login Instructions17 Student Test Process17 Secure Browser18 Session Number19 Proctor Approval20 Re-Entry Procedures20 Login Requirements & Student Roster

22 After Testing is Complete22 Registrar Data22 Accessing the Registrar Data Tool23 Entering Mean SAT or ACT scores23 Registrar Data Template24 Review and Submit Data25 Accessing Reports25 Email Addresses for Spring Score Reports

3


The CLA+ testing platform is compatible with most systems, but there are some exceptions. Review the list below, and contact your CLA+ Rep if your institution is planning to use anything listed. We may provide additional support for systems that aren’t directly compatible with our testing requirements.

IF YOUR INSTITUTION PLANS ON USING:• Thin clients• Virtual machines• XBoxes• Cloud environments• Remote desktops• Tablets• Remote proctoring• iPads with keyboard• Chromebooks

IF YOU NEED ADDITIONAL INFORMATION ON:• Instructions for creating a demonstration session• Informed consent form template • Sample correspondence emails and suggested timeline• An explanation of the CLA+’s value-added methodology• Special accommodations or extended timing for students

CONTACT INFORMATION

CONTACT YOUR REP IF...

General Contact Information

ITS Technical SupportMonday-Friday, 8 AM-8 PM ETToll Free: (800) 514-8494Local: (443) [email protected]

Outside of business hours and on weekends, press 1 to be taken to after-hours support or 2 to leave a voicemail (you will be contacted the next business day).

General [email protected](212) 217-0700

Ashley BrailsfordPartnership Development [email protected](212) 217-0850

Lee FinkelsteinProgram Manager, [email protected](212) 204-8420(215) 280-0405

Zac VarieurProgram [email protected](212) 217-0815

Mi Sun KwonAssistant Director of Assessment [email protected](212) 217-0865

Kelly RotholzDirector of Assessment [email protected](212) 217-0839

4


The following list summarizes the proctoring procedures that occur during each day of testing.

1. Prior to testing:• Log into the Proctor Interface (http://admin-cla.cae.org) to ensure you have access to the system and

the ability to create Test Sessions.• Use the Check Readiness site, found on the upper right hand corner of the Proctor Interface, to run the

system check, evaluate your network, and utilize the advanced readiness and troubleshooting tools.

2. Before students arrive:• Make sure you have the phone number for your institution’s computer services/help desk in case of

technical difficulties. • Log in to the Proctor Interface and set up the testing session:

· Go to http://admin-cla.cae.org. · Select Test Delivery. · Enter a Session Name and select the Test Administration. · Click Create Session, which will bring you to the Proctor Session page. · Record the Test Session Number.

• Write the session number on the classroom board.• Place a few pieces of scrap paper at each computer station (optional). All scrap paper should be

collected and disposed of after testing is complete.

3. Administering CLA+ to students:• Take attendance and place a sign on the outside of the classroom door noting that CLA+ testing is in

progress.• Direct students to http://cla.cae.org.• Assist students with downloading of the secure browser.• Read the Proctor Script.• Provide students with the Test Session Number.• Use the Waiting for Approval table to approve students into CLA+.

· Verify the students and click Approve All Waiting; or · Approve students individually by clicking Approve next to each name.

• If students ask you questions during the exam, refer them to the Help button on their screen.• Collect all scrap paper before students leave.

If a student has moved past the Performance Task by accident, have the student stop work, and contact your CLA+ Rep immediately. If a student has issues logging in or has any other technical irregularities, call ITS Technical Support at (800) 514-8494 immediately. Contact your CLA+ Rep if your problem is not resolved.

4. When the institution has completed CLA+ testing:• Contact your administrator or CLA+ Rep to confirm all testing at your institution is finished.• Coomplete and submit Registrar Data using the downlowded template.

Quick Reference for Proctors

5


OVERVIEW

CLA+ is delivered online through a secure browser and is designed to be completed in approximately 90 minutes. Total time spent on CLA+ will vary by each individual student, so please expect the students to finish at different times. The assessment will include the completion of a 60-minute Performance Task and a 30-minute section of Selected-Response Questions. CLA+ must be administered under standardized and controlled testing conditions, with all students monitored by a proctor.

It is highly recommended that you review all of the directions in this manual prior to the first test session so you are familiar with all aspects of test administrations. If you are coordinating the efforts of several proctors, we suggest an orientation for everyone in which the contents of the Day of Testing Guide and CLA+ instrument itself are reviewed. In addition, online video trainings for proctors are available on the Users Portal at http://www.cae.org/CLAusersportal. These proctor training videos can be accessed at http://www.cae.org/CLAtrainingvideos. All proctors are encouraged to view the training videos prior to administering their first test session for content updates.

ROOM AND STAFFING ASSIGNMENTSThe test sessions should take place in a computer laboratory (or a room equipped with one computer for each individual taking the exam) with a high speed Internet connection. Students should have enough space to work on a computer without distracting other students. The proctor must have access to a phone during the exam to contact Technical Support if needed. Staff should be familiar with the computers in order to assist with hardware or software problems. We recommend at least one proctor per 30 students and a minimum of one proctor per room (if multiple rooms are used).

On the day of the assessment, we recommend that you arrive 30 minutes in advance and allot approximately two hours for each session of testing. Put a sign on the door indicating that the room is in use for testing. Upon arrival at the testing location, find the nearest bathrooms so that you can advise students of their locations accordingly. Students should be advised to use the restroom before beginning the test.

You may supply scrap paper to students for use during the assessment. Remember to collect all scrap paper from students at the conclusion of the assessment. The use of a calculator is not allowed.

Prior to testing, we recommend that Proctors check student identification before students are approved into the testing environment.

CHECK READINESSPrior to testing, we recommend that you utilize the Check Readiness site to run the System Check. Also available are tools to evaluate your network, perform a test delivery check, and run a bandwidth check. The Check Readiness site can be found by going to http://admin-cla.cae.org and clicking Check Readiness in the upper right hand corner.

We recommend that you use the Check Readiness site with help from your institution’s IT Department. They will be able to interpret the information provided and identify any potential problems.

Please note: Both the Student Interface and the Proctor Interface are supported by Windows and Mac operating systems. Review the CLA+ Configuration Requirements at the end of this manual for browser compatibility information. No specific software is required, but you may need to make adjustments to your current settings.

In Advance of Testing

6


CHECK READINESS (CONTINUED)

7


CONTACTING TECHNICAL SUPPORTIf you require technical support during a testing session between 8:00 AM and 8:00 PM Eastern Time, Monday through Friday, please call (800) 514-8494. You will be connected to the next available support representative at Internet Testing Systems (ITS). Additionally, there is a Contact Support function available on the top right hand corner of the Proctor Interface. This function allows you to email and/or chat with tech support. The chat function is only available during regular business hours. For most effective support, contact ITS while you are experiencing your technical difficulties.

If you require technical support outside of the hours stated above, please call (800) 514-8494, press 1, and wait for the system to locate an on-call support representative. You can also dial 2 for the voicemail, and will be called back the next business day. Please be patient while the system pages an on-call technical support representative.

ACCOMMODATIONS FOR STUDENTS REQUIRING ADDITIONAL TIMEIf any students require additional time for CLA+ or need a reader (provided by your institution), please contact your CLA+ Rep at least two weeks in advance of the scheduled testing date. Documentation from your school, including a signed letter on letterhead verifying the student’s accommodations, will need to be provided for the students at least one week in advance of testing. Your CLA+ Rep will provide guidance for the accommodation. Additional time is available for both students testing remotely through ProcterU and those testing locally.

PROCTOR DEMONSTRATION CAPABILITIESContact your CLA+ Rep for instructions on creating demonstration sessions. If you wish to practice proctoring and student testing procedures prior to a scheduled test date, or if you would like your faculty and/or proctors to view CLA+, you will need to create a separate demonstration session.

CONSENT FORMSRecognizing that securing student consent is a process best directed by individual schools, CAE does not require informed consent forms. Please consult your Institutional Research office to determine your school’s requirements. For your convenience, electronic consent form templates are available upon request from your CLA+ Rep.

The Proctor Interface was developed to provide individual institutions flexibility in managing CLA+ testing on campus. Proctors are responsible for setting up the test session(s) on each day of testing; each individual test session will expire at 11:59 PM local time on the day of the test. Sessions cannot be set up prior to the day of testing.

Only registered users will have the ability to access the Proctor Interface and administer CLA+. Returning users will use the same username and password they created in previous years of testing. New users will receive information once they submit their testing plan. If you are unable to remember your password, please access the Forgot Your Password? feature on the Proctor Interface.

If you plan to host one session at a time, you will only need to register one proctor. If your institution plans to host multiple test sessions simultaneously, each proctor must have a unique login for the Proctor Interface. Details on creating additional users appear later in this guide.

ENTERING THE PROCTOR INTERFACE AND SETTING UP USERSTo access the Proctor Interface, type in the following URL: http://admin-cla.cae.org. Please be sure to type the URL each time you visit the website. Please do not attempt to access the Proctor Interface by bookmarking the site, as doing so may cause complications during testing.

After entering your username (typically your institutional email address) and password, click Submit. If it is your first time administering the CLA+, you will be asked to change your Proctor Interface password upon your initial login for security purposes. If you have previously administered CLA+, please note that your password will remain the same. Please contact your CLA+ Rep if you have any difficulty logging in to the Proctor Interface.

THE PROCTOR INTERFACE

8


HOME PAGEOnce logged in, you will see the Home Page. This page provides a brief welcome message and table outlining the features available on the Proctor Interface. The available components appear within the menu on the left-hand side of your screen.

9


CREATING NEW USERSPrimary contacts are granted the ability to create and modify users (adding additional users) for your institution. Only users with permissions set to Create and Modify Users will be able to view the Users Page. From the Home Page, click the Users component on the left-hand side of your screen, which will bring up a list of all currently approved users at your institution. Users that are created with Administrative Access set to None will not be able to view the Users Page when logged in to the Proctor Interface. Institutional users are able to set accounts as Inactive, but are unable to delete old accounts. Contact your CLA+ Rep if you would like to permanently remove any users.

To add additional users, click Create a New User near the top right of your screen and follow the steps below:1. Complete the user’s profile information by providing their First Name, Last Name, and Email.

• Please note that the email address provided must be a functioning email address. The email address will become their username.

2. If you would like this new user to have the ability to create and modify user accounts for your institution set their Administration Access to Create and Modify. Be mindful that this allows individuals to delete other users.

3. Under Program Access, check the Enable box and choose the user’s role.• Proctors are able to administer CLA+, look up student information, and configure Custom Questions.• Administrators are also able to administer the CLA+, look up student information, and configure

Custom Questions. They are also able to submit Registrar Data, and access your institution’s Reports.4. Lastly you are able to add and preview a personalized message to the activation email that the user will

receive.

10


CUSTOM QUESTIONS (OPTIONAL)Custom Questions allow institutions to create unique multiple-choice questions giving the chance for more customized data collection. Results will be collected and sent to your institution in your Student Data File along with students’ CLA+ results. Your institution may practice setting up the Custom Questions by requesting a demonstration. Custom questions must be set up before students test and cannot be edited once your institution begins testing. Contact your CLA+ Rep for more information on acquiring access to the demonstration event or if you would like for your Rep to review your Custom Question set up prior to testing.

User TipPlease note that any Custom Questions you create will automatically be delivered to all students.

To create Custom Questions for your institution, select Test Administrations from the menu on the left. This will bring you to the Test Administrations Page, which will display the Active Test Administrations to which your institution has been assigned. Fall [current year] CLA+ will be listed during the fall testing window and Spring [current year] CLA+ will be listed during the spring testing window (see next image).

Use the Active Test Administrations table to identify the Test Administration to which you would like to add Custom Questions and click Custom Questions under the Manage column. Doing so will bring up the Configure Custom Questions page, as illustrated in the next image.

On the Configure Custom Questions page, you may configure up to 20 custom questions, with up to six response options per question. Instructions on adding Custom Questions appear in the yellow box at the top of the screen. Please keep in mind that response options must be separated by a vertical bar ( | ) without commas or spaces. To create a vertical bar, hold the shift key and press the backslash key.

• Please avoid using hyphens, dashes, or ampersands in your survey questions and response options.• To create ratios, such as “1:2,” enter the ratio in text form: “One to two.”• Character limits are 200 for each question and 400 for each multiple-choice response.

If questions and answers are not configured by following the above parameters, there may be errors in the rendering of the data in your institution’s student data file.

Once you have entered all of your questions and response options, click Save near the top right of your screen. An expanded list of sample questions, with associated response options, is available in this document’s appendices.

11


During the CLA+ testing window, proctors will have the ability to view aggregate student responses to the Custom Questions. To access these responses during a testing session, click Custom Questions on the Test Administrations page.

Once you click Custom Questions, you will be brought to the View Custom Questions page, shown below, which will show the aggregated results as the students test. The results will also be available after testing is complete. This page provides counts and percentages of students across available responses for each Custom Question. Students who choose not to answer a question will be totaled under No Response.

12


PROCTOR PERSPECTIVE

On The Day Of TestingThis section details the steps necessary to proceed with CLA+ on the scheduled date and time of a test session. Remember to utilize the Check Readiness tools prior to testing; this can be found by clicking on the Check Readiness link on the Proctor Interface log in page.

CHECKING MATERIALSOn the day of testing, we suggest you bring the following:

• The Day of Testing Guide, paying attention to the following sections in particular: · Quick Reference for Proctors · CLA+ Configuration Requirements

We encourage you to distribute copies of the Day of Testing Guide to all proctors for reference at each test session.

CREATING A SESSIONIn order for students to access the test, a proctor must first initiate the login process by creating a test session from the Proctor Interface. Each test session will contain a unique ID number that students will use to login.

To create a test session from the Proctor Interface, select Test Delivery on the left, which will bring you to the following screen. If any test sessions have already been created for the day, they will appear in table format as shown in the image below. All sessions must be created on the day of testing.

The three steps required to create a new session are outlined below. If all testing will take place during one sessionon one day, we recommend creating unique test sessions for each proctor so attendance and approving students into the interface may be done carefully and accurately. All other testing scenarios will require the creation of multiple testing sessions. Sessions must be created on the day of testing (not before) and will expire at 11:59 PM local timeon the day of the session.

13


PROCTOR SESSION DETAILSOnce you have created a session, you will be brought to the Proctor Session page. This page will provide the Test Session Number, which students use to access CLA+. You will also use this page to approve students into CLA+. The next image illustrates what the page will look like if there are no students waiting for proctor approval.

While students are waiting for approval, their names will appear in the Waiting for Approval table. You have two options to approve the students to test: you may click Approve next to each individual student name, or you may click Approve All to approve all of the students as a group.

1. Enter a Session Name under Create a New Session. The name of the session can be anything you wish. We recommend using unique session names for each individual session. As an example, you may name your session after the proctor, location, and the time of testing.

2. Select your CLA+ testing event from the Test Administration drop-down menu. Standard four-year institutions and two-year institutions will select the Fall [current year] CLA+ Test Administration in the fall testing window and the Spring [current year] CLA+ Test Administration in the spring testing window.

3. Click Create Session.

14


VIEWING THE STUDENT ROSTERTo view a roster of the students who tested within the active CLA+ testing window, select Test Administrations on the left-hand menu and then click Roster under the Manage column. This will bring up the Roster page, where you can review the students who tested Locally or Remotely. This list will be populated with all students who have tested within the active CLA+ testing window. Please note that only users with Administrator privileges have access to the Roster.

15


Administrators will need to import a .csv file with all student information in order to preregister students. Please note that each student must have a unique Student ID and Email. Students testing locally should be coded as “0” in the Remote column. Should a student have already taken the CLA+, please ensure that the same login and data are being used, otherwise the data will not import.

Once the file is imported, the administrator can review the student roster. When imported students log in to test, they should indicate Yes, I have Registered and enter their information exactly as it was imported by the Administrator. Imported students will not be directed to the profile page.

To import a list of students, select Import Students and follow the step-by-step instructions provided. For students testing remotely, you may download the list of authorization codes by clicking the download button at the bottom of the roster.

IMPORTING STUDENTS (OPTIONAL)On the Test Administration page, the administrator will see their current test administration. To preregister students for testing, select Roster. Only users with Administrator privileges have access to the Roster. If your institution is planning to use this feature for Remote Proctoring, please contact your CLA+ Rep for further instructions.

16


STANDARDIZING TESTING

STANDARDIZING TESTING: PROCTOR SCRIPT OR TEST OVERVIEW VIDEOProctors have two options before students enter the Student Interface:

• Read the Proctor Script • Instruct students to watch the Test Overview by going to http://www.cae.org/testoverview.

Please read the script in the bordered box to students when you are ready to begin taking CLA+. It is important to adhere to this script because it permits standardization of testing environments within and across institutions.

PROCTOR SCRIPT

Thank you for coming today to take CLA+. Your performance on this test will help (name of college or university) assess how it contributes to your critical-thinking and written communication skills. It is important that you put forth your best effort on this assessment.

As a test-taker, you will complete a Performance Task and a group of selected-response questions. The Performance Task is an essay-based question and requires a written response. Please be sure that you have finished writing before moving past the yellow warning screen to begin the selected-response questions. The entire testing process will take up to 90 minutes*.

For test security reasons, we ask that you do not discuss the specific contents of the exam with other students.

Testing will occur through a secure Internet browser; you will not be able to access websites or computer files while testing. Please launch CLA+ by going to http://cla.cae.org (or clicking the “CLA and CWRA Browser” icon if one appears on your computer’s desktop).

When you are ready, please enter the following Test Session Number: ______________, and fill out the requested information in the student profile presented to you.

Please do not talk during the assessment. Let me know if you have any questions or difficulty with your computer. If for some reason you are unintentionally logged out of the assessment, please notify me immediately.

It is vital that proctors do not comment on, or add in any way to, the assessment. During the session, if the student has a task-related question, say, “Do the best you can” or refer them to the Help button on their computer screen.

*Adjust time for students receiving extended time. Students receiving time and a half (50%) will have 135 minutes. Students receiving double time (100%) will have 180 minutes.

17


STUDENT PERSPECTIVE

STUDENT LOGIN INSTRUCTIONSThis section outlines the CLA+ experience from the student perspective. To begin the assessment, students’ computers must be open to the CLA+ Student Interface: http://cla.cae.org.

As a precaution please do not allow students to use a bookmarked version of this website.

CLA+ STUDENT TEST PROCESSThe diagram below briefly shows the process that students will move through during CLA+ testing.

Student Interfacehttp://cla.cae.org

Survey of Motivation andBackground Information

Enter Test Session Number(provided by your proctor)

Selected-ResponseQuestions

Student will click Start Online

Performance Task

Student Registration

Selected-ResponseQuestions Tutorial

Review Test OverviewPage

Review PerformanceTask Tutorial

SECURE BROWSERStudents will take CLA+ through a secure browser, which prohibits access to any other programs. To download the secure browser, students testing on a PC will click the Download Button on the Student Interface. They will select Run to access CLA+ in secure browser mode. Mac users will click the Download Button, prompting the browser to automatically begin downloading to a default location (usually the desktop). If the computer does not automatically unzip the file, double-click the file to decompress the secure browser. At this point, Mac users may double-click the icon on the desktop and begin taking CLA+.

The secure browser will auto-update on dekstops and laptops. Once downloaded, you will not need to download an updated version.

18


SESSION NUMBERStudents will enter in the relevant Test Session Number provided by the proctor and click Submit. Next, students will enter their email addresses. Student score notifications will be sent to the email addresses provided here unless the institution indicates otherwise.

First time test takers, when asked if they have registered, should select No, I Have Not Registered and hit Submit. Students will then be prompted to provide their last and first names, student ID, and date of birth. Students who have previously taken CLA+ (in most cases as freshmen) should enter the same email address they used previously and select Yes, I Have Registered. They will then be prompted for their date of birth. If a student tested in the past, but is having issues logging in, the proctor can assist the student by looking up registration information through the Students component of the Proctor Interface.

Student will then see a Waiting for Approval screen. They will not be able to begin CLA+ until they receive proctor approval.

19


PROCTOR APPROVALAt this point, the proctor must approve students for access to CLA+. As a reminder, this process takes place on the Proctor Session page, which appears after the session has been created. The proctor will approve students into the test once they appear in the Waiting for Approval table. This page automatically refreshes every 30 seconds. The proctor must approve each student into the exam.

Once approved into the Student Interface, students will see the Start Page. Students should click Start Online and CLA+ will launch. Before starting each section of the exam, students will have an opportunity to view a short tutorial on responding to Performance Tasks and completing the selected-response questions. Proctors should encourage students to read through the tutorial to familiarize themselves with the Student Interface prior to their task.

Students first complete the Performance Task, and then move on to the selected-response questions. Students will view a yellow warning screen to confirm they have completed the Performance Task, and would like to move into the selected-response questions. Once a student clicks past the yellow warning screen, they will be unable to return to the Performance Task.

20


RE-ENTRY PROCEDURESIf any technical disturbances are encountered and a student loses connection to the testing platform, they are able to re-enter the exam. Work is saved automatically every 10 seconds. To log back in the student will need to relaunch the secure browser and log back in using the same email address and indicate that they have registered before.

If the student mistakenly exits the exam before they are finished, please contact your CLA+ Rep for further instructions.

If the proctor has technical problems, contact Tech Support immediately at (800) 514-8494.

LOGIN REQUIREMENTS & STUDENT ROSTERStudents that have taken CLA+ previously will login with their email address and should select Yes, I Have Registered and enter their date of birth. Any student who has taken the exam at your institution in previous year(s) will need to log in using the same information they provided in prior test administrations.

For students who cannot remember their email addresses to login, the proctor can look up their login credentials in one of two ways. The simplest method is selecting the Students menu item, which allows the proctor to search for students by name or ID number in the search bar.

The proctor can also use the Browse Students table and search by All students or By Last Name by selecting the associated hyperlink. Please note, using either of the features in the Browse Students table will bring up all students who have ever tested at your institution; it will not filter by the active CLA+ test administration. The proctor’s search will return the student’s ID, last name, first name, middle initial, date of birth, and username (email address).

21


To view a roster of the students who tested within the active CLA+ testing window, select Test Administrations on the left-hand menu and then click on the hyperlinked Roster. Doing so will bring up the Roster page, where you can review the students tested Locally or Remotely. These lists will be populated with all students who have tested within the active CLA+ testing window.

22


After Testing Is Complete

Registrar data must be submitted for report generation, and to confirm the accuracy of reporting. Institutions should carefully review the instructions for descriptions and accepted ranges for requested student-level data in the Registrar Data Template. If any of the requested information is not available for a student, please leave the field blank. Students must have the appropriate Class Level to be included in the analysis. Please note that Class Level is the only required field in the Registrar Data Template. Information such as GPA or Test Scores is option and will be included in the Student Data File if reported.

It is very important that correct data is imported and submitted. Once a report has been generated, you are unable to correct your imported data. We ask that you review your import summary (Step 4) carefully before you submit

ACCESSING THE REGISTRAR DATA TOOL1. Click on Test Administrations in the navigation menu on the left-hand side of the Proctor Interface.2. In the Active Test Administrations table, click on Registrar Data located in the right-most column. (If you do

not see this link, your user access is not designated as Administrator on the Proctor Interface; please notify the primary CLA+ contact at your institution).

3. You will be redirected to the Registrar Data page, which outlines the process for entering the requested data.

REGISTRAR DATA

23


Depending on the reporting you will receive, not all columns will be appear on your template. If you have any questions please contact your CLA+ Rep.

ID Number, Date of Birth, First Name, Middle Name, and Last Name (Columns A-F): These columns and the column headers cannot be altered while submitting registrar data or there will be an import error. Templates are pre-loaded with information in columns A-F; this information is provided by students in the profiles that are completed as a part of CLA+. If necessary, institutions can correct information in columns A-F when they receive the Student Data file with their reports.

Program Code (Column G): Program Codes are only used for very special cases that CAE has preapproved with an institution. This column should be left blank, unless otherwise instructed by your CLA+ Rep.

SAT/Mathematics (Column H): This field is optional. Enter each student’s section score in values between 200 and 800. If a student does not have an SAT Math score, this column should be left blank.

SAT/Critical Reading (Column I): This field is optional. Enter each student’s section score in values between 200 and 800. If a student does not have an SAT Critical Reading score, this column should be left blank. This section is now called Evidence-Based Reading and Writing.

SAT/Writing (Column J): This field is optional and can be reported for students who took the SAT from 2006 to 2015. Enter each student’s score in values between 200 and 800. If a student does not have an SAT Writing score, this column should be left blank.

REGISTRAR DATA TEMPLATE1. Click on Download the Registrar Data Template. The downloaded template is sorted by last name, first name,

and then middle initial.2. Save this file to a known location on your computer, such as your desktop.3. All pre-populated information in the template is locked and cannot be modified.4. Once you have entered in all available data, save the file, and import to the Proctor Interface.

ENTERING MEAN SAT OR ACT SCORES• Report only for freshmen in fall testing or for seniors in spring testing.• The mean score provided should be of the entire population, not just the students sampled. If you are not

testing any freshmen in the fall or seniors in the spring you may skip this step. Please note that reporting mean SAT or ACT scores is optional.

24


REVIEW AND SUBMIT THE DATAOnce the data has been imported, click Return in the top-right corner. On the Registrar Data Tool page you will now see two additional steps to review to ensure data accuracy. Please carefully review the summary and compare to the imported template.

Review Import Summary: Please confirm that students have been coded with the appropriate Class Level. To be included in the value-added analysis students must be entering students in the fall and exiting students in the spring administration.

• This section will review a brief summary of all the data that has been imported, such as the number of students coded at a given class level.

• If there is uncertainty or an error, please correct the data in your template and reimport it on the Import Your Data page.

• If there are questions after viewing this summary, please contact your CLA+ Rep.• The Review Import Summary will show the number of incoming students in the fall and upperclassmen in the spring.

ACT Composite (Column K): This field is optional. Enter each student’s score in values between 1 and 36.

Cumulative GPA (Column L): Enter each student’s most recent cumulative GPA, only including courses completed at your institution or an affiliated site. This variable is not collected for entering students.

Transfer Status (Column M): Transfer students are classified as any student that has previous college credit from another higher education institution. Excluding first time freshmen. Enter each student’s transfer status by entering a 0 for a native student or a 1 for a non-native or transfer student. This column is not available for cross-sectional reporting in the fall.

Class Level (Column N): This column is used to confirm a student’s class standing, and is necessary for reports to be generated. Coding has bearing on the inclusion of student results in data analysis. Do not leave this column blank and only use numbers, which may be referenced below.

If there are less than five students in a Class Level, please enter “1” in the Program Code column for all students in that Class Level.

Enter each student’s Class Level using the following code: 1=entering student/freshmen, 2=sophomore/exiting community college student, 3=junior, 4=senior/exiting student, 5=graduate student/other.

First-time, full-time students with prior earned credit should not be coded as sophomores. Community colleges should code exiting students as 2. Coding has bearing on the inclusion of student results in data analysis.

25


ACCESSING INSTITUTIONAL AND STUDENT REPORTS

GENERAL INFORMATIONReports will be available on the Proctor Interface four to six weeks after your institution has completed Registrar Data and all student responses have been scored. You will receive notification from your CLA+ Rep when institutional reports are available and student results have been sent.

USER ACCESSOnly users who have their roles set as Administrator have access to download reports at your institution. The primary point of contact at your institution is designated as Administrator by default and has access to results.

DIRECTIONS FOR ACCESSING INSTITUTIONAL REPORTSTo view institutional results, select Test Administrations on the left-hand menu and then click on the hyperlinked Reports. Doing so will bring up the Reports page, where you can download the Student Data File, Student Data File Key, Institutional Reports, and Student Reports.

EMAIL ADDRESS FOR SPRING SCORE REPORTSStudent score reports are generated for each student that takes CLA+. Students will access their score reports through their notification email, which directs them to the CLA+ Student Portal. It is recommended that students use a personal email address in order to ensure access to score reports, especially if students lose access to school email addresses after graduation. If Registrar Data has not yet been submitted, it may be possible to change an email address by contacting your CLA+ Rep.

Please note: Student reports are sent out approximately one to two weeks after institutional reports. Please ensure any changes that need to be made are done before the Registrar Data is submitted.

Submit Registrar Data: This indicates that once registrar data is submitted, all testing will be marked complete and your institution cannot deliver additional exams this administration.

• Only click Submit if you are certain that all testing at your school is complete and that you have entered and imported all available registrar data.

Please ensure that all information is correct before submission by reviewing the Import Summary provided in Step 4. If a report generates with incorrect registrar data and must be corrected, your school will be subjected to a $300 fee for report regeneration. You may ask your CLA+ Rep to review imported data before submitting.

26


Configuration RequirementsThis page includes a list of supported and recommended operating systems, browsers, connection speeds, and hardware.

Please remember to conduct Check Readiness prior to your testing session(s). Check Readiness can be accessed using the following URL: http://admin-cla.cae.org. Click the Check Readiness link on the upper right hand corner of the screen.

The Secure Browser does not need to be re-downloaded if it is already available on your computer. It will auto-update if you attempt to launch an older copy.

We recommend downloading the most current browser(s) for both the Proctor and Student Interface.

Operating Systems BrowsersProctor Interfacehttp://admin-cla.cae.org

Windows 7 or higherMac OS X 10.9 or higher

Edge1

Chrome1

Internet Explorer 9.0Safari 7

Student Interfacehttp://cla.cae.org

Windows 7 or higherMac OS X 10.9 or higherMac iOS7 or higher (iPads)iPad 2 or higher

Internet Explorer 9.0Safari 7

OPERATING SYSTEM AND BROWSER REQUIREMENTS

RECOMMENDED CONNECTION SPEEDS AND HARDWAREInternet Connection While dial-up modems are supported, we recommend a broadband connection (DSL,

Cable, or T1) for best results.

Monitor Size We recommend a 17” monitor or larger.

Resolution Tests can be delivered at any resolution. Most are designed for resolutions of 800x600 or better. We recommend a 1024x768 resolution or higher.

Color Depth On Windows, the highest resolution is 32-bit color. Ninety-seven percent of users have 16-bit color or greater. Although 8-bit color resolution is supported, it may not present complex graphics perfectly. We recommend a 16-bit color depth or greater.

VISUALLY IMPAIRED STUDENT ACCOMMODATIONSStudents with visual impairments are able to use a magnification feature in the Secure Browser. The minimum system requirement for this function for PCs is Windows 7 operating system and Internet Explorer 10 (or higher). To zoom in, students should press “Ctrl+” and to zoom out, students should press “Ctrl-.” On Macs you may use “Command +” and “Command -.”

PLEASE CONTACT YOUR CLA+ REP IF YOU PLAN ON USING:• Thin clients• Virtual machines• XBoxes• Cloud environments

• Remote Desktops• Tablets/iPads with

keyboards• Chromebooks

1. Edge and Chrome support is limited to the current version plus one previous version due to forced automatic updates.

27


COMMON TESTING ISSUES AND TROUBLESHOOTING

Through feedback collected from institutions, we have compiled six common testing issues and solutions.

SCENARIO 1A student has clicked through the test, passed the yellow warning screen, and a pop-up reads:

Warning! Time remaining. This is the end of the task. You have not used all the time allotted for completing this task.Once you click ‘Next’ you will not be allowed to return to this task. If you would like to review your responses, pleaseuse the ‘Back’ button. Click the ‘Next’ button only if you are finished with the task.

In this scenario, the student will no longer be shown to be in the PT on the Proctor Session page.

Solution: If this occurs, the proctor should call the institution’s CLA+ Rep for additional instructions.

SCENARIO 2A student’s computer fails or a power outage occurs during testing, causing the student to be logged off of the Student Interface.

Solution: If this happens during testing, your institution will have one week for the student to log back in using the same email address and date of birth. The student’s work and location in the exam will be saved automatically (the system automatically saves student answers every 10 seconds). The remaining time will resume when the student logs back in.

SCENARIO 3A test may freeze when the Internet connection becomes slow, causing the connection to time out.

Solution: Run the system check found in the Check Readiness feature to see if current bandwidth is an issue. If bandwidth is not a problem, we advise you to clear the cache and resume.

In Internet Explorer this can be found under Tools and then Internet Options. When the window appears, click Delete under Browsing History. When prompted, delete all Temporary Internet Files, Cookies, and History. It is also recommended you restart your computer after clearing the cache.

In Safari this can be found under Preferences in the Security tab on the top browser menu bar. Click the Show Cookies button and then Remove All. It is also recommended that you restart your computer.

SCENARIO 4A student completes the exam and clicks exit, but is brought back to the login page instead of receiving messaging that they have completed the task and should close the browser.

Solution: Proctors should not worry if this occurs. This is often the result of local browser settings. Instruct the student to exit the Student Interface by closing the browser. By default, a student’s work is saved every 10 seconds and by clicking exit, the task is marked complete.

28


SCENARIO 5There is an emergency event where a student is unable to complete CLA+ testing (e.g. an unplanned fire drill, a student becomes suddenly ill etc.)

Solution: Proctors should instruct the students to shut down their computers by clicking the manual power button. Students will be able to complete their CLA+ exams where they stopped by logging back into another proctor session within seven days. Please contact your CLA+ Rep for further instructions. Do not instruct students to click through the tests, or they will not be able to reenter.

SCENARIO 6In the selected response section, a student accidentally hits the Review button on the lower left hand corner and isbrought to the end of the section.

Solution: On the Review Screen, the student may return to the section by clicking on the Review All or ReviewIncomplete button on the lower left hand corner. The student should not click Mark Complete, as it will complete andend the section.

COMMUNICATION ERRORSIf you are receiving communication errors, restart your local Internet connection. This doesn’t necessarily mean that the connection is “down,” but it usually means that the connection was running at a high level at that time, causing the connection to slow down to a point at which it timed out.

This can happen because your Internet connection is shared with other people around you, so the more people that are using the Internet, the slower your connection will be. When your connection slows down, it can cause your test link to disconnect.

You should be able to click OK to continue testing when those messages appear. If you continue to have issues, you should close out of the test altogether and restart your computer. When the computer powers up again, please log in to your test again and you will be able to pick up where you left off. If this continues to happen, please contact ITS Tech Support and your institution’s IT department.

29


The following are sample questions that your institution could use when creating Custom Questions*:

HOW OFTEN DID YOUR FACULTY AT THIS SCHOOL…• Give clear explanations?• Make good use of examples and illustrations to explain difficult points?• Effectively review and summarize the material?• Interpret abstract ideas and theories clearly?• Give assignments that helped in learning the course material?• Present material in a well organized fashion?• Come well prepared for class?• Use class time effectively?• Have a good command of what they were teaching?

RESPONSE OPTIONSResponse options should be separated by the vertical bar character (|). For instance, enter the responses to thequestion “How often did your faculty at this school give clear explanations?” as:

Very Often|Often|Sometimes|Rarely|Never

To create the vertical line, press shift followed by the backslash key.

KEY CHARACTERS TO AVOIDIn order to ensure that your data render correctly in the Student Data File, the following instructions should be adhered to when creating questions and responses in the Custom Questions. To create ratios, such as “1:2,” please enter the ratio in text: “one to two.” Should you wish to include fractions, please enter them in word form, such as “three-fourths.” We recommend that institutions refrain from using special characters (e.g., *, &, ^, %, #).

TAILORINGTo apply these questions to incoming freshmen, consider changing “How often did your faculty at this school…” to “How often do you expect faculty at this school to…” as a way to frame them as expectations.

Appendix A: Sample Questions

SAMPLE CUSTOM QUESTIONS

* The sample questions provided in this document constitute a modified version of the Teaching Clarity and Organization scale employed by the Wabash National Study of Liberal Arts Education. The scale has been shown to relate to retention and persistence. CLA+ thanks Charlie Blaich, Director of Inquiries, Center of Inquiry in the Liberal Arts at Wabash College, for guidance on modification and allowing for the use of this scale by CLA+ schools.

30


Appendix B: Glossary of Terms

CLA+ Rep: Your primary contact at CLA+. Your CLA+ Rep is available to answer questions and help you through all phases of the CLA+ testing process.

College and Work Readiness Assessment (CWRA+): An assessment program designed for high schools to measure institutional contributions to student learning. The assessment uses Performance Tasks to measure students’ mastery of higher-order skills, including: analysis and problem solving, writing effectiveness, and writing mechanics.

Collegiate Learning Assessment (CLA+): An assessment program designed to measure institutional contributions to student learning. CLA+ uses Performance Tasks and selected response questions to calculate the value a school adds to its students’ learning of critical-thinking skills, including: analytic reasoning and evaluation, writing effectiveness, writing mechanics, and problem solving. The value-added assessment of these skills is determined through the use of Performance Tasks and Selected-Response Questions. CLA+ is also an umbrella term used to refer to all of our assessment programs.

Critique an Argument (CA): A SRQ section in which students are presented with a brief argument about an authentic issue, and must use their critical-thinking skills to critique the given argument. Some of the questions may require students to: evaluate alternate conclusions; address additional information that could strengthen or weaken the argument; detect logical flaws and questionable assumptions in the argument; and evaluate the reliability of information, including recognizing potential biases or conflicts of interest. There are 5 questions in this section.

Critical Reading & Evaluation (CRE): A SRQ section that requires students to use information and arguments from (an) accompanying document(s) to: support or refute a position; identify connected and conflicting information; analyze logic; identify assumptions in arguments; make justifiable inferences; or evaluate the reliability of the information provided. There are 10 questions in this section.

Cross-Sectional: Testing approach that compares the CLA+ scores of two separate samples—entering students (freshmen) and exiting students (seniors)—at the same institution during the same academic year.

Entering Academic Ability (EAA): Entering Academic Ability is the estimated performance of a student on CLA+ prior to entering the institution as freshmen.

Freshmen: First-time, full-time entering students.

In-Depth Sampling/Oversampling: Occasionally, schools are interested in conducting in-depth analyses of particular subgroups of students, where they oversample students with the characteristic of interest in addition to, or instead of, their representative sample. Special studies such as this must be established with your CLA+ Rep prior to the opening of the testing window. For special studies with in-depth sampling, additional reporting fees may apply. Students identified as part of special studies will be excluded from institution-level analyses, as they are not representative of the school’s population, but will be included in the Student Data Files with all relevant statistics. If you are interested in learning more about in-depth sampling, please contact your CLA+ Rep for more information. Regardless of whether you have oversampled a specific population at your school, additional students tested beyond the contracted number may result in an “in-depth” sampling charge (see your MOA for details).

Performance Task (PT): A task that asks students to craft responses to several open-ended questions about a hypothetical but realistic scenario using a library of relevant documents.

Scientific & Quantitative Reasoning (SQR): A SRQ section that require students to use information and arguments provided in (an) accompanying document(s) to: make inferences and hypotheses based on given results; support or refute a position; identify connected and conflicting information; detect questionable assumptions (such as implications of causation based on correlation); evaluate the reliability of the information provided (such as the experimental design or data collection methodology); draw a conclusion or decide on a course of action to solve

31


a problem; evaluate alternate conclusions; or recognize that the text leaves some matters uncertain and propose additional research to address these matters. There are 10 questions in this section.

Secure Browser: CLA+ is administered through a secure browser site, which prevents students from accessing other Internet browsers or computer applications while testing.

Selected-Response Questions (SRQ): Selected-Response questions are designed to evaluate targeted skill sets. Students are presented with a set of questions as well as one or two documents to refer to when answering each question. The supporting documents include a range of information sources, such as letters, memos, photographs, charts, or newspaper articles. There are 25 questions and students are given 30 minutes to complete the section.

Seniors: Exiting students on schedule to graduate in the testing calendar year.

Test Delivery: A component on the Proctor Interface where proctors administer the test by creating a Test Session (see Day of Testing Guide) and approving students into the Student Interface during a testing session. In the Test Delivery component, Test Administration is used to select the appropriate test version when creating a testing session.

Test Administrations: A component on the Proctor Interface used for the Registrar Data Tool and to create and monitor the optional Custom Questions. Test Administration also refers to the type of exams that are selected when a proctor creates a test.

Testing Window: The designated period each semester when an institution is able to administer CLA+. No student may test before the testing window opens, or after the testing window closes.

administrator manual - cae.org · this commitment helps cae and its member institutions maximize...

Documents