wow! i did not know i could do that; tricks for configuring smartermeasure & analyzing data

WOW, I Did Not Know I Could Do That!Tricks for configuring and

analyzing SmarterMeasure Data

Cynthia Pascal, Northern Virginia Community CollegeLou Ann Martin, Tri-County Technical CollegeCourtney Tillett, Tri-County Technical CollegeDr. Mac Adkins, President, SmarterServices

All attendee lines are muted. Please submit questions via chat. Q & A at the end. The session is being recorded and an archive

will be available.

Housekeeping

Introduction – Making Sense of your Data Northern Virginia Community College Tri-County Technical College Five C’s to Maximize SmarterMeasure

Agenda

Level One – Students learn about themselves• Aha moment• The tiger looking in the mirror

Level Two – Schools learn how to best support students • Integration into orientation courses, first year experience

programs• Recommendation of appropriate remedial / support resourcesLevel Three – Leaders learn how to maximize data• Computing retention gains, comparing populations• Fulfilling accreditation requirements, justifying costs

Levels of Benefit

Level One – Students learn about themselves• Aha moment• The tiger looking in the mirror

Level Two – Schools learn how to best support students • Integration into orientation courses, first year experience

programs• Recommendation of appropriate remedial / support resourcesLevel Three – Leaders learn how to maximize data• Computing retention gains, comparing populations• Fulfilling accreditation requirements, justifying costs

Levels of Benefit

Which level would you say best describes the highest level of how your school is using SmarterMeasue?

Level One – Using it as a student serviceLevel Two – Integrated into your software and processesLevel Three – Conducting statistical analysis of your data

Poll

Progression of SmarterMeasure Data Utilization

Predictive

Correlation

Comparison

Descriptive

Student Service Metrics

Approaches to Research Projects

Internally

Conducted

Company

Assisted

Professionall

y Assist

ed

Cynthia Pascal Coordinator of Academic Advising & Retention

Northern Virginia Community College

Northern Virginia Community College

Northern Virginia Community College’s Extended Learning Institute

Current Use of

SmarterMeasure

• Website• Orientation• Welcome

Messages• Student Planners• Embedded in Bb• Freshman

Seminar/ College Skills Assignment

• Advising partner colleges of students’ “goodness of fit”

Northern Virginia Community College

Tool to assess incoming students

Northern Virginia Community College’s Extended Learning

Institute

Northern Virginia Community College

Northern Virginia Community College’s Extended Learning Institute

Data used to have conversations with students

who are on academic probation or taking a class

for the third time.

Data used to have conversations with

students who are on academic probation or

repeating a class for the third time.

Northern Virginia Community College

Northern Virginia Community College’s Extended Learning Institute

Lou Ann Martin – Title III Activity Director for the Academic Support Network

Courtney Tillett – Information Tech-instruction-applications Analyst

Tri-County Technical College

Current use Completed voluntarily by incoming students

Completion encouraged through acceptance letter Results captured in Banner Results reviewed briefly with students by advisors

Students enrolled in Freshman Seminar/College Skills classes use results as springboard for discussions and journal entries

Goal Would like to be able to identify students at-risk due to

non-academic issues Initial research found no specific correlations Tried different approach after consulting with

SmarterMeasure Preliminary results are promising and suggest the need

to tweak our readiness ranges

SmarterMeasure Results at TCTC

Acceptance Letter

Results in Banner

Readiness Ranges Analysis

  Default Cut

Points

 N

 p-value

 F

Adjusted Cut Points

 p-

value

 F

MeanGPA

PersonalAttribute

s 

70 / 85 1,457 0.2723 1.30 80 /90 0.2327

1.46 Fail: 2.15Question: 2.30Pass: 2.34

LifeFactors

70 / 85 1,482 0.2453 1.41 80 / 90 0.1217

2.11 Fail: 2.21Question: 2.24Pass: 2.56

Reading Recall

30 / 65 1,380 0.1237 2.09 70 / 90 0.0002

8.52 Fail: 2.21 Question: 2.24Pass: 2.56

Technical

Competency

80 / 90 1,362 0.0011 6.82 90 / 100 0.0367

3.31 Fail: 1.84Question: 2.26Pass: 2.34

Technical

Knowledge 

50 / 75 1,352 0.0039 5.56 70 / 80 0.0057

5.18 Fail: 2.03Question: 2.26Pass: 2.45

TypingRate

21 / 31 1,346 0.0303 3.51 25 / 40 0.1226

2.10 Fail: 2.17Question: 2.36Pass: 2.36

Notes: Based on “Making SmarterMeasure a Strong Predictor of

Academic Success” Used ANOVA to determine the strength of the relationships

between the readiness ranges and first-semester GPA 6,931 FTPS students from the fall and spring semesters of the last

three academic years (2011-2012 – 2013-2014) 1,593 matched to SmarterMeasure results Any test taken after the first day of class for each semester was

excluded from the analyses

Readiness Ranges Analysis

How can the ranges for Personal Attributes and Life Factors be adjusted? Would any of the individual subscales in either

section be more predictive? What is the practical significance of the

findings?

Questions Raised

Configure the assessment engine Control for self-selection bias Consider adding your own questions Classify using your own demographic factors Create a sanitized data set

Five C’s to Maximize SmarterMeasure

Your Account Manager is eager to train you on any of these configuration options.

Configure the Assessment Engine

Julie Owen Tara McLaughlin

Login Tutorial Video

Add Google Analytics

Integrate into LMS or SIS

Require student email

Turn on/off demographics

School Level

More demographics

Adjust time zone

Custom Remedial Resources

Custom Remedial Text

Require student email

Renewal alert

School Level

Integrate into third party software such as an LMS, SIS or Advising suite.

Single Sign On

Automated data transfer

Gradebook integration

School Level

Re-order sections

Skip sections

Add Math and Writing

Adjust Readiness Ranges

Select Versions

Group Level

Change scoring type – 4 point or percentage

Assessment Mode – Sequential or Disaggregate

Show National Averages

Results options – Report, Custom Text or URL

Group Level

Custom Interface

Text

Select each of the ways that your school has configured SmarterMeasure.Re-order or delete sections LMS integrationAdding custom questionsCustom introductory or score report textAdjusting Readiness Ranges

Poll

Control for Self-Selection Bias

Consider Adding Your Own Questions

Classify Using Your Own Demographic Factors

Create a Sanitized Data Set

Based on the webinar today how likely is your school to enhance the way it uses SmarterMeasure?Very likely – I learned some actionable ideas.Likely – This has me thinking.Not likely – This is interesting but there is no need to change what we are doing.

Poll

Let Us Know If We Can Help

Julie OwenJulie@SmarterServices.com

Tara McLaughlinTara@SmarterServices.com

Please Give Us Your Feedback

You will receive a post-webinar email with a link to the survey.