Digital Proctor A Detailed Look Behind the Scenes

2011

Digital Proctor Inc. 1/1/2011

2 | P a g e

Welcome.

3 | P a g e

Client Objectives

Detailed market and client analysis has yielded the following objectives as indicated by institutions,

without limitation, when implementing a solution for student identity verification purposes:

• Institutions would want a solution for authenticating the identity of students who register for

and participate in online courses and programs

• Institutions would want a solution to enhance their current single sign on architecture through

true multi-factor authentication

• Institutions would want a solution that is committed to protecting the privacy of students and

the security of their personal data.

• Institutions would want a solution that students would not perceive as intrusive or privacy

insensitive

• Institutions would want a solution that is able to serve a global student base

• Institutions would want a solution that is cost-effective and would allow for unlimited use with

respect to each student

• Institutions would want a solution that requires minimal if any time for students to set up or

upgrade

• Institutions would want a solution that does not exceed the system requirements of their LMS

• Institutions would want a solution that allows for potential authentications at any time of the

day or night

• Institutions would want a solution that is robust to accommodates students with slow speed

internet connections

• Institutions would want a solution that accommodates students using multiple computers from

different locations

• Institutions would want a solution that features a robust and intuitive reporting infrastructure

with varying levels of privileged access

• Institutions would want a solution that integrates with the learning management system and

potentially other University systems

• Institutions would want a solution that minimizes the need for technical support.

• Institutions would want a solution that would accommodate a minimum need for training or

orientation.

• Institutions would want a solution that maintains the highest security standards for

administrative, technical, and physical safeguards to protect the security, confidentiality, and

integrity of the University’s confidential information

4 | P a g e

Key Challenges

Institutions are currently looking out for a solution, which should meet the following challenges faced by

them:

� Stronger mechanisms for student identity verification

� Systematic and centralized approach to academic integrity

� Reduction of exposure to risk

Digital Proctor has appraised itself of the above challenges and is offering a solution delineated in the

following pages. The solution addresses the above challenges and brings about enhanced institutional

experience with regards to student identity verification.

5 | P a g e

Digital Proctor Solution

Overview

Digital Proctor has developed a powerful set of technologies designed to prevent and detect cheating in

online education.

Digital Proctor provides an unprecedented view into the online learning environment. We implement a

transparent authentication solution that analyzes student behavior invisibly in the background. As a

byproduct of students’ normal interactions with their assignments, we are able to create an individual

identity profile for each student using multiple data points, including our groundbreaking typing

recognition system. Our software verifies student identities, reports atypical cut/copy/paste usage,

detects collusion, and gives faculty members a set of intelligent questions that they can ask students to

confirm or discount any suspicious activity.

We are able to identify students by the unique way that they type on their keyboard while interacting

within the learning management system. Throughout all course activity for a particular student, we

check his/her typing patterns for consistency and can identify if a particular assignment has been

outsourced. That is, if a student has enlisted a friend or paid someone to complete an assignment for

them. Our reporting interface is further capable of detecting if an entire course has been outsourced.

With our software, the identification (authentication) of students is intrinsically bound with the

completion of assignments. The students interact normally within the learning management system,

and as a byproduct of this interaction, we can uniquely identify them.

If the typing pattern is inconsistent and it appears that the student has in fact outsourced an

assignment, the adminstrators, faculty, and staff members have access to a comprehensive reporting

interface which incorporates a robust set of data points around the suspicious assignment or course.

From this data, the faculty member can ask intelligent questions and investigate the suspicious activity.

This affords the faculty member a light touch, non accusatory way to open a dialogue with the student

and confirm or deny the suspicious activity.

In addition to preventing outsourcing through student identity verification, our product also provides

faculty with insight into unusual cut/copy/pasting activity that a student is executing within the learning

management system. Our reporting interface also highlights cases of blatant collusion, that is, when

students are working on particular assignments together (when they should not be).

Our most recent feature is a commenting system that allows administrators, faculty, and staff to make

notes on particular students within the reporting interface.

Digital Proctor takes the privacy of our clients and their students extremely seriously. It is important to

understand that we do not collect what a student types, and are not classified as keylogging software.

In fact, the order of the keys that a student types are scrambled and unable to be reconstructed before

they are sent to our server for analysis. Further, all data sent to and from our servers is protected by

256-bit encryption and our reporting interface resides on an HTTPS server.

6 | P a g e

The primary objective of our product is to stop the most blatant forms of cheating (ie outsourcing

assignments, pasting answers from the web, working together on assignments) with the least amount of

invasion. Students already give off unique identifiers as they complete coursework. Digital Proctor

analyzes these identifiers, checks them for consistency, and packages them in an intelligible and, if

needed, an actionable format for the faculty members and administration.

7 | P a g e

Product Description

Digital Proctor provides a complete solution for student identity verification, that is, credible verification

that a student who registers for a course is the same student who completes the course and receives

credit. The design of the student authentication solution affords an additional layer of functionality.

That is, an electronic proctoring tool, specifically capable of capturing instances of collusion and atypical

cut/copy/paste activity.

Because the solution is fully hosted, students and faculty are not required to download or install

anything and the solution transfers from computer to computer, accounting for the mobility of the

modern student.

The solution begins when a student logs in, running transparently and non-confrontationally in the

background. Throughout a student’s activity, the solution collects several data points including: typing

pattern samples, location information, browser characteristics, software environment, date and time,

and then maps all of this information to a particular assignment or activity for the duration of the

student’s course(s).

This data is then packaged and sent to our server for analysis. If the typing pattern analysis for a student

yields consistent results, the student is successfully authenticated and receives a passing check mark. If

the typing pattern for a student is inconsistent, a probability calculation considers the likelihood that the

student outsourced an assignment or course, and then a student receives either a passing check mark or

is flagged with an “x” indicating that suspicious activity has been detected.

This data culminates in an intuitive reporting user interface that is continuously accessible to

administrators and faculty. If a student is flagged for suspicious activity, the administrator or faculty

member can access the assignment or course history of the student, find out what specifically we

flagged as suspicious, and then investigate the situation appropriately. Equipped with multiple data

points and information around the suspicious activity, the faculty member or administrator can make an

intelligent inquiry to the student and properly investigate the assignment(s) or course(s). A best

practices guideline outlines the recommended course of action, but of course, the institution will

ultimately decide this.

8 | P a g e

Electronic Proctoring Functionality

In addition to the student identity verification, some institutions have expressed an interest in additional

electronic proctoring tools to further strengthen academic integrity. The two primary tools packaged into our

product in addition to student identity verification are the cut/copy/paste detection tool and the collusion

discovery tool.

The electronic proctoring solutions are capable of detecting instances of suspected collusion and

atypical cut/copy/paste activity (CCPA).

Detecting suspected collusion is accomplished through a creative combination and filtering process of

the existing data points that we collect for our student authentication solution.

CCPA is an added functionality that allows us to capture the amount of this activity for any given

assignment, and then determines what exactly was pasted so that the faculty member or administrator

can determine whether or not the paste was a legitimate action.

9 | P a g e

Software Updates

Updates occur seamlessly for the end-user, because all updates occur on the server side. End-users are

not required to implement an update.

A later version of our software would deploy seamlessly for the end-user with no implications.

10 | P a g e

Administrator, Faculty, and Staff Training

Students do not need to be trained how to use our solution, as we are only interested in what students

are already doing naturally. Administrators, faculty, and staff members receive training sessions, made

available at a frequency determined by the institution, on how to navigate and interpret the data in the

reporting user interface. Depending on the institutions preference, these training sessions can be given

on-site or remotely via webinars. Typically, each instituion receieves two access periods to review a

recorded webinar, which is followed up by a live webinar in which a representative from Digital Proctor

answers any remaining questions.

11 | P a g e

True Multi-factor Authentication

Digital Proctor provides true multi factor authentication in strict accordance with the Federal Financial

Institutions Examination Council’s (FFIEC) conclusion that, “By definition true multifactor authentication

requires the use of solutions from two or more of the three categories of factors. Using multiple

solutions from the same category at different points in the process may be part of a layered security or

other compensating control approach, but it would not constitute multifactor authentication." The

categories of factors including:

• Something the user knows (e.g., password, PIN);

• Something the user has (e.g., ATM card, smart card); and

• Something the user is (e.g., biometric characteristic, such as a unique typing pattern).

Digital Proctor leverages each category to provide true multi factor authentication. Specifically:

Something the user knows

Utilizes the secure login/password combinations currently issued by the client through its SSO

architecture.

Something the user has – Including:

A particular browser environment identifiable by cookie files, height and width characteristics, and other

metadata such as the particular version of the browser.

A particular location where assignments are completed.

A particular software environment.

A particular schedule when assignments are completed identifiable by date and time of activity

The client can opt in to all or none of these particular data collection points.

Something the user is

Analyzes students’ unique typing patterns, an established behavioral biometric, as they interact within

the learning environment. Checks each student’s typing pattern for consistency, ensuring all

assignments are completed by the same student.

12 | P a g e

Data Collection and Student Privacy

All data to and from our server is protected by 256 bit encryption.

All analyses of students is conducted blindly, without using their names. For our typing pattern analysis,

we do not collect the order of keystrokes that students enter into the learning management system. In

order to obtain a biometric watermark of students' typing patterns, we only need timing measurements.

This allows us to scramble the order of keys into an unreconstructable order before they are even sent

to our server for analysis. In this manner, the solution is legally not classified as a keylogger according to

DLA Piper’s professional opinion.

Additional data points that are optionally collected include: IP address, browser characteristics, software

environment, time of activity, and data that is cut, copied, and/or pasted into the learning management

system.

The institution owns the data collected on students, but there are restrictions due to FERPA. For more

information about FERPA, please see section 6.3 in the Digital Proctor Software License and Hosting

Agreement.

13 | P a g e

Multiple Computers

Because the solution is fully hosted, it is transferable from computer to computer without any need for

a download or installation.

A student using a different keyboard will exhibit a slightly different typing pattern from time to time;

however, we automatically detect if a student is using a different keyboard and take this into account to

limit false positives resulting from different keyboard use. But even more importantly, while a student

may exhibit a slightly different typing pattern from one keyboard to the next, the difference between

these samples is still far less than the typing pattern of another student. Therefore, we can verify

student identity across multiple computers.

14 | P a g e

Reporting

Administrators, faculty, and staff can access reports at any time through our reporting user interface.

Access privileges are currently designed to give faculty members access to their specific courses only

and administrators access to all courses. Currently, students do not have access to the reporting user

interface.

Primary indicators of suspicious behavior:

The detection of more than one distinct typing pattern under a single student account

The same typing distinct typing pattern across more than one account

Secondary indicators of suspicious behavior:

Different location detected for “higher stakes” assignments

Different time of activity for “higher stakes” assignments

Different browser characteristics for “higher stakes” assignments

Different software environment for “higher stakes” assignments

Reporting capabilities include:

• Failure to match one student authenticating at time of registration with attempt during the

semester

• Failure to match one student authenticating at multiple points during semester across

multiple courses

• Failure to match one student authenticating in two different semesters

• Matches between two or more “different” students in a given semester or across semesters

• Failure to match one student authenticating at multiple points in a given semester in a

single course as well as across multiple courses

• Detecting one student posing as one or more other students (exhibiting the same profile for

authentication)

• High level statistical reporting for administrators

• Identify only the top x% of suspicious students

15 | P a g e

Approach to Providing the Scope of Services

Implementation Methodology

The solution has three main components:

The first component is installed on or alongside the learning management system. This collects typing

data and other unique characteristics of students' activities while they are registering and completing

assignments at an institution. It is important to emphasize that the privacy of students is assured: 1)

Typing data is randomized before being sent to us so we cannot see what a student types, only how they

type it; 2) all data is sent over secure SSL (which is the same security used by banking websites); and 3)

typing data is signed using a 256-bit encryption scheme (this is the highest level of security of the

options the US government recommends using) to assure it genuinely comes from the right student.

Another key aspect of this component is that it is very light weight. It runs seamlessly in the background

on a user's computer. Not one student at the schools we have serviced has complained about this

software. Also, the total amount of data sent from a user's computer is only a few kilobytes per

minute. Historically, the installation process for this component has taken about fifteen minutes of

system administrator’s time, plus another thirty minutes of us providing background information and

testing.

The second component is the server on our end that receives the data sent from students. This server is

amazingly stable. Last semester (Spring 2011), there were no crashes or unscheduled downtime. There

was only one fifteen minute period of scheduled maintenance, and that occurred at 1am on a weekend,

when activity was at its lowest. It is important to note that, in the extremely unlikely event that our

collection servers go down, students would still be able to complete assignments as normal. There is no

negative impact except the assignments students complete during that time would not be verified.

The third component is our web user interface. This analyzes, organizes, and displays a wealth of

information about student activity. Faculty members are able to see activity from their courses, and

administrators can see all the activity at their institution. There are pages to narrow in by a course, by

an assignment, by a student, and more. Each page is intuitive but also contains embedded help

dialogues.

16 | P a g e

False Positives

Solutions that involve biometrics are susceptible to type I (false positives) and type II errors (false

negatives).

First, we want to clarify terminology to ensure an accurate response. Digital proctor adopts the

following standard definitions of type I and type II errors and their implications:

In biometrics, the null hypothesis is that the input does identify someone in the searched list of people.

For this solution specifically, the null hypothesis is that the input (authentication) of a student matches

the previous input of the same student. Again, the null hypothesis is that the student authenticating is

the same student who registered for the course.

Type I error (false positive) – The error of rejecting the null hypothesis when it should not have been. In

the context of student authentication, a type I error occurs when the biometric system fails to

authenticate the student when it should have authenticated the student. That is, a false positive would

indicate the honest student is not completing their own work, when in fact they were completing their

own work.

Type II error (false negative) – The error of failing to reject the null hypothesis when it is in fact not true.

In the context of student authentication, a type II error occurs when the biometric system authenticates

the student when it should have failed to authenticate a student. That is, a false negative would

indicate that a dishonest student, who is outsourcing their assignment(s), was completing their own

work, when in fact they were not completing their own work.

Inherent to biometric systems is the correlation between type I and type II errors. Our statistical system

is designed to keep the number of false positives to an absolute minimum, even at the cost of allowing a

small number of false negatives dishonest students to go unnoticed. Digital Proctor’s philosophy is that

it is far worse to falsely accuse an honest student than to let a dishonest student go through undetected.

In combination with the biometric component of our solution, we have implemented a human

intelligence based component that serves to further reduce the number of false positives that our

system might reveal. This component looks at the “stakes” of an assignment after we have detected a

different typing pattern and calculates the likelihood, using a number of different methods, that the

student would have outsourced that particular assignment.

Further, the best practices guidelines that we encourage institutions to follow encourages investigating

instances of suspicious activity using a non accusatory, data oriented line of questioning as opposed to a

quick pass or fail judgment. A dishonest student who is confronted over suspicious activity using our

recommended method will most likely cease all future suspicious activity and/or admit to some form of

deviance. Knowing that someone is looking over their shoulder, will be a compelling force to keep

students honest. If an honest student is confronted, they should easily be able to account for any

suspicious activity and deny any claims to the contrary without hesitation.

17 | P a g e

Special/Unique Qualifications

Digital Proctor embodies the ideal synergy of technical talent and client relations.

The technical team is led by Andrew Mills, who in addition to his striking technical background and

accomplishments, is a clear communicator and works excellent in team environments.

Client relationships are managed by Shaun Sims, who works ceaselessly to make sure client expectations

are promptly met and exceeded. Shaun leverages his carefully cultivated network of leaders in the

space to stay ahead of the current issues facing higher education and sets internal policies that keep

Digital Proctor in line with industry best practices.

The size and organizational structure of Digital Proctor allows us to respond quickly to customer

requests without delay. Digital Proctor has access to one of the country’s most accomplished talent

pools in Austin, Texas, including relationships with premium employers and sources of capital to help us

grow securely and source customer requests as needed.

Digital Proctor has been recognized for the following awards:

DFJ-Cisco Global Business Plan Competition Finalist

1st Place Milken-Penn GSE Competition '10

1st Place UT Idea to Product '10

2nd Place Texas Moot Corp '10

McGinnis Venture Competition Semi-Finalist '10

Selection DLA Venture Pipeline

Digital Proctor is also represented by one of the world’s largest international and most respected law

firms, DLA Piper.