gist qualtrics : using the unt online survey system

GIST QUALTRICS: USING THE UNT ONLINE SURVEY SYSTEMDr. Russell Ruffu, Office of Institutional Research and Effectiveness

Gist Qualtrics: Using the UNT online survey system

TODAY’S AGENDA

Electronic surveys at UNT Qualtrics at UNT: Brief History Resources for Problem-Solving Ethical Use of Data Qualtrics Demo


ELECTRONIC SURVEYS AT UNT 2000: IRE purchased Inquisite (not open across

campus) – hard to learn and very complex 2005: Division of Student Affairs purchased Survey

Monkey (monthly subscription) 2007: DSA moved to annual Survey Monkey

subscription 2009: DSA purchased second, private Survey Monkey

account to lock down sensitive data (limited access) 2010: Colleges of Education and Business purchased

Qualtrics (limited license) 2010/2011: IRE approached campus users to purchase

university-wide license for Qualtrics April 2011: Qualtrics added to MyUNT


STEPS FOR PROBLEM-SOLVING Start off at Qualtrics University

www.qualtrics.com/universitywww.qualtrics.com/university/learn-qualtrics/

Access the Qualtrics self-help module at http://institutionalresearch.unt.edu/sites/default/files/Qualtrics%20101Training%20Series%2010_1_12.pdf

Contact Institutional Research and Effectiveness Qualtrics Coordinator

Dr. Russell [email protected] 7894


HELP FOR SPECIFIC TYPES OF QUESTIONS/ITEMS: GO TO

HTTP://WWW.QUALTRICS.COM/UNIVERSITY/RESEARCHSUITE/BASIC-BUILDING/BASIC-SURVEY-OPTIONS-SETTINGS/SURVEY-EXPERIENCE


ETHICAL USE OF DATA

All data collected should be treated as security-sensitive You should only access data for which you have a

legitimate work purpose Use care when sharing data with others, whether

formally or casually Be aware of collaborating with external parties, you may

unwittingly be compromising FERPA National Institutes of Health training

(phrp.nihtraining.com/users/login.php) – useful even if not IRB study

Avoid downloading individual response data to a laptop or portable file/device


HOW TO USE THIS GUIDE:

The following slides are not designed to show you every detail of Qualtrics – just the most frequently used.

This presentation has been organized to mirror the actual process of creating a survey, and distributing the instrument

Where appropriate, screen shots and notes in the margin are designed to help first-time users.

QUALTRICS TRAINING SERIESPART I

Survey Creation


OPENING PAGE: USE MYUNTFACULTY QUALTRICS

Page shows all of your surveys as well as surveys in collaboration (separated by the gray divider bars).

Click on Create Survey tab or the Create Survey icon


CREATE SURVEY: QUICK SURVEY BUILDER

Create from Copy provides a feature that copies a survey from your Qualtrics file.

The Survey Library links to templates that you have created or borrowed that exist in your Qualtrics account, or are available from the Qualtrics library.

Click Quick Survey Builder.


ENTER YOUR SURVEY’S NAME AND IDENTIFY THE FOLDER IT WILL BE KEPT. THEN CLICK “CREATE SURVEY.”


BLOCK AND BLOCK OPTIONS

A block can contain specific categories of questions which can be manipulated for navigation and analyses purposes. For example, blocks might be labeled Track 1 and Track 2 for respondents who ‘branch’ off within the survey; or Blocks might be labeled as Demographics, Activities, and Perceptions to segregate the type of information being sought.

(If you simply want one question to appear at a time, just add a Page Break.)


CREATE A NEW ITEM: CLICK CREATE NEW ITEM. WRITE THE QUESTION TEXT. BOTTOM RIGHT CORNER, CLICK “ADD BLOCK.” WRITE SECOND QUESTION. WHAT IF YOU DECIDE YOU WANT BOTH QUESTIONS DISPLAYED IN SAME BLOCK? THEN REMOVE THE LAST BLOCK. BUT…

Block deletion is a little funky

BEST:Plan your blocks/question groups before you start developing the survey.

FUNKY:Check box for each question below the block marker that you want to removemove all questions up by clicking up-arrow at far left of questions. Remove Block option now appears in bottom right corner of last block.

FUNKIER:Block OptionsDelete Block, questions go to trash at bottom of surveyclick on question in trash, then click RestoreMove question to desired location.


MOST COMMONLY USED QUESTION TYPESo Descriptive Text – allows you to communicate with

respondent (no active data collection)o Graphic – allows you to upload your logo or other imageo Multiple Choice – allows respondent to pick from a list

(either one item or multiple items)o Matrix Table – Series of statements being measured on a

common scale (e.g. strongly disagree, disagree, neutral, agree, strongly agree)

o Text Entry – allows for write in (qualitative) responses in either single lines, short essay or long essay formats

o Slider – Graphically interesting scale itemo Rank Order – Users can re-arrange answers in order of

importanceo Side by Side – Lets users choose answers in a forced choice

model


ITEM TYPES

Let’s start with Text/Graphic items


TEXT/GRAPHIC ITEM: A BIT ABOUT BUTTONS & GRAPHICS

FYI, code is available from the Qualtric site to serve many functions. This is a sample.

Button functions, like Next and Back, are automatically embedded. The Back button is inserted/selected by going to the Survey Options icon at the top of the page.


SURVEY OPTIONS

Items selected are inserted automatically.


Labels can be added to the Next and Back buttons. Go to Look and Feel, locate Next Button Text and type in your label.


DISPLAY LOGIC: IF ANSWER ITEM SELECTED, THEN DISPLAY ANOTHER QUESTION

If Question Q5 “In what type of h… do you reside”tent is Selected

(then display this question)

Force Response is activated


SKIP LOGIC: IF CHOICE SELECTED, THEN SKIP TO QUESTION X OR END OF SURVEY

Skip Logic must be applied for each answer selection. That is, there is no condition for applying “all” to all of the answer choices.


BRANCHING: A QUICK LOOK SO YOU KNOW IT EXISTS

Branching will send participants who choose certain specified answers down a different road to answer a set of questions that others will not see.

Choose Survey Flow from the tool bar, then click Add Below from the options provided on the right in the blocks. Choose Branch from the yellow menu. Define the conditions. Notice that you can select And…And, or you can select Or, Or, or a combination of these.

Drag the defined conditions to “Add a New Element Here.

Save Flow


ELEMENTS IN BRANCHING OPERATION

Block= new section of questions

Branch=new road based on previous answer(s)

Embedded Data=information about respondents long before you survey them

Randomizer=Changes order of response choices

Web Service=bring additional information from a website into your survey

Authenticator=The Authenticator makes it possible to have respondents “log in” to a survey, along with other features to authenticate the respondent.

Conjoint=A conjoint study usually involves showing respondents a set of features and asking them to indicate how much they like or prefer the different attributes of that feature.

End of Survey=takes participant to end of survey

QUALTRICS TRAINING SERIESPART II

Survey Distribution


DISTRIBUTE SURVEY SUB-TABS

Survey Link: According to Qualtrics University,

“It’s the easiest way to distribute a survey: a simple, anonymous link that can be taken by anyone. You can:

•Send it out with your personal email.•Post it on a web page.•Use social media to distribute it.”

There are a couple of ways to distribute your survey. The first is called Survey Link. The other one we will focus on is the use of Panels, which seems to be the most popular here at UNT.

FIRST – you have to activate the survey using the link above!


http://www.qualtrics.com/university/researchsuite/distributing/basic-distribution/anonymous-survey-link


The Survey Link

Highlight and Copy it.

Go to the Email Survey sub-tab to an email.

Paste the link here.


MAILING LISTS/PANELS


UPLOAD A MAILING LIST FROM A CSV EXCEL SHEET

There is a way to upload large (and small) sets of email recipients. It does require, however, that the email list be in a csv Excel format that includes the first 3 column variables EXACTLY as follows: FirstName LastName PrimaryEmail (the columns may be in any order) From there, go to PanelsCreate New Panel click “Create”Import From a Fileselect fileimport For automatic reminders for those who have not completed the survey, go to DistributeEmail Historyon tool bar, select ActionsSend Reminder or Thank You (thank you message can be customized after this item is selected).

The FirstName LastName are NOT mandatory, but are commonly included in spreadsheets with email addresses. They are not necessary for uploading survey mailing addresses.

1.

2.

3.

4.


IMPORT FROM A FILE: CHOOSE THIS ONE


ONLY EXCEL CSV CAN BE IMPORTED


-OR-

1.

2.

3.

4.


EMAIL SURVEY

You can type in, copy & paste, select everyone or individuals from a previous panel.


CREATE CONDITIONAL SAMPLE FROM EXISTING PANEL

Set the Panel conditions


-OR-

Enter by hand.

Note expansion of panel to include embedded data.

You can use this to test the integrity of emailing your survey. Be aware that...

Using this email choice produces…


…NOT ALL FEATURES ARE ACTIVATED IN THE SEND TEST EMAIL

Not so good (using “Send Test Email”)

Much better (using “Schedule Mailing”)


AUTHENTICATION: AVOID BALLOT BOX STUFFING; ASSURE CORRECT AUDIENCE ONLY IS TAKING YOUR SURVEY. AUTHENTICATION USUALLY OCCURS BEFORE A PARTICIPANT IS ALLOWED TO TAKE THE SURVEY, AND MAY RELY ON EMBEDDED DATA TYPICALLY INCLUDED IN A PANEL. THE PANEL IS OFTEN ASSOCIATED WITH A CSV FILE THAT INCLUDES A PASSWORD, EUID, OR OTHER UNIQUE EMBEDDED DATA. IT CAN BE THE SAME PANEL USED FOR DISTRIBUTION.

Start with the Edit Survey tab Survey Flow click Add a New Element Here select Authenticator

click Move and drag Authenticate Using Panel to the top of the Survey Flow.

Place blocks and other element beneath the Authentication box.

Using the drop-down menus, select the fields used for authentication process.

Optional labels can be typed in if you want the field names to be different from those used in the Panel.

The minus and plus icons are used to add or remove fields from the authenticator.

Options bottom right corner = set parameters and text messages for authentication.

Click OK and save changes.

Go to http://www.qualtrics.com/university/researchsuie/advanced-building/survey-flow select Authenticator for a 2.5 minute video describing the authentication procedures.

QUALTRICS TRAINING SERIESPART III

Downloading and Analyzing Results


OVERALL TIPS: ANALYZING DATA In general, look for trends in the responses Evaluate blanks as a possible clue that the respondent

didn’t understand the question Be aware of “Christmas Treeing” where respondents

answer all the same way throughout every item – consider deleting

Look at your response rates and compute a confidence interval – go to: http://www.surveysystem.com/sscalc.htm for a great resource

Share data internally and externally to drive changes in practice

Look at your demographics, do they match your overall population?

Are you missing certain members of your community? How does this non-response affect your findings?

When in doubt, contact IR&E for a consultation on your data and findings.


VIEW RESULTS

Click here first!


RECORDED RESPONSES: INDIVIDUAL


DOWNLOAD DATA: TO CSV, SPSS (COMMONLY USED)


CSV OUTPUT

Answers as Values

Answers as Labels


SPSS DATA DOWNLOAD


THANK YOU FOR YOUR ATTENDANCE TODAY.

THIS POWERPOINT IS AVAILABLE ON THE IR&E WEB PAGE AT: HTTP://INSTITUTIONALRESEARCH.UNT.EDU/INSTITUTIONAL-

ASSESSMENT

GIST QUALTRICS: SURVEY BUILDER TRAINING MODULE


CONTACT INFORMATION:

Dr. Russell RuffuOffice of Institutional Research and EffectivenessHurley 121E

Phone:940.369.7894

Email:[email protected]

Web:http://institutionalresearch.unt.edu/institutional-assessment

gist qualtrics : using the unt online survey system

Documents

survey icongist qualtrics

survey tab

qualtrics university

qualtrics library

qualtrics account

unt online survey systemhelp

unt online survey systemsteps

unt online survey systemhow