using qualtrics to create automated online trainings
TRANSCRIPT
USING QUALTRICS TO CREATE AUTOMATED ONLINE TRAININGS
Digital Poster Session
Big XII Teaching and Learning Conference (3rd Annual) Theme: Transformational Teaching and Learning
Kansas State University
June 2 – 3, 2016
OVERVIEW
When thinking about “transformational teaching and learning,” training would not be the first thing to come to mind.
The Qualtrics® research suite offers a number of design tools and features that enable the building of automated online trainings. There are the baseline features such as the ability to integrate multimedia, apply various question designs, enable accessibility features (like alt-texting), deliver a mobile experience, reach learners across distances, and provide basic security and data integrity features.
2
OVERVIEW (CONT.)
Other features actually make this tool phenomenally powerful. One is the ability to richly customize learning sequences—by learner profile, by performance (behavior), by selection, or a mix of factors. There is a feature that enables the scoring of learner responses and the ability to set a threshold for passing. This tool has a rich data analytics capability (including a light item analysis), including online analytics and even cross-tabulation analysis. A Qualtrics® API enables the recording of online assessment scores and learner behaviors, in an automated way to faculty / staff / student information systems.
Trainings are critical for effective workplace functioning and professional development. The same features in Qualtrics® that enable the effective building of automated trainings also enable the effective building of pre-learning modules or sequences for learners who need to refresh their skills for a new course. This digital slideshow introduces the use of Qualtrics® as a customizable training and pre-learning module tool.
3
4
AUTOMATED…ONLINE…TRAININGS
Automated: learner and the machine, non-human-led, computer-captured details of the learning, computerized record-keeping
Online: deliverable to any location on Earth (and in space) with Web connectivity
Training: workplace-based teaching of skills, standards, values, behaviors, and attitudes
5
GENERAL PARTS TO AN ONLINE TRAINING
Title
Overview of learning objectives (observable, measurable)
Overview of topics covered
Authorizing regulation or policy (if relevant)
Estimated length of training (time commitment)
Content (information): Text, images, stories, audio, and video (with source citations)
Decision-making and walk-throughs of real-world scenarios
Assignments (actionable, optional)
Formative questions and answers
Summative assessment (with results to the learner)
Attestation
Feedback from learners about the training and assessment (for continuing improvement)
Contact information of the training provider
6
POST-TRAINING RESOURCES
Downloadables and takeaways (checklists, tip sheets, posters)
Memory enhancers like mnemonics, images, and stories
Refresher contents (post-training): Online sites, scenarios, decision walk-throughs
Encouragement to practice-practice-practice
Support and direction to continue learning (through other reputable sources)
7
METADATA ABOUT THE ONLINE TRAINING
Descriptive metadata (to help tie to other trainings and training sequences, to enhance re-usability in various contexts)
8
AFFORDANCES OF THE QUALTRICSRESEARCH SUITE FOR AUTOMATED
ONLINE TRAININGS9
WHY THE QUALTRICS RESEARCH SUITE?
#1: Rich design structures for experiential learning sequences
#2: Multimedia-rich builds (with accessibility accommodations)
#3: Connections to information systems for data tracking
#4: Branching logic
#5: Customizations
#6: Ability to build questions on prior dynamically captured data
#7: Smart data collection
10
WHY THE QUALTRICS RESEARCH SUITE? (CONT.)
#8: Smart in-tool and out-tool data analytics
#9: Scoring and threshold features
#10: Panel triggers
#11: Email triggers
#12: Fast development and deployment cycles
#13: On-the-fly revision
#14: Ease-of-collaboration through email verified accounts and shared libraries / folders
11
WHY THE QUALTRICS RESEARCH SUITE? (CONT.)
#15: Templating through libraries
#16: Security features for identity-supported controls
#17: Data integrity protections
12
#1: RICH DESIGN STRUCTURES FOR EXPERIENTIAL LEARNING SEQUENCES
Ability to build surveys / blocks / questions and to mix-and-match
Diverse range of question types
Complex sequencing based on various types of dependencies (geographical location, technologies used, answers to certain questions, performance on certain questions)
Customization capabilities
13
14
#2: MULTIMEDIA-RICH BUILDS (WITH ACCESSIBILITY ACCOMMODATIONS)
Ability to integrate links, iframes (inline frames), video, imagery, interactive objects, and other elements
Built-in accessibility tests of the sequences
Alt-texting capability for imagery
15
#3: CONNECTIONS TO INFORMATION SYSTEMS FOR DATA TRACKING
Computerized record-keeping and tracking over time (legal-grade verification) using a Qualtrics API
Enablement for mass-scale delivery of trainings (such as compliance trainings, which occur regularly and require records)
16
#4: BRANCHING LOGIC
Ability to create various training experiences for various trainees with different needs
May be based on…
performance (online behavior)
time spent in a certain block of learning
submitted (or pre-determined) learner profile
professional role (and need to acquire certain skills and knowledge sets)
geographical location
technologies used to access the training (particularly for cases where the underlying technologies change the information needed for the training)
other factors or mixes of factors
17
#4: BRANCHING LOGIC (CONT.)
May branch at any point in time in a learning sequence between blocks and block sequences
May branch between surveys
If used, must test every path through the training to ensure accuracy
Will result in different extracted performance data at the end of the training (if the particular branching paths are scored)
18
19
#5: CUSTOMIZATIONS
Piped text function {a} to deliver unique information to various parts of the training
Customization of trainings to the individual’s name and the individual’s responses
20
#6: ABILITY TO BUILD QUESTIONS ON PRIOR DYNAMICALLY CAPTURED DATA
Ability to build subsequent questions on prior dynamically captured data to elicit elaborations
“Loop & merge” question types enable asking a closed-answer multiple selection question prior and then using the unique selected selections to elicit textual responses about those selections
“Carry forward” responses enable the repeating of responses to a prior question (including open-ended ones) in a future question for more elaboration or a new elicitation
21
22
#7: SMART DATA COLLECTION
Ability to capture a wide range of quantitative and qualitative data
Capability to capture text responses
Enablement to capture digital file uploads (for digital imagery, text files, audio, video, and other basic digital file formats)
May designated acceptable file extensions only
23
#8: SMART IN-TOOL AND OUT-TOOL DATA ANALYTICS
About the Training
Analysis of the training through the Reporting -> Survey Statistics function, including light item analysis and summary performance data
About Learner Performance
Automated data summarization visualizations (both in the dashboard and the auto-created reports function)
Easy data download
Cross-tabulation analysis for certain close-ended question types
24
#9: SCORING AND THRESHOLD FEATURES
Ability to score question responses based on point values
Capacity to set thresholds for certain learning sequences
Those who do not apparently grasp certain aspects of the training may be sent for more extensive training on-the-fly
Capability to set thresholds for pass/fail (binary) or thresholds to go on various learning sequences (at a number of thresholds)
Can set thresholds to populate various panels that may lead to invites to additional other trainings (with human mediation)
Can set thresholds to send learners to new learning branches or whole trainings in an automated way
25
26
#10: PANEL TRIGGERS
Enablement to populate panels based on various factors, such as performance, geographical region, and other factors (as trigger conditions)
27
#11: EMAIL TRIGGERS
Ability to have automated emails sent when certain trigger conditions are met:
Reaching pre-set quotas (of numbers of respondents to a certain survey)
When certain answers are given
Email recipients may be to certain members (to control for potential spam)
28
#12: FAST DEVELOPMENT AND DEPLOYMENT CYCLES
Capability of developing automated online trainings in a fast way
Ability to deploy quickly in an open way (public shared URL, including semantic-friendly ones) or in a closed invitation-only way
Ability to pilot-test a training and to collect information about the training on Qualtrics®
Light item analysis
29
#13: ON-THE-FLY REVISION
Enablement to update or change a training on-the-fly
Ability to preserve the then-version of a training (vs. an updated one) for a particular trainee at the time of the first log-in
30
#14: EASE-OF-COLLABORATION THROUGH EMAIL VERIFIED ACCOUNTS AND SHARED LIBRARIES / FOLDERSAbility to invite others (even those without Qualtrics) by email to collaborate on this cloud-based tool
Ability to share user-owned survey libraries / folders of trainings (surveys), blocks, and questions
31
#15: TEMPLATING THROUGH LIBRARIES
Collection of original templates through user-owned survey libraries on Qualtrics
Access to Qualtrics’ survey templates
32
33
#16: SECURITY FEATURES FOR IDENTITY-SUPPORTED CONTROLS
Ability to hide open trainings from web browser indexers or spiders
Ability to create unique one-use URLs for trainees invited by email
Ability to analyze amount-of-time spent on training to detect potential ‘bot responses and potential dishonesty (using a crib sheet or cheat sheet to finish the training quickly)
Ability to prevent access to trainings from the same IP address (what is called prevention against “ballot box stuffing” but may serve a different purpose in an automated online training context)
34
#17: DATA INTEGRITY PROTECTIONS
Protection of data stateside, without data moving across undersea lines to overseas data centers
35
IN SUMMARY:
Not about stand-alone capabilities but about tactical combinations in order to achieve training objectives
36
AND SOME LIMITS…
37
AND SOME SYSTEM LIMITS…
Trainings are non-portable. Trainings are downloaded as proprietary .qsf files, and linked data may be downloaded as .csv, but they require Qualtrics to reconstitute online.
Currently, it is not possible to archive trainings with data in a library (only the trainings themselves can be archived, without the related data).
All complex sequences have to be thoroughly tested for clear behaviors. (The branching logic visualizations could benefit from richer details, such as a full decision-tree layout vs. just a linear listing.)
There could be more effective in-system analytics that focus on learner behaviors and inferences about their learning.
38
REQUIREMENT FOR DEVELOPER SUPPORT
Connecting to information systems (databases) will require developer work to use the Qualtrics API for one-directional automated data sharing.
This feature usually works with closed panel invites to trainings (with learner identifiers emplaced).
Another option is to have an open URL training and the collection of identifiers during the training (but this may introduce some noise and some room for mis-identification of the self).
39
AND? SO?
40
THINK AFFORDANCES AND EFFICIENCIES
Think about tech tools based on capabilities, not the initial design intentions of the developers or the depictions of the marketers.
Integrated solutions, which address a broad range of interests (legal, record-keeping, training designer, learner), benefit the entire process.
There are costs to investing in any socio-technical platform, and it helps to consider various scenarios and use cases.
41
CONCLUSION AND CONTACT
Dr. Shalin Hai-Jew
iTAC, Kansas State University
212 Hale / Farrell Library
785-532-5262
Another Resource: Using Qualtrics for Online Trainings
The presenter has no formal tie to Qualtrics, Inc.
Thanks to Dr. Jana R. Fallin and her team at the K-State Teaching & Learning Center for accepting this digital poster session for the Big XII Teaching and Learning Conference (2016)! Thanks to Dr. Rebecca Gould for supporting this work!
42