institutional reflection on readiness: the learning analytics readiness instrument (237810993)
TRANSCRIPT
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 1/80
Institutional Reflection on Readiness:The Learning Analytics Readiness Instrument
May 19, 2014
Kimberly Arnold, University of Wisconsin-Madison
Steven Lonn, University of Michigan
Matthew D. Pistilli, Purdue University
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 2/80
Poll
Who is attending this seminar with you?
A) I’m attending by myself
B) There are a small handful of us here
(< 5)C) There are many people here (6+)
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 3/80
Who are we?
Where are we from?Why are we here?
Introductions and Purpose
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 4/80
Introduction
What is Learning Analytics?
The Learning Analytics Readiness Instrument
Breaking Down the Factors
Concluding Thoughts
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 5/80
What is Learning Analytics:
Definitions
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 6/80
Definitions of Learning Analytics Using analytic techniques to help target instructional,
curricular, and support resources to support theachievement of specific learning goals (van Bareneveld,Arnold, & Campbell, 2012)
the process of developing actionable insights throughproblem definition and the application of statisticalmodels and analysis against existing and/or simulatedfuture data (Cooper, 2012)
The use of data, statistical analysis, and explanatoryand predictive models to gain insights and act oncomplex issues (Bichsel, 2012)
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 7/80
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 8/80
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 9/80
Challenge: How do you find the student at risk?
https://reader010.{domain}/reader010/html5/0613/5b2057b131b74/5b2057b683499.jpg
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 10/80
https://reader010.{domain}/reader010/html5/0613/5b2057b131b74/5b2057b79bc43.jpg
Challenge: How do you find the student at risk?
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 11/80
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 12/80
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 13/80
Analytics is about…
Actionable intelligence
Moving research to practice
Basis for design, pedagogy, self-awareness
Changing institutional culture
Understanding the limitations andrisks
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 14/80
http://i.imgur.com/nZArTnc.jpg
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 15/80
New Possibilities
Using data that exists on campus
Taking advantages of existing programs
Bringing a “complete picture” beyond academics
Focusing on the “Action” in “Actionable Intelligence”
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 16/80
The Concept of LA Readiness
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 17/80
What is “Readiness”
• What things do you think of when
you hear Institutional Readinessfor Learning Analytics?
• Discuss / Brainstorm
– What does the institution need?
• For whom? When?
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 18/80
Institutional Readiness
- Share Out -
• What did you / your team list?
• What is the highest / first priority?
– Why?
• What is already in place vs. needs to be
purchased / developed / trained?
• Are their larger / cultural issues to discuss
before logistical issues can be addressed?
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 19/80
Poll
How ready is your institution to
implement learning analytics?
A.We’re ready!
B.We need a little planning, but we are
basically ready.C.We’ve talked about learning analytics,
but made no plans.
D.We are not ready.
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 20/80
Readiness (ˈred-ē-nis)
noun1 willingness to do something.
2 the state of being fully prepared
for something.
3 immediacy, quickness, or promptness.
• Adopting learning analytics requiressignificant investment of:
– Time
– Resources – Money
http://www.flickr.com/photos/gadgetgirl70/2918038830/
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 21/80
Readiness for
Learning Analytics Will not happen
accidentally
Logistics can
be daunting
Institutional
reflection is
criticalhttp://ieatgrass.com/2011/05/ready‐set‐go/
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 22/80
The Need for Reflection
• Comprehensive understanding
• Multiple perspectives needed
– Cross-disciplinary
– Diverse expertise & skills• More realistic understanding
of resources
• Limit bias that inadvertentlyinterjected
“Thinking” designed by
Ashley Fiveash
from
The Noun Project
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 23/80
Assessing Readiness
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 24/80
Analytics “Maturity”Educause’s (2012) Center for Analysis and Research’s Analytics Maturity Index
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 25/80
Framework for Creation
Provide an institutional profile with
readiness indicators
Serve as a prescriptive diagnostic
Situated within L.A. literature
Formative instrument focusing on“3 Ps”
Parsimony
Practicality
Proactive
http://www.flickr.com/photos/tschiae/8417927326/
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 26/80
The Learning Analytics
Readiness Instrument(LARI)
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 27/80
Item Development Questions based on:
Literature
ECAR Maturity Index
Practitioners’ experience
Original Factors:
ABILITY DATAGOVERNANCE/
INFRASTRUCTURE
CULTURE PROCESS
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 28/80
Poll
What is the most important factor to
consider when preparing for a learning
analytics implementation?
A. Ability
B. DataC. Culture and process
D. Governance and infrastructure
E. Readiness perception
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 29/80
Participants• 33 participants across 9 institutions
(8 US, 1 Canadian)• R1s
• Already established foothold in LA
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 30/80
Exploratory Factor Analysis Began with 139 items
Varimax rotation with Kaiser normalization Exclusion criteria of |.30|
42 survey items eliminated
Second factor analysis 7 additional items eliminated
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 31/80
New Factors
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE
PROCESS
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 32/80
New Factors
ABILITY ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE
PROCESS
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 33/80
New Factors
ABILITY
DATA
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE
PROCESS
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 34/80
New Factors
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE
PROCESS
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 35/80
New Factors
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE /
PROCESS
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE
PROCESS
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 36/80
New Factors
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE /
PROCESS
OVERALL
READINESS
PERCEPTIONS
ABILITY
DATA
GOVERNANCE /INFRASTRUCTURE
CULTURE
PROCESS
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 37/80
Exploratory Factor Analysis
Final 90 survey items
Explained 55.7% of variation
Chronbach’s alpha of .946
Factor # of Items Median Skewness Kurtosis α
Ability 19 -.038 -.401 -.640 .940
Data 22 .228 -1.157 .686 .945Culture &
Process23 .146 -.420 .073 .898
Governance &
Infrastructure15 .164 -1.299 2.131 .896
OverallReadiness
Perceptions11 .006 -.734 1.421 .841
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 38/80
Results
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 39/80
-2
-1.5
-1
-0.5
0
0.5
1
1.5
Ability Data
Culture / Process Governance / Infrastructure
Readiness Perceptions
Institutional Differences(at least 2 respondents)
A B C E F G I
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 40/80
-2
-1.5
-1
-0.5
0
0.5
1
1.5
Ability Data
Culture / Process Governance / Infrastructure
Readiness Perceptions
Institutional Differences(at least 2 respondents)
A B C E F G I
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 41/80
-2
-1.5
-1
-0.5
0
0.5
1
1.5
Ability Data
Culture / Process Governance / Infrastructure
Readiness Perceptions
Institutional Differences(at least 2 respondents)
A B C E F G I
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 42/80
Limitations• Sample size
• Institution type
• Policy implication in international
settings
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 43/80
Breaking Down the Factors
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 44/80
Breaking Down the Factors: Data
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 45/80
Data is…
Foundational
Abundant
Diverse
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 46/80
Data is often…
Abundant (confusing)
Not archived
Dispersed
Siloed
Inaccessible
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 47/80
Data considerations…
“Big” or “small”
Validity and reliability of data sources
Storage, archival, and access
Ownership/stewardship/utilization
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 48/80
LARI Items
The extent to which my institution currently
collects admissions data (e.g., prospect
demographics, interests, applicationevaluation)
At my institution, the primary “owner” (the
person / group responsible for granting accessand stewarding the data) of the following data
sources is....
At my institution there are routine scripts /processes for refreshing / updating institutional
reports.
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 49/80
Reflection Questions…
On what topic(s)/issue(s) do you plan to
focus and why?
What data is important to gather? How will
you know if you have the “right” data? The
“best” data?
What are the challenges associated with
gathering, assembling, and maintaining
data?
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 50/80
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 51/80
Breaking Down the Factors: Ability
P ll
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 52/80
Poll
Which abilities and expertise are most
important for a successful learning analytics
initiative?
A. Technical architecture
B. Analytics proficiencyC. Strategic leadership
D. Instructional design
Abilit
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 53/80
Ability
Wide-ranging abilities and expertise will
need to accessible
Different abilities will need to be accessed
at different points
Abilities needed will be different dependingon goals
Abilit
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 54/80
Ability
Technical expertise (architecture, coding,
development)
Analytic proficiency
Strategic leadership
Student support skills Instructional and curriculum expertise
Researchers and evaluators
Abilit C id ti
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 55/80
Ability Considerations
Technical expertise (architecture, coding,
development)
Analytic proficiency
Strategic leadership
Student support skills Instructional and curriculum expertise
LARI Items
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 56/80
LARI Items
My institution has professionals with
mathematical/statistical experience in deriving
and validating statistically sound algorithms(e.g., simulations, cross validation).
My institution has professionals who have
specialized analytics training. My institution has professionals with technical
knowledge / expertise in documenting data
store / warehouse structure and datadictionaries.
Reflection Questions
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 57/80
Reflection Questions
How do you identify where expertise exists?
How will you manage “territorial disputes”
between departments/units that have staff
with needed expertise?
To what extent is your executive leadershipinvolved in moving this project to fruition?
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 58/80
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 59/80
The Learning Analytics Readiness Instrument (LARI)
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 60/80
How would you describe your institution’s approach to using data on a day‐
to‐day basis?
A) We ALWAYS use data to bolster actions we’re going to take
B) Data is available, and sometimes we access it to inform our decision
C) Data is available, but we usually don’t refer to it
D) Data is difficult to get, so we only ask for it when absolutely necessary
E) Data? What’s data? We don’t need data to make good decisions
Poll
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 61/80
• Shared vision of what analytics is
• Shared vision of student success
• Awareness and acceptance of data‐driven decision
making
• Goal
orientation
for
implementing/using
analytics• Stakeholder support of the effort
• Policies and practices amenable to examining/using
student data
• Communication between units about data/data use is
commonplace
Culture
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 62/80
• Stakeholder involvement in the effort
• Project
management
resources• Sustaining new initiatives
• Data and report use in practice
• Established procedures for collecting, organizing, storing, accessing, sharing, using, and securing data
Process
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 63/80
• My institution has the ability to store, manage, and analyze increasingly large volumes of
data.• Institutional reports routinely inform
institutional decisions.
• There is
a clear
vision
of
where
my
institution
can make changes to help students be more
successful academically.
• My institution
has
measurable
objectives
that
will indicate if success has been achieved.
LARI
Items
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 64/80
• What are the different policies that exist on
campus regarding access to and use of data?
• To
what
extent
is
data
currently
reported
on
your
campus?
What
makes
the
determination
of
what is reported, and the audience(s) to whom
those reports go? How are those reports used
by
the
recipients?
To
what
extent
is
the
information presented trustworthy?
• In
what
ways
has
your
institution
defined
student
success?
If
multiple
definitions
exist, are they competing?
Campus
Discussion
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 65/80
Institutional Report Out
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 66/80
Breaking down the factors:
Governance & InfrastructureThe Learning Readiness Instrument (LARI)
Poll
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 67/80
Poll
How would you characterize your current
governance and infrastructure around learning
analytics?
A) A benefit. Resources and professionals are
ready to assist in LA efforts.
B) A hindrance. No cohesive rules or resources.
Individuals can block progress.
C) A mixed bag. Some elements work great, someneed a lot of work.
67
Governance
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 68/80
Governance
• Broad, long-term involvement of LA efforts
• Sustainability
• Policy, goals ownership
(related to Culture & Process) – Policies that allow appropriate use
68
Infrastructure
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 69/80
Infrastructure
• Professional knowledge / expertise
– Building & maintaining large data warehouses
– Documenting data sources
– Query building, extraction – Manipulating & combining data sources
• Customer Service
– Diverse constituencies
– Documentation, tutorials
69
LARI Items - Governance
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 70/80
LARI Items Governance
• My institution has had conversations regarding
the sustainability of an analytics effort.
• People from multiple offices across my institution
are involved in the effort.
• My institution views learning analytics as a long
term investment, rather than a short term
expense.
• My institution has IT professionals who know
how to support analytics.
70
LARI Items - Infrastructure
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 71/80
• My institution has professionals with expertise in:
– Building / Maintaining complex data stores /warehouses
– Query building and data extraction
– Manipulating data from multiple sources and
platforms to conform to institutional
specifications
– documenting data store / warehouse structure
and data dictionaries
71
Worksheet
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 72/80
Discuss the following questions:
1. What institutional data is collected, curated, andmaintained for use in research projects?
2. How accessible are the subject matter experts on
institutional data?3. What institutional policies exist that govern the use of
varying kinds of data?
1. To what extent do these policies compete with one
another?
2. Who has the ability to determine which policies apply or
are superfluous to the project?72
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 73/80
Institutional Report Out
73
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 74/80
Future of the LARI
Next Steps
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 75/80
• Continue iterating the instrument
• Create tailored feedback and automate
– Overall profile score; domain scores
– Low, medium, high readiness
– Broad,
domain‐
specific
feedback – Prompts for exploration
• Build out scaffolds and activities
• Identifying international
partners
Beta Partners
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 76/80
• Lead individual
at
American
institution
• 10‐15 individuals in various roles representing
the diversity of job roles associated with
analytics
• Provide honest and constructive feedback
• Participate
in
follow‐
up
study
on
use
and
usefulness of report
• To become involved, email
Expectations Reality
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 77/80
p y
Plug and Play
Immediate results
Solve every problem –
ever!
Universal adoption
Everyone would love it!
Fits, starts, reboots
Mostly long termoutcomes
Solve some problems,
create some newproblems
Lackluster use
Not everyone loved it
Institutional Challenges
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 78/80
Data in many places, “owned” by manypeople/organizations
Different processes, procedures, and regulationsdepending on data owner
Sustainability
Everyone can see potential, but all want something slightlydifferent
“can’t you just…”
Faculty participation is essential
Staffing can be a challenge
In the end…
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 79/80
It is challenging
It can be difficultBUT
It is doable
This is the purpose of the LARI
Please consider joining the beta
8/11/2019 Institutional Reflection on Readiness: The Learning Analytics Readiness Instrument (237810993)
http://slidepdf.com/reader/full/institutional-reflection-on-readiness-the-learning-analytics-readiness-instrument 80/80
Institutional Reflection on Readiness:
The Learning Analytics Readiness Instrument
Kimberly Arnold, University of Wisconsin-Madison
Steven Lonn, University of MichiganMatthew D. Pistilli, Purdue University