1. Analytics & Higher EducationVince Kellen, Ph.D. Senior Vice Provost Analytics and Technologies University of KentuckyVince.Kellen@uky.edu January, 2014This is a living document subject to substantial revision.

2. Higher education has a last mile problem Education in any form is struggling to address families and communities with economic and other readiness problemsFree or low-cost educational content does not easily solve readiness problems which have a multitude of factorsFor profit models rightfully struggle with last-mile problems. Public policy matters!2 3. What would Abraham Lincoln think of things? Abraham Lincoln Autodidactic Books, books, books Became a skilled military strategist Penchant for poetry, Shakespeare, politics and historyMy nephew Not an autodidact Good worker, smart kid, but It takes a village After a few low-security colleges and much money borrowed He has found an intellectual home3 4. Analytics in higher education is hot right now A number of vendors, old and new are making sales now Knewton, Starfish, Civitas, Education Advisor Board, Banner (Signals) and others Value proposition: we collect lots of data, we have data scientists who can analyze, we collect lots of best practice that we can share with you, we host the data and systems so you dont have to Adaptive learning got a boost this past year Gates Foundation spurred implementations with funding for colleges and universities to implement pilot programs Reports by Education Growth Advisors, Foundation report identified and evaluated eight adaptive learning vendors APLU formed a Personalized Learning Consortia (also partially funded by the Gates Foundation) to spur collaboration between universities for personalized learning content and platform development Industry press has picked up coverage (Chronicle, InsideHigherEd) and vendors are making acquisitions 4 5. Analytics have been around awhile Thousands of studies and research on many aspects of student successand the psychology of learning over several decades Most institutions understand the common causes High school GPA, test scores, family familiarity with higher education, personal and family expectations, wealth, high identification with an academic discipline, high motivation, conscientiousness, involvement in academics and co-curricular activities, strength of and placement within a social network, problems in course progressions and degree choice Most institutions understand and are starting to act upon the signals First semester performance, mid-term grades, the first few weeks of progress in academic classes, identification of gaps in needed skills and remediation Many institutions are taking steps to reform the teaching and support Hybrid designs, active learning designs, guide-on-the-side versus sage-on-the-stage Early intervention, live and learn communities, peer mentoring, professional advising5 6. What is different now? The technology for analytics has undergone a recent renaissance Different forms of high-speed, big-data processing are coming forward Structured and unstructured data can be rapidly analyzed Queries can be run against both structured and unstructured stores simultaneously The hardware is now more parallel enabling scale-out designs to handle big data Apple Siri, IBM Watson and others have grabbed mainstream attention Data visualization is well established Many tools offer interesting ways of visualizing data, enabling better communication of insights MOOCs have brought attention to eLearning opportunities MOOCs are expected to incorporate analytics to help improve learning outcomes Organizations and vendors are facing some critical transitions How do old-line database and analytic vendors change their tools to compete with new approaches (e.g., the Hadoop bandwagon, SAP HANA, etc.) How do institutions adopt and take advantage of the new tools? What skill sets are lacking? What organizational pieces need to be put into place? Can institutions integrate the data needed? 6 7. What we have done and what we would like to do First steps over the past year Mobile micro-surveys: Learning from the learner Student enrollment, retention, demographics, performance, K-Score, facilities utilization, instructor workload and more High speed, in-memory analytics architectural differences Open data and organizational considerations Coming down the road? Micro-segmentation tool to enhance user and IT productivity, develop personalized mobile student interaction/intervention Models for learner technographics, psychographics, in addition to behaviors, performance, background Advanced way-finding for streaming content like lecture capture Content metadata extraction and learner knowledge discovery Real-time measures of concept engagement and mastery Real-time learner recommendations and support engine Use graphing algorithms to perform more sophisticated degree audit what ifs 7 8. ModelDescriptionEnrollmentEnrollment in a class, midterm and final grades, credit hours attempted and earned, instructor teaching the classStudent retention and graduationStudent demographics and cohort identification (e.g., John Doe is in the 2009 entering first-year student cohort)Student demographicsDemographics, such as age, high school GPA, entrance test scores (SAT, ACT) and subcomponent scores. Also, in a secure location, additional personally identifiable demographic details such as name, address, email, etc.Student performancePresent the enrollment data in such a way as to easily show the students performance for each term, including credit hours earned, term GPA, cumulative GPA for that term, etc.Student academic career Keep a list of the majors and minors for each student and degrees awarded. Also, include details on students who transfer in and out, including transfer institution, credit hours transferred in, etc. ProductivityThe room utilization model contains every building, every potential classroom and lets users analyze the room capacity and enrollments for the class or event in the room at five minute intervals. The faculty stats per term model pulls together the number of students and sections taught per term and will contain other important data such as research expenditures per term and grant proposals submitted and won.Micro-surveysCapture questions and answers from the My UK Mobile micro-survey featureStudent involvementInteraction history with various applications including the learning management system, clickers, course capture and playback, academic alerts. Provide the basis for calculating the students K-Score. 8 9. My UK Mobile usage stats9 10. 10 11. 11 12. Predictive analytics at its finestOur Ph.D. data scientists locked themselves in a room and worked very hard on an approach to more reliably predict first-year student retention from fall to spring term.What they came up with amazed us12 13. A question sent via MyUK Mobile to freshman who are not doing so well just prior to midterms:Do you plan on coming back in the spring?13 14. 14 15. Academic Health Notifications: View in student mobile app 16. 18 17. K-Feed: Intelligent, personalized alerts, news, reminders19 18. Taxonomy? Automatic metadata? Automatic atomic metadata? Let learners navigate anaudio/visual stream Let the system learn what are topterms. Let the system map terms to concepts. Let instructional designers lightly bump the taxonomy, post production Record student engagement withspecific terms / concepts See http://p.uky.edu Deliver personalized messages tostudents20 19. University of Kentucky 20. Key questions Can the audio and slides be reliably converted into useful text? Can a concept map be derived automatically from the text generated or easily edited by an instructor? How easy will it be for designers-instructors to create an assessment and guide its placement in the right location in the video? Can we personalize the recommendations to reflect prior knowledge, student ability and individual differences in information processing? Can the interface support real-time integration with high-speed analytic back-ends (e.g., HANA)? Can advising, learning and general support processes be integrated? Can this be cost-effective for existing courses? This is just one conceptualization. What other interface designs might exist? How effective will they be? 21. Personalize learning and support in one architecture Real-time personalized interactions Target on-demand peer tutoring based on students profile Deliver micro-surveys and assessments to capture additional information needed to improve personalization Give students academic health indicators that tell students where they can improve in study, engagement, support, etc. Let students opt their parents in to this information so the family can support the student Tailor and target reminder services, avoid over messaging, enable timing of message delivery based on user temporal proclivitiesAllow for open personalized learning How content gets matched to students is psychologically complex Several theories of how humans learn give many insights Students differ in the following abilities and attributes: visual-object, visualspatial, reasoning, cognitive reflection, need for sensation, need for cognition, various verbal abilities, confidence, persistence, prospective memory, etc. We need an open architecture to promote rapid experimentation, testing and sharing of what works and what doesnt University of Kentucky 22. Is learning analytics a sustainable opportunity? Lets look at the pieces to the puzzle of the value proposition We have the expertise, you dont Researchers have been examining many aspects of students success and learning, but do all vendors have experts knowledgeable of the breadth of this literature? Research into how the brain learns is yielding much recent insight, but a single theory of mind for learning has not emerged, nor will one soon. There is complexity, counter-intuitiveness and much more to understand regarding how the brain functions While data science is a scarce skill sets, universities, especially research universities, tend to have these skills We have large data across multiple institutions that you dont Does one need large data to gain valid insights? Exactly where does large data provide learning analytics benefits? For inductive, atheoretical approaches, perhaps big data can help (e.g., millions of rows of clickstream). Do students brains vary that much that very large sample sizes are needed? Have we exhausted existing and emerging theoretical approaches? We have access to multi-institutional best practices that you dont Good point 24 23. Where are we going? Question 1 Should knowledge of how students learn be considered a private orpublic good? Learning analytics are nonrival, that is, if I gain knowledge of how students learn that does not simultaneously deprive you of the same opportunity Learning analytics are excludable, that is, if I have a piece of software that collects data on how students learn, I can prevent you from getting it How excludable are learning analytics? Most if not all learning analytic companies base their analytics off of published research. While a single vendors knowledge of learning analytics might be excludable, the prior art is often commonly available How easy would it be for a competitor or a customer to reverseengineer an approach or design an alternative? 25 24. Questions 2 and 3 Is making knowledge about how students learn excludable the right thing to do? How would you feel about the same scenario, but now regarding a difficult, lifesaving, heart surgical procedure? Image a company that has figured out how to improve learning for 90% of human beings by increasing learning outcomes over any time frame by 100% while maintaining or reducing costs of instruction? Suppose this approach is available at the following price: $75,000 per student Who gets to benefit? Who does not?While in the U.S. medical procedure patents have been permissible, many countries ban them. Most, if not all medical procedures are based on prior art. Many in the medical community are opposed to these types of patents as they can interfere with educating doctors, and impede public health objectivesWhat does this mean for vendors? Learning analytic procedures (algorithms) would have to be free from reliance on prior art which might be difficult for most vendors While copyright law can protect the software written regarding learning analytics, copyright law cannot compel anyone from revealing what aids learning Vendor goals and university goals are not always aligned 26 25. Is this something to worry about? Perhaps not Cognitive psychology, neuroscience and learning theory are rapidly evolving. Recent brain imaging and sensor advances are expanding knowledge quickly How humans learn is amazingly complex, and even harder to apply in single event cases. How many of you have family in college you just cant seem to help? Universities can easily create open source versions of learning analytic tools, and sharing specific knowledge about their students with others As learning analytic knowledge diffuses, universities will then shift the competitive effort on to those activities that their faculty and staff perform (skill of the doctor) versus gaining access to the analytically-power learning tool (medical procedure) We live in a connected world. Countries might demand learning insights be public goods hurting companies relying on keeping knowledge excludable. Research globally can undermine vendors locallyPerhaps so Universities sometimes move in haste and en masse, and dont have the expertise to build their own tools, thus will be reliant on vendors Vendors, by design, are motivated to make data and insights into data excludable Regulation in a single nation can encourage further privatization of education, thus locking up insights into things universities must purchase (and not generally use) 27 26. Scarce / Not scarce Scarce1. Management ability to know how to build an organization to take advantage of analytics 2. Enterprise architecture and data science skills 3. Ability to integrate from disparate sources quickly 4. Order of magnitude improvement in cost-effectiveness Not scarce1. 2. 3. 4.Ideas for analytics Raw data (dark data) Tools Willing students28 27. Dj vu? MOOCsLarge lecturesPHI 698???http://www.thelongtail.com/conceptual.jpg 29 28. Questions?http://www.independent.co.uk/life-style/gadgets-and-tech/features/from-fighter-jets-to-google-glass-headup-displays-make-the-jump-to-mainstream-gadgets-8854485.html30