st century: using data to - bluenotes group€¦ · century: using data to respond to opportunities...

Higher Education in the 21st

Century: Using Data to Respond to Opportunities

and Challenges

August 1 – 4, 2017

Bluenotes Americas 2017 About eXplorance At eXplorance, a Learning Experience Management (LEM) solutions provider, we empower organizations in making the right decisions with fact‐based learning analytics. Our offerings, Blue® and Bluepulse®, help instill a culture of continuous improvement by evaluating, analysing, and improving stakeholder needs, expectations, skills, knowledge, and competencies. Founded in 2003, eXplorance is a privately held corporation based in Montreal, Canada with offices in APAC, Europe, and Latin America. With employees working across the globe, eXplorance is renowned for its strong company culture and is deemed one of the Best Workplaces in Canada by the Great Places to Work Institute®. Recognized for its growth, technical innovation, and entrepreneurship, eXplorance is listed in Deloitte’s North American Technology Fast 500™. eXplorance’s clients include academic institutions such as the University of Melbourne, University of Pennsylvania, University of Toronto, Zayed University, University of Louisville, Del Mar College, Bowdoin College, IESE Business School, Xi'an Jiaotong‐Liverpool University, UDLAP, and Liverpool John Moores University and organizations including Aramco, National Bank of Canada, and NASA.

Keynote Speaker

Wednesday, August 2

Cofounder and Chief Learning Officer, Civitas Learning

Dr. Mark David Milliron

Dr. Mark David Milliron is an award‐winning leader, author, speaker, and consultant, who works with universities, community colleges, K‐12 schools, foundations, corporations, associations, and government agencies across the country and around the world. Mark also serves as Executive Director of the Next‐Gen Leadership Academy, a program dedicated to grounding rising educational leaders in the issues and innovations of the day, and the leadership skills necessary to help institutions thrive in dynamic times. In previous roles, Mark served as the Deputy Director for Postsecondary Improvement with the Bill & Melinda Gates Foundation; founding Chancellor of WGU Texas; Endowed Fellow and Director of the National Institute of Staff and Organizational Development at The University of Texas at Austin; Vice President for Education and Medical Practice with SAS; and President and CEO of the League for Innovation in the Community College.

Speak with eXplorance Staff

Wednesday, August 2: 9:15 AM to 4:30 PM

“I need help” Support station: Sara Alkadri, Software Support Engineer Michael Cochrane, Customer Support Engineer

“I have feedback”

Product Management station: Long Hua, Chief Product Officer Zelbrey Bedard, VP, Product Management (Blue) Georges Geha, VP, Product Management (Blue)

“I want to learn more” Demo station: Margot MacNutt, Training Manager Rola Omar Al‐Ghoul, Manager – Project Success

Product features in sandbox Blue Lab (open sessions)

Thursday, August 3: 9:15 AM to 4:30 PM





Product features in sandbox Blue Lab (open sessions 9:45 AM to 2:45 PM; 3:15 PM to 4:30 PM) Blue Lab (reserved sessions 9:15 AM to 9:45 AM) Blue Lab (reserved sessions 2:45 PM to 3:15 PM)

Friday, August 4: 8:30 AM to 10:45 AM





Product features in sandbox Blue Lab (open sessions)

Tuesday, August 1, 2017

Thought Leadership Roundtables and Workshops

8:00 AM to 9:45 AM

Thought Leadership Roundtable: Response Rates: Drawing Insights from This (not so) Simple Metric Moderators: Gregory Hum, Kosha Bramesfeld, and Victoria James, University of Toronto Broadway A

8:00 AM to 9:45 AM Workshop: Autonomy as Driver for User EngagementInstructors: Margot MacNutt and Krimo Bouaou, eXplorance Broadway B

9:45 AM to 10:15 AM Morning Break

10:15 AM to Noon

Thought Leadership Roundtable: Course Evaluation Feed Forward: Students as Stakeholders in the Learning Process Through Deep Formative Feedback Practices Moderator: Krisy Mahome, Indiana University Bloomington Broadway A

10:15 AM to Noon Workshop: Automation of Your Feedback ProcessInstructors: Margot MacNutt and Krimo Bouaou, eXplorance Broadway B

Noon to 1:00 PM Lunch Bluegrass Room

1:00 PM to 2:45 PM

Thought Leadership Roundtable: Rethinking Course Evaluation Administration for Competency‐Based Education Courses Moderator: Lee Bewley, University of Louisville Broadway A

1:00 PM to 2:45 PM Workshop: Insights from Your Qualitative QuestionsInstructors: Margot MacNutt and Krimo Bouaou, eXplorance Broadway B

2:45 PM to 3:15 PM Afternoon Break

3:15 PM to 5:00 PM

Thought Leadership Roundtable: The Blue Guru: Sharing Your Expertise and Gaining New Insights Moderators: Katie Partin and Katie Bixby, University of Louisville, Thomas Dohm, University of Minnesota Broadway A

3:15 PM to 5:00 PM Workshop: Tips and TricksInstructors: Margot MacNutt and Krimo Bouaou, eXplorance Broadway B

7:00 PM to 9:00 PM Opening Night Reception Brown Hotel Rooftop

Wednesday (August 2)

7:15 AM to 8:00 AM

Registration/Breakfast

8:00 AM to 9:00 AM

Keynote Address

Dr. Mark Milliron, Civitas Learning

9:00 AM to 9:15 AM

Early Morning Break

9:15 AM to 10:45 AM

Stories from the Past, and Vision of the Future – eXplorance CEO Update

Samer Saab, eXplorance

10:45 AM to 11:00 AM

Mid‐Morning Break

11:00 AM to 11:50 AM

The Blue Cascaded Course Evaluation

Model

A Tale of Two Faculties: Course

Evaluation Implementation at a Large University

Blue (ART) and Other Musings

Complete the LEM Framework with Embedded Faculty

Development

eXplorance University of Toronto

University of Minnesota

Verteres Partner, LLC

Noon to 1:00 PM

Lunch

1:00 PM to 1:50 PM

Leveraging Blue’s Enhanced Data

Structure Capabilities

Measuring What We Do: Using Blue to Survey and Improve Our Landscape

What Can We Learn about Teaching and Learning

The Role of an End of Semester Module

Questionnaire

eXplorance University of Louisville

Dalhousie University

Xi’an Jiaotong‐Liverpool University

2:00 PM to 2:50 PM

Implementation of Blue Text Analytics

at Temasek Polytechnic

Blue Implementation in a Decentralized Environment

How to Embrace Your Data

Blue Survey to Advance

Institutional Research and Initiatives

eXplorance UNC‐Chapel Hill Nuventive University of Louisville

2:50 PM to 3:10 PM

Afternoon Break

3:10 PM to 4:00 PM

Advanced Integration with LMS and SIS

Contextualizing Student Feedback

Opting In & Changing Dates

Monitoring Academic Quality: When Data are Challenged

eXplorance University of

California Berkeley James Madison

University Southern Cross

University

Thursday (August 3)

7:45 AM to 8:15 AM

Breakfast

8:15 AM to 9:15 AM

Structured Networking

9:15 AM to 9:45 AM

Morning Break

9:45 AM to 10:35 AM

Lessons Learned from Implementing Bluepulse

Thinking Beyond the Traditional ‘Instructor Evaluation’

Response Rate Monitoring

Beyond Numbers: Using Qualitative

Results

eXplorance, Education Affiliates

Galen College of Nursing

eXplorance, University of Louisville

Auckland University of Technology

10:45 AM to 11:35 AM

Unveiling BlueX

No 360? No Problem

Designing and Improving CBE

Programs

Piloting Blue Online Course Evaluation

Solution

eXplorance Oregon Health & Science University

Blackboard, Inc. Durham University

11:45 AM to 12:45 PM

Lunch

12:45 PM to 1:35 PM

The Latest and Greatest of Bluepulse

The Journey of Kent State University

Factors Contributing to a Rapid and Successful Implementation of Blue

Is Over‐surveying Undermining your

Efforts?

eXplorance Kent State University

Monash University University of Louisville

1:45 PM to 2:45 PM

Data Preparation

Made Easy and Less Painful!

Marketing Outreach to

Improve Response Rates

The Elephant in the Room: A Reflection of a

South African Case

Journey Into “Blue” online course evaluation

eXplorance Duquesne,

Pittsburgh, UofL Vaal University of

Technology Blinn College

2:45 PM to 3:15 PM

Afternoon Break

3:15 PM to 4:05 PM

Lessons Learned and Best Practices

Using Evaluation Data Strategically

Implementing Data Integrity Gateway (DIG)

Making the Most of Text Analytics

eXplorance Galen College of

Nursing Belmont University


4:15 PM to 5:00 PM

Question Bank 2.0 – How We Made it Easier for Our Users to Submit Questions

Using Blue to go Beyond: Meeting the Challenges of

Developing Business Leaders

Removing Obstacles in Using Blue in Your Environments

Implementing Blue in a Large Multi‐School System: Lessons Learned

University of Kentucky

Lake Forest Graduate School of Management

eXplorance Education Affiliates

Friday (August 4)

8:00 AM to 8:30 AM

Breakfast

8:30 AM to 9:30 AM

Bluepulse [closed session]

Blue Tips and Tricks (part I)

Blue 360 ‐ Taking

Advantage of Multi‐Rater Feedback

Harnessing Feedback in the Next

Generation Digital Learning Ecosystem

eXplorance eXplorance eXplorance Instructure

9:30 AM to 10:35 AM

BlueX [closed session]

Blue Tips and Tricks (part II)

Data at the Fingertips of Faculty – Unveiling a New Blue Dashboard

Bluenotes Group

Community Update and Customer Satisfaction

(CSAT) Survey Results

eXplorance eXplorance eXplorance eXplorance

10:45 AM to 11:15 AM

Closing Ceremony

Wednesday, August 2, 2017

TIME SESSION DETAILS

7:15 AM to 8:00 AM Registration/Breakfast Mezzanine/Crystal Ballroom/South Alcove

8:00 AM to 9:00 AM

Welcome/Introduction to ConferenceBecky Patterson, University of Louisville Keynote Address: The Art and Science of Student Success: Using Design Thinking and Data Science to Help Students Learn Well and Finish Strong Speaker: Dr. Mark David Milliron, Civitas Learning Crystal Ballroom

9:00 AM to 9:15 AM Early Morning Break Crystal Ballroom

9:15 AM to 10:45 AM Session: Stories from the Past and Vision of the Future – eXplorance CEO Update Presenter: Samer Saab, eXplorance Track: Strategic Crystal Ballroom

10:45 AM to 11:00 AM Mid‐Morning Break Crystal Ballroom

11:00 AM to 11:50 AM

Session: The Blue Cascaded Course Evaluation Model to Make Data More Meaningful by Leveraging Institutional Hierarchy Presenter: Long Hua, eXplorance Track: Blue Administrators Broadway A Session: A Tale of Two Faculties: Course Evaluation Implementation at a Large University, a Study of Contrasts and Lessons Learned Presenters: Victoria James and Gregory Hum, University of Toronto Track: Strategic Broadway B Session: Blue Advanced Relationship Tables (ART) and Other Musings Presenter: Daniel Glover, University of Minnesota Track: Technology Broadway C Session: Complete the LEM Framework with Embedded Faculty Development Presenters: Pete Baccile, Kristin Good, and Brenda Barnes, Verteres Partners, LLC Track: Innovation Labs Crystal Ballroom

Noon to 1:00 PM Lunch Bluegrass Room

1:00 PM to 1:50 PM

Session: Leveraging Blue’s Enhanced Data Structure Capabilities for Dynamic Report Access and Teaching Roles Support Presenter: Zelbrey Bedard, eXplorance Track: Blue Administrators Broadway A Session: Measuring What We Do: Using Blue to Survey and Improve Our Landscape Presenter: IL Barrow, University of Louisville Track: Strategic Broadway B


1:00 PM to 1:50 PM

Session: What Can We Learn about Teaching and Learning from Our Student Evaluation of Teaching Data? Presenters: Brad Wuetherick and Bruno Roy, Dalhousie University Track: Strategic Broadway C Session: The Role of an End of Semester Module Questionnaire in Enhancing Students’ Learning Experience and Supporting Staff Professional Development Presenter: Yao Wu, Xi’an Jiaotong‐Liverpool University Track: Innovation Labs Crystal Ballroom

2:00 PM to 2:50 PM

Session: Implementation of Blue Text Analytics at Temasek Polytechnic Presenter: Samer Jaffar, eXplorance Track: Blue Administrators Broadway A Session: Blue Implementation in a Decentralized Environment: The Case of UNC‐Chapel Hill Presenter: Jason Block and Heather Thompson, University of North Carolina Chapel Hill Track: Strategic Broadway B Session: How to Embrace Your Data and Use it to Inform Learning and Continuous Improvement Presenter: John Miller, Nuventive Track: Technology Broadway C Session: Blue Survey to Advance Institutional Research and Initiatives Presenter: Jason Immekus, University of Louisville Track: Innovation Labs Crystal Ballroom

2:50 PM to 3:10 PM Afternoon Break – 1st drawing for door prizes (3:00 PM) Presenters: Samer Jaffar and Mary Cash, eXplorance Crystal Ballroom

3:10 PM to 4:00 PM

Session: Advanced Integration with LMS and SIS: Improving Student Learning ExperiencePresenters: Margot MacNutt and Georges Geha, eXplorance Track: Blue Administrators Broadway A Session: Contextualizing Student Feedback by Mashing Up Course Evaluations and Learning Interaction Data to Make Student Feedback More Meaningful Presenter: Miles Lincoln, University of California Berkeley Track: Strategic Broadway B Session: Opting In & Changing Dates: Giving Faculty More Control Over Evaluations Presenter: Donna Davis, James Madison University Track: Innovation Labs Broadway C Session: Monitoring Academic Quality: When Data are Challenged and Opportunities are Lost in the Discourse Presenter: Julie Arthur, Southern Cross University Track: Strategic Crystal Ballroom

4:00 PM to 7:30 PM Louisville Mega Cavern (Please be in the lobby at your designated time.)

Thursday, August 3, 2017


7:45 AM to 8:15 AM Breakfast Crystal Ballroom/South Alcove

8:15 AM to 9:15 AM Session: Structured Networking Crystal Ballroom

9:15 AM to 9:45 AM Morning Break – 2nd drawing for door prizes (9:35 AM)Presenters: Samer Jaffar and Mary Cash, eXplorance Crystal Ballroom

9:45 AM to 10:35 AM

Session: Lessons Learned from Implementing Bluepulse in Classrooms with Faculty ChampionsPresenters: Nitin Sharma, eXplorance and Stephanie Romano, Education Affiliates Track: Blue Administrators Broadway A Session: Thinking Beyond the Traditional ‘Instructor Evaluation’ ‐ Using ONE Simplified Survey toEvaluate Instructors Presenter: Nick Ullrich, Galen College of Nursing Track: Strategic Broadway B Session: Response Rate Monitoring: Technical Overview and Implementation Case Study Presenters: Mohammed Sheraidah, eXplorance and Becky Patterson, University of Louisville Track: Technology Broadway C Session: Beyond Numbers: Using Qualitative Results to Drive Change at the Programme Level Presenter: Ineke Kranenburg, Auckland University of Technology Track: Innovation Labs Crystal Ballroom

10:45 AM to 11:35 AM

Session: Unveiling BlueX – The Generalized Survey ToolPresenter: Samer Saab, eXplorance Track: Blue Administrators Broadway A Session: No 360? No Problem: Creating a 360 Evaluation Project by Using Blue’s Evaluation Module Presenter: Larry Williams, Oregon Health & Science University Track: Strategic Broadway B Session: Designing and Improving Competency‐Based Education Programs: The Role of Student, Faculty, and Employer Feedback in Competency‐Based Education Presenter: Dr. Van Davis, Blackboard, Inc. Track: Innovation Labs Broadway C Session: Piloting Blue Online Course Evaluation Solution in a UK Higher Education Institution Presenter: Julie Mulvey, Durham University Track: Strategic Crystal Ballroom

11:45 AM to 12:45 PM Lunch Bluegrass Room


12:45 PM to 1:35 PM

Session: The Latest and Greatest of Bluepulse: UnveiledPresenter: Samer Saab, eXplorance Track: Blue Administrators Broadway A Session: The Journey of Kent State University Presenter: Paul Creed III, Kent State University Track: Strategic Broadway B Session: Factors Contributing to a Rapid and Successful Implementation of Blue at Monash University Presenter: Elizabeth Toy, Monash University Track: Innovation Labs Broadway C Session: Is Over‐surveying Undermining your Efforts? Create a University‐wide Survey Administration Process FTW Presenters: Katie Partin and Katie Bixby, University of Louisville Track: Innovation Labs Crystal Ballroom

1:45 PM to 2:45 PM

Session: Data Preparation Made Easy and Less Painful!Presenter: Krimo Bouaou, eXplorance Track: Blue Administrators Broadway A Panel Discussion: Marketing Outreach to Improve Response Rates Presenters: Karen Seach, Duquesne University, Lisa Votodian, University of Pittsburgh, and Shari

Barrow, University of Louisville Track: Strategic Broadway B Session: The Elephant in the Room: A Reflection of a South African Case Study of Moving Towards an Automated Teaching Evaluation Process Presenter: L. B. Matee, Vaal University of Technology Track: Innovation Labs Broadway C Session: Journey Into “Blue” Online Course Evaluation Presenter: Lillian Marshall, Blinn College Track: Strategic Crystal Ballroom

2:45 PM to 3:15 PM Afternoon Break – 3rd drawing for door prizes (3:05 PM)Presenters: Samer Jaffar and Mary Cash, eXplorance Crystal Ballroom

3:15 PM to 4:05 PM

Session: Lessons Learned and Best Practices to Ensure a Successful and Effective Course Evaluation Cycle Presenter: Rola Omar Al‐Ghoul, eXplorance Track: Blue Administrators Broadway A Session: Using Evaluation Data Strategically: The Evolution of Evaluations and Strategic Decision Making at Galen College of Nursing Presenter: Carissa Shafto, Galen College of Nursing Track: Strategic Broadway B


3:15 PM to 4:05 PM

Session: Implementing Data Integrity Gateway (DIG) Presenters: Patricia White and Mary Lucus, Belmont University Track: Strategic Broadway C Session: Making the Most of Text Analytics – Pushing Ourselves to Improve Text Analytics Offerings in Ways that Matter to University Staff Presenter: Bradley Patten, Auckland University of Technology Track: Innovation Labs Crystal Ballroom

4:15 PM to 5:00 PM

Session: Question Bank 2.0 – How We Made it Easier for Our Users to Submit QuestionsPresenter: Brett McDaniel, University of Kentucky Track: Blue Administrators Broadway A Session: Using Blue to go Beyond: Meeting the Challenges of Developing Business Leaders Presenters: Neil Holman and Kirstin Mahar, Lake Forest Graduate School of Management Track: Strategic Broadway B Session: Removing Obstacles in Using Blue in Your Environments Presenter: Mohammed Sheraidah, eXplorance Track: Blue Administrators Broadway C Session: Implementing Blue in a Large Multi‐School System: Lessons Learned Presenters: Stephanie Romano and Eric Goodman, Education Affiliates Track: Innovation Labs Crystal Ballroom

5:30 PM to 9:00 PM

Please be in the lobby no later than 5:15 PM.Social Activity: Depart for Kentucky Derby Museum/Cocktails/Dinner/Tour Presentation of Best New Presenter and Best New Presentation Presenter: Samer Saab, eXplorance

Friday, August 4, 2017


8:00 AM to 8:30 AM Breakfast Crystal Ballroom/South Alcove

8:30 AM to 9:30 AM

Session: Bluepulse [closed session]Presenter: Samer Saab, eXplorance Track: Blue Administrators Broadway A Session: Blue Tips and Tricks (part I) Presenter: Margot MacNutt, eXplorance Track: Blue Administrators Broadway B Session: Blue 360 ‐ Taking Advantage of Multi‐Rater Feedback Presenter: Jeff Anderson, eXplorance Track: Blue Administrators Broadway C Session: Harnessing Feedback in the Next Generation Digital Learning Ecosystem Presenter: Melissa Loble, Instructure Track: Technology Crystal Ballroom

9:30 AM to 9:45 AM Morning Break – 4th drawing for door prizes (9:35 AM) Presenters: Samer Jaffar and Mary Cash, eXplorance Crystal Ballroom

9:45 AM to 10:35 AM

Session: Data at the Fingertips of Faculty – Unveiling a New Blue Dashboard Experience and RoadMap Presenter: Georges Geha, eXplorance Track: Blue Administrators Broadway A Session: Blue Tips and Tricks (part II) Presenter: Margot MacNutt, eXplorance Track: Blue Administrators Broadway B Session: BlueX [closed session] Presenter: Samer Saab, eXplorance Track: Blue Administrators Broadway C Session: We Heard You, Now What Happens Next? Bluenotes Group Community Update and Customer Satisfaction (CSAT) Survey Results Presenter: Yeona Jang, eXplorance Track: Strategic Crystal Ballroom

10:45 AM to 11:15 AM Closing Ceremony (Interactive Session – Please Bring Your Mobile Device)

Session Descriptions Thought Leadership Roundtable: Response Rates: Drawing Insights from This (not so) Simple Metric Moderators: Gregory Hum, Assistant Director, Teaching Assessment, University of Toronto Kosha Bramesfeld, Data Analyst, University of Toronto Victoria James, Data Coordinator, University of Toronto

“What are your response rates”? “I hear online course evaluations have really low response rates”. Questions and concerns such as these are frequently expressed by those switching from paper to online surveys. Underlying this is a fear that response rates will be “too low”. We aim to shift the discussion from such questions towards a more sophisticated and nuanced understanding of the underlying issues. First, we aim to address the question of “too low”. Namely, how does a response rate relate to the actual interpretability of the data and why? As well, how can we statistically determine these thresholds? And, once determined, how often is the response rate actually “too low”? Second, we aim to re‐examine the calculation of response rates in terms of their levels of aggregation. Specifically, we show how calculating response rates by survey‐student pairings, by course, and by student, can each provide different ways to understand the data both theoretically and practically.

Thought Leadership Roundtable: Course Evaluation Feed Forward: Students as Stakeholders in the Learning Process Through Deep Formative Feedback Practices Moderator: Krisy Mahome, Assistant Director of Bloomington Evaluation Services and Testing, Indiana University Bloomington and Higher Education Administration Ph.D. student with a focus on Formative Feedback

Engaging students in establishing and improving the learning process can seem like a daunting task for faculty when using a traditional faculty‐centered pedagogical approach. For the purpose of improving the level of quality in the learners’ experience, continuous formative feedback is being examined as an avenue for faculty to assess learning‐in‐process via two‐way dialogues with students. In this thought leadership roundtable session, we discuss not only the “why” behind the use of formative feedback but also the faculty voices collected to understand better the needs, benefits, and challenges associated with the use of formative feedback tools like Bluepulse. Join this session to discuss the advantages and difficulties in cultivating this type of collaborative learning environment that involves including all stakeholders in the learning process. The participants will have the opportunity to leave this session with insight on how to utilize formative feedback tools to establish and improve their learning process along with key factors to consider.

Thought Leadership Roundtable: Rethinking Course Evaluation Administration for Competency‐Based Education Courses Moderator: Lee Bewley, Program Director, B.S. OLL‐Healthcare Leadership, Associate Professor, University of Louisville

Competency‐Based Education is an educational approach that focuses on competency‐development and demonstrated mastery of competencies as the prime variables of educational process in a time‐flexible, non‐term format. Students may progress more quickly or more deliberately than normal term courses matched to the needs and goals of each student.

Thought Leadership Roundtable: The Blue Guru: Sharing Your Expertise and Gaining New Insights Moderators: Katie Partin, Assistant Director, Office of Institutional Effectiveness, University of Louisville Katie Bixby, Institutional Effectiveness Analyst II, Office of Institutional Effectiveness, University of Louisville Thomas Dohm, Director, University of Minnesota

Whether you’re new to Blue or an old pro, you have something to offer the collective! Come share your secrets for success and pet peeves, along with your “what I wish I would have known” and “in a perfect [electronic] world” scenarios of what you hope to be able to do with Blue! Be prepared to share in the conversation so we can all learn from each other!

Workshop: Autonomy as Driver for User EngagementInstructors: Margot MacNutt, Training Manager, eXplorance Krimo Bouaou, Manager, Process and Innovation, eXplorance

Learn how to build in Blue an evaluation process that engages your faculty and students by giving them maximum autonomy to control and access their tasks and reports.

Workshop: Automation of Your Feedback ProcessInstructors: Margot MacNutt, Training Manager, eXplorance Krimo Bouaou, Manager, Process and Innovation, eXplorance

Learn how to build a data preparation process that will ensure you have all the data needed to automate your feedback process.

Workshop: Insights from Your Qualitative QuestionsInstructors: Margot MacNutt, Training Manager, eXplorance Krimo Bouaou, Manager, Process and Innovation, eXplorance

Learn how to build in Blue reports that analyze your comments and quantify them along 28 dimensions. Also learn how to interpret these results and evaluate the accuracy and coverage of it.

Workshop: Tips and Tricks Instructors: Margot MacNutt, Training Manager, eXplorance Krimo Bouaou, Manager, Process and Innovation, eXplorance

Learn quick ways and features that will help you capture further the value of Blue.

Wednesday, August 2, 2017

Keynote Address: The Art and Science of Student Success: Using Design Thinking and Data Science to Help Students Learn Well and Finish Strong Speaker: Dr. Mark David Milliron, Cofounder and Chief Learning Officer, Civitas Learning

Using examples from other sectors that bring together the best of design thinking and data science to achieve their diverse missions—from business to athletics to healthcare—we’ll catalyze a conversation about how we leverage this approach in education. We’ll then dive into examples of these insight and action analytics efforts in education, showing how schools, colleges, and universities are innovating with everything from predictive modeling to early‐warning systems to engaging apps for students, faculty, and advisers that help more students learn well and successfully navigate their educational journeys.

Session: Stories from the Past and Vision of the Future – eXplorance CEO UpdatePresenter: Samer Saab, CEO, eXplorance

The last year has been a very eventful one for eXplorance. During this session, we will share with you all pertinent updates since we last met in August of 2016. We will unveil our latest and greatest software and services news. Get ready to learn more about Blue 7, DIG 2, Bluepulse 2, and BlueX. We will also give you a sneak peek of the advancements and

enhancements our hard working team is bringing your way and share what is to come over the next 12 months. As we close the session off, we invite you to dream with us a little by sharing our vision and technology roadmap.

Session: The Blue Cascaded Course Evaluation Model to Make Data More Meaningful by Leveraging Institutional Hierarchy Presenter: Long Hua, Chief Product Officer, eXplorance

Institutions managing a decentralized course evaluation approach are looking for ways to address their need for a more comprehensive configuration of their survey instruments. The aim is to ensure that stakeholders at all levels in an organization have the ability to collect meaningful data, and query and receive results pertinent to their areas of interest. The Blue cascaded course evaluation model provides a unique way to link organizational units (e.g., school of arts, department of computer science, etc.) with their respective list of course evaluation questions. In addition, this capability leads to many time saving ways of managing course evaluation projects and reports.

Session: A Tale of Two Faculties: Course Evaluation Implementation at a Large University, a Study of Contrasts and Lessons Learned Presenters: Victoria James, Data Coordinator, University of Toronto Gregory Hum, Assistant Director, Teaching Assessment, University of Toronto

The University of Toronto is in a phased process of implementing the University of Toronto’s Cascaded Course Evaluation Framework (CCEF). The CCEF aims to represent both centralized and decentralized teaching and learning priorities across various multiple levels of the University. In this session, we share some of the insights gained through two contrasting cases of implementation, one which was immediately successful, and another with ongoing challenges. Lessons learned include how to deeply understand and adapt to local contexts/goals, the importance of community consultation and “buy‐in”, and the need for long‐term sustainable solutions. We will discuss how success is determined from data, and how to bring troubled implementations back on track.

Session: Blue Advanced Relationship Tables (ART) and Other MusingsPresenter: Daniel Glover, Developer II, University of Minnesota

So maybe you have heard about Blue Advanced Relationship Tables (ART). Are you wondering how they can help manage your survey needs? We will demonstrate the approach we have found to be most effective in getting supervisory information whose source of truth is currently distributed across the organization in to Blue. Then we will discuss how we expect to leverage this process to facilitate the new ART and in managing dynamic report viewership access coming to Blue.

Session: Complete the LEM Framework with Embedded Faculty DevelopmentPresenters: Pete Baccile, Business & Relations Officer, Verteres Partners, LLC Kristin Good, Learning and Development Expert, Verteres Partners, LLC Brenda Barnes, Solutions Engineer, Verteres Partners, LLC

Imagine: A faculty member uses all of the best‐in‐class benefits of eXplorance’s Learning Experience Management (LEM) framework. The data indicates an opportunity for potential teaching improvement. Within a few simple clicks, the faculty member is launched into a community of targeted online professional development facilitated by teaching and learning specialists. Through a partnership with eXplorance, Verteres Partners is completing the LEM framework by embedding training and peer networking online. Come explore and network with us!

Session: Leveraging Blue’s Enhanced Data Structure Capabilities for Dynamic Report Access and Teaching Roles Support Presenter: Zelbrey Bedard, VP, Product Management (Blue), eXplorance

When it comes to deans, department chairs, and other academic leaders for whom access to reports is essential, institutions have long struggled to replicate the organizational links between these constituents and the reports originating from multiple course evaluation campaigns. When an academic leader changes roles or a new leader is appointed, a further need arises to propagate changes to report access. In addition, when institutions wish to evaluate teachers and teaching assistants with a separate set of questions for each instructor type, it requires implementing separate processes ‐ one for teachers and another for TAs (or lab assistants, secondary lecturers, etc.). What this means is that students will be asked to complete multiple forms for the same course. For the survey administrator, it also means managing multiple processes and analyses, which adds time and effort. This session focuses on the new data structure capabilities that make it possible to better support these two use cases.

Session: Measuring What We Do: Using Blue to Survey and Improve Our Landscape Presenter: IL Barrow, QEP Specialist for Assessment, University of Louisville

The Delphi Center for Teaching and Learning strives to make data‐driven decisions based on information collected from our faculty, staff, and students. Blue survey is a key solution for how we collect these data. This session will focus on the various ways the Delphi Center utilizes Blue survey to articulate need, satisfaction, and impact of our services and programs. The presenter will also integrate best practices in assessment and evaluation to complement our use of Blue survey.

Session: What Can We Learn about Teaching and Learning from Our Student Evaluation of Teaching Data?Presenters: Brad Wuetherick, Executive Director, Learning and Teaching, Dalhousie University Bruno Roy, SRI Technical Administrator, Dalhousie University

The role of student ratings of instruction as part of the landscape for the evaluation of teaching has become very contested, particularly in recent years as some recent studies have called for us to dismiss student evaluations as biased (for example, due to what has been identified as gender bias against particular faculty). Within the context of some lessons learned from the past six years using Blue for online student evaluations of teaching at Dalhousie University (a Canadian, medical‐doctoral, public research university located in Halifax, Nova Scotia), this session will explore three different projects using student evaluation of teaching data as part of a broader academic analytics framework on campus to explore the teaching and learning environment on campus. After integrating our Blue data set with our institutional data (provided by our institutional student information system), we were able to explore three major questions – 1. What do student evaluations of teaching tell us about student persistence, success, and retention?; 2. Are student evaluations of teaching actually measuring teaching, or are they really more determined by other variables (like gender); and 3. What other ways of analyzing data at the program, department, or faculty level (beyond the standard reporting at the level of individual courses and instructors, and the reporting of departmental and institutional means) might inform our understanding of the teaching and learning environment? In addition to sharing the results of our exploration of these three questions, this session will explore the ways of acting on student evaluation of teaching data, as an institution, to improve the student experience of the teaching and learning environment, including through a facilitated discussion of two cases representing hypothetical data at the institutional and department levels.

Session: The Role of an End of Semester Module Questionnaire in Enhancing Students’ Learning Experience and Supporting Staff Professional Development Presenter: Yao Wu, Centre Administrator, Xi’an Jiaotong‐Liverpool University

This presentation will firstly offer a brief introduction about Xi’an Jiaotong‐Liverpool University (XJTLU), a Sino‐British University in China and its Module Feedback Questionnaire (MQ) practice with Blue. Then it will provide evidence of good practice in relation to how one particular department instigates a process of closing the feedback loop to students. This is achieved by publishing the summaries of the MQ reports on the staff and students communication page on the XJTLU VLE platform. The presentation will also examine the developments of MQ as a fundamental teaching evaluation tool. The distinctive feature about MQ is that it not only provides a comprehensive overview of students’ feedback regarding a module’s objectives, materials, assessments and delivery, but also offers comparative perspectives among different modules and departments on a university scale, which is helpful for addressing both students’ learning needs and staff’s professional development.

Session: Implementation of Blue Text Analytics at Temasek PolytechnicPresenter: Samer Jaffar, EVP Global Customer Loyalty, eXplorance

As attributes, thematic taxonomy and sentiment in teaching and learning are important to help us gain deeper understanding of students’ satisfaction, Temasek Polytechnic (TP), Singapore, has successfully used Blue Text Analytics ( BTA ), a text analytics solution, to mine free‐form textual data from its students’ responses in its Evaluation of Teaching Effectiveness survey. Join us in this eXplorance‐led presentation to find out how TP uses BTA to uncover hidden insights, validate quality of its students’ feedback and obtain new intelligence for further research.

Session: Blue Implementation in a Decentralized Environment: The Case of UNC‐Chapel Hill Presenters: Jason Block, Program Compliance Coordinator, Office of Institutional Research and Assessment, University of North Carolina‐Chapel Hill Heather Thompson, Course Evaluation Coordinator, College of Arts and Sciences, University of North Carolina‐Chapel Hill

One eXplorance executive described the Blue implementation at the University of North Carolina as the most complex implementation the company has done. Administrators from UNC Chapel Hill will use this session to describe the complex nature of course evaluations at UNC‐Chapel Hill and the challenges they faced, and mostly overcame, in the hopes of helping institutions new to Blue, considering Blue, or considering switching their approach to Blue to avoid a difficult implementation process.

Session: How to Embrace Your Data and Use it to Inform Learning and Continuous Improvement Presenter: John Miller, Eastern Regional Executive, Nuventive

This presentation is designed to help you answer the “so what” questions surrounding your data. What did you do with it? What actions will you take? Will the actions drive a budget request? Most importantly, did the results drive improvement? In this session, based on our experiences @ Nuventive, we will talk about ways to provide individuals, departments and leadership with a place to document their efforts, record the actions taken, and report achievement on key performance objectives.

Session: Blue Survey to Advance Institutional Research and InitiativesPresenter: Jason Immekus, Associate Professor, College of Education and Human Development, University of Louisville

This session examines the use of Blue surveys to advance faculty and graduate student research and institutional initiatives. Specifically, we will explore the ways in which faculty and student research can be aligned with university‐wide initiatives to promote institutional decision‐making while making significant contributions to the knowledge base. For didactic purposes, the discussion will be situated in research conducted over the 2016‐17 academic year on doctoral students’ experiences in graduate education and faculty involvement in community‐engaged scholarship. Implications faculty and student development and institutional‐research will be presented.

Session: Advanced Integration with LMS and SIS: Improving Student Learning ExperiencePresenters: Margot MacNutt, Training Manager, eXplorance Georges Geha, VP, Product Management (Blue), eXplorance

eXplorance is developing new SIS and LMS integration tools to improve the student learning experience by streamlining the course evaluation and registration process. Deeper integration with PeopleSoft will give students access to course reports and aid in their course selection decisions, and the grade blocking feature will boost response rates. By leveraging native capabilities in Blackboard, Canvas, Moodle and Brightspace, such as to‐do lists, calendar notices, assignments, and grade pathways, access to course evaluations is front and center.

Session: Contextualizing Student Feedback by Mashing Up Course Evaluations and Learning Interaction Data to Make Student Feedback More Meaningful Presenter: Miles Lincoln, Course Evaluation Service Lead, University of California Berkeley

At the end of each term, we ask students to evaluate their courses and instructors, but what happens next? How can we best equip instructors to interpret this feedback? How might we combine student feedback with other data to create a more comprehensive and actionable record of the students’ learning experience? In this presentation, we will share our efforts at UC Berkeley to contextualize course evaluation reports to improve their value for the instructor. We will discuss our learning and progress in using learning interaction data and other readily‐available sources to create enriched course evaluation reports. Finally, building on our experience, we will outline next steps to scale this effort, with the goal of empowering instructors to interpret their feedback in context and react in a more informed manner.

Session: Opting In & Changing Dates: Giving Faculty More Control Over EvaluationsPresenter: Donna Davis, Course Evaluation Administrator, James Madison University

Could something as simple as a date change be the tool necessary for instructor support of the Blue evaluations process? Would Unit Heads allow it? Learn how evaluation timing could be a near‐perfect solution to Blue implementation concerns.

Session: Monitoring Academic Quality: When Data are Challenged and Opportunities are Lost in the DiscoursePresenter: Julie Arthur, Director, Planning Quality and Review, Southern Cross University

Changes in the external landscape of higher education in Australia present increasing challenges to keep internal quality assurance processes aligned with public comparisons of teaching quality. The updated Higher Education Standards Framework (Threshold Standards) (2015) placed greater emphasis on the quality assurance of teaching and learning in Australian universities. The Quality Indicators for Learning and Teaching (QILT) website publicly represents nationally collected data on students and graduates. At Southern Cross University, a pilot project has been implemented to improve

the oversight of academic quality at the unit (subject) level using data on student success and feedback. This paper reports on the challenges and opportunities currently being experienced.

Thursday, August 3, 2017

Session: Lessons Learned from Implementing Bluepulse in Classrooms with Faculty Champions Presenters: Nitin Sharma, Product Specialist, eXplorance Stephanie Romano, National Dean of Hybrid and Blended Learning, Education Affiliates

In this session, we will present lessons learned from having implemented Bluepulse in various institutions. The two most common areas of focus in implementing Bluepulse expressed by institutions are “How to get students onto the Bluepulse platform and engaged” and “What feedback questions to ask” during a term, as learning occurs. We will share success stories from institutions that are implementing Bluepulse, along with critical success factors, tips, and best practices. Building a culture of feedback as more departments and faculty in an institution begin to use the platform and identifying “faculty champions” as early as possible emerge as a key critical success factor for teaching and learning with Bluepulse. In this presentation, we will talk about how the role of faculty champions and their interactions with each other can impact the successful implementation of Bluepulse in integrating formative feedback practices to teaching and learning.

Session: Thinking Beyond the Traditional ‘Instructor Evaluation’ ‐ Using ONE Simplified Survey to Evaluate Instructors Presenter: Nick Ullrich, Data Support Specialist, Galen College of Nursing

Is your intuition struggling with ideas on how to easily evaluate instructors? Galen College of Nursing will present an alternative way for supervisors to evaluate their instructors using a simplified survey. By using one open‐sourced link generated through Blue, we converted an existing paper and pencil evaluation tool for supervisors to evaluate their faculty members more easily and effectively. During this session, we will preview an evaluation, show an example of the results, and discuss the benefits and limitations this method offers our College.

Session: Response Rate Monitoring: Technical Overview and Implementation Case Study Presenters: Mohammed Sheraidah, Director, Product Knowledge & Engagement, eXplorance Becky Patterson, Executive Director, Office of Institutional Research and Planning, University of Louisville

This session will provide a technical overview of the response rate monitoring feature in Blue. The University of Louisville will share how response rate monitoring data informed changes in business practices to increase response rates.

Session: Beyond Numbers: Using Qualitative Results to Drive Change at the Programme Level Presenter: Ineke Kranenburg, Academic Director, Auckland University of Technology

Surveys provide important information on student perceptions of course and teacher quality in many Universities. As a focus on the importance of the student voice expands, identifying the best ways to listen to, represent, and advocate for that voice will become more important. While quantitative items are easy to summarise, the depth of insight into/understanding of the student experience that they offer is often limited. Qualitative results typically offer the exact opposite: a rich source of detailed information that quickly becomes difficult to summarise as comments expand from 10s, to 100s, to 1000s. Where the volume of comments becomes overwhelming, text analytic software can be used to quickly summarise student comments in ways that engage staff and represent student voices fairly. Using relevant case studies, we will focus on how Blue Text Analytics can be used to summarise student experiences, test ideas, identify areas of strength, and provide a description of areas identified for improvement in the context of an integrated University wide quality assurance process.

Session: Unveiling BlueX – The Generalized Survey ToolPresenter: Samer Saab, CEO, eXplorance

After much reflection and following a thorough beta program with many of your colleagues, eXplorance brings to you the latest in surveying technology. Get ready for an enhanced survey management experience, one that is referred to as: simple, smart, and delightful. During this session, we will cover the basics of survey creation, notifications, panel management, and the respondent experience.

Session: No 360? No Problem: Creating a 360 Evaluation Project by Using Blue’s Evaluation Module Presenter: Larry Williams, Educational Technology Specialist, Oregon Health & Science University

Let’s say you’ve been tasked with creating a 360 evaluation in Blue. It’s a great tool, but what if you don’t have access to that module? And what if you have to create the evaluation project in a short time frame? This was the case for the evaluation manager at Oregon Health and Sciences University, as he created a peer, self, and facilitator evaluation for the Clinical Nutrition department. In this presentation, he’ll address how to build a 360 evaluation with only using a few CSV datasource files, in Blue’s evaluation module.

Session: Designing and Improving Competency‐Based Education Programs: The Role of Student, Faculty, and Employer Feedback in Competency‐Based Education Presenter: Dr. Van Davis, Associate Vice President Higher Education Policy and Research, Blackboard, Inc.

As competency‐based education shifts the educational focus to student mastery of clearly defined competencies, feedback from students, faculty, and employers will become more critical for program design and review. How can such feedback be elicited and how should it impact both the design and continuous improvement process will be the topic of this presentation.

Session: Piloting Blue Online Course Evaluation Solution in a UK Higher Education Institution Presenter: Julie Mulvey, Learning Technologist, Durham University

This talk will discuss the year‐long pilot that was run within Durham's Business School that ran alongside an MEQ (Module Evaluation Questionnaires) Task & Finish Group looking at evaluation questions and evaluation questionnaires. The pilot of Blue at Durham was completed in late May this year with the decision to implement Blue online course evaluation solution across the institution for the next academic year. She will share the pilot experience, highlighting the issues that arose and how Blue overcame obstacles to provide a streamlined solution. She will also present a list of questions from a UK perspective on implementing Blue for discussion.

Session: The Latest and Greatest of Bluepulse: UnveiledPresenter: Samer Saab, CEO, eXplorance

Join us for an interactive session where you will get a full preview of the next generation release of Bluepulse that brings your way an expanded feature set, a larger scope transcending beyond just courses to services, teams and people; and an exciting roadmap to pave the path ahead.

Session: The Journey of Kent State University Presenter: Paul Creed III, Technology Project Director, Kent State University

Paul Creed will tell the story of Kent State University and their journey through the world of online course evaluations. From the days of paper through a home‐grown system, to the implementation of Blue, to “the pilot”, to the current state of surveys, to the future, Paul will present the story of The Good, The Bad, and the Ugly.

Session: Factors Contributing to a Rapid and Successful Implementation of Blue at Monash University Presenter: Elizabeth Toy, Learning and Teaching Quality Manager, Monash University

Monash University is the largest university in Australia with over 70,000 students. In 2015, it commenced a project to replace the student survey platform in order to achieve a range of institutional objectives. Through the application of Extended Enterprise Architecture, the project re‐conceptualized the relationship between the survey administration and the utilization of the survey results to improve learning and teaching. It also reshaped the procurement process so that implementation was rapid, with the university knowing exactly what it needed and how it would be applied once it had selected Blue as its survey platform. Monash became an autonomous user of Blue very quickly and the project was delivered on time; successfully delivering the Semester 2 2016 main survey round with no procedural or technical issues – a survey that reached 61,000 students, with 199,000 individual surveys, and included over 3,000 academic staff for individual feedback. The project identified a range of wider issues in relation to data quality that has resulted in further work at Monash, as well as enabling the development of new applications and presentations of the data to enhance the utilisation of student feedback to improve learning and teaching, spaces/facilities, IT and other dimensions of the student experience.

Session: Is Over‐surveying Undermining your Efforts? Create a University‐wide Survey Administration Process FTWPresenters: Katie Partin, Assistant Director, Office of Institutional Effectiveness, University of Louisville Katie Bixby, Institutional Effectiveness Analyst II, Office of Institutional Effectiveness, University of Louisville

As an institution, we must pay greater attention to a) the rationale for and quality design of surveys, b) the coordination of survey administration through one central office to maximize response rates, and c) the meaningful use of results to inform and guide continuous program improvement. Therefore, UofL has facilitated a university‐wide survey administration process. During this session, attendees will learn about how the process was created and the required components for a successful implementation.

Session: Data Preparation Made Easy and Less Painful!Presenter: Krimo Bouaou, Manager, Process and Innovation, eXplorance

When it comes time to launch course evaluations, one of the biggest challenges institutions face is the lack of accuracy of their SIS data often leading to laborious course evaluation experiences for teachers, students, and staff. So, if you’re tired of laboring for weeks to get SIS data ready chasing after people to confirm that data is accurate, be sure to attend this session. In this session, you will learn how an automated solution like DIG can improve data quality, while significantly saving time and resources. With DIG, data clean‐up can be distributed to a wide group of users minimizing the burden on any one individual. In this manner, users are assigned to work on data they are familiar with. DIG can decrease course evaluation preparation time from seven to just two weeks, giving you more time to spend on analyzing data ‐ leading to better insights and learning experience for all.

Panel Discussion: Marketing Outreach to Improve Response RatesPresenters: Karen Seach, Application Specialist, Duquesne University Lisa Votodian, Assistant to the Director, University of Pittsburgh Shari Barrow, Institutional Research Analyst II, University of Louisville

What marketing and communication efforts does your institution utilize to get the word out to students and faculty regarding the importance (or value) of completing course evaluations? Student participation is vital to the success of the survey process. Instructors and administrators would like to see 100% response rates. Join Duquesne University, University of Pittsburgh, and University of Louisville to discuss marketing and advertising campaigns and share what your institution does to promote evaluations to ultimately increase response rates.

Session: The Elephant in the Room: A Reflection of a South African Case Study of Moving Towards an Automated Teaching Evaluation Process Presenter: L. B. Matee, Manager, Vaal University of Technology

Vaal University of Technology (VUT) South Africa embarked on implementing an online teaching evaluation system starting March 2017 named VQUILT, an acronym for VUT Quality in Teaching and Learning. The objective of VQUILT is to solicit constructive feedback to Lecturers and allow Lecturers to use this feedback as the basis for improvement. In this paper, we share experiences of this implementation process, challenges for students and academics, adoption of the systems by faculties and the university. We explore the impact of such implementation on the strategic direction of the university and the impact of it on the performance of lecturers, in a developing African context.

Session: Journey Into “Blue” Online Course EvaluationPresenter: Lillian Marshall, Data Analyst, Blinn College

Blinn College recently made a decision to migrating its existing online course evaluation platform to Blue. In this session, I will briefly talk about the key factors considered and the steps taken by the College. I will also talk about what the College plans to achieve through this migration, as well as the implementation strategy and progress being made for discussion.

Session: Lessons Learned and Best Practices to Ensure a Successful and Effective Course Evaluation Cycle Presenter: Rola Omar Al‐Ghoul, Manager – Project Success, eXplorance

Over the past 14 years, eXplorance has had the privilege of working alongside a large number of higher education institutions and helped in the planning, development, and launching of thousands of course evaluations of varying types and scopes. Over the years, we have accumulated (and continue to accumulate) a vast pool of best practices that we follow in order to deliver the most effective course evaluation cycles term after term. In this presentation, we will focus on highlighting some key lessons learned and best practices in different phases of the course evaluation implementation cycle that will add value to your overall process.

Session: Using Evaluation Data Strategically: The Evolution of Evaluations and Strategic Decision Making at Galen College of Nursing Presenter: Carissa Shafto, Director of Institutional Effectiveness, Galen College of Nursing

Galen College of Nursing adopted Blue in the spring of 2015. At that time, we had just transitioned from paper to electronic surveys in Class Climate, a platform that provided limited options for data analysis and trending. Since the implementation of Blue, data from course evaluations and other surveys have been available for use in academic program evaluation, faculty performance evaluations, and the evaluation of student support services. This presentation will walk

through specific examples of the use of survey/evaluation data in decision‐making at Galen, including the role of other technological tools adopted during that time frame.

Session: Implementing Data Integrity Gateway (DIG) Presenters: Patricia White, Director of Business Intelligence and Data Management, Belmont University Mary Lucus, Director of Institutional Research, Belmont University

In the fall of 2016, Belmont University went live with Blue course evaluations and the Data Integrity Gateway (DIG) tool university‐wide. In this session we will present the story of our DIG implementation, what worked and what didn’t. We will cover setting up the data feed, creating users, creating the DIG project and data quality rules. We will talk about our overall process and how we train our users to validate and clean data for a smooth course evaluation process.

Session: Making the Most of Text Analytics – Pushing Ourselves to Improve Text Analytics Offerings in Ways that Matter to University Staff Presenter: Bradley Patten, Survey & Evaluation Analyst, Auckland University of Technology

Running with eXplorance’s ‘improvement at heart’ and ‘continuous improvement’ values, AUT are looking at ways to improve the way that we report (and report on) student comments. We will outline some high level plans on how we might supplement Blue’s current text analysis offerings in order to meet staff requests/expectations. This includes allowing staff to explore student comments by theme, using text analysis to edit/restrict access to certain content, and (ideally) create a system that will learn/develop overtime.

Session: Question Bank 2.0 – How We Made it Easier for Our Users to Submit QuestionsPresenter: Brett McDaniel, IT Manager, University of Kentucky

This presentation will briefly review how the University of Kentucky managed the Question Bank process last year. Then you will get to hear about and see the new system we have developed to allow users (college contacts and department contacts) to review, add, edit, and delete questions in the Question Bank through a GUI. Finally, you will see the Question Bank and Question Bank Mapping CSV files that the system generates for us to upload into Blue.

Session: Using Blue to go Beyond: Meeting the Challenges of Developing Business Leaders Presenters: Neil L. Holman, Ph.D., Dean, Educational Programs & Development, Lake Forest Graduate School of Management Kirstin Mahar, Senior Manager, Corporate Learning Solutions, Lake Forest Graduate School of Management

At Lake Forest, we design, develop and deliver customized training for corporate clients, informed by the leadership principles we teach in our master’s degree programs. We use Blue not only to evaluate how well we did after a training event; we also frequently use Blue prior to the event, to help us diagnose needs; evaluate the climate; test prior knowledge; allow participants to receive 360‐degree views of how they are perceived; and set benchmarks. Following the event, we use Blue to assess evidence of learning, and particularly the application of learning, or “business impact.” Our aim is often to go to Level 3, and measure behavior change. In this presentation, we’ll discuss how you might use Blue to go “beyond” at your institution.

Session: Removing Obstacles in Using Blue in Your EnvironmentsPresenter: Mohammed Sheraidah, Director, Product Knowledge & Engagement, eXplorance

How do I automate course evaluations using Blue in my institution? How do I automate surveys using Blue in my

institution? How do I get the reports from Blue that my key stakeholders need? And are there any other questions on using Blue? In this session, an eXplorance Blue eXpert will work directly with you to hear your obstacles in your current implementation of Blue, and will provide you with practical recommendations on how to move forward and get back on track to success with Blue.

Session: Implementing Blue in a Large Multi‐School System: Lessons Learned Presenters: Stephanie Romano, National Dean of Hybrid and Blended Learning, Education Affiliates Eric Goodman, Vice President of Education, Education Affiliates

This session will explore a large school system’s initial Blue survey implementation that highlights the complexities of a multi‐institution, multi‐campus higher education environment where survey questions, survey timing, and survey delivery method vary. In addition, we will examine change management strategies, data considerations, and lessons learned from this system‐wide implementation.

Friday, August 4, 2017

Session: Bluepulse [closed session] Presenter: Samer Saab, CEO, eXplorance

This session is by invitation only.

Session: Blue Tips and Tricks (part I) Presenter: Margot MacNutt, Training Manager, eXplorance


Session: Blue 360 ‐ Taking Advantage of Multi‐Rater FeedbackPresenter: Jeff Anderson, Director, Sales Engagement, eXplorance

In this session, we will present how best to use Blue's multi‐rater capabilities in both 360 and course evaluation processes, including definition building, workflows, and reporting. We will also share examples of how existing institutions are using these capabilities to enrich their reports with multi‐rater feedback.

Session: Harnessing Feedback in the Next Generation Digital Learning EcosystemPresenter: Melissa Loble, Vice President, Platform, Instructure & Instructor, University of California, Irvine

Many institutions are striving to implement the Next Generation Digital Learning Ecosystem (NGDLE) in order to have the greatest positive impact on student success. Critical to NGDLE effectiveness is the role of feedback, particularly student feedback, in the overall assessment process. Together we will explore how to encourage, gather, and utilize student feedback throughout a broader learning ecosystem in which an LMS plays a foundational, platform role. Come prepared to share you own ideas and experiences in utilizing student feedback across multiple sources and solutions.

Session: Blue Tips and Tricks (part II) Presenter: Margot MacNutt, Training Manager, eXplorance


Session: BlueX [closed session] Presenter: Samer Saab, CEO, eXplorance

This session is by invitation only.

Session: Data at the Fingertips of Faculty – Unveiling a New Blue Dashboard Experience and RoadMap Presenter: Georges Geha, VP, Product Management (Blue), eXplorance

Faculty often express the need for additional analysis of their results that are not included in their standard end‐of‐term course evaluation reports. The more insights they can receive, the more likely they are to bring about the necessary changes for improvement. It would be too onerous for a Blue administrator alone to accommodate all these individual faculty requests, which is why some institutions have asked eXplorance for the tools that can facilitate a self‐service approach to analysis by instructors. We are introducing a new way for faculty to consume their end‐of‐course evaluation results. The benefits include: faculty can see how the different genders view them, how students in different academic years are rating course difficulty, and many more. The Blue Feedback Dashboard becomes a self‐service area for faculty to gain the insights they need to improve.

Session: We Heard You, Now What Happens Next? Bluenotes Group Community Update and Customer Satisfaction (CSAT) Survey Results Presenter: Yeona Jang, EVP Strategic Initiatives Office, eXplorance; Adjunct Professor, McGill University

Interested in learning what’s planned for the Bluenotes Group Community over the next year? Join this session for an update on community activities which will help build a stronger network. Wondering what happened to your feedback? We will also share the results from the last two client satisfaction (CSAT) surveys and present improvement efforts that we are making in response to your input.

Congratulations to the Bluenotes Americas 2016 Award Winners

Best New Presenters

Best Presentation

Question Bank: Why We are Using it and HowBrett McDaniel, Manager, Planning and Assessment Technology, University of Kentucky Abstract: The University of Kentucky moved away from Question Personalization to the Question Bank for Fall 2015. Come hear why we are using the Question Bank instead of Question Personalization. We will explain our rollout and how we have achieved buy‐in on our campus. We will also give some strategies and tips for implementing the Question Bank.

The Many Uses of Blue Survey Katie Partin, Assistant Director for Institutional Effectiveness, University of Louisville Katie Bixby, Institutional Effectiveness Analyst II, University of Louisville Abstract: The University of Louisville (UofL) uses Blue surveys as the institution’s enterprise survey application. UofL uses Blue surveys to gather feedback in several situations, including but not limited to: faculty/staff exit surveys, student perception surveys, and following special events. This presentation will highlight the many uses of Blue surveys by over 50 project managers on campus. An open dialogue will be highly encouraged so that session participants can discuss how their institution has used Blue surveys.

Pictured left to right: Katie Partin, Katie Bixby, Samer Saab

Pictured left to right: Bob Goldstein, Samer Saab, Brett McDaniel

A Special “Thank You” to the Bluenotes Americas 2017 Volunteers

Donna Davis Blue Applications Administrator

Larry Williams Education Technology Specialist

Brett McDaniel IT Manager

Molly Hall Administrative Specialist

Thomas Dohm Director

Daniel Glover Developer II

Names of attendees certified in Blue (as of July 14) are highlighted in blue font. Attendees Institution/Organization Name Title Email

Aarhus University Line Krog Heltoft Project Coordinator [email protected]

Aarhus University Morten Palsgaard Application Manager

Aarhus University Mustapha El‐Ahmad Data Manager [email protected]

Alabama A&M University Dianne Bozemann Interim Assessment Coordinator [email protected]


Bradley Patten Survey and Evaluation Analyst [email protected]


Ineke Kranenburg Academic Director [email protected]

Belmont University Mary Lucus Director [email protected]

Belmont University Patricia White Director of Business Intelligence and Data Management

[email protected]

Blackboard Van Davis Associate Vice President, Policy and Research

Blinn College Lillian Marshall Data Analyst [email protected]

Brandeis University Mitchell Albury Senior Institutional Research Analyst [email protected]

British Columbia Institute of Technology (BCIT) Nancy Morello Supervisor [email protected]

California Institute of Integral Studies

Peter Jones Director of Institutional Effectiveness [email protected]

Central Piedmont Community College

Diane Frazier Special Projects Coordinator [email protected]

Civitas Learning Dr. Mark David Milliron

Co‐Founder & Chief Learning Officer

Clark State Community College Cindra Phillips Director, Center for Teaching and Learning [email protected]

College of the Ouachitas Chris Robbins Title III Project Manager

College of the Ouachitas Jessica Allen Distance Education Support Specialist

College of the Ouachitas Pat Simms Vice President for Academic Affairs

College of the Ouachitas Tricia Baar Dean of Learning

College of William and Mary Carol MacVaugh Planning Analyst [email protected]

College of William and Mary Tricia Whisnant Director of Academic Affairs [email protected]

Cuesta College Aimee LaRue Analyst, Institutional Research [email protected]

Dalhousie University Brad Wuetherick Executive Director, Learning and Teaching [email protected]

Dalhousie University Bruno Roy SRI Technical Administrator

Duquesne University Karen Seach Application Specialist [email protected]

Durham University Julie Mulvey Learning Technologist [email protected]

Eastern Kentucky University Earl Angel Data Manager

Edison State Community College Amy Crow Distance Learning

Education Affiliates Eric Goodman VP of Education

Education Affiliates Stephanie Romano National Dean of Hybrid and Blended Learning [email protected]

eXplorance Chanel Sutherland Marketing Content Specialist [email protected]

eXplorance Charles Assaf Chief Revenue Officer [email protected]

eXplorance Georges Geha VP Product Manager [email protected]

eXplorance Ian Haugh VP Business Development [email protected]

eXplorance Jeff Anderson Director, Sales Engagement [email protected]

eXplorance Krimo Bouaou Manager, Process and Innovation [email protected]

eXplorance Long Hua Chief Product Officer [email protected]

eXplorance Margot MacNutt Training Manager [email protected]

eXplorance Mary Cash Customer Relationship Manager [email protected]


eXplorance Michael Cochrane Customer Support Engineer [email protected]

eXplorance Michael Weisman Customer Relationship Manager [email protected]

eXplorance Miltiadis Vadrahanis VP Global Services [email protected]

eXplorance Mohammed Sheraidah

Director, Product Knowledge & Engagement [email protected]

eXplorance Nitin Sharma Product Specialist [email protected]

eXplorance Rola Al‐Ghoul Manager, Project Success [email protected]

eXplorance Samer Jaffar EVP Global Customer Loyalty [email protected]

eXplorance Samer Saab CEO [email protected]

eXplorance Sara Alkadri Software Support Engineer [email protected]

eXplorance Shadi Khader VP Customer Success [email protected]

eXplorance Stacey Martini Events & Special Projects Coordinator [email protected]

eXplorance Tim Brennan General Manager, Asia‐Pacific [email protected]

eXplorance Yeona Jang EVP Strategic Initiatives Office, eXplorance; Adjunct Professor, McGill University

[email protected]

eXplorance Zelbrey Bedard VP, Product Management [email protected]

Florida Gateway College Rebecca van Hoek Director

Fox Valley Technical College Amy Schiedermayer Blackboard Support Technician

Fox Valley Technical College Nate Lison Blackboard Support Technician [email protected]

Galen College of Nursing Carissa Shafto Director of Institutional Effectiveness [email protected]

Galen College of Nursing Christina Bollinger Data Manager [email protected]

Galen College of Nursing Nick Ullrich Data Support Specialist [email protected]

Galen College of Nursing Sarah Berg Senior Research Analyst

Harvard Business School Eran Spira Systems Administrator

Humber College Keshia Ellis Student Support Advisor

Humber College Mark Ihnat Director, eLearning [email protected]

Humber College Naveed Aqeel Digital Learning Specialist

Indiana University Bloomington Amanda Abrell Online Systems Education Manager

Indiana University Bloomington Krisy Mahome Assistant Director [email protected]

Indiana University Bloomington Shane Kearney Software/Technology Specialist

Indiana University Kokomo Cherie Dodd Instructional Strategies Specialist [email protected]

Indiana University Kokomo Julie Saam Assistant Vice Chancellor for Academic Affairs [email protected]

Indiana University‐Purdue University Columbus

Barb Dobbs Information Management Analyst [email protected]

Indiana University‐Purdue University Indianapolis

Howard Mzumara Director of Testing Center [email protected]


Jon McMahel Director of Technology [email protected]


Kate Forrest Coordinator [email protected]


Kent Stoelting Assistant Director [email protected]

Instructure Melissa Loble Vice President, Platform & Partnerships [email protected]

James Madison University Donna Davis Blue Applications Administrator [email protected]

Kent State University Paul Creed Technology Project Director


Kirstin Mahar Sr. Manager



Neil Holman Dean, Educational Programs [email protected]

Loyola Marymount University Michelle Castellanos Associate Director of Survey Research [email protected]

Marian University Elizabeth Osika Director, Center for Teaching and Learning

Mercy College Piyaporn Nawarat Associate Director of Data and Analysis

Monash University Elizabeth Toy Learning and Teaching Quality Manager [email protected]

Mount Royal University Crystal Koch Special Project Administrator [email protected]

New Brunswick Community College

Denise Doucet Institutional Research Analyst

Nuventive John Miller Eastern Regional Executive [email protected]

Oregon Health and Science University

Lawrence Williams Education Technology Specialist [email protected]

Pikes Peak Community College Jennifer Petross Administrative Assistant III [email protected]

Princeton University Ashley Boost Assistant Registrar for Course Information

Princeton University Jonatha LeBouef Associate Registrar

Princeton University Polly Griffin Registrar

Purdue University Northwest Emily Hixon Professor of Education [email protected]

Rasmussen College Stacy Spencer Institutional Research & Assessment Specialist

Saint Louis University Kathleen Thatcher University Assessment Coordinator [email protected]

Saint Louis University Megan Buckley Manager, Enterprise Applications

Saint Louis University Steven Sanchez Assistant Provost

Saint Mary's University of Minnesota

Mark Barber Associate Vice President for Academic Affairs

Sheridan College Dean Langan Analyst, Institutional Research [email protected]

Simon Fraser University Juanitta Dare Course Evaluations Systems Specialist [email protected]

Southern Cross University Julie Arthur Director [email protected]

Stonehill College Patty Mead Coordinator [email protected]

Texas A&M University ‐ Corpus Christi

Christine Shupala Associate Vice President [email protected]

The College of New Rochelle Diane Lama Asst to the AVP for Program Development & Assessment

The Ohio State University Cindy Davis Assistant Registrar – Testing Services [email protected]

The University of Alabama Jason Phillips Web Resources Integration Analyst Senior

The University of Auckland Lizzie Li Evaluations Coordinator [email protected]

Tufts University Candace Girard Education Technology Specialist

Tufts University Janet Hill Director, Educational Technology Systems and Services

University of California Anne Jones Chief of Staff to the Provost and Executive Vice President

University of California ‐ Berkeley Miles Lincoln Course Evaluation Service Lead [email protected]

University of Connecticut Alexander Walsh Research Analyst [email protected]

University of Kentucky Brett McDaniel IT Manager [email protected]

University of Kentucky Kelita Mayorga IS Technical Support Specialist II [email protected]

University of Louisville Arnold Hook Assistant Director [email protected]

University of Louisville Becky Patterson Executive Director [email protected]

University of Louisville Chandler Hora Analyst [email protected]

University of Louisville Dan Bye Assistant to the Vice Provost

University of Louisville Erika Ruffin Senior IR Analyst [email protected]


University of Louisville Il Barrow QEP Specialist for Assessment [email protected]

University of Louisville Jason Immekus Associate Professor [email protected]

University of Louisville Katie Bixby Research Analyst II [email protected]

University of Louisville Katie Partin Assistant Director [email protected]

University of Louisville Leslee Martin Director of Undergraduate Medical Education

[email protected]

University of Louisville Lisa Horn Assistant Director [email protected]

University of Louisville Manish Sharma Director of Assessment, Educator Preparation & Human Development

[email protected]

University of Louisville Molly Hall Administrative Specialist [email protected]

University of Louisville Paul Klein Evaluation Specialist

University of Louisville Shari Barrow Institutional Research Analyst II [email protected]

University of Louisville Stephen Fry Institutional Research Analyst II [email protected]

University of Maryland – Baltimore County

Jared Fincke Business Applications Specialist

University of Maryland ‐ College Park

Eowyn Ellison Manager [email protected]

University of Minnesota Daniel Glover Developer II [email protected]

University of Minnesota Thomas Dohm Director [email protected]

University of New Haven Gregory Sanzone Operations and Research Analyst [email protected]

University of North Carolina ‐ Chapel Hill

Heather Thompson Course Evaluation Coordinator [email protected]


Jason Block Program Compliance Coordinator

University of North Dakota Jodi Steiner Data Coordinator [email protected]

University of Pennsylvania – Wharton

Pamela Knight IT Director [email protected]

University of Pittsburgh Lisa Votodian Assistant to the Director [email protected]

University of Pittsburgh Nancy Reilly Director, OMET [email protected]

University of Toronto Gregory Hum Assistant Director, Teaching Assessment [email protected]

University of Toronto Kosha Bramesfeld

University of Toronto Victoria James Data Coordinator

University of Western Ontario Krista Dziuba Course Evaluation Coordinator [email protected]

Ursinus College Ruth Sprague User Support Specialist [email protected]

Vaal University of Technology Corneli Van Der Walt Manager First Year Experience

Vaal University of Technology Lesego Bruce Matee Manager: Evaluation Unit [email protected]

Verteres Partners, LLC Brenda Barnes Solutions Engineer [email protected]

Verteres Partners, LLC Kristin Good Learning and Development Expert [email protected]

Verteres Partners, LLC Pete Baccile Business & Relations Officer [email protected]

Villanova University Julie Helverson Systems Administrator

Washington State University Joshua Yeidel Information Systems Manager [email protected]

Washington State University Peggy Collins Associate Director

Washington University in St. Louis

John Murphy Application Design Lead [email protected]

Washington University in St. Louis

Laura Setchfield Registrar Services Specialist [email protected]

Washtenaw Community College

Gloria Eccleston Director for Online Learning Services


Wesleyan University Lisa Sacks Assistant Director [email protected]

Wesleyan University Michael Whitcomb Director of Institutional Research

West Virginia University Amanda Rittenhouse IT Support Specialist [email protected]

West Virginia University Kathryn Fletcher IT Support Specialist

West Virginia University Robert Hastings Assistant Director [email protected]

Xi'an Jiaotong Liverpool University

Yao Wu Centre Administrator [email protected]

Institution Directory

Institution Years Using Blue

Survey 360 Evaluation Bluepulse Blue Text Analytics

evalUT Data

Integrity Gateway

Aarhus University 3

Alabama A&M University 1

Auckland University of Technology 3

Belmont University 1

Blinn College <1

Brandeis University 2

British Columbia Institute of Technology (BCIT)

1

California Institute of Integral Studies

<1

Central Piedmont Community College

3

Clark State Community College <1

College of New Rochelle 1

College of the Ouachitas

College of William and Mary 5

Cuesta College 4

Dalhousie University 5

Duquesne University 4

Durham University <1

Eastern Kentucky University 2

Edison State Community College 5

Education Affiliates <1

Florida Gateway College 5

Fox Valley Technical College 4

Galen College of Nursing 3

Harvard Business School 1

Humber College 1

Indiana University Bloomington 4

Indiana University Kokomo <1

Indiana University‐Purdue University Columbus

2


3

James Madison University 6

Kent State University 4


1

Loyola Marymount University 5

Marian University <1

Mercy College 2

Monash University 1




evalUT Data

Integrity Gateway

Mount Royal University 4

New Brunswick Community College <1

Oregon Health Sciences University 4

Pikes Peak Community College 3

Princeton University

Purdue University Northwest <1

Rasmussen College 4

Saint Louis University 2

Saint Mary's University of Minnesota

Sheridan College 3

Simon Fraser University 3

Southern Cross University 3

Stonehill College 5

Texas A&M University ‐ Corpus Christi

<1

The Ohio State University

The University of Auckland 1

Tufts University 2

University of Alabama 9

University of California ‐ Berkeley 4

University of California system

University of Connecticut 5

University of Kentucky 4

University of Louisville 10

University of Maryland – Baltimore County

2

University of Maryland ‐ College Park 2

University of Minnesota 4

University of New Haven 1


<1

University of North Dakota 1

University of Pennsylvania – Wharton

1

University of Pittsburgh 2

University of Toronto 6

University of Western Ontario 1

Ursinus College 10




evalUT Data

Integrity Gateway

Vaal University of Technology <1

Villanova University 1

Washington State University 3

Washington University in St. Louis 1

Washtenaw Community College 2

Wesleyan University 2

West Virginia University 2

Xi'an Jiaotong Liverpool University 3

Future Bluenotes Americas Conferences: Save the Dates!

2018

Blue eXpert Certification Camp:

Sunday, August 5 through Tuesday, August 7

Bluenotes Americas conference: Tuesday, August 7 through Friday, August 10

2019

Blue eXpert Certification Camp: Sunday, August 4 through Tuesday, August 6

Bluenotes Americas Conference:

Tuesday, August 6 through Friday, August 9

2020

Blue eXpert Certification Camp:

Sunday, August 2 through Tuesday, August 4

Bluenotes Americas Conference: Tuesday, August 4 through Friday, August 7

Notes

Thank you for attending Bluenotes Americas 2017

st century: using data to - bluenotes group€¦ · century: using data to respond to opportunities...

Documents