Abdulla Janahi Marie Vachovsky Tina Vachovsky

Thomas Zhao CS194H

Final Report Abdulla Janahi, Marie Vachovsky, Tina Vachovsky, Thomas Zhao

Problem Through our needfinding, we discovered that students often have trouble taking action to try and understand confusing material while in a large lecture classroom. The professors need a way to gauge the level of understanding of all, rather than just some, students in large lectures. Currently, students find it unreasonable to ask a professor in a large class to adjust their pace and perceive a stigma associated with asking questions. Professors, on the other hand, have no simple way to gauge the level of confusion in the class and need to filter relevant questions.

Solution

Value proposition & Mission enGauge creates a virtual communication line: students can give feedback on the pace of the class and ask questions, and professors can address that feedback, all in real-time.

Solution Overview enGauge provides a dual interface that consists of an iOS app for students and a Web app for professors. The solution consists of three tasks of simple, medium and complex difficulty:

● Simple: gauging the speed of the class. ● Medium: asking good questions. ● Complex: seeing which topics confused students.

Preview of the enGauge student interface

Preview of the enGauge professor interface

Tasks The student app allows for completion of the simple and medium tasks, and the professor app allows for completion of all three tasks. They are described in further detail below:

Student 1. Simple: Student can give feedback to professor based on the speed of class. Students

often don’t want to ask questions because they become so confused they don’t know what to ask. In these cases, they just want the professor to slow down. Similarly, they may also not want to ask questions when they fully understand the content: they just want the professor to speed up. This is a simple task of tapping the hare button (speed up), or the tortoise button (slow down).

2. Medium: Students can semi-anonymously ask relevant questions. Students are anxious about asking a question in class because they are afraid of being judged by their classmates. Furthermore, if one of the students asks a question that is not relevant for the whole class, most students will not learn, taking time away from relevant questions. To make the class relevant for the majority of the class, the professor can use answer the most upvoted question. This requires reading other students’ questions.

Professor 1. Simple: Professor can gauge the understanding level of class. Currently the professors

don’t have a good way to know the overall engagement of the class during the times when students don’t ask questions and when they just want the professor to speed up or slow down. This can help the professor speed up or slow down based on the class’s needs, and thus make lecture as useful for the students as possible.

2. Medium: Professor can answer relevant questions. The professor can see which questions the most students want answered and answer those questions, keeping the class engaged.

3. Complex: Professor can analyze the class’s understanding in the different topics taught. The professor analyzes how engaged students were during specific topics in the class and what questions were asked at which part of the lecture. They can use this data to better address confusing topics in future classes or address potential questions preemptively in future lectures.

Task Flows Here one can see exactly how the different task flows take place for the student and professor interface:

Student

SIMPLE: Gauging speed of class

MEDIUM: Asking good questions

Professor

SIMPLE: Gauging speed of class

MEDIUM: Asking good questions

COMPLEX: See which topics confused students

Design Evolution Although our design changed significantly throughout the two quarters, the original design still consisted of the core elements of slow-down and speed-up buttons, a meter, questions on both sides, and a report on the professor side. We mostly changed the way in which these ideas were implemented. Thus, we can look at how the UI changed for each task on the student and professor side:

Student

Initial sketches of student prototype

SIMPLE: Gauging speed of class CS 147

CS 194h

Major Changes in Simple Task for Student interface

The general application stayed very similar from the beginning to the end in terms of the functionality it provided, since user testing revealed that students liked the big, easy-to-hit buttons and easy-to-understand calls to action. However, the design greatly evolved. Initially there was only a hare and tortoise but students did not understand what that really meant (did a hare mean “go faster” or “too fast”?). Therefore, we added labels to the images and cleaned up the UI to make it more accommodating. Subtle haptic feedback was also added (for supported devices) to reinforce interactions so that students would not have to glance down at their phone again to confirm that they made a selection.

MEDIUM: Asking good questions CS 147

CS 194h

Major Changes in Medium Task for Student Interface

There were not many major changes in the functionality of the questions screen. However, the interface was cleaned up in order for it to be more clear for the user and encourage the user to ask better questions. We also added “top” and “new” sort options which allow users to switch in between the most upvoted questions and the latest questions. We also changed the voting metaphor from “upvotes” to “likes,” as we found in user testing that some students attempted to upvote multiple times or did not understand the upvote icon, as there was no corresponding downvote icon, as is common with upvote-downvote systems. Also, unlike the previous system, new questions appear automatically, rather than requiring the user to pull to refresh.

New Features

In the final prototype we added the ability to create accounts, which is necessary to track which classes students are in and for the professor to see which students are asking which questions. Additionally, we added the ability for students to control the speed feedback using their device’s volume buttons when they are on their lock screen or when the screen was off. The aim was for students to still be able to use the application without being distracted by other things on the phone.

Professor

Initial sketches of professor prototype

SIMPLE: Gauging speed of class + MEDIUM: Asking good questions

Major Changes

Here, we made major changes to the speed bar at the top of the screen. We quickly realized through user testing that people had a very difficult time reading the bar, for two reasons: 1) the bar changed colors too gradually for users to notice, and red-green color deficient users could not notice it at all, and 2) the small black triangle located beneath the bar was extremely difficult to see against the dark gray background. Therefore, we completely redesigned the interface using distinct bricks rather a meter. This was a far clearer design with a physical change that circumvented any problems that could arise due to differences in color perception. The questions part was cleaned up but stayed largely the same.

COMPLEX: Seeing which topics confused students

Major Changes

During user testing, we found that instructors universally appreciated the information provided. However, the way in which it was displayed was rarely liked by the users and many felt that it was inconvenient or hard to understand at a glance. After several iterations we ended up using a line graph for the speed votes and a bar chart for the questions with the ability to hide either

graph if the user so desired. The charts were also modified to update in real time, so the instructor could use them as a more detailed way of gauging student feedback.

New Features Login Screen

Classes and Sessions

We added new features only in our latest prototype. On the professor side, these included the ability to create accounts, classes and sessions (lectures/slide decks) within those classes. Although these were not important for our initial tasks, they are necessary in order to scale in the application among multiple users. They also allow users to personalize the experience for themselves.

Evaluation Technique Out of all our evaluation techniques, the most useful were the ones in which we had long discussions with expert users on the product. For example, one expert user was a professor who not only had an extensive teaching background but also a strong understanding of visual and interaction design, and was also a business school student who had previously worked as both a high school teacher as well as in the ed-tech industry. We had long discussions with these individuals that encompassed both the minutia of certain design details on our product as well as our overall design goals. The insight we gained helped us solidify our product.

Final Interface

Final UI Design

Student

Design and Functionality We tried to make sure that the design of the student app would be more fun and accessible than that of the professor app. The goal is to encourage the student to engage with the application when they want to. However, we also wanted to keep the application simple enough that the student would not get distracted during class. The student logs into the application with their Google account. They can then add classes to their class list by scanning a QR code that the professor can present whenever they start a session. The student can then simply tap speed up or slow down and have their vote be reflected in the professor interface. Additionally, the student can tap “Ask Questions” to be brought to the Questions screen, where they can like questions or ask new ones.

Professor

Design and Functionality For the professor app, we wanted the design to seem professional and accessible. The goal of simplicity had to remain, since much like the students, the professor should also not be distracted from engaging with the material and the class. We wanted to the interface to work in the background and flag the professor when something important is happening, such as many students wanting the class to slow down or one question receiving a significant amount of likes. As for the graph, we wanted it to be simple to read at a glance and to allow the professor to focus on the information most relevant to them. In terms of operation, the professor registers an account and signs into the application. They then join an existing class or create a new class. Within that class they create a session by uploading a PDF file of their slide deck and giving a name to the session. After this is done, the professor enters the session and a new tab solely with the slides opens, which they can display to the rest of the class on a separate monitor while using the enGauge “presenter view”-style display. The professor can also open up a QR code and share it on the large screen so that students can join the class, if they have not done so already. The class can then begin. If students ask to speed up or slow down, this will be displayed as bars on the top of the screen. If the students ask questions, these will appear on the side. The professor can then address the questions during the class and click on the “X” to remove a question from the screen. At the end of the class they can click on the graph icon and look at the feedback of the students and alternate between the votes and the questions graphs.

What is unimplemented The main feature we initially planned to implement but ultimately didn’t involved anonymity: showing student names next to questions that would pop up in the professor interface, to handle potential cases of students abusing the question-asking functionality. We omitted this feature because we needed to prioritize optimizing the functionality and design of the core interactions and therefore did not have time to add that specific feature. We did not need to use any Wizard of Oz techniques in our final implementation. Moving forward we want to include android support for the applications. Additionally, we want to implement an in class quiz functionality with which the professor can prompt multiple choice questions on the student’s phone and have them answer questions in real time. Finally, we would like to implement better analytics and moderation tools for professors that encompass multiple sessions.

Tools Used The iOS app was built using Swift with the native UIKit framework as one of our team members had a very strong grasp of iOS development and was therefore able to design a well-made application. The web app was initially built in React but we switched to Angular.JS as we realized that we had a far better overall understanding of Angular as a team. Finally, we built our backend using Firebase to allow us to easily setup a database and adapt it as we were rapidly iterating and testing our product.

Download Directions The student application can be downloaded from the App Store—just search for “enGauge”. Note that, due to App Store restrictions, the version from the App Store does not contain the volume button-related functionality. However, the version downloadable from our website does. As for the professor application, it can be obtained from the enGauge website. It can be found on the Prototypes page.

Making it Real

Team As a team we are in a perfect position to start, develop and expand enGauge. Not only are we building a product whose target user-base we are very familiar with, since we are currently students and have also taught small classes, but we also each bring unique skillsets to the table that will allow enGauge to be successful. All four of us are Computer Science students; two of us have a concentration in HCI and two of us in AI. Thus, not only can we improve the design and think of new product features to implement but we can also leverage AI to provide actionables to the professor based on the feedback data that we will collect over time. From a development standpoint, one of us is skilled in iOS development while the others have good web development experience, which allows us to continue developing this dual-sided application. Additionally, one member of our team has significant experience in Venture Capital, raising funding, marketing, product and business development. Hence, not only can we build a great product but we can also take it to market and execute on its expansion.

Business Model

With regards to the business model, we decided as a team that we did not want to charge students for the service as we wanted to make sure that if they did not want to pay for something then they should not. Therefore, we considered charging schools for the service. However, once we spoke to individuals with domain knowledge in selling to schools, we realized the difficulty we would have in doing such a thing. The sales cycle is complex and schools are not willing to spend money on many applications such as these, especially when they do not need the product directly as an institution. Thus, we decided that we would provide the product for free to schools and students that would be interested. However, we would use this as a user acquisition strategy and make sure that we would develop a large and strong user base within our application. We would aim to continue growing and expanding without initially monetizing the service. Once our user base is strong enough and we have iterated enough on the application that we have achieved product-market fit and started growing organically, we would begin monetizing the service. We would do this by offering other verticals on top of the core features of the both the student and teacher application. Hence, no one would be forced to pay the initial service that enGauge start off as, but if they want to obtain other services that will be built on top of the platform, then they would have to spend money. Certain initial verticals we are planning on including on the student side are connecting students with tutors, providing online tutors, providing digital textbooks and recruiting services for internships and jobs. Similarly, with more needfinding we would development similar verticals for the Professors. This is a sustainable model as we are applying a variable pricing model. Each individual will pay for as many services as they can and want relative to their spending power. This way, we can capture the entire spending opportunity from each user while not alienating non-spending users. Currently there is a large initial customer base for our application: in the USA alone there are 20.5 million students enrolled in higher education. However, we will eventually expand to K-12, which has around 50 million students in US. Our eventual goal is to capture the entire ed-tech market. According to leading education technology-focused conference, EdTechXGlobal, ed-tech spend is projected to reach $252bn by 2020. Our product will be highly impactful and will ultimately optimize global education, which is further discussed below. Initially, however, enGauge will be impactful in that it will bring mobile phones, which students use all time, and integrate it into the classroom environment. Rather than shunning smartphones in the classroom and having them be a source of distraction, instructors will be able to leverage them to improve the learning experience.

Summary Human behaviour has evolved with time, and, with the advent of projectors and now interactive whiteboards, classrooms have adapted to this change. However, there has been no significant

change in the classroom model in the past 10 years, with the explosive growth in mobile technology. enGauge’s key innovation is that it is bringing classes into the present by taking a behavior that is now ubiquitous among the young generation—using mobile phones—and leveraging it to improve the learning experience. Indeed, mobile phones have become an extension of the youth’s identity rather than an external piece of technology. While teachers are currently trying to push back against this inevitable behaviour, they should instead embrace and leverage it. With enGauge, they can achieve just that. enGauge can also be used for the flipped classroom experience, where students can use the application when watching the lecture prior to class and the professor can know what topics to focus on when the students are physically present. Eventually, there will also be an entire suite of other verticals students can use on enGauge which are part of the monetization strategy. But this is just the beginning of enGauge’s impact on the world. enGauge will be the go-to application for learning in the next several decades. Eventually, as classrooms move increasingly online and teaching becomes more personalized through AI, enGauge can become the bridge between the teaching entity and the student. Through the enGauge application the teaching experience will become tailored specifically for the student. The impact of this has no boundaries all around the world and can help students with lesser means obtain a private tutor-like experience. Thus, enGauge will optimize learning worldwide.