final report for cutgroup #17 - ventra mobile app

CUTGroup 17 Ventra App

Remote Test then In-person test on July 15 & 16, 2015

Introduction For our seventeenth Civic User Testing session, we tested the new Ventra mobile transit app. The Ventra app allows you to manage your Ventra account, buy mobile tickets for use on Metra, get notifications about your account, and other features.

Target Group:

On June 28, we sent out an email to all of our 880 CUTGroup participants. We wanted to know if they would be available for a remote test within the timeframe of July 7 15, and then an in-person test either on July 15 or 16. When segmenting our testers we were interested in:

A mix of testers with different types of mobile devices (Android/iPhone)

Testers who did or didnt have a Ventra card

Testers who add value in different circumstances

A mix of riders on the CTA bus, CTA trains, Metra, or Pace

Testers who commute on a regular basis

Test Type:

Remote test component where testers would download the beta version of the Ventra app and

use it in their normal day-to-day routine and answer brief questions about their experiences

In-person test component where testers would view different features on their own device and

complete tasks

In addition, we also did some testing during the in-person test component on test devices so

testers could purchase tickets and create new accounts

All testers were paired with a proctor, who asked them questions and took down their

responses and actions

Goals of Test:

Here is what we heard from Tony Coppoletta, Manager, External Electronic Communications of the

Chicago Transit Authority, about what the Ventra app team wanted to learn from this CUTGroup test:

The CTA has an incredibly varied set of use cases and we look forward to how all our riders can

benefit from this app's feature set. In addition to some public outreach to invite people to test

our app before public release, we want to try and the diverse and experienced testers the

CUTGroup offers would be an excellent addition to our real-world user testing of the app. The

focus is to trap bugs, ensure transactions happen as expected, good usability and a solid UX

overall.

1

http://www.transitchicago.com/assets/1/media_relations_documents/Ventra_App_Feature_Onesheet_-_Optimized_-_final.pdfhttp://us5.campaign-archive1.com/?u=085247dea37361c266002462c&id=3a3c6efb9b&e=%5bUNIQID%5d



Part I Responses

27 testers completed Part 1 of the CUTGroup test, which was a remote test where testers used the

Ventra app in their normal day-to-day routine.

When using the app for the first time which of these actions did you do?

Logged in to an existing account 88% (24)

Created a new account 4% (1)

Registered a Ventra Card 8% (2)

Purchased a Ventra Card 0%

Other: 0%

Have you added CTA transit value or passes to your account using the Ventra app?

Yes 30% (8)

No 70% (19)

Have you purchased Metra tickets?

Yes 19% (5)

No 81% (22)

If you have made a purchase, please tell us about that experience.

Here are some comments from testers who did not make a purchase during the remote test:

ChgoKC says, I tried to add value to the account, but it asked for my Ventra password, which

I entered (it's 4 digits), and it said the password had to be 8 digits. I tried my password for my

bank account, but that was incorrect. So not sure what password it's looking for.

Mischievous Metra maverick has not made a purchase and is still waiting on function to

upload existing Metra month pass into app. I purchase monthly Metra passes through a Conexis

pre-tax commuter benefit program.

Gr8fl4CTA said, I haven't made a purchase because I have a monthly pass. I do not like that

you can only add $5 to the card. There should be an option to either fill out the amount-

especially since most rides are $2.25 or $2.50. People shouldn't be forced to put $5 on their

card if they don't need all of that.

Here are some responses from testers who made a purchase during the remote test:

Brown line rider / occasional UP-N Metra rider says, There was a little trouble with the

Metra ticket purchase: my original Ventra password was too short. I attempted to change my

password in the app, but was not successful. (Unbeknownst to me at the time, there are further

restrictions on passwords beyond their minimum eight-character length.) I then logged into the

2



website and changed my password there, where the other password requirements (a numeral, a

special character, etc) were made explicit. Once that was done, I was able to buy a Metra ticket

through the app.

CTA/Metra Commuter/Evil Genius says, Purchasing in general is easy. Multiple times, I've

purchased a ticket while walking to the train. I love the ability to pay with my Ventra auto-

reloading balance.

Frequent CTA / Occasional Metra says, Purchasing the Metra tickets was very easy - I only

purchased one-way tickets so far using the app, but it seemed very straightforward in choosing

which stops and identifying the ticket type I wanted. The conductor that checked my mobile

ticket on UP-N was educated on the system and he had no trouble accepting the ticket; that

was my only concern about using the Ventra app to buy a Metra ticket.

Systems engineer CTA rider says, I added a 30 day pass to an existing transit account and

it went very smoothly. I'm also REALLY, REALLY impressed with how split tenders for

payments were implemented. It's really intuitive and I really like that feature.

What are some features that you have used in the app? Check all that apply.

Review account history 81% (22)

Used Trip Tools 78% (21)

Looked up Ventra Value 67% (18)

Add Transit Value 22% (6)

Buy Metra Tickets 15% (4)

Add Autoload 8% (2)

Add CTA Passes 4% (1)

Here are some other features testers tried to use:

Tried to remove a Ventra card that wasnt needed on the account, but the feature was not

implemented yet

Looked up transit history

Removed old cards

Changed password

Tell us about your overall experience using the Ventra app.

3 testers mentioned that sometimes the app was slow to load. Additionally, 3 testers either expected or

wanted Near Field Communication (NFC) capabilities. Otherwise, testers had a wide range of responses

when describing their overall experience using the app. Here are some improvements we heard:

Show password

Be able to transfer value from one card to another

Improve the to back button so it does not kick the user out of the app

Route planning in Trip Tools

Blind Transit rider says, I spent a bit of time trying to make my active reduced fare card for

people with disabilities. I do not find the app or the website compatible. All the customer

service agents are able to see my reduced fare permit/ventra card, but I cannot see my active

3



DRAFT

card. I can only see my closed paratransit Ventra card. It is a closed account because I never

activated it. I am trying to correct that issue, but in the meantime I have been using the card that

is not visible to me in either the app, or the website. I was able to use the trip tools and found

that to be extremely helpful and cover all the transit options I use.

Metra commuter says, Overall it's pretty nice. I think it will definitely make commuting a lot

easier. Especially with metra. If you're late and aren't able to buy a ticket in the station, you can

always buy it in the app and avoid the insane fees. I like it a lot.

CTA Rider says, It's a good, reliable app. One thing I noticed is that when you add value it

appears instantly, versus the online site, where it usually doesn't update until you log off and log

back in again, so that's a plus. I also like that Bus Tracker is built into the app, which saves time

from having to search it on the internet.

How appealing do you find the visual design of the app?

5 Very appealing 15% (4)

4 Appealing 52% (14)

3 Neutral 33% (9)

2 Not very appealing 0%

1 Not at all appealing 0%

Tell us about what you like or do not like about the visual design of the app. How would you improve it?

These responses are from testers who were neutral about the visual design of the app:

Daily CTA Rider MW says, I like it. Its functional. Its not eye candy but then, it doesnt

have to be.

Blind Transit rider says, There are some buttons or links that do not have text labels, or alt

text labels. This is important for accessibility.

CTA rider KB says, Things are too close together in many places.

These are responses from testers who thought the visual design of the app is appealing, but had some

suggestions for improving the design:

Frequent CTA / Occasional Metra mentioned, I have two Ventra cards registered to my

account, and there are two semicircles that indicate two cards below the card name. This is a

problematic design for an iPhone because shaded dots usually indicate something that a user can

swipe on the screen, but in the Ventra app the user is expected to tap on the arrow on the far

right to switch cards. I know the correct way to switch between them and I'm still doing it

wrong.

CTA rider AC would change the color of the cursor from white to black or another darker

color. CTA Rider AC says It is really annoying that I cant see where the cursor is.

4



Occasional CTA rider says, The alerts in the Trip tools are a very light font and not very

easy to read.

Here are a couple of responses from testers who thought the visual design is very appealing:

Gr8fl4CTA says, Overall, I like the look and ease of using the website. I think that I would

make the categories of Transit value, CTA & Pace passes and Metra Mobile Tickets more

bold or make the grey heading a little darker. The delineation between the categories needs to

be clearer.

CTA Rider OP, says, It's recognizable with the light blue of the website and it has a clean,

streamlined interface which makes it easy to use.

How easy has it been to use the Ventra app?

5 Very Easy 33% (9)

4 Easy 48% (13)

3 Neutral (not easy, not difficult) 11% (3)

2 Difficult 4% (1)

1 Very Difficult 4% (1)

How would you improve this app to make it easier to use?

During the remote test, testers gave a lot of suggestions for improvements to the Ventra app. There

was not an overwhelming request for one feature over another and, therefore, here is a list of

improvements we heard from our testers (the number indicates the number of times a suggestion was

mentioned):

2 Back button needs to be consistent with users expectations

2 Add map features

2 NFC capabilities

1 Favorite stops should be shown first in Trip Tools

1 Change requirements to reduce typing password multiple times within app

1 Make icons more intuitive

1 Hide Metra features if not a Metra rider

1 Spanish option for bilingual speakers

1 Add the ability to verify funds with bus drivers if card is lost

Part II Responses

26 testers came to the in-person sessions (14 came on Day 1, while 12 came on Day 2). Two of these

testers did not complete the remote questionnaire, but still provided great feedback during the in-

person section.

How often did you use the Ventra app while you have had it?

Multiple times a day 19% (5)

Once a day 23% (6)

5



A few times since I have had it 54% (14)

Once for testing purposes 0%

Other: Unable to login at home 4% (1)

How easy was it to make a CTA ticket or pass purchase using the Ventra app?

5 Very Easy 19% (5)

4 Easy 8% (2)

3 Neutral 4% (1)

2 Difficult 0%

1 Very Difficult 0%

Not applicable 69% (18)

How easy was it to purchase a Metra ticket using the Ventra app?

5 Very Easy 11% (3)

4 Easy 11% (3)

3 Neutral 0%

2 Difficult 0%

1 Very Difficult 4% (1)


If you purchased a Metra ticket, did you use it on the train?

Yes 15% (4)

No 8% (2)


How was the experience using Metra tickets on the train?

4 out of 26 testers purchased a Metra ticket and used it on a train during the remote portion of the

test. Here is what these testers said about using the Metra ticket on the train:

Frequent CTA / Occasional Metra says, Very cool. I was the first one this conductor saw

and he was well-trained and knew how to activate it and use it.

SR thought it was excellent! The conductor was excited since it was the first time he got to see

it and then had her tap the ticket and then also wanted to see the QR code.

Daily CTA Rider (MW) says, "It was really cool. I wasn't sure what it'd be like and didn't

want to activate at first, wasn't sure if should until saw conductor." Daily CTA Rider (MW)

learned that he can activate when on board. He mentioned that there wasn't a round trip

option, but liked that he was able to buy one A-E and E-A.

6



CTA/Metra Commuter/Evil Genius said that the conductors werent trained when he rode

it on July 11. They were not sure about QR scanners.

Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it.

Testers, in general, had a good response to the My Ventra Cards page, but there were a lot of

suggestions made to improve this page:

A few testers had trouble loading the Transit History page, and it generally, took multiple tries

to get it to load. 6 testers mentioned liking the Transit History page when reviewing this page.

Frequent CTA Rider showed us on their LG phone that when the keyboard was up, they

were unable to scroll on the page and could not see where they were typing.

Metra rider88 wants to be able to scan their credit card to add the payment information.

Frequent CTA / Occasional Metra thought that if autoload has already been set up for a

card, the option should not be there. I didnt know what it meant since I already have it set up

from online.

Route 66 Book Stalker expected to see their frequently used Ventra card first, instead of the

one they never use.

Testers had questions about why Autoload was listed twice, and one tester, Mischievous

Metra Maverick mentioned that you do not get a warning on the autoload page and wants a

pop-up explaining exactly what you are doing.

Occasional CTA Rider wanted to change the names of the card in the app.

ChgoKC noted that the back button exits the app from the My Ventra Cards page rather

than going back to the menu.

Metra rider88 never registered their card, but did it on the Ventra app because it was right there!

How would you explain what the following features mean to you. Think about how these values are different from one another:

"Add Transit Value"

This was very explanatory to all testers, who understood immediately that this would add value to their

Ventra cards.

"Add Pass"

6 testers were not sure what Add Pass meant at first, and two of these testers specifically had to click

Autoload to better understand the difference between pass and autoload. Daily CTA Rider thought

passes should be an option under Add Transit Value.

"Add Autoload"

6 testers were not sure exactly what the Add Autoload feature would do, and there was still some

confusion between adding autoload and adding passes.

7



What are some websites or apps that you currently use to check on transit times?

Here are the websites or apps that testers mentioned (number indicates the number of times an answer

was mentioned):

8 Google Maps

6 CTA Bus Tracker

5 Transit Chicago

5 Transit Stop app

3 RTA Trip Planner

2 City Mapper

1 Goroo

1 Metra Rail Time Tracker

1 Boothscoot

1 Embark CTA

1 iTrans Metra

1 iTrans CTA

1 Chicago Transit Tracker Lite

1 Buster

1 CTA Train tracker

Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience.

Here are improvements or suggestions testers made when searching for transit information near them:

3 testers mentioned that they were more likely to use other trip tools because it does not map

out routes. 2 additional testers said that they were expecting a wayfinding feature since the

name of the feature was called Trip Tools.

14 testers did not use the Locate Me feature when searching for transit information near

them. Some of this was due to the fact that some testers were aware of the route they would

need to take, but 6 testers did not see the Locate Me icon. Overall, 8 testers thought the

icons (especially the Locate Me and Favorites icons) could be more intuitive.

4 testers mentioned the inconsistency or lack of the Back Button, and the experience that the

Back button would sometimes kick the tester out of the app.

Here are some other comments from testers:

Systems engineer CTA rider would use Google Maps to see the scheduled routes and then

use the app to determine when the next available time really is. Systems engineer CTA rider did

not use the Locate me feature and assumed that stops were sorted by distance. In addition,

this tester showed that the second line overlaps with direction information.

CTA Savant had some questions about the stop names especially with similarities such as

Michigan & South Water or Michigan/South Water. CTA Savant wants to clarify different stops

and then provide them on the map to see if they actually located her on the map.

8



DRAFT

CTA Rider says, Term is very misleading, it is actually a bus and train tracker. Trip Tools

make me think of real time trip planner Step by step directions were expected based on the

naming convention.

How did you search?

Searched by bus/train route 54% (14)

Used "Locate Me" feature 46% (12)

Searched by stop number 0%

Went to favorites 0%

Now search for another transit route near your home. How was this experience?

7 out of the 26 testers were able to find a transit route near their home very easily and quickly and did

not come across any troubles during their experience. Here is a list of issues that the remainder of the

testers faced when finding a transit route near their home (the number indicates the number of times a

tester mentioned an issue):

7 Back button/had to start over

6 Looking for route planning/tracking that is similar to other apps (such as Google maps)

5 Stop order/organization did not make sense or could be better

3 Unfamiliar or reverse routes were difficult

Back Button

Here are some experiences testers faced with the back button during this portion of the test:

Metra rider88 briefly looked at Trip Tools during the remote test, but notices and likes the

bus routes. Metra rider88 mentioned that there was no back button on the train route options

page, and had to go back to the main menu.

CTA rider (KB) says, itd be nice if back button had a larger hot area because it did not feel

responsive enough.

Systems engineer CTA rider liked searching for stop IDs, but did not like that he could not

use the Android back button when using Trip Tools.

Stop Order Organization

Here are some experiences testers faced with the organization of the bus stops:

CTA rider (AC) scrolled a lot to find the correct stop, and then later used A-Z sort. This

tester was confused about the order of how stops were listed. CTA Rider (AC) prefers to have

stops listed alphabetically, but thinks that order by station might be better for users that are

from out of town.

9



Metra rider88 also had to scroll a lot to find the bus route, and did not see Sort A-Z. Tester

refers to the bus routes by the name vs the number, but it is sorted by bus number.

Route Tracking/Planning

Testers shared with us by using the phrase Trip Tools they were expecting more route planning

features similar to using Google Maps. For example, one tester specifically called out wanting to see a

map during this part of the test. Testers thought this feature would help them choose bus and train

routes to get to locations, not only tell when the next train or bus is coming. Changing the name of this

feature might assist with peoples expectations of the tool.

How did you search?

Searched by bus/train route 92% (24)

Used "Locate Me" feature 4% (1)

Searched by stop number 4% (1)

Went to favorites 0%

How easy was it for you to find the correct stops you were looking for?

5 Very Easy 35% (9)

4 Easy 31% (8)

3 Neutral 23% (6)

2 Difficult 11% (3)

1 Very Difficult 0%

How would you improve this experience?

Here are some changes that testers recommended for the Trip Tools page (the number indicates the

number of times it was mentioned):

6 Be able to search by name of stop/bus (ex: testers did not know bus number, but cross streets might be better)

4 Add a map

3 Improve the sorting of busses/stops

3 Icons could make more sense (Locate me, Favorites,)

2 Improve back button

1 Make it easier to determine direction of bus

1 Make the cursor black for text boxes to make it easier to see

1 Make the alerts more noticeable

1 Less steps to determine transit times

How does this feature compare to other trip websites and apps that you have used in the past?

9 out of 26 testers found the Trip Planning feature to be similar to or on par with other websites or

apps that they use. Here are some responses from testers:

10



Brown line rider / occasional UP-N Metra rider says that this is Similar to what youd get

on the CTA website only this app looks more this decade.

CTA rider (KB) says, It's good, has about the same functionality that many other apps have,

and the alerts here are easier to find because they are shown more prominently.

9 out of 26 testers thought the Trip Planning features on Ventra were better than other websites or

apps they currently use. Here are some comments:

CTA rider (AC) says, "I trusted it more because it was affiliated with Ventra."

Daily CTA Rider (MW) says, "Actually easier than my other apps! And I think that may be

because of simplicity and consistency on the screen and not use of a lot of different colors."

6 out of 26 testers thought this feature was worse than other tools that they use. Here are their

comments:

Frequent CTA / Occasional Metra says, This is stop driven. I think in terms of routes. The

other apps think in terms of route and not stops. Other apps locate you on a map and tell you

what is around. Helps you find the bus and route you need to take.

Riding the 20 every day says, It's similar, but the lack of map function is a problem.

2 testers did not provide enough of a thorough response to this question to be added to any of these

groups.

How would you register a new Ventra card if you wanted to add a card? Where would you find this?

17 out of 26 testers found the plus (+) sign quickly and understood that it meant adding a new Ventra

card. 4 additional testers found it confusing, but eventually found it. Here are some responses from

testers who had trouble or thought it was confusing:

TA thought it was confusing, but went to My Ventra Cards, but mentioned that a new one

wouldnt be my Ventra Card, right? Also, you need to remember all of your account

information to set this up.

RIDER was having a difficult time finding how to enter a second Ventra card, and accidentally

closed the app several times using the hard back button.

CTA Rider and Metra rider88 thought that adding a text description would be better.

Did you create a new account during the remote test? Yes 38% (10)

No 62% (16)

11



Create new account section On day 1 of testing, we decided we wanted testers to create a new account to see how that experience worked. We decided to remove the second on day 2 since we got a lot of great feedback from day 1 and did not think it was necessary on the second testing date.

We asked testers how easy it was for them to create an account if they created one during the remote

test OR if they created one on the test device on day 1 of testing.

Please create a new account on the test device. Tell us about your experience as you go through it.

Only 9 testers created a new account on the test device. 3 out of the 9 testers missed the password

instructions and had to go back and re-do the password.

CTA rider (KB) noted that the keyboard does not have a next field skip button.

SR pointed out a couple of issues such as it being nice to indicate the formatting of the phone

number. In addition the error for not entering the city was Please add a(n) address locality

which could have been explained better.

How easy was it for you to create a new account?

19 testers responded:

5 Very Easy 26% (5)

4 Easy 47% (9)

3 Neutral 5% (1)

2 Difficult 21% (4)

1 Very Difficult 0%


While the majority of testers had an easy or very easy time creating a new account, here are some

suggestions from testers on how to improve the experience:

2 Make password rules more prominent

2 Fewer questions

2 Add arrows to move to the next field

1 Show password as typing

1 Alert when account is already created

1 Wanted to add in zip code first and have it auto-populate

1 Bring you to the error if you miss a field

CTA Savant thought it was strange that the state field only had 10 states listed and they were

not in alpha order.

Daily CTA Rider and CTA Savant had trouble with the password because when the

password/confirm password field is highlighted the characters are a different size so it is difficult

to tell if the passwords are the same length.

12



How often do you purchase Metra tickets?

Every day 0%

2 - 3 times a week 4% (1)

Once a week 0%

Once or twice a month 31% (8)

Once every six months 19% (5)

Once every year 27% (7)

I have never purchased a Metra ticket before 19% (5)

Anything else to add?

Most testers did not have much to add, except that CTA Savant and Metra rider88 both used to be

heavy Metra riders, but now live in the city and do not use it anymore. In addition, Frequent CTA /

Occasional Metra buys 10 rides at one time and that usually lasts them a month.

Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it.

Here are the top things that testers liked about the process of purchasing a Metra ticket.

7 Split payment

3 Using transit value as payment

2 Billing info saved

2 Stops are in order of location on the line.

More information about testers experiences:

SR would like to see a when selecting a ticket that indicates why Metra Monthly isnt available

during the part of the month when its not sold.

CTA/Metra Commuter/Evil Genius likes that you can add items to the card once you are

in it, but does not like the fact that the app cleared the cart if you go back and change the ticket

type.

Riding the 20 every day tried to buy the card using the test device, he forgot to confirm

password and an error message popped up but did not indicate what caused the error. Other

than that, Riding the 20 every day thought it was easy, organized, logical

Occasional CTA Rider was not sure when they were supposed to activate the ticket.

Occasional CTA Rider wants to say that ticket should be activated when on the train.

How easy was it for you to purchase a Metra ticket?

5 Very Easy 69% (18)

4 Easy 23% (6)

3 Neutral 8% (2)

2 Difficult 0%

13



1 Very Difficult 0%


Testers thought that purchasing a Metra ticket was pretty easy to do. 3 testers mentioned that it might

be difficult if you were not sure where to go. Here are some other suggestions from testers, although

most testers did not have anything to add:

SR says, it would be nice if screen could scroll if the keyboard is in the way of entering a

password.

Route 66 Book Stalker wants to be able to search by station.

CTA rider (KB) would like a round-trip option.

CTA/Metra Commuter/Evil Genius wants the cart to be automatically cleared if you

change your ticket type.

Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page?

12 testers mentioned specifically that they liked this Metra ticket page. Here are some suggestions from

other testers:

CTA rider (AC) was confused about the time on the top of the ticket because it showed the

current time, not the valid until time. In addition, CTA rider (AC) thought the design was a little

much and was concerned about the battery life.

Brown line rider / occasional UP-N Metra rider was unclear how to activate and when to

activate the tickets you bought. Can there be a hint screen or help screen? He wouldnt be

confident using these tickets.

CTA Savant didnt know where her second ticket was at first. When opening up the QR code,

she commented that the Metra logo is over the QR code and was not sure if that affected the

ticket.

Wireframe Section On day 2 of testing, the Ventra team wanted to ask a couple of questions about a wireframe to see if

testers understood when the next train was coming. Therefore, we asked 12 testers to look at a

screenshot to tell us if the train was coming in 17 minutes or at 5:46 PM.

Please take a look at this screen, and tell us when the next train is.

At 5:46 PM 50% (6)

In 17 min 42% (5)

Other: Sort of Both 8% (1)

14



Do you have any other comments or insight about this page?

Here are a couple of comments from testers who said the next train is coming at 5:46 PM:

Riding the 20 every day says, It's off that it says both the time and the minutes on the right.

Instead of the minutes on the right, maybe noting how early or late they are?

Cta my way thinks it should say Delayed if it coming later than the regularly scheduled time.

Here are comments from testers who said the next train is coming in 17 minutes:

CTA rider (AC) expects on the left to be actual arrival time. Time on right is saying that the next train leaves in 17 minutes. CTA Rider (AC) think it would help to say "Delayed."

MC assumed the time (5:46) would update in real time.

Gr8fl4CTA and Occasional CTA Rider both think the information is redundant and the

same.

Do you like the Ventra app?

Yes 92% (24)

No 8% (2)

Why or why not?

Here are the top reasons why testers liked the Ventra app (number indicates number of times the same

reason was given):

7 Easy to use

7 Metra functionality

5 Convenient

3 Good visual designs

Here are some additional responses from testers:

Brown line rider / occasional UP-N Metra rider said that this app is going to replace the

Ventra website for him.

Metra rider88 says, It was really convenient! Normally Metra rider88 has to go down to the

vending machine at the train station to add value via credit card and this can be very

inconvenient due to the lines that form.

Frequent CTA / Occasional Metra says, I will ride Metra more often because of the app. I

wouldnt have usually considered it as an option.

15



Blind Transit rider says, It has the promise to provide me with access on the go to the

information my Ventra pass has. [This app] can be more accessible and useful than any other

available option as a blind transit rider.

Do you believe the Ventra app fits into your day-to-day routine?

Yes 69% (18)

No 31% (8)

Why or why not?

Here are some responses from testers who said, Yes the Ventra app fits into their day-to-day

routine:

ChgoKC says, It'd be similar to the parking apps, where you'd keep it around and put on the

home screen of your phone.

CTA/Metra Commuter/Evil Genius has wanted this app for a long time.

Mischievous Metra Maverick rides the Metra daily and says this eliminates the need to look

for a soggy ticket that is ruined halfway through the month.

Geek Sheek says, this will probably be one of his top five apps.

Here are some responses from testers who said, No the Ventra app does not fit into their day-to-day

routine:

Frequent CTA Rider JH and Occasional CTA Rider are both looking for use of NFC

technology so that they can use their phone at the turnstiles to get on the trains or busses.

Frequent CTA / Occasional Metra says, Not day to day. But that is because I don't ride

Metra every day. If I did ride Metra every day, I would open it every day. I like the autoload

option and like I don't have to think about it. But I do like having the app to check balance.

SR likes that the ticket is protected from loss, but worries about having phone charged or

problems with wireless carrier services.

CTA rider (KB) says making it easier to get to [the arrivals page] more directly on open

would potentially replace use of other apps, but would use it a lot more if loading fare manually

or riding Metra daily.

How likely will you keep and use the Ventra app after this test?

5 Very Likely 81% (21)

4 Likely 8% (2)

3 Not sure 8% (2)

2 Unlikely 0%

16



1 Very Unlikely 4% (1)

What is your reason for keeping or not keeping the Ventra app?

Only three testers were not sure or thought it was very unlikely that they would keep the Ventra app

on their phones. Here is what they said:

CTA Savant says that the app is not doing enough for her to keep it. She would use it if it uses

NFC or if the Trip Tools worked better and did more than her other apps.

Gr8fl4CTA is not sure because the trip planning does not work well enough.

TA is not sure if she would keep it because she knows her daily value and would not need to

use trip tools often. It is handy for adding a value and getting a pass.

9 of the 23 testers who said that it was likely or very likely that they would keep the Ventra app

mentioned how easy or convenient it was to use the Ventra app. Here are some other responses

from tester:

CTA Rider says they are using Ventra every day and this does not make them jump between

apps.

Frequent CTA Rider believes the Ventra app will be part of her daily activities and is happy

that she will not need to go to the Ventra station to add value.

CTA/Metra Commuter/Evil Genius is very likely to continue to use the Ventra app because

it allows you to use Metra without a wallet and only ever carried cash before for Metra tickets.

How would you improve the Ventra app? How should it be improved to make you more likely to use it?

Testers shared a lot of improvements they would make throughout the rest of the test. 7 out of 26

testers wanted improvements to the Trip Tools, with 4 of these testers wanting a map incorporated. 6

out of 26 testers were interested in NFC mechanics so that they could use the app to pay for their

bus/train fare without carrying around their Ventra card. 2 testers mentioned the biggest improvement

would be fixing the back button.

Was this your first CUTGroup test?

Yes 73% (19)

No 27% (7)

Do you like CUTGroup?

Yes 100% (26)

No 0%

17



Did you like this CUTGroup test?

Yes 100% (26)

No 0%


CTA Rider says, I love being able to help contribute to the development of this product.

Frequent CTA Rider says Keep tests coming!

CTA/Metra Commuter/Evil Genius shared that people were very jealous that she had the

app.

Route 66 Book Stalker liked testing something that is important and impacts a lot of people.

18

Tester Profile (given by tester):

Blind Transit rider

Device (listed in initial screening email)

iphone 5

Remote Test


Logged in to an existing account

Have you added CTA

transit value or passes to

your account using the

Ventra app?

No


No


I couldn't because my active card was not accessible to me.


Used Trip Tools


I am blind and use voice over for this app. I spent a bit of time trying to make my active reduced faire

card for people with disabilities. I do not find the app or the website compatible. All the customer service

agents are able to see my reduced fare permit/ventra card, but I cannot see my active card. I can only see

my closed paratransit Ventra card. It is a closed account because I never activated it. I am trying to

correct that issue, but in the meantime I have been using the card that is not visible to me in either the

app, or the website. I was able to use the trip tools and found that to be extremely helpful and cover all

the transit options I use.

How appealing do you find

the visual design of the app?

3 - Neutral

Tell us about what you like or do not like about the visual

design of the app. How would you improve it?

There are some buttons or links that do not have text labels, or alt

text labels. This is important for accessibility.

How easy has it been to use

the Ventra app?

2 - Difficult


Make it fully accessible for blind voice over users. This means making

sure that text is part of all the activating links, buttons, etc.

19

7/15/2015 9:04:12 AM

In-person Component

Proctored by:

Tony Tom


A few times since I have had it

How easy was it to make a

CTA ticket or pass

purchase using the Ventra

app?

Not applicable - I did not make

a CTA ticket or pass purchase

How easy was it to purchase a Metra ticket using the

Ventra app?

Not applicable - I did not purchase a Metra ticket

If you purchased a Metra

ticket, did you use it on the

train?

Not applicable


Go to "My Ventra Cards" and review the page. We want to find out if this page works for

you and how to improve it.

Registration T/C's created a big issue when creating an account. Not straight forward for limited sight

user.

How would you explain what the following features mean to you. Think about how these

values are different from one another:

"Add Transit Value"

"Add Pass"

"Add Autoload"


Transit Stop (bad), nothing good that she's found.

20

Visit the "Trip Tools" page on your device and search for transit information near you. Tell

us about your experience.

Worked better than any other tool that she's found.

How did you search?

Searched by bus/train route


N/A

How did you search?


How easy was it for you to

find the correct stops you

were looking for?

3 - Neutral


The function itself is good. Getting there is difficult within the current

version of the app. Likes that the information is provided in a list view

and is easily read by voice over function.

How does this feature compare to other trip websites and apps that you have used in the

past?

Much better than other options she has. Websites work but are not convenient.

How would you register a new Ventra card if you wanted to add a card? Where would you

find this?

N/A

Did you create a new account during the remote test?

Yes

Please create a new account on the test device. Tell us about your experience as you go

through it.


create a new account?

2 - Difficult


Date of birth field issue. Form is too long. Check box doesn't work

and last box doesn't have a label of what it wanted. Lack of

consistency in how all of the fields were presented.

How often do you purchase

Metra tickets?

Once every six months


21

Please purchase a Metra ticket using the test device. Tell us about your experience as you

go through it.

N/A


purchase a Metra ticket?

3 - Neutral


Date of birth field issue. Form is too long. Check box doesn't work

and last box doesn't have a label of what it wanted. Lack of

consistency in how all of the fields were presented.

Now look for the Metra ticket you just purchased. Look through this page and the tickets.

What do you think about this page?

Please take a look at this

screen, and tell us when the

next train is.


OVERALL REVIEW


Yes

Why or why not?

It has the promise to provide me with access on the go to the

information my Ventra pass has. Can be more accessible and useful

than any other available option as a blind transit rider.

Do you believe the Ventra

app fits into your day-to-

day routine?

Yes

Why or why not?

How likely will you keep

and use the Ventra app

after this test?

5 - Very Likely

What is your reason for keeping or not keeping the Ventra

app?

Easier than other option. It allows me independence!

How would you improve the Ventra app? How should it be imporved to make you more

likely to use it?

Everything is labeled with words which is rare. Don't add pictures!

22


No


Yes


Yes


7/15/2015 6:01:07 PM

23


Brown line rider / occasional UP-N Metra rider


HTC One M8

Remote Test



Have you added CTA



Ventra app?

No


Yes


There was a little trouble with the Metra ticket purchase: my original Ventra password was too short. I

attempted to change my password in the app, but was not successful. (Unbeknownst to me at the time,

there are further restrictions on passwords beyond their minimum eight-character length.) I then logged

into the website and changed my password there, where the other password requirements (a numeral,

a special character, etc) were made explicit. Once that was done, I was able to buy a Metra ticket

through the app. My next use of the app will be applying my federal mass transit employee stipend to my

Ventra account later this month.


Review account history

Buy Metra Tickets

Used Trip Tools


Overall pretty useful. Changing the password (and setting forth the password requirements) could be a

little more clear-cut.



3 - Neutral



It's certainly serviceable, but it does not yet have the polish of peer

apps, such as Uber or Spotify.

24


the Ventra app?

4 - Easy


I feel navigation isn't always consistent. Sometimes I can use my

phone's back button, other times I have to use an in-app menu

button.

7/12/2015 10:32:23 PM

In-person Component

Proctored by:

Denise Linn


Once a day


CTA ticket or pass


app?




Ventra app?

4 - Easy



train?

No




Gets a federal transit benefit - normally adds that on and buys a pass with his own money. Hasn't done

that yet, but he does he'll have to figure out how to do that. He knows how to do this on the website.

For the app, it seems he can do the same process. Familiar. It has all the information he would want to

see at a glance.



"Add Transit Value" cash in the account

25

"Add Pass" fare

"Add Autoload" automatically charging a card to predetermined amount


CTA train tracker (notices they have one here and it seems better - more moden, auto uploads)



Did this just before getting here since he doesn't work in this part of the Loop. He likes you can specify

bus vs. rail. Intuitive.

How did you search?

Used "Locate Me" feature


He had done that before. Seems clear cut how to do it.

How did you search?




were looking for?

5 - Very Easy


Give me all the information I would normally want/expect at a glance.

The bookmarking function isn't as intuitive as it could be. Icons could

make more sense.


past?

Similar to what you'd get on the CTA website only this app "looks more this decade." The back button

doesn't get your back the main screen. Here you do the right swipe to get to navigation. This isn't all the

clear at first. It has to be learned.


find this?

My Ventra cards, then touch the plus sign, then register Ventra card.


No


through it.

26



Not applicable


His original passwork was only 4 characters long and he had trouble

changing the password. Also, the password requirements weren't on

the app. He had to change his password correctly online before he

could use the app.


Metra tickets?



specifically once every 2 months


go through it.

Note: didn't have password for the dummy account so didn't finish purchase.



4 - Easy


His original passwork was only 4 characters long and he had trouble

changing the password. Also, the password requirements weren't on

the app. He had to change his password correctly online before he

could use the app.



Unclear how to activate and when to activate the tickets you bought. Can there be a hint screen or a

help screen? He wouldn't be confident using these. This is a new feature to users of CTA, so more help

would be good.



next train is.

At 5:46 PM


What I would expect. It's intuative.

OVERALL REVIEW


Yes

Why or why not?

Looking forward to paying with the cellphone, but overall very useful

app. For him, it will replace going to the Ventra website.



Why or why not?

27

day routine?

No

Just because I don't need to go to the website day-to-day.



after this test?

5 - Very Likely


app?

Likes the trip tools a lot.


likely to use it?

The password issue was a one-time difficulty.


Yes


Yes


Yes


7/16/2015 7:33:00 PM

28


ChgoKC


Android Version 4.4.2

Remote Test



Have you added CTA



Ventra app?

No


No


I tried to add value to the account, but it asked for my Ventra password, which I entered (it's 4 digits),

and it said the password had to be 8 digits. I tried my password for my bank account, but that was

"incorrect". So not sure what password it's looking for.



Add Transit Value

Used Trip Tools

Looked up Ventra Value


not always easy to navigate. you couldn't hit a "back" key, or the back on the phone. Had to go

completely out to the app, and then re-log in. Now the little bars at the top left are visible to get to the

main menu, but it wasn't always visible.



4 - Appealing



I like the design of the app.


the Ventra app?

3 - Neutral (not easy, not

difficult)


works pretty well! adding $$ was my only issue.

29

7/13/2015 11:45:44 AM

In-person Component

Proctored by:

Tony Coppoletta




CTA ticket or pass


app?

4 - Easy


Ventra app?




train?

Not applicable




Noted that back button exits app from My Ventra Cards rather than going back to menu. Otherwise had

no additional comments.



"Add Transit Value" To add more value

"Add Pass" Doesn't know

"Add Autoload" Doesn't know


n/a

30



No comments (worked as expected)

How did you search?



Would like to be able to search for stops by name.

How did you search?




were looking for?

4 - Easy


Would like to be able to search for stops by name, or type in route

by number or name.


past?

n/a


find this?

Don't know, tried My Account--would probably just go online


No


through it.



Not applicable



Metra tickets?




go through it.

31

Understood transit value and chose to use that split with card. Very easy to use.



5 - Very Easy




Very self-explanatory, says what it is, when it expires.



next train is.

Sort of both


Should be the clock time, could be clearer.

OVERALL REVIEW


Yes

Why or why not?

I think it's cool. I love the whole idea of it. Everyone has a

smartphone, and you don't have paper to lose and it's easy.



day routine?

Yes

Why or why not?

It'd be similar to the parking apps, where you'd keep it around and

put on the home screen of your phone.



after this test?

5 - Very Likely


app?

On the weeks that she rides a lot she'd check every day.


likely to use it?

Nothing additional. Thinks it looks nice.


32

Yes


Yes


Yes


7/16/2015 7:11:46 PM

33


Cta my way


I phone

Remote Test


Registered a Ventra Card

Have you added CTA



Ventra app?

Yes


No


Did not purchase metra




Had to change password check balance


It was not as user friendly as I had hoped. Can I use my phone as the ventra card instead of the actual

card, like at Starbucks?



3 - Neutral



It was okay- nice color simple


the Ventra app?

4 - Easy


It asked for the nickname of card and I had no idea what thAt was

7/13/2015 3:13:09 PM

34

In-person Component

Proctored by:

Sonja Marziano


Once a day


CTA ticket or pass


app?

5 - Very easy


Ventra app?

1 - Very difficult



train?

Not applicable




Had to reset the password and had to know the name of the card, but that was tough to remember

Wants to be able to scan phone to get on transit (NFC)



"Add Transit Value" Typically use this feature

"Add Pass" Don't buy passes normally so these features are not as clear

"Add Autoload" First autoload (likes the warning) is for the card. Second autoload might be for pass?

seemed confused at first but then made sense after clicking


CTA/RTA search for "Find my bus route"



35

Trip Tools "CTA Busses" was set automatically alphabetical order but Sort A-Z was not highlighted.

Would make more sense to use your own location. Searched by route and stop, never saw the Locate

me button or the heart button. Would be better to say "Nearby locations" or "Nearby stops"

How did you search?



Searches by route, but wants to go back to the back screen "Find new stop" makes you start over

"Show all routes at this stop" Was curious about the alert button, would click that normally since it

looks like a message for me.

How did you search?




were looking for?

3 - Neutral


Click on the icons to know what they meant, wasn't intuitive. Stops:

were not clear that it was alphabetical


past?

Easier than searching CTA/RTA site, but likes having the map of the stops. Would like to see this

feature in Ventra app.


find this?

Went straight to "Ventra cards" searched around the screen, almost pressed "Add Pass" and then

clicked the + symbol.


No


through it.



Not applicable



Metra tickets?

I have never purchased a Metra

ticket before


36


go through it.

Went through it very quickly, very smooth process. Used transit value, liked that feature



5 - Very Easy




This seems very clear, no questions. Doesn't ride the Metra



next train is.

At 5:46 PM


"Delayed" if it's coming later. Shopping cart wasn't clear: thought it

might show you your value.

OVERALL REVIEW


Yes

Why or why not?

User-friendly



day routine?

Yes

Why or why not?

Take the bus more than the train, no machines to put the money in.

Usually uses to vending machine and has to go out of the way



after this test?

5 - Very Likely


app?

Ease of adding value


likely to use it?

Trouble "Find a Retail Location" on test device but works on her own iPhone -- says she doesn't really

need that.

37


Yes


Yes


Yes


Easy location from work! I got interrupted a lot, she mentioned that.

7/16/2015 6:18:34 PM

38


CTA Rider


LG G2

Remote Test



Have you added CTA



Ventra app?

Yes


No


There were no problems. I added value to my Ventra using a credit card, just as I do when using the

website. It was very simple.



Add Transit Value

Used Trip Tools


It's a good, reliable app. One thing I noticed is that when you add value it appears instantly, versus the

online site, where it usually doesn't update until you log off and log back in again, so that's a plus. I also

like that Bus Tracker is built into the app, which saves time from having to search it on the internet.



5 - Very appealing



It's recognizable with the light blue of the website & it has a clean,

streamlined interface which makes it easy to use.


the Ventra app?

5 - Very Easy


It's pretty simple to use @ the moment so there's nothing really that

I can think of, that could make it any simpler. It's pretty

straightforward.

39

7/13/2015 8:32:33 AM

In-person Component

Proctored by:

Tony Tom


Once a day


CTA ticket or pass


app?

5 - Very easy


Ventra app?




train?

Not applicable




Very streamlined. Related to the website, found it easy. Unlike website though you can remove a card,

this feature is not yet implemented in the app so she had to go back to the website and use it. Transit

History bogs down at times which can be frustrating. Overall, it's been better than the website because

it's instantaneous.



"Add Transit Value" Seemed self explantory

"Add Pass" Seemed self explanatory

"Add Autoload" Seemed self explanatory


40

CTA Trip Planner, Google Maps



Term is very misleading. Is actually a bus and train tracker. Trip Tools make me think of real time trip

planner, so all alerts, types of alerts didn't seem to go through. Step by step directions were expected

based naming convention.

How did you search?



Use bus/train tracker. Search stop. Search direction.

How did you search?




were looking for?

5 - Very Easy


Real-time alerts being more noticeable. More information would be

needed on what a 'delay' means to make the alert functions more

useable.


past?

Trip Planning is the big missing feature. 'Delay' statement could use more information to make it helpful.


find this?

For an existing user, need to add a description to the Plus button in the top right. Had difficulty finding

the button to add a card.


Yes


through it.

She already had a web account so did not create an account during her testing. We set up account on

the device. Issue with birthdate field.



4 - Easy


Make the year in birthdate two digits.

41


Metra tickets?

Once ever year



go through it.

Easy.



5 - Very Easy


Make the year in birthdate two digits.





next train is.


OVERALL REVIEW


Yes

Why or why not?

Flexibity and ease of use. Was very dependable and accurate. Almost

instantaneous.



day routine?

Yes

Why or why not?

Because I'm a commuter.



after this test?

5 - Very Likely


app?

Already using Ventra every day. Makes me not have to jump between

apps.


likely to use it?

42

Make Trip Tools more than a bus tracker. Real-time updates need to provide more info around 'delays'.

Provide access to 24-hour support for Ventra. Put the card on the phone.


Yes


Yes


Yes


Loved being able to help contribute the development of this product.

7/15/2015 4:17:03 PM

43


CTA rider AC


Nexus 4

Remote Test



Have you added CTA



Ventra app?

Yes


No


It went seamlessly. The system remembered my credit card info from my earlier purchases and could

not really have to make any effort to add value.


Add Transit Value

Used Trip Tools



It seems a bit slow to load but otherwise it works fine.



4 - Appealing



I would change the color of cursor from white to black or something

dark. It is really annoying that I can't see where the cursor is.


the Ventra app?

4 - Easy


Some icons are not very telling. I had to click on them to figure out

what they are for.

7/13/2015 9:16:21 PM

44

In-person Component

Proctored by:

Sonja Marziano




CTA ticket or pass


app?

5 - Very easy


Ventra app?




train?

Not applicable




Last four digit of the card number doesn't seem necessary (only has one card though) PhD student so

adds value during the summer Transit History takes multiple tries before it shows up. Attempted 5 times

before it showed Wanted to know the timeframe of the Ventra cards Also wanted to add a second card.

Was confused about "Register a Ventra Card" because she had two cards under two different user

names. It seems to allow her to register a second card that is under a different user name.



"Add Transit Value" just value

"Add Pass" Monthly pass

"Add Autoload" a pass and autocharge; Does not think that autoload needs to be a separate option

but instead part of the pass option


Chicago Transit Tracker Lite

45



It's easier to use on the Ventra app. Less clicks to find the correct stop. Uses the bus stop number when

she is nearby the stop Locate me feature, first but then went back has background knowledge about

which line to ride Has pink line in her favorites

How did you search?



Lots of scrolling through trying to find the stop, and then went to A-Z sort after a little while Was

confused about the order of how stops were listed Prefers to have stops listed alphabetically, but thinks

that order by station might be better for users that are from out of town

How did you search?




were looking for?

4 - Easy


Cursor is white on the white background when typing is required

Locate me icon did not make sense until she clicked it, thought text

might be helpful.


past?

"I trusted it more because it was affiliated with Ventra" More "design-y" The "Locate me" function was

not part of the old app The enter stop number also did not exist


find this?

Found it during the remote test but had confusion about what "Register a Ventra Card" meant

specifically


No


through it.



Not applicable


How often do you purchase Anything else to add?

46

Metra tickets?

Once ever year


go through it.

Not familiar with Metra at all, so didn't know which stops to pick Ordering of stops are interesting but

could not figure out the logic. Cursor should be black



5 - Very Easy




Time on the top of the ticket is the current time, not the valid until time. Tester liked sharing the valid

until time, but only shown in QR code. Design is a little much, concerned about battery life. Explained

why it was moving and she seemed to feel better about the design element.



next train is.

In 17 min


Expects on the left to be actual arrival time. Time on right is next

train leaves in 17 minutes. Would help to say "Delay"

OVERALL REVIEW


Yes

Why or why not?

"Really easy to put money on your card." Used to put money on card

online. Likes its the first thing you see if the amount left on the card.



day routine?

Yes

Why or why not?



after this test?

5 - Very Likely


app?

easy to put money on card, and trip tools

47


likely to use it?

Showed me error to add a New Card: "Webpage not available". Keyboard is large and overlaps with the

fields and "Save" and "OK" buttons. Seems to be an Android issue.


No


Yes


Yes


7/16/2015 5:01:47 PM

48


CTA rider KB


iPhone 6

Remote Test



Have you added CTA



Ventra app?

No


No


N/A



Used Trip Tools



Good, and much easier than the mobile app, which I used to use frequently. It is sometimes slow to load

though.



3 - Neutral



Things are too close together in many places.


the Ventra app?

4 - Easy


Many pages are missing a back button, making navigation difficult.

7/15/2015 10:46:43 AM

49

In-person Component

Proctored by:

Tony Coppoletta




CTA ticket or pass


app?




Ventra app?




train?

Not applicable




Thinks there's a lot on the page. Noted difference between slider for transit value autoload but a button

for adding autoload under pass. Likes that balance is clear and what's regular value and benefit value, as

someone who has both in their account. Might like more to highlight what things do, though not sure

exactly what that would be. Might like to be able to see both value and passes in one place rather than

have separate flows to get to each and have it more explicit that there are no passes in queue when

there aren't any. Transit history takes a while to load but not evident that when tapped it is loading, so

tapped multiple times not realizing the tap had been accepted. Would like to be able to see further back

in history.



"Add Transit Value" Will add value

"Add Pass" Will add a pass (7-day, 30-day)

"Add Autoload" Will add autoload for value or pass (unclear)


50

TransitStop, CityMapper (but felt too much info, just needs quick stop info)



Experience was good. Wants there to be a functioning back button from results screen and felt list of

nearby stops could present what stop has what services more clear (direction per stop, for bus, for

example).

How did you search?



During stop selection, it'd be nice if back button had a larger hot area (doesn't feel responsive enough).

Train info can be more intuitively sorted. Would be nice to be able to see, when selecting stops, which

one is closest. Section headers on main screen seem restating what's implicit, perhaps unnecessary.

How did you search?




were looking for?

3 - Neutral


Make it easier to find that feature and get to it from more places

within the Trip Tools section.


past?

It's good, has about the same functionality that many other apps have, and the alerts here are easier to

find (shown prominently). Navigation between pages could be easier, though. Though the default area is

My Ventra Cards, would make a better substitution for other transit apps used if the favorites/nearby

were easy to find throughout the app/right up front. Would like link to view full schedule, or otherwise

determine first bus/last bus. If no results shown, would be nice to pull to refresh.


find this?

Used (+) button. Noted might be confusing to some that you can see balance in My Ventra Cards or

separately look up value. Noted that "I'm logged in but it wants me to log in" on being redirected to

mobile website flow. User understands need for auth in mobile web experience, but noted it would be

nice if would at least autofill username

final report for cutgroup #17 - ventra mobile app

Technology