materials developed by k. watkins, j. lamondia and c. brakewood data collection with surveys unit 2:...

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Data Collection With Surveys

Unit 2: Describing Transit Systems with Data

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Outline

• Types of surveys

• Methods of sampling

• Survey content

• Survey instruments

• Data collection plan

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

TYPES OF SURVEYS

Last time we discussed the type of information we collect. Now, let’s discuss how we go about collected detailed survey data.

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Types of Surveys

• Longitudinal survey– Over time

• Cross-sectional survey – Single point in time

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Survey Modes

1. On-board/Intercept 2. Telephone 3. Mail 4. Web-based 5. Smartphones (emerging)

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

On-Board & Intercept Surveys

Overview

• Conducted on-board vehicles, in stations or at stops

• Target (study) population is current riders

• Very common: 96% of transit agencies conducted on-board surveys between 2002-2004*

Pros & Cons+ Easy to administer+ Inexpensive+ Relatively high response

rates (33%-67%*)

- Constraint of trip time on length of survey instrument (mail-back option)

- Non-riders excluded

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

On-Board Survey Example

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Telephone Survey

Overview

• Sampling frame is the household

• Often use Random Digit Dialing (RDD)

• Most common in cities with high use of transit– 71% of transit agencies

conducted telephone surveys between 2002-2004*

Pros & Cons+ Riders and non-riders+ Medium to long survey

instruments+ Good response rates

- Interviewer training (Computer Aided Telephone Interviewing, or CATI, can help)

- Respondent needs a (local) telephone

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Mail Survey

Overview

• Sampling frame is the household

• Somewhat common– 38% of transit agencies

conducted mail surveys*

Pros & Cons+ Riders and non-riders+ Longer survey instruments+ Easy to administer+ Eliminates interviewer bias

- Response rates can be very low; typically below 20%*

- Usually not used for origin-destination surveys

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Web Survey

Overview

• Sampling frame is riders (usually)

• Increasingly common– 44% of transit agencies

conducted mail surveys between 2002-2004*

Pros & Cons+ Very inexpensive + Very easy to administer+ Longer survey instruments+ Eliminates interviewer bias+ Data already coded

- Respondent needs to have internet

- Difficult to achieve random sample

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Smartphone Survey

Overview

• Native smartphone applications can be used for travel surveys

• Geocoded travel information supplemented with survey questions

• Still in research & development– Some state DOTs; limited (if

any) use in transit

Pros & Cons+ Geocoded travel information+ Longer survey instruments+ Relatively easy to administer+ Data already coded

- Respondent needs to have a smartphone

- Difficult to achieve random sample

- Privacy concerns

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

METHODS OF SAMPLING

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Methods of Sampling

• Simple Random Sampling• Systematic Sampling– every nth unit selected

• Stratified Sampling – divide the population into homogenous groups

(e.g. areas of the city) • Cluster Sampling – natural groups, sample within groups

• Probability Proportionate to Size – selection probability proportional to size

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Sample Size Determination

• Depends on the population size (e.g. number of riders) and the level of precision desired

• Increase sample size increase costs• Example for Simple Random Sampling

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

SURVEY CONTENTWhat is commonly included in transit surveys?

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Research Goals & Content

• Origin & Destination Surveys– “This trip”: Origin, destination, purpose, access mode, egress

mode, etc.• General Travel Behavior

– Frequency of travel, how long using transit, alternative modes• Demographics

– Gender, age, race, driver’s license, HH income, HH size, employment status

• Satisfaction– Attributes of service: timely service, speed of service,

cleanliness, comfort, safety, crowing, information• Other

– Fare payment, new technologies, communications, etc.

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Translating Research into Survey Questions

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

SURVEY INSTRUMENTHow to write a good survey

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Types of Questions

• Closed-Ended– Dichotomous (2 answers, e.g. Yes/No)– Nominal (Categorical)– Ordinal (a.k.a. Likert, Ranking)– Filter/Contingency (Qualified)

• Partially-Closed– Last answer is “Other______”

• Open-Ended

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Do’s and Don’ts

• Questions should be:– Concise (multiple choice)– Understandable– Useful– Exhaustive

• Questions should NOT be:– Double-barreled– Double negative– Technical jargon– Long-winded– Redundant– Obtrusive or embarrassing

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

IN-CLASS EXERCISENow, its time for you to practice

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

DATA COLLECTION PLANPutting it all together in order to create a…

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Creating a Data Collection Plan

1. Determine research goals 2. Select survey mode & sampling methodology3. Determine necessary sample size4. Create detailed sampling plan5. Draft questionnaire6. Pre-test the questionnaire7. Conduct staff training8. Distribute survey & revise sampling plan 9. Code/process data & error check

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Other Considerations

• Language – English & Spanish

• Special groups – Those who cannot read/write

• Longitudinal comparisons– Consistency of questions with prior transit surveys

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

CHICAGO TRANSIT AUTHORITY

What does a (good) transit survey look like?

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

2003 Chicago Transit AuthorityCustomer Satisfaction Survey

• Objective: Comprehensive review of customer satisfaction conducted every 2 to 3 years

• Content Included:– General travel behavior (modes, trip purpose)– Satisfaction with service delivery attributes– Loyalty towards CTA– Technology use (internet access, CTA website)– Fare payment – Demographics

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

CTA: Mode & Sampling

• Mode: Household Telephone survey– Random Digit Dialing– Computer Assist

• Stratified Sampling– Area of residence (N, S, NW, SW, W, Suburbs)– Mode (rail or bus)

• Sample Size– Minimum of 200 respondents per strata– Total of 2,577 responses

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

CTA: Questionnaire

• Question types– Mostly closed-ended questions (scales typically 5

points)– A few partially closed (other) & open ended

questions • Interview length– Average of 21.4 minutes to complete– Pre-tested to assure length

• Response rate– 64,986 telephone numbers attempted

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

CTA: Results

• Satisfaction with bus service:– Has improved terms of reliability, information services, and comfort,

among other things.– Has decreased in regards to access to service, personal safety, and

intramodal transferring (within CTA).

• Satisfaction with rail service:– Has improved or stay constant in terms of operator attributes

(personnel), information sources, reliability and fare payment, among other things.

– Has decreased in regards to access to service, intramodal travel, personal safety, and appearance.

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

CTA: Results

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Conclusion

• Surveys are a good way for agencies to know who their riders are and what they want.

• There are several survey types with varying costs, penetration and bias. They should be chosen to match their target.

• Surveys should be easy to understand and should encourage participants to complete them.

Materials developed by K. Watkins, J. LaMondia and C. Brakewood

Reference

Materials in this lecture were taken from:• Schaller. “On-board and Intercept Transit

Survey Techniques”. Transit Cooperative Research Program Synthesis 63, National Research Council, Washington, DC (2005).

• CTA, Customer Satisfaction Survey, 2003.