Analysis of Driving simulator Analysis of Driving simulator validation for speed researchvalidation for speed research

Stuart T. Godley, Thomas J. Triggs, Brian N. Stuart T. Godley, Thomas J. Triggs, Brian N. FildesFildes

1.Purpose of Research ideas/question1.Purpose of Research ideas/question

To evaluate countermeasures for mean speed using behavioral validation of an advanced driving simulator

Using behavioral validation : relative validity, absolute validity and interactive validity:

to validate the Monash University Accident Research Centre (MUARC) driving simulator.

2. Background support2. Background support

Blaauw(1982) proposed two levels of Blaauw(1982) proposed two levels of validity:validity:

Physical validity is correspondence of Physical validity is correspondence of simulator components, layout, simulator components, layout, dynamics with real cardynamics with real car

Behavioral Validity is correspondence Behavioral Validity is correspondence between simulator and car in the between simulator and car in the way the human operator behavesway the human operator behaves

Rel val. is necessary unlike absoluteRel val. is necessary unlike absolute

Background support: Absolute Vs Relative ValidityBackground support: Absolute Vs Relative Validity

3. Theoretical Basis/ Hypothesis3. Theoretical Basis/ Hypothesis Analysis of behavioral validation, absolute Analysis of behavioral validation, absolute

validity and relative validity is the validity and relative validity is the theoretical basis.theoretical basis.

Beh. Val. :comparison between simulator Beh. Val. :comparison between simulator and real car drivingand real car driving

Absolute Validity: comparison of Absolute Validity: comparison of performance difference between performance difference between experimental conditions in sim. and carexperimental conditions in sim. and car

Relative Validity: difference between Relative Validity: difference between experimental conditions of identical experimental conditions of identical magnitudemagnitude

4. Practical Contribution4. Practical Contribution

Simulator provides experimental control, Simulator provides experimental control, efficiency, expenses, safety, ease of data efficiency, expenses, safety, ease of data collection which may/may not be provided collection which may/may not be provided by real car by real car

Driving patterns in the simulator observed Driving patterns in the simulator observed to improve future carsto improve future cars

5. Theoretical Contribution5. Theoretical Contribution

For the purpose of effective For the purpose of effective countermeasure, speed is validated to be countermeasure, speed is validated to be dependent variable for research using dependent variable for research using simulator.simulator.

Validity of speed covers research Validity of speed covers research investigating the relative differences investigating the relative differences between treatment and control roads.between treatment and control roads.

6. Appropriate Methodologies6. Appropriate Methodologies Driver’s speed response to rumble strips was experimental Driver’s speed response to rumble strips was experimental

stimuli, simulator adjustedstimuli, simulator adjusted Two experiments conducted; driving instrumented car and Two experiments conducted; driving instrumented car and

simulatorsimulator VariablesVariables Instrumented car : 12 males, 12 females, average age 29.8 Instrumented car : 12 males, 12 females, average age 29.8

years years Three sites; approach to stop sign, right curve, left curveThree sites; approach to stop sign, right curve, left curve 2 Treatment sites for simulator and real road: Consisted of 2 Treatment sites for simulator and real road: Consisted of

rumble stripsrumble strips 2 control sites for simulator and real road: without rumble 2 control sites for simulator and real road: without rumble

stripsstrips Treatment and control matched for on road off road detailsTreatment and control matched for on road off road details

Methodologies...Methodologies... Rumble strips : width 60 cm, preceded Rumble strips : width 60 cm, preceded

intersection by 30-50 m, spaced at decreasing intersection by 30-50 m, spaced at decreasing distancesdistances

Two experimental routes: half male/female on Two experimental routes: half male/female on each route, had vehicles parked on themeach route, had vehicles parked on them

MethodMethod Instructed to drive normally, experimenter in Instructed to drive normally, experimenter in

front seat, technician at backfront seat, technician at back First a practice drive:11 min, then experimental First a practice drive:11 min, then experimental

drive: ~40-50 min.drive: ~40-50 min.Data collectionData collectionInstantaneous speed recorded at each intersectionInstantaneous speed recorded at each intersection

Methodologies...Methodologies...

Some participants excluded from data pool for Some participants excluded from data pool for individual treatment-control analysis: unrestricted individual treatment-control analysis: unrestricted drive to intersection, slow moving car, school drive to intersection, slow moving car, school children walking, etcchildren walking, etc

No participant removed from all analysisNo participant removed from all analysis

Validity of measuresValidity of measures Participants had to sign a consent form approved Participants had to sign a consent form approved

by Monash University Ethics Committeeby Monash University Ethics Committee

Methodologies...Methodologies...

22ndnd method: Simulator Experiment method: Simulator Experiment

VariablesVariables Simulator car : 12 males, 8 females, average age Simulator car : 12 males, 8 females, average age

26.4 years 26.4 years Projected through computer :resolution 640x 480 Projected through computer :resolution 640x 480

pixels, rate of 30 Hz. 180 deg forward, 60 deg pixels, rate of 30 Hz. 180 deg forward, 60 deg horizontal, 45 deg vertical viewhorizontal, 45 deg vertical view

Rest all variables are sameRest all variables are sameMethod and data collection were the same; except there was Method and data collection were the same; except there was

no hindrance while driving; hence collection of datano hindrance while driving; hence collection of data

7. Statistical Analysis7. Statistical Analysis A modified correlation analysis based on A modified correlation analysis based on

canonical correlation was used.canonical correlation was used. Correlation of each participant’s data not feasible: Correlation of each participant’s data not feasible:

separate participants used in each expseparate participants used in each exp Measures of effect size in ANOVA are measures of Measures of effect size in ANOVA are measures of

degree of association between effect and degree of association between effect and dependent variable (ex: main effect, linear dependent variable (ex: main effect, linear contrast, an interaction)contrast, an interaction)

Two one-way ANOVA calculated: one for Two one-way ANOVA calculated: one for treatment, other for controltreatment, other for control

Omega squared and interclass correlation Omega squared and interclass correlation estimate degree of association in presentationestimate degree of association in presentation

Statistical Analysis...Statistical Analysis...

Non significant results validate simulator, but Non significant results validate simulator, but may arise from inadequate statistical power may arise from inadequate statistical power rather than genuine absence of difference.rather than genuine absence of difference.

Small omega square reflects genuine non-Small omega square reflects genuine non-differences; large value suggests a difference differences; large value suggests a difference exists, but insufficient sample size.exists, but insufficient sample size.

Omega square at or below 0.01 meaninglessOmega square at or below 0.01 meaningless

•F- test used to interpret the significant differences in

experiments

•Alpha = 0.05

•95% confident that differences are significant

•Correlation used to support interactive relative validity

•Size effect verify if the non-significant result can be

conclusive

8.Presentation of Results8.Presentation of Results

Results : Stop Sign ApproachResults : Stop Sign Approach

Results : Stop Sign ApproachResults : Stop Sign ApproachSignificant difference between mean speed at treatment site relative to the control site ( P<0.001 )

Very small size effect (0.002) to support this non-significant result

Average relative validity was established

The pattern of speeds for the treatment site and control site similar in both experiments, supported by a significant correlation (0.40)

Interactive relative validity established

Speeds for the two sites converged as they approach the intersection in the simulator

This pattern was not observed in the data for the on-road experiment

Absolute validity was not established

Results:Right Curve Approach ApproachResults:Right Curve Approach Approach

Results:Right Curve Approach ApproachResults:Right Curve Approach ApproachCar: No significant difference between the mean speed Simulator: Significant difference between the mean speed Significant interaction between 2 experiments (P<0.01)Average relative validity was not established Speed pattern similar for the first three-quarters of the measurement areaSignificant correlation (0.52) supported interactive relative validityRight curve treatment site speeds not statistically different between the car and simulator experiments (P = 0.590)Small effect size(0.007) supports this non-significant result.Control site speed slower for instrumented car experiment compared to simulator experiment (P<0.05)Absolute validity established for treatment sites, but not control sites

Results:Left Curve Approach ApproachResults:Left Curve Approach Approach

Results:Left Curve Approach ApproachResults:Left Curve Approach Approach

Significant difference between mean speed at treatment site relative to the control site (P<0.001) Significant speed differences between the two experiments (P<0.001)Averaged relative validity not establishedSame speed pattern for the first three quarters of data collection areaSignificant correlation (0.50) supported interactive relative validity Significant speed differences in treatment site for instrumented car and simulator experiments (P<0.001)Control site speeds not significantly different, but produced a size effect close to medium Non-significant result cannot be considered as a conclusive. Absolute validity was not established.

Relative validation: Treatment speeds started to reduce relative to

their control sites, before the rumble strips Significant correlations in relation to interactive

relative validity. Rumble strips led to an average slower

approach speed to the stop sign intersection in both the simulator and on-road trials

Relative validity well established for mean speed

Absolute Validation:Absolute speed values were generally different in the two experimentsGeneral trend: simulator induce slower speeds than instrumented cars

Results: Relative and Absolute ValidationResults: Relative and Absolute Validation

9. Conclusion : Relative Validity9. Conclusion : Relative Validity

Simulator produced larger average speed differences between its treatment and control sites during the curve approaches

Two reasons: 1. On-Road : Speeds initially faster at the treatment relative to the

control sites ; Non Perfect Site Matching

Wider lane widths conducive to faster driving Treatment site roads may have geographical characteristic that

encouraged faster speeds

2. Treatment site speeds started to reduce further back before the first of the rumble strips in the simulator experiment

main difference between the simulator and the on-road experiments

Without the two experimental differences, average relative validation for the curves may have been established.

Absence of G-force motion cues in simulator

Intentionally short practice section may have contributed to simulator participants to slow earlier when they perceive the rumble strips

Converging speed pattern could only be determined when the vehicle stopped for on-road trial, rather than when the front of the simulator car crossed the stop line

Simulator may have slightly enhanced the speed differences

Address limitations and these differences should be able to be minimized.

Simulator does seem to be a valid tool for generating and generalizing relative speed results for experiments involving road based speeding countermeasures aiming to influence decelerating.

9. Conclusion : Relative Validity9. Conclusion : Relative Validity

9. Conclusion : Absolute Validity9. Conclusion : Absolute Validity

Different absolute speed values in two experiments not particularly negative because simulator experiments only concerned speed differences between road environments

Experiments did not attempt to establish the numerical speeds at which the investigated road manipulations encourage drivers to drive

9. Conclusion :Overall9. Conclusion :Overall Evidence to conclude that speed is a valid measure to use for

the experiments on driving simulator involving road based speeding countermeasures.

Speed profiles found indicate a speed reduction relative to control roads or other roads.

Speed has been clearly validated as a dependent variable for research using the simulator.

Validity of speed only covers research investigating the relative differences between road treatments and control roads.

Inconsistencies between the two experiments account for:

1. Differences in characteristics of the road and road environment between the treatment and control sites in the on-road experiments hindered the validation.

2. Procedural methods with the simulator, notably practice, may have also contributed.

10. Future Work/Research Directions:10. Future Work/Research Directions:Reduce the inconsistencies between the two experiments.

Include investigating absolute numerical speed values for validity of speed

More similar initial speeds for the on-road curve sites, so that a more similar treatment-control site speed difference to the simulator experience may be found.

Conduct experiment with same participants for the on-road and simulator trials so as to observe any difference in the results.

Experiments can be modified so that converging patterns can appear for both on-road and simulator data.

Attempt to establish the numerical speeds at which the investigated road manipulations encourage drivers to drive

Validity of speed to investigate the absolute differences between road treatments and control roads.