Research Papers

On-road and Simulated Driving: Concurrent and Discriminant Validation

Filename FINAL-PAPER-67.doc
Filesize 162 KB
Version 1
Date added June 10, 2012
Downloaded 3 times/fois
Category 2012 CMRSC XXII Banff
Tags Session 2B
Author/Auteur Katherine Wood, Daniel Mayhew, Herb Simpson, Lawrence Lonero


Purpose: A multi-site, multi-level evaluation of the effectiveness of driver education - a large scale evaluation of driver education (LSEDE) -- is currently being conducted by the Traffic Injury Research Foundation (TIRF) and Northport Associates (NPA) under funding from a consortium that includes: AAA Foundation for Traffic Safety (AAAFTS), Centers for Disease Control and Prevention (CDC), Manitoba Public Insurance (MPI), and the National Highway Traffic Safety Administration (NHTSA). One segment of the evaluation involves an examination of changes in student outcomes, including knowledge, attitudes, opinions, driving practices and skills. Most of these outcomes are being measured using a questionnaire. Although self-reports provide valuable insights into driving skills, a more objective measure such as that obtained from an on-road test or from a simulator, would greatly enhance the validity of inferences. Driving simulators provide a more controlled and safer environment for measuring performance than an on-road test. Most simulators, however, are used to teach driving skills, not assess them, and the validity of simulators for assessing driving skills, especially those of young drivers, has not been well established. This paper describes a converging pair of studies that examined the validity of a driving simulator to measure driving skills.

Methods: A concurrent validity study was designed to compare driving performance on-the-road with driving performance on a simulator - i.e., how well does a drive test on road correlate with a drive test on a simulator. A discriminant validity study compared driving performance on the simulator across three groups of drivers who differ in their level of experience - a group of true beginners who had no driving experience, a group of novice drivers who had completed driver education and had a learner's permit, and a group of fully licensed, experienced drivers. This study design determined the degree to which a simulated drive test discriminates between "unskilled" and "skilled" drivers defined in terms of their driving experience. These two validation studies were conducted in Winnipeg, Manitoba, Canada.

Results: The concurrent validity study results showed a reasonable degree of concordance in terms of the distribution of driving errors on-road and errors on the simulator. This study found a significant relationship between the two tests when performance was rank ordered according to errors - simply put, in general, the worst drivers as measured by their on-road performance were also the worst drivers as measured on the simulator. The discriminant validity study results showed significant differences in driving errors and anticipating hazards among the groups in the expected direction -- the various measures of driving errors showed that beginners performed worse than novice drivers and that experienced drivers had the fewest errors.

Conclusions: This investigation established that on-road driving performance and performance on the simulator are significantly related and that the simulator can discriminate between drivers with different levels of experience. These results support the use of the simulator as a valid measure of driving performance for research purposes.

Katherine Wood, Daniel Mayhew, Herb Simpson, Lawrence Lonero