|

More is Less: How Increasing Damage Thresholds for Reporting Collisions Erodes Sample Sizes and Produces Misleading Analytical Results

Author(s): Pooley

Slidedeck Presentation Only:

7C_Pooley

Abstract:

Background/Context: Law enforcement and city planners use collision statistics to predict high-incident areas and the most prevalent causes of traffic collisions. As with all statistical analyses, the sample size of the data is everything. Manitoba has recently increased the damage threshold for reporting collisions to $5,000, and other provinces are considering this move as well. Ontario currently has a damage threshold of $2,000, increased from $1,000 in 2015.

Aims/Objectives: The goal is to show how excluding the data from collisions in the $2,000 - $5,000 damage range produces misleading results. Particularly important data points are: injury frequency and severity, vehicle maneuvers and initial impact type, driver action and condition, and high-incident intersections. It is believed that by excluding the large number of accidents that fall within this mid-range of damage estimates, any conclusions derived regarding the causes and conditions of traffic incidents will not reflect real-world problems that face drivers, police officers, and insurers every day.

Methods/Targets: The methodology for this is simple. Using a database of approximately 400,000 self-reported collisions taken in Ontario since 2016, common analytics were performed: ranking of collisions types and severity; injury frequency and severity; involvement of pedestrians, bicyclists, and motorcyclists; action and condition of involved drivers; and collision frequency by intersection. These are common reports that are produced for insurance companies and police services on a regular and frequent basis. These were then re-evaluated using a minimum threshold of $5,000 in estimated damages.

Results/Activities: The most striking result is that excluding collision data for those where the total damage fell between the $2k - $5k range reduces the sample size of the data by about half. As expected, this led to differences in rankings for the top 10 intersections in Ontario broadly, as well as how that is determined in specific communities. The vehicle maneuvers, initial impact types, and other factors that go into determining causes of incidents were undermined and misrepresented by the lack of data that these lower-severity reports provide.

Discussion/Deliverables: While the difference in data and results is marked, there is still some discussion about whether this affects the stated goals of civil planning projects. Further investigation into whether law enforcement and civil engineers are focusing solely on collisions that involve injury (which generally follows along with higher damage estimates) needs to be done. One limit or challenge of the study was that the values presented are not actual damage amounts, but estimates provided at the time of the self-report. However, since this directly speaks to whether drivers would be required to voluntarily report based on their non-professional assessment of the damage, the results are valid.

Conclusions: While it was expected that an increase of the minimum reportable threshold would reduce the volume of collision data that would be available, the surprise was the extent of the effect. Depending on whether extremely low-severity collisions (for example $0 - $500) are included or excluded, the effect size changes slightly. But by all metrics, the number of collisions that would be erased from datasets is significant and overwhelming, and not accounting for them in proactive law enforcement and city planning would be problematic.