Skip to main content
ICTD 2024 Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
ViewAttendees 1

Abstract for presentation (Poster or Podium)

Safety, Security, and Standards

Enhanced Safety Evaluation for Autonomous Vehicles Considering Perception-Critical Scenarios

Sunday, June 16, 2024
3:00 PM - 4:15 PM ET
Room: Galleria

    Primary Author(s)

  • Chengyuan Ma, n/a

    Postdoc
    University of Wisconsin-Madison
    MADISON, Wisconsin, United States

    • Presenter(s)

  • Chengyuan Ma, n/a

    Postdoc
    University of Wisconsin-Madison
    MADISON, Wisconsin, United States

    • Co-Author(s)

  • Hang Zhou, n/a

    Research Assistant
    University of Wisconsin-Madison
    Madison, Wisconsin, United States

  • Zheng Li, n/a

    Research Assistant
    University of Wisconsin, Madison
    Madison, Wisconsin, United States

  • Ke Ma, PhD

    Research Assistant
    UW-Madison
    Madison, WI, United States

  • Xiaopeng Li, PhD

    Professor
    University of Wisconsin–Madison, United States

    • Presenter(s)

  • Xiaopeng Li, PhD

    Professor
    University of Wisconsin–Madison, United States

    • Submitter(s)

  • CM

    Chengyuan Ma

    University of Wisconsin-Madison
    MADISON, Wisconsin, United States

Abstract:

The testing approach for autonomous driving systems, particularly in safety-critical scenarios, has garnered significant attention within both academia and industry. Existing research has predominantly focused on designing road geometries and vehicle maneuvers while assuming perfect autonomous vehicle (AV) perception capabilities. In practice, perception errors due to varying environmental factors, including weather conditions and lighting, can be significant contributors to the AV accidents. This paper aims to accelerate the AV evaluation process by increasing the probability of crucial perception states occurring during testing using the important sampling method. Based on the naturalistic driving data, we first calculate the optimal probability distribution for sampling test scenarios through important sampling theory. A Stochastic Dynamic Programming (SDP) model is employed to iteratively increase the occurrence frequency of critical states in test episodes. A counterfactual approach is applied to capture such critical perception states during the iterative process, including the movements of road users and stationary variables like weather conditions, road conditions, and vehicle colors. We validate the proposed approach using the Waymo public dataset, demonstrating its effectiveness in accelerating the evaluation process, while simultaneously proficiently recording safety-critical perception scenarios for AV training and development.

Learning Objectives:

Attendees can expect to learn the following from this session:
  • quantitatively assess the impact of perception-related factors in automated vehicle accidents
  • understand how to accelerate the validation of rare events, such as AV accidents, by important sampling method
  • understand how to generate test traffic scenarios by the near optimal probability distribution using stochastic dynamic programming model
  • know the practical steps for creating AV testing environments using the Waymo open dataset