When Segment Anything Model (SAM) Meets Inventorying of Roadway Assets

Abstract: Automating the identification, localization, and monitoring of traffic management system (TMS) assets distributed widely in the roadway network is critical. It will efficiently provide up-to-date information in supporting TMS asset management. Collecting videos with vehicle-mounted cameras and processing the data with computer vision (CV)-based deep learning methods is a technology garnering increased attention. While promising, challenges arise from the lack of high-quality annotations of TMS assets in the image data, which need to be provided by domain experts. The Segment Anything Model (SAM) presents a potential breakthrough for general image segmentation. It motivates the study of this paper that aims to evaluate SAM’s applicability and efficiency in extracting TMS assets from video data collected on the road. Specifically, the paper examined how the model size and the quality of bounding box prompts impact SAM’s performance in segmenting TMS assets such as traffic lights and signals. The evaluation of SAM was made against five prominent semantic segmentation models. Results show that a lightweight SAM has outperformed other deep learning semantic segmentation models. However, the superior performance heavily relies on detailed domain expert knowledge. In the end, the paper suggests embedding valuable domain expert knowledge in CV-based deep learning models by utilizing SAM, leading to a harmonious collaboration between human experts and artificial intelligence.