A Lightweight Camera and Lidar Fusion Framework for Railway Transit Obstacle Detection

The presence of obstacles on railway tracks poses a significant threat to the safety of train operations. In order to accurately obtain information about obstacles, we propose a novel obstacle detection method that combines camera and lidar technology. Initially, the improved YOLOv5 network is employed to extract 2D positional information of obstacles, enhancing feature representation capabilities through dual-feature fusion channels. A two-stage serial fusion structure is then utilized to progressively refine feature layer information. Subsequently, the coordinate transformation relationship between lidar and image coordinates is leveraged to extract point cloud features of obstacles from lidar data. This procedure facilitates the derivation of obstacle distance information, contributing to the precise acquisition of obstacle data essential for tasks like emergency obstacle avoidance in rail vehicle scenarios. To assess the algorithm's performance, a multi-sensor data collection platform is established, and the algorithm is tested in a real-world train operating environment. For performance evaluation, a dataset containing various types of track impediments is compiled. Experimental results demonstrate an average detection accuracy of 85.1% for common obstacles, with a detection speed reaching 94 FPS. Through on-track testing and experiments, the real-time performance and accuracy of the algorithm meet the safety requirements of rail vehicles.