This project focuses on improving the accuracy of object recognition and situational awareness of connected and autonomous vehicles by having multiple vehicles and smart street IoTs (like smart lights and smart intersections) combine their sensor data through a mobile edge computing (MEC) node located at a nearby roadside unit (RSU). A vehicle, even one with the most sophisticated array of sensors, can only perceive a limited area around itself; this vision is further impacted by possible adverse weather and light conditions and occlusions created by the presence of other road users, like other vehicles and pedestrians, as well as buildings and road infrastructure. To address the above challenge, we are developing a novel collaborative vehicular vision approach, where multiple vehicles (and smart street IoTs) share their sensor data at a RSU MEC node, and through multi-modal data fusion and analytics, a constantly updated representation of the environment surrounding the vehicles can be created and used by the vehicles to better recognize the objects in its surroundings.

This shared and timely environment representation aims to offer connected and autonomous vehicles a more comprehensive understanding of both close and distant road conditions, thus improving the safety and efficiency of driving. In the first phase of the project, we are using camera, GPS, and IMU sensor data shared by vehicles. Subsequently, we plan to use other types of vehicular sensor data, like radar, lidar, and data from other smart street IoTs.

An example scenario highlighting the advantage of such a system is depicted in the figure below. The figure shows RSUs consisting of small cells and MECs covering a set of roads and vehicles. In particular, we can see vehicle 1 and vehicle 2 in the area covered by RSU3 (SC/MEC-3) sharing their data at MEC3. Vehicle 1 can detect a set of objects in front of itself that are too far away from or are being occluded from vehicle 2. Through collaborative vision at the MEC node, vehicle 2 will now be informed about the location of these objects (shown by the dotted arrows). While this example scenario shows two vehicles, our project will enable MEC-based collaborative vehicular vision between multiple vehicles and smart street IoTs, extending the benefit beyond data sharing between two vehicles.