This was a team project done under supervision of Technische Hochschule University, Ingolstadt. The goal was to perform late-fusion on radar & camerea detections.

This was really fun to work on - got to learn about nitty-gritties of model training, evaluation and spatial fusion of detections. I would like to highlight a few sections here from the project.

Github Repo

Image Augmentation:

Primary problem when it comes to working on road user data is high levels of data imbalance among different classes, we were working with 6 road user classes with the following distribution.

Tried a bunch of different techniques for image augmentation, one of the most interesting was Image Inpainting.

Model Fine-Tuning:

Used Ultralytics library along with YOLOv8-large for fine tuning on the INFRA-3DRC dataset

Sensor Fusion:

Performed spatial fusion of detection objects from two different sensors: Radar & Camera. As seen below, the box is detection result from the camera (YOLO v8) and the red dot represents the centroid of cluster (detection object from radar).