Fusing LiDAR and Radar Into One Collision Decision
- Raffay Hassan
- Mar 3
- 3 min read
Updated: Mar 30
With the LiDAR running on the Jetson and the radar streaming tracks from the Pi, the next step was getting them talking to each other. Two independent sensors, two separate data streams, and I needed one unified answer: is the path ahead safe or not?
Still all bench testing at this stage sensors on a desk pointed across the room. The goal here was to get the fusion logic, threading model and live GUI working correctly before anything goes near the actual car.
How the Two Devices Connect
The architecture is pretty clean. The Pi runs one Python process that handles all the radar signal processing and streams JSON track data over UDP. The Jetson runs one Python process that reads the LiDAR over serial, receives the radar tracks over the network, fuses them together and renders the GUI.
They communicate over a single UDP socket. The Jetson GUI updates at roughly 9.5 FPS combining both data sources each frame.
Threading Without Blocking the GUI
The fusion script runs four threads LIDAR, radar receiver, camera (added in Part 4), and the Qt GUI thread. Each sensor thread updates shared state protected by a threading.Lock. The GUI polls that shared state every 100ms via a QTimer and renders whatever is current.
The important thing is the GUI thread never waits for sensor data. If the radar hasn't sent anything new in the last 100ms, it just renders the last known radar state. If the LiDAR serial buffer is empty, same deal. The display stays responsive regardless of what the sensors are doing.
Deciding Who Wins When They Disagree
The tricky part of fusion is what to do when two sensors give different answers. I went with worst-case wins whichever sensor reports the more serious condition takes precedence.
def fuse(lidar_level, radar_level):
priority = {"IMMINENT": 3, "CAUTION": 2, "SAFE": 1}
lidar_p = priority.get(lidar_level, 0)
radar_p = priority.get(radar_level, 0)
if lidar_p >= radar_p:
return lidar_level, "LiDAR"
return radar_level, "Radar"
If LiDAR says CAUTION and radar says SAFE, the output is CAUTION from LiDAR. The GUI shows which sensor triggered the alert so you can see at a glance whether it's the geometry sensor or the velocity sensor driving the decision.
This is deliberately conservative. For something that's supposed to stop a car from hitting things, you want to err on the side of stopping.
The Dashboard Layout
he GUI is PyQt5 with pyqtgraph for the plots. I split it into three main areas:
The left panel is the LiDAR bird's-eye view a scatter plot in car frame coordinates with points coloured by zone (red for centre, blue for left, green for right). Two horizontal lines mark the STOP and SLOW thresholds. There's a small rectangle at the origin for the car silhouette.
The centre panel is the radar range-velocity plot. Each confirmed track appears as a dot at its (velocity, range) position, coloured by collision level green, amber or red. TTC is annotated next to each dot.
The right panel has a TTC bar chart for all active tracks and a table below listing range, velocity, TTC and level per track.
At the very bottom there's a single status bar that gives a snapshot of the entire system
Problems That Came Up
False IMMINENT alerts :
Single stray LiDAR points were occasionally triggering red alerts. Fixed with the 3-frame persistence filter and MIN_POINTS_PER_ZONE = 3, both covered in Part 1.
PyQt5 not found in venv:
pip-installed PyQt5 fails on this platform. Covered above with the system package approach.
Pip broken after venv recreation:
Ran pip from inside a directory that had just been deleted, got a confusing FileNotFoundError. The fix is simple: always cd ~ before running pip commands. Obvious in hindsight.
Next Blog is where the camera comes in Arducam IMX477, GStreamer, YOLOv8n on the Jetson GPU, and cross-validated fusion that only triggers an alert when the LiDAR and camera both agree there's something there.
Comments