Artificial intelligence (AI) combined with a novel bio-inspired camera achieves 100-times faster detection of pedestrians and obstacles than current automotive cameras. This significant advancement by researchers at the University of Zurich could greatly enhance the safety of automotive systems and self-driving cars.
A Major Leap in Automotive Safety
Every driver's worst nightmare is a pedestrian stepping out in front of the car unexpectedly, leaving only a fraction of a second to react. While some cars are equipped with camera systems that can alert the driver or activate emergency braking, these systems are not yet fast or reliable enough for autonomous vehicles where no human driver is present.
Enhanced Detection with Reduced Computational Power
Daniel Gehrig and Davide Scaramuzza from the Department of Informatics at the University of Zurich (UZH) have developed a system that combines a novel bio-inspired camera with AI, enabling much quicker detection of obstacles around a car while using less computational power. Their study is published in Nature.
Limitations of Current Camera Systems
Most current automotive cameras are frame-based, capturing 30 to 50 frames per second, and rely on artificial neural networks trained to recognize objects such as pedestrians and other vehicles. However, these systems can miss events occurring in the milliseconds between frames. Increasing the frame rate would generate more data, requiring more real-time processing power.
Innovative Event Cameras
Event cameras operate on a different principle. They have smart pixels that record information whenever they detect fast movements, eliminating blind spots between frames. Known as neuromorphic cameras, they mimic human vision but can miss slow-moving objects and have challenges converting images into data suitable for AI training.
A Hybrid System for Superior Detection
Gehrig and Scaramuzza developed a hybrid system that includes a standard camera capturing 20 images per second and an AI system trained to recognize vehicles and pedestrians. This is paired with an event camera linked to an asynchronous graph neural network, ideal for analyzing 3D data that change over time. The event camera's detections help anticipate and enhance the standard camera's performance.
"The result is a visual detector that matches the speed of a standard camera taking 5,000 images per second but requires the bandwidth of a 50-frame-per-second camera," explains Gehrig.
Superior Performance with Less Data
Testing against current top automotive cameras and algorithms showed the hybrid system achieved 100-times faster detections, reducing data transmission and computational power needs without compromising accuracy. It effectively detects objects entering the field of view between frames, enhancing safety, especially at high speeds.
Future Potential
The researchers believe that integrating this system with LiDAR sensors, similar to those in self-driving cars, could further improve its capabilities. "Hybrid systems like this are crucial for enabling autonomous driving, ensuring safety without a substantial increase in data and computational power," says Scaramuzza.
More: https://techxplore.com/news/2024-05-bio-cameras-ai-drivers-pedestrians.html
