How thermal imaging is contributing to the development of a new generation of nighttime pedestrian detection systems

Road user safety has always been a major concern for governments. Over the last few years, changes in international safety standards and regulations have focused on offering greater protection for vulnerable road users like pedestrians, cyclists, and motorcyclists.

Every year, more than 1.35 million people lose their lives on the road, 54% of whom are vulnerable road users. And these accidents occur more often at night and in the winter, when visibility is poor (fog, rain, etc.).

Although huge progress has recently been made in pedestrian protection, testing conditions for advanced driver assistance systems do not take low light, total darkness or glare into account. Under these conditions, the gaps left by existing systems can be filled by thermal imaging.

So, what are the limitations of existing systems and what can thermal imaging bring to the table? Let’s take a look.

• The existing solutions and their limits

Perception in and around a vehicle is a major issue for the transport market. Visible CMOS imaging and radar are the most used technologies. They receive the most investment and their ecosystems are the most developed and best suited to the high market demand.

Only the latest studies have shown that these technologies, which are highly effective in daytime and under ideal driving conditions, perform far less well at detecting pedestrians in low light or when it’s raining or foggy.

The Austrian and Swiss organizations ETSC (European Transport Safety Council), KFV (Kuratorium für Verkehrssicherheit) and BFU (Beratungsstelle für Unfallverhütung) carried out over 200 tests to better integrate real world driving conditions such as rain and foggy weather and poor light conditions at night. The authors of these studies confirmed that in bright, daytime conditions, detection and emergency braking systems work very well. However, the results of tests conducted when visibility was less favorable are much poorer. In rain, fog or low light, these systems do not always identify vulnerable road users and do not trigger the emergency braking system.

In partnership with the Automobile Club of Southern California's Automotive Research Center, AAA evaluated the performances of four vehicles equipped with an automatic emergency braking system with pedestrian detection in order to determine its efficacy. The tests were carried out on a closed circuit using simulated pedestrian targets for the following scenarios. The daytime scenarios generally produced satisfying results. However, none of the systems detected or reacted to pedestrians at night.

The US Insurance Institute for Highway Safety also ran a nighttime test in response to the high percentage of pedestrian accidents that occur on dark roads. Only 4 of the first 23 cars tested obtained the highest superior score, but over half obtained a low score.

"As we were expecting, most of these AEB (Automatic Emergency Braking) systems for pedestrians do not work well in the dark,” stated the IIHS president, David Harkey. "

It should be noted that an advanced or superior score for the nighttime test is set to become a requirement for ther TOP SAFETY PACK + rating in the years to come. 

These images show a pedestrian at distances of 20 m to 90 m from a vehicle, on the side of a road that is only lit by dimmed car headlights.  On the left is the visible camera vision and on the right is the thermal imaging vision. At a distance of over 20 m, the visible camera does not effectively detect the pedestrian, whereas the thermal camera can detect them up to 90 m away, which is well beyond the reach of the headlights.

A pedestrian walking at night on a country road. On the left, in the visible camera image, the camera is dazzled by the headlights of the car coming towards it. On the right, on the thermal image, the pedestrian is perfectly visible and the light of the vehicle headlights does not affect the thermal image.

• Thermal imaging, a vital addition to current systems

Current technology may display some limits, but thermal imaging can help to extend the performance of automatic emergency braking (AEB) in poorer visibility.

Thermal or LWIR cameras capitalize on the fact that all objects emit thermal energy, which eliminates the need for a light source. They also provide additional data to visible cameras and radar, day and night, without being affected by the sun, headlight glare from other vehicles, and most darkening effects.

This means that thermal imaging can detect and categorize vulnerable road users in the dark and is not affected by ambient light conditions.

Thermal imaging is largely appreciated on the one hand for providing data redundancy with current detection systems in good light and on the other, providing additional information in poor visibility, particularly at night when visible cameras can be limited, while also maintaining a good resolution to help hone the radar data. Thermal imaging is therefore a crucial technology for detecting obstacles, automatic braking, and automating vehicles in all conditions.

Certification bodies such as the NCAP (New Car Assessment Program) are integrating ADAS (Advanced Driver Assistance Systems) into their scoring and some governments are legislating for a mandatory inclusion of these systems in all new vehicles. Visible CMOS and radar technologies are already well enough established to meet the needs of ADAS and particularly VRU detection, but we have seen that they also have limitations when visibility is poor due to light or weather conditions.

Thermal cameras remove the majority of these limitations thanks to their capacity to detect light conditions independently, helping emergency braking systems to work however a vehicle is being used.

Download our free case study, which addresses the topic of autonomous emergency braking at night from a theoretical perspective.