A drone with microphone arrays to hear and help victims in disaster zones

When a natural disaster such as an earthquake occurs, every minute counts. Unmanned aerial vehicles (UAVs) are often used to assist the search for survivors as they can provide an initial overview of difficult-to-reach areas and help to detect victims—provided they are visible.

Researchers at the Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE are now looking to close a gap in the provision of disaster management services with a new technology: In the future, drones equipped with microphone arrays will be able to precisely locate cries for help and other acoustic signals from victims from the air and supply information about their location to the rescue teams. This significantly increases the chances of a rapid rescue for victims who cannot be spotted by camera.

Floods in Libya, Greece and Slovenia, fires in Hawaii and Tenerife, earthquakes in Turkey and Morocco—when a region is hit by a natural disaster, every minute counts in the efforts to save victims. But searching for survivors is complex work as buildings and roads may be damaged and large areas cut off. The use of drones equipped with daylight cameras and thermal imaging cameras is therefore becoming increasingly widespread—they can quickly fly over large areas of ruined infrastructure, locate victims and speed up the response of the rescue teams.

The problem is that victims trapped under rubble cannot be seen by these imaging sensors, and factors such as thick smoke, fog or darkness also limit the effectiveness of the cameras. For scenarios such as these, researchers at Fraunhofer FKIE are working on a solution which enables acoustic sensors to be added to the cameras: LUCY—short for Listening system Using a Crow’s nest arraY—is a piece of technology developed by FKIE scientist Macarena Varela in collaboration with colleagues and research group leader Dr. Marc Oispuu to save the lives of people buried in rubble or trapped by fires.

Microphone array receives signals from all directions

LUCY is an array of MEMS microphones—known as a crow’s nest array—which is mounted on drones in order to determine the direction from which noises such as cries for help, clapping or knocking signals are coming. The tiny, robust MEMS microphones are inexpensive and used in applications such as smartphones.

The special thing about this system is that the microphones are attached to the underside of the drone in a special geometric configuration and can perceive sound from all directions. “The highest lookout point on ships which allows you to see in all directions is known as the crow’s nest. The same goes for LUCY—our system can hear in all directions with virtually no restrictions,” explains Macarena Varela.

LUCY works in a similar way to the human ear, which takes in sound information and conveys it to the brain where it is analyzed. In the case of the array system, the ears are replaced by microphones and the brain is replaced by a signal processing unit which gages the direction from which the noises are coming. LUCY currently features 48 microphones, enabling the direction of the sound source to be determined with excellent precision.

“Spatial hearing obviously works better with 48 or more microphones than with two acoustic sensors, and both targeted hearing in a particular direction and the ability to ignore certain sounds are also improved,” says Dr. Oispuu. Furthermore, the system is able to perceive frequencies which the human ear cannot register. In the future, the number of microphones will be increased to 256 sensors capable of processing signals in real time.

Distracting ambient noises are filtered out

The system blocks out distracting ambient noises such as from rescue equipment, wind or birds, as well as from the whirring rotors of the drone itself. Artificial intelligence (AI) methods and adaptive filters are used to filter out signals, and at the same time the system is taught to detect sound patterns such as shouting, banging or clapping which might be used by people in need to attract attention.

To enable the system to do this, it uses a database of different sounds or signatures that the AI has been trained on in advance. In combination with signal processing techniques such as coherent beamforming, this makes it possible to detect and classify noises and determine their angle of incidence precisely. Furthermore, a compact processing unit ensures that signals can be processed very quickly. When a disaster occurs, the location data that is received will be conveyed to the rescue teams, who can then use tablets, for example, to identify the exact positions of the victims.

Lightweight LUCY

Thanks to their scalability, the sensor modules and microphone arrays can be used on numerous commercially available drones. As both the MEMS technology and the drones are relatively cheap, multiple unmanned aerial vehicles can be used to investigate the disaster zone effectively.

Due to its low weight, emergency responders can carry the LUCY system with them to also use it on the ground, and it can be mounted on vehicles or used as stationary equipment. The FKIE researchers are currently working on further improvements to the experimental system.