Master thesis »Deep Learning-Based Audio Analysis for Avian Population Estimation« in Ilmenau bei Digital Media Technology

The Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft. Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security. At the headquarters, on the campus of “Technische Universität Ilmenau” researchers work on technologies for robust, trustworthy AI-based analysis and classification of audio and video data. These are used, among other things, to monitor industrial production processes, but also in traffic monitoring or in the media context, for example when it comes to automatic metadata extraction and audio manipulation detection. Another focus is the development of algorithms for the areas of virtual product development, intelligent actuator-sensor systems and audio for the automotive sector. There are currently around 70 employees working at Fraunhofer IDMT in Ilmenau.

In the group Semantic Music Technologies at the Fraunhofer IDMT, one of the main research focuses is on extracting meaningful information, identifying patterns, and making sense of complex acoustic recording. For this purpose, methods from audio signal processing and machine learning are often combined.

What you will do

This thesis aims to explore and develop audio analysis methods specifically designed for counting bird populations in outdoor environments. The study will primarily focus on developing deep learning-based audio analysis methods to count distinct bird calls in complex and noisy outdoor soundscapes. These calls may originate from individuals of the same species or from individuals of different species. The ability to detect and count bird calls within a short audio segment enables the derivation of long-term statistics about population density at specific acoustic sensor locations.
Existing bioacoustics research has mainly focused on the task of bird species classification. The most popular model, BirdNet [1], uses a convolutional neural network (CNN) architecture to classify up to 3,000 bird species. Some research exists on animal counting, which employs either computer vision-based methods [2, 3] or audio-based algorithms [4,5]. In previous research at Fraunhofer IDMT, various approaches for polyphony estimation were developed to characterize audio signals. The term “polyphony” can refer to the number of simultaneous pitches [6], the ensemble size in a music recording [7], or the number of audible sound sources in a short environmental audio recording [8].

Objectives

(1) As part of the first objective, the student should investigate whether there exist suitable bird song datasets with polyphony annotations. Alternatively, the student should examine the following approach to dataset creation: After defining a taxonomy of approximately 10-20 common bird species (with assistance from domain experts from an ongoing EU research project), corresponding audio recordings should be collected from the Xeno-Canto platform [9], which houses numerous audio recordings of individual bird calls. These recordings can then be randomly mixed to generate a larger dataset of mixtures, covering different degrees of polyphony and allowing for model training and evaluation.

(2) For the second objective, two deep learning-based approaches should be identified from the literature and re-implemented. Utilizing the previously compiled dataset, models should be trained and evaluated for the task of bird counting. The student should compare two strategies: an explicit counting approach, where the number of birds is predicted directly, and implicit counting approach, where the number of unique species is counted that are classified before by the BirdNet model.

(3) In the third objective, the robustness of the counting method should be tested having a real-world application scenario of a passive acoustic monitoring (PAM) sensor in mind. For this purpose, different data augmentation methods such as the simulation of various microphone characteristics, room impulse responses, and different signal-to-noise levels with background sounds shall be simulated and their effect on the counting accuracy shall be evaluated. Finally, the results should be documented in a written thesis.

References
[1] S. Kahl, C. M. Wood, M. Eibl, H. Klinck, BirdNET: A deep learning solution for avian diversity monitoring, Ecological Informatics, Volume 61, 2021, https://birdnet.cornell.edu/
[2] C. Arteta, V. Lempitsky, A. Zisserman: Count