An App That Recognises Birds - UPDATE: Video clip available

Researchers at Chemnitz University of Technology have developed an AI-based app that recognises bird songs

(Update: 25 Juli 2019: Video statement added (Categorie "Multimedia")

Multimedia: An assessment of how the app works and the result leading to this app by Stefan Kahl (M.Sc.) is available as part of the "Statements From Research" series on the Chemnitz University of Technology YouTube channel.

Original article (2 July 2019)

Analysing the sounds that birds make has become much easier for ornithologists and bird enthusiasts alike. A mobile app for the recording and recognition of the different sounds that birds make, developed at Chemnitz University of Technology, is finished and can be downloaded from the Google Play Store.

According to the latest figures, around 180,000 people are currently using the app, which Stefan Kahl, a research associate at the Professorship of Media Informatics (chaired by Prof. Dr. Maximilian Eibl) at Chemnitz University of Technology, and his team have developed. The researchers from Chemnitz note around 3,000 new users and 30,000 observations per average day. This means that users transfer their audio data to the application’s server, where it is analysed by the so-called “BirdNET” artificial intelligence programme (developed at the Professorship of Media Informatics), and the corresponding bird species is identified.

In addition to Stefan Kahl, researchers from Hochschule Mittweida (Prof. Dr. Marc Ritter) and Cornell University in Ithaca, New York (Prof. Holger Klinck) were also involved in the development of the app.

The European Social Fund (ESF) supported the project over a period of three years with a total of 57,600 euros. In addition, the project was supported by the Professorship of Media Informatics at Chemnitz University of Technology and the Junior Professorship of Media Computing (JP Dr. Danny Kowerko) at Chemnitz University of Technology, along with various companies from the region.

Better data thanks to “citizen science” - data collection in cooperation with Tierpark Chemnitz

Several gigabytes of audio data has been fed into the system. “We use the data supplied by our users to improve our system. Building such a large dataset is only possible with the help of app users, because that is the only way to get pictures from different devices, places, and soundscapes,” Stefan Kahl explains. “The ‘citizen science’ approach which his app is based on means that users can actively participate in the research and development process themselves.

Another component of the “citizen science” approach is the cooperation that the programme has with Tierpark Chemnitz. “We have four passive recorders mounted at the zoo that are in use around the clock, and they record audio data that is used for later analysis. The zoo is primarily concerned with the question of which species live freely at the zoo, where they are concentrated, and how this data can be used in a pedagogically valuable way,” explains Kahl.

Cross-border success

Most enquiries come to the team by way of Germany, the United Kingdom, France and the USA, where apparently the largest user base can be found. “We are very happy that, with the release of the final version of the app, we will be making an important contribution to the discipline of ornithology, which will enable researchers to more effectively evaluate their data. Above all, the cross-border success of the app, even prior to its release, shows us the great level of interest that comes with this app,” says Kahl. Due to high demand for the app, Kahl and his team also created a special version of the app in French.

Bird sounds on the app, thanks to AI support

The background of this Chemnitz research project is very simple: for the ongoing protection of wild species such as birds, constant monitoring is essential. However, wild animals can only be permanently observed by humans under certain conditions. As a rule, therefore, image and sound recorders which record long-term data are used. However, this data must first be collected and evaluated, and doing so with wild animals usually brings high costs.

In order to obtain the necessary data for the app, Stefan Kahl and his team compiled the largest, completely annotated data set of soundscapes – which posed a special challenge for the researchers. “In addition to bird sounds, many other sounds appear on the recordings such as wind, rain, other animals such as frogs and insects, and of course humans,” says Kahl. In order to filter this data, the team implemented learning methods having to do with artificial neural network learning techniques. The research team trained these networks in advance, with around 50,000 images and over 350 hours of test material from well-known bird species. Currently, the database contains selections of the roughly 500 most common bird species in North America and Europe. The app is currently most widely used in these two regions. Further regions can be added in the future.

This approach also applies in the classroom

In addition, the researchers developed a mobile and completely autonomous monitoring station for the recording of bird sounds that is based on “Raspberry Pi”, a miniature computer device that can be easily and flexibly programmed.

The system is currently operating in the United States, and analyses the data in real time. After its test phase, the system is going to be mass-produced and placed, for example, at schools. “We want to give children and teachers the opportunity to observe their environment, get to know it, and protect it. Along with the collection of ornithological knowledge, this is our overall goal. The stations will function as bird feeding sites, and thus the students will have an opportunity to visit the sites and take care of them,” explains Kahl. The Chemnitz-based researcher is confident that the first installations will become operational at the end of this year.

And more success has come Kahl’s way – last year, he and his team were the sole participants in the “LifeCLEF Bird Recognition Challenge”, and they are now in charge of organising the event. In the truest sense, the research project and the app are off to a fast start.

Background: LifeCLEF Bird Recognition Challenge

The “LifeCLEF Bird Recognition Challenge” is the largest of its kind in the field of bird recognition. It is an international event within the framework of the “Conference and Labs of the Evaluation Forum”[1] , formerly known as the “Cross-Language Evaluation Forum” (CLEF).

Background: Functions of the bird recognition app

• Automatic recognition of bird sounds based on short audio clips.
• Visualisation of the sounds occurring in the environment and the immediate appearance of the bird sounds, thus making it easier to decide whether something is actually heard in the recording.
• The app deduces not only on the basis of audio data, but on the basis of location and date, whether or not a particular species of bird can be heard.
• If a certain species has been detected, you can see details about it.
• Observations can be saved and shared with friends.
• Each observation is registered and used for research purposes.
• The design and the features of the app are open to feedback from users.
• The app is designed to help people better get to know their environment, and ideally, it will help strengthen ecological awareness.

For more information, please contact Stefan Kahl (M. Sc.), Phone +49 371 531-32219, E-mail stefan.kahl@informatik.tu-chemnitz.de. 

(Translation: Jeffrey Karnitz)

Mario Steinebach
02.07.2019

All "University News" articles