Where Computers Consider Human Emotions

In order to make air travel even safer, air traffic controllers’ operating procedure and emotional strain are studied at TU Chemnitz “Virtual Humans” Competency Center

Certainly, everyone who gets on an airplane wants to arrive safely at their destination. Along with pilots, approximately 2,000 German air traffic controllers are responsible for making sure this is possible. They coordinate around three million aircraft movements each year and ensure that planes are operated safely and do not come too close to one another. Because air traffic is continuously growing, the strain on air traffic controllers is also on the rise and they cannot afford any mistakes. Therefore, researchers at the Chemnitz University of Technology “Virtual Humans” Competency Center are working to make air traffic controlling even safer. In the “MACeLot” project, they are studying the cooperative work done by controllers and their strain in the workplace environment. Using various measurement techniques, they can assess the workers’ emotional strain, or stress level. “For example, if an air traffic controller is experiencing too much stress, the assistance system will send a warning to the supervisor on duty,” explains project director Prof. Dr. Guido Brunnett, spokesperson for the “Virtual Humans” Competency Center. Then the supervisor can immediately react and take appropriate action to relieve the controller – for instance by calling in support from neighboring control sectors.

Computers learn to recognize stress

But how can a computer system reliably recognize cognitive and emotional strain? The Chemnitz-based researchers approach the problem from an interdisciplinary standpoint: psychologists research the air traffic controllers’ operating procedures. They measure pupil size using eye-tracking glasses and skin conductance using an armband. Both of these metrics are good indicators of stress. Computer scientists research body posture, facial expressions and language used by the air traffic controllers, in order to infer their emotional states. “We, as humans, often find it difficult to correctly identify the moods of others and we have to rely on our past experiences – and that’s also how we have to train the computer,” says Brunnett. In order to allow the computer to make sound assessments, many examples of emotion-related posture, facial expressions or audio recordings need to be introduced into the program. “Using all of our recorded data, a model to calculate the cognitive and emotional strain was derived and integrated into a prototypical assistance system,” reports Brunnett. This system shows the supervisor the air traffic controllers’ current stress levels and therefore provides him with decision-making support to prevent excessive strain in the future. 

Keeping track of everything in hazardous situations

Through their research, the scientists in Chemnitz have also developed suggestions to redesign air traffic controllers’ working environment. Based on interviews she conducted with controllers, doctoral candidate Linda Pfeiffer has determined how to display all elements in the control system most effectively. “One of the interfaces we recommend allows the user to better keep track of everything in hazardous situations, because it highlights key information,” explains Pfeiffer.

Such studies and recommendations for improvement are also of great interest to project partner DFS Deutsche Flugsicherung GmbH. “Collecting information about the status of the air traffic control team using special sensor technology opens up completely new possibilities to control and reduce work strain,” says Jörg Buxbaum, leader of the DFS’s Air Traffic Management Research Team. Integrating innovations into the everyday routines of air traffic controllers is, however, very difficult, as they are required to pass strict safety tests. Additionally, in this case, personality rights and privacy must be considered. Nevertheless, there is one possible way to profit from research findings in the near future: “To start with, the methods developed can be applied in developments and trainings using a simulator,” says Buxbaum and continues: “We have clear indications that we can better ascertain cognitive and emotional strain with these tools than by simply directly asking the air traffic controllers themselves.”

Keyword: “Virtual Humans” Competency Center

The “Virtual Humans” Competency Center at the Chemnitz University of Technology has been a driving force in “Humans and Technology” research since 2011 and brings together expert knowledge on digital human modelling at the TU.  Competency center researchers work closely together on interdisciplinary topics. „Digital human modeling, which involves an appropriately high degree of complexity, is a central component of emotion-sensitive assistance systems,” explains Brunnett. Additional areas of application include human-machine interaction and the simulation of individual human models in virtual reality. “Humans will remain a central factor in technical advancements, even with the progression of digitalization and mechanization. That’s why we are working to conceptualize our digital human models so that they are capable of recognizing human emotions and can react accordingly,” explains the center director. Digital human models help machines to consider human feelings. https://www.tu-chemnitz.de/forschung/virtual_humans/ (German only)

Background: The German Federal Government sponsored projects from 2015-2018 that confront one of the core challenges of assistance systems: technical systems need to approximate the complexity of human communication, which is determined by emotional and social communication. The “MACeLot” project is one of the sponsored projects and received roughly 1.7 million euros in funding. The Chemnitz-based researchers presented their project results to the general public and representatives of the Federal Ministry of Education and Research in Bonn from January 31 through February 2, 2018.

Project homepage: https://www.tu-chemnitz.de/forschung/virtual_humans/macelot/ (German only)

More information is available from Prof. Dr. Guido Brunnett, phone 0371 531-31533, e-mail guido.brunnett@informatik.tu-chemnitz.de, and Micha Fiedlschuster, phone 0371 531-30302, e-mail micha.fiedlschuster@informatik.tu-chemnitz.de.

(Translation: Sarah Wilson/Alissa Hölzel)

Matthias Fejes
19.02.2018

All "University News" articles