Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers
 
research article

A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers

Zuluaga-Gomez, Juan
•
Prasad, Amrutha
•
Nigmatulina, Iuliia
Show more
May 22, 2023
Aerospace

In this paper we propose a novel virtual simulation-pilot engine for speeding up air traffic controller (ATCo) training by integrating different state-of-the-art artificial intelligence (AI)-based tools. The virtual simulation-pilot engine receives spoken communications from ATCo trainees, and it performs automatic speech recognition and understanding. Thus, it goes beyond only transcribing the communication and can also understand its meaning. The output is subsequently sent to a response generator system, which resembles the spoken read-back that pilots give to the ATCo trainees. The overall pipeline is composed of the following submodules: (i) an automatic speech recognition (ASR) system that transforms audio into a sequence of words; (ii) a high-level air traffic control (ATC)-related entity parser that understands the transcribed voice communication; and (iii) a text-to-speech submodule that generates a spoken utterance that resembles a pilot based on the situation of the dialogue. Our system employs state-of-the-art AI-based tools such as Wav2Vec 2.0, Conformer, BERT and Tacotron models. To the best of our knowledge, this is the first work fully based on open-source ATC resources and AI tools. In addition, we develop a robust and modular system with optional submodules that can enhance the system's performance by incorporating real-time surveillance data, metadata related to exercises (such as sectors or runways), or even a deliberate read-back error to train ATCo trainees to identify them. Our ASR system can reach as low as 5.5% and 15.9% absolute word error rates (WER) on high- and low-quality ATC audio. We also demonstrate that adding surveillance data into the ASR can yield a callsign detection accuracy of more than 96%.

  • Details
  • Metrics
Type
research article
DOI
10.3390/aerospace10050490
Web of Science ID

WOS:000995051300001

Author(s)
Zuluaga-Gomez, Juan
•
Prasad, Amrutha
•
Nigmatulina, Iuliia
•
Motlicek, Petr  
•
Kleinert, Matthias
Date Issued

2023-05-22

Publisher

MDPI

Published in
Aerospace
Volume

10

Issue

5

Start page

490

Subjects

Engineering, Aerospace

•

Engineering

•

air traffic controller training

•

simulation-pilot agent

•

bert

•

automatic speech recognition and understanding

•

speech synthesis

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
LIDIAP  
Available on Infoscience
June 19, 2023
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/198447
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés