How is the Pilot Doing: VTOL Pilot Workload Estimation by Multimodal Machine Learning on Psycho-physiological Signals
0
Sign in to get full access
Overview
- This paper explores using multimodal machine learning on psycho-physiological signals to estimate the workload of pilots operating Vertical Take-Off and Landing (VTOL) aircraft.
- The researchers conducted a user study where participants flew a VTOL simulator while their physiological data, such as heart rate and brain activity, were recorded.
- Machine learning models were then trained on this data to predict the pilots' workload levels during the simulated flights.
Plain English Explanation
The main idea of this research is to find ways to understand how much mental effort and stress pilots are experiencing when flying VTOL aircraft. VTOL planes, like those used in air taxis, can be challenging to fly because the pilot has to control both vertical lift and forward motion. The researchers wanted to see if they could use sensors that measure things like heart rate and brain activity to estimate the pilot's workload, or how hard they are working.
To do this, they had people fly a VTOL simulator while wearing sensors. The researchers then trained machine learning models to look for patterns in the sensor data that matched up with the pilots' reported workload levels. The goal was to develop a system that could automatically detect when a pilot is under a lot of stress or mental strain during a real VTOL flight, which could help improve safety and performance.
Technical Explanation
The researchers conducted a user study where participants flew a VTOL aircraft simulator while their physiological data was recorded. This included measures of heart rate, skin conductance, pupil dilation, and brain activity. Participants also self-reported their workload levels at various points during the simulated flights.
The researchers then used this dataset to train and evaluate several machine learning models, including decision trees, support vector machines, and neural networks. The goal was to develop a system that could accurately predict the pilot's workload based on the multimodal psycho-physiological signals.
The results showed that the machine learning models were able to estimate the pilots' workload levels with reasonable accuracy. The best-performing model achieved an R-squared value of 0.73, indicating it could explain a significant portion of the variation in workload.
Critical Analysis
The researchers acknowledge several limitations and caveats to their work. For example, the study was conducted in a simulated environment, which may not fully capture the stresses and demands of real-world VTOL flight. Additionally, the sample size was relatively small, and the participants were mostly young, healthy individuals, which may limit the generalizability of the findings.
Another potential issue is the reliance on self-reported workload data, which can be subjective and influenced by various factors. It would be valuable to explore ways to validate the workload estimates against objective performance metrics or expert assessments.
Conclusion
This research demonstrates the potential of using multimodal machine learning on psycho-physiological signals to estimate VTOL pilot workload. If further developed and validated, such a system could help improve the safety and performance of VTOL aircraft operations by providing real-time feedback on the pilot's cognitive state. However, more research is needed to address the limitations and expand the scope of this approach.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
How is the Pilot Doing: VTOL Pilot Workload Estimation by Multimodal Machine Learning on Psycho-physiological Signals
Jong Hoon Park, Lawrence Chen, Ian Higgins, Zhaobo Zheng, Shashank Mehrotra, Kevin Salubre, Mohammadreza Mousaei, Steven Willits, Blain Levedahl, Timothy Buker, Eliot Xing, Teruhisa Misu, Sebastian Scherer, Jean Oh
Vertical take-off and landing (VTOL) aircraft do not require a prolonged runway, thus allowing them to land almost anywhere. In recent years, their flexibility has made them popular in development, research, and operation. When compared to traditional fixed-wing aircraft and rotorcraft, VTOLs bring unique challenges as they combine many maneuvers from both types of aircraft. Pilot workload is a critical factor for safe and efficient operation of VTOLs. In this work, we conduct a user study to collect multimodal data from 28 pilots while they perform a variety of VTOL flight tasks. We analyze and interpolate behavioral patterns related to their performance and perceived workload. Finally, we build machine learning models to estimate their workload from the collected data. Our results are promising, suggesting that quantitative and accurate VTOL pilot workload monitoring is viable. Such assistive tools would help the research field understand VTOL operations and serve as a stepping stone for the industry to ensure VTOL safe operations and further remote operations.
Read more6/11/2024
0
Vertiport Terminal Scheduling with Multiple Surface Directions along with Throughput Analysis
Ravi Raj Saxena, T. V. Prabhakar, Joy Kuri, Manogna Yadav
Vertical Take-Off and Landing (VTOL) vehicles have gained immense popularity in the delivery drone market and are now being developed for passenger transportation in urban areas to efficiently enable Urban Air Mobility (UAM). UAM aims to utilize the urban airspace hidetxt{vertical dimension} to address the problem of heavy road congestion in dense urban cities. VTOL vehicles require vertiport terminals for landing, take-off, passengers boarding or deboarding, refuelling (or charging), and maintenance. An efficient scheduling algorithm is essential to maximize the throughput of the vertiport terminal (vertiminal)hidetxt{ as well as efficient use of airspace} while maintaining safety protocols to handle the UAM traffic. While traditional departure and taxiing operations can be applied in the context of vertiminal, specific algorithms are required for take-off and landing schedules. Unlike fixed-wing aircraft that require a runway to take-off and climb in a single direction, VTOL vehicles can approach and climb in several directions. We propose a Mixed Integer Linear Program (MILP) formulation to schedule flights for taxiing, climbing (or approaching) using multiple directions after take-off (before landing) and turnaround on gates. We also derived equations to thoroughly analyze the throughput capacity of a vertiminal considering all its core elements. We have shown that our MILP can achieve the maximum throughput obtained through the equations. Given the input parameters, our analysis can be used to analyze the capacity of a vertiminal without running any simulation, while our MILP can be used to get the most efficient schedule.
Read more8/6/2024
๐
0
Performance Level Evaluation Model based on ELM
Qian Mei
Human factor evaluation is crucial in designing civil aircraft cockpits. This process relies on the physiological and cognitive characteristics of the flight crew to ensure that the cockpit design aligns with their capabilities and enhances flight safety. Modern physiological data acquisition and analysis technology, developed to replace traditional subjective human evaluation, has become an effective method for verifying and evaluating cockpit human factors design. Given the high-dimensional and complex nature of pilot physiological signals, these uncertainties significantly impact pilot performance. This paper proposes a pilot performance evaluation model based on an Extreme Learning Machine (ELM) to predict flight performance through pilots' physiological signals and further explores the quantitative relationship between human factors and civil aviation safety.
Read more9/4/2024
๐งช
0
A Multimodal Learning-based Approach for Autonomous Landing of UAV
Francisco Neves, Lu'is Branco, Maria Pereira, Rafael Claro, Andry Pinto
In the field of autonomous Unmanned Aerial Vehicles (UAVs) landing, conventional approaches fall short in delivering not only the required precision but also the resilience against environmental disturbances. Yet, learning-based algorithms can offer promising solutions by leveraging their ability to learn the intelligent behaviour from data. On one hand, this paper introduces a novel multimodal transformer-based Deep Learning detector, that can provide reliable positioning for precise autonomous landing. It surpasses standard approaches by addressing individual sensor limitations, achieving high reliability even in diverse weather and sensor failure conditions. It was rigorously validated across varying environments, achieving optimal true positive rates and average precisions of up to 90%. On the other hand, it is proposed a Reinforcement Learning (RL) decision-making model, based on a Deep Q-Network (DQN) rationale. Initially trained in sumlation, its adaptive behaviour is successfully transferred and validated in a real outdoor scenario. Furthermore, this approach demonstrates rapid inference times of approximately 5ms, validating its applicability on edge devices.
Read more5/22/2024