First MR system for intuitive drone navigation in unknown environments, slashing operator workload and enhancing exploration capabilities.
Abstract
The widespread use of aerial robots in inspection, search and rescue, and monitoring has created a growing need for intuitive human-drone interfaces. These aim to streamline and enhance the user interaction and collaboration process during drone navigation, ultimately expediting mission success and accommodating users’ inputs. In this paper, we present a novel human-drone mixed reality interface that aims to (a) increase human-drone spatial awareness by sharing relevant spatial information and representations between the human equipped with a Head Mounted Display (HMD) and the robot and (b) enable safer and intuitive human-drone interactive and collaborative navigation in unknown environments beyond the simple command and control or teleoperation paradigm. Our framework is validated through extensive user studies and experiments conducted in simulated post-disaster scenarios, with performance compared to traditional First-Person View (FPV) control systems. Multiple tests on several users underscore the advantages of the proposed solution, which offers intuitive and natural interaction with the system. This demonstrates the solution’s ability to assist humans during a drone navigation mission, ensuring its safe and effective execution.
Key contributions
- Bi-directional Spatial Representation: The system allows the drone and the user (wearing a Head-Mounted Display or HMD) to share spatial information continuously. This means the user can see a real-time map of the environment created by the drone, even when the drone is far away. The user’s view is augmented with digital spatial elements.
- Tight Integration of MR and Autonomous Navigation: The MR interface and the drone’s autonomous navigation system are deeply connected. The user can set goals or trajectories for the drone in the MR environment, and the drone can then autonomously navigate to those locations, avoiding obstacles. The system can even re-plan paths if the user’s initial trajectory is not safe.
- Improved User Experience and Efficiency: The paper shows that the MR interface leads to a reduced mental workload for the user compared to a traditional First-Person View (FPV) control system. It also shows that the MR system allows the drone to explore a larger area within a given timeframe.
Method
Explain the method, and add diagrams, figures, videos, or result tables as needed.
Results
Summarize the main findings and link to external resources above.
BibTeX
@inproceedings{salunkhe2025intuitive,
title={Intuitive human-drone collaborative navigation in unknown environments through mixed reality},
author={Salunkhe, Sanket A and Nedunghat, Pranav and Morando, Luca and Bobbili, Nishanth and Li, Guanrui and Loianno, Giuseppe},
booktitle={2025 International conference on unmanned aircraft systems (ICUAS)},
pages={862--868},
year={2025},
organization={IEEE}
}