Forsey-Smerek, A., Paige, C., Ward, F., Haddad, D. D., Sanneman, L., Todd, J., ... & Newman, D. (2022, March). Assessment of Depth Data Acquisition Methods for Virtual Reality Mission Operations Support Tools. In 2022 IEEE Aerospace Conference (AERO) (pp. 1-14). IEEE.

Abstract

NASA intends to be back on the Moon within the next two years, and to have long-duration, manned missions to Mars in the late 2030s. These future exploration goals demand a paradigm shift. Mission operational complexity is increasing and - with the development of heavy lift launch capabilities and increased funding of lunar orbital and surface missions - cadence of lunar missions will increase. A sustained human presence on the Moon, and eventually Mars, demands new enabling technologies and capabilities to support in situ resource utilization (ISRU). The development of ISRU technologies requires precursory scientific and prospecting missions to identify and characterize available resources. These missions will employ robotic and human explorers to perform traverses over the lunar surface and collect data to fulfill scientific objectives. The time and monetary resources required to support a mission make maximizing the scientific return of each mission critical. Given the wide range of scientific objectives often found within a mission, the vast scope of diverse expertise within the Earth-located science team will prove invaluable to strategic decision making. Essential to maximizing scientific return on these missions is the ability of the Earth-located science team to be central to rapid science decision making, between and during traverses. Human-computer interaction needs to lead mission planning priorities to enable rapid decision processes. Treating machines as collaboration tools allows for improved cross-team communication, improved decision-making processes, reduced task loads, and flexibility in temporal and spatial planning. Multi-user naturalistic visualization techniques can be used to analyze, discuss, and interpret near-real-time data with the potential to dramatically improve science support room situation awareness, maximizing scientific return on robotic and human exploration missions. The virtual reality Mission Simulation System (vMSS), is a virtual reality platform designed at MIT by the Resource Exploration and Science of our Cosmic Environment (RESOURCE) team, which will provide teams with a collaboration interface for planetary exploration missions. As an early step in development of vMSS, we examine various methods to acquire depth data necessary for development of a high-resolution three-dimensional map of the lunar surface, which will serve as a basis of the platform. In this paper we argue the importance of high-resolution depth data for scientific return, and the limitations of current planetary surface mapping tools using methods such as orbital data and Structure-from-Motion ( $S$ fM) Photogrammetry. We present a comparative analysis of four different methods to achieve depth-mapping using stereo cameras, short-range time-of-flight, LiDAR, and 360° 3D VR imagery. For this analysis, we performed a field experiment with the Boston Dynamics Spot robot, taking advantage of its ability to maneuver in geologically relevant terrain. Finally, we present planned future integration of science analysis tools based on depth imagery into vMSS, with the goal of handling the expected proliferation of real-time science data throughout science and resource prospecting missions.