A modular architecture for an interactive real‐time simulation and training environment for satellite on‐orbit servicing

Maintaining or repairing satellites in orbit is a delicate task that requires expert skills. The planning, training and analysis of on‐orbit servicing (OOS) missions performed by astronauts or through remote operation using a robot is often time consuming and costly. Virtual Reality (VR) enables simulation and training in a flexible and safe environment. This paper describes an interactive real‐time environment that supports a number of OOS tasks within an immersive VR environment. The system simulates the dynamic and kinematic behaviour of satellite components and provides photo‐realistic visualization of satellite parts and the space environment. It integrates user interaction with haptic force feedback through a bi‐manual haptic human machine interface, as well as simulates and interfaces to a humanoid robot for tele‐operation.In order to provide a realistic experience at interactive frame rates, we propose a distributed system architecture, where the load of computing the physics simulation, haptic feedback and visualization of the complex scene is transferred to dedicated machines. The modular architecture is designed to allow the inclusion of further simulation processes. Several mechanisms for reducing the communication traffic have been implemented. This paper gives an overview of the system architecture, outlines the software implementation and documents an evaluation of the real‐time performance of our system in detail. We describe how system performance was measured in terms of simulation timings and distribution load, as well as report on latencies at several stages. Results show that our distributed system is capable of providing visual and haptic feedback at high frame rates required for user interaction with end‐to‐end latencies of less than 8ms and 3ms, respectively.