The Automation, Robotics & Simulation Division (AR&SD) is developingan approach to operating Robonaut, called telepresence, which usesvirtual reality display technology to visually immerse the operator inthe robot's workspace via views captured from cameras positioned in therobot. This way the teleoperator feels as if he or she is in the placeof the robot. The telepresence system will generate robot positioncommands through teleoperator pose tracking. Head and arm pose sensingwill be based on operator motions as sensed through attached sensors.Finger tracking will be accomplished through glove based finger posesensors. Visual feedback will be provided by a stereo display helmetand will include live video, from the robot head cameras. The operatorwould then perform the arm, head and hand motions for the requiredtasks and a master-slave control mechanism would duplicate the samemotions in the Robonaut. The goal of this project is to provide anintuitive, unobtrusive, accurate and low cost method for trackingthese operator motions and communicating them to the Robonaut system.
AR&SD surveyed a number of potential technologies for tracking thewaist-up posture (together with arm and head positions) of a human.These technologies include: directly placing mechanical, or magneticsensors on the body joints of interest; visual tracking of bodymarkers; non-contact methods such as sonar, structured light, andmachine vision; etc.. There are advantages and disadvantages to all ofthem for telepresence. For example, several of these techniques offerimpressive tracking accuracy. However, very few of these operate fastenough (i.e., in real-time) for teleoperating Robonaut. Also, to meetthe challenges of a space environment and the NASA thrust for lesscostly space systems some additional characteristics in the selectedtechnology are required, such as: highly intuitive and unobtrusiveinterfaces to minimize crew training and system operations, use of COTScomponents, safe operation in a spacecraft, compact stowagerequirements, low power consumption, etc.