A Robotic Teleoperation System Enhanced by Augmented Reality for Natural Human–Robot Interaction

Telekinesis, as commonly portrayed in science fiction literature and cinema, is a super power wherein users control and manipulate objects absent in physical interaction. In real world, enhancing human–robot interaction needs the synthesis of human intuitive processes with robotic arm. This paper introduces a robotic teleoperation system achieving the essence of telekinetic operations, combining the profound capabilities of augmented reality (AR) with the robotic arm operations. Utilizing AR, the proposed methodology offers operators with a visual feedback, facilitating a level of control surpassing the capacities of natural interfaces. By using AR-driven visual recognition, this system achieves operations in a virtual environment, subsequently actualized in the real world through the robotic arm. Through multiple experiments, we found that the system has a small margin of error in telekinesis operations, meeting the needs of remote operation. Furthermore, our system can operate on objects in the real world. These experiments underscore the capability of the remote control system to assist humans in accomplishing a wider range of tasks through the integration of AR and robotic arms, providing a natural human–robot interaction approach.