Search

In this paper, we propose and analyze the novel approach for contactless surgery. The base of contactless surgery is the system for motion tracking. For motion tracking, we use a complex stereo vision depth camera system. Once having the patient data, by using the VR techniques and depth tracking cameras, it is possible to analyze and segment the region of interest in pre-operative preparation. Using a depth tracking camera enables an operation to be simulated, and the outcome is viewed and analyzed before the patient undergoes surgery in the same environment as in the real surgery. This enables medical specialists to optimize their surgical approaches. We think specific VR/AR technologies and approaches should be incorporated, such as analyzing 3D workflow and 3D virtual scene so medical specialists would be able to make the most of the benefits during the surgery. A VR-based application is useful due to avoiding any dangers for the patients. Training applications usually have an elevation part for any step of training. However, the main challenge is to produce a step with adequate fidelity to be simulated like what is performing on the patients. The other ability of training and education simulators is preparing a situation for research surgeons to simply practice what has been suggested by other colleagues and even reading from websites.

Full-Text HTML Abstract Full-Text PDF Full-Text XML How to Cite Research Article What is the Future of Minimally Invasive Sinus Surgery: Computer-Assisted Navigation, 3D-Surgical Planner, Augmented Reality in the Operating Room with ‘in the Air’ Surgeon’s Commands as “Biomechanics” of the New Era in Personalized Contactless Hand- Gesture Non-Invasive SurgeonComputer Interaction? Klapan Ivica1, 2, 3, 4*, Duspara Alen6, Majhen Zlatko5, 7, Benić Igor8, Kostelac Milan8, Kubat Goranka9, Berlengi Nedjeljka10, Zemba Mladen10, Žagar Martin11 Author Affiliations Received: July 11, 2019 | Published: July 23, 2019 Corresponding author: Ivica Klapan, Klapan Medical Group Polyclinic, Ilica 191A, HR-10000 Zagreb, Croatia, EU DOI: 10.26717/BJSTR.2019.19.003377 Abstract Purpose: We were focused on the development of personal-3D-navigation system and application of augmented reality in the operating room per viam personalized contactless hand-gesture non-invasive surgeon-computer interaction, with higher intraoperative safety, reduction of operating time, as well as the length of patient postoperative recovery. Methods: Simultaneous use of video image, 3D anatomic fields and navigation in space, with the application of our original special plug-in application for OsiriX platform, enabling users to use LM-sensor as an interface for camera positioning in 3DVR and VE views, and integrating speech recognition as a VC solution, in an original way. Results: Management of image 2D-3D-video-medical documentation, as well as the control of marker- based virtual reality simulation in real time during real operation with per viam our personalized contactless “in the air” surgeon’s commands. Conclusion: The use of modern technologies in head and neck surgery in the last 30 years (e.g., FESS, NESS, and robotic surgery) has enabled surgeons to demonstrate spatial anatomic elements in the operating field, which was quite inconceivable before. This approach has not yet been used in rhinosinusology or otorhinolaryngology, and to our knowledge, not even in general surgery. The question that we have to ask ourselves now is what prerequisites, realizable in the future, should be realized in our “on the fly” gesturecontrolled and incisionless virtual surgical interventions for the eventual utilization to meet the most demanding requirements in the OR?

Objectives: This study presents the use of our original contactless interface as a plug-in application for OsiriX-DICOM-viewer platform using a hardware sensor devicecontroller that supports hand/finger motions as input, with no hand contact, touching, or voice navigation. It would be possible to modify standard surgical parameters in the fly gesture-controlled incisionless surgical interventions. Methods: The accuracy of computer-generated models was analyzed according to T. Galeta/2017. Our original special plug-in- application provided different types of gestures for three-dimensional- virtual reality navigation. Our hardware sensor device controlling the system without touching any other device served as an interface for camera positioning in three- dimensional virtual endoscopy-views. The impression of panoramic three-dimensional volume rendering-viewing was given by pivoting the camera around a focus fixed on the object. Results: This novel technique enables surgeons to get complete and aware orientation in the operative field, where ‘overlapping’ of the real and virtual anatomic models is inevitable. Our human mind and understanding of this new surgery work by creating completely new models of human behavior and understanding spatial relationships, along with devising assessment that will provide an insight into our human nature. Any model and/or virtual model of the surgical field is defined as it actually exists in its natural surroundings. Conclusion: We offer an alternative to closed software systems for visual tracking, with an initiative for developing the software framework that will interface with depth cameras, with a set of standardized methods for medical applications such as hand gestures and tracking, face recognition, navigation, etc. This software should be an open source, operation system agnostic, approved for medical use and independent of hardware. Comparison with previous doctrine in human medicine clearly indicates that both preoperative/intraoperative manipulation with three-dimensional-volume rendering slices of the human anatomy per viam touchless surgical navigation system with simulation of virtual activities has become reality in the operation room.