1. Augmented reality navigation for liver resection with a stereoscopic laparoscope
- Author
-
He Shenghao, Shuhang Liang, Xiao Guo, Linmao Sun, Wei Cai, Lianxin Liu, Shugeng Zhang, Wenyu Zhang, Baochun He, Chihua Fang, Deqiang Xiao, Hongrui Guo, Huoling Luo, Shuxun Liu, Yanfang Zhang, Qingmao Hu, Shuo Zhou, Dalong Yin, Fucang Jia, and Fanzheng Meng
- Subjects
Laparoscopic surgery ,Computer science ,Swine ,medicine.medical_treatment ,Video Recording ,Health Informatics ,Stereoscopy ,030218 nuclear medicine & medical imaging ,Resection ,law.invention ,03 medical and health sciences ,0302 clinical medicine ,Deep Learning ,Imaging, Three-Dimensional ,law ,Neoplasms ,medicine ,Image Processing, Computer-Assisted ,Animals ,Computer vision ,Depth Perception ,Augmented Reality ,business.industry ,Orientation (computer vision) ,Iterative closest point ,Navigation system ,Reproducibility of Results ,Laparoscopes ,Computer Science Applications ,Disease Models, Animal ,Liver ,Surgery, Computer-Assisted ,Augmented reality ,Laparoscopy ,Artificial intelligence ,business ,Tomography, X-Ray Computed ,030217 neurology & neurosurgery ,Algorithms ,Software - Abstract
Objective Understanding the three-dimensional (3D) spatial position and orientation of vessels and tumor(s) is vital in laparoscopic liver resection procedures. Augmented reality (AR) techniques can help surgeons see the patient's internal anatomy in conjunction with laparoscopic video images. Method In this paper, we present an AR-assisted navigation system for liver resection based on a rigid stereoscopic laparoscope. The stereo image pairs from the laparoscope are used by an unsupervised convolutional network (CNN) framework to estimate depth and generate an intraoperative 3D liver surface. Meanwhile, 3D models of the patient's surgical field are segmented from preoperative CT images using V-Net architecture for volumetric image data in an end-to-end predictive style. A globally optimal iterative closest point (Go-ICP) algorithm is adopted to register the pre- and intraoperative models into a unified coordinate space; then, the preoperative 3D models are superimposed on the live laparoscopic images to provide the surgeon with detailed information about the subsurface of the patient's anatomy, including tumors, their resection margins and vessels. Results The proposed navigation system is tested on four laboratory ex vivo porcine livers and five operating theatre in vivo porcine experiments to validate its accuracy. The ex vivo and in vivo reprojection errors (RPE) are 6.04 ± 1.85 mm and 8.73 ± 2.43 mm, respectively. Conclusion and Significance Both the qualitative and quantitative results indicate that our AR-assisted navigation system shows promise and has the potential to be highly useful in clinical practice.
- Published
- 2019