Search

Introduction: Thermal imaging can be used for the non-invasive detection of blood vessels of the skin. However, mapping the results to the patient currently lacks user-friendliness. Augmented reality may provide a useful tool to superimpose thermal information on the patient., Methods: A system to support planning in reconstructive surgery using a thermal camera was designed. The obtained information was superimposed on the physical object using a Microsoft HoloLens. An RGB, depth, and thermal camera were combined to capture a scene of different modalities and reconstruct a virtual scene in real time. To register the different cameras and the AR device, an active calibration target was developed and evaluated. A Vuforia marker was used to register the hologram in the virtual space. The accuracy of the projected hologram was evaluated in a laboratory setting with participants by measuring the error between the physical object and the hologram., Results: The AR-based system was evaluated by 21 participants in a laboratory setting. The mean projection error is 10.3 ± 9.4 mm. The system is able to stream a three-dimensional scene with augmented thermal information in real time at 5 frames per second. The active calibration target can be used independently of the environment., Conclusion: The calibration target provides an easy-to-use method for the registration of cameras capturing the visible to long-infrared spectral range. The inside-out tracking of the HoloLens in combination with a Vuforia marker is not accurate enough for the intended clinical use., (© 2024. CARS.)

Blood is the most encountered type of biological evidence in violent crimes and contains pertinent information to a forensic investigation. The false presumption that blood encountered at a crime scene is human may not be realised until after costly and sample-consuming tests are performed. To address the question of blood origin, the novel application of visible-near infrared hyperspectral imaging (HSI) is used for the detection and discrimination of human and animal bloodstains. The HSI system used is a portable, non-contact, non-destructive method for the determination of blood origin. A support vector machine (SVM) binary classifier was trained for the discrimination of bloodstains of human (n = 20) and five animal species: pig (n = 20), mouse (n = 16), rat (n = 5), rabbit (n = 5), and cow (n = 20). On an independent test set, the SVM model achieved accuracy, precision, sensitivity, and specificity values of 96, 97, 95, and 96%, respectively. Segmented images of bloodstains aged over a period of two months were produced, allowing for the clear visualisation of the discrimination of human and animal bloodstains. The inclusion of such a system in a forensic investigation workflow not only removes ambiguity surrounding blood origin, but can potentially be used in tandem with HSI bloodstain age determination methods for rapid on-scene forensic analysis., (© 2023. The Author(s).)

Significance: Multispectral imaging (MSI) is an approach for real-time, quantitative, and non-invasive tissue perfusion measurements. Current laparoscopic systems based on mosaic sensors or filter wheels lack high spatial resolution or acceptable frame rates., Aim: To develop a laparoscopic system for MSI-based color video and tissue perfusion imaging during gastrointestinal surgery without compromising spatial or temporal resolution., Approach: The system was built with 14 switchable light-emitting diodes in the visible and near-infrared spectral range, a 4K image sensor, and a 10 mm laparoscope. Illumination patterns were created for tissue oxygenation and hemoglobin content monitoring. The system was calibrated to a clinically approved laparoscopic hyperspectral system using linear regression models and evaluated in an occlusion study with 36 volunteers., Results: The root mean squared errors between the MSI and reference system were 0.073 for hemoglobin content, 0.039 for oxygenation in deeper tissue layers, and 0.093 for superficial oxygenation. The spatial resolution at a working distance of 45 mm was 156    μ m . The effective frame rate was 20 fps., Conclusions: High-resolution perfusion monitoring was successfully achieved. Hardware optimizations will increase the frame rate. Parameter optimizations through alternative illumination patterns, regression, or assumed tissue models are planned. Intraoperative measurements must confirm the suitability during surgery., (© 2023 The Authors.)

Introduction: Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA., Material and Methods: The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works., Results: The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method., Conclusions: Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.

Background: Hyperspectral Imaging (HSI) is a reliable and safe imaging method for taking intraoperative perfusion measurements. This is the first study translating intraoperative HSI to an in vivo laparoscopic setting using a CE-certified HSI-system for minimally invasive surgery (HSI-MIS). We aim to compare it to an established HSI-system for open surgery (HSI-Open)., Methods: Intraoperative HSI was done using the HSI-MIS and HSI-Open at the Region of Interest (ROI). 19 patients undergoing gastrointestinal resections were analyzed in this study. The HSI-MIS-acquired images were aligned with those from the HSI-Open, and spectra and parameter images were compared pixel-wise. We calculated the Mean Absolute Error (MAE) for Tissue Oxygen Saturation (StO 2 ), Near-Infrared Perfusion Index (NIR-PI), Tissue Water Index (TWI), and Organ Hemoglobin Index (OHI), as well as the Root Mean Squared Error (RMSE) over the whole spectrum. Our analysis of parameters was optimized using partial least squares (PLS) regression. Two experienced surgeons carried out an additional color-change analysis, comparing the ROI images and deciding whether they provided the same (acceptable) or different visual information (rejected)., Results: HSI and subsequent image registration was possible in 19 patients. MAE results for the original calculation were StO 2 orig. 17.2% (± 7.7%) , NIR-PI orig.  16.0 (± 9.5), TWI orig.  18.1 (± 7.9), OHI orig.  14.4 (± 4.5). For the PLS calculation, they were StO 2 PLS  12.6% (± 5.2%), NIR-PI PLS  10.3 (± 6.0), TWI PLS  10.6 (± 5.1), and OHI PLS  11.6 (± 3.0). The RMSE between both systems was 0.14 (± 0.06). In the color-change analysis; both surgeons accepted more images generated using the PLS method., Conclusion: Intraoperative HSI-MIS is a new technology and holds great potential for future applications in surgery. Parameter deviations are attributable to technical differences and can be reduced by applying improved calculation methods. This study is an important step toward the clinical implementation of HSI for minimally invasive surgery., (© 2023. The Author(s).)

Background: Recent studies have shown that hyperspectral imaging (HSI) combined with neural networks can detect colorectal cancer. Usually, different pre-processing techniques (e.g., wavelength selection and scaling, smoothing, denoising) are analyzed in detail to achieve a well-trained network. The impact of post-processing was studied less., Methods: We tested the following methods: (1) Two pre-processing techniques (Standardization and Normalization), with (2) Two 3D-CNN models: Inception-based and RemoteSensing (RS)-based, with (3) Two post-processing algorithms based on median filter: one applies a median filter to a raw predictions map, the other applies the filter to the predictions map after adopting a discrimination threshold. These approaches were evaluated on a dataset that contains ex vivo hyperspectral (HS) colorectal cancer records of 56 patients., Results: (1) Inception-based models perform better than RS-based, with the best results being 92% sensitivity and 94% specificity; (2) Inception-based models perform better with Normalization, RS-based with Standardization; (3) Our outcomes show that the post-processing step improves sensitivity and specificity by 6.6% in total. It was also found that both post-processing algorithms have the same effect, and this behavior was explained., Conclusion: HSI combined with tissue classification algorithms is a promising diagnostic approach whose performance can be additionally improved by the application of the right combination of pre- and post-processing.

Background: Colon conduit is an alternative approach to reconstructing the alimentary tract after esophagectomy. Hyperspectral imaging (HSI) has been demonstrated to be effective for evaluating the perfusion of gastric conduits, but not colon conduits. This is the first study to describe this new tool addressing image-guided surgery and supporting esophageal surgeons to select the optimal colon segment for the conduit and anastomotic site intraoperatively., Patients and Methods: Of 10 patients, eight who underwent reconstruction with a long-segment colon conduit after esophagectomy between 01/05/2018 and 01/04/2022 were included in this study. HSI was recorded at the root and tip of the colon conduit after clamping the middle colic vessels, allowing us to evaluate the perfusion and appropriate part of the colon segment., Results: Anastomotic leak (AL) was detected in only one (12.5%) of all the enrolled patients (n = 8). None of the patients developed conduit necrosis. Only one patient required re-anastomosis on postoperative day 4. No patient needed conduit removal, esophageal diversion, or stent placement. There was a change in the anastomosis site to proximal in two patients intraoperatively. There was no need to change the side of colon conduit intraoperatively in any patient., Conclusion: HSI is a promising and novel intraoperative imaging tool to objectively assess the perfusion of the colon conduit. It helps the surgeon to define the best perfused anastomosis site and the side of colon conduit in this type of operation., (© 2023. The Author(s).)

Problem: Similarity measures are widely used as an approved method for spectral discrimination or identification with their applications in different areas of scientific research. Even though a range of works have been presented, only a few showed slightly promising results for human tissue, and these were mostly focused on pathological and non-pathological tissue classification., Methods: In this work, several spectral similarity measures on hyperspectral (HS) images of in vivo human tissue were evaluated for tissue discrimination purposes. Moreover, we introduced two new hybrid spectral measures, called SID-JM-TAN(SAM) and SID-JM-TAN(SCA). We analyzed spectral signatures obtained from 13 different human tissue types and two different materials (gauze, instruments), collected from HS images of 100 patients during surgeries., Results: The quantitative results showed the reliable performance of the different similarity measures and the proposed hybrid measures for tissue discrimination purposes. The latter produced higher discrimination values, up to 6.7 times more than the classical spectral similarity measures. Moreover, an application of the similarity measures was presented to support the annotations of the HS images. We showed that the automatic checking of tissue-annotated thyroid and colon tissues was successful in 73% and 60% of the total spectra, respectively. The hybrid measures showed the highest performance. Furthermore, the automatic labeling of wrongly annotated tissues was similar for all measures, with an accuracy of up to 90%., Conclusion: In future work, the proposed spectral similarity measures will be integrated with tools to support physicians in annotations and tissue labeling of HS images.

Currently, one of the most common causes of death worldwide is cancer. The development of innovative methods to support the early and accurate detection of cancers is required to increase the recovery rate of patients. Several studies have shown that medical Hyperspectral Imaging (HSI) combined with artificial intelligence algorithms is a powerful tool for cancer detection. Various preprocessing methods are commonly applied to hyperspectral data to improve the performance of the algorithms. However, there is currently no standard for these methods, and no studies have compared them so far in the medical field. In this work, we evaluated different combinations of preprocessing steps, including spatial and spectral smoothing, Min-Max scaling, Standard Normal Variate normalization, and a median spatial smoothing technique, with the goal of improving tumor detection in three different HSI databases concerning colorectal, esophagogastric, and brain cancers. Two machine learning and deep learning models were used to perform the pixel-wise classification. The results showed that the choice of preprocessing method affects the performance of tumor identification. The method that showed slightly better results with respect to identifing colorectal tumors was Median Filter preprocessing (0.94 of area under the curve). On the other hand, esophagogastric and brain tumors were more accurately identified using Min-Max scaling preprocessing (0.93 and 0.92 of area under the curve, respectively). However, it is observed that the Median Filter method smooths sharp spectral features, resulting in high variability in the classification performance. Therefore, based on these results, obtained with different databases acquired by different HSI instrumentation, the most relevant preprocessing technique identified in this work is Min-Max scaling.

Background: Hyperspectral imaging (HSI) during surgical procedures is a new method for perfusion quantification and tissue discrimination. Its use has been limited to open surgery due to large camera sizes, missing color video, or long acquisition times. A hand-held, laparoscopic hyperspectral camera has been developed now to overcome those disadvantages and evaluated clinically for the first time., Methods: In a clinical evaluation study, gastrointestinal resectates of ten cancer patients were investigated using the laparoscopic hyperspectral camera. Reference data from corresponding anatomical regions were acquired with a clinically approved HSI system. An image registration process was executed that allowed for pixel-wise comparisons of spectral data and parameter images (StO 2 : oxygen saturation of tissue, NIR PI: near-infrared perfusion index, OHI: organ hemoglobin index, TWI: tissue water index) provided by both camera systems. The mean absolute error (MAE) and root mean square error (RMSE) served for the quantitative evaluations. Spearman's rank correlation between factors related to the study design like the time of spectral white balancing and MAE, respectively RMSE, was calculated., Results: The obtained mean MAEs between the TIVITA® Tissue and the laparoscopic hyperspectral system resulted in StO 2 : 11% ± 7%, NIR PI: 14±3, OHI: 14± 5, and TWI: 10 ± 2. The mean RMSE between both systems was 0.1±0.03 from 500 to 750 nm and 0.15 ±0.06 from 750 to 1000 nm. Spearman's rank correlation coefficients showed no significant correlation between MAE or RMSE and influencing factors related to the study design., Conclusion: Qualitatively, parameter images of the laparoscopic system corresponded to those of the system for open surgery. Quantitative deviations were attributed to technical differences rather than the study design. Limitations of the presented study are addressed in current large-scale in vivo trials., (© 2022. The Author(s).)

Background: Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article., Objective: What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery?, Material and Methods: Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors., Results: In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91., Discussion: This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice., (© 2022. The Author(s), under exclusive licence to Springer Medizin Verlag GmbH, ein Teil von Springer Nature.)

Laparoscopic procedures can be assisted by intraoperative modalities, such as quantitative perfusion imaging based on fluorescence or hyperspectral data. If these modalities are not available at video frame rate, fast image registration is needed for the visualization in augmented reality. Three feature-based algorithms and one pre-trained deep homography neural network (DH-NN) were tested for single and multi-homography estimation. Fine-tuning was used to bridge the domain gap of the DH-NN for non-rigid registration of laparoscopic images. The methods were validated on two datasets: an open-source record of 750 manually annotated laparoscopic images, presented in this work, and in-vivo data from a novel laparoscopic hyperspectral imaging system. All feature-based single homography methods outperformed the fine-tuned DH-NN in terms of reprojection error, Structural Similarity Index Measure, and processing time. The feature detector and descriptor ORB1000 enabled video-rate registration of laparoscopic images on standard hardware with submillimeter accuracy., (© 2022. The Author(s).)

Objectives: Hand-sewn and stapled intestinal anastomoses are both daily performed routine procedures by surgeons. Yet, differences in micro perfusion of these two surgical techniques and their impact on surgical outcomes are still insufficiently understood. Only recently, hyperspectral imaging (HSI) has been established as a non-invasive, contact-free, real-time assessment tool for tissue oxygenation and micro-perfusion. Hence, objective of this study was HSI assessment of different intestinal anastomotic techniques and analysis of patients' clinical outcome., Methods: Forty-six consecutive patients with an ileal-ileal anastomoses were included in our study; 21 side-to-side stapled and 25 end-to-end hand-sewn. Based on adsorption and reflectance of the analyzed tissue, chemical color imaging indicates oxygen saturation (StO 2 ), tissue perfusion (near-infrared perfusion index [NIR]), organ hemoglobin index (OHI), and tissue water index (TWI)., Results: StO 2 as well as NIR of the region of interest (ROI) was significantly higher in stapled anastomoses as compared to hand-sewn ileal-ileal anastomoses (StO 2 0.79 (0.74-0.81) vs. 0.66 (0.62-0.70); p<0.001 NIR 0.83 (0.70-0.86) vs. 0.70 (0.63-0.76); p=0.01). In both groups, neither anastomotic leakage nor abdominal septic complications nor patient death did occur., Conclusions: Intraoperative HSI assessment is able to detect significant differences in tissue oxygenation and NIR of hand-sewn and stapled intestinal anastomoses. Long-term clinical consequences resulting from the reduced tissue oxygenation and tissue perfusion in hand-sewn anastomoses need to be evaluated in larger clinical trials, as patients may benefit from further refined surgical techniques., Competing Interests: Competing interests: Authors state no conflict of interest., (© 2022 the author(s), published by De Gruyter, Berlin/Boston.)

Anastomotic insufficiencies still represent one of the most severe complications in colorectal surgery. Since tissue perfusion highly affects anastomotic healing, its objective assessment is an unmet clinical need. Indocyanine green-based fluorescence angiography (ICG-FA) and hyperspectral imaging (HSI) have received great interest in recent years but surgeons have to decide between both techniques. For the first time, two data processing pipelines capable of reconstructing an ICG-FA correlating signal from hyperspectral data were developed. Results were technically evaluated and compared to ground truth data obtained during colorectal resections. In 87% of 46 data sets, the reconstructed images resembled the ground truth data. The combined applicability of ICG-FA and HSI within one imaging system might provide supportive and complementary information about tissue vascularization, shorten surgery time, and reduce perioperative mortality., Competing Interests: The authors declare no conflicts of interest., (© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement.)

Esophageal cancer is the sixth leading cause of cancer-related death worldwide. Histopathological confirmation is a key step in tumor diagnosis. Therefore, simplification in decision-making by discrimination between malignant and non-malignant cells of histological specimens can be provided by combination of new imaging technology and artificial intelligence (AI). In this work, hyperspectral imaging (HSI) data from 95 patients were used to classify three different histopathological features (squamous epithelium cells, esophageal adenocarcinoma (EAC) cells, and tumor stroma cells), based on a multi-layer perceptron with two hidden layers. We achieved an accuracy of 78% for EAC and stroma cells, and 80% for squamous epithelium. HSI combined with machine learning algorithms is a promising and innovative technique, which allows image acquisition beyond Red-Green-Blue (RGB) images. Further method validation and standardization will be necessary, before automated tumor cell identification algorithms can be used in daily clinical practice., (© 2022. The Author(s).)

Background: A perfusion deficit is a well-defined and intraoperatively influenceable cause of anastomotic leak (AL). Current intraoperative perfusion assessment methods do not provide objective and quantitative results. In this study, the ability of hyperspectral imaging (HSI) to quantify tissue oxygenation intraoperatively was assessed., Methods: 115 patients undergoing colorectal resections were included in the final analysis. Before anastomotic formation, the bowel was extracted and the resection line was outlined and imaged using a compact HSI camera, in order to provide instantaneously quantitative perfusion assessment., Results: In 105 patients, a clear demarcation line was visible with HSI one minute after marginal artery transection, reaching a plateau after 3 min. In 58 (55.2%) patients, the clinically determined transection line matched with HSI. In 23 (21.9%) patients, the clinically established resection margin was entirely within the less perfused area. In 24 patients (22.8%), the HSI transection line had an irregular course and crossed the clinically established resection line. In four cases, HSI disclosed a clinically undetected lesion of the marginal artery., Conclusions: Intraoperative HSI is safe, well reproducible, and does not disrupt the surgical workflow. It also quantifies bowel surface perfusion. HSI might become an intraoperative guidance tool, potentially preventing postoperative complications.

Innovations and new advancements in intraoperative real-time imaging have gained significant importance in the field of gastric cancer surgery in the recent past. Currently, the most promising procedures include indocyanine green fluorescence imaging (ICG-FI) and hyperspectral imaging or multispectral imaging (HSI, MSI). ICG-FI is utilized in a broad range of clinical applications, e.g., assessment of perfusion or lymphatic drainage, and additional implementations are currently investigated. HSI is still in the experimental phase and its value and clinical relevance require further evaluation, but initial studies have shown a successful application in perfusion assessment, and prospects concerning non-invasive tissue and tumor classification are promising. The application of machine learning and artificial intelligence technologies might enable an automatic evaluation of the acquired image data in the future. Both methods facilitate the accurate visualization of tissue characteristics that are initially indistinguishable for the human eye. By aiding surgeons in optimizing the surgical procedure, image-guided surgery can contribute to the oncologic safety and reduction of complications in gastric cancer surgery and recent advances hold promise for the application of HSI in intraoperative tissue diagnostics.

Background: Novel intraoperative imaging techniques, namely, hyperspectral (HSI) and fluorescence imaging (FI), are promising with respect to reducing severe postoperative complications, thus increasing patient safety. Both tools have already been used to evaluate perfusion of the gastric conduit after esophagectomy and before anastomosis. To our knowledge, this is the first study evaluating both modalities simultaneously during esophagectomy., Methods: In our pilot study, 13 patients, who underwent Ivor Lewis esophagectomy and gastric conduit reconstruction, were analyzed prospectively. HSI and FI were recorded before establishing the anastomosis in order to determine its optimum position., Results: No anastomotic leak occurred during this pilot study. In five patients, the imaging methods resulted in a more peripheral adaptation of the anastomosis. There were no significant differences between the two imaging tools, and no adverse events due to the imaging methods or indocyanine green (ICG) injection occurred., Conclusions: Simultaneous intraoperative application of both modalities was feasible and not time consuming. They are complementary with regard to the ideal anastomotic position and may contribute to better surgical outcomes. The impact of their simultaneous application will be proven in consecutive prospective trials with a large patient cohort.

Background: Intraoperative ultrasound (iUS), using a navigation system and preoperative magnetic resonance imaging (pMRI), supports the surgeon intraoperatively in identifying tumour margins. Therefore, visual tumour enhancement can be supported by efficient segmentation methods., Methods: A semi-automatic and two registration-based segmentation methods are evaluated to extract brain tumours from 3D-iUS data. The registration-based methods estimated the brain deformation after craniotomy based on pMRI and 3D-iUS data. Both approaches use the normalised gradient field and linear correlation of linear combinations metrics. Proposed methods were evaluated on 66 B-mode and contrast-mode 3D-iUS data with metastasis and glioblastoma., Results: The semi-automatic segmentation achieved superior results with dice similarity index (DSI) values between [85.34, 86.79]% and contour mean distance values between [1.05, 1.11] mm for both modalities and tumour classes., Conclusions: Better segmentation results were obtained for metastasis detection than glioblastoma, preferring 3D-intraoperative B-mode over 3D-intraoperative contrast-mode., (© 2021 John Wiley & Sons Ltd.)

The accurate detection of malignant tissue during colorectal surgery impacts operation outcome. The non-invasive spectral imaging combined with machine learning (ML) methods showed to be promising for tumor identification. However, large spectral range implies large computing time. To reduce the number of features, ML methods (e.g. logistic regression and convolutional neuronal network CNN) were evaluated based on four physiological tissue parameters to automatically classify cancer and healthy mucosa in resected colon tissue. A ROC AUC of 0.81 was achieved with the CNN. This study shows that the use of only specific wavelengths bands can detect cancer.Clinical Relevance- These outcomes support the possibility to automatically classify colon tumor based on physiological parameters calculated using only specific wavelength bands. Hence, future image-guided colorectal surgeries can be performed with real-time multispectral imaging.

Introduction: Restorative proctocolectomy with ileal pouch-anal anastomosis (IPAA) is a challenging operation. Especially the mobilization of the pouch into the pelvis can be complex. Adequate perfusion of the pouch is required for optimal healing and functioning., Methods: With hyperspectral imaging (HSI) wavelengths between 500 and 1,000 nm can be analyzed in addition to visible light and by reflecting patterns. This intraoperative procedure is non-invasive, contact-free, and no contrast medium is needed. Fifteen patients undergoing IPAA were examined prospectively, and the pouch was evaluated by HSI intraoperatively. HSI was measured in standardized fashion at 4 defined locations of the J-pouch. Each measurement took about 10 s. The clinical postoperative course was assessed in all patients and correlated to the intraoperative HSI findings., Results: Mean near-infrared perfusion and oxygenation of patients showed values ≥74% for all defined pouch areas, revealing good blood supply. Three minor anastomotic leaks were detected by standard pouchoscopy in the postoperative course, which could be treated conservatively with endosponge therapy., Conclusion: HSI values of perfusion and oxygenation of the IPAA were high. The leak rate is associated with redo procedures. This is reflected by the current literature and most likely related to the higher complexity of the revisional pouch operation. HSI has proved itself as a quick and effective new intraoperative tool to evaluate pouch perfusion objectively and quantitatively., Competing Interests: All authors have no competing interests, except Hannes Koehler − he is an employee of the company “Diaspective vision,” the manufacturer of the used hyperspectral camera., (Copyright © 2021 by S. Karger AG, Basel.)

There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.

Introduction: Fluorescence angiography (FA) with indocyanine green (ICG) and hyperspectral imaging (HSI) are novel intraoperative visualization techniques in abdominal, vascular and transplant surgery. With the purpose of precision surgery, and in order to increase patient's safety, these new tools aim at reducing postoperative morbidity and mortality. This review discusses and highlights recent developments and the future potential of real-time imaging modalities., Methods: The underlying mechanisms of the novel imaging methods and their clinical impact are displayed in the context of avoiding anastomotic leaks, the most momentous complications in gastrointestinal surgery after oncologic resections., Results: While FA is associated with the admission of a fluorescence agent, HSI is contact-free and non-invasive. Both methods are able to record physiological tissue properties in real-time. Additionally, FA also measures dynamic phenomena. The techniques take a few seconds only and do not hamper the operative workflow considerably. With regard to a potential change of the surgical strategy, FA and HSI have an equal significance. Our own advancements reflect, in particular, the topics of data visualization and automated data analyses together with the implementation of artificial intelligence (AI) and minimalization of the current devices to install them into endoscopes, minimal-invasive and robot-guided surgery., Conclusion: There are a limited number of studies in the field of intraoperative imaging techniques. Whether precision surgery in the "high-tech" OR together with FA, HSI and robotics will result in more secure operative procedures to minimize the postoperative morbidity and mortality will have to be evaluated in future multicenter trials., Competing Interests: Hannes Köhler war bis August 2020 Mitarbeiter der Firma Diaspective Vision. Zurzeit ist er Beschäftigter der Universität Leipzig (ICCAS, Innovation Center Computer Assisted Surgery). Er erhält derzeit keine finanzielle Unterstützung der Firma, somit besteht zurzeit kein Interessenkonflikt. Alle weiteren Autoren geben an, zum gegenwärtigen Zeitpunkt ebenso keine Interessenkonflikte zu haben., (Thieme. All rights reserved.)

The surgical treatment of gastric cancer has arrived at a turning point towards the routine application of minimally invasive techniques. After the first results of prospective randomized trials from Asia confirmed the surgical and oncological safety, the latest results of international trials provided evidence for minimally invasive gastrectomy of advanced gastric cancer in a multimodal setting. A new addition in the field of minimally invasive procedures is robotic-assisted surgical techniques, which have already been implemented for these indications in many centers in Germany. The technical advantages that are applicable in the robotics setting in comparison to laparoscopy lead to a rapid dissemination of the procedure but still need to be evaluated in controlled trials. Further developments for the surgical treatment of gastric cancer are found in the field of intraoperative imaging procedures. In this field various technologies are available, such as fluorescence imaging using a near-infrared camera, which requires the use of a fluorescent agent or the hyperspectral camera system, which does not require the application of a fluorophore and merges pictures from visible and non-visible wavelengths to a functional image. It is to be expected that in the future various technological advancements can make a valuable contribution to the surgical treatment of gastric cancer in the clinical routine, especially if they support and facilitate the use of minimally invasive surgical techniques.

Oncologic visceral surgery has recently been revolutionized by robotics, artificial intelligence (AI), sparing of functionally important structures and innovative intraoperative imaging tools. These techniques enable new dimensions of precision surgery and oncology. Currently, data-driven, cognitive operating rooms are standing at the forefront of the latest technical and didactic developments in abdominal surgery. Rectal low anterior resection with total mesorectal excision (TME) for lower- and middle-third rectal cancer is a challenging operation due to the narrow pelvis and the tender guiding structures. Thus, new approaches have been needed to simplify the procedure and to upgrade the results. The combination of robotics with pelvic intraoperative neuromonitoring (pIONM) and new possibilities of visualization, such as multi- and hyperspectral imaging (MSI / HSI) or fluorescence imaging (FI) with indocyanine green (ICG) is a forward-looking modality to enhance surgical precision and reduce postoperative complications while improving oncologic and functional outcomes with a better quality of life. The aim of our video-paper is to show how to achieve maximum precision by combining robotic surgery with pelvic intraoperative neuromonitoring and new imaging devices for rectal cancer.

Currently, colorectal cancer (CRC) is mainly identified via a visual assessment during colonoscopy, increasingly used artificial intelligence algorithms, or surgery. Subsequently, CRC is confirmed through a histopathological examination by a pathologist. Hyperspectral imaging (HSI), a non-invasive optical imaging technology, has shown promising results in the medical field. In the current study, we combined HSI with several artificial intelligence algorithms to discriminate CRC. Between July 2019 and May 2020, 54 consecutive patients undergoing colorectal resections for CRC were included. The tumor was imaged from the mucosal side with a hyperspectral camera. The image annotations were classified into three groups (cancer, CA; adenomatous margin around the central tumor, AD; and healthy mucosa, HM). Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. Hyperspectral imaging combined with automatic classification can be used to differentiate between CRC and healthy mucosa. Additionally, the biological changes induced by chemotherapy to the tissue are detectable with HSI.

Purpose: One relevant aspect for anastomotic leakage in colorectal surgery is blood perfusion of both ends of the anastomosis. The clinical evaluation of this issue is limited, but new methods like fluorescence angiography with indocyanine green or non-invasive and contactless hyperspectral imaging have evolved as objective parameters for perfusion evaluation., Methods: In this prospective, non-randomized, open-label and two-arm study, fluorescence angiography and hyperspectral imaging were compared in 32 consecutive patients with each other and with the clinical assessment by the surgeon. After preparation of the bowel and determination of the surgical resection line, the tissue was evaluated with hyperspectral imaging for 5 min before and after cutting the marginal artery and assessed by 6 hyperspectral pictures followed by fluorescence angiography with indocyanine green., Results: In 30 of 32 patients, the image data could be evaluated and compared. Both methods provided a comparable borderline between well-perfused and poorly perfused tissue (p = 0.704). In 15 cases, the surgical resection line was shifted to the central position due to the imaging. The border zone was sharper in fluorescence angiography and best assessed 31 s after injection. With hyperspectral imaging, the border zone was visualized wider and with more differences between proximal and distal border., Conclusion: Hyperspectral imaging and fluorescence angiography provide similar results in determining the perfusion border. Both methods allow a good and safe visualization of the blood perfusion at the central resection margin to create a well-perfused anastomosis., Trial Registration: This study was registered at Clinicaltrials.gov ( NCT04226781 ) on January 13, 2020.

Background: Thyroidectomy is one of the most commonly performed surgical procedures. The region of the neck has a very complex structural organization. It would be beneficial to introduce a tool that can assist the surgeon in tissue discrimination during the procedure. One such solution is the noninvasive and contactless technique, called hyperspectral imaging (HSI)., Methods: To interpret the HSI data, we implemented a supervised classification method to automatically discriminate the parathyroid, the thyroid, and the recurrent laryngeal nerve from surrounding tissue(muscle, skin) and materials (instruments, gauze). A leave-one-patient-out cross-validation was performed., Results: The best performance was obtained using support vector machine (SVM) with a classification and visualization in less than 1.4 seconds. A mean patient accuracy of 68% ± 23% was obtained for all tissues and material types., Conclusions: The proposed method showed promising results and have to be confirmed on a larger cohort of patient data., (© 2020 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background: Acute mesenteric ischemia is a life-threatening acute condition, which requires an interdisciplinary approach, including vascular recanalization and surgical treatment. Visual evaluation of intestinal perfusion might be misleading, and therefore, additional tools are necessary to reliably be able to resect the ischemic intestine. Hyperspectral imaging (HSI) has been shown to be feasible and safe for real-time assessment of tissue perfusion in visceral surgery but has never been used in cases of acute mesenteric ischemia. Therefore, we applied HSI in acute mesenteric ischemia to evaluate it for potential aid in the objectively discriminating ischemic and well-perfused intestine during explorative laparotomy., Methods: We recorded HSI measurements in 11 cases of acute mesenteric ischemia during explorative laparotomy. We evaluated the recorded images for macroscopic visual perfusion quality and divided it into three groups. Of those three groups, we calculated and compared the HSI indexes of tissue saturation, near-infrared perfusion index, organ hemoglobin index, and tissue water index, as well as the reflectance spectra., Results: We found significant differences in tissue saturation (0.7% versus 0.45%; P = 0.002) and near-infrared perfusion index (0.58 versus 0.23; P < 0.001) in poorly perfused intestinal segments compared with the viable intestine. Furthermore, we could detect an increasing peak at 630 nm of the reflectance spectra in less viable tissues, indicating a maximum in necrotic tissues. We attributed this peak to an increase in met-hemoglobin content in necrotic tissues, which is supported by the increase in the HSI organ hemoglobin index., Conclusions: HSI is able to discriminate tissue perfusion in acute mesenteric ischemia reliably and therefore might be helpful for resection. In addition, HSI gives information on tissue viability via reflectance spectra., (Copyright © 2020. Published by Elsevier Inc.)

Significance: Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups., Aim: An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging., Approach: Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources., Results: The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm)., Conclusions: Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.

Competing Interests: Die Autorinnen/Autoren geben an, dass kein Interessenkonflikt besteht.

HyperSpectral Imaging (HSI) technology enables quantitative tissue analyses beyond the limitations of the human eye. Thus, it serves as a new diagnostic tool for optical properties of diverse tissues. In contrast to other intraoperative imaging methods, HSI is contactless, noninvasive, and the administration of a contrast medium is not necessary. The duration of measurements takes only a few seconds and the surgical procedure is only marginally disturbed. Preliminary HSI applications in visceral surgery are promising with the potential of optimized outcomes. Current concepts, possibilities and new perspectives regarding HSI technology together with its limitations are discussed in this article.

Background: Hyperspectral imaging (HSI) is a relatively new method used in image-guided and precision surgery, which has shown promising results for characterization of tissues and assessment of physiologic tissue parameters. Previous methods used for analysis of preconditioning concepts in patients and animal models have shown several limitations of application. The aim of this study was to evaluate HSI for the measurement of ischemic conditioning effects during esophagectomy., Methods: Intraoperative hyperspectral images of the gastric tube through the mini-thoracotomy were recorded from n = 22 patients, 14 of whom underwent laparoscopic gastrolysis and ischemic conditioning of the stomach with two-step transthoracic esophagectomy and gastric pull-up with intrathoracic anastomosis after 3-7 days. The tip of the gastric tube (later esophagogastric anastomosis) was measured with HSI. Analysis software provides a RGB image and 4 false color images representing physiologic parameters of the recorded tissue area intraoperatively. These parameters contain tissue oxygenation (StO 2 ), perfusion-(NIR Perfusion Index), organ hemoglobin (OHI), and tissue water index (TWI)., Results: Intraoperative HSI of the gastric conduit was possible in all patients and did not prolong the regular operative procedure due to its quick applicability. In particular, the tissue oxygenation of the gastric conduit was significantly higher in patients who underwent ischemic conditioning ([Formula: see text] = 78%; [Formula: see text] = 66%; p = 0.03)., Conclusions: HSI is suitable for contact-free, non-invasive, and intraoperative evaluation of physiological tissue parameters within gastric conduits. Therefore, HSI is a valuable method for evaluating ischemic conditioning effects and may contribute to reduce anastomotic complications. Additional studies are needed to establish normal values and thresholds of the presented parameters for the gastric conduit anastomotic site.

Purpose: Esophageal carcinoma is the eighth most common cancer worldwide. Esophageal resection with gastric pull-up is a potentially curative therapeutic option. After this procedure, the specimen is examined by the pathologist to confirm complete removal of the cancer. An intraoperative analysis of the resectate would be less time-consuming and therefore improve patient safety., Methods: Hyperspectral imaging (HSI) is a relatively new modality, which has shown promising results for the detection of tumors. Automatic approaches could support the surgeon in the visualization of tumor margins. Therefore, we evaluated four supervised classification algorithms: random forest, support vector machines (SVM), multilayer perceptron, and k-nearest neighbors to differentiate malignant from healthy tissue based on HSI recordings of esophago-gastric resectates in 11 patients., Results: The best performances were obtained with a cancerous tissue detection of 63% sensitivity and 69% specificity with the SVM. In a leave-one patient-out cross-validation, the classification showed larger performance differences according to the patient data used. In less than 1 s, data classification and visualization was shown., Conclusion: In this work, we successfully tested several classification algorithms for the automatic detection of esophageal carcinoma in resected tissue. A larger data set and a combination of several methods would probably increase the performance. Moreover, the implementation of software tools for intraoperative tumor boundary visualization will further support the surgeon during oncologic operations.

Purpose: This study evaluated the use of hyperspectral imaging for the determination of the resection margin during colorectal resections instead of clinical macroscopic assessment., Methods: The used hyperspectral camera is able to record light spectra from 500 to 1000 nm and provides information about physiologic parameters of the recorded tissue area intraoperatively (e.g., tissue oxygenation and perfusion). We performed an open-label, single-arm, and non-randomized intervention clinical trial to compare clinical assessment and hyperspectral measurement to define the resection margin in 24 patients before and after separation of the marginal artery over 15 min; HSI was performed each minute to assess the parameters mentioned above., Results: The false color images calculated from the hyperspectral data visualized the margin of perfusion in 20 out of 24 patients precisely. In the other four patients, the perfusion difference could be displayed with additional evaluation software. In all cases, there was a deviation between the transection line planed by the surgeon and the border line visualized by HSI (median 1 mm; range - 13 to 13 mm). Tissue perfusion dropped up to 12% within the first 10 mm distal to the border line. Therefore, the resection area was corrected proximally in five cases due to HSI record. The biggest drop in perfusion took place in less than 2 min after devascularization., Conclusion: Determination of the resection margin by HSI provides the surgeon with an objective decision aid for assessment of the best possible perfusion and ideal anastomotic area in colorectal surgery.

Purpose: Knowing the location of the blood vessels supplying the skin and subcutaneous tissue is a requirement during the planning of tissue transfer in reconstructive surgery. Commonly used imaging techniques such as computed tomography angiography and indocyanine green angiography expose the patient to radiation or a contrast agent, respectively. Infrared thermal imaging was evaluated with success as a non-invasive alternative. To support the interpretation of thermograms, a method to automatically detect the perforators was developed and evaluated., Methods: A system consisting of a thermal camera, a PC and custom software was developed. The temperature variations of the skin surface were analysed to extract the perforator locations. A study was conducted to assess the performance of the algorithm by comparing the detection results of the algorithm with manually labelled thermal images by two clinicians of the deep inferior epigastric perforator flap of 20 healthy volunteers., Results: The F measure, precision and recall were used to evaluate the system performance. The median F measure is 0.833, the median precision is 0.80, and the median recall is 0.907., Conclusion: The results of this study showed that it is possible to automatically and reliably detect the skin perforators in thermograms despite their weak temperature signature. Infrared thermal imaging is a non-invasive and contactless approach suitable for intraoperative use. Combined with a computer-assisted tool for the automatic detection of perforator vessels, it is a relevant alternative intraoperative imaging method to the standard indocyanine green angiography.

Introduction: Anastomotic insufficiency (AI) remains the most feared surgical complication in gastrointestinal surgery, which is closely associated with a prolonged inpatient hospital stay and significant postoperative mortality. Hyperspectral imaging (HSI) is a relatively new medical imaging procedure which has proven to be promising in tissue identification as well as in the analysis of tissue oxygenation and water content. Until now, no data exist on the in vivo HSI analysis of gastrointestinal anastomoses., Methods: Intraoperative images were obtained using the TIVITA™ tissue system HSI camera from Diaspective Vision GmbH (Pepelow, Germany). In 47 patients who underwent gastrointestinal surgery with esophageal, gastric, pancreatic, small bowel or colorectal anastomoses, 97 assessable recordings were generated. Parameters obtained at the sites of the anastomoses included tissue oxygenation (StO 2 ), the tissue hemoglobin index (THI), near-infrared (NIR) perfusion index, and tissue water index (TWI)., Results: Obtaining and analyzing the intraoperative images with this non-invasive imaging system proved practicable and delivered good results on a consistent basis. A NIR gradient along and across the anastomosis was observed and, furthermore, analysis of the tissue water and oxygenation content showed specific changes at the site of anastomosis., Conclusion: The HSI method provides a non-contact, non-invasive, intraoperative imaging procedure without the use of a contrast medium, which enables a real-time analysis of physiological anastomotic parameters, which may contribute to determine the "ideal" anastomotic region. In light of this, the establishment of this methodology in the field of visceral surgery, enabling the generation of normal or cut off values for different gastrointestinal anastomotic types, is an obvious necessity.

Purpose: Complications in wound healing after neurosurgical operations occur often due to scarred dehiscence with skin blood perfusion disturbance. The standard imaging method for intraoperative skin perfusion assessment is the invasive indocyanine green video angiography (ICGA). The noninvasive dynamic infrared thermography (DIRT) is a promising alternative modality that was evaluated by comparison with ICGA., Methods: The study was carried out in two parts: (1) investigation of technical conditions for intraoperative use of DIRT for its comparison with ICGA, and (2) visual and quantitative comparison of both modalities in a proof of concept on nine patients. Time-temperature curves in DIRT and time-intensity curves in ICGA for defined regions of interest were analyzed. New perfusion parameters were defined in DIRT and compared with the usual perfusion parameters in ICGA., Results: The visual observation of the image data in DIRT and ICGA showed that operation material, anatomical structures and skin perfusion are represented similarly in both modalities. Although the analysis of the curves and perfusion parameter values showed differences between patients, no complications were observed clinically. These differences were represented in DIRT and ICGA equivalently., Conclusions: DIRT has shown a great potential for intraoperative use, with several advantages over ICGA. The technique is passive, contactless and noninvasive. The practicability of the intraoperative recording of the same operation field section with ICGA and DIRT has been demonstrated. The promising results of this proof of concept provide a basis for a trial with a larger number of patients.

Purpose: Intraoperative ultrasound (iUS) imaging is commonly used to support brain tumor operation. The tumor segmentation in the iUS images is a difficult task and still under improvement because of the low signal-to-noise ratio. The success of automatic methods is also limited due to the high noise sensibility. Therefore, an alternative brain tumor segmentation method in 3D-iUS data using a tumor model obtained from magnetic resonance (MR) data for local MR-iUS registration is presented in this paper. The aim is to enhance the visualization of the brain tumor contours in iUS., Methods: A multistep approach is proposed. First, a region of interest (ROI) based on the specific patient tumor model is defined. Second, hyperechogenic structures, mainly tumor tissues, are extracted from the ROI of both modalities by using automatic thresholding techniques. Third, the registration is performed over the extracted binary sub-volumes using a similarity measure based on gradient values, and rigid and affine transformations. Finally, the tumor model is aligned with the 3D-iUS data, and its contours are represented., Results: Experiments were successfully conducted on a dataset of 33 patients. The method was evaluated by comparing the tumor segmentation with expert manual delineations using two binary metrics: contour mean distance and Dice index. The proposed segmentation method using local and binary registration was compared with two grayscale-based approaches. The outcomes showed that our approach reached better results in terms of computational time and accuracy than the comparative methods., Conclusion: The proposed approach requires limited interaction and reduced computation time, making it relevant for intraoperative use. Experimental results and evaluations were performed offline. The developed tool could be useful for brain tumor resection supporting neurosurgeons to improve tumor border visualization in the iUS volumes.