Search

The article discusses the met averse services and business models and Met averse architectures. Topics include the ways of social interaction are significantly augmented with persistent virtual content, consumed via VR or AR; met averse is an emerging concept which embraces three important elements, and Experimental demonstrations and prototypes of met averse.

The article reports that The metaverse is an emerging concept which embraces three important elements. First, it embraces a social element, i.e. it is not only a virtual space where users spend time (and money) on their own or with a selected few. Rather, the metaverse is intended to resonate with the very social fabric which underpins human society.

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

The paper provides a high-level review of the current state of techno-pedagogical design in Augmented Reality Learning Experiences (ARLEs). The review is based on a rubric constructed from the Meaningful Learning with ICT framework and the Orchestration Load reduction framework, providing, respectively, a view of primarily student- and primarily teacher-focused dimensions in ARLEs. Fifty-nine ARLEs are reviewed based on an extensive search for papers based on both previous work in the field as well as academic databases. Exemplars for specific situations are provided as well as a correlation analysis between dimensions and an overall view of the maturity of the field in terms of relevant techno-pedagogical considerations, giving ARLE practitioners practical information on the current state of the field as well as assistance in better positioning their future ARLE design efforts. [ABSTRACT FROM PUBLISHER]

With the development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary value of AR is to promote the fusion of digital contents and real-world environments, however, studies on how this fusion will influence the Quality of Experience (QoE) of these two components are lacking. To achieve better QoE of AR, whose two layers are influenced by each other, it is important to evaluate its perceptual quality first. In this paper, we consider AR technology as the superimposition of virtual scenes and real scenes, and introduce visual confusion as its basic theory. A more general problem is first proposed, which is evaluating the perceptual quality of superimposed images, i.e., confusing image quality assessment. A ConFusing Image Quality Assessment (CFIQA) database is established, which includes 600 reference images and 300 distorted images generated by mixing reference images in pairs. Then a subjective quality perception experiment is conducted towards attaining a better understanding of how humans perceive the confusing images. Based on the CFIQA database, several benchmark models and a specifically designed CFIQA model are proposed for solving this problem. Experimental results show that the proposed CFIQA model achieves state-of-the-art performance compared to other benchmark models. Moreover, an extended ARIQA study is further conducted based on the CFIQA study. We establish an ARIQA database to better simulate the real AR application scenarios, which contains 20 AR reference images, 20 background (BG) reference images, and 560 distorted images generated from AR and BG references, as well as the correspondingly collected subjective quality ratings. Three types of full-reference (FR) IQA benchmark variants are designed to study whether we should consider the visual confusion when designing corresponding IQA algorithms. An ARIQA metric is finally proposed for better evaluating the perceptual quality of AR images. Experimental results demonstrate the good generalization ability of the CFIQA model and the state-of-the-art performance of the ARIQA model. The databases, benchmark models, and proposed metrics are available at: https://github.com/DuanHuiyu/ARIQA. [ABSTRACT FROM AUTHOR]

The rejection rates of upper-limb prosthetic devices in adults are high, currently averaging 26% and 23% for body-powered and electric devices, respectively. While many factors influence acceptance, prosthesis training methods relying on novel virtual reality systems have been cited as a critical factor capable of increasing the likelihood of long-term, full-time use. Despite that, these implementations have not yet garnered widespread traction in the clinical setting, and their use remains immaterial. This review aims to explore the reasons behind this situation by identifying trends in existing research that seek to advance Extended Reality “X-Reality” systems for the sake of upper-limb prosthesis rehabilitation and, secondly, analyzing barriers and presenting potential pathways to deployment for successful adoption in the future. The search yielded 42 research papers that were divided into two categories. The first category included articles that focused on the technical aspect of virtual prosthesis training. Articles in the second category utilize user evaluation procedures to ensure applicability in a clinical environment. The review showed that 75% of articles that conducted whole system testing experimented with non-immersive virtual systems. Furthermore, there is a shortage of experiments performed with amputee subjects. From the large-scale studies analyzed, 71% of those recruited solely non-disabled participants. This paper shows that X-Reality technologies for prosthesis rehabilitation of upper-limb amputees carry significant benefits. Nevertheless, much still must be done so that the technology reaches widespread clinical use. [ABSTRACT FROM AUTHOR]

Visual localization enables autonomous vehicles to navigate in their surroundings and augmented reality applications to link virtual to real worlds. Practical visual localization approaches need to be robust to a wide variety of viewing conditions, including day-night changes, as well as weather and seasonal variations, while providing highly accurate six degree-of-freedom (6DOF) camera pose estimates. In this paper, we extend three publicly available datasets containing images captured under a wide variety of viewing conditions, but lacking camera pose information, with ground truth pose information, making evaluation of the impact of various factors on 6DOF camera pose estimation accuracy possible. We also discuss the performance of state-of-the-art localization approaches on these datasets. Additionally, we release around half of the poses for all conditions, and keep the remaining half private as a test set, in the hopes that this will stimulate research on long-term visual localization, learned local image features, and related research areas. Our datasets are available at visuallocalization.net , where we are also hosting a benchmarking server for automatic evaluation of results on the test set. The presented state-of-the-art results are to a large degree based on submissions to our server. [ABSTRACT FROM AUTHOR]

Experiments suggest that using head-up displays like Google Glass to support parts picking for distribution results in fewer errors than current processes. Making Glass opaque instead of transparent further improves selection efficiency. The Web extra at http://youtu.be/yUZFaCP6rP4 is a video demonstrating that order picking assisted by head-up display (HUD) is faster, has fewer errors, requires less workload, and is preferred to order picking assisted by pick-by-light, paper pick list, and cart-mounted display (CMD). The second Web extra at http://youtu.be/RApBJ0U3XpI is a video demonstrating that order picking assisted by a Google Glass with an opaque display is three percent faster than with a transparent display (default). [ABSTRACT FROM PUBLISHER]

In the last years, more and more research activities have focused on the application of Virtual Reality (VR) and related cutting-edge immersive technologies like Augmented Reality (AR) and Mixed Reality (MR) to vehicular systems for supporting the design of novel Human-Machine Interaction techniques, simulating the functioning of newly developed components, recreating hazardous test environments, establishing trust in Connected and Autonomous Vehicles (CAVs), etc. The benefits brought by immersive environments for supporting simulations in the field of vehicular technology have already been recognized by many companies and academic institutions worldwide, as confirmed by the huge investments and a large number of publications. As we approach the time in which immersive simulations will become fundamental for a wide range of applications in the field of vehicular technology, the goal of this Special Section was to attract high-quality submissions reporting state-of-the-art research activities targeted to next-generation vehicles and related infrastructures. The Special Section received 12 submissions, which were prescreened and reviewed by experts in the field. Four papers were ultimately accepted. [ABSTRACT FROM AUTHOR]

The ExtendedHand interface projects a computer graphics (CG) hand that synchronizes with a user’s physical hand movements onto a real environment, visually extending the user’s reach. This paper focuses on enhancing the user’s tactile perception of an object through cross-modal phenomena by providing a sound texture (auditory information that matches the object) when the CG hand touches it. Here, ExtendedHand enables users to touch objects beyond their physical reach, an experience that cannot be achieved with their physical body. In such situations, the appropriateness of adjusting sound pressure based on physical laws according to distance for users is unclear. Additionally, we have empirical knowledge that the speed at which we touch objects with our hands results in different sounds. Within ExtendedHand, since the movement of the user’s physical hand is amplified and reflected in the CG hand’s movement, the physical hand’s speed does not match the CG hand’s speed. This raises the question of whether sound texture feedback should align with the visual information of the CG hand or the proprioceptive sensory information of the physical hand. In this paper, we conducted two user studies to explore appropriate sound texture feedback for the projected CG hand. The results indicate that when the CG hand touches objects at various distances, the sound pressure should follow the same sound pressure attenuation as observed in physical phenomena. Additionally, the results suggest that despite swift tracing actions with the CG hand, users perceive sounds produced at a slower pace to be more suitable.

Six degree-of-freedom (6-DoF) visual tracking of dynamic objects is fundamental to a large variety of robotics and augmented reality (AR) applications. A key to this problem is accurate distance measurement of dynamic objects, which is usually obtained via stereo cameras, RGB-D sensors, or LiDARs. In this paper, however, we address the problem using only a monocular camera rigidly mounted with a low-cost inertial measurement unit. This is a light-weight, small-size, and low-cost solution, which is particularly suitable for tracking dynamic objects on drones or on mobile phones. Starting from a generic image-based two-dimensional tracker, we propose a novel method to resolve the object scale ambiguity in monocular vision in a geometric manner based on correlation analysis. This enables accurate metric three-dimensional tracking of arbitrary objects without requiring any prior knowledge about the object shape or size. We discuss the applicability by analyzing the observability condition and degenerated cases for object scale recovery. Simulation and real-world experimental results with ground truth comparison, along with AR application examples, demonstrate the feasibility of the proposed 6-DoF tracking method. [ABSTRACT FROM AUTHOR]

This paper proposes a weighted large margin nearest center (WLMNC) distance-based human depth recovery method for tele-immersive video interaction systems with limited bandwidth consumption. In the remote stage, the proposed method highly compresses the depth data of the remote human into skeletal block structures by learning the WLMNC distance, which is equivalent to downsampling the human depth map at $64{\times}$ the sampling rate. In the local stage, the method first recovers a rough human depth map based on a WLMNC distance augmented clustering approach and then obtains a fine depth map based on a rough depth-guided autoregressive model to preserve the depth discontinuities and suppress texture copy artifacts. The proposed WLMNC distance is learned by the large margin clustering problem with a weighted hinge loss to balance the clustering accuracy and depth recovery accuracy and is verified to be able to preserve depth discontinuities between skeletal block structures with occlusion. A theoretical analysis is conducted to verify the effectiveness of using the weighted hinge loss. Furthermore, a novel data set containing various types of human postures with self-occlusion is built to benchmark the human depth recovery methods. The quantitative comparison with the state-of-the-art depth recovery methods on the introduced benchmark data set demonstrates the effectiveness of the proposed method for human depth recovery with such a high upsampling rate. [ABSTRACT FROM AUTHOR]

With the full development of intelligent mobile communications, wireless mixed reality (MR) provides a more visually immersive experience and stronger interaction with environments than virtual reality (VR) and augmented reality (AR). However, the asymmetric characteristic of wireless MR traffic creates a huge challenge to current mobile networks. Dynamic time division duplex (D-TDD) is considered as a promising technology to improve wireless MR users’ quality of experience (QoE) due to its potentials and advantages in delivering asymmetric traffic. Therefore, in this paper, we propose a QoE-driven distributed multidimensional resource allocation (MRA) supplemented by inter-cell interference (ICI) mitigation scheme for wireless MR in multi-cell D-TDD systems. First, to improve QoE of MR users, we formulate the joint optimization of subframe configuration, channel assignment and computation offloading as a mixed-integer nonlinear programming problem. A novel fully-decentralized multi-agent deep Q-network (DQN) algorithm is developed to solve the problem. Then, to mitigate ICI, a water filling based power control algorithm is investigated to minimize the total power of each small base station and its associated MR users. Simulation results demonstrate that our proposed scheme improves QoE of MR users in a realizable way as compared to existing schemes. [ABSTRACT FROM AUTHOR]

Medically oriented Brain Computer Interfaces (BCIs) have been proposed as a promising approach addressed to individuals suffering from severe paralysis. Steady-State Visual Evoked Potentials (SSVEPs) in particular have been proven successful in many different applications, achieving high information throughput with short or even no training. However, efficient electric wheelchair navigation combining high accuracy and comfort is still not demonstrated. In this paper, we propose the use of an SSVEP-based universal control system featuring augmented reality (AR) glasses in an attempt to increase ease of use and patient acceptability without making compromises on BCI performance. The system received positive user-feedback, reaching a mean accuracy of 90%. Merits and pitfalls of the system proposed are also addressed.

Multiple patterning lithography has been recognized as one of the most promising solutions, in addition to extreme ultraviolet lithography, directed self-assembly, nanoimprint lithography, and electron beam lithography, for advancing the resolution limit of conventional optical lithography. Multiple patterning layout decomposition (MPLD) becomes more challenging as advanced technology introduces complex coloring rules. Existing works model MPLD as a graph coloring problem; nevertheless, when complex coloring rules are considered, layout decomposition can no longer be modeled accurately by graph coloring. Therefore, in this paper, for capturing the essence of layout decomposition with complex coloring rules, we model the MPLD problem as an exact cover problem. We then propose a fast and exact MPLD framework based on augmented dancing links. Our method is flexible and general: it can consider the basic and complex coloring rules simultaneously, can maintain density balancing, and can handle quadruple patterning and beyond. Experimental results show that our approach outperforms state-of-the-art works on reported conflicts and stitches and is promising for handling complex coloring rules and density balancing as well. [ABSTRACT FROM PUBLISHER]

A holistic understanding of dynamic scenes is of fundamental importance in real-world computer vision problems such as autonomous driving, augmented reality and spatio-temporal reasoning. In this paper, we propose a new computer vision benchmark: Video Panoptic Segmentation (VPS). To study this important problem, we present two datasets, Cityscapes-VPS and VIPER together with a new evaluation metric, video panoptic quality (VPQ). We also propose VPSNet++, an advanced video panoptic segmentation network, which simultaneously performs classification, detection, segmentation, and tracking of all identities in videos. Specifically, VPSNet++ builds upon a top-down panoptic segmentation network by adding pixel-level feature fusion head and object-level association head. The former temporally augments the pixel features while the latter performs object tracking. Furthermore, we propose panoptic boundary learning as an auxiliary task, and instance discrimination learning which learns spatio-temporally clustered pixel embedding for individual thing or stuff regions, i.e., exactly the objective of the video panoptic segmentation problem. Our VPSNet++ significantly outperforms the default VPSNet, i.e., FuseTrack baseline, and achieves state-of-the-art results on both Cityscapes-VPS and VIPER datasets. The datasets, metric, and models are publicly available at https://github.com/mcahny/vps. [ABSTRACT FROM AUTHOR]

Due to the increased popularity of augmented and virtual reality experiences, the interest in capturing high-resolution real-world point clouds has never been higher. Loss of details and irregularities in point cloud geometry can occur during the capturing, processing, and compression pipeline. It is essential to address these challenges by being able to upsample a low Level-of-Detail (LoD) point cloud into a high LoD point cloud. Current upsampling methods suffer from several weaknesses in handling point cloud upsampling, especially in dense real-world photo-realistic point clouds. In this paper, we present a novel geometry upsampling technique, PU-Dense, which can process a diverse set of point clouds including synthetic mesh-based point clouds, real-world high-resolution point clouds, real-world indoor LiDAR scanned objects, as well as outdoor dynamically acquired LiDAR-based point clouds. PU-Dense employs a 3D multiscale architecture using sparse convolutional networks that hierarchically reconstruct an upsampled point cloud geometry via progressive rescaling and multiscale feature extraction. The framework employs a UNet type architecture that downscales the point cloud to a bottleneck and then upscales it to a higher level-of-detail (LoD) point cloud. PU-Dense introduces a novel Feature Extraction Unit that incorporates multiscale spatial learning by employing filters at multiple sampling rates and receptive fields. The architecture is memory efficient and is driven by a binary voxel occupancy classification loss that allows it to process high-resolution dense point clouds with millions of points during inference time. Qualitative and quantitative experimental results show that our method significantly outperforms the state-of-the-art approaches by a large margin while having much lower inference time complexity. We further test our dataset on high-resolution photo-realistic datasets. In addition, our method can handle noisy data well. We further show that our approach is memory efficient compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Omnidirectional (or 360°) images and videos are emergent signals being used in many areas, such as robotics and virtual/augmented reality. In particular, for virtual reality applications, they allow an immersive experience in which the user can interactively navigate through a scene with three degrees of freedom, wearing a head-mounted display. Current approaches for capturing, processing, delivering, and displaying 360° content, however, present many open technical challenges and introduce several types of distortions in the visual signal. Some of the distortions are specific to the nature of 360° images and often differ from those encountered in classical visual communication frameworks. This paper provides a first comprehensive review of the most common visual distortions that alter 360° signals going through the different processing elements of the visual communication pipeline. While their impact on viewers’ visual perception and the immersive experience at large is still unknown—thus, it is an open research topic—this review serves the purpose of proposing a taxonomy of the visual distortions that can be encountered in 360° signals. Their underlying causes in the end-to-end 360° content distribution pipeline are identified. This taxonomy is essential as a basis for comparing different processing techniques, such as visual enhancement, encoding, and streaming strategies, and allowing the effective design of new algorithms and applications. It is also a useful resource for the design of psycho-visual studies aiming to characterize human perception of 360° content in interactive and immersive applications. [ABSTRACT FROM AUTHOR]

Head-mounted holographic displays (HMHD) are projected to be the first commercial realization of holographic video display systems. HMHDs use liquid crystal on silicon (LCoS) spatial light modulators (SLM), which are best suited to display phase-only holograms (POH). The performance/watt requirement of a monochrome, 60 fps Full HD, 2-eye, POH HMHD system is about 10 TFLOPS/W, which is orders of magnitude higher than that is achievable by commercially available mobile processors. To mitigate this compute power constraint, display-ready POHs shall be generated on a nearby server and sent to the HMHD in compressed form over a wireless link. This paper discusses design of a feasible HMHD-based augmented reality system, focusing on compression requirements and per-pixel rate-distortion trade-off for transmission of display-ready POH from the server to HMHD. Since the decoder in the HMHD needs to operate on low power, only coding methods that have low-power decoder implementation are considered. Effects of 2D phase unwrapping and flat quantization on compression performance are also reported. We next propose a versatile PCM-POH codec with progressive quantization that can adapt to SLM-dynamic-range and available bitrate, and features per-pixel rate-distortion control to achieve acceptable POH quality at target rates of 60–200 Mbit/s that can be reliably achieved by current wireless technologies. Our results demonstrate feasibility of realizing a low-power, quality-ensured, multi-user, interactive HMHD augmented reality system with commercially available components using the proposed adaptive compression of display-ready POH with light-weight decoding. [ABSTRACT FROM AUTHOR]

In this paper, the global pinning synchronization problem is studied for a complex dynamical network to follow a dynamic target system. A distinguished feature of the present network model is that the target system may have some predesigned control inputs. This implies that, when pinning synchronization is guaranteed, the states of the nodes within such a pinning-controlled dynamical network may approach a specified trajectory which does not satisfy the system equation of the uncoupling individual node system within the network. The practical constraint that the external control inputs acting on the target system are unknown to any node in the considered network poses a big challenge in solving such a pinning synchronization problem. The designed scheme for achieving pinning synchronization is executed in two steps. Specifically, the first step is to select some nodes to pin such that the augmented interaction topology has at least one directed spanning tree rooted at the node describing the target system, while the second step is to construct a coupling law to synchronize all the states of nodes within the network. Moreover, two kinds of discontinuous coupling laws with static and adaptive coupling gains are, respectively, proposed to achieve pinning synchronization. Meanwhile, by utilizing nonsingular ${M}$ -matrix theory and Lyapunov stability analysis for nonsmooth system, some efficient criteria are established for guaranteeing synchronization in the pinning-controlled networks. Numerical simulations on pinning synchronization of networking Chua’s circuit systems are finally given to verify the analytic results. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) and virtual reality (VR) applications have seen rising popularity in recent years. Omnidirectional 360° video is a video format often used in AR and VR applications. To address the industry needs, a new HEVC edition recently published includes several supplemental enhancement information (SEI) messages to enable the carriage of omnidirectional video using HEVC. However, further improvement in 360° video compression efficiency is needed. In order to address this challenge, the Joint Video Exploration Team (JVET) of ITU-T VCEG and ISO/IEC MPEG has been investigating 360° video coding technologies, including projection formats, pre- and post-processing technologies, as well as 360°-video-specific coding tools since 2016. The joint call for proposals (CfP) recently issued by ITU-T VCEG and ISO/IEC MPEG on video compression technologies beyond HEVC included a category on 360° video. Twelve CfP responses in the 360° video category were received. This paper describes technologies relevant to 360° video for VVC. A summary of projection formats, pre- and post-processing methods, and 360°-video specific coding tool modifications in these proposals is provided. [ABSTRACT FROM AUTHOR]

Geometric optical waveguide display can significantly miniaturize the augmented reality eyeglasses, but suffers from the ghost image. In this paper, a method using surface micro-reflector array is proposed, which can dramatically suppress the ghost image. The transmission and expansion of the light bearing image are completed by using planar waveguide and micro-reflector array embedded in the surface layer of waveguide. The image quality is improved by using the dual channel to eliminate ghost image. The imaging process of optical system is modeled and simulated. The surface micro-reflector array waveguide element of 3 mm thickness is prepared for verification. The whole optical system has the advantages of simple structure, easy preparation and high structural strength. The display field of view is shown to be 25° × 25°, and the size of the full field of view area is 15 mm with even light distribution. What's more, the uniformity of imaging intensity is better than 20%, and the image is clear without ghost image, which can effectively realize the fusion of the virtual image and the real environment. Thus, the efficiency of augmented reality is greatly improved, and the danger of human eye safety is effectively avoided. [ABSTRACT FROM AUTHOR]

Objective: The focal length of available optical see-through (OST) head-mounted displays (HMDs) is at least 2 m; therefore, during manual tasks, the user eye cannot keep in focus both the virtual and real content at the same time. Another perceptual limitation is related to the vergence-accommodation conflict, the latter being present in binocular vision only. This paper investigates the effect of incorrect focus cues on the user performance, visual comfort, and workload during the execution of augmented reality (AR)-guided manual task with one of the most advanced OST HMD, the Microsoft HoloLens. Methods: An experimental study was designed to investigate the performance of 20 subjects in a connect-the-dots task, with and without the use of AR. The following tests were planned: AR-guided monocular and binocular, and naked-eye monocular and binocular. Each trial was analyzed to evaluate the accuracy in connecting dots. NASA Task Load Index and Likert questionnaires were used to assess the workload and the visual comfort. Results: No statistically significant differences were found in the workload, and in the perceived comfort between the AR-guided binocular and monocular test. User performances were significantly better during the naked eye tests. No statistically significant differences in performances were found in the monocular and binocular tests. The maximum error in AR tests was 5.9 mm. Conclusion: Even if there is a growing interest in using commercial OST HMD, for guiding high-precision manual tasks, attention should be paid to the limitations of the available technology not designed for the peripersonal space. [ABSTRACT FROM AUTHOR]

Task mapping has been a hot topic in multiprocessor system-on-chip software design for decades. During the mapping process, load balance (LB) and communication optimization have been two important performance optimization factors. This paper studies the relations between LB, interprocessor communications, and communication pipeline technique during the mapping process, and proposes an integer linear programming (ILP)-based static task mapping approach, which considers both LB and communication optimization. The approach consists of an optimized ILP model for task mapping with fewer variables compared to previous ILP mapping works. Moreover, to enhance the scalability of the ILP task mapping, the task-processor-cluster algorithm is proposed to reduce the scale of the task graph and the number of processors and then solve the coarse-grained input by the ILP mapping. To increase the adaptability of the ILP task mapping, the improved augmented $\epsilon $ -constraint method is further integrated with the ILP formulations to select the best mapping for different applications. Experimental results on a 2/4/8/16/24-CPU platform of both synthetic and real-life benchmarks demonstrate the efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper investigates the effectiveness of multipath-decorrelating antenna motion in reducing the initialization time of global navigation satellite system (GNSS) receivers employing low-cost single-frequency antennas for carrier-phase differential GNSS (CDGNSS) positioning. Fast initialization times with low-cost antennas will encourage the expansion of CDGNSS into the mass market, bringing the benefits of globally referenced centimeter-accurate positioning to many consumer applications, such as augmented reality and autonomous vehicles, that have so far been hampered by the several-meter-level errors of traditional GNSS positioning. Poor multipath suppression common to low-cost antennas results in large and strongly time-correlated phase errors when a receiver is static. Such errors can result in the CDGNSS initialization time, the so-called time to ambiguity resolution (TAR), extending to hundreds of seconds—many times longer than for higher cost survey-grade antennas, which have substantially better multipath suppression. This paper demonstrates that TAR can be significantly reduced through antenna motion, particularly gentle wavelength-scale random antenna motion. Such motion acts to decrease the correlation time of the multipath-induced phase errors. A priori knowledge of the motion profile is shown to further reduce TAR, with the reduction shown to be more pronounced as the initialization scenario is more challenging. [ABSTRACT FROM AUTHOR]

Along with the evolution of wireless technologies, users are expected to have high-throughput and low-latency indoor mobile applications, such as augmented reality and virtual reality. Therefore, the use of millimeter-wave for low-latency communications becomes attractive. Under low-latency criteria, a fast multi-link establishment is desired. In this paper, we present a novel concurrent multi-beam training jointly with a proposed multi-beam codebook. First, we propose a progressive multi-beam estimation (PMBE). This algorithm makes multiple coarse beams emerge within a few training steps. Second, we propose a Fast Fourier Transform (FFT)-based codebook which can be considered as a Discrete Fourier Transform based codebook built in a bit-reversal scrambler, thus forming a scattering multi-beam codebook. This FFT-based codebook design is optimally implemented on a sub-connected architecture, a hardware-efficient architecture. For fast multi-link establishment, we design our concurrent multi-beam training by combining the proposed PMBE jointly with the proposed FFT-based codebook, so as to further improve multiplexing gain. Since multiplexing gain can occur immediately, even under strictly-limited training steps, the high data rate can be achieved. Simulation results show that when reaching more than 95% capacity gain of exhaustive channel estimation, our work only spends 1% training steps of the exhaustive case. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) provides huge opportunities for online teaching in science and engineering, as these disciplines place emphasis on practical training and unsuited to completely nonclassroom training. This paper proposes a new concept in virtual and remote laboratories: the augmented remote laboratory (ARL). ARL is being tested in the first and second years of the new degrees in industrial engineering and computer engineering, respectively, at the School of Engineering, University of Huelva, Huelva, Spain. By means of augmented reality techniques, ARL allows the student to experience sensations and explore learning experiences that, in some cases, may exceed those offered by traditional laboratory classes. The effectiveness of this methodology for remote laboratory work is evaluated by comparing it to practical sessions in the laboratory at the university itself with the same group of students. Students completed a questionnaire after having experienced both types of practicals, and the results show that the use of ARL improves student outcomes. As discussed in the paper, the potential of AR to configure different experiments from the same physical configuration is virtually limitless. [ABSTRACT FROM AUTHOR]

This paper proposes a scalable Augmented Reality (AR) framework enabled by a simple extension to the Internet of Things (IoT) infrastructure, called the "ARIoT." Through ARIoT (1) potential user-proximal target objects are dynamically identified in any IoT-enabled space, (2) tracking feature information is directly obtained from the IoT objects enabling fast recognition and tracking (independent of the number of the total IoT objects everywhere), and (3) generic interactive contents are augmented as an IoT service to the AR client. To further improve the AR tracking performance, which often depends on feature appearances, this paper introduces a method in which the preferred tracking method is determined by the very IoT object in hand. The proposed concept of ARIoT is demonstrated, and its performance is evaluated and compared with the conventional single serveror client-based AR implementation. In addition, a pilot usability test is conducted between a spatially registered visual in-situ AR interface using ARIoT and a conventional interface based only on an online connection, and the results are then compared. This study shows how AR and IoT can be complementary to each other: AR as an attractive visual insitu interface for IoT objects and IoT as a basis for scalable AR1 [ABSTRACT FROM PUBLISHER]

Hand pose understanding is essential to applications such as human computer interaction and augmented reality. Recently, deep learning based methods achieve great progress in this problem. However, the lack of high-quality and large-scale dataset prevents the further improvement of hand pose related tasks such as 2D/3D hand pose from color and depth from color. In this paper, we develop a large-scale and high-quality synthetic dataset, PBRHand. The dataset contains millions of photo-realistic rendered hand images and various ground truths including pose, semantic segmentation, and depth. Based on the dataset, we firstly investigate the effect of rendering methods and used databases on the performance of three hand pose related tasks: 2D/3D hand pose from color, depth from color and 3D hand pose from depth. This study provides insights that photo-realistic rendering dataset is worthy of synthesizing and shows that our new dataset can improve the performance of the state-of-the-art on these tasks. This synthetic data also enables us to explore multi-task learning, while it is expensive to have all the ground truth available on real data. Evaluations show that our approach can achieve state-of-the-art or competitive performance on several public datasets. [ABSTRACT FROM AUTHOR]

The current target detection network in deep learning has been widely used in plant growth state detection. However, with the development of deep learning, within the field of plant growth state detection, the performance of the detection network is no longer the primary factor limiting the detection accuracy and model generalization ability. The construction of high-quality and large-scale plant datasets is more significant for the improvement of model detection accuracy and generalization ability. However, traditional methods for building deep learning datasets for plants have a large time span and low efficiency. And it is difficult to construct and expand the dataset for plants with complex growth environments and difficult image acquisition by existing methods. To address this problem, this paper proposes a method for constructing plant datasets based on augmented reality techniques. The method proposed in this paper allows for the rapid and efficient construction of large-scale field datasets that match the actual inspection environment in the lack of data. Meanwhile, this paper proposes an automatic annotation method for datasets in conjunction with the imaging environment in virtual space. In this paper, we experimentally compare the proposed method with the method of expanding the dataset using GAN networks. Using the virtual dataset constructed by the method proposed in this paper as the training set, the trained YOLOv5 model achieves an average accuracy (@0.5:0.95) of 0.71 for the three detection categories on the test set. The detection accuracy of the six mixed datasets constructed using the two data expansion methods on the test set was experimentally tested. The proposed method in this paper improved the accuracy by 2.2%, 3.1%, and 7.0%, respectively. The smaller the percentage of real images, the greater the accuracy improvement. Experiments show that the method proposed in this paper can well solve the problems faced in the field of plant growth state detection, such as the lack of data, and provides a new idea for the production and expansion of datasets in plant detection tasks.

The term Metaverse is emerging as a result of the late push by multinational technology conglomerates and a recent surge of interest in Web 3.0, Blockchain, NFT, and Cryptocurrencies. From a scientific point of view, there is no definite consensus on what the Metaverse will be like. This paper collects, analyzes, and synthesizes scientific definitions and the accompanying major characteristics of the Metaverse using the methodology of a Systematic Literature Review (SLR). Two revised definitions for the Metaverse are presented, both condensing the key attributes, where the first one is rather simplistic holistic describing “a three-dimensional online environment in which users represented by avatars interact with each other in virtual spaces decoupled from the real physical world”. In contrast, the second definition is specified in a more detailed manner in the paper and further discussed. These comprehensive definitions offer specialized and general scholars an application within and beyond the scientific context of the system science, information system science, computer science, and business informatics, by also introducing open research challenges. Furthermore, an outlook on the social, economic, and technical implications is given, and the preconditions that are necessary for a successful implementation are discussed.

As technological, economic, and social factors drive scientific publishing toward electronic formats, opportunities open beyond traditional reading and writing frameworks. Journal articles now, and in the future, can increasingly include a variety of supplemental multimedia and interactive materials for augmented reading that will impact both the nature and presentation of scientific research. The IEEE Computer Society is preparing for this evolution. The first Web extra at http://youtu.be/06J3yg0SV8o is a video that serves as a short abstract of the paper "Augmented Reading: The Present and Future of Electronic Scientific Publications," presented by the two authors with the support of two young promising scientists. The second Web extra at http://youtu.be/86nN1gCMc1E is a video by David Alan Grier (IEEE Computer Society President, 2013) that discusses the Computer Society and its plans for scientific publications. The third Web extra at http://youtu.be/GCCMYR5Zy70 is a video that collects the personal views of Dejan Milosevic (IEEE CS President, 2014), David A. Grier (IEEE CS President, 2013), Sorel Reisman (IEEE CS President, 2012), Jon Rokne (IEEE CS Publications), Fabrizio Lombardi (EIC IEEE Transactions on Emerging Topics in Computing and member of the IEEE CS Board of Governors), and Steve Woods (IEEE CS Senior Manager Multimedia, Production) about the future of scientific publications. The fourth Web extra at http://youtu.be/7cLNV4SNdjY is a video presentation of the multimedia enhancements Computer magazine offers to its readers each month. The fifth Web extra at http://youtu.be/4csCKb685XQ is a video that presents a collection of definitions for the term "supplemental material." The sixth Web extra at http://youtu.be/d7a7XxMT6eY is a video that reviews the announcement from the Journal of Neuroscience that authors can no longer submit supplemental material. The seventh Web extra at http://youtu.be/NMFwLJhv11c is a video in which author Paolo Montuschi summarizes the key concepts of the article "Augmented Reading: The Present and Future of Electronic Scientific Publications". [ABSTRACT FROM PUBLISHER]

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. As a consequence, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. The main contribution of this paper is to reveal how exemplary workflows are redefined by taking full advantage of head-mounted displays when entirely co-registered with the imaging system at all times. The awareness of the system from the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. Our system achieved an error of 4.76 ± 2.91mm for placing K-wire in a fracture management procedure, and yielded errors of 1.57 ± 1.16° and 1.46 ± 1.00° in the abduction and anteversion angles, respectively, for total hip arthroplasty (THA). We compared the results with the outcomes from baseline standard operative and non-immersive AR procedures, which had yielded errors of [4.61mm, 4.76°, 4.77°] and [5.13mm, 1.78°, 1.43°], respectively, for wire placement, and abduction and anteversion during THA. We hope that our holistic approach towards improving the interface of surgery not only augments the surgeon’s capabilities but also augments the surgical team’s experience in carrying out an effective intervention with reduced complications and provide novel approaches of documenting procedures for training purposes. [ABSTRACT FROM AUTHOR]

This paper presents a new approach for self-calibration of static cameras in the context of surveillance applications. Initially, a pedestrian detector is applied and the responses are validated using background removal. Then, foreground-related pixels within the detection results are used to estimate the feet-head line segments of each person (called poles), which are used to find a linear estimate for the camera matrix. Finally, a nonlinear cost function is used to refine the initial estimate, aiming to mostly improve the orientation of the reprojected poles. We also present different applications of self-calibration in tasks related to video surveillance itself, such as improvements to pedestrian detection and tracking algorithms, and augmented reality applications, such as the insertion of virtual cameras to aid the placement of real cameras in the scene. [ABSTRACT FROM PUBLISHER]

Fastener assembly is a tedious and time-consuming work because operators have to check assembly manuals and find right fastener for each hole. Hence, this paper aims to develop a three-dimensional (3-D) projection system that projects assembly instruction onto the work piece surface directly to guide operators to assemble. However, in order to project the instruction accurately, the corresponding part of the computer-aided design model of the physical scanned area needs to be attained through the rapid and accurate registration. In order to achieve this goal, first, a high-accuracy and rapid 3-D measurement system is developed; second, a fast registration method based on local multiscale geometric feature vector is proposed to accelerate the registration speed and improve the registration reliability. Experimental results demonstrate the measurement accuracy of the developed system, and verify the feasibility of the proposed registration method. Hence, the proposed method can lead to improved assembly efficiency and decreased error probability, making great contributions to large-scale structure assembly. [ABSTRACT FROM PUBLISHER]

This paper describes a novel light-sensor-based information transmission system for indoor positioning and navigation with particular benefits for mobile and wearable computers. It can seamlessly extend outdoor GPS tracking to the indoor environment. In a novel manner, fluorescent light is used as the medium to transmit information, which is encoded by using a pulse-frequency modulation technique. The user receives the encoded light information through a photoreceiver. The information is passed into the wearable or mobile computer after the data are decoded. This information allows positioning information to be given to indoor mobile and wearable computers. We design an economical transmitter circuit by adding a few components to a commercial electronic ballast circuit for fluorescent lamps with a price of less than 10 USD. The proposed system can be used in indoor guidance and navigation applications. Exemplary applications are included in the paper, with experimented results detailed. [ABSTRACT FROM AUTHOR]

Light field imaging, which captures spatial-angular information of light incident on image sensors, enables many interesting applications such as image refocusing and augmented reality. However, due to the limited sensor resolution, a trade-off exists between the spatial and angular resolutions. To increase the angular resolution, view synthesis techniques have been adopted to generate new views from existing views. However, traditional learning-based view synthesis mainly considers the image quality of each view of the light field and neglects the quality of the refocused images. In this paper, we propose a new loss function called refocused image error (RIE) to address the issue. The main idea is that the image quality of the synthesized light field should be optimized in the refocused image domain because it is where the light field is viewed. We analyze the behavior of RIE in the spectral domain and test the performance of our approach against previous approaches on both real (INRIA) and software-rendered (HCI) light field datasets using objective assessment metrics such as MSE, MAE, PSNR, SSIM, and GMSD. Experimental results show that the light field generated by our method results in better refocused images than previous methods. [ABSTRACT FROM AUTHOR]

Physically based rendering has been widely used to generate photo-realistic images, which greatly impacts industry by providing appealing rendering, such as for entertainment and augmented reality, and academia by serving large scale high-fidelity synthetic training data for data hungry methods like deep learning. However, physically based rendering heavily relies on ray-tracing, which can be computational expensive in complicated environment and hard to parallelize. In this paper, we propose an end-to-end deep learning based approach to generate physically based rendering efficiently. Our system consists of two stacked neural networks, which effectively simulates the physical behavior of the rendering process and produces photo-realistic images. The first network, namely shading network, is designed to predict the optimal shading image from surface normal, depth and illumination; the second network, namely composition network, learns to combine the predicted shading image with the reflectance to generate the final result. Our approach is inspired by intrinsic image decomposition, and thus it is more physically reasonable to have shading as intermediate supervision. Extensive experiments show that our approach is robust to noise thanks to a modified perceptual loss and even outperforms the physically based rendering systems in complex scenes given a reasonable time budget. [ABSTRACT FROM AUTHOR]

This paper presents a new approach to recognizing vanishing-point-constrained building planes from a single image of street view. We first design a novel convolutional neural network (CNN) architecture that generates geometric segmentation of per-pixel orientations from a single street-view image. The network combines two-stream features of general visual cues and surface normals in gated convolution layers, and employs a deeply supervised loss that encapsulates multi-scale convolutional features. Our experiments on a new benchmark with fine-grained plane segmentations of real-world street views show that our network outperforms state-of-the-arts methods of both semantic and geometric segmentation. The pixel-wise segmentation exhibits coarse boundaries and discontinuities. We then propose to rectify the pixel-wise segmentation into perspectively-projected quads based on spatial proximity between the segmentation masks and exterior line segments detected through an image processing. We demonstrate how the results can be utilized to perspectively overlay images and icons on building planes in input photos, and provide visual cues for various applications. [ABSTRACT FROM AUTHOR]

With the rapid development of deep learning technology and other powerful tools, 3D object detection has made great progress and become one of the fastest growing field in computer vision. Many automated applications such as robotic navigation, autonomous driving, and virtual or augmented reality system require estimation of accurate 3D object location and detection. Under this requirement, many methods have been proposed to improve the performance of 3D object localization and detection. Despite recent efforts, 3D object detection is still a very challenging task due to occlusion, viewpoint variations, scale changes, and limited information in 3D scenes. In this paper, we present a comprehensive review of recent state-of-the-art approaches in 3D object detection technology. We start with some basic concepts, then describe some of the available datasets that are designed to facilitate the performance evaluation of 3D object detection algorithms. Next, we will review the state-of-the-art technologies in this area, highlighting their contributions, importance, and limitations as a guide for future research. Finally, we provide a quantitative comparison of the results of the state-of-the-art methods on the popular public datasets. [ABSTRACT FROM AUTHOR]

This paper proposes a wearable monitoring system for inspection in the framework of Industry 4.0. The instrument integrates augmented reality (AR) glasses with a noninvasive single-channel brain–computer interface (BCI), which replaces the classical input interface of AR platforms. Steady-state visually evoked potentials (SSVEP) are measured by a single-channel electroencephalography (EEG) and simple power spectral density analysis. The visual stimuli for SSVEP elicitation are provided by AR glasses while displaying the inspection information. The real-time metrological performance of the BCI is assessed by the receiver operating characteristic curve on the experimental data from 20 subjects. The characterization was carried out by considering stimulation times from 10.0 down to 2.0 s. The thresholds for the classification were found to be dependent on the subject and the obtained average accuracy goes from 98.9% at 10.0 s to 81.1% at 2.0 s. An inspection case study of the integrated AR-BCI device shows encouraging accuracy of about 80% of lab values. [ABSTRACT FROM AUTHOR]

Ever since the development of the first applications in image-guided therapy (IGT), the use of head-mounted displays (HMDs) was considered an important extension of existing IGT technologies. Several approaches to utilizing HMDs and modified medical devices for augmented reality (AR) visualization were implemented. These approaches include video-see through systems, semitransparent mirrors, modified endoscopes, and modified operating microscopes. Common to all these devices is the fact that a precise calibration between the display and three-dimensional co-ordinates in the patient's frame of reference is compulsory. In optical see-through devices based on complex optical systems such as operating microscopes or operating binoculars—as in the case of the system presented in this paper—this procedure can become increasingly difficult since precise camera calibration for every focus and zoom position is required. We present a method for fully automatic calibration of the operating binocular Varioscope™ MS AR for the full range of zoom and focus settings available. Our method uses a special calibration pattern, a linear guide driven by a stepping motor, and special calibration software. The overlay error in the calibration plane was found to be 0.14-0.91 mm, which is less than 1% of the field of view. Using the motorized calibration rig as presented in the paper, we were also able to assess the dynamic latency when viewing augmentation graphics on a mobile target; spatial displacement due to latency was found to be in the range of 1.1-2.8 mm maximum, the disparity between the true object and its computed overlay represented latency of 0.1 s. We conclude that the automatic calibration method presented in this paper is sufficient in terms of accuracy and time requirements for standard uses of optical see-through systems in a clinical environment. [ABSTRACT FROM AUTHOR]

This paper presents the MPEG-4 Animation Frame- work extension (AFX) standard, ISOIIEC 14496-16. Initiated by the MPEG Synthetic/Natural Hybrid Coding group in 2000, MPEG-4 AFX proposes an advanced framework for interactive multimedia applications using both natural and synthetic objects. Following this model, new synthetic objects have been specified, increasing content realism over existing MPEG-4 synthetic objects. The general overview of MPEG-4 AFX is provided on top of the review of MPEG-4 standards to explain the relationship between MPEG-4 and MPEG-4 AFX. Then we give a bird's-eye view of new tools available in this standard and further explained in other paper in this special issue. [ABSTRACT FROM AUTHOR]

In image-guided robotic surgery, segmenting the endoscopic video stream into meaningful parts provides important contextual information that surgeons can exploit to enhance their perception of the surgical scene. This information provides surgeons with real-time decision-making guidance before initiating critical tasks such as tissue cutting. Segmenting endoscopic video is a challenging problem due to a variety of complications including significant noise attributed to bleeding and smoke from cutting, poor appearance contrast between different tissue types, occluding surgical tools, and limited visibility of the objects’ geometries on the projected camera views. In this paper, we propose a multi-modal approach to segmentation where preoperative 3D computed tomography scans and intraoperative stereo-endoscopic video data are jointly analyzed. The idea is to segment multiple poorly visible structures in the stereo/multichannel endoscopic videos by fusing reliable prior knowledge captured from the preoperative 3D scans. More specifically, we estimate and track the pose of the preoperative models in 3D and consider the models’ non-rigid deformations to match with corresponding visual cues in multi-channel endoscopic video and segment the objects of interest. Further, contrary to most augmented reality frameworks in endoscopic surgery that assume known camera parameters, an assumption that is often violated during surgery due to non-optimal camera calibration and changes in camera focus/zoom, our method embeds these parameters into the optimization hence correcting the calibration parameters within the segmentation process. We evaluate our technique on synthetic data, ex vivo lamb kidney datasets, and in vivo clinical partial nephrectomy surgery with results demonstrating high accuracy and robustness. [ABSTRACT FROM AUTHOR]

The rejection rates of upper-limb prosthetic devices in adults are high, currently averaging 26% and 23% for body-powered and electric devices, respectively. While many factors influence acceptance, prosthesis training methods relying on novel virtual reality systems have been cited as a critical factor capable of increasing the likelihood of long-term, full-time use. Despite that, these implementations have not yet garnered widespread traction in the clinical setting, and their use remains immaterial. This review aims to explore the reasons behind this situation by identifying trends in existing research that seek to advance Extended Reality “X-Reality” systems for the sake of upper-limb prosthesis rehabilitation and, secondly, analyzing barriers and presenting potential pathways to deployment for successful adoption in the future. The search yielded 42 research papers that were divided into two categories. The first category included articles that focused on the technical aspect of virtual prosthesis training. Articles in the second category utilize user evaluation procedures to ensure applicability in a clinical environment. The review showed that 75% of articles that conducted whole system testing experimented with non-immersive virtual systems. Furthermore, there is a shortage of experiments performed with amputee subjects. From the large-scale studies analyzed, 71% of those recruited solely non-disabled participants. This paper shows that X-Reality technologies for prosthesis rehabilitation of upper-limb amputees carry significant benefits. Nevertheless, much still must be done so that the technology reaches widespread clinical use.

This paper presents a complete 3D model reconstruction of an object with edge quality enhancements using multiple inward depth sensors to create closed 3D model. In the reconstruction pipeline, a pattern of incorrect depth information was consistently observed at the edges of the mesh generated by each sensor stream, which we refer to in this paper as a “drift-effect”. In order to mitigate this, we introduced a filtering approach with a localized threshold value that is used to remove drift faces from a mesh. We also present a mesh stitching technique incorporating Laplacian mesh smoothing to generate a closed 3D model from the smoothened multi-view meshes. The primary objective of this research was to implement a system that could capture a static physical object with a minimum scan time and at a low cost while retaining accurate details in the model. For the demonstration, we used four Intel RealSense D435 depth sensors to capture a clothing article that can be imported into a virtual dressing room application. We captured the entire object within three seconds, which is quicker than traditional techniques such as table rotation and sensor rotation. The final results indicate that the system is able to provide a satisfactory reconstruction of a clothing model which can be used in a live virtual dressing room application.

Recreation of 3D crime scenes is critical for law enforcement in the investigation of serious crimes for criminal justice responses. This work presents a premier systematic literature review (SLR) that offers a structured, methodical, and rigorous approach to understanding the trend of research in 3D crime scene reconstruction as well as tools, technologies, methods, and techniques employed thereof in the last 17 years. Major credible scholarly database sources, Scopus, and Google Scholar, which index journals and conferences that are promoted by entities such as IEEE, ACM, Elsevier, and SpringerLink were explored as data sources. Of the initial 17, 912 papers that resulted from the first search string, 258 were found to be relevant to our research questions after implementing the inclusion and exclusion criteria. To summarize the existing efforts, we compared and analysed various classical 3D reconstruction approaches. This study presents the first comprehensive review of key milestones in the development of methods for 3D crime scene reconstruction, gaps for improvement and where immersive technology has been used to enhance crime scene findings. This study found that the implementation of light detection and ranging (LiDAR) scanners and immersive technologies, alongside traditional methods, has been beneficial in the recreation of crime scenes. The SLR is limited to existing applications with peer-reviewed papers published between 2005 and 2021. Results based on the analysed published data indicated that 20.2% of the articles implemented immersive technologies in crime scene reconstruction, of which Augmented Reality (AR) accounted for 15.3%, Virtual Reality (VR) accounted for 75%, Mixed reality (MR) accounted for 5.9% and VR and AR mixture accounted for 3.8%. Finally, we summarize the development trend of design and key technology prospects of crime scene recreation using immersive technology and provide insights into potential future research. To the best of the researchers’ knowledge, this is the first survey that accomplishes such goals.

This paper presents a new algorithm for the fast, shared memory, multi-core computation of augmented contour trees on triangulations. In contrast to most existing parallel algorithms our technique computes augmented trees, enabling the full extent of contour tree based applications including data segmentation. Our approach completely revisits the traditional, sequential contour tree algorithm to re-formulate all the steps of the computation as a set of independent local tasks. This includes a new computation procedure based on Fibonacci heaps for the join and split trees, two intermediate data structures used to compute the contour tree, whose constructions are efficiently carried out concurrently thanks to the dynamic scheduling of task parallelism. We also introduce a new parallel algorithm for the combination of these two trees into the output global contour tree. Overall, this results in superior time performance in practice, both in sequential and in parallel thanks to the OpenMP task runtime. We report performance numbers that compare our approach to reference sequential and multi-threaded implementations for the computation of augmented merge and contour trees. These experiments demonstrate the run-time efficiency of our approach and its scalability on common workstations. We demonstrate the utility of our approach in data segmentation applications. [ABSTRACT FROM AUTHOR]

Hybrid surgical simulators based on augmented reality (AR) solutions benefit from the advantages of both the box trainers and the virtual reality simulators. This paper reports on the results of a long development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and the virtual components. We first outline the specifications of the AR simulator and then we explain the strategy adopted for implementing it based on a careful selection of its simulated anatomical components, and characterized by a real-time tracking of both a target anatomy and of the laparoscope. The former is tracked by means of an electromagnetic field generator, while the latter requires an additional camera for video tracking. The new system was evaluated in terms of AR visualization accuracy, realism, and hardware robustness. Obtained results show that the accuracy of AR visualization is adequate for training purposes. The qualitative evaluation confirms the robustness and the realism of the simulator. In conclusion, the proposed AR simulator satisfies all the initial specifications in terms of anatomical appearance, modularity, reusability, minimization of spare parts cost, and ability to record surgical errors and to track in real-time the Calot's triangle and the laparoscope. Thus, the proposed system could be an effective training tool for learning the task of identification and isolation of Calot's triangle in laparoscopic cholecystectomy. Moreover, the presented strategy could be applied to simulate other surgical procedures involving the task of identification and isolation of generic tubular structures, such as blood vessels, biliary tree, and nerves, which are not directly visible. [ABSTRACT FROM AUTHOR]