Search

The article discusses the met averse services and business models and Met averse architectures. Topics include the ways of social interaction are significantly augmented with persistent virtual content, consumed via VR or AR; met averse is an emerging concept which embraces three important elements, and Experimental demonstrations and prototypes of met averse.

The article reports that The metaverse is an emerging concept which embraces three important elements. First, it embraces a social element, i.e. it is not only a virtual space where users spend time (and money) on their own or with a selected few. Rather, the metaverse is intended to resonate with the very social fabric which underpins human society.

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

With the full development of intelligent mobile communications, wireless mixed reality (MR) provides a more visually immersive experience and stronger interaction with environments than virtual reality (VR) and augmented reality (AR). However, the asymmetric characteristic of wireless MR traffic creates a huge challenge to current mobile networks. Dynamic time division duplex (D-TDD) is considered as a promising technology to improve wireless MR users’ quality of experience (QoE) due to its potentials and advantages in delivering asymmetric traffic. Therefore, in this paper, we propose a QoE-driven distributed multidimensional resource allocation (MRA) supplemented by inter-cell interference (ICI) mitigation scheme for wireless MR in multi-cell D-TDD systems. First, to improve QoE of MR users, we formulate the joint optimization of subframe configuration, channel assignment and computation offloading as a mixed-integer nonlinear programming problem. A novel fully-decentralized multi-agent deep Q-network (DQN) algorithm is developed to solve the problem. Then, to mitigate ICI, a water filling based power control algorithm is investigated to minimize the total power of each small base station and its associated MR users. Simulation results demonstrate that our proposed scheme improves QoE of MR users in a realizable way as compared to existing schemes. [ABSTRACT FROM AUTHOR]

With the development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary value of AR is to promote the fusion of digital contents and real-world environments, however, studies on how this fusion will influence the Quality of Experience (QoE) of these two components are lacking. To achieve better QoE of AR, whose two layers are influenced by each other, it is important to evaluate its perceptual quality first. In this paper, we consider AR technology as the superimposition of virtual scenes and real scenes, and introduce visual confusion as its basic theory. A more general problem is first proposed, which is evaluating the perceptual quality of superimposed images, i.e., confusing image quality assessment. A ConFusing Image Quality Assessment (CFIQA) database is established, which includes 600 reference images and 300 distorted images generated by mixing reference images in pairs. Then a subjective quality perception experiment is conducted towards attaining a better understanding of how humans perceive the confusing images. Based on the CFIQA database, several benchmark models and a specifically designed CFIQA model are proposed for solving this problem. Experimental results show that the proposed CFIQA model achieves state-of-the-art performance compared to other benchmark models. Moreover, an extended ARIQA study is further conducted based on the CFIQA study. We establish an ARIQA database to better simulate the real AR application scenarios, which contains 20 AR reference images, 20 background (BG) reference images, and 560 distorted images generated from AR and BG references, as well as the correspondingly collected subjective quality ratings. Three types of full-reference (FR) IQA benchmark variants are designed to study whether we should consider the visual confusion when designing corresponding IQA algorithms. An ARIQA metric is finally proposed for better evaluating the perceptual quality of AR images. Experimental results demonstrate the good generalization ability of the CFIQA model and the state-of-the-art performance of the ARIQA model. The databases, benchmark models, and proposed metrics are available at: https://github.com/DuanHuiyu/ARIQA. [ABSTRACT FROM AUTHOR]

A holistic understanding of dynamic scenes is of fundamental importance in real-world computer vision problems such as autonomous driving, augmented reality and spatio-temporal reasoning. In this paper, we propose a new computer vision benchmark: Video Panoptic Segmentation (VPS). To study this important problem, we present two datasets, Cityscapes-VPS and VIPER together with a new evaluation metric, video panoptic quality (VPQ). We also propose VPSNet++, an advanced video panoptic segmentation network, which simultaneously performs classification, detection, segmentation, and tracking of all identities in videos. Specifically, VPSNet++ builds upon a top-down panoptic segmentation network by adding pixel-level feature fusion head and object-level association head. The former temporally augments the pixel features while the latter performs object tracking. Furthermore, we propose panoptic boundary learning as an auxiliary task, and instance discrimination learning which learns spatio-temporally clustered pixel embedding for individual thing or stuff regions, i.e., exactly the objective of the video panoptic segmentation problem. Our VPSNet++ significantly outperforms the default VPSNet, i.e., FuseTrack baseline, and achieves state-of-the-art results on both Cityscapes-VPS and VIPER datasets. The datasets, metric, and models are publicly available at https://github.com/mcahny/vps. [ABSTRACT FROM AUTHOR]

Due to the increased popularity of augmented and virtual reality experiences, the interest in capturing high-resolution real-world point clouds has never been higher. Loss of details and irregularities in point cloud geometry can occur during the capturing, processing, and compression pipeline. It is essential to address these challenges by being able to upsample a low Level-of-Detail (LoD) point cloud into a high LoD point cloud. Current upsampling methods suffer from several weaknesses in handling point cloud upsampling, especially in dense real-world photo-realistic point clouds. In this paper, we present a novel geometry upsampling technique, PU-Dense, which can process a diverse set of point clouds including synthetic mesh-based point clouds, real-world high-resolution point clouds, real-world indoor LiDAR scanned objects, as well as outdoor dynamically acquired LiDAR-based point clouds. PU-Dense employs a 3D multiscale architecture using sparse convolutional networks that hierarchically reconstruct an upsampled point cloud geometry via progressive rescaling and multiscale feature extraction. The framework employs a UNet type architecture that downscales the point cloud to a bottleneck and then upscales it to a higher level-of-detail (LoD) point cloud. PU-Dense introduces a novel Feature Extraction Unit that incorporates multiscale spatial learning by employing filters at multiple sampling rates and receptive fields. The architecture is memory efficient and is driven by a binary voxel occupancy classification loss that allows it to process high-resolution dense point clouds with millions of points during inference time. Qualitative and quantitative experimental results show that our method significantly outperforms the state-of-the-art approaches by a large margin while having much lower inference time complexity. We further test our dataset on high-resolution photo-realistic datasets. In addition, our method can handle noisy data well. We further show that our approach is memory efficient compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Visual localization enables autonomous vehicles to navigate in their surroundings and augmented reality applications to link virtual to real worlds. Practical visual localization approaches need to be robust to a wide variety of viewing conditions, including day-night changes, as well as weather and seasonal variations, while providing highly accurate six degree-of-freedom (6DOF) camera pose estimates. In this paper, we extend three publicly available datasets containing images captured under a wide variety of viewing conditions, but lacking camera pose information, with ground truth pose information, making evaluation of the impact of various factors on 6DOF camera pose estimation accuracy possible. We also discuss the performance of state-of-the-art localization approaches on these datasets. Additionally, we release around half of the poses for all conditions, and keep the remaining half private as a test set, in the hopes that this will stimulate research on long-term visual localization, learned local image features, and related research areas. Our datasets are available at visuallocalization.net , where we are also hosting a benchmarking server for automatic evaluation of results on the test set. The presented state-of-the-art results are to a large degree based on submissions to our server. [ABSTRACT FROM AUTHOR]

Hand pose understanding is essential to applications such as human computer interaction and augmented reality. Recently, deep learning based methods achieve great progress in this problem. However, the lack of high-quality and large-scale dataset prevents the further improvement of hand pose related tasks such as 2D/3D hand pose from color and depth from color. In this paper, we develop a large-scale and high-quality synthetic dataset, PBRHand. The dataset contains millions of photo-realistic rendered hand images and various ground truths including pose, semantic segmentation, and depth. Based on the dataset, we firstly investigate the effect of rendering methods and used databases on the performance of three hand pose related tasks: 2D/3D hand pose from color, depth from color and 3D hand pose from depth. This study provides insights that photo-realistic rendering dataset is worthy of synthesizing and shows that our new dataset can improve the performance of the state-of-the-art on these tasks. This synthetic data also enables us to explore multi-task learning, while it is expensive to have all the ground truth available on real data. Evaluations show that our approach can achieve state-of-the-art or competitive performance on several public datasets. [ABSTRACT FROM AUTHOR]

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. As a consequence, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. The main contribution of this paper is to reveal how exemplary workflows are redefined by taking full advantage of head-mounted displays when entirely co-registered with the imaging system at all times. The awareness of the system from the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. Our system achieved an error of 4.76 ± 2.91mm for placing K-wire in a fracture management procedure, and yielded errors of 1.57 ± 1.16° and 1.46 ± 1.00° in the abduction and anteversion angles, respectively, for total hip arthroplasty (THA). We compared the results with the outcomes from baseline standard operative and non-immersive AR procedures, which had yielded errors of [4.61mm, 4.76°, 4.77°] and [5.13mm, 1.78°, 1.43°], respectively, for wire placement, and abduction and anteversion during THA. We hope that our holistic approach towards improving the interface of surgery not only augments the surgeon’s capabilities but also augments the surgical team’s experience in carrying out an effective intervention with reduced complications and provide novel approaches of documenting procedures for training purposes. [ABSTRACT FROM AUTHOR]

Omnidirectional (or 360°) images and videos are emergent signals being used in many areas, such as robotics and virtual/augmented reality. In particular, for virtual reality applications, they allow an immersive experience in which the user can interactively navigate through a scene with three degrees of freedom, wearing a head-mounted display. Current approaches for capturing, processing, delivering, and displaying 360° content, however, present many open technical challenges and introduce several types of distortions in the visual signal. Some of the distortions are specific to the nature of 360° images and often differ from those encountered in classical visual communication frameworks. This paper provides a first comprehensive review of the most common visual distortions that alter 360° signals going through the different processing elements of the visual communication pipeline. While their impact on viewers’ visual perception and the immersive experience at large is still unknown—thus, it is an open research topic—this review serves the purpose of proposing a taxonomy of the visual distortions that can be encountered in 360° signals. Their underlying causes in the end-to-end 360° content distribution pipeline are identified. This taxonomy is essential as a basis for comparing different processing techniques, such as visual enhancement, encoding, and streaming strategies, and allowing the effective design of new algorithms and applications. It is also a useful resource for the design of psycho-visual studies aiming to characterize human perception of 360° content in interactive and immersive applications. [ABSTRACT FROM AUTHOR]

Light field imaging, which captures spatial-angular information of light incident on image sensors, enables many interesting applications such as image refocusing and augmented reality. However, due to the limited sensor resolution, a trade-off exists between the spatial and angular resolutions. To increase the angular resolution, view synthesis techniques have been adopted to generate new views from existing views. However, traditional learning-based view synthesis mainly considers the image quality of each view of the light field and neglects the quality of the refocused images. In this paper, we propose a new loss function called refocused image error (RIE) to address the issue. The main idea is that the image quality of the synthesized light field should be optimized in the refocused image domain because it is where the light field is viewed. We analyze the behavior of RIE in the spectral domain and test the performance of our approach against previous approaches on both real (INRIA) and software-rendered (HCI) light field datasets using objective assessment metrics such as MSE, MAE, PSNR, SSIM, and GMSD. Experimental results show that the light field generated by our method results in better refocused images than previous methods. [ABSTRACT FROM AUTHOR]

Physically based rendering has been widely used to generate photo-realistic images, which greatly impacts industry by providing appealing rendering, such as for entertainment and augmented reality, and academia by serving large scale high-fidelity synthetic training data for data hungry methods like deep learning. However, physically based rendering heavily relies on ray-tracing, which can be computational expensive in complicated environment and hard to parallelize. In this paper, we propose an end-to-end deep learning based approach to generate physically based rendering efficiently. Our system consists of two stacked neural networks, which effectively simulates the physical behavior of the rendering process and produces photo-realistic images. The first network, namely shading network, is designed to predict the optimal shading image from surface normal, depth and illumination; the second network, namely composition network, learns to combine the predicted shading image with the reflectance to generate the final result. Our approach is inspired by intrinsic image decomposition, and thus it is more physically reasonable to have shading as intermediate supervision. Extensive experiments show that our approach is robust to noise thanks to a modified perceptual loss and even outperforms the physically based rendering systems in complex scenes given a reasonable time budget. [ABSTRACT FROM AUTHOR]

This paper presents a new approach to recognizing vanishing-point-constrained building planes from a single image of street view. We first design a novel convolutional neural network (CNN) architecture that generates geometric segmentation of per-pixel orientations from a single street-view image. The network combines two-stream features of general visual cues and surface normals in gated convolution layers, and employs a deeply supervised loss that encapsulates multi-scale convolutional features. Our experiments on a new benchmark with fine-grained plane segmentations of real-world street views show that our network outperforms state-of-the-arts methods of both semantic and geometric segmentation. The pixel-wise segmentation exhibits coarse boundaries and discontinuities. We then propose to rectify the pixel-wise segmentation into perspectively-projected quads based on spatial proximity between the segmentation masks and exterior line segments detected through an image processing. We demonstrate how the results can be utilized to perspectively overlay images and icons on building planes in input photos, and provide visual cues for various applications. [ABSTRACT FROM AUTHOR]

Head-mounted holographic displays (HMHD) are projected to be the first commercial realization of holographic video display systems. HMHDs use liquid crystal on silicon (LCoS) spatial light modulators (SLM), which are best suited to display phase-only holograms (POH). The performance/watt requirement of a monochrome, 60 fps Full HD, 2-eye, POH HMHD system is about 10 TFLOPS/W, which is orders of magnitude higher than that is achievable by commercially available mobile processors. To mitigate this compute power constraint, display-ready POHs shall be generated on a nearby server and sent to the HMHD in compressed form over a wireless link. This paper discusses design of a feasible HMHD-based augmented reality system, focusing on compression requirements and per-pixel rate-distortion trade-off for transmission of display-ready POH from the server to HMHD. Since the decoder in the HMHD needs to operate on low power, only coding methods that have low-power decoder implementation are considered. Effects of 2D phase unwrapping and flat quantization on compression performance are also reported. We next propose a versatile PCM-POH codec with progressive quantization that can adapt to SLM-dynamic-range and available bitrate, and features per-pixel rate-distortion control to achieve acceptable POH quality at target rates of 60–200 Mbit/s that can be reliably achieved by current wireless technologies. Our results demonstrate feasibility of realizing a low-power, quality-ensured, multi-user, interactive HMHD augmented reality system with commercially available components using the proposed adaptive compression of display-ready POH with light-weight decoding. [ABSTRACT FROM AUTHOR]

With the rapid development of deep learning technology and other powerful tools, 3D object detection has made great progress and become one of the fastest growing field in computer vision. Many automated applications such as robotic navigation, autonomous driving, and virtual or augmented reality system require estimation of accurate 3D object location and detection. Under this requirement, many methods have been proposed to improve the performance of 3D object localization and detection. Despite recent efforts, 3D object detection is still a very challenging task due to occlusion, viewpoint variations, scale changes, and limited information in 3D scenes. In this paper, we present a comprehensive review of recent state-of-the-art approaches in 3D object detection technology. We start with some basic concepts, then describe some of the available datasets that are designed to facilitate the performance evaluation of 3D object detection algorithms. Next, we will review the state-of-the-art technologies in this area, highlighting their contributions, importance, and limitations as a guide for future research. Finally, we provide a quantitative comparison of the results of the state-of-the-art methods on the popular public datasets. [ABSTRACT FROM AUTHOR]

In this paper, the global pinning synchronization problem is studied for a complex dynamical network to follow a dynamic target system. A distinguished feature of the present network model is that the target system may have some predesigned control inputs. This implies that, when pinning synchronization is guaranteed, the states of the nodes within such a pinning-controlled dynamical network may approach a specified trajectory which does not satisfy the system equation of the uncoupling individual node system within the network. The practical constraint that the external control inputs acting on the target system are unknown to any node in the considered network poses a big challenge in solving such a pinning synchronization problem. The designed scheme for achieving pinning synchronization is executed in two steps. Specifically, the first step is to select some nodes to pin such that the augmented interaction topology has at least one directed spanning tree rooted at the node describing the target system, while the second step is to construct a coupling law to synchronize all the states of nodes within the network. Moreover, two kinds of discontinuous coupling laws with static and adaptive coupling gains are, respectively, proposed to achieve pinning synchronization. Meanwhile, by utilizing nonsingular ${M}$ -matrix theory and Lyapunov stability analysis for nonsmooth system, some efficient criteria are established for guaranteeing synchronization in the pinning-controlled networks. Numerical simulations on pinning synchronization of networking Chua’s circuit systems are finally given to verify the analytic results. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) and virtual reality (VR) applications have seen rising popularity in recent years. Omnidirectional 360° video is a video format often used in AR and VR applications. To address the industry needs, a new HEVC edition recently published includes several supplemental enhancement information (SEI) messages to enable the carriage of omnidirectional video using HEVC. However, further improvement in 360° video compression efficiency is needed. In order to address this challenge, the Joint Video Exploration Team (JVET) of ITU-T VCEG and ISO/IEC MPEG has been investigating 360° video coding technologies, including projection formats, pre- and post-processing technologies, as well as 360°-video-specific coding tools since 2016. The joint call for proposals (CfP) recently issued by ITU-T VCEG and ISO/IEC MPEG on video compression technologies beyond HEVC included a category on 360° video. Twelve CfP responses in the 360° video category were received. This paper describes technologies relevant to 360° video for VVC. A summary of projection formats, pre- and post-processing methods, and 360°-video specific coding tool modifications in these proposals is provided. [ABSTRACT FROM AUTHOR]

This paper proposes a wearable monitoring system for inspection in the framework of Industry 4.0. The instrument integrates augmented reality (AR) glasses with a noninvasive single-channel brain–computer interface (BCI), which replaces the classical input interface of AR platforms. Steady-state visually evoked potentials (SSVEP) are measured by a single-channel electroencephalography (EEG) and simple power spectral density analysis. The visual stimuli for SSVEP elicitation are provided by AR glasses while displaying the inspection information. The real-time metrological performance of the BCI is assessed by the receiver operating characteristic curve on the experimental data from 20 subjects. The characterization was carried out by considering stimulation times from 10.0 down to 2.0 s. The thresholds for the classification were found to be dependent on the subject and the obtained average accuracy goes from 98.9% at 10.0 s to 81.1% at 2.0 s. An inspection case study of the integrated AR-BCI device shows encouraging accuracy of about 80% of lab values. [ABSTRACT FROM AUTHOR]

Objective: The focal length of available optical see-through (OST) head-mounted displays (HMDs) is at least 2 m; therefore, during manual tasks, the user eye cannot keep in focus both the virtual and real content at the same time. Another perceptual limitation is related to the vergence-accommodation conflict, the latter being present in binocular vision only. This paper investigates the effect of incorrect focus cues on the user performance, visual comfort, and workload during the execution of augmented reality (AR)-guided manual task with one of the most advanced OST HMD, the Microsoft HoloLens. Methods: An experimental study was designed to investigate the performance of 20 subjects in a connect-the-dots task, with and without the use of AR. The following tests were planned: AR-guided monocular and binocular, and naked-eye monocular and binocular. Each trial was analyzed to evaluate the accuracy in connecting dots. NASA Task Load Index and Likert questionnaires were used to assess the workload and the visual comfort. Results: No statistically significant differences were found in the workload, and in the perceived comfort between the AR-guided binocular and monocular test. User performances were significantly better during the naked eye tests. No statistically significant differences in performances were found in the monocular and binocular tests. The maximum error in AR tests was 5.9 mm. Conclusion: Even if there is a growing interest in using commercial OST HMD, for guiding high-precision manual tasks, attention should be paid to the limitations of the available technology not designed for the peripersonal space. [ABSTRACT FROM AUTHOR]

Task mapping has been a hot topic in multiprocessor system-on-chip software design for decades. During the mapping process, load balance (LB) and communication optimization have been two important performance optimization factors. This paper studies the relations between LB, interprocessor communications, and communication pipeline technique during the mapping process, and proposes an integer linear programming (ILP)-based static task mapping approach, which considers both LB and communication optimization. The approach consists of an optimized ILP model for task mapping with fewer variables compared to previous ILP mapping works. Moreover, to enhance the scalability of the ILP task mapping, the task-processor-cluster algorithm is proposed to reduce the scale of the task graph and the number of processors and then solve the coarse-grained input by the ILP mapping. To increase the adaptability of the ILP task mapping, the improved augmented $\epsilon $ -constraint method is further integrated with the ILP formulations to select the best mapping for different applications. Experimental results on a 2/4/8/16/24-CPU platform of both synthetic and real-life benchmarks demonstrate the efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper investigates the effectiveness of multipath-decorrelating antenna motion in reducing the initialization time of global navigation satellite system (GNSS) receivers employing low-cost single-frequency antennas for carrier-phase differential GNSS (CDGNSS) positioning. Fast initialization times with low-cost antennas will encourage the expansion of CDGNSS into the mass market, bringing the benefits of globally referenced centimeter-accurate positioning to many consumer applications, such as augmented reality and autonomous vehicles, that have so far been hampered by the several-meter-level errors of traditional GNSS positioning. Poor multipath suppression common to low-cost antennas results in large and strongly time-correlated phase errors when a receiver is static. Such errors can result in the CDGNSS initialization time, the so-called time to ambiguity resolution (TAR), extending to hundreds of seconds—many times longer than for higher cost survey-grade antennas, which have substantially better multipath suppression. This paper demonstrates that TAR can be significantly reduced through antenna motion, particularly gentle wavelength-scale random antenna motion. Such motion acts to decrease the correlation time of the multipath-induced phase errors. A priori knowledge of the motion profile is shown to further reduce TAR, with the reduction shown to be more pronounced as the initialization scenario is more challenging. [ABSTRACT FROM AUTHOR]

This paper presents a new algorithm for the fast, shared memory, multi-core computation of augmented contour trees on triangulations. In contrast to most existing parallel algorithms our technique computes augmented trees, enabling the full extent of contour tree based applications including data segmentation. Our approach completely revisits the traditional, sequential contour tree algorithm to re-formulate all the steps of the computation as a set of independent local tasks. This includes a new computation procedure based on Fibonacci heaps for the join and split trees, two intermediate data structures used to compute the contour tree, whose constructions are efficiently carried out concurrently thanks to the dynamic scheduling of task parallelism. We also introduce a new parallel algorithm for the combination of these two trees into the output global contour tree. Overall, this results in superior time performance in practice, both in sequential and in parallel thanks to the OpenMP task runtime. We report performance numbers that compare our approach to reference sequential and multi-threaded implementations for the computation of augmented merge and contour trees. These experiments demonstrate the run-time efficiency of our approach and its scalability on common workstations. We demonstrate the utility of our approach in data segmentation applications. [ABSTRACT FROM AUTHOR]

Six degree-of-freedom (6-DoF) visual tracking of dynamic objects is fundamental to a large variety of robotics and augmented reality (AR) applications. A key to this problem is accurate distance measurement of dynamic objects, which is usually obtained via stereo cameras, RGB-D sensors, or LiDARs. In this paper, however, we address the problem using only a monocular camera rigidly mounted with a low-cost inertial measurement unit. This is a light-weight, small-size, and low-cost solution, which is particularly suitable for tracking dynamic objects on drones or on mobile phones. Starting from a generic image-based two-dimensional tracker, we propose a novel method to resolve the object scale ambiguity in monocular vision in a geometric manner based on correlation analysis. This enables accurate metric three-dimensional tracking of arbitrary objects without requiring any prior knowledge about the object shape or size. We discuss the applicability by analyzing the observability condition and degenerated cases for object scale recovery. Simulation and real-world experimental results with ground truth comparison, along with AR application examples, demonstrate the feasibility of the proposed 6-DoF tracking method. [ABSTRACT FROM AUTHOR]

Hybrid surgical simulators based on augmented reality (AR) solutions benefit from the advantages of both the box trainers and the virtual reality simulators. This paper reports on the results of a long development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and the virtual components. We first outline the specifications of the AR simulator and then we explain the strategy adopted for implementing it based on a careful selection of its simulated anatomical components, and characterized by a real-time tracking of both a target anatomy and of the laparoscope. The former is tracked by means of an electromagnetic field generator, while the latter requires an additional camera for video tracking. The new system was evaluated in terms of AR visualization accuracy, realism, and hardware robustness. Obtained results show that the accuracy of AR visualization is adequate for training purposes. The qualitative evaluation confirms the robustness and the realism of the simulator. In conclusion, the proposed AR simulator satisfies all the initial specifications in terms of anatomical appearance, modularity, reusability, minimization of spare parts cost, and ability to record surgical errors and to track in real-time the Calot's triangle and the laparoscope. Thus, the proposed system could be an effective training tool for learning the task of identification and isolation of Calot's triangle in laparoscopic cholecystectomy. Moreover, the presented strategy could be applied to simulate other surgical procedures involving the task of identification and isolation of generic tubular structures, such as blood vessels, biliary tree, and nerves, which are not directly visible. [ABSTRACT FROM AUTHOR]

Along with the evolution of wireless technologies, users are expected to have high-throughput and low-latency indoor mobile applications, such as augmented reality and virtual reality. Therefore, the use of millimeter-wave for low-latency communications becomes attractive. Under low-latency criteria, a fast multi-link establishment is desired. In this paper, we present a novel concurrent multi-beam training jointly with a proposed multi-beam codebook. First, we propose a progressive multi-beam estimation (PMBE). This algorithm makes multiple coarse beams emerge within a few training steps. Second, we propose a Fast Fourier Transform (FFT)-based codebook which can be considered as a Discrete Fourier Transform based codebook built in a bit-reversal scrambler, thus forming a scattering multi-beam codebook. This FFT-based codebook design is optimally implemented on a sub-connected architecture, a hardware-efficient architecture. For fast multi-link establishment, we design our concurrent multi-beam training by combining the proposed PMBE jointly with the proposed FFT-based codebook, so as to further improve multiplexing gain. Since multiplexing gain can occur immediately, even under strictly-limited training steps, the high data rate can be achieved. Simulation results show that when reaching more than 95% capacity gain of exhaustive channel estimation, our work only spends 1% training steps of the exhaustive case. [ABSTRACT FROM AUTHOR]

The viewing safety is one of the main issues in viewing virtual reality (VR) content. In particular, VR sickness could occur when watching immersive VR content. To deal with the viewing safety for VR content, objective assessment of VR sickness is of great importance. In this paper, we propose a novel objective VR sickness assessment (VRSA) network based on deep generative model for automatically predicting the VR sickness score. The proposed method takes into account motion patterns of VR videos in which an exceptional motion is a critical factor inducing excessive VR sickness in human motion perception. The proposed VRSA network consists of two parts, which are VR video generator and VR sickness score predictor. By training the VR video generator with common videos with non-exceptional motion, the generator learns the tolerance of VR sickness in human motion perception. As a result, the difference between the original and the generated videos by the VR video generator could represent exceptional motion of VR video causing VR sickness. In the VR sickness score predictor, the VR sickness score is predicted by projecting the difference between the original and the generated videos onto the subjective score space. For the evaluation of VR sickness assessment, we built a new dataset which consists of 360° videos (stimuli), corresponding physiological signals, and subjective questionnaires from subjective assessment experiments. Experimental results demonstrated that the proposed VRSA network achieved a high correlation with human perceptual score for VR sickness. [ABSTRACT FROM AUTHOR]

This paper proposes a weighted large margin nearest center (WLMNC) distance-based human depth recovery method for tele-immersive video interaction systems with limited bandwidth consumption. In the remote stage, the proposed method highly compresses the depth data of the remote human into skeletal block structures by learning the WLMNC distance, which is equivalent to downsampling the human depth map at $64{\times}$ the sampling rate. In the local stage, the method first recovers a rough human depth map based on a WLMNC distance augmented clustering approach and then obtains a fine depth map based on a rough depth-guided autoregressive model to preserve the depth discontinuities and suppress texture copy artifacts. The proposed WLMNC distance is learned by the large margin clustering problem with a weighted hinge loss to balance the clustering accuracy and depth recovery accuracy and is verified to be able to preserve depth discontinuities between skeletal block structures with occlusion. A theoretical analysis is conducted to verify the effectiveness of using the weighted hinge loss. Furthermore, a novel data set containing various types of human postures with self-occlusion is built to benchmark the human depth recovery methods. The quantitative comparison with the state-of-the-art depth recovery methods on the introduced benchmark data set demonstrates the effectiveness of the proposed method for human depth recovery with such a high upsampling rate. [ABSTRACT FROM AUTHOR]

In recent years, augmented reality (AR) is considered a promising technology that combines virtual information such as videos, images, and three-dimensional objects with a real camera view in mobile platforms. Interactive AR further provides human–computer interaction to allow the user to interact with virtual objects on the mobile display. In this paper, we proposed a cloud-based mobile augmented reality interactive system (MARIS), which includes MARIS-I for image target tracking and MARIS-H for hand motion tracking. MARIS-I estimates the position of the image target by adopting a feature-based mean-shift algorithm, which is feasible for real-time applications with its small region feature detection. MARIS-H provides two tracking modes for fingertip and back of hand tracking to enhance user experiences (UX) for interaction. Either the center position for the back of hand or fingertip is first estimated by particle filtering technique, which calculates the weighting of each particle according to hand or fingertip model. Afterward, the contour of the fingertip is estimated by level-set-based contour evolution in the fingertip tracking mode. Furthermore, we implement a device/cloud architecture for the proposed MARIS to decrease memory requirement and computational complexity on the device side. Experimental results show that MARIS including MARIS-I and MARIS-H can outperform other existing methods for image and hand motion tracking, respectively. The proposed MARIS is demonstrated in a picture book to provide fruitful interactive UX for digital learning systems. [ABSTRACT FROM AUTHOR]

This paper proposes a scalable Augmented Reality (AR) framework enabled by a simple extension to the Internet of Things (IoT) infrastructure, called the "ARIoT." Through ARIoT (1) potential user-proximal target objects are dynamically identified in any IoT-enabled space, (2) tracking feature information is directly obtained from the IoT objects enabling fast recognition and tracking (independent of the number of the total IoT objects everywhere), and (3) generic interactive contents are augmented as an IoT service to the AR client. To further improve the AR tracking performance, which often depends on feature appearances, this paper introduces a method in which the preferred tracking method is determined by the very IoT object in hand. The proposed concept of ARIoT is demonstrated, and its performance is evaluated and compared with the conventional single serveror client-based AR implementation. In addition, a pilot usability test is conducted between a spatially registered visual in-situ AR interface using ARIoT and a conventional interface based only on an online connection, and the results are then compared. This study shows how AR and IoT can be complementary to each other: AR as an attractive visual insitu interface for IoT objects and IoT as a basis for scalable AR1 [ABSTRACT FROM PUBLISHER]

One camera and one low-cost inertial measurement unit (IMU) form a monocular visual-inertial system (VINS), which is the minimum sensor suite (in size, weight, and power) for the metric six degrees-of-freedom (DOF) state estimation. In this paper, we present VINS-Mono: a robust and versatile monocular visual-inertial state estimator. Our approach starts with a robust procedure for estimator initialization. A tightly coupled, nonlinear optimization-based method is used to obtain highly accurate visual-inertial odometry by fusing preintegrated IMU measurements and feature observations. A loop detection module, in combination with our tightly coupled formulation, enables relocalization with minimum computation. We additionally perform 4-DOF pose graph optimization to enforce the global consistency. Furthermore, the proposed system can reuse a map by saving and loading it in an efficient way. The current and previous maps can be merged together by the global pose graph optimization. We validate the performance of our system on public datasets and real-world experiments and compare against other state-of-the-art algorithms. We also perform an onboard closed-loop autonomous flight on the microaerial-vehicle platform and port the algorithm to an iOS-based demonstration. We highlight that the proposed work is a reliable, complete, and versatile system that is applicable for different applications that require high accuracy in localization. We open source our implementations for both PCs (https://github.com/HKUST-Aerial-Robotics/VINS-Mono) and iOS mobile devices (https://github.com/HKUST-Aerial-Robotics/VINS-Mobile). [ABSTRACT FROM AUTHOR]

This paper proposes a novel multirate co-simulation method to improve simulation efficiency of large-scale AC plus multiterminal DC (MTDC) grids. In this method, the whole system is partitioned into different AC and MTDC subsystems. They each are simulated with different time steps according to their requirements of accuracy. Unlike the existing methods where simplified models are adopted, the proposed approach fully reserves both the detailed behavior of converters and the nonlinear dynamics of large-scale AC systems. To realize the coordination between different subsystems, updating and discretization of the interface models are significant to guarantee the overall numerical accuracy and stability of the co-simulation method. Accordingly, the interface models of the partitioned AC and DC networks are represented by time-varying Thevenin and Norton equivalents. To eliminate the aliasing or time-delay errors, their parameters are derived using moving-window prediction, stepwise correction, and averaging techniques. A numerical oscillation suppression algorithm is developed for the discretization of the interface model. Thus, numerical stability is well guaranteed. The variables inside each subsystem as well as the interfaces, which are derived by augmented network equations (ANEs), are calculated simultaneously. The rate ratio of large AC and MTDC systems is determined based on our proposed criterion of electromagnetic transient simulation accuracy. The effectiveness of the cosimulation method is validated on a practical system integrating large AC and modular-multilevel-converter-based MTDC grids. [ABSTRACT FROM AUTHOR]

Multiple patterning lithography has been recognized as one of the most promising solutions, in addition to extreme ultraviolet lithography, directed self-assembly, nanoimprint lithography, and electron beam lithography, for advancing the resolution limit of conventional optical lithography. Multiple patterning layout decomposition (MPLD) becomes more challenging as advanced technology introduces complex coloring rules. Existing works model MPLD as a graph coloring problem; nevertheless, when complex coloring rules are considered, layout decomposition can no longer be modeled accurately by graph coloring. Therefore, in this paper, for capturing the essence of layout decomposition with complex coloring rules, we model the MPLD problem as an exact cover problem. We then propose a fast and exact MPLD framework based on augmented dancing links. Our method is flexible and general: it can consider the basic and complex coloring rules simultaneously, can maintain density balancing, and can handle quadruple patterning and beyond. Experimental results show that our approach outperforms state-of-the-art works on reported conflicts and stitches and is promising for handling complex coloring rules and density balancing as well. [ABSTRACT FROM PUBLISHER]

Parkinson's disease (PD) is a chronic progressive disease caused by loss of dopaminergic neurons in the substantia nigra, degenerating the nervous system of a patient over time. Freezing of gait (FOG), which is a form of akinesia, is a symptom of PD. Meanwhile, recent studies show that the gait of PD patients experiencing FOG can be significantly improved by providing the regular visual or auditory patterns for the patients. In this paper, we propose a gait-aid system built upon smart glasses. Our system continuously monitors the gait and so on of a PD patient to detect FOG, and upon detection of FOG it projects visual patterns on the glasses as if the patterns were actually on the floor. Conducting experiments involving ten PD patients, we demonstrate that our system achieves the accuracy of 92.86$\%$ in detecting FOG episodes and that it improves the gait speed and stride length of PD patients by 15.3 $\sim$ 37.2% and 18.7  $\sim$ 31.7%, respectively. [ABSTRACT FROM AUTHOR]

In this paper, we present an efficient numerical scheme for the recently introduced geodesic active fields (GAF) framework for geometric image registration. This framework considers the registration task as a weighted minimal surface problem. Hence, the data-term and the regularization-term are combined through multiplication in a single, parametrization invariant and geometric cost functional. The multiplicative coupling provides an intrinsic, spatially varying and data-dependent tuning of the regularization strength, and the parametrization invariance allows working with images of nonflat geometry, generally defined on any smoothly parametrizable manifold. The resulting energy-minimizing flow, however, has poor numerical properties. Here, we provide an efficient numerical scheme that uses a splitting approach; data and regularity terms are optimized over two distinct deformation fields that are constrained to be equal via an augmented Lagrangian approach. Our approach is more flexible than standard Gaussian regularization, since one can interpolate freely between isotropic Gaussian and anisotropic TV-like smoothing. In this paper, we compare the geodesic active fields method with the popular Demons method and three more recent state-of-the-art algorithms: NL-optical flow, MRF image registration, and landmark-enhanced large displacement optical flow. Thus, we can show the advantages of the proposed FastGAF method. It compares favorably against Demons, both in terms of registration speed and quality. Over the range of example applications, it also consistently produces results not far from more dedicated state-of-the-art methods, illustrating the flexibility of the proposed framework. [ABSTRACT FROM AUTHOR]

In the last years, more and more research activities have focused on the application of Virtual Reality (VR) and related cutting-edge immersive technologies like Augmented Reality (AR) and Mixed Reality (MR) to vehicular systems for supporting the design of novel Human-Machine Interaction techniques, simulating the functioning of newly developed components, recreating hazardous test environments, establishing trust in Connected and Autonomous Vehicles (CAVs), etc. The benefits brought by immersive environments for supporting simulations in the field of vehicular technology have already been recognized by many companies and academic institutions worldwide, as confirmed by the huge investments and a large number of publications. As we approach the time in which immersive simulations will become fundamental for a wide range of applications in the field of vehicular technology, the goal of this Special Section was to attract high-quality submissions reporting state-of-the-art research activities targeted to next-generation vehicles and related infrastructures. The Special Section received 12 submissions, which were prescreened and reviewed by experts in the field. Four papers were ultimately accepted. [ABSTRACT FROM AUTHOR]

This paper presents a new approach for self-calibration of static cameras in the context of surveillance applications. Initially, a pedestrian detector is applied and the responses are validated using background removal. Then, foreground-related pixels within the detection results are used to estimate the feet-head line segments of each person (called poles), which are used to find a linear estimate for the camera matrix. Finally, a nonlinear cost function is used to refine the initial estimate, aiming to mostly improve the orientation of the reprojected poles. We also present different applications of self-calibration in tasks related to video surveillance itself, such as improvements to pedestrian detection and tracking algorithms, and augmented reality applications, such as the insertion of virtual cameras to aid the placement of real cameras in the scene. [ABSTRACT FROM PUBLISHER]

Fastener assembly is a tedious and time-consuming work because operators have to check assembly manuals and find right fastener for each hole. Hence, this paper aims to develop a three-dimensional (3-D) projection system that projects assembly instruction onto the work piece surface directly to guide operators to assemble. However, in order to project the instruction accurately, the corresponding part of the computer-aided design model of the physical scanned area needs to be attained through the rapid and accurate registration. In order to achieve this goal, first, a high-accuracy and rapid 3-D measurement system is developed; second, a fast registration method based on local multiscale geometric feature vector is proposed to accelerate the registration speed and improve the registration reliability. Experimental results demonstrate the measurement accuracy of the developed system, and verify the feasibility of the proposed registration method. Hence, the proposed method can lead to improved assembly efficiency and decreased error probability, making great contributions to large-scale structure assembly. [ABSTRACT FROM PUBLISHER]

Augmented reality (AR) provides huge opportunities for online teaching in science and engineering, as these disciplines place emphasis on practical training and unsuited to completely nonclassroom training. This paper proposes a new concept in virtual and remote laboratories: the augmented remote laboratory (ARL). ARL is being tested in the first and second years of the new degrees in industrial engineering and computer engineering, respectively, at the School of Engineering, University of Huelva, Huelva, Spain. By means of augmented reality techniques, ARL allows the student to experience sensations and explore learning experiences that, in some cases, may exceed those offered by traditional laboratory classes. The effectiveness of this methodology for remote laboratory work is evaluated by comparing it to practical sessions in the laboratory at the university itself with the same group of students. Students completed a questionnaire after having experienced both types of practicals, and the results show that the use of ARL improves student outcomes. As discussed in the paper, the potential of AR to configure different experiments from the same physical configuration is virtually limitless. [ABSTRACT FROM AUTHOR]

This paper describes a novel light-sensor-based information transmission system for indoor positioning and navigation with particular benefits for mobile and wearable computers. It can seamlessly extend outdoor GPS tracking to the indoor environment. In a novel manner, fluorescent light is used as the medium to transmit information, which is encoded by using a pulse-frequency modulation technique. The user receives the encoded light information through a photoreceiver. The information is passed into the wearable or mobile computer after the data are decoded. This information allows positioning information to be given to indoor mobile and wearable computers. We design an economical transmitter circuit by adding a few components to a commercial electronic ballast circuit for fluorescent lamps with a price of less than 10 USD. The proposed system can be used in indoor guidance and navigation applications. Exemplary applications are included in the paper, with experimented results detailed. [ABSTRACT FROM AUTHOR]

Ever since the development of the first applications in image-guided therapy (IGT), the use of head-mounted displays (HMDs) was considered an important extension of existing IGT technologies. Several approaches to utilizing HMDs and modified medical devices for augmented reality (AR) visualization were implemented. These approaches include video-see through systems, semitransparent mirrors, modified endoscopes, and modified operating microscopes. Common to all these devices is the fact that a precise calibration between the display and three-dimensional co-ordinates in the patient's frame of reference is compulsory. In optical see-through devices based on complex optical systems such as operating microscopes or operating binoculars—as in the case of the system presented in this paper—this procedure can become increasingly difficult since precise camera calibration for every focus and zoom position is required. We present a method for fully automatic calibration of the operating binocular Varioscope™ MS AR for the full range of zoom and focus settings available. Our method uses a special calibration pattern, a linear guide driven by a stepping motor, and special calibration software. The overlay error in the calibration plane was found to be 0.14-0.91 mm, which is less than 1% of the field of view. Using the motorized calibration rig as presented in the paper, we were also able to assess the dynamic latency when viewing augmentation graphics on a mobile target; spatial displacement due to latency was found to be in the range of 1.1-2.8 mm maximum, the disparity between the true object and its computed overlay represented latency of 0.1 s. We conclude that the automatic calibration method presented in this paper is sufficient in terms of accuracy and time requirements for standard uses of optical see-through systems in a clinical environment. [ABSTRACT FROM AUTHOR]

This paper presents the MPEG-4 Animation Frame- work extension (AFX) standard, ISOIIEC 14496-16. Initiated by the MPEG Synthetic/Natural Hybrid Coding group in 2000, MPEG-4 AFX proposes an advanced framework for interactive multimedia applications using both natural and synthetic objects. Following this model, new synthetic objects have been specified, increasing content realism over existing MPEG-4 synthetic objects. The general overview of MPEG-4 AFX is provided on top of the review of MPEG-4 standards to explain the relationship between MPEG-4 and MPEG-4 AFX. Then we give a bird's-eye view of new tools available in this standard and further explained in other paper in this special issue. [ABSTRACT FROM AUTHOR]

In image-guided robotic surgery, segmenting the endoscopic video stream into meaningful parts provides important contextual information that surgeons can exploit to enhance their perception of the surgical scene. This information provides surgeons with real-time decision-making guidance before initiating critical tasks such as tissue cutting. Segmenting endoscopic video is a challenging problem due to a variety of complications including significant noise attributed to bleeding and smoke from cutting, poor appearance contrast between different tissue types, occluding surgical tools, and limited visibility of the objects’ geometries on the projected camera views. In this paper, we propose a multi-modal approach to segmentation where preoperative 3D computed tomography scans and intraoperative stereo-endoscopic video data are jointly analyzed. The idea is to segment multiple poorly visible structures in the stereo/multichannel endoscopic videos by fusing reliable prior knowledge captured from the preoperative 3D scans. More specifically, we estimate and track the pose of the preoperative models in 3D and consider the models’ non-rigid deformations to match with corresponding visual cues in multi-channel endoscopic video and segment the objects of interest. Further, contrary to most augmented reality frameworks in endoscopic surgery that assume known camera parameters, an assumption that is often violated during surgery due to non-optimal camera calibration and changes in camera focus/zoom, our method embeds these parameters into the optimization hence correcting the calibration parameters within the segmentation process. We evaluate our technique on synthetic data, ex vivo lamb kidney datasets, and in vivo clinical partial nephrectomy surgery with results demonstrating high accuracy and robustness. [ABSTRACT FROM AUTHOR]

This paper reports empirical evidence on having students use AR-SaBEr, a simulation tool based on augmented reality (AR), to discover the basic principles of electricity through a series of experiments. AR-SaBEr was enhanced with knowledge-based support and inquiry-based scaffolding mechanisms, which proved useful for discovery learning in Web-based simulation environments. Learning performance factors evaluated included students' learning behavior while interacting with the system, AR-SaBEr's learning effectiveness, and students' motivation in using the AR-based simulation environment. The study suggests that AR simulators can be exploited as effective learning environments for learning the basic principles of electricity. It would, however, be necessary to consider a number of scaffolding options to improve the learning effectiveness of AR-based simulation environments when used with students who have low levels of either: 1) self-regulation skills, or 2) motivation to engage in AR-based simulation activities. [ABSTRACT FROM AUTHOR]

Planning surgical interventions is a complex task, demanding a high degree of perceptual, cognitive, and sensorimotor skills to reduce intra- and post-operative complications. This process requires spatial reasoning to coordinate between the preoperatively acquired medical images and patient reference frames. In the case of neurosurgical interventions, traditional approaches to planning tend to focus on providing a means for visualizing medical images, but rarely support transformation between different spatial reference frames. Thus, surgeons often rely on their previous experience and intuition as their sole guide is to perform mental transformation. In case of junior residents, this may lead to longer operation times or increased chance of error under additional cognitive demands. In this paper, we introduce a mixed augmented-/virtual-reality system to facilitate training for planning a common neurosurgical procedure, brain tumour resection. The proposed system is designed and evaluated with human factors explicitly in mind, alleviating the difficulty of mental transformation. Our results indicate that, compared to conventional planning environments, the proposed system greatly improves the nonclinicians’ performance, independent of the sensorimotor tasks performed ($p\,<\,0.01$ ). Furthermore, the use of the proposed system by clinicians resulted in a significant reduction in time to perform clinically relevant tasks ($p\,<\,0.05$ ). These results demonstrate the role of mixed-reality systems in assisting residents to develop necessary spatial reasoning skills needed for planning brain tumour resection, improving patient outcomes. [ABSTRACT FROM AUTHOR]

Experiments suggest that using head-up displays like Google Glass to support parts picking for distribution results in fewer errors than current processes. Making Glass opaque instead of transparent further improves selection efficiency. The Web extra at http://youtu.be/yUZFaCP6rP4 is a video demonstrating that order picking assisted by head-up display (HUD) is faster, has fewer errors, requires less workload, and is preferred to order picking assisted by pick-by-light, paper pick list, and cart-mounted display (CMD). The second Web extra at http://youtu.be/RApBJ0U3XpI is a video demonstrating that order picking assisted by a Google Glass with an opaque display is three percent faster than with a transparent display (default). [ABSTRACT FROM PUBLISHER]

By overlaying preoperative radiological 3-D models onto the intraoperative laparoscopic video, augmented-reality (AR) displays promise to increase surgeons’ visual awareness of high-risk surgical targets (e.g., the location of a tumor). Existing AR surgical systems lack in robustness and accuracy because of the many challenges in endoscopic imagery, such as frequent changes in illumination, rapid camera motions, prolonged organ occlusions, and tissue deformations. The frequent occurrence of these events can cause the loss of image (anchor) points, and thus, the loss of the AR display after a few frames. In this paper, we present the design of a new AR system that represents a first step toward long term and accurate augmented surgical display for monocular (calibrated and uncalibrated) endoscopic videos. Our system uses correspondence-search methods, and a new weighted sliding-window registration approach, to automatically and accurately recover the overlay by predicting the image locations of a high number of anchor points that were lost after a sudden image change. The effectiveness of the proposed system in maintaining a long term (over 2 min) and accurate (less than 1 mm) augmentation has been documented over a set of real partial-nephrectomy laparascopic videos. [ABSTRACT FROM PUBLISHER]

The diffusion of electronics in everyday life requires for the development of sensing systems, based on new materials and technologies. Deformation sensors are of interest in many application fields, ranging from assisted rehabilitation, virtual reality and augmented reality, and gaming applications, juts to mention a few. The diffusion of low-cost sensing systems outlines the need for environmentally friendly and low-cost sensors. Cellulose is one of the most abundant materials on earth. Eventually, it is green, cheap, and flexible. In this article, the deformation sensing properties of a three-layer structure-based bacterial cellulose (BC) compound based on BC, impregnated by ionic liquids, and covered by conducting polymers are investigated. More specifically, it is shown that this novel composite is capable of sensing flexural deformations and producing a corresponding open-circuit voltage. The obtained results pave the road to the possibility of realizing a new class of deformation sensors that fit the requirements of more sustainable sensor technologies. [ABSTRACT FROM AUTHOR]

The efficiency and quality of a feature descriptor are critical to the user experience of many computer vision applications. However, the existing descriptors are either too computationally expensive to achieve real-time performance, or not sufficiently distinctive to identify correct matches from a large database with various transformations. In this paper, we propose a highly efficient and distinctive binary descriptor, called local difference binary (LDB). LDB directly computes a binary string for an image patch using simple intensity and gradient difference tests on pairwise grid cells within the patch. A multiple-gridding strategy and a salient bit-selection method are applied to capture the distinct patterns of the patch at different spatial granularities. Experimental results demonstrate that compared to the existing state-of-the-art binary descriptors, primarily designed for speed, LDB has similar construction efficiency, while achieving a greater accuracy and faster speed for mobile object recognition and tracking tasks. [ABSTRACT FROM AUTHOR]