Search

Machine vision measurement is desirable to permit real-time non-contact measuring and positioning for hot forgings, among which edge extraction is a most essential issue to extract the contour and effective area. However, conventional edge detection methods are prone to get unsatisfactory edging extraction results, thus have poor effectiveness, and are not suitable for hot forging images. In this paper, an efficient and robust edge extraction approach for passive vision images of hot forgings is proposed. Grayscale images of hot forgings converted into discrete gray surface, the approach is based on the geometric properties and the continuity of the equivalent discrete grayscale surface. The presented algorithm detects three types of edges by various continuity criterions, which are corresponded to the geometric properties and vary with the primary and secondary edges. The geometric properties dependent nature of the algorithm ensures the primary and the secondary edges of the forges are identified in the different environmental conditions and for forging parts with various heat radiation intensities. Moreover, an edge thinning and connection approach is presented by defining the edging direction, which can be used to improve the qualities of types of edges. Finally, experimentations for images of various sorts of hot forgings are carried out to extract three types of edges; the relevant experimental results and validation indicators show that the proposed method takes better performance as 17.4453 in PSNR and 0.1146 in entropy for G0 edge for a typical forging image while 0.0342 for G2 edge compared with existing methods. The results demonstrate that the proposed approach is validated to have satisfactory performance, as well as efficacy and robustness. [ABSTRACT FROM AUTHOR]

Multi-View Stereo (MVS) is a long-standing and fundamental task in computer vision, which aims to reconstruct the 3D geometry of a scene from a set of overlapping images. With known camera parameters, MVS matches pixels across images to compute dense correspondences and recover 3D points. Traditional MVS methods often encounter difficulties in addressing complex scenes due to limitations in robustness. However, recent advancements in deep-learning-based MVS have demonstrated remarkable performance gains, attributed to their enhanced feature extraction and cost volume processing capabilities. This comprehensive survey delves into deep-learning-based MVS algorithms, highlighting key challenges and avenues for future exploration. Our focus is on studies aimed at mitigating algorithmic memory requirements and elevating processing speeds. To better understand the developments in this field, we propose a new taxonomy. The deep-learning-based MVS algorithms are categorized into two categories: scene-oriented and view-oriented methods. Scene-oriented methods typically employ voxels or neural implicit representations as the representative form of the scene and infer the complete scene directly. View-oriented methods focus on estimating the depth map of a single view. Detailed classifications and reviews have been conducted for each category. Furthermore, we introduce several widely used datasets and metrics and empirical evaluation of representative algorithms across three popular datasets. This article concludes by discussing prevalent challenges in the MVS domain and promising research directions for the future. [ABSTRACT FROM AUTHOR]

Stereo matching is a popular topic in the image processing and computer vision fields. Although deep learning-based stereo matching approaches have achieved remarkable performance with respect to both estimation accuracy and computation efficiency, textureless and occluded regions and regions with edge blurring remain significant challenges for most existing stereo matching methods. To address these issues, we propose a geometric prior-guided stereo matching method with disparity fusion refinement, named GPDF-Net, in this paper. First, we exploit a geometric prior guidance module in the feature extraction part to enable features to obtain the global information of cross-view interactions and pay attention to the structural information contained in the input image. Second, we construct cost volume with disparity fusion refinement that introduces disparity-related geometric features to the concatenation volume and suppresses its redundant information to provide a better similarity measure. Third, we explore a method for replacing 3D convolution, making the cost aggregation module lighter and more efficient. Finally, we compare the proposed method with several state-of-the-art approaches on the Scene Flow and KITTI test databases. The experimental results demonstrate that the proposed method achieves competitive performance with respect to both accuracy and robustness, and it produces better results than those of other methods when given blurred edges and textureless and occluded areas. (The code is available at https://github.com/PCwenyue.) [ABSTRACT FROM AUTHOR]

Fully exploiting the parallax information of stereo images for super-resolution (SR) can obtain remarkable performance. The most challenging issue for stereo image SR is how to capture complementary correlation information between the stereo image pair to accurately guide reconstruction. In this paper, we propose a multi-column smoothed dilated convolution based back projection network (MSDBPN) for stereo SR by explicitly learning and exploiting the parallax information. In particular, we incorporate adaptive weighted multi-column smoothed dilated convolutions to rapidly expand the receptive field while maintaining excellent inter-pixel correlation. Meanwhile, we reweight different column feature with adaptive learnable parameter to distinguish contributions. Furthermore, we employ a deep back projection mechanism to calculate projection error and implement self-correction to guide precise reconstruction. Extensive experiments on benchmark datasets demonstrate that our proposed method outperforms other state-of-the-art approaches on both quantitative and qualitative evaluations. [ABSTRACT FROM AUTHOR]

Featured Application: This research addresses the critical problem of accurate distance estimation using stereo camera systems, which are increasingly relevant for applications in metrology, robotics, and automated systems. The manuscript provides theoretical insights and practical experimental results on the performance and accuracy of stereo vision for distance estimation. Distance measurement plays a key role in many fields of science and technology, including robotics, civil engineering, and navigation systems. This paper focuses on analyzing the precision of a measurement system using stereo camera distance measurement technology in the context of measuring two objects of different sizes. The first part of the paper presents key information about stereoscopy, followed by a discussion of the process of building a measuring station. The Mask R-CNN algorithm, which is a deep learning model that combines object detection and instance segmentation, was used to identify objects in the images. In the following section, the calibration process of the system and the distance calculation method are presented. The purpose of the study was to determine the precision of the measurement system and to identify the distance ranges where the measurements are most precise. Measurements were made in the range of 20 to 70 cm. The system demonstrated a relative error of 0.95% for larger objects and 1.46% for smaller objects at optimal distances. A detailed analysis showed that for larger objects, the system exhibited higher precision over a wider range of distances, while for smaller objects, the highest accuracy was achieved over a more limited range. These results provide valuable information on the capabilities and limitations of the measurement system used, while pointing out directions for its further optimization. [ABSTRACT FROM AUTHOR]

This letter presents a novel self‐supervised learning strategy to improve the robustness of a monocular depth estimation (MDE) network against motion blur. Motion blur, a common problem in real‐world applications like autonomous driving and scene reconstruction, often hinders accurate depth perception. Conventional MDE methods are effective under controlled conditions but struggle to generalise their performance to blurred images. To address this problem, we generate blur‐synthesised data to train a robust MDE model without the need for preprocessing, such as deblurring. By incorporating self‐distillation techniques and using blur‐synthesised data, the depth estimation accuracy for blurred images is significantly enhanced without additional computational or memory overhead. Extensive experimental results demonstrate the effectiveness of the proposed method, enhancing existing MDE models to accurately estimate depth information across various blur conditions. [ABSTRACT FROM AUTHOR]

Stereo vision systems are increasingly utilized in various applications, however, the presence of noise significantly hampers the quality of the captured images. Traditional denoising methods often fail to address the complex noise patterns in such scenarios, which can adversely affect feature encoding and subsequent processing tasks. This paper introduces a novel stereo denoising approach that leverages cross-view information to enhance the robustness of noise reduction. A Cross-Channel and Spatial Context Information Mining Module is employed to encode long-range spatial dependencies and to bolster inter-channel feature interaction. This module utilizes large convolutional kernels, channel attention mechanisms, and a simple gating structure to enhance feature representation. Our approach relies on an encoder-decoder architecture, which facilitates cross-view and cross-scale feature interactions. The network is trained with a composite loss function that includes both spatial and perceptual domain constraints, ensuring a comprehensive optimization of the denoising process. Extensive experiments conducted on our proposed NoisyST dataset demonstrate the superior performance of our method in terms of noise removal and detail preservation. Notably, the method outperforms existing State-Of-The-Art techniques, as evidenced by its effectiveness in various evaluation metrics.

Three-dimensional (3-D) stereo images can be generated via computer-based image processing of CORONA stereo pairs. To a certain extent, important terrain and surface feature data extracted from these stereo images can improve the survey of archaeological sites and the identification and mapping of major landscapes. In this study, we focused on the identification of the archaeological ruins of Liangzhu City. An optical stereo model (red/blue stereo image) of the Liangzhu site was created through computer-based mosaicking and processing of CORONA remote-sensing stereo pairs taken in the 1960s and 1970s. By importing the optical stereo model into mobile phones, tablet computers, and other mobile devices, the research team undertook real-time locating of ruins via human observation, on-site investigation, and image overlay during a field survey and identified several Liangzhu-period dams, some of which have been confirmed via archaeological field investigations. The research team later applied the same method to the identification of tombs at the site of the mausoleums of the six emperors of the Southern Song dynasty. The results further prove that this method is feasible and reliable and can be widely promoted and used for the identification of archaeological ruins. [ABSTRACT FROM AUTHOR]

Abstract Generalizable and accurate stereo depth estimation is vital for 3D reconstruction, especially in surgery. Supervised learning methods obtain best performance however, limited ground truth data for surgical scenes limits generalizability. Self‐supervised methods don't need ground truth, but suffer from scale ambiguity and incorrect disparity prediction due to inconsistency of photometric loss. This work proposes a two‐phase training procedure that is generalizable and retains the high performance of supervised methods. It entails: (1) performing self‐supervised representation learning of left and right views via masked image modelling (MIM) to learn generalizable semantic stereo features (2) utilizing the MIM pre‐trained model to learn robust depth representation via supervised learning for disparity estimation on synthetic data only. To improve stereo representations learnt via MIM, perceptual loss terms are introduced, which improve the model's stereo representations learnt by explicitly encouraging the learning of higher scene‐level features. Qualitative and quantitative performance evaluation on surgical and natural scenes shows that the approach achieves sub‐millimetre accuracy and lowest errors respectively, setting a new state‐of‐the‐art. Despite not training on surgical nor natural scene data for disparity estimation.

Recently, deep learning‐based stereo matching has achieved great success. However, models trained on the source domain dataset encounter substantial performance degradation when directly tested on an unseen target domain dataset because of neglecting the generalization to out‐of‐distribution (OOD) stereo samples. This paper proposes a stereo sample generation‐based domain generalization network (SGDG‐Net) for stereo matching. Specifically, to expand the distribution span of training samples, OOD stereo samples are generated to assist training. To effectively generate OOD left samples, a style transfer‐based generation mechanism is proposed to transmit perturbations to the source left samples. In addition, to generate the OOD right samples, a disparity‐assisted generation strategy is proposed by using disparity map labels as auxiliary information. Experimental results demonstrate that the proposed SGDG‐Net produces remarkable results on four benchmark datasets. [ABSTRACT FROM AUTHOR]

In recent years, vision systems have become essential in the development of advanced driver assistance systems or autonomous vehicles. Although deep learning methods have been the center of focus in recent years to develop fast and reliable obstacle detection solutions, they face difficulties in complex and unknown environments where objects of varying types and shapes are present. In this study, a novel non-AI approach is presented for finding the ground-line and detecting the obstacles in roads using v-disparity data. The main motivation behind the study is that the ground-line estimation errors cause greater deviations at the output. Hence, a novel ground plane is defined as a region in the v-disparity map by using random variables to minimize these errors. In this new approach, weighted least squares regression, outlier detection, and camera height approximation were utilized for determining the ground region with higher accuracy. KITTY-2 dataset was chosen to conduct validation and evaluation experiments of the proposed approach. The experiment results were presented in GitHub, and the performance comparison shows that the proposed approach provides at least 20% improvement over Hough transform, which is a widely used non-AI algorithm. The results were also compared with a recently published article data and the best outcome was obtained among them for the recall metric. [ABSTRACT FROM AUTHOR]

Abstract Depth estimation from monocular panoramic image is a crucial step in 3D reconstruction, which is a close relationship with virtual reality and metaverse technologies. In recent years, some methods, such as HRDFuse, BiFuse++, and UniFuse, have employed a two‐branch neural network leveraging two common projections: equirectangular and cubemap projections (CMPs). The equirectangular projection (ERP) provides a complete field of view but introduces distortion, while the CMP avoids distortion but introduces discontinuity at the boundary of the cube. In order to address the issue of distortion and discontinuity, the authors propose an efficient depth estimation fusion module to balance the feature mapping of the two projections. Moreover, for the ERP, the authors propose a novel inflated network architecture to extend the receptive field and effectively harness visual information. Extensive experiments show that the authors’ method predicts more clear boundaries and accurate depth results while outperforming mainstream panoramic depth estimation algorithms.

Abstract Depth map super resolution has been paid much attention in 3D applications due to the limitation of depth sensors. Few textures in objects with clear contours along them is the most important characteristic of depth map. An efficient image representation should be directional, multiscale and anisotropic. From this we propose a novel multiscale and multidirection depth map super resolution framework with semantic inference to improve the quality of depth maps. In this framework, a multiscale and multidirection depth map contour fusion scheme captures and assembles intrinsic geometrical structures through a multiview non‐subsampled contourlet transform manner. This scheme not only isolates the discontinuities of contours but retains the smoothness along the contours. The semantic inference is also utilized to segment and label the depth map into objects/backgrounds‐level which are coplanar. Furthermore, a semantic‐aware label refinement strategy is introduced to correct the rarely inaccurate labels of the label map for upscaling the target pixel with pixels in the same object or background. Experimental results on benchmark depth map dataset demonstrate that the proposed multiscale and multidirection depth map super resolution framework with semantic inference has a significant improvement than the state‐of‐the‐art algorithms both visually and quantitatively.

Depth estimation from monocular panoramic image is a crucial step in 3D reconstruction, which is a close relationship with virtual reality and metaverse technologies. In recent years, some methods, such as HRDFuse, BiFuse++, and UniFuse, have employed a two‐branch neural network leveraging two common projections: equirectangular and cubemap projections (CMPs). The equirectangular projection (ERP) provides a complete field of view but introduces distortion, while the CMP avoids distortion but introduces discontinuity at the boundary of the cube. In order to address the issue of distortion and discontinuity, the authors propose an efficient depth estimation fusion module to balance the feature mapping of the two projections. Moreover, for the ERP, the authors propose a novel inflated network architecture to extend the receptive field and effectively harness visual information. Extensive experiments show that the authors' method predicts more clear boundaries and accurate depth results while outperforming mainstream panoramic depth estimation algorithms. [ABSTRACT FROM AUTHOR]

Recently, online optimisation‐based scene flow estimation has attracted significant attention due to its strong domain adaptivity. Although online optimisation‐based methods have made significant advances, the performance is far from satisfactory as only flow priors are considered, neglecting scene priors that are crucial for the representations of dynamic scenes. To address this problem, the authors introduce a dual‐branch MLP‐based architecture to encode implicit scene representations from a source 3D point cloud, which can additionally synthesise a target 3D point cloud. Thus, the mapping function between the source and synthesised target 3D point clouds is established as an extra implicit regulariser to capture scene priors. Moreover, their model infers both flow and scene priors in a stronger bidirectional manner. It can effectively establish spatiotemporal constraints among the synthesised, source, and target 3D point clouds. Experiments on four challenging datasets, including KITTI scene flow, FlyingThings3D, Argoverse, and nuScenes, show that our method can achieve potential and comparable results, proving its effectiveness and generality. [ABSTRACT FROM AUTHOR]

Visual discomfort significantly limits the broader application of stereoscopic display technology. Hence, the accurate assessment of stereoscopic visual discomfort is a crucial topic in this field. Electroencephalography (EEG) data, which can reflect changes in brain activity, have received increasing attention in objective assessment research. However, inaccurately labeled data, resulting from the presence of individual differences, restrict the effectiveness of the widely used supervised learning methods in visual discomfort assessment tasks. Simultaneously, visual discomfort assessment methods should pay greater attention to the information provided by the visual cortical areas of the brain. To tackle these challenges, we need to consider two key aspects: maximizing the utilization of inaccurately labeled data for enhanced learning and integrating information from the brain’s visual cortex for feature representation purposes. Therefore, we propose the weakly supervised graph convolution neural network for visual discomfort (WSGCN-VD). In the classification part, a center correction loss serves as a weakly supervised loss, employing a progressive selection strategy to identify accurately labeled data while constraining the involvement of inaccurately labeled data that are influenced by individual differences during the model learning process. In the feature extraction part, a feature graph module pays particular attention to the construction of spatial connections among the channels in the visual regions of the brain and combines them with high-dimensional temporal features to obtain visually dependent spatio-temporal representations. Through extensive experiments conducted in various scenarios, we demonstrate the effectiveness of our proposed model. Further analysis reveals that the proposed model mitigates the impact of inaccurately labeled data on the accuracy of assessment. [ABSTRACT FROM AUTHOR]

Abstract In the current stereo matching field, the accuracy of the derived disparity map is highly dependent on the processing capability of ill regions. Fortunately, we find that the use of local information will eliminate the negative effects associated with ill regions. As a result of the above discovery, we propose the Concatenated Dilated Convolution (CDC) block and the Multi‐scale Feature Fusion module (MFF), which are capable of effectively extracting regional context by increasing the receptive field in parallel and channel‐wise ways. The CDC block can expand the receptive field by applying multiple dilated convolutions at different dilation rates in parallel to enhance the smoothness of the feature map. By constructing parallel CDC blocks in a multiple dilated manner, the MFF module can improve the smoothness of the feature map. In addition, to control the number of parameters in the MFF network, a high‐performance channel‐distribution algorithm is proposed, capable of adjusting the weights of each module and convolution in an adaptive manner while reducing the number of parameters. Extensive experiments have demonstrated that MFF and CDC can effectively improve the performance of ill areas and networks with a minimal number of parameters.

Depth map super resolution has been paid much attention in 3D applications due to the limitation of depth sensors. Few textures in objects with clear contours along them is the most important characteristic of depth map. An efficient image representation should be directional, multiscale and anisotropic. From this we propose a novel multiscale and multidirection depth map super resolution framework with semantic inference to improve the quality of depth maps. In this framework, a multiscale and multidirection depth map contour fusion scheme captures and assembles intrinsic geometrical structures through a multiview non‐subsampled contourlet transform manner. This scheme not only isolates the discontinuities of contours but retains the smoothness along the contours. The semantic inference is also utilized to segment and label the depth map into objects/backgrounds‐level which are coplanar. Furthermore, a semantic‐aware label refinement strategy is introduced to correct the rarely inaccurate labels of the label map for upscaling the target pixel with pixels in the same object or background. Experimental results on benchmark depth map dataset demonstrate that the proposed multiscale and multidirection depth map super resolution framework with semantic inference has a significant improvement than the state‐of‐the‐art algorithms both visually and quantitatively. [ABSTRACT FROM AUTHOR]

This paper introduces a systematic approach to pothole detection, emphasizing its significance in road maintenance and safety. The proposed system is comprised of meticulously designed modules that collectively contribute to achieving accurate and effective results. Commencing with data collection through cameras capturing road areas of interest, the images undergo manual cropping and resizing for standardization. The core principle of the system revolves around image enhancement, involving grayscale conversion, application of blurring techniques, and adjustments to brightness and contrast. Automatic thresholding extracts essential information encoded within pixels, paving the way for precise edge detection refined through morphological operations. The focus then shifts to pothole detection, incorporating a stereo camera setup to calculate depth and disparity of road surfaces. Critical steps, including image rectification, correspondence matching, and disparity calculations, contribute to the accurate identification and delineation of potholes using depth thresholds. Subsequent modules employ K-Means clustering to segregate regions of interest from the image background, and post-processing steps, including filtering, masking, and refinement, fine-tune the images. Utilizing the HSV color space for grayscale image refinement and the connected component method to isolate white pixel objects further enhance the system's capability. The final step involves the addition of bounding boxes around identified potholes, streamlining their identification process. This comprehensive methodology ensures effective image processing, precise pothole detection, and contributes significantly to road safety and maintenance efforts. [ABSTRACT FROM AUTHOR]

Abstract Cost aggregation is crucial to the accuracy of stereo matching. A reasonable cost aggregation algorithm should aggregate costs within homogeneous regions where pixels have the same or similar disparities. Otherwise, the estimated disparity map is prone to the well‐known edge‐fattening issue and the problem of losing fine structures. However, current state‐of‐the‐art convolutional neural networks (CNNs) mainly do cost aggregation with square‐kernel convolutional layers that learn to adjust their kernel elements to make the actual receptive fields of the aggregated costs adapt to homogeneous regions with various shapes. This is a mechanism that easily leads to the above issues due to the translation equivalence and content‐insensitivity properties of CNNs. To tackle these problems, a novel densely connected asymmetric convolution block (Dense‐ACB) based on asymmetric convolutions is proposed to explicitly construct receptive fields with various shapes, which effectively alleviates the issues caused by mismatching shapes of receptive fields and homogeneous regions. The proposed Dense‐ACB brings new insight to CNN‐based stereo matching networks. Based on the proposed cost aggregation method, an efficient and effective stereo matching network is built, which not only achieves competitive overall accuracy compared with state‐of‐the‐art models but also effectively alleviates the edge‐fattening problem and preserves fine structures.

This paper constructs an energy model based on local features used in stereo matching. The local features include the similarity between different image areas, the matching cost function pattern, the connection between neighbor pixels, and the occlusion geometric relationship. Based on these features, we define the weight of each data term and smoothing term in the energy function and then design an algorithm to solve the energy model and get disparity results. The significant improvements of this paper include as following. 1) We modify the structure of the energy function. First, we define the weight of the data term based on the reliability of its corresponding disparity result, which is obtained by cost function features and the occlusion geometric relationship. Then we define the weight of the smoothing term by analyzing the characteristic relation between neighbor super-pixels. We can also reduce the computational complexity by detecting and reducing some low-strength connections. 2) We proposed an algorithm based on pairwise Markov random field (MRF) (Taniai et al., IEEE Trans Pattern Anal Machine Intell 40(11): 2725–2739, 2017) and local greedy iteratively, which can be used to solve the energy model. 3) In post-optimation, we select some areas with severe occlusion and fewer matching clues for post-interpolation fitting to optimize the results. The experiment shows that the proposed method reduced the average percentage of bad pixels (in bad 3) to 6.06 on the Middlebury dataset and 1.42 on the KITTI dataset. Finally, we compare our results with those of MC-Cnn (Zbontar and LeCun 2015), CF-Net (Shen et al., 2021), Guided-Stereo (Poggi et al., 2019), Gwc-Net (Guo et al., 2019) and Patchmatch-Net(PM-Net) (Wang et al., 2021) to verify the improved speed and accuracy of our algorithm, especially at recognizing the depth of changing edges and small objects. This paper's relevant research can contribute to practical engineering practices such as assisted vision, intelligent driving, and robot grasping control. [ABSTRACT FROM AUTHOR]

In the current stereo matching field, the accuracy of the derived disparity map is highly dependent on the processing capability of ill regions. Fortunately, we find that the use of local information will eliminate the negative effects associated with ill regions. As a result of the above discovery, we propose the Concatenated Dilated Convolution (CDC) block and the Multi‐scale Feature Fusion module (MFF), which are capable of effectively extracting regional context by increasing the receptive field in parallel and channel‐wise ways. The CDC block can expand the receptive field by applying multiple dilated convolutions at different dilation rates in parallel to enhance the smoothness of the feature map. By constructing parallel CDC blocks in a multiple dilated manner, the MFF module can improve the smoothness of the feature map. In addition, to control the number of parameters in the MFF network, a high‐performance channel‐distribution algorithm is proposed, capable of adjusting the weights of each module and convolution in an adaptive manner while reducing the number of parameters. Extensive experiments have demonstrated that MFF and CDC can effectively improve the performance of ill areas and networks with a minimal number of parameters. [ABSTRACT FROM AUTHOR]

Monocular 3D object detection has drawn increasing attention in various human-related applications, such as autonomous vehicles, due to its cost-effective property. On the other hand, a monocular image alone inherently contains insufficient information to infer the 3D information. In this paper, we propose a new monocular 3D object detector that can recall the stereoscopic visual information about an object, given a left-view monocular image. Here, we devise a location embedding module to handle each object by being aware of its location. Next, given the object appearance of the left-view monocular image, we devise Monocular-to-Stereoscopic (M2S) memory that can recall the object appearance of the right-view and depth information. For this purpose, we introduce a stereoscopic vision memorizing loss that guides the M2S memory to store the stereoscopic visual information. Furthermore, we propose a binocular vision association loss to guide the M2S memory that can associate the information of the left-right view about the object when estimating the depth. As a result, our monocular 3D object detector with the M2S memory can effectively exploit the recalled stereoscopic visual information in the inference phase. The comprehensive experimental results on two public datasets, KITTI 3D Object Detection Benchmark and Waymo Open Dataset, demonstrate the effectiveness of the proposed method. We claim that our method is a step-forward method that follows the behaviors of humans that can recall the stereoscopic visual information even when one eye is closed. [ABSTRACT FROM AUTHOR]

An accurate measurement method of crack fabric is proposed. Considering the density, geometry and spatial distribution in cracks, a novel method suggests the formula of high-order fabric tensor in the orthogonal space to describe cracks quantitatively. The crack is automatically identified by morphology and determined by stereo scans of image processing techniques. The fabric tensor is defined based on the idea of probability and statistics. The zero-order tensor describes the average effect of the density of rectangular cracks, the second-order tensor describes the two dominant directions of spatial cracks, and the higher-order tensor more accurately describes the anisotropy of cracks. With the increase of the tensor order, the more accurate direction and degree of the anisotropy can be described, and the morphological automatic identification of crack and its physical meaning of quantitative determination are clear. The plane description of the crack can be described by the invariants of the plane tensor. One invariant describes the degree of the anisotropy, and the other invariant describes its direction. The scan line is rotated and scanned on the rock plane image to obtain the accurate distribution of plane cracks, and the distribution law can be accurately described by the proposed high-order fabric tensor. In this paper, the rationality and effectiveness of the proposed measurement method are well verified by CT image analyses. Highlights: An accurate measurement method of crack fabric is proposed. The expression of crack fabric tensor is redefined by the normalized idea. The plane crack fabric is automatically identified by morphology, determined by stereo scans, and described by the its invariants equivalently. On the basis of stereology, the rationality and effectiveness of the proposed measurement method are well verified by SEM and CT image analyses. [ABSTRACT FROM AUTHOR]

Objective: Determination of species from animal hair is an effective method in veterinary forensic investigations, research, endangered species and prevention of poaching. Since the hairs are resistant to deterioration, they can be stored as evidence for many years. In addition, pig hairs are often used in making brushes. When these brushes are used in the food industry, it raises questions about halal food. This study aimed to identify these hairs by examining the hair structure of wild pigs living in the Balıkesir region and revealing their characteristics. Materials and Methods: The hairs of 3 wild boars obtained from the İvrindi (Balıkesir) region were used. After the hairs taken from different parts of the pigs were cleaned, stereomicroscopy and macroscopic examination were performed and routine procedures were applied for scanning electron microscopic imaging. Result: In stereomicroscopy and macroscopic examination, it was determined that the length and thickness of the hairs in different regions varied significantly. In the study, the hairs were generally bifurcated from the upper 1/3 part. In the scanning electron microscopic images, the hardened cuticle patterns on the hair shaft, which have a scaly appearance, were detected, and their measurements were made. Scanning electron microscopic images determined that there were very small bifurcations from the hair shaft. However, it was thought that these hairs could not be used for species separation, since these parts would break off in the hairs used as brushes. Significant images could not be obtained in cross-sections. Conclusion: It is thought that it will be used as a source for the identification of the hairs of wild boars in the Balıkesir region. [ABSTRACT FROM AUTHOR]

Cost aggregation is crucial to the accuracy of stereo matching. A reasonable cost aggregation algorithm should aggregate costs within homogeneous regions where pixels have the same or similar disparities. Otherwise, the estimated disparity map is prone to the well‐known edge‐fattening issue and the problem of losing fine structures. However, current state‐of‐the‐art convolutional neural networks (CNNs) mainly do cost aggregation with square‐kernel convolutional layers that learn to adjust their kernel elements to make the actual receptive fields of the aggregated costs adapt to homogeneous regions with various shapes. This is a mechanism that easily leads to the above issues due to the translation equivalence and content‐insensitivity properties of CNNs. To tackle these problems, a novel densely connected asymmetric convolution block (Dense‐ACB) based on asymmetric convolutions is proposed to explicitly construct receptive fields with various shapes, which effectively alleviates the issues caused by mismatching shapes of receptive fields and homogeneous regions. The proposed Dense‐ACB brings new insight to CNN‐based stereo matching networks. Based on the proposed cost aggregation method, an efficient and effective stereo matching network is built, which not only achieves competitive overall accuracy compared with state‐of‐the‐art models but also effectively alleviates the edge‐fattening problem and preserves fine structures. [ABSTRACT FROM AUTHOR]

In recent years, convolutional‐neural‐network based stereo matching methods have achieved significant gains compared to conventional methods in terms of both speed and accuracy. Current state‐of‐the‐art disparity estimation algorithms require many parameters and large amounts of computational resources and are not suited for applications on edge devices. In this paper, an end‐to‐end light‐weight network (LWNet) for fast stereo matching is proposed, which consists of an efficient backbone with multi‐scale feature fusion for feature extraction, a 3D U‐Net aggregation architecture for disparity computation, and color guidance in a 2D convolutional neural network (CNN) for disparity refinement. MobileNetV2 is adopted as an efficient backbone in feature extraction. The channel attention module is applied to improve the representational capacity of features and multi‐resolution information is adaptively incorporated into the cost volume via cross‐scale connections. Further, a left‐right consistency check and color guidance refinement are introduced and a robust disparity refinement network is designed with skip connections and dilated convolutions to capture global context information and improve disparity estimation accuracy with little computational cost and memory space. Extensive experiments on Scene Flow, KITTI 2015, and KITTI 2012 demonstrate that the proposed LWNet achieves competitive accuracy and speed when compared with state‐of‐the‐art stereo matching methods. [ABSTRACT FROM AUTHOR]

In order to cultivate superior varieties of abalone, it is necessary to accurately measure the useful trait data of them. This paper presents a novel underwater binocular system for measuring the size of abalone. The measurement process can be divided into two main stages. In the first stage, underwater images are corrected using our proposed algorithm to eliminate the impact of refraction. In the second stage, the corrected images are utilized for the 3D reconstruction of the abalone shell. The length and width can be calculated by analyzing the contour of the shell. To validate the effectiveness of our method, a set of abalone shell samples are measured with the binocular system. The average measurement errors for the length and the width are 2.02% and 2.54%, respectively. • Developed a novel non-contact binocular system for underwater abalone measurement. • Proposed an correction algorithm to eliminate the impact of refraction on images. • Average measurement errors of the length and width of abalone samples are satisfied. [ABSTRACT FROM AUTHOR]

Abstract Stereo matching algorithms have been developed for many years but basically focus only on the implementation of existing datasets and are rarely applied to real scenarios, such as industrial robot scenarios. Traditional stereo matching algorithms have a high error rate, and deep learning algorithms are difficult to obtain good results in real scenarios because of their weak generalisation ability and difficult access to training data. In order to use stereo matching algorithms for industrial robot guidance, it is better to design a new traditional algorithm with low time complexity for the characteristics of industrial robot scenarios dominated by planar facets. This paper proposes a new matching method based on subrows of pixels, instead of individual pixels, in order to improve robustness of matching and reduce running time. First, the pixel strings from the same row of the left and right images are divided into several colour‐identical or colour‐gradient segments. Then, the colour and length of the two left and right pixel segment are used as clues to determine a matching relation and obtain the matching type. Then, all match types can be determined according to non‐crossing mapping. Each match type can reason backward to the corresponding spatial state of the stimulus source so that the disparity of pixels in pixel segments representing the spatial state can be calculated. This new matching method makes full use of the stimulus homology constraints and projective geometric constraints of row‐aligned images. The method can obtain good results in industrial robot scenarios and be applied for industrial robot guidance.

China's first optical stereo mapping satellite with a sub-meter resolution, GaoFen-7 (GF-7), launched in November 2019, shows significant potential for providing high-resolution topographic and geomorphic data for quantitative research on active tectonics. However, no studies have evaluated the capability of the GF-7-generated digital elevation model (DEM) for quantitatively studying active tectonics. This study aimed to validate the accuracy of the DEMs extracted from GF-7 stereo imagery, with or without ground control points (GCPs), and evaluated the potential of applying GF-7 DEMs to active tectonics. First, GF-7 stereo images were processed to obtain DEMs with a spatial resolution of 2 m, utilizing three different methods, including block adjustment without GCPs, block adjustment with the aid of Google Earth images and SRTM DEM, and block adjustment with GCPs derived from the Ice, Cloud, and land Elevation Satellite-2 (ICESat-2) data. These three generated DEMs were called GF-7 DEMMethod1, GF-7 DEMMethod2, and GF-7 DEMMethod3, respectively, and were verified by the airborne LiDAR data in the Hasishan section of the Haiyuan fault. Second, the capability of the GF-7 DEMs for identifying active faults, fault scarps, and horizontal offsets was evaluated. Finally, 8 vertical and 13 horizontal offsets were measured based on three different GF-7 DEMs, and airborne LiDAR data were used to verify the measurements' accuracies. The results indicated that the accuracy of GF-7 DEMMethod1 was the worst and that of GF-7 DEMMethod3 was superior to that of GF-7 DEMMethod2. The GF-7 DEMs could effectively identify the apparent fault scarps and horizontal offsets. The RMSE values of the vertical offsets measured based on GF-7 DEMMethod1, GF-7 DEMMethod2, and GF-7 DEMMethod3 were 0.55 m, 0.55 m, and 0.41 m, respectively. The horizontal offsets yielded RMSE values of 3.98 m, 2.52 m, and 1.37 m, respectively. These findings demonstrated that vertical and horizontal offsets could be accurately measured using the DEMs generated from GF-7 stereo images. Meanwhile, our study indicated that the GCPs derived from ICESat-2 data could be utilized to improve the accuracies of the GF-7 DEM, and the measurements of vertical and horizontal offsets. [ABSTRACT FROM AUTHOR]

This paper presents a method on how to compute the position of a robotic arm in the 2D and 3D spaces. This method is slightly different from the well-known methods, such as forward or inverse kinematics. The method presented in this paper is an optical method, which uses two video cameras in stereo vision configuration to locate and compute the next move of a robotic arm in space. The method recognizes the coordinates of the markers placed at the joints of the robotic arm using the two video cameras. The coordinate points of these markers are connected with straight lines. Around certain points, circles are drawn. From the tangent to the circles, a non-Cartesian (orthogonal) coordinate system is drawn, which is enough to compute the target position of the robotic arm. All of these drawings are overlaid on the live video feed. This paper also presents another method for calculating the stereo distance using the triangulation method. An alternative method is also presented when a non-Cartesian (orthogonal) 3D coordinate system is created, which is used to compute the target position of the robotic arm in the 3D space. Because the system is in a loop, it can make micro-adjustments of the robotic arm, in order to be exactly in the desired position. In this way, there is no need to make calibrations for the robotic arm. In an industrial system, there is no need to stop the production line, which can be a really big cost saver. [ABSTRACT FROM AUTHOR]

Abstract Stereo image pairs can effectively enhance the performance of super‐resolution (SR) since both intra‐view and cross‐view information can be used. However, exploiting cross‐view information accurately is extremely challenging. Most recent methods use the attention mechanism to get stereo correspondence. But these methods ignore the high‐frequency information since they only utilise first‐order statistics, which leads to reducing the discriminative ability of the network. To address this issue, in this work, a parallax‐based second‐order mixed attention stereo SR network (PSMASSRnet) is proposed to integrate the cross‐view information from a stereo image pair for SR. Specifically, a novel parallax‐based second‐order mixed attention module (PSMAM) is developed to combine second‐order channel features and spatial features to obtain more discriminative representations. Furthermore, a dense cross‐atrous spatial pyramid pooling (ASPP) module is also presented, which can effectively explore the local and the multi‐scale features with different dilation rates to extract more discriminative features with fewer parameters and less execution. The extensive experiments on the KITTI2012, KITTI2015 and Middlebury datasets have demonstrated the superiority of the proposed PSMASSRnet over the state‐of‐the‐art methods in the aspects of both the quantitative metrics and the visual quality.

Abstract Despite recent advances in 3‐D pose estimation of human hands, thanks to the advent of convolutional neural networks (CNNs) and depth cameras, this task is still far from being solved in uncontrolled setups. This is mainly due to the highly non‐linear dynamics of fingers and self‐occlusions, which make hand model training a challenging task. In this study, a novel hierarchical tree‐like structured CNN is exploited, in which branches are trained to become specialised in predefined subsets of hand joints called local poses. Further, local pose features, extracted from hierarchical CNN branches, are fused to learn higher order dependencies among joints in the final pose by end‐to‐end training. Lastly, the loss function used is also defined to incorporate appearance and physical constraints about doable hand motions and deformations. Finally, a non‐rigid data augmentation approach is introduced to increase the amount of training depth data. Experimental results suggest that feeding a tree‐shaped CNN, specialised in local poses, into a fusion network for modelling joints' correlations and dependencies, helps to increase the precision of final estimations, showing competitive results on NYU, MSRA, Hands17 and SyntheticHand datasets.

This paper introduces PlaneLoc2 - a novel indoor global localization system designed to harness the potential of stereo cameras. A need for robust global localization that does not produce incorrect results (false positives) is present in almost every life-long autonomy task. We show that planar segments extracted from stereo vision data by a neural network enable such robust localization. Planar segments are easier to discriminate than keypoint features and provide easy-to-use geometric constraints. We propose an architecture that exploits a single deep neural network (DNN) to detect planar segments, produce appearance descriptors, and estimate segment geometry. Moreover, we introduce a novel view-based segment map and a novel pose retrieval procedure that considers the uncertainty of features to efficiently use the geometric constraints provided by them. We also show that the new learned descriptor provides better discrimination than the hand-crafted one. Finally, we present experimental results that show that our solution outperforms other state-of-the-art global localization methods and does not produce incorrect agent poses. For both test scenes it recognizes at least 15% more poses than the second best method without incorrect recognitions.

Non-contact vibration measurements are relevant for non-invasively characterizing the mechanical behavior of structures. This paper presents a novel methodology for full-field vibrational analysis at high frequencies using the three-dimensional digital image correlation technique combined with the projection of a speckle pattern. The method includes stereo calibration and image processing routines for accurate three-dimensional data acquisition. Quantitative analysis allows the extraction of several deformation parameters, such as the cross-correlation coefficients, shape and intensity, as well as the out-of-plane displacement fields and mode shapes. The potential of the methodology is demonstrated on an Unmanned Aerial Vehicle wing made of composite material, followed by experimental validation with reference accelerometers. The results obtained with the projected three-dimensional digital image correlation show a percentage of error below 5% compared with the measures of accelerometers, achieving, therefore, high sensitivity to detect the dynamic modes in structures made of composite material. [ABSTRACT FROM AUTHOR]

[Display omitted] Discrete orthogonal moments (DOMs) are widely applied in the field of image processing. However, most existing DOMs are used for flat image analysis and are less suitable for color stereo image processing. In light of this limitation, this paper firstly presents a moments type called octonion Hahn moments (OHMs) for the compact analysis of color stereo images, and also for preserving the internal relations between all the color channels of the stereo image. OHMs are introduced in both Cartesian and polar coordinates. Secondly, we present a modified version of 2D Henon map that exhibits improved chaotic behavior over its original version. This version is introduced for potential deployment in the field of information security. Thirdly, we present an efficient color stereo image encryption scheme in the transform domain using the modified Henon's map and OHMs that are defined in Cartesian space. This scheme guarantees a perfect reconstruction of the stereo image. Finally, we present a local zero-watermarking scheme for color stereo image copyrighting based on the proposed version of Henon's map and OHMs that are defined in the polar coordinate space. This scheme is originally designed to be robust against image rotation attack. Compared to excellent existing encryption schemes, our scheme shows superior performance in terms of high security level and good robustness to various attacks (classical, statistical, noise addition, cropping, etc.). Moreover, the proposed zero-watermarking method shows stronger resistance to different types of attacks (geometric, common image processing, noise, desynchronization, etc.) in comparison to recent zero-watermarking methods. [ABSTRACT FROM AUTHOR]

Electronic holography is an ideal 3-D display technique to ameliorate the vergence-accommodation conflict, which is a possible cause of visual fatigue and discomfort from viewing conventional stereoscopic 3-D displays. Previous studies have measured accommodative and vergence responses to holographic images and to real objects, and the results showed that these responses are in good agreement. To demonstrate the effectiveness of electronic holography as a possible solution to the vergence-accommodation conflict caused by viewing conventional stereoscopic 3-D displays, we measured both accommodative and vergence responses to reconstructed images of holograms and super multiview holographic stereograms. We also measured responses to real objects located at various distances and conventional two-view stereo images for comparison. The results indicate that the accommodative responses to the electro-holographic display and super multiview holographic stereogram change with the vergence response in a similar manner, as is the case with real objects as compared to conventional two-view stereo display. This suggests that the electronic holographic display and super multiview holographic stereograms are promising candidates for resolving the vergence-accommodation conflict. [ABSTRACT FROM AUTHOR]

Virtual Reality (VR) is becoming ubiquitous with the rise of consumer displays and commercial VR platforms. Such displays require low latency and high quality rendering of synthetic imagery with reduced compute overheads. Recent advances in neural rendering showed promise of unlocking new possibilities in 3D computer graphics via image-based representations of virtual or physical environments. Specifically, the neural radiance fields (NeRF) demonstrated that photo-realistic quality and continuous view changes of 3D scenes can be achieved without loss of view-dependent effects. While NeRF can significantly benefit rendering for VR applications, it faces unique challenges posed by high field-of-view, high resolution, and stereoscopic/egocentric viewing, typically causing low quality and high latency of the rendered images. In VR, this not only harms the interaction experience but may also cause sickness. To tackle these problems toward six-degrees-of-freedom, egocentric, and stereo NeRF in VR, we present the first gaze-contingent 3D neural representation and view synthesis method. We incorporate the human psychophysics of visual- and stereo-acuity into an egocentric neural representation of 3D scenery. We then jointly optimize the latency/performance and visual quality while mutually bridging human perception and neural scene synthesis to achieve perceptually high-quality immersive interaction. We conducted both objective analysis and subjective studies to evaluate the effectiveness of our approach. We find that our method significantly reduces latency (up to 99% time reduction compared with NeRF) without loss of high-fidelity rendering (perceptually identical to full-resolution ground truth). The presented approach may serve as the first step toward future VR/AR systems that capture, teleport, and visualize remote environments in real-time. [ABSTRACT FROM AUTHOR]

Modeling 3D visual saliency has received great attention due to the development of emerging 3D display technologies. Traditional methods relying on low-level features may not be efficient in interpreting 3D visual content from high-level semantic perspective. Despite numerous efforts dedicated to this area, existing 3D visual saliency detection methods do not necessarily excel in exploring the stereoscopic image saliency driven by the intra-view and inter-view dependencies among left and right views. In this paper, we propose a visual saliency detection method for stereoscopic images grounded on adaptive viewpoint feature enhancement via binocular vision. More specifically, the correlation among left and right views is investigated through a delicately designed binocular stereoscopic saliency feature aggregation module, enabling the generation of more representative saliency features towards binocular vision. Subsequently, to further aggregate the saliency features in multiple scales, we design a progressive attention-based saliency feature pyramid extraction module to effectively integrate the features from top-level to down-level based on the network hierarchy mechanism. The saliency maps are ultimately produced for stereoscopic images by evaluating the obtained saliency features. In addition, we create a stereoscopic image saliency dataset (SIS-3D) that includes 1086 stereoscopic image pairs with various content and their corresponding human eye fixation annotations, aiming to further facilitate the research on visual saliency detection for stereoscopic images. Extensive experiments demonstrate that our proposed method improves CC by an average of 4.02% compared to representative counterparts on the newly built saliency dataset and another publicly available dataset. [ABSTRACT FROM AUTHOR]

This paper is devoted to the important problem of subsea infrastructure inspection using autonomous underwater vehicles (AUVs). The accomplishment of the inspection mission requires high accuracy of AUV coordination with respect to objects, which is not always provided by standard hydroacoustic equipment. That is why we propose an approach based on stereo image processing that can provide submeter accuracy of AUV coordination required under subsea conditions. In the framework of a unified approach based on the use of an a priori formed point model of an object and the application of a structural coherence criterion, we propose and analyze several methods for recognition of underwater objects and coordinate referencing of the AUV to these objects. As a geometric model of an object, we consider its representation based on characteristic points, which determines the spatial structure of the object. In addition to the standard method for object model construction from preliminary measurements, a new method is proposed, which is based on automated technique of model generation from images offline. The proposed object recognition methods use the structural coherence criterion to match 3D point clouds with object models. The 3D point clouds are formed from stereo images captured by a camera using visual navigation. Feature points are generated and matched using the SURF detector and Harris corner detector. To improve the reliability of object identification, we propose joint processing of source images with their vectorized forms. The use of line segments obtained by vectorization significantly increases the number of identified characteristic points when matching a 3D point cloud with a model. Based on the identified points of an object, the matrix of AUV coordinate referencing to this object is computed. The efficiency of the considered methods is estimated and compared. In the computational experiments with model scenes, a virtual geometric model of a distributed subsea production system is used. The experiments on real-world data are carried out under laboratory conditions with the use of a Karmin2 stereo camera (Nerian's 3D stereo camera, baseline 25 cm). [ABSTRACT FROM AUTHOR]

Existing forgery image localization methods have achieved impressive performance on monocular images but struggle to maintain comparable performance on stereo super-resolution (SR) images. The stereo image SR process usually introduces additional noise, i.e., reconstruction artifacts, which interferes with the extraction of forgery clues and thus undermines the localization performance. To mitigate this issue, we present a multi-scale attention framework named SSR-IFL for stereo SR image forgery localization, which explores multi-scale forgery clues in stereo SR images to minimize disturbances caused by reconstruction artifacts. Specifically, the novel framework integrates two significant components: the multi-scale feature extraction (MSFE) network to thoroughly mine forgery clues against the inference of reconstruction artifacts, and the bidirectional progressive feature fusion (BPFF) network to effectively reason the learned forgery clues and obtain robust forgery feature representation. Under the MSFE network, the multi-dilated self-attention (MDSA) block makes full use of multi-resolution pixel correlation to adapt to changes in pixel correlation distributions induced by the stereo SR reconstruction process, thus extracting more precise multi-scale forgery clues. The BPFF network introduces a bidirectional feature fusion strategy to fuse multi-scale forgery clues, effectively eliminating redundant noise such as reconstruction artifacts and enhancing the robustness of forgery feature representation. Experimental results on public image datasets demonstrate that our method can achieve not only competitive performance on stereo SR images but also on monocular images. • Multi-scale attention framework for robust forgery localization in stereo SR images. • The multi-scale feature extraction network is introduced to against reconstruction artifacts. • The multi-dilated self-attention block adapts to pixel correlation changes for precise multi-scale clues. • The bidirectional progressive feature fusion network is proposed to enhance model robustness. • Competitive performance on both stereo SR and monocular images in experiments. [ABSTRACT FROM AUTHOR]

Abstract Image matching is a well‐studied problem in computer vision. Conventional image matching is solved using image feature matching algorithms, and later deep learning techniques are also applied to tackle the problem. Here, a slope‐modelling framework is proposed by adopting the image matching techniques. First, image pairs of a slope scene are captured and camera calibration as well as image rectification are performed. Then, PatchMatch Filter (PMF‐S) and PWC‐Net techniques are adapted to solve the matching of image pairs. In the proposed PatchMatch Filter‐Census (PMF‐Census), slanted‐plane modelling, image census transform and gradient difference are employed in matching cost formulation. Later, nine matching points are manually selected from an image pair. Matching point pairs are further used in fitting a transformation matrix to relate the matching between the image pair. Then, the transformation matrix is applied to obtain a ground truth matching image for algorithm evaluation. The challenges in this matching problem are that the slope is of a homogenous region and it has a slanted‐surface geometric structure. In this work, it is found out that the error rate of the proposed PMF‐Census is significantly lower as compared with the PWC‐Net method and is more suitable in this slope‐modelling task. In addition, to show the robustness of the proposed PMF‐Census against the original PMF‐S, further experiments on some image pairs from Middlebury Stereo 2006 dataset are conducted. It is demonstrated that the error percentage by the proposed PMF‐Census is reduced significantly especially in the low‐texture and photometric distorted region, in comparison to the original PMF‐S algorithm. This further verifies the suitability of the PMF‐Census in modelling the outdoor low‐texture slope scene.

Existing stereo matching methods achieve satisfactory average accuracy on a whole predicted disparity map under common global metrics, but ignore the fine-grained performance at region level, especially for far regions in the scene, which is more crucial in actual auto-driving scenarios. There are two factors accounting for this problem: 1) Depth resolution. Existing methods use disparity-based sampling to extract matching candidates uniformly according to the disparity range, but leads to sparser sampling density at far regions than that of close regions in terms of depth range, resulting in low depth resolution in far regions. 2) Feature discriminability. Limited image resolution and inferior feature extraction at far regions result in the obtained features with low discrimination, which influences the subsequent matching process between stereo images. To improve the estimation accuracy of far regions and thus achieve a balanced performance at region level, we design a novel two-stage Balanced Stereo Matching Network (BSMNet) to address the above problems. The coarse stage of BSMNet introduces a direct depth-based sampling strategy, which generates matching candidates according to scene depth instead of disparity, thus improving the depth resolution and obtaining initial depth map with more balanced accuracy. Then, a depth refinement stage is proposed to solve the problem of low feature discriminability and further optimizes the initial depth map obtained from the coarse stage. It selects matching candidates and computes their similarity scores from a carefully designed adaptive feature volume guided by a learnable scale map, thus making the final estimation more accurate. Different from the existing methods that construct stereo matching based on disparity prediction, our proposed pipeline is to directly optimize on depth information. Experiments show that our BSMNet can obtain an obvious performance improvement at far regions without discarding that at close regions, so as to largely outperform existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In contemporary society full of stereoscopic images, how to assess visual quality of 3D images has attracted an increasing attention in field of Stereoscopic Image Quality Assessment (SIQA). Compared with 2D-IQA, SIQA is more challenging because some complicated features of Human Visual System (HVS), such as binocular interaction and binocular fusion, must be considered. In this paper, considering both binocular interaction and fusion mechanisms of the HVS, a hierarchical no-reference stereoscopic image quality assessment network (StereoIF-Net) is proposed to simulate the whole quality perception of 3D visual signals in human cortex, including two key modules: BIM and BFM. In particular, Binocular Interaction Modules (BIMs) are constructed to simulate binocular interaction in V2-V5 visual cortex regions, in which a novel cross convolution is designed to explore the interaction details in each region. In the BIMs, different output channel numbers are designed to imitate various receptive fields in V2-V5. Furthermore, a Binocular Fusion Module (BFM) with automatic learned weights is proposed to model binocular fusion of the HVS in higher cortex layers. The verification experiments are conducted on the LIVE 3D, IVC and Waterloo-IVC SIQA databases and three indices including PLCC, SROCC and RMSE are employed to evaluate the assessment consistency between StereoIF-Net and the HVS. The proposed StereoIF-Net achieves almost the best results compared with advanced SIQA methods. Specifically, the metric values on LIVE 3D, IVC and WIVC-I are the best, and are the second-best on the WIVC-II. [ABSTRACT FROM AUTHOR]

Stereoscopic visual fatigue (SVF) due to prolonged immersion in the virtual environment can lead to negative user experience, thus hindering the development of virtual reality (VR) industry. Previous studies have focused on investigating the evaluation indicators associated with SVF, while few studies have been conducted to reveal the underlying neural mechanism, especially in VR applications. In this paper, a modified Go/NoGo paradigm was adopted to induce SVF in VR environment with Go trials for maintaining participants’ attention and NoGo trials for investigating the neural effects under SVF. Random dot stereograms (RDSs) with 11 disparities were presented to evoke the depth-related visual evoked potentials (DVEPs) during 64-channel EEG recordings. EEG datasets collected from 15 participants in NoGo trials were selected to conduct individual processing and group analysis, in which the characteristics of the DVEPs components for various fatigue degrees were compared and independent components were clustered to explore the original cortex areas related to SVF. Point-by-point permutation statistics revealed that DVEPs sample points from 230 ms to 280 ms (component P2) in most brain areas changed significantly when SVF increased. Additionally, independent component analysis (ICA) identified that component P2 which originated from posterior cingulate cortex and precuneus, was associated statistically with SVF. We believe that SVF is rather a conscious status concerning the changes of self-awareness or self-location awareness than the performance reduction of retinal image processing. Moreover, we suggest that indicators representing higher conscious state may be a better indicator for SVF evaluation in VR environments. [ABSTRACT FROM AUTHOR]

Abstract Depth is essential information for autonomous robotics applications that need environmental depth values. The depth could be acquired by finding the matching pixels between stereo image pairs. Depth information is an inference from a matching cost volume that is composed of the distances between the possible pixel points on the pre-aligned horizontal axis of stereo images. Most approaches use matching costs to identify matches between stereo images and obtain depth information. Recently, researchers have been using convolutional neural network-based solutions to handle this matching problem. In this paper, a novel method has been proposed for the refinement of matching costs by using recurrent neural networks. Our motivation is to enhance the depth values obtained from matching costs. For this purpose, to attain an enhanced disparity map by utilizing the sequential information of matching costs in the horizontal space, recurrent neural networks are used. Exploiting this sequential information, we aimed to determine the position of the correct matching point by using recurrent neural networks, as in the case of speech processing problems. We used existing stereo algorithms to obtain the initial matching costs and then improved the results by utilizing recurrent neural networks. The results are evaluated on the KITTI 2012 and KITTI 2015 datasets. The results show that the matching cost three-pixel error is decreased by an average of 14.5% in both datasets.

This paper presents a novel approach to provide immersive free navigation with 6 Degrees of Freedom in real-time for natural and virtual scenery, for both static and dynamic content. Stemming from the state-of-the-art in Depth Image-Based Rendering and the OpenGL pipeline, this new View Synthesis method achieves free navigation at up to 90 FPS and can take any number of input views with their corresponding depth maps as priors. Video content can be played thanks to GPU decompression, supporting free navigation with full parallax in real-time. To render a novel viewpoint, each selected input view is warped using the camera pose and associated depth map, using an implicit 3D representation. The warped views are then blended all together to generate the chosen virtual view. Various view blending approaches specifically designed to avoid visual artifacts are compared. Using as few as four input views appears to be an optimal trade-off between computation time and quality, allowing to synthesize high-quality stereoscopic views in real-time, offering a genuine immersive virtual reality experience. Additionally, the proposed approach provides high-quality rendering of a 3D scenery on holographic light field displays. Our results are comparable - objectively and subjectively - to the state of the art view synthesis tools NeRF and LLFF, while maintaining an overall lower complexity and real-time rendering.