Search

The recent success in supervised multi-view stereopsis (MVS) relies on the onerously collected real-world 3D data. While the latest differentiable rendering techniques enable unsupervised MVS, they are restricted to discretized (e.g., point cloud) or implicit geometric representation, suffering from either low integrity for a textureless region or less geometric details for complex scenes. In this paper, we propose SurRF, an unsupervised MVS pipeline by learning Surface Radiance Field, i.e., a radiance field defined on a continuous and explicit 2D surface. Our key insight is that, in a local region, the explicit surface can be gradually deformed from a continuous initialization along view-dependent camera rays by differentiable rendering. That enables us to define the radiance field only on a 2D deformable surface rather than in a dense volume of 3D space, leading to compact representation while maintaining complete shape and realistic texture for large-scale complex scenes. We experimentally demonstrate that the proposed SurRF produces competitive results over the-state-of-the-art on various real-world challenging scenes, without any 3D supervision. Moreover, SurRF shows great potential in owning the joint advantages of mesh (scene manipulation), continuous surface (high geometric resolution), and radiance field (realistic rendering).

Automatic inspections of overhead contact lines (OCLs) are developed to implement anomaly detection during normal operation. It is an essential prerequisite for efficient maintenance of railway electrification system. This paper presents a comprehensive survey on the inspections of OCLs with an emphasis on computer vision technology, which has developed rapidly due to its ability to understand images. Our survey begins with a brief introduction on anomalies in OCL inspections and generic procedures of computer vision inspection for anomaly detection. Subsequently, for detecting deviations of parameters and defective components during OCL inspections, the existing techniques involving stereo vision and object vision, especially convolution neural networks are described in detail from two aspects: measurement of OCL parameters and identification of OCL conditions. Some interference factors in OCL inspection are analyzed. Actual cases of the inspections are also briefly shown. Challenges and suggestions for further research on OCL inspection are drawn toward the end of the paper.

In-hand object localization and manipulation has always been a challenging task in robotic community. In this paper, we address this problem by vision-based tactile sensing with high spatial resolution. Specifically, we design a novel tactile sensor based on stereo vision, named GelStereo, which can perceive tactile point cloud with high spatial resolution (< 1 mm). A tactile-based in-hand object localization pipeline composed of saliency detection and probabilistic point-set registration algorithms of the perceived contact point cloud is presented. Furthermore, extensive qualitative and quantitative analyses of perceived tactile point cloud and in-hand localization and insertion experiments of small parts are performed on our robot platform. The experimental results verify the accuracy and robustness of the tactile point cloud sensed by the novel GelStereo tactile sensor and the proposed in-hand object localization pipeline. This novel high-resolution visuotactile sensing technology has predictable application potential in the field of dexterous robotic manipulation.

The performance of stereo vision tasks degrades when haze exists in the input stereo image pair. Independently applying single image dehazing algorithm on left and right images is not optimal. To overcome the problem, we propose an effective framework, called SRDNet, for simultaneously dehazing stereo images. The main idea of SRDNet is to make full use of the stereo information from cross views improving dehazing performance. It does not explicitly employ the disparity estimation and the correlation matrix. SRDNet comprises two parts: a weight-sharing coarse dehazing network (WSCDN) and a guided separated refinement network (GSRN). The WSCDN is utilized to predict a coarse dehazed image pair. Then the GSRN is introduced to predict the residues for different views by extracting the fused information of cross views and separating the features of different views with a guided channel and spatial refinement module. The residues are added to the coarse dehazed pair so as to make refinement and remove the remained haze. Experimental results demonstrate that our proposed SRDNet surpasses previous image dehazing methods by a significant margin both quantitatively and qualitatively. Moreover, our SRDNet could be a preprocessing step of the stereo-based 3D object detection and boosts the 3D detection accuracy in hazy scenes.

Efficient real-time disparity estimation is critical for the application of stereo vision systems in various areas. Recently, stereo network based on coarse-to-fine method has largely relieved the memory constraints and speed limitations of large-scale network models. Nevertheless, all of the previous coarse-to-fine designs employ constant offsets and three or more stages to progressively refine the coarse disparity map, still resulting in unsatisfactory computation accuracy and inference time when deployed on mobile devices. This paper claims that the coarse matching errors can be corrected efficiently with fewer stages as long as more accurate disparity candidates can be provided. Therefore, we propose a dynamic offset prediction module to meet different correction requirements of diverse objects and design an efficient two-stage framework. In addition, a disparity-independent convolution is proposed to regularize the compact cost volume efficiently and further improve the overall performance. The disparity quality and efficiency of various stereo networks are evaluated on multiple datasets and platforms. Evaluation results demonstrate that, the disparity error rate of the proposed network achieves 2.66% and 2.71% on KITTI 2012 and 2015 test sets respectively, where the computation speed is 2× faster than the state-of-the-art lightweight models on high-end and source-constrained GPUs.

Free-space detection is considered as one of the most important tasks in intelligent vehicle, which remains the essential requirement for obstacle detection and path planning. As a part of Internet of Things, vehicle-based device produces a large amount of data requiring to be processed in realtime. Therefore, the present study proposes a fast free-space detection system with a mobile platform based stereo vision. The current system processes the image-based perception signal in terminal equipment, which avoids lots of data transmissions between cloud and vehicle. In terms of flexibility and robustness, a fast spare disparity matching algorithm is suggested to be real-timely processed in a low-power mobile platform. Based on the spare disparity map, the current work improves the u-disparity space algorithm as the u-log-disparity space with the aim to detect the free-space for significantly lowering the running time. Besides, we also provide a complete system design including hardware and software. The experiments demonstrate that our system is precise and robust at complex real-time driving environment.

International audience; Several works have focused on Simultaneous Localization and Mapping (SLAM), which is a topic that has been studied for more than a decade to meet the needs of robots to navigate in an unknown environment. SLAM is an essential perception functionality in several applications, especially in robotics and autonomous vehicles. RGB-D cameras are among the sensors commonly used with recent SLAM algorithms. They provide an RGB image and the associated depth map, making it possible to solve scale drift with less complexity and create a dense 3D environment representation. Many RGB-D SLAM algorithms have been studied and evaluated on publicly available datasets without considering sensor specifications or image acquisition modes that could improve or decrease localization accuracy. In this work, we deal with indoor localization, taking into account the sensor specifications. In this context, our contribution is a deep experimental study to highlight the impact of the sensor acquisition modes on the localization accuracy, and a parametric optimization protocol for a precise localization in a given environment. Furthermore, we apply the proposed protocol to optimize a depth-related parameter of the SLAM algorithm. The study is based on a publicly available dataset in an indoor environment with a depth sensor. The reconstruction results’ analysis is founded on the study of different metrics involving translational and rotational errors. These metrics errors are compared with those obtained with a state-of-the-art stereo vision-based SLAM algorithm.

Background: Stereo-vision-based three-dimensional coordinates measurement technology has been widely applied in the military or civil fields. There are two problems that need to be solved. The first problem is that each camera’s internal parameters and the two cameras' external parameters need to be calibrated. To increase the measurement range, usually, the turntable is used with the stereo vision system together. The second problem is the calibration of the turntable. Objective: The aim of the study is to construct and calibrate a stereo-vision-based coordinates measurement system via a two-axis turntable. Methods: Considering that the internal parameters of each camera do not change during the measurement process and the complicated optimization process of one-step self-calibration, a two-step stereo vision calibration method is proposed. In the first step, we calibrate the internal parameters of each camera through a specially designed planar target with circular points. In the second step, on the basis of the calibrated results of the internal parameters, the two cameras' external parameters are calibrated through a simple target which could be distributed in the measurement volume. For the calibration of the two-axis turntable, we calibrated the rotation axes of the turntable and the coordinates of points in the 3D space could be measured considering the non-orthogonality of the axes. Results: Some experiments are provided to examine the calibration methods we proposed. They are the plane target measurement experiments; the standard ball center coordinates measurement experiments and target pose measurement experiments. Experiment results demonstrate the superiority of the calibration method we proposed. Conclusion: We studied the calibration methods of the stereo-vision-based coordinates measurement system via a two-axis turntable. The experimental results show the measurement accuracy of our system is less than 0.1mm.

It is still a grand challenge to implement a high-accuracy and high-performance stereo matching algorithm on a resource-limited hardware platform in stereo vision systems. This paper proposes a resource-efficient pipelined hardware architecture with four-cycle time-sharing for the semi-global matching (SGM) algorithm with weighted path cost aggregation. To save hardware resources, we also combined image down-sampling and disparity skipping in the SGM algorithm. The presented architecture is synthesized and implemented on a Zynq-7 FPGA board, which results in a throughput of 1280 × 960/62.5 fps with 75 disparity levels at the maximum frequency of 216 MHz. To improve the accuracy of the disparity map at close range, we also adapt the presented architecture with two-cycle time-sharing, and the disparity range is increased to 128, which attains the processing of 1280×960/116 fps at 200 MHz on VCU-118 FPGA board; the throughput reaches 18245 MDE/s. The result shows that the whole architecture only takes 50465 LUTs, 48046 Registers, 125.5 BRAMs with 128 disparity levels, which is much more efficient than the latest reference work.

Stereo image super-resolution exploits additional features from cross view image pairs for high resolution (HR) image reconstruction. Recently, several new methods have been proposed to investigate cross view features along epipolar lines to enhance the visual perception of recovered HR images. Despite the impressive performance of these methods, global contextual features from cross view images are left unexplored. In this paper, we propose a cross view capture network (CVCnet) for stereo image super-resolution by using both global contextual and local features extracted from both views. Specifically, we design a cross view block to capture diverse feature embeddings from the views in stereo vision. In addition, a cascaded spatial perception module is proposed to redistribute each location in feature maps according to the weight it occupies to make the extraction of features more effective. Extensive experiments demonstrate that our proposed CVCnet outperforms the state-of-the-art image super-resolution methods to achieve the best performance for stereo image super-resolution tasks. The source code is available at https://github.com/xyzhu1/CVCnet.

Accurate volume measurement of flying droplets in the Mura-free manufacturing of inkjet printed organic light-emitting diodes (IJP OLEDs) remains a challenge due to the micrometer scale and high speeds of the droplets. Most existing vision-based methods pay little attention to image segmentation and estimate the droplet volume based on a symmetry assumption, which yields poor accuracy, especially for droplets with an unknown shape. In this paper, an accurate measurement method for flying droplet volume based on stereo vision is proposed. First, an image acquisition technique based on stereo vision and synchronous triggering is used to obtain multiview high-resolution droplet images. Then, an accurate binocular droplet image segmentation algorithm is proposed to segment the images with different degrees of blur to achieve similar accuracy. Finally, a polar-coordinate-Hermite-interpolation-based droplet reconstruction (PHDR) algorithm is proposed to reconstruct droplets and to calculate volume on the basis of the contours obtained by the binocular droplet image segmentation algorithm without any symmetry assumptions. The droplet measurement system (DMS) is set up, and a series of experiments are conducted to verify the performance of the proposed method. The results show the advantages of our algorithms, that the volume measurement accuracy is ±3% and that the relative standard uncertainty is better than 2.1%.

Time-of-Flight (ToF) sensors and stereo vision systems are both widely used for capturing depth data. They have some complementary strengths and limitations, which have been exploited in prior research to produce more accurate depth maps by fusing data from the two sources. However, among these diverse data fusion approaches, none of them provides an end-to-end neural network solution. In this work, we propose the first end-to-end ToF and stereo data fusion network using the coarse-to-fine matching framework, where the prior of ToF depth is integrated into the stereo matching process by constraining the search range of stereo matching within an interval around the ToF camera depth measurement. We adopt a dynamic search range for each pixel according to an estimated ToF error map, which is more efficient and effective than a constant one when handling various errors. The ToF error map is estimated by the ToF error estimator branching out from the stereo matching network. Both ToF error estimation and stereo matching are performed in a joint framework, with the two tasks assisting each other mutually. We also propose an upsampling module to replace the naive bilinear upsampling in the coarse-to-fine stereo matching network, which reduces the error caused by the upsampling. The proposed deep network is trained end-to-end on synthetic datasets and generalizable to real-world datasets without further fine-tuning. Experimental results show that our fusion method achieves higher accuracy than either ToF or stereo alone, and outperforms state-of-the-art fusion methods on both synthetic and real data.

In the previous chapter we learnt about corner detectors which find particularly distinctive points in a scene. These points can be reliably detected in different views of the same scene irrespective of viewpoint or lighting conditions. Such points are characterised by high image gradients in orthogonal directions and typically occur on the corners of objects. However the 3-dimensional coordinate of the corresponding world point was lost in the perspective projection process which we discussed in Chap. 11 - we mapped a 3-dimensional world point to a 2-dimensional image coordinate. All we know is that the world point lies along some ray in space corresponding to the pixel coordinate, as shown in Fig. 11.1. To recover the missing third dimension we need additional information. In Sect. 11.2.3 the additional information was camera calibration parameters plus a geometric object model, and this allowed us to estimate the object’s 3-dimensional pose from the 2-dimensional image data.

Stereo matching is the key technology in stereo vision. Given a pair of rectified images, stereo matching determines correspondences between the pair images and estimate depth by obtaining disparity between corresponding pixels. The current work has shown that depth estimation from a stereo pair of images can be formulated as a supervised learning task with an end-to-end frame based on convolutional neural networks (CNNs). However, 3D CNN puts a great burden on memory storage and computation, which further leads to the significantly increased computation time. To alleviate this issue, atrous convolution was proposed to reduce the number of convolutional operations via a relatively sparse receptive field. However, this sparse receptive field makes it difficult to find reliable corresponding points in fuzzy areas, e.g., occluded areas and untextured areas, owing to the loss of rich contextual information. To address this problem, we propose the Group-based Atrous Convolution Spatial Pyramid Pooling (GASPP) to robustly segment objects at multiple scales with affordable computing resources. The main feature of the GASPP module is to set convolutional layers with continuous dilation rate in each group, so that it can reduce the impact of holes introduced by atrous convolution on network performance. Moreover, we introduce a tailored cascade cost volume in a pyramid form to reduce memory, so as to meet real-time performance. The group-based atrous convolution stereo matching network is evaluated on the street scene benchmark KITTI 2015 and Scene Flow and achieves state-of-the-art performance.

To evaluate binocular visual function after unilateral and bilateral implantation of segmented refractive multifocal intraocular lenses (MIOLs). This prospective comparative pilot study included patients who underwent SBL-3 (Lenstec; + 3.00 D) implantation at Peking University Third Hospital. Patients were divided into two groups (monocular or binocular surgery). Thirty-two patients with emmetropic presbyopic contralateral eyes and 49 patients with bilateral SBL-3 implantation within a week between eyes were included in the unilateral SBL-3 and bilateral groups, respectively. At 3-month follow-up, the main outcomes were binocular uncorrected distant, intermediate, and near visual acuity (UDVA, UIVA, and UNVA). Secondary outcomes included binocular best-corrected visual acuity at all distances, defocus curve, contrast sensitivity, photic phenomena, spectacle independence, patient satisfaction, and National Eye Institute Visual Function Questionnaire (NEI-VFQ-25) score. The essential perceptual phenomena constituting binocular vision, simultaneous perception, fusion, and stereopsis were also evaluated. Both groups showed similar binocular UDVA and UIVA, but UNVA was significantly better in the bilateral group (0.07 ± 0.07 versus 0.12 ± 0.07, P = 0.008). Better binocular defocus curve at intermediate to near focal points, higher percentage of spectacle independence at near distance, and NEI-VFQ-25 near activity scores were observed in the bilateral group. No significant differences in contrast sensitivity, photic phenomena, overall satisfaction, other NEI-VFQ-25 subscales, fusional amplitude, and stereoacuity were found between groups. Unilateral implantation of segmented refractive MIOL provided desirable distant visual acuity and high patient satisfaction, but inferior intermediate and near visual outcomes compared with bilateral implantation.

Purpose: To compare the stereoacuity between patients with anisometropic amblyopia who were treated and achieved normal visual acuity (VA) and normal children and evaluate the factors associated with stereoacuity.Methods: We retrospectively reviewed the records of 37 pediatric patients with anisometropic amblyopia who recovered to normal VA with glasses and occlusion treatment (amblyopia group) and 34 normal children (control group). The Worth 4-dot test, Lang II test, Titmus test, and TNO test were performed to measure stereoacuity. Clinical characteristics were compared between the two groups, and factors affecting stereoacuity outcomes were also analyzed in the amblyopic group.Results: The mean age at diagnosis of amblyopia was 5.3 ± 1.4 years, and the mean VAs at diagnosis were 0.41 ± 0.24 and 0.06 ± 0.07 in amblyopic and fellow eyes, respectively. The mean duration of occlusion was 19.00 ± 9.44 months, and VA of amblyopic eyes improved to 0.04 ± 0.04 after occlusion treatment. The patient characteristics did not differ significantly between the two groups, except for the final VA of the amblyopic eye. The final mean logarithm of minimal angle of resolution VA of the amblyopic eye in the amblyopia group was significantly worse than that in the control group. The number of patients with normal stereoacuity was significantly lower in the amblyopia group than in the control group on Lang II, Titmus, and TNO tests. Factors associated with poor stereoacuity were severe amblyopia in the Lang II test and poor post-treatment VA of the amblyopic eye in the Titmus test.Conclusions: Stereoacuity was worse in the amblyopia group than in the control group, despite normal visual development. The depth of amblyopia and post-treatment VA were associated with stereoacuity outcomes. Thus, VA improvement should be closely monitored in the amblyopic eye to obtain good stereoacuity.

PURPOSE Mild to moderate vision loss affects many children and can negatively impact a child's early literacy and academic achievement. Nevertheless, there is no consensus on which factors present in early childhood indicate the need for long-term ophthalmic follow up, particularly in children with a history of perinatal adversity. This study identified the relationship between visual, cognitive, motor and demographic factors at 2 years of age and visual acuity (VA) and stereoacuity at 4.5 years of age. METHODS Five hundred sixteen children identified as being at risk of neonatal hypoglycaemia were recruited soon after birth. At 2 years of age, binocular VA, stereoacuity and non-cycloplegic refraction were measured and a clinical neuro-developmental assessment with the Bayley Scales of Infant Development III (BSID-III) was conducted by a trained examiner. Monocular VA and stereoacuity were measured at 4.5 years of age. RESULTS Three hundred twenty-eight children completed both the 2 and 4.5 year vision and neurodevelopmental assessments. Multiple linear regression showed oblique astigmatism and motor function at 2 years were significantly associated with VA at 4.5 years of age, while spherical equivalent refraction, motor scores and stereoacuity at 2 years were significantly associated with stereoacuity at 4.5 years of age. BSID-III motor scores had the best sensitivity (81.8%) and specificity (51.5%) for identifying impaired stereoacuity at 4.5 years. However, all measures at 2 years were poorly associated with VA at 4.5 years old. CONCLUSION Vision and neurodevelopmental measures at 2 years were poorly associated with visual function at 4.5 years of age. However, lower scores on tests of motor function at 2 years may be associated with vision abnormalities, particularly reduced stereopsis, at 4.5 years of age and referral for comprehensive vision assessment for these children may be warranted.

The necessity for intelligent welding robots that meet the demand in real industrial production, according to the objectives of Industry 4.0, has been supported owing to the rapid development of computer vision and the use of new technologies. To improve the efficiency in weld location for industrial robots, this work focuses on trajectory extraction based on color features identification on three-dimensional surfaces acquired with a depth-RGB sensor. The system is planned to be used with a low-cost Intel RealSense D435 sensor for the reconstruction of 3D models based on stereo vision and the built-in color sensor to quickly identify the objective trajectory, since the parts to be welded are previously marked with different colors, indicating the locations of the welding trajectories to be followed. This work focuses on 3D color segmentation with which the points of the target trajectory are segmented by color thresholds in HSV color space and a spline cubic interpolation algorithm is implemented to obtain a smooth trajectory. Experimental results have shown that the RMSE error for V-type butt joint path extraction was under 1.1 mm and below 0.6 mm for a straight butt joint, in addition, the system seems to be suitable for welding beads of various shapes.

Point clouds from scanners or stereo vision inevitably contain outliers which have negative effects on subsequent procedures. Previous works classify outliers according to their characteristics, which guide the design of targeted outlier removal methods. Thus, outlier classification is critical to the corresponding method and outlier removal effect. The proposed type-based outlier removal framework (TBORF) aims to classify outliers more elaborately by considering both the characteristics of the underlying point cloud and the outliers. Therefore, the designed outlier removal methods can be more targeted. To this end, the framework first quantifies the characteristics of the input point cloud using three proposed metrics. According to this quantitative result, the input point clouds are classified into four types. Meanwhile, three new single-criterion methods are proposed to improve the effect on specific types of outlier removal. Based on the point cloud classification and the proposed single-criterion methods, an appropriate combined method is carefully designed for each type of point cloud. Performance evaluations on both outdoor and indoor point cloud datasets demonstrate that TBORF can effectively remove various outliers, facilitating subsequent digital geometry processing operations.

A key function of multitarget tracking (MTT) is the state estimation of unknown targets. In this paper, we propose a new state estimator, the probabilistic data association finite impulse response filter (PDAFIRF), which is designed to overcome the drawbacks of the existing state estimators used for MTT. Because the existing state estimation methods used for MTT employ all past measurements, they may exhibit poor performance because of the accumulated errors caused by modeling uncertainties and numerical errors. To overcome the weaknesses of the existing methods, the proposed PDAFIRF uses recent finite measurements, and therefore, can prevent accumulated errors. The proposed PDAFIRF requires recent finite measurements from lidar and stereo vision and processes data in a probabilistic manner. Owing to its FIR-type structure, the PDAFIRF overcomes the structural and conditional defects of the existing stochastic filters. An MTT algorithm employing the PDAFIRF and lidar and stereo vision data is developed for multi-object tracking and target information estimation with high accuracy. The fusion of lidar and stereo vision sensor data is provided for the PDAFIRF. For verifying the high accuracy of the PDAFIRF-based MTT, a simulation of tracking eight objects under fast-varying conditions is conducted. An experimental test is performed using lidar and stereo vision for tracking three pedestrians. It is demonstrated that the proposed PDAFIRF-based MTT considerably enhances the tracking performance compared to the existing Kalman filter-, unscented Kalman filter-, and particle filter-based algorithms.

Stereo vision is widely deployed on robots and drones to enable depth estimation at a low cost. The combination of lightweight deep neural network (DNN) and cost volumes algorithm is proved to possess the advantages of both high depth estimation accuracy and speed. However, currently there is no accelerator architecture compatible with both efficient DNN inference and cost generation algorithms such as stereo matching. This work proposes a stereo vision accelerator called Dadu-eye, dedicated to real-time processing of high-resolution image streams. The proposed architecture adopts a pipelined hardware design with the techniques of operation approximation and scheduling-level optimization. First, a cost estimation block is designed to generate cost volumes from both luminance and color information. Second, a super pipelined multiplication and accumulation array with a row scan-based fused-layer convolution scheduling is proposed to perform the encoding and decoding neural network efficiently. Finally, an optical flow block is designed and cooperates with the array to approximately predict half of the frames’ depth to achieve real-time (30fps) processing on 1080p view. Based on the SMIC 40 nm CMOS process, this stereo vision accelerator achieves 5.3 TOPS/W power efficiency and significantly reduces 81% off-chip memory access.

Stereo vision systems in autonomous vehicles can come with camera sensors with different models and lenses to detect objects at long and short ranges. The intrinsic/extrinsic parameters of the camera sensors, such as the focal length and distortion parameters used for calibration, are not provided. In this article, we present a novel rectification method for uncalibrated stereo cameras based on their corresponding feature points. To obtain reliable key/feature points and descriptors from stereo images, we propose a deep neural network, called a multilevel feature pooling network. The model processes images individually to learn both local and semiglobal features, from which the location and descriptor of each pixel in the image are estimated. We also introduce a method to balance the zoom effect caused by the different focal lengths of the camera lenses and rectify the images by estimating homography matrices for stereo images based on their refined corresponding feature points.

The use of continuum manipulators in surgical applications can be beneficial because of their inherent safety from their structural compliance. However, the tip pose (position and orientation) accuracy of a continuum manipulator can be low when an external load or disturbance is applied. Closed-loop pose control is preferred to improve the pose accuracy. Considering the requirements stemmed from sterilization and medical device electromagnetic compatibility, the approach of fully utilizing the vision feedback from a stereo endoscopic camera for such a closed-loop pose controller is adopted. Wrist markers are attached to the end-effectors of the surgical continuum manipulators for pose information detection. And these wrist markers are designed in a well-considered way to generate enough variants. The closed-loop pose controller is then designed based on a stereo vision tracking algorithm that detects the position and orientation of the wrist marker. A switching feature is also incorporated into the controller to handle the situation when the closed-loop control is disrupted due to the visual occlusion or the operation transition in dual-arm manipulation. Experimental verification showed that the average errors of the pose feedback are 0.65 mm for the position and 1.05° for the orientation. The experiments of trajectory following under a 120-gram external load and automated dual-arm suturing were also conducted to verify the effectiveness of the proposed controller.

En el desarrollo de los robots de asistencia un reto importante consiste en mejorar la percepción espacial de los robots para la identificación de objetos en diversos escenarios. Para ello, es preciso desarrollar herramientas de análisis y procesamiento de datos de visión estereoscópica artificial. Por esta razón, el presente artículo describe un algoritmo de redes neuronales convolucionales (CNN) implementado en una Raspberry Pi 3 ubicada en la cabeza de una réplica del robot humanoide de código abierto InMoov para estimar la posición en X, Y, Z de un objeto dentro de un entorno controlado. Este artículo explica la construcción de la parte superior del robot InMoov, la aplicación de Transfer Learning para detectar y segmentar un objeto dentro de un entorno controlado, el desarrollo de la arquitectura CNN y, por último, la asignación y evaluación de parámetros de entrenamiento. Como resultado, se obtuvo un error promedio estimado de 27 mm en la coordenada X, 21 mm en la coordenada Y y 4 mm en la coordenada Z. Estos datos son de gran impacto y necesarios al momento de usar esas coordenadas en un brazo robótico para que alcance el objeto y lo agarre, tema que queda pendiente para un futuro trabajo.

In this paper, we propose a low error rate and realtime stereo vision system on G PU. Many stereo vision systems on G PU have been proposed to date. In those systems, the error rates and the processing speed are in trade-off relationship. We propose a real-time stereo vision system on GPU for the high resolution images. This system also maintains a low error rate compared to other fast systems. In our approach, we have implemented the cost aggregation (CA), cross-checking and median filter on GPU in order to realize the real-time processing. Its processing speed is 40 fps for $1436\times 992$ pixels images when the maximum disparity is 145, and its error rate is the lowest among the GPU systems which are faster than 30 fps.

Three-dimensional multiple fish tracking has gained significant research interest in quantifying fish behavior. However, most tracking techniques use a high frame rate, which is currently not viable for real-time tracking applications. This study discusses multiple fish-tracking techniques using low-frame-rate sampling of stereo video clips. The fish were tagged and tracked based on the absolute error of the predicted indices using past and present fish centroid locations and a deterministic frame index. In the predictor sub-system, linear regression and machine learning algorithms intended for nonlinear systems, such as the adaptive neuro-fuzzy inference system (ANFIS), symbolic regression, and Gaussian process regression (GPR), were investigated. The results showed that, in the context of tagging and tracking accuracy, the symbolic regression attained the best performance, followed by the GPR, that is, 74% to 100% and 81% to 91%, respectively. Considering the computation time, symbolic regression resulted in the highest computing lag of approximately 946 ms per iteration, whereas GPR achieved the lowest computing time of 39 ms.

Purpose The purpose of this paper is to present a programming by demonstration (PbD) system based on 3D stereoscopic vision and inertial sensing that provides a cost-effective pose tracking system, even during error-prone situations, such as camera occlusions. Design/methodology/approach The proposed PbD system is based on the 6D Mimic innovative solution, whose six degrees of freedom marker hardware had to be revised and restructured to accommodate an IMU sensor. Additionally, a new software pipeline was designed to include this new sensing device, seeking the improvement of the overall system’s robustness in stereoscopic vision occlusion situations. Findings The IMU component and the new software pipeline allow the 6D Mimic system to successfully maintain the pose tracking when the main tracking tool, i.e. the stereoscopic vision, fails. Therefore, the system improves in terms of reliability, robustness, and accuracy which were verified by real experiments. Practical implications Based on this proposal, the 6D Mimic system reaches a reliable and low-cost PbD methodology. Therefore, the robot can accurately replicate, on an industrial scale, the artisan level performance of highly skilled shop-floor operators. Originality/value To the best of the authors’ knowledge, the sensor fusion between stereoscopic images and IMU applied to robot PbD is a novel approach. The system is entirely designed aiming to reduce costs and taking advantage of an offline processing step for data analysis, filtering and fusion, enhancing the reliability of the PbD system.

The creation of three-dimensional models from an unorganized set of points is an active research area in computer graphics. One of the purposes of this study is to explore the 3D reconstruction of a cube-type artificial reef (CTAR) set by linear structured light and binocular stereo vision technology in an underwater environment. The experimental setup is composed of two cameras in a stereo vision configuration. The alpha shapes method can be used to construct a surface that most closely reflects the artificial reef set described by the points. A parameter study is conducted to assess the scales of the set (i.e., usable volume, surface area, projected area, height, and base diameter) on the basis of 3D reconstruction. Experimental results show that the quality of 3D reconstruction in an underwater environment is acceptable for estimating the scale size of the CTAR set. According to the measurement of the scale sizing of the CTAR set, the relationships between the parameters of the CTAR set and the number of CTAR modules were determined. Moreover, the usable volume of the CTAR set can be estimated depending on the basis of the number of CTAR modules.

Purpose. To report the functional outcomes and reading speeds following PRESBYOND laser blended vision (LBV) using nonlinear aspheric ablation profiles with micro-monovision with the Carl Zeiss Meditec MEL 90 platform. Methods. Data have been collected retrospectively for all patients who underwent PRESBYOND LBV using the MEL 90 excimer laser. Postoperative binocular uncorrected distance and near visual acuity, stereo-acuity, contrast sensitivity, and reading performance were compared with pre-op values measured with patient’s progressive glasses. Mean follow-up was 6 ± 1.2 months. Results. Sixty eyes of 30 patients (mean age 50.47 ± 6.43 years) were included. Of these, 18 patients were hyperopic and 12 patients were myopic with mean SE of 1.28 ± 1.38 D and −2.84 ± 1.86 D, respectively. At 6 months, the mean binocular UDVA was ≥−0.03 ± 0.06 log MAR and the mean binocular UNVA was ≥0.22 ± 0.04 log MAR. The uncorrected reading speeds (words per minute) at the preferred reading distance of 46.17 cm, 60 cm, and 80 cm were significantly better ( p value p > 0.05 for all distances). Uncorrected contrast sensitivity log values showed mild reduction; however, this was not statistically significant for any spatial frequency. There was a significant reduction in post uncorrected stereopsis to 89.67 arc sec, compared with pre-op corrected stereopsis (50.67 arc sec); however, it recovered fully with near correction (53.33 arc sec, p > 0.05 compared with pre). Conclusion. PRESBYOND LBV resulted in significantly better reading speeds and satisfactory functional visual outcomes, without a permanent change in stereo-acuity and contrast sensitivity 6 months postoperatively.

We evaluated the effects of strabismus repair on fixational eye movements (FEMs) and stereopsis recovery in patients with fusion maldevelopment nystagmus (FMN) and patients without nystagmus. Twenty-one patients with strabismus, twelve with FMN and nine without nystagmus, were tested before and after strabismus repair. Eye-movements were recorded during a gaze-holding task under monocular viewing conditions. Fast (fixational saccades and quick phases of nystagmus) and slow (inter-saccadic drifts and slow phases of nystagmus) FEMs and bivariate contour ellipse area (BCEA) were analyzed in the viewing and non-viewing eye. Strabismus repair improved the angle of strabismus in subjects with and without FMN, however patients without nystagmus were more likely to have improvement in stereoacuity. The fixational saccade amplitudes and intersaccadic drift velocities in both eyes decreased after strabismus repair in subjects without nystagmus. The slow phase velocities were higher in patients with FMN compared to inter-saccadic drifts in patients without nystagmus. There was no change in the BCEA after surgery in either group. In patients without nystagmus, the improvement of the binocular function (stereopsis), as well as decreased fixational saccade amplitude and intersaccadic drift velocity, could be due, at least partially, to central adaptive mechanisms rendered possible by surgical realignment of the eyes. The absence of improvement in patients with FMN post strabismus repair likely suggests the lack of such adaptive mechanisms in patients with early onset infantile strabismus. Assessment of fixation eye movement characteristics can be a useful tool to predict functional improvement post strabismus repair.

This paper proposes a method for monocular underwater depth estimation, which is an open problem in robotics and computer vision. To this end, we leverage publicly available in-air RGB-D image pairs for underwater depth estimation in the spherical domain with an unsupervised approach. For this, the in-air images are style-transferred to the underwater style as the first step. Given those synthetic underwater images and their ground truth depth, we then train a network to estimate the depth. This way, our learning model is designed to obtain the depth up to scale, without the need of corresponding ground truth underwater depth data, which is typically not available. We test our approach on style-transferred in-air images as well as on our own real underwater dataset, for which we computed sparse ground truth depths data via stereopsis. This dataset is provided for download. Experiments with this data against a state-of-the-art in-air network as well as different artificial inputs show that the style transfer as well as the depth estimation exhibit promising performance.

Background: Studies had also shown that 86% of children with refractive error living in rural area are without correction as many are unaware of their problem. Refractive error by itself plays a significant role on one’s quality of life and literature had also proven it. Aim: To study the visual functions in ametropes with 6/6 BCVA and compare it with emmetropes. Materials and Methods : A cross-sectional study was conducted for a period of one year. All the 200 subjects were tested for refractive error using auto refractor, distance and near visual acuity was assessed with the ETDRS chart, contrast sensitivity was assessed using LEA symbol chart, color vision was assessed using FM 15 online test and field vision was tested using Bjerrum tangent screen, stereopsis was examined using TNO cards, morphometric measurements was done by measuring the axial length using A scan and the K reading was obtained using auto keratometer and finally the functional assessment was analysed using the reading and writing speed. Results: A total of 200 subjects with 100 ametropes and 100 emmetropes were included in our study. Colour vision, near point accommodation, near point convergence, axial length were significantly altered in ametropes. Contrast sensitivity, steropsis and reading speed were significantly reduced in ametropes compared to emmetropes. There was no significant difference in visual functions between different types refractive error subjects like myopia, hypermetropia and astigmatism. Conclusion: The present study proves that visual acuity alone is not an indicator for assessing the quality of vision instead the complete visual functions has to be screened on all patients reporting with refractive error. Keywords : Visual functions, Refractive error, Emmetropes.

In the 4.0 era, which is progressing towards the 5.0 era, many studies have been carried out on visual sensing in robots. Which has a function as monitoring the movement of robots, especially in the scope of defense in the military. Technological developments in military defense equipment play an important role in supporting the defense of the country's territory. One of them is the development of sensing systems on robots, there are many methods to decide the view of enemy targets against weapons. One of them using the triangulation distance measurement method with the help of a stereo vision camera. This method has advantages in the combat robot system, which is that it can reduce people casualties caused by direct weapon contact with the enemy on the battlefield, especially in urban areas, while the principle of this study is to adopt the similarity of the human eye work system, this method requires a least of two images to get information coordinates the target distance to the camera. Then, we can process the distance information to decide the position of the target against the weapon, while the distance function is carried out to place the robot to take measures. From this background, the author wants to build a visual sensing system using stereo vision. Two webcams with 1080HD resolution each are placed parallel to the combat robot. From the images captured by the two cameras, it can observe the comparison gap between the position of the object with the term disparity, while the results of the disparity are usually used to calculate the distance of the real object and each camera has a viewpoint that can be known, so the distance of the object can be calculated by the more method of triangulation. The results show that the error between the real and the detected distance is 5% and that the camera resolution influences reducing the error.

Wire-driven flexible manipulators are widely used in medical interventions and most of them are controlled in open loop. Their closed-loop control relies on shape sensing, which is challenging. In this paper, a shape sensing method for two-dimensional flexible manipulators based on Statics Model and Bezier curve is proposed. This method builds on the statics modeling of the flexible manipulator and the assumption that the deformed shape is close to a Bezier curve. By employing the motion and tension of the wires together with one or more joints’ locations measured by a stereo vision system, the manipulator shape can be well estimated. Compared with purely vision-based method, this method applies to wider conditions, including visual occlusion. The proposed method is experimentally validated. Results show that for both load-free and loaded conditions this method could well estimate the shape of the flexible manipulator.

Crack assessment of reinforced concrete structures using stereo cameras is a potential way for increasing the efficiency and safety of infrastructure maintenance routines. However, existing damage methods for reinforced concrete structures are based on the segmentation of two-dimensional planes without consideration to the actual size of concrete damage. Furthermore, on-site structural monitoring requires the installation of a large number of contact-based sensing devices, resulting in the potentially excessive consumption of time and financial resources. Therefore, a new vision-based damage assessment method for reinforced concrete structures using a novel intelligent inspection robot with Internet of things–enabled data communication system is proposed in this article. In the first part of this article, the data acquisition system of the inspection robot and the algorithm for three-dimensional structural reconstruction using a stereo camera is discussed. The discussion is followed by a description of the method for crack quantification based on a new proposed deep-learning technique. Finally, to accomplish damage localization, the quantified concrete damage with actual size information is projected onto a three-dimensional surface point cloud reconstruction of the inspected structure. To verify the proposed method, a reinforced concrete column that has undergone cyclic loading failure is used as an inspection subject. The validation experiment demonstrated the ability of the proposed system to segment, localize, and quantify the damage in three-dimensional space with high accuracy.

The tracking and measurement methods of drogue in autonomous aerial refueling which based 2-D image, because of the loss of depth information, are susceptible to light changes, occlusion, etc., resulting in false tracking and the failure of position and orientation of drogue measurement, which may lead to collision of drones. In order to solve these problems, this article intends to introduce 3-D information to solve the position and orientation of drogue based on the original 2-D image tracking method. The method mainly includes two parts, tracking and measurement algorithms, using the decision function to effectively combine the two algorithms. The tracking algorithm provides a positioning area for the measurement algorithm, thereby reducing the amount of measurement operation data. The 3-D measurement result of the measurement algorithm based on stereo vision is used for measuring the drogue's position and orientation, and fed back to the tracking algorithm, which eliminates the candidate area and reduces the error rate to improve accuracy of the tracking algorithm. The measurement algorithm can still be realized stably high-precision under occlusion and no cooperation markers, which is not available in traditional 2-D image-based methods. We perform semiphysical simulation experiments to verify the effectiveness of the proposed method.

Industrial robots are increasingly applied in the automatic die repair welding via the prevalent wire and arc additive manufacturing (WAAM) technology. However, the precise calibration of work coordinates is indispensable for the off-line programming of robotic welding paths, which often results in positioning error, path deviation, or even tool collisions. The die is pre-heated at about 500 °C before the robotic WAAM processes. Thus, it is challenging to calibrate work coordinates by touch sensing because those points on the X-axis and the Y-axis to determine the location of the part need to be caught by human eyes. In this paper, a camera vision calibration (CVC) method based on stereo vision is developed. Image feature points are extracted by a multi-saliency fusion algorithm based on the human visual attention mechanism. Through stereo vision, 3D information of the feature points is obtained, and the workpiece coordinate system (WCS) is finely calibrated. Compared with the random error of human vision calibration (HVC), the proposed method could improve the workpiece’s calibration accuracy, reduce the unexpected collisions in limited space, and improve the dimensional precision of the welding layer.

With the continuous development of computer science and technology, symbol recognition systems may be converted from two-dimensional space to three-dimensional space. Therefore, this article mainly introduces the symbol recognition system based on 3D stereo vision. The three-dimensional image is taken by the visual coordinate measuring machine in two places on the left and right. Perform binocular stereo matching on the edge of the feature points of the two images. A corner detection algorithm combining SUSAN and Harris is used to detect the left and right camera calibration templates. The two-dimensional coordinate points of the object are determined by the image stereo matching module, and the three-dimensional discrete coordinate points of the object space can be obtained according to the transformation relationship between the image coordinates and the actual object coordinates. Then draw the three-dimensional model of the object through the three-dimensional drawing software. Experimental data shows that the logic resources and memory resources occupied by image preprocessing account for 30.4% and 27.4% of the entire system, respectively. The results show that the system can calibrate the internal and external parameters of the camera. In this way, the camera calibration result will be more accurate and the range will be wider. At the same time, it can effectively make up for the shortcomings of traditional modeling techniques to ensure the measurement accuracy of the detection system.

Jordan Murray,1 Kiran Garg,2 Fatema Ghasia3 1Cleveland Clinic Cole Eye Institute, Ophthalmological Research, Cleveland, OH, USA; 2Case Western Reserve University School of Medicine, Cleveland, OH, USA; 3Cleveland Clinic Cole Eye Institute, Department of Ophthalmology, Cleveland, OH, 44195-0001, USACorrespondence: Fatema GhasiaCleveland Clinic Cole Eye Institute, Department of Ophthalmology, 2022, East 105th Street, Cleveland, OH, 44195-0001, USAEmail fatemaghasia@gmail.comPurpose: The aim of the study is to examine the association between amblyopia type and the presence of nystagmus on binocular and monocular functions of the fellow (FE) and amblyopic eye (AE).Methods: We recruited 19 controls and 44 amblyopes (anisometropes=13, strabismic=10, mixed=21). We measured visual, grating, and vernier acuities and high/low spatial frequency (SF) contrast sensitivities in each eye using a staircase method. Stereoacuity was measured with the Titmus fly test. We recorded fixation eye movements (FEM) using high-resolution video-oculography. Subjects were classified as having either no nystagmus (n=18), fusion maldevelopment nystagmus syndrome (FMNS) (n=12), or nystagmus without any structural anomalies that does not meet the criteria for FMNS or infantile nystagmus (n=14).Results: Analysis of visual function by clinical amblyopia type showed that patients with strabismus/mixed amblyopia (F (2,54)=9.5, p< 0.001) were more likely to have poor stereopsis while controlling for AE grating acuity deficit. The FE of patients with anisometropia had greater contrast sensitivity deficits at low (F (2,43)=4.4, p=0.018) and high SF (F (2,42)=10.1, p< 0.001). Analysis of visual function by FEM characteristics (low SF: (F (3,43)=4.3, p=0.010) and high SF: (F (3,42)=7.1, p=0.001) showed that the FE of patients with FMNS had worse low and high SF contrast sensitivities, whereas those without FMNS had greater contrast sensitivity deficits only at high SF compared to controls. Patients with FMNS (F (3,54) = 12.9, p< 0.001) were more likely to have poor stereopsis while controlling for AE grating acuity deficit compared to patients without FMNS. All amblyopic patients had worse high SF contrast sensitivity of the AE irrespective of type or FEM characteristics (Type = F (2,43)=8.8, p=0.001; FEM characteristics= F (3,43)=5.1, p=0.004).Conclusion: The presence of FMNS in patients with strabismic/mixed amblyopia is associated with poor/absent stereopsis. FE deficits vary across amblyopia type. Like FEM abnormalities, visual function deficits are seen in the FE of patients with and without nystagmus.Keywords: amblyopia, fellow eye, contrast sensitivity, fusion maldevelopment nystagmus, stereopsis

Stereo vision (SV) is widely used for noncontact three-dimensional industrial measurement. In SV systems, camera orientation (the angles formed by the two cameras optical axes across the baseline) is among the factors that influence the measurement accuracy. Presently, there is a great divergence regarding the optimal value of camera orientation, i.e., existing theoretical analyses suggest that the optimal angle lies in 30 $^\circ$ –50 $^\circ$ ; however, a camera orientation angle between 60 $^\circ$ –80 $^\circ$ is typically used in practical SV systems and commercial products. Unlike previous models that treated system parameters as uncorrelated, the proposed model introduces covariance to model the correlation between camera orientation and the accuracy of detecting corresponding points. The proposed model proves the optimal camera orientation angle lies in 60 $^\circ$ –80 $^\circ$ , which was verified experimentally via measurements with a circle center tracker (CCT), fringe projection profilometry (FPP), and stereo digital image correlation (DIC).

We propose a novel method to measure and analyze depth resolution in near range. Depth resolution is the smallest depth difference that can be detected by a depth camera, which is an important parameter but is hard to be measured accurately. Intuitively, a stair-shaped target is preferred to measure depth resolution but has many limitations. To overcome the difficulties, we use a flat target and move a depth camera at specified steps to form a stair-shaped depth map. Then, the depth map is compressed to one-dimensional profile by an average cross-section method. The 68-95-99.7 rule is applied for identifying the depth resolution. In order to obtain accurate data, a computer-controlled positioning platform is used to move or rotate a depth camera and a target in high precision. Three types of targets are made and tested, including a stair-shaped metal plate, a flat acrylic board, and a tensioned electric screen. A procedure is provided for aligning the optical plane of depth camera to the target before measuring. The Intel RealSense D400 series depth cameras are adopted for verification. Their theoretical depth resolution is derived and compared to the measured depth resolution. The effect of image resolution (x, y) on the depth resolution (z) is also analyzed. The experimental results offer useful information for researchers to design a depth camera and verify its depth resolution.

Various approaches using neural networks have been proposed to address multi-view stereopsis, but most of them lack capabilities to handle large textureless regions. Hence, a compelling matching network learning comprehensive information from stereo images is constructed to enforce smoothness constraints globally. Trained over binocular stereo datasets only, we show that the network can directly handle the DTU multi-view stereo dataset. When merging together multiple depth maps obtained using either stereo matching, an additional point consolidation procedure is often needed for removing outliers and better aligning individual patches. A second network that consolidates 3D point clouds through directly projecting individual 3D points based on point distributions in their neighborhoods is proposed. Unlike the matching network, this network is trained on local information and is scalable for handling point clouds of any sizes and is capable of processing selected areas of interest as well. Quantitative evaluation on the DTU dataset demonstrates our two networks together can generate point clouds comparable to existing state-of-the-art approaches.

To evaluate the visual outcomes, visual performance, and stereoacuity in presbyopic patients following treatment by a change in the corneal asphericity and micro-monovision. Presbyopic patients with diverse refractive errors and emmetropes (n = 30 eyes) were treated with a custom Q-ablation profile and micro-monovision in the non-dominant eye. There with a difference of Q − 0.30 in the Q profiles between dominant and non-dominant eyes. Patients were assigned in two groups based on the preoperative spherical equivalent (Group 1 + 4.00 to + 0.50, and group 2 neutral to − 3.00). Binocular uncorrected distance visual acuity (binocular UCVA), best-corrected visual acuity (BCVA), binocular uncorrected near visual acuity (binocular UNVA) preoperative and postoperative, spherical equivalent refraction, contrast sensitivity, and stereopsis were analyzed at 1, 3, and 6 months. The mean age was 52.6 ± 5.1 (SD) years. At six months post-operation, the mean binocular uncorrected distance visual acuity (binocular UDVA) was 0.15 ± 0.04 logMAR (20/25-) in group 1, and 0.11 ± 0.05 logMAR (20/25) in group 2, and binocular uncorrected near vision UNVA was 0.5 ± 0.1 M (20/25 J2) in group 1 and 0.45 ± 0.2 M (20/25 J2) group 2. An increase in stereoacuity was found in both groups. The correction of refractive defects using customized corneal asphericity was an effective treatment in presbyopic patients. Furthermore, the treatment was well tolerated in this group of patients. Following surgery, the quality of vision was adequate, and the stereovision improved in this cohort of patients.

Stereo vision is one of the most crucial operations in many computer vision applications, and stereo matching is its most important step. In recent years, stereo matching has developed in the direction of increasing accuracy and higher computational speed, and the real-time stereo matching algorithm based on hardware architecture has been increasingly emphasized due to its applicability in embedded systems. The most frequently used method on FPGA is the census transform (CT) because of its simple structure and easy parallelization and the high quality of the generated disparity maps, but CT has the drawbacks of mismatches in some regions and dependence on a central pixel. In this paper, an improved CT-based semi-global stereo matching algorithm with pipeline and parallel operation based on FPGA is proposed in order to increase matching accuracy in specific regions while satisfying real-time constraints. In the matching cost step, pixels are divided into two parts, with the two methods calculated to generate the bit-vector feature. A four-path semi-global matching algorithm with twice aggregate is proposed in the cost aggregation stage. The left–right check, mismatching point filling, and the median filter to enhance the final disparity map are also required. The novel algorithm is evaluated on the Middleburry benchmark and implemented on Xilinx ZYNQ-7000 SoCs, which results in a throughput of 640 × 480/60 fps, with 64 disparity levels at 100 MHz. Compared with the related work, we improve the average accuracy by 1.61%, making our approach suitable for real-time embedded systems.

Computer vision algorithms are important for many areas of human activity. In particular, the number of applications related to the need to process images of real-world objects with computerized tools and the subsequent use of descriptive information in a variety of interactive and automated decision-making systems is increased. An important tool for analyzing real-world scenes are approaches to the application of stereo vision algorithms. The important step of many stereo matching algorithms is a disparity map. Depending on the content of the observed scene, part of the values on the disparity map can be immediately attributed to background values on a certain basis, or form a "natural" background, which is characterized by loss of informative data due to unacceptable error of subsequent resultant distance values. The calculated disparity map of any algorithm may contain some shortcomings in the form of discontinuities of continuous information areas caused by the complexity of shooting conditions, the impact of noise of various natures, hardware imperfections, and so on. An approach to mitigating the undesirable influence of negative factors on the resulting disparity is the use of mathematical morphology operations to process disparity maps at the post-processing stage. This paper presents information technology for increasing the content of disparity maps based on the mathematical morphology methods. The technology is based on a combination of morphological operations of erosion and dilation, which eliminates the typical problems of discontinuities of monotone regions and erroneous values on disparity maps. The proposed approach allows reducing the impact of common problems that arise during the operation of stereo matching algorithms, as well as increase the overall informativeness of disparity maps for images of real objects in the absence of partial or complete initial data on the characteristics of the observed scene. The results of testing morphological operations with disparity maps for real objects allow us to conclude about the possibility of partial restoration of areas of disparity maps with gaps in continuous information areas, as well as to reduce the impact of random anomalous values on the overall content of the disparity maps.

Reflection happening on a specular surface severely degrades the performance of structured light-based 3D measurements. Traditional structured light techniques remain limited to ideal scenes. Inspired by the multiple exposure fusion technique in photography, which fuses a number of exposure sequences into a well-exposed image, we propose an exposure fusion (EF)-based structured light method in this work. With well-exposed images, the brightness difference of the specular surface can be reduced. Binary gray code is adopted as the primary structured light pattern. With the help of the EF technique, the generated well-exposed structured light images are used for 3D reconstruction of the specular surface. To evaluate the performance of our method, the defective batteries with reflective properties are tested in this experiment. The high dynamic range (HDR)-based structured light method, auto-HDR and ABFDP methods are introduced for comparison. In these experiments, an average error of 0.088 mm and a standard deviation of 0.1921 mm are obtained. These results show that the EF method performs better than other methods for the reconstruction of specular surfaces and can be used to distinguish defects on the surface of a battery.

Objective:The purpose of this study is to determine the effect of latent strabismus on stereoacuity.Methods:Stereoacuities of normal individuals, ranging in age from 18 to 35 years, were measured using the TNO or Titmus tests. The study population was divided into 2 groups regarding the achievement of the accepted excellent stereoacuity threshold of 30 arcseconds (group A) or not (group B). The relationship between latent deviation angles and stereoacuity levels were evaluated.Results:Latent deviation angles ranged from 0 to 35 prism diopters (PD). Both TNO (r=0.380 p=0.002) and Titmus (r=0.306 p=0.015) stereothresholds tended to increase as the latent deviation angles increased. Group A included 27 participants who had either exophoria (18) or orthophoria (9), and the mean deviation angle of this group was 3.07±3.26 PD. Group B included 36 participants, 28 of whom had exophoria, three had esophoria, and the remaining five had orthophoria. The mean angle of deviation in group B was 6.50±6.92 PD. This value was statistically higher than the mean deviation angle in group A (p=0.012). Though none of the esophoric participants achieved excellent stereoacuity, the distribution of the deviation types did not cause any difference between the groups (p=0.077). The mean age of group A was found to be higher than that of group B (p=0.006).Conclusion:Latent deviation can mask true stereosensitivity potentials of subjects to some extent during the stereotesting procedure.

Object Stereo Vision has conventionally been one of the deeply examined areas in computer vision. Stereo matching is employed in numerous modern applications, including robot navigation, augmented reality, and automotive applications. Even though it has a long research history, it is still challenging for the edges of textureless, discontinues, and occluded regions under radiometric variation. This research article proposes a modified histogram equalization, a novel feature extraction, a spatial gradient model, and matching cost, which is robust and stable to images taken in different radiometric variations. The proposed method reduced the average percentage of bad pixels to 3.35 and reduced the relative mean square error (RMSE) up to 30.08 on the Middlebury dataset for different illumination and exposure values. Quantitative and qualitative evaluation of the proposed method demonstrates significant improvement in increasing PSNR and decreasing bad pixel percentage against radiometric variation and state-of-the-art local stereo matching algorithms.