Search

This investigation focuses on servoing a robot's tool with respect to a smooth workpiece surface by making use of the surface's characteristic local differential properties. A novel formulation for 6 degree-of-freedom (DoF) textureless visual servoing based on these properties is proposed, which extends an existing 3-DoF scheme. Our approach naturally combines the geometric tools of computer-aided design and machining (CAD/CAM) theory with the manipulator control tools of visual servoing synergistically to achieve full 6-DoF pose control. A novel family of observed feature sets and their associated interaction matrices are presented. A geometric condition on the surface shape is derived under which local asymptotic stability for 6-DoF is guaranteed. Validation of the proposed method is performed in simulation and experiment using an articulated desktop robot equipped with only a monocular camera and 16 laser pointers.

Inferring the camera pose with high accuracy is deemed crucial in many applications such as robot manipulation and virtual reality. Traditionally, features extracted from camera images are processed by Kalman-based schemes due to their high efficacy and fast response. Yet, the performance of the filter deteriorates if the parameters of the filter are uncertain. Estimating the noise parameters through conventional adaptive schemes alleviates this problem, yet the estimation’s accuracy still suffers from rough parameter estimations. To improve the accuracy of pose estimations further, this work proposes a novel adaptive scheme which employs a multi-model approach to approximate the system posteriori via sampling the noise and initial state covariance priories and progressively adjusting the filter parameters using the drawn samples. The experimental results confirm the enhanced performance of the proposed adaptive method compared to previously applied adaptive and non-adaptive pose estimation schemes, at the expense of additional complexity.

The control of robotic manipulators in unstructured environments is a challenging task. Exploiting the camera images for that purpose, known as visual servoing, offers an interesting solution to the problem. Classic visual servoing techniques were the first attempts towards this goal. However, these methods suffered from system’s shortcomings such as ones imposed by the limited camera’s field of view and the robot’s reachability. Numerous approaches were proposed to overcome these limitations. Nevertheless, most of these techniques assumed full knowledge about the system and did not account for uncertainties. Uncertainties in visual servoing systems are introduced by multiple sources such as camera image noise and robot parameters. The lack of knowledge about system’s parameters may lead to reduced accuracy or even total failure. Adaptive techniques were introduced previously to cope with this matter. However, those techniques were usually useful for deterministic uncertainties (e.g., camera calibration errors). Alternately, robust methods were employed to improve the performance of the system under uncertainties. Yet, those methods were usually conservative and more concerned with the stability of the system, rather than its accuracy. This work proposes three steps towards robust and accurate visual servoing. First, the pose estimation algorithm, used in many visual servoing systems, is revised by introducing novel sensor fusion techniques. Multiple fusion algorithms at different levels of estimation are introduced to enhance the accuracy and robustness of the estimation against system uncertainties. Second, a novel uncertainty estimation technique is presented to approximate the level of uncertainties induced by image noise at different levels of the system. A general approach is used for that matter which has applicability over a wide range of controllers. Finally, multiple constraint-aware and robust controllers with improved stability and numerical feasibility are proposed to enhance the performance of the visual servoing systems in presence of uncertainties. The developed uncertainty model is exploited in robust control design. The effectiveness of each proposed technique is verified through numerous simulations and experiments. As it is expected, the proposed methods are capable of handling the uncertainties and enhancing the accuracy, while accounting for system constraints.

Background : There has been an increasing surge of interest on development of advanced Reinforcement Learning (RL) systems as intelligent approaches to learn optimal control policies directly from smart agents’ interactions with the environment. Objectives : In a model-free RL method with continuous state-space, typically, the value function of the states needs to be approximated. In this regard, Deep Neural Networks (DNNs) provide an attractive modeling mechanism to approximate the value function using sample transitions. DNN-based solutions, however, suffer from high sensitivity to parameter selection, are prone to overfitting, and are not very sample efficient. A Kalman-based methodology, on the other hand, could be used as an efficient alternative. Such an approach, however, commonly requires a-priori information about the system (such as noise statistics) to perform efficiently. The main objective of this paper is to address this issue. Methods : As a remedy to the aforementioned problems, this paper proposes an innovative Multiple Model Kalman Temporal Difference (MM-KTD) framework, which adapts the parameters of the filter using the observed states and rewards. Moreover, an active learning method is proposed to enhance the sampling efficiency of the system. More specifically, the estimated uncertainty of the value functions are exploited to form the behaviour policy leading to more visits to less certain values, therefore, improving the overall learning sample efficiency. As a result, the proposed MM-KTD framework can learn the optimal policy with significantly reduced number of samples as compared to its DNN-based counterparts. Results : To evaluate performance of the proposed MM-KTD framework, we have performed a comprehensive set of experiments based on three RL benchmarks, namely, Inverted Pendulum; Mountain Car, and; Lunar Lander. Experimental results show superiority of the proposed MM-KTD framework in comparison to its state-of-the-art counterparts.

Gaussian mixtures (GMs) are widely used in signal processing applications to capture the multimodal behavior of dynamic systems. Due to an exponential increase in the number of GM components in such applications, Gaussian mixture reduction (GMR) approaches are deemed necessary. Traditionally, the Kullback-Leibler divergence (KLD) is used for GMR along with a moment-matching merging approach to minimize the information loss. However, in certain applications such as image retrieval, preserving the geometric shape of the GM is more appealing. For such applications, this work prescribes the use of the Wasserstein distance (WD), which quantifies the minimum cost of converting one density into another and, therefore, is mostly concerned with the shape difference between the densities. To minimize the change in the shape of the GM, first, similar GM components are identified utilizing the WD. Next, these components are merged by proposing a novel WD-based averaging method. The simulation results confirm the success of the proposed WD-based GMR techniques in providing a better approximation of the original GM in the WD sense as compared to KLD-based methods.

Multiple model adaptive estimation (MMAE) methods are frequently used to overcome the parametric uncertainty of the system's model. Most MMAE methods approximate the state posteriori (posterior probability) by a weighted arithmetic average of model posteriories using a Bayesian weighting scheme. Despite its effectiveness, arguably arithmetic averaging is not the most proper type of averaging for probability densities. Besides, the exploited Bayesian weighting scheme eventually reduces the MMAE to the single best candidate model, which is problematic in many scenarios. Motivated by such shortcomings, this paper proposes a similarity-based approach for MMAE which enhances the estimation accuracy by generalizing the model averaging scheme and providing realistic weights for each model. The proposed approach provides a posteriori which on average is closest to all posteriories and assigns weights to each model based on their similarity to the true model. The choice of similarity measure leads to various schemes. The simulation results confirm the superiority of the proposed MMAE methods as compared to the conventional method.

It is well known that the performance of the Kalman filter deteriorates when the system noise statistics are not available a priori . In particular, the adjustment of measurement noise covariance is deemed paramount as it directly affects the estimation accuracy and plays the key role in applications such as sensor selection and sensor fusion. This letter proposes a novel adaptive scheme by approximating the measurement noise covariance distribution through finite samples, assuming the noise to be white with a normal distribution. Exploiting these samples in approximation of the system state a posteriori leads to a Gaussian mixture model (GMM), the components of which are acquired by Kalman filtering. The resultant GMM is then reduced to the closest normal distribution and also used to estimate the measurement noise covariance. Compared to previous adaptive techniques, the proposed method adapts faster to the unknown parameters and thus provides a higher performance in terms of estimation accuracy, which is confirmed by the simulation results.

Estimating the 3D pose of objects is an important problem in vision-based robotics. Kalman filters are commonly used as efficient solutions to this problem. However, the performance of these filters deteriorates when system’s noise statistics are not known a priori. This work proposes an adaptive scheme based on particle swarm optimization (PSO) to adjust the measurement noise covariance of the filter. The experimental results confirm the effectiveness of the proposed adaptive solution for Kalman-based pose estimation with uncertain noise statistics.

Sensor fusion has found a lot of applications in today’s industrial and scientific world with Kalman filtering being one of the most practiced methods. Despite their simplicity and effectiveness, Kalman filters are usually prone to uncertainties in system parameters and particularly system noise covariance. This paper proposes a Kalman filtering framework for sensor fusion, which provides robustness to the uncertainties in the system parameters such as noise covariance and state initialization. Two methods are developed based on the proposed approach. The effectiveness of the proposed methods is verified through numerous simulations and experiments.

Exploiting multiple cameras for object pose estimation contributes toward achieving enhanced accuracy and robustness to measurement faultiness. Nonlinear optimization frameworks, such as the Gauss-Newton, were traditionally employed for this purpose. This paper proposes a novel algorithm for multicamera pose estimation by using the concept of virtual visual servoing. Two fusion structures, namely centralized and decentralized fusion, are considered. The former offers higher accuracy at the price of increased computation. The latter improves the computation speed, partially sacrificing the system accuracy. The experimental results imply the proposed configurations to be faster than the Gauss-Newton pose estimation method, while offering the same or even higher level of accuracy.

Visual servoing provides a smart solution for robot control in unstructured environments. However, exploiting camera images for control imposes additional constraints to the system. Previously image-based predictive controllers were proposed as a remedy for this problem. Nevertheless, these methods suffered from local stability and possible numerical infeasibility. This work presents two hybrid predictive controllers that are globally stable and have minimum computation time. One of the controllers is based on hybrid visual servoing, while the other minimizes the error in both image and Cartesian space simultaneously with selected ratios. The effectiveness of the proposed methods is verified through numerous simulations.

Robotic manipulators have found wide applications in various industries such as automotive industry and aerospace. Successful robot manipulation usually requires a highly structured workspace, which is expensive and time consuming. Visual servoing offers an interesting remedy to this problem by exploiting camera images for robot control. Typically, a single degree of freedom proportional controller is employed for servoing, which has to compromise between uncertainties and rate of convergence. This work proposes a two degree of freedom controller to address this problem. The proposed controller separates the rate of convergence from the uncertainty, does not require online depth estimation, and is faster compared to classic proportional controller. The effectiveness of the proposed controller is verified through simulations.

Object pose estimation in Cartesian space has found a fundamental role is various applications such as object recognition and visual servoing. Performance and reliability of many of these applications are highly dependent on accuracy and robustness of pose estimation. While monocular systems are usually sufficient for this purpose, multi-camera configurations offer enhanced accuracy and field of view (FOV). Sensor fusion is one smart strategy to exploit data from multiple cameras. This work presents a decentralized sensor fusion scheme which is fault tolerant and does not demand external camera or robot calibrations. Experimental and Simulation data are provided to verify the effectiveness of this scheme.

Object pose estimation is of great importance to many applications, such as augmented reality, localization and mapping, motion capture, and visual servoing. Although many approaches based on a monocular camera have been proposed, only a few works have concentrated on applying multicamera sensor fusion techniques to pose estimation. Higher accuracy and enhanced robustness toward sensor defects or failures are some of the advantages of these schemes. This paper presents a new Kalman-based sensor fusion approach for pose estimation that offers higher accuracy and precision, and is robust to camera motion and image occlusion, compared to its predecessors. Extensive experiments are conducted to validate the superiority of this fusion method over currently employed vision-based pose estimation algorithms.

Robustness is the key feature for practical visual servoing systems. Uncertainty modeling in these systems is the first step towards accurate and robust visual servoing. Camera displacement error due to image noise has been investigated using the noise boundaries in the previous works. Also error covariance propagation has already been developed to model the uncertainties of different visual servoing system components. However, these works were either open-loop or not fully successful under controlled simulations. This work presents a new error modeling method based on error covariance propagation in closed-loop visual servoing systems, considering the discrete-time nature of the system. The proposed framework could be employed for a broad range of controllers. Simulation results suggest the effectiveness of the proposed method.

Multi-camera systems usually offer higher accuracy and robustness to sensor faultiness; however the volume of the processed data in these systems makes them computationally expensive. Moreover, they often require major modifications in the processing units. This work proposes a general pre-processing fusion technique which reduces the computation cost, while enhancing the accuracy of the system. Simulation results confirm the effectiveness of the proposed method.

Accurate visual servoing depends extensively on the quality of pose estimation. Sensor fusion provides a solution to improve accuracy and robustness of pose estimation. This paper introduces sensor fusion methods using two cameras to reduce the inaccuracy of pose estimation. Simulation results are reported to verify the efficiency of the proposed methods.

Fast transportation of the loads without unintentional load sway has become a regular problem with cranes. Any improvement in the time of load transportation leads to higher rate of load accommodation and thus lower costs. In this paper, a four step process is taken to design an optimized fuzzy controller independent and free of any large and expensive computer help. Such a controller can suppress the load sway faster compared to previous reported results of fuzzy crane controls. The proficiency of this controller is confirmed through experimental results.