Search

Countersink inspection is crucial in various automated assembly lines, especially in the aerospace and automotive sectors. Advancements in machine vision introduced automated robotic inspection of countersinks using laser scanners and monocular cameras. Nevertheless, the aforementioned sensing pipelines require the robot to pause on each hole for inspection due to high latency and measurement uncertainties with motion, leading to prolonged execution times of the inspection task. The neuromorphic vision sensor, on the other hand, has the potential to expedite the countersink inspection process, but the unorthodox output of the neuromorphic technology prohibits utilizing traditional image processing techniques. Therefore, novel event-based perception algorithms need to be introduced. We propose a countersink detection approach on the basis of event-based motion compensation and the mean-shift clustering principle. In addition, our framework presents a robust event-based circle detection algorithm to precisely estimate the depth of the countersink specimens. The proposed approach expedites the inspection process by a factor of 10 × compared to conventional countersink inspection methods. The work in this paper was validated for over 50 trials on three countersink workpiece variants. The experimental results show that our method provides a standard deviation of 0.025 mm and an accuracy of 0.026 mm for countersink depth inspection despite the low resolution of commercially available neuromorphic cameras. Video Link: https://www.dropbox.com/s/pateqqwh4d605t3/final_video_new.mp4?dl=0. [ABSTRACT FROM AUTHOR]

In recent years, significant advancements have been made in the field of smart greenhouses, particularly in the application of computer vision and robotics for pollinating flowers. Robotic pollination offers several benefits, including reduced labor requirements and preservation of costly pollen through artificial tomato pollination. However, previous studies have primarily focused on the labeling and detection of tomato flowers alone. Therefore, the objective of this study was to develop a comprehensive methodology for simultaneously labeling, training, and detecting tomato flowers specifically tailored for robotic pollination. To achieve this, transfer learning techniques were employed using well-known models, namely YOLOv5 and the recently introduced YOLOv8, for tomato flower detection. The performance of both models was evaluated using the same image dataset, and a comparison was made based on their Average Precision (AP) scores to determine the superior model. The results indicated that YOLOv8 achieved a higher mean AP (mAP) of 92.6% in tomato flower and bud detection, outperforming YOLOv5 with 91.2%. Notably, YOLOv8 also demonstrated an inference speed of 0.7 ms when considering an image size of 1920 × 1080 pixels resized to 640 × 640 pixels during detection. The image dataset was acquired during both morning and evening periods to minimize the impact of lighting conditions on the detection model. These findings highlight the potential of YOLOv8 for real-time detection of tomato flowers and buds, enabling further estimation of flower blooming peaks and facilitating robotic pollination. In the context of robotic pollination, the study also focuses on the deployment of the proposed detection model on the 3P2R gantry robot. The study introduces a kinematic model and a modified circuit for the gantry robot. The position-based visual servoing method is employed to approach the detected flower during the pollination process. The effectiveness of the proposed visual servoing approach is validated in both un-clustered and clustered plant environments in the laboratory setting. Additionally, this study provides valuable theoretical and practical insights for specialists in the field of greenhouse systems, particularly in the design of flower detection algorithms using computer vision and its deployment in robotic systems used in greenhouses. [ABSTRACT FROM AUTHOR]

Robotic vision plays a key role for perceiving the environment in grasping applications. However, the conventional framed-based robotic vision, suffering from motion blur and low sampling rate, may not meet the automation needs of evolving industrial requirements. This paper, for the first time, proposes an event-based robotic grasping framework for multiple known and unknown objects in a cluttered scene. With advantages of microsecond-level sampling rate and no motion blur of event camera, the model-based and model-free approaches are developed for known and unknown objects' grasping respectively. The event-based multi-view approach is used to localize the objects in the scene in the model-based approach, and then point cloud processing is utilized to cluster and register the objects. The proposed model-free approach, on the other hand, utilizes the developed event-based object segmentation, visual servoing and grasp planning to localize, align to, and grasp the targeting object. Using a UR10 robot with an eye-in-hand neuromorphic camera and a Barrett hand gripper, the proposed approaches are experimentally validated with objects of different sizes. Furthermore, it demonstrates robustness and a significant advantage over grasping with a traditional frame-based camera in low-light conditions. [ABSTRACT FROM AUTHOR]

Semi-autonomous self driving car technologies are commercially available today, but making them fully autonomous under guaranteed safety is still an open challenge. There are holonomic robot path planning algorithms for generating guaranteed collision-free paths even in very complex but completely known environments. However, it is not feasible to apply these algorithms to non-holonomic cars. In this paper we propose a novel approach to solve this problem in real-time, by generating a series of incremental collision-free path segments from the start to end configuration using an incremental sampling-based planner, such as Rapidly-exploring Random Trees (RRT). The proposed approach employs only local sensor data (point cloud data) for path planning of nonholonomic self driving cars in quasi static unknown environments. The proposed planner generates real-time paths that guarantee safety. The algorithms are extensively testedand validated in popular benchmark simulation environments. [ABSTRACT FROM AUTHOR]

This paper reviews the existing vision-based tactile sensor (VBTS) designs reported in the literature. Although some reviews on VBTSs already exist in the literature. We believe it is necessary to review existing VBTS designs to formulate a guideline for developing such systems considering recent developments in the manufacturing and imaging technologies. Therefore, the main emphasis of this paper is to investigate current manufacturing trends and component selection criteria for developing a complete VBTS system. Further, the motivation behind this review is to identify the shortcomings in the current VBTS development technology and to furnish viable solutions to overcome such challenges. First, three different modalities of VBTSs are discussed: i) Waveguide-type designs, ii) marker-tracking based designs, and ii) reflective membrane designs. Next, a detailed discussion on various design aspects, like manufacturing, selection, and arrangements of the various sensor components, of the VBTSs is included. Then, a discussion on the validation/testing of various VBTSs is presented. Finally, based on the review, several challenges related to the development of VBTS are presented and the future research directions to overcome such challenges are recommended. This will serve the research community in determining the future research directions in the area of VBTS development. [ABSTRACT FROM AUTHOR]

This paper explores methods that make use of visual cues aimed at generating actual haptic sensation to the user, namely pseudo-haptics. We propose a new pseudo-haptic feedback-based method capable of conveying 3D haptic information and combining visual haptics with force feedback to enhance the user's haptic experience. We focused on an application related to tumor identification during palpation and evaluated the proposed method in an experimental study where users interacted with a haptic device and graphical interface while exploring a virtual model of soft tissue, which represented stiffness distribution of a silicone phantom tissue with embedded hard inclusions. The performance of hard inclusion detection using force feedback only, pseudo-haptic feedback only, and the combination of the two feedbacks was compared with the direct hand touch. The combination method and direct hand touch had no significant difference in the detection results. Compared with the force feedback alone, our method increased the sensitivity by 5%, the positive predictive value by 4%, and decreased detection time by 48.7%. The proposed methodology has great potential for robot-assisted minimally invasive surgery and in all applications where remote haptic feedback is needed. [ABSTRACT FROM AUTHOR]

In this paper, we propose multiple parameter models based adaptive switching control system for robot manipulators. We first uniformly distribute the parameter set into a finite number of smaller compact subsets. Then, distributed candidate controllers are designed for each of these smaller compact subsets. Using Lyapunov inequality, a candidate controller is identified from the finite set of distributed candidate controllers that best estimates the plant at each instant of time. The design reduced the observer-controller gains by reducing modeling errors and uncertainties via identifying an appropriate control/model via choosing largest guaranteed decrease in the value of the Lyapunov function energy function. Compared with CE based CAC design, the proposed design requires smaller observer-controller gains to ensure stability and tracking performance in the presence of large-scale modeling errors and disturbance uncertainties. In contrast with existing adaptive method, multiple model based distributed hybrid design can be used to reduce the energy consumption of the industrial robotic manipulator for large scale industrial automation by reducing actuator input energy. Finally, the proposed hybrid adaptive control design is experimentally tested on a 3-DOF Phantom robot manipulator to demonstrate the theoretical development for real-time applications. [ABSTRACT FROM AUTHOR]

This paper presents a unified inverse kinematics and dynamics model of a metamorphic parallel mechanism with pure rotation and pure translation phases. By altering one rotation axis of the reconfigurable Hooke (rT) joints in the limbs, the mechanism can be switched into one of the two phases. To provide joint reaction forces for optimal design and control, Newton method is used in developing the dynamics model which is unified by combining geometric constraints and parameters in both phases. An analytical investigation provides special joint force decomposition and limb coordinate setup to decouple the dynamics equations between the platform and the limbs. This reduces the dynamics computation load from solving a 15 × 15 matrix to a 6 × 6 matrix. A numerical example is given to illustrate the proposed method and simulation results are explained and compared between the two phases. The work on this paper gives good reference for optimal design and control of this metamorphic parallel mechanism in different applications using two phases. [ABSTRACT FROM AUTHOR]

In this paper, a vision-based haptic feedback system has been proposed with the aim to assist the movement of an endoscopic device during capsule endoscopy (CE) procedures. We present a general system architecture consisting of three modules responsible for vision, haptic guidance and control of movements. The vision module generates 3D local maps as well as local navigation trajectory for endoluminal navigation. The haptic guidance module consists of a haptic device that allows the user to control the movement of the capsule along the generated path. The haptics module also helps the operator by transforming the 3D maps and the relative paths into a guiding virtual force. Measuring the current relative distance between the user input and the maps boundaries, the haptic guidance module will check if the user is moving away or toward the colonic walls and will generate a feedback force with the aim to assist the operator during the navigation procedure. The user will also sense an attractive virtual feedback force toward the generated path that will help the user in the navigation. Finally, the movement control module is the interface between the haptics module and the chosen manipulator. The final goal is to develop a complete active CE robotic platform with haptic feedback in order to enhance safety, to reduce cost (using the same system as a training simulator as well as real endoscopic platform) and to help the operator during the navigation by combining all 3D local maps into a full 3D reconstructed colon. [ABSTRACT FROM AUTHOR]

In this paper we introduce a novel contact-sensing algorithm for a robotic fingertip which is equipped with a 6-axis force/torque sensor and covered with a deformable rubber skin. The design and the sensing algorithm of the fingertip for effective contact information identification are introduced. Validation tests show that the contact sensing fingertip can estimate contact information, including the contact location on the fingertip, the direction and the magnitude of the friction and normal forces, the local torque generated at the surface, at high speed (158-242 Hz) and with high precision. Experiments show that the proposed algorithm is robust and accurate when the friction coefficient $$\le $$ 1. Obtaining such contact information in real-time are essential for fine object manipulation. Using the contact sensing fingertip for surface exploration has been demonstrated, indicating the advantage gained by using the identified contact information from the proposed contact-sensing method. [ABSTRACT FROM AUTHOR]

This paper investigates various topologies and mobility of a class of metamorphic parallel mechanisms synthesized with reconfigurable rT PS limbs. Based on the reconfigurable Hooke (rT) joint, the rT PS limb has two phases which result in parallel mechanisms having ability of mobility change. While in one phase the limb has no constraint to the platform, in the other it constrains the spherical joint center to lie on a plane which is used to demonstrate different topologies of the nrT PS metamorphic parallel mechanisms by investigating various relations (parallel or intersecting) among the n constraint planes ( n = 2,3,..,6). Geometric constraint equations of the platform rotation matrix and translation vector are set up based on the point-plane constraint, which reveals mobility and redundant geometric conditions of the mechanism topologies. By altering the limbs into the non-constraint phase without constraint plane, new mechanism phases are deduced with mobility change based on each mechanism topology. [ABSTRACT FROM AUTHOR]

In this paper, the problem of time varying telecommunication delays in passive teleoperation systems is addressed. The design comprises delayed position, velocity and position-velocity signals with the local position and velocity signals of the master and slave manipulators. Nonlinear adaptive control terms are employed locally to cope with uncertain parameters associated with the gravity loading vector of the master and slave manipulators. Lyapunov-Krasovskii function is employed for three methods to establish asymptotic tracking property of the closed loop teleoperation systems. The stability analysis is derived for both symmetrical and unsymmetrical time varying delays in the forward and backward communication channel that connects the local and remote sites. Finally, evaluation results are presented to illustrate the effectiveness of the proposed design for real-time applications. [ABSTRACT FROM AUTHOR]

This paper investigates the use of inverse finite-element modeling (IFEM)-based methods for tissue parameter identification using a rolling indentation probe for surgical palpation. An IFEM-based algorithm is proposed for tissue parameter identification through uniaxial indentation. IFEM-based algorithms are also created for locating and identifying the properties of an embedded tumor through rolling indentation of the soft tissue. Two types of parameter identification for the tissue tumor are investigated (1) identifying the stiffness (μ) of a tumor at a known depth and (2) estimating the depth of the tumor (D) with known mechanical properties. The efficiency of proposed methods has been evaluated through silicone and porcine kidney experiments for both uniaxial indentation and rolling indentation. The results show that both of the proposed IFEM methods for uniaxial indentation and rolling indentation have good robustness and can rapidly converge to the correct results. The tissue properties estimated using the developed method are generic and in good agreement with results obtained from standard material tests. The estimation error of μ through uniaxial indentation is below 3 % for both silicone and kidney; the estimation error of μ for the tumor through rolling indentation is 7-9 %. The estimation error of D through rolling indentation is 1-2 mm. [ABSTRACT FROM AUTHOR]

This paper presents a novel technique for identifying soil parameters for a wheeled vehicle traversing unknown terrain. The identified soil parameters are required for predicting vehicle drawbar pull and wheel drive torque, which in turn can be used for traversability prediction, traction control, and performance optimization of a wheeled vehicle on unknown terrain. The proposed technique is based on the Newton Raphson method. An approximated form of a wheel-soil interaction model based on Composite Simpson’s Rule is employed for this purpose. The key soil parameters to be identified are internal friction angle, shear deformation modulus, and lumped pressure-sinkage coefficient. The fourth parameter, cohesion, is not too relevant to vehicle drawbar pull, and is assigned an average value during the identification process. Identified parameters are compared with known values, and shown to be in agreement. The identification method is relatively fast and robust. The identified soil parameters can effectively be used to predict drawbar pull and wheel drive torque with good accuracy. The use of identified soil parameters to design a traversability criterion for wheeled vehicles traversing unknown terrain is presented. [ABSTRACT FROM AUTHOR]

A path planning algorithm for a mobile robot subject to nonholonomic constraints is presented. The algorithmemploys a global- local strategy, and solves the problem in the 2D workspace of the robot, without generating the complexconfiguration space. Firstly, a visibility graph is constructed for finding a collision-free shortest path for a point. Secondly,the path for a point is evaluated to find whether it can be used as a reference to build up a feasible path for the mobile robot.If not, this path is discarded and the next shortest path is selected and evaluated until a right reference path is found. Thirdly,robot configurations are placed along the selected path in the way that the robot can move from one configuration to the nextavoiding obstacles. Lemmas are introduced to ensure that the robot travels using direct, indirect or reversal manoeuvres. Thealgorithm is computationally efficient and runs in time O(nk + n log n) for k obstacles andn vertices. The path found is near optimal in terms of distance travelled. The algorithm is tested in computersimulations and test results are presented to demonstrate its versatility in complex environments. [ABSTRACT FROM AUTHOR]