Search

The rapid shift towards electric vehicles (EVs) demands effective end-of-life strategies for lithium-ion batteries (LIBs), necessitating examining recycling methodologies, particularly the disassembly process. This study presents a technoeconomic analysis of EV battery disassembly, focusing on incorporating robotics to address challenges and capitalize on opportunities. Based on the case study of the Mitsubishi Outlander PHEV battery pack, we identify the most labor and cost-intensive components and introduce a structured approach to evaluate automating disassembly tasks. Classifying tasks based on their automation potential, we find 57% of pack-to-module (P2M) tasks readily automatable, with an additional 24% requiring minimal human intervention. Incorporating this into an analysis of task feasibility, level of automation and robot time efficiency, we establish a roadmap towards robotizing the disassembly process based on human-robot colllaboration, demonstrating a 12.85% increase in net annual revenue and 43.75% reduction in disassembly cost with current level of task feasibility. Our findings indicate a long-term roadmap towards increasing level of automation, while surmounting feasibility of complex disassembly tasks remains an unsolved challenge.

IntroductionThe transition to electric vehicles (EVs) has highlighted the need for efficient diagnostic methods to assess the state of health (SoH) of lithium-ion batteries (LIBs) at the end of their life cycle. Electrochemical Impedance Spectroscopy (EIS) offers a non-invasive technique for determining battery degradation. However, automating this process in industrial settings remains a challenge.MethodsThis study proposes a robotic framework for automating EIS testing using a KUKA KR20 robot arm mounted on a 5 m rail track, equipped with a force/torque sensor and a custom-designed End-of-Arm Potentiostat (EOAT). The system operates in a shared-control mode, enabling the robot to function both autonomously and semi-autonomously, with the option for human intervention to assume control as needed. An admittance controller ensures stable connections, with forces optimized for accuracy and safety. The EOAT’s mechanical strength was validated through finite element analysis.ResultsExperimental validation demonstrated the effectiveness of the developed robotized framework in identifying varying levels of battery degradation. Internal resistance measurements reached up to 1.5 mΩ in the most degraded cells, correlating with significant capacity reductions. The robotic setup achieved consistent and reliable EIS testing across multiple LIB modules.DiscussionThis automated robotic framework enhances battery diagnostics by improving testing accuracy, reducing human intervention, and minimizing safety risks. The proposed approach shows promise for scaling EIS testing in industrial environments, contributing to efficient EV battery reuse and recycling processes.

Abstract 3D edge features, which represent the boundaries between different objects or surfaces in a 3D scene, are crucial for many computer vision tasks, including object recognition, tracking, and segmentation. They also have numerous real-world applications in the field of robotics, such as vision-guided grasping and manipulation of objects. To extract these features in the noisy real-world depth data, reliable 3D edge detectors are indispensable. However, currently available 3D edge detection methods are either highly parameterized or require ground truth labelling, which makes them challenging to use for practical applications. To this extent, we present a new 3D edge detection approach using unsupervised classification. Our method learns features from depth maps at three different scales using an encoder–decoder network, from which edge-specific features are extracted. These edge features are then clustered using learning to classify each point as an edge or not. The proposed method has two key benefits. First, it eliminates the need for manual fine-tuning of data-specific hyper-parameters and automatically selects threshold values for edge classification. Second, the method does not require any labelled training data, unlike many state-of-the-art methods that require supervised training with extensive hand-labelled datasets. The proposed method is evaluated on five benchmark datasets with single and multi-object scenes, and compared with four state-of-the-art edge detection methods from the literature. Results demonstrate that the proposed method achieves competitive performance, despite not using any labelled data or relying on hand-tuning of key parameters.

We propose a framework for contact-rich path following with reinforcement learning based on a mixture of visual and tactile feedback to achieve path following on unknown environments. We employ a curriculum-based domain randomisation approach with a time-varying sampling distribution, rendering our approach is robust to parametric uncertainties in the robot-environment system. Based on evaluation in simulation for compliant path-following case studies with a random uncertain environment, and comparison with LBMPC and FDM methods, the robustness of the obtained policy over a stiffness range $10^{4}$ – $10^{9}$ N/m and friction range 0.1–1.2 is demonstrated. We extend this concept to unknown surfaces with various surface curvatures to enhance the robustness of the trained policy in terms of changes in surfaces. We demonstrate $\sim 15\times $ improvement in trajectory accuracy compared to the previous LBMPC method and $\sim 18\times $ improvement compared to using the FDM approach. We suggest the applications of the proposed method for learning more challenging tasks such as milling, which are difficult to model and dependent on a wide range of process variables.

With the surging interest in electric vehicles (EVs), there is a need for advancements in the development and dismantling of lithium-ion batteries (LIBs), which are highly important for the circular economy. This paper introduces an intelligent hybrid task planner designed for multi-robot disassembly and demonstrates its application to an EV lithium-ion battery pack. The objective is to enable multiple robots to operate collaboratively in a single workspace to execute battery disassembly tasks efficiently and without collisions. This approach can be generalized to almost any disassembly task. The planner uses logical and hierarchical strategies to identify object locations from data captured by cameras mounted on each robot’s end-effector, orchestrating coordinated pick-and-place operations. The efficacy of this task planner was assessed through simulations with three trajectory-planning algorithms: RRT, RRTConnect, and RRTStar. Performance evaluations focused on completion times for battery disassembly tasks. The results showed that completion times were similar across the planners, with 543.06 s for RRT, 541.89 s for RRTConnect, and 547.27 s for RRTStar, illustrating that the effectiveness of the task planner is independent of the specific joint-trajectory-planning algorithm used. This demonstrates the planner’s capability to effectively manage multi-robot disassembly operations.

Abstract The real-time unknown parameter estimation and adaptive tracking control problems are investigated in this paper for a six degrees of freedom (6-DOF) of under-actuated quadrotor unmanned aerial vehicle (UAV). A virtual proportional derivative (PD) controller is designed to maintain the translational dynamics. Two adaptive schemes are developed to handle the attitude dynamics of the UAV with several unknown parameters. In the beginning, a classical adaptive scheme (CAS) using the certainty equivalence principle is proposed and designed. The idea is to design a controller for an ideal situation by assuming the unknown parameters were known. Then the unknown parameters are replaced by their estimation. A theoretical analysis is provided to ensure the trajectory tracking of the adaptive controller. However, an inherent drawback of this scheme is that there is no guarantee for the estimated parameters to converge to the actual values. To address this issue, a new adaptive scheme (NAS) is developed as the next step by adding a continuously differentiable function to the control structure. The proposed technique guarantees handling of the parametric uncertainties with an appropriate design manifold. A rigorous analytical proof, numerical simulation analyses, and experimental validation are presented to show the effectiveness of the proposed control design.

This paper studies the real-time parameter estimation and adaptive tracking control problem for a six degrees of freedom (6-DOF) of quadrotor unmanned aerial vehicle (UAV) as an under-actuated system. A virtual proportional derivative (PD) is proposed to maintain position dynamics. Two adaptive control schemes are designed and compared to maintain the attitude dynamics of UAV while several parameters of UAV are unknown. In the first scheme, a classical adaptive scheme using the certainty equivalence principle is extended and designed for tracing control of the systems with unknown time-varying parameters. To improve the performance of the first scheme, a new control scheme is designed in the second scheme by proposing additional continuous function to handle the unknown parameters. An additional robust term is designed in both schemes to handle the perturbation caused by unknown time-varying parameters. The rigorous analytical proof and numerical simulation analysis are provided to support the efficacy of the proposed controller.

In this study, knowledge of multiple controllers was used and combined with deep reinforcement learning (RL) to train a visual servoing (VS) technique. Deep RL algorithms were successful in solving complicated control problems, however they generally require a large amount of data before they achieve an acceptable performance. We developed a method that generates online hyper-volume action bounds from demonstrations of multiple controllers (experts) to address the issue of insufficient data in RL. The agent then continues to explore the created bounds to find more optimized solutions and gain more rewards. By doing this, we cut out pointless agent explorations, which results in a reduction in training time as well as an improvement in performance of the trained policy. During the training process, we used domain randomization and domain adaptation to make the VS approach robust in the real world. As a result, we showed a 51% decrease in training time to achieve the desired level of performance, compared to the case when RL was used solely. The findings showed that the developed method outperformed other baseline VS methods (image-based VS, position-based VS, and hybrid-decoupled VS) in terms of VS error convergence speed and maintained higher manipulability.

Disassembly of electric vehicle batteries is a critical stage in recovery, recycling and re-use of high-value battery materials, but is complicated by limited standardisation, design complexity, compounded by uncertainty and safety issues from varying end-of-life condition. Telerobotics presents an avenue for semi-autonomous robotic disassembly that addresses these challenges. However, it is suggested that quality and realism of the user’s haptic interactions with the environment is important for precise, contact-rich and safety-critical tasks. To investigate this proposition, we demonstrate the disassembly of a Nissan Leaf 2011 module stack as a basis for a comparative study between a traditional asymmetric haptic-“cobot” master-slave framework and identical master and slave cobots based on task completion time and success rate metrics. We demonstrate across a range of disassembly tasks a time reduction of 22%–57% is achieved using identical cobots, yet this improvement arises chiefly from an expanded workspace and 1:1 positional mapping, and suffers a 10%–30% reduction in first attempt success rate. For unbolting and grasping, the realism of force feedback was comparatively less important than directional information encoded in the interaction, however, 1:1 force mapping strengthened environmental tactile cues for vacuum pick-and-place and contact cutting tasks.

This paper showcases the integration of the Interfacing Toolbox for Robotic Arms (ITRA) with our newly developed hybrid Visual Servoing (VS) methods to automate the disassembly of electric vehicle batteries, thereby advancing sustainability and fostering a circular economy. ITRA enhances collaboration between industrial robotic arms, server computers, sensors, and actuators, meeting the intricate demands of robotic disassembly, including the essential real-time tracking of components and robotic arms. We demonstrate the effectiveness of our hybrid VS approach, combined with ITRA, in the context of Electric Vehicle (EV) battery disassembly across two robotic testbeds. The first employs a KUKA KR10 robot for precision tasks, while the second utilizes a KUKA KR500 for operations needing higher payload capacity. Conducted in T1 (Manual Reduced Velocity) mode, our experiments underscore a swift communication protocol that links low-level and high-level control systems, thus enabling rapid object detection and tracking. This allows for the efficient completion of disassembly tasks, such as removing the EV battery’s top case in 27 s and disassembling a stack of modules in 32 s. The demonstrated success of our framework highlights its extensive applicability in robotic manufacturing sectors that demand precision and adaptability, including medical robotics, extreme environments, aerospace, and construction.

With the proliferation of electric vehicles (EVs) and the consequential increase in EV battery circulation, the need for accurate assessments of battery health and remaining useful life (RUL) is paramount, driven by environmentally friendly and sustainable goals. This study addresses this pressing concern by employing data-driven methods, specifically harnessing deep learning techniques to enhance RUL estimation for lithium-ion batteries (LIB). Leveraging the Toyota Research Institute Dataset, consisting of 124 lithium-ion batteries cycled to failure and encompassing key metrics such as capacity, temperature, resistance, and discharge time, our analysis substantially improves RUL prediction accuracy. Notably, the convolutional long short-term memory deep neural network (CLDNN) model and the transformer LSTM (temporal transformer) model have emerged as standout remaining useful life (RUL) predictors. The CLDNN model, in particular, achieved a remarkable mean absolute error (MAE) of 84.012 and a mean absolute percentage error (MAPE) of 25.676. Similarly, the temporal transformer model exhibited a notable performance, with an MAE of 85.134 and a MAPE of 28.7932. These impressive results were achieved by applying Bayesian hyperparameter optimization, further enhancing the accuracy of predictive methods. These models were bench-marked against existing approaches, demonstrating superior results with an improvement in MAPE ranging from 4.01% to 7.12%.

This article addresses the problem of superpixel-bases segmentation of synthetic aperture radar (SAR) images. Most superpixel segmentation methods have difficulties in segmenting adjacent regions with similar gray values, due to only considering spatial and gray information. To solve this problem and improve segmentation accuracy, this article proposes an SAR image segmentation method based on Fisher vector superpixel generation and label revision. First, the Fisher vector is obtained by processing the Gaussian mixture function. By introducing the Fisher vector, a distance formula is constructed for superpixel segmentation. Therefore, the adjacent regions with similar gray values can be segmented effectively in the generated superpixel map. Second, the superpixels are clustered using the K-means algorithm to obtain the initial label map. Then, with extracted edge information as constraints, the pixel labels obtained by K-means are repaired pixel by pixel to get the middle label map, according to the number and gray value difference of labels. This overcomes the influence of noise generated by K-means. Finally, the middle label map is relabeled using the region growth algorithm to divide pixel blocks. Isolated pixel blocks surrounded by similar labels are corrected, based on the gray mean difference. The final label result has a better segmentation accuracy. Experiments on synthetic SAR images and real images demonstrate that the proposed algorithm achieves higher segmentation accuracy than six state-of-the-art clustering algorithms for SAR image segmentation.

Recent times are witnessing an emergence of sites that are hazardous for human access. This has created a global demand to equip agents with the ability to autonomously inspect such environments by computing a coverage path effectively and efficiently. However, inspection of such sites requires agents to consider the correlation of work, providing precedence provision in visiting regions. The current approaches to compute coverage path in the hazardous sites, however, do not consider precedence provision. To this end, coverage path planning strategies are proposed, which provide precedence provision. To meet the challenges, the problem is divided into two phases: inter-region and intra-region path planning. In the `inter-region' path planning of the approach, the site comprising of multiple disjoint regions is modelled as connectivity graph. Two novel approaches, Mixed Integer Linear Programming (MILP) solution and heuristic based techniques, are proposed to generate the ordered sequence of regions to be traversed. In the `intra-region' path planning of the approach, each region is decomposed into a grid and Boustrophedon Motion is planned over each region. The ability of combined approach to provide complete coverage is proved under minor assumption. An investigative study has been conducted to elucidate the efficiency of the proposed approach in different scenarios using simulation experiments. The proposed approach is evaluated against baseline approaches. The results manifest a significant reduction in cost and execution time, which caters to inspection of target sites comprising of multiple disjoint regions with precedence provision.

In this paper, the problem of simultaneous localization and mapping (SLAM) using a modified Rao Blackwellized Particle Filter (RBPF) (a modified FastSLAM) is developed for a quadcopter system. It is intended to overcome the problem of inaccurate localization and mapping caused by inertial sensory faulty measurements (due to biases, drifts and noises) injected in the kinematics (odometery based) which is commonly used as a motion model in FastSLAM approaches. In this paper, the quadcopter’s dynamics with augmented bias and drift models is employed to eliminate these faults from the localization and mapping process. A modified FastSLAM is then developed in which both Kalman Filter (KF) and Extended Kalman Filter (EKF) algorithms are embedded in a PF with modified particles weights to estimate biases, drifts and landmark locations, respectively. In order to make the SLAM process robust to model mismatches due to parameter uncertainties in the dynamics, measurements are incorporated in the PF and in the particle generation process. This leads to a cascaded two-stage modified FastSLAM in which the extended FastSLAM 1.0 (to include dynamics and sensory faults) is employed in first stage and the results are used in second stage in which probabilistic inverse sensor models are incorporated in the particle generation process of the PF. The efficiency of the proposed approach is demonstrated through a co-simulation between MATLAB-2019b and Gazebo in the robotic operating system (ROS) in which the quadcopter model is simulated in Gazebo in ROS using a modified version of the Hector quadcopter ROS package. The collected pointcloud data using LiDAR is then utilised for feature extraction in the Gazebo. The simulation environment used to this aim is validated based on experimental data.

The market dynamics, and their impact on a future circular economy for lithium-ion batteries (LIB), are presented in this roadmap, with safety as an integral consideration throughout the life cycle. At the point of end-of-life (EOL), there is a range of potential options—remanufacturing, reuse and recycling. Diagnostics play a significant role in evaluating the state-of-health and condition of batteries, and improvements to diagnostic techniques are evaluated. At present, manual disassembly dominates EOL disposal, however, given the volumes of future batteries that are to be anticipated, automated approaches to the dismantling of EOL battery packs will be key. The first stage in recycling after the removal of the cells is the initial cell-breaking or opening step. Approaches to this are reviewed, contrasting shredding and cell disassembly as two alternative approaches. Design for recycling is one approach that could assist in easier disassembly of cells, and new approaches to cell design that could enable the circular economy of LIBs are reviewed. After disassembly, subsequent separation of the black mass is performed before further concentration of components. There are a plethora of alternative approaches for recovering materials; this roadmap sets out the future directions for a range of approaches including pyrometallurgy, hydrometallurgy, short-loop, direct, and the biological recovery of LIB materials. Furthermore, anode, lithium, electrolyte, binder and plastics recovery are considered in order to maximise the proportion of materials recovered, minimise waste and point the way towards zero-waste recycling. The life-cycle implications of a circular economy are discussed considering the overall system of LIB recycling, and also directly investigating the different recycling methods. The legal and regulatory perspectives are also considered. Finally, with a view to the future, approaches for next-generation battery chemistries and recycling are evaluated, identifying gaps for research. This review takes the form of a series of short reviews, with each section written independently by a diverse international authorship of experts on the topic. Collectively, these reviews form a comprehensive picture of the current state of the art in LIB recycling, and how these technologies are expected to develop in the future.

The control of the interaction between the robot and environment, following a predefined geometric surface path with high accuracy, is a fundamental problem for contact-rich tasks such as machining, polishing, or grinding. Flexible path-following control presents numerous applications in emerging industry fields such as disassembly and recycling, where the control system must adapt to a range of dissimilar object classes, where the properties of the environment are uncertain. We present an end-to-end framework for trajectory-independent robotic path following for contact-rich tasks in the presence of parametric uncertainties. We formulate a combination of model predictive control with image-based path planning and real-time visual feedback, based on a learned state-space dynamic model. For modeling the dynamics of the robot-environment system during contact, we introduce the application of the differentiable neural computer, a type of memory augmented neural network (MANN). Although MANNs have been as yet unexplored in a control context, we demonstrate a reduction in RMS error of ∼21.0% compared with an equivalent Long Short-Term Memory (LSTM) architecture. Our framework was validated in simulation, demonstrating the ability to generalize to materials previously unseen in the training dataset.

Task-aware robotic grasping is critical if robots are to successfully cooperate with humans. The choice of a grasp is multi-faceted; however, the task to perform primes this choice in terms of hand shaping and placement on the object. This grasping strategy is particularly important for a robot companion, as it can potentially hinder the success of the collaboration with humans. In this work, we investigate how different grasping strategies of a robot passer influence the performance and the perceptions of the interaction of a human receiver. Our findings suggest that a grasping strategy that accounts for the subsequent task of the receiver improves substantially the performance of the human receiver in executing the subsequent task. The time to complete the task is reduced by eliminating the need of a post-handover re-adjustment of the object. Furthermore, the human perceptions of the interaction improve when a task-oriented grasping strategy is adopted. The influence of the robotic grasp strategy increases as the constraints induced by the object's affordances become more restrictive. The results of this work can benefit the wider robotics community, with application ranging from industrial to household human-robot interaction for cooperative and collaborative object manipulation.

Recognizing material categories is one of the core challenges in robotic nuclear waste decommissioning. All nuclear waste should be sorted and segregated according to its materials, and then different disposal post-process can be applied. In this paper, we propose a novel transfer learning approach to learn boundary-aware material segmentation from a meta-dataset and weakly annotated data. The proposed method is data-efficient, leveraging a publically available dataset for general computer vision tasks and coarsely labeled material recognition data, with only a limited number of fine pixel-wise annotations required. Importantly, our approach is integrated with a Simultaneous Localization and Mapping (SLAM) system to fuse the per-frame understanding delicately into a 3D global semantic map to facilitate robot manipulation in self-occluded object heaps or robot navigation in disaster zones. We evaluate the proposed method on the Materials in Context dataset over 23 categories and that our integrated system delivers quasi-real-time 3D semantic mapping with high-resolution images. The trained model is also verified in an industrial environment as part of the EU RoMaNs project, and promising qualitative results are presented. A video demo and the newly generated data can be found at the project website1 (Supplementary Material).

As robots make their way out of factories into human environments, outer space, and beyond, they require the skill to manipulate their environment in multifarious, unforeseeable circumstances. With this regard, pushing is an essential motion primitive that dramatically extends a robot's manipulation repertoire. In this work, we review the robotic pushing literature. While focusing on work concerned with predicting the motion of pushed objects, we also cover relevant applications of pushing for planning and control. Beginning with analytical approaches, under which we also subsume physics engines, we then proceed to discuss work on learning models from data. In doing so, we dedicate a separate section to deep learning approaches which have seen a recent upsurge in the literature. Concluding remarks and further research perspectives are given at the end of the paper.

This study proposes a hybrid visual servoing technique that is optimised to tackle the shortcomings of classical 2D, 3D and hybrid visual servoing approaches. These shortcomings are mostly the convergence issues, image and robot singularities, and unreachable trajectories for the robot. To address these deficiencies, 3D estimation of the visual features was used to control the translations in Z-axis as well as all rotations. To speed up the visual servoing (VS) operation, adaptive gains were used. Damped Least Square (DLS) approach was used to reduce the robot singularities and smooth out the discontinuities. Finally, manipulability was established as a secondary task, and the redundancy of the robot was resolved using the classical projection operator. The proposed approach is compared with the classical 2D, 3D and hybrid visual servoing methods in both simulation and real-world. The approach offers more efficient trajectories for the robot, with shorter camera paths than 2D image-based and classical hybrid VS methods. In comparison with the traditional position-based approach, the proposed method is less likely to lose the object from the camera scene, and it is more robust to the camera calibrations. Moreover, the proposed approach offers greater robot controllability (higher manipulability) than other approaches.

In this paper, we present a novel concept and primary investigations regarding automated unfastening of hexagonal nuts by means of surface exploration with a compliant robot. In contrast to the conventional industrial approaches that rely on custom-designed motorised tools and mechanical tool changers, we propose to use robot fingers to position, grasp and unfasten unknown random-sized hexagonal nuts, which are arbitrarily positioned in the robot’s task space. Inspired by how visually impaired people handle unknown objects, in this work, we use information observed from surface exploration to devise the unfastening strategy. It combines torque monitoring with active compliance for the robot fingers to smoothly explore the object’s surface. We implement a shape estimation technique combining scaled iterative closest point and hypotrochoid approximation to estimate the location as well as contour profile of the hexagonal nut so as to accurately position the gripper fingers. We demonstrate this work in the context of dismantling an electrically driven vehicle battery pack. The experiments are conducted using a seven degrees of freedom (DoF)–compliant robot fitted with a two-finger gripper to unfasten four different sized randomly positioned hexagonal nuts. The obtained results suggest an overall exploration and unfastening success rate of 95% over an average of ten trials for each nut.

The process of recycling electric vehicle (EV) batteries currently represents a significant challenge to the waste management automation industry. One example of it is the necessity of removing and sorting dismantled components from EV battery pack. This paper proposes a novel framework to semi-automate the process of removing and sorting different objects from an EV battery pack using a mobile manipulator. The work exploits the Behaviour Trees model for cognitive task execution and monitoring, which links different robot capabilities such as navigation, object tracking and motion planning in a modular fashion. The framework was tested in simulation, in both static and dynamic environments, and it was evaluated based on task time and the number of objects that the robot successfully placed in the respective containers. Results suggested that the robot’s success rate in accomplishing the task of sorting the battery components was 95% and 82% in static and dynamic environments, respectively.

The continually expanding number of electric vehicles in circulation presents challenges in terms of end-of-life disposal, driving interest in the reuse of batteries for second-life applications. A key aspect of battery reuse is the quantification of the relative battery condition or state of health (SoH), to inform the subsequent battery application and to match batteries of similar capacity. Impedance spectroscopy has demonstrated potential for estimation of state of health, however, there is difficulty in interpreting results to estimate state of health reliably. This study proposes a model-free, convolutional-neural-network-based estimation scheme for the state of health of high-power lithium-ion batteries based on a dataset of impedance spectroscopy measurements from 13 end-of-first-life Nissan Leaf 2011 battery modules. As a baseline, this is compared with our previous approach, where parameters from a Randles equivalent circuit model (ECM) with and without dataset-specific adaptations to the ECM were extracted from the dataset to train a deep neural network refined using Bayesian hyperparameter optimisation. It is demonstrated that for a small dataset of 128 samples, the proposed method achieves good discrimination of high and low state of health batteries and superior prediction accuracy to the model-based approach by RMS error (1.974 SoH%) and peak error (4.935 SoH%) metrics without dataset-specific model adaptations to improve fit quality. This is accomplished while maintaining the competitive performance of the previous model-based approach when compared with previously proposed SoH estimation schemes.

Motion control in dynamic environments is one of the most important problems in using mobile robots in collaboration with humans and other robots. In this paper, the motion control of a four-Mecanum-wheeled omnidirectional mobile robot (OMR) in dynamic environments is studied. The robot’s differential equations of motion are extracted using Kane’s method and converted to discrete state space form. A nonlinear model predictive control (NMPC) strategy is designed based on the derived mathematical model to stabilize the robot in desired positions and orientations. As a main contribution of this work, the velocity obstacles (VO) approach is reformulated to be introduced in the NMPC system to avoid the robot from collision with moving and fixed obstacles online. Considering the robot’s physical restrictions, the parameters and functions used in the designed control system and collision avoidance strategy are determined through stability and performance analysis and some criteria are established for calculating the best values of these parameters. The effectiveness of the proposed controller and collision avoidance strategy is evaluated through a series of computer simulations. The simulation results show that the proposed strategy is efficient in stabilizing the robot in the desired configuration and in avoiding collision with obstacles, even in narrow spaces and with complicated arrangements of obstacles.

In recent years, regional algorithms have shown great potential in the field of synthetic aperture radar (SAR) image segmentation. However, SAR images have a variety of landforms and a landform with complex texture is difficult to be divided as a whole. Due to speckle noise, traditional over-segmentation algorithm may cause mixed superpixels with different labels. They are usually located adjacent to two areas or contain more noise. In this paper, a new semantic segmentation method of SAR images based on texture complexity analysis and key superpixels is proposed. Texture complexity analysis is performed and on this basis, mixed superpixels are selected as key superpixels. Specifically, the texture complexity of the input image is calculated by a new method. Then a new superpixels generation method called neighbourhood information simple linear iterative clustering (NISLIC) is used to over-segment the image. For images with high texture complexity, the complex areas are first separated and key superpixels are selected according to certain rules. For images with low texture complexity, key superpixels are directly extracted. Finally, the superpixels are pre-segmented by fuzzy clustering based on the extracted features and the key superpixels are processed at the pixel level to obtain the final result. The effectiveness of this method has been successfully verified on several kinds of images. Comparing with the state-of-the-art algorithms, the proposed algorithm can more effectively distinguish different landforms and suppress the influence of noise, so as to achieve semantic segmentation of SAR images.

Semantic segmentation of high-resolution remote sensing images is highly challenging due to the presence of a complicated background, irregular target shapes, and similarities in the appearance of multiple target categories. Most of the existing segmentation methods that rely only on simple fusion of the extracted multi-scale features often fail to provide satisfactory results when there is a large difference in the target sizes. Handling this problem through multi-scale context extraction and efficient fusion of multi-scale features, in this paper we present an end-to-end multi-scale adaptive feature fusion network (MANet) for semantic segmentation in remote sensing images. It is a coding and decoding structure that includes a multi-scale context extraction module (MCM) and an adaptive fusion module (AFM). The MCM employs two layers of atrous convolutions with different dilatation rates and global average pooling to extract context information at multiple scales in parallel. MANet embeds the channel attention mechanism to fuse semantic features. The high- and low-level semantic information are concatenated to generate global features via global average pooling. These global features are used as channel weights to acquire adaptive weight information of each channel by the fully connected layer. To accomplish an efficient fusion, these tuned weights are applied to the fused features. Performance of the proposed method has been evaluated by comparing it with six other state-of-the-art networks: fully convolutional networks (FCN), U-net, UZ1, Light-weight RefineNet, DeepLabv3+, and APPD. Experiments performed using the publicly available Potsdam and Vaihingen datasets show that the proposed MANet significantly outperforms the other existing networks, with overall accuracy reaching 89.4% and 88.2%, respectively and with average of F1 reaching 90.4% and 86.7% respectively.

In this paper, a novel Pixel-Voxel network is proposed for dense 3D semantic mapping, which can perform dense 3D mapping while simultaneously recognizing and labelling the semantic category each point in the 3D map. In our approach, we fully leverage the advantages of different modalities. That is, the PixelNet can learn the high-level contextual information from 2D RGB images, and the VoxelNet can learn 3D geometrical shapes from the 3D point cloud. Unlike the existing architecture that fuses score maps from different modalities with equal weights, we propose a softmax weighted fusion stack that adaptively learns the varying contributions of PixelNet and VoxelNet and fuses the score maps according to their respective confidence levels. Our approach achieved competitive results on both the SUN RGB-D and NYU V2 benchmarks, while the runtime of the proposed system is boosted to around 13 Hz, enabling near-real-time performance using an i7 eight-cores PC with a single Titan X GPU.