Search

Deep reinforcement learning (DRL) provides a new solution for rehabilitation robot trajectory planning in the unstructured working environment, which can bring great convenience to patients. Previous researches mainly focused on optimization strategies but ignored the construction of reward functions, which leads to low efficiency. Different from traditional sparse reward function, this paper proposes two dense reward functions. First, azimuth reward function mainly provides a global guidance and reasonable constraints in the exploration. To further improve the efficiency, a process-oriented aspiration reward function is proposed, it is capable of accelerating the exploration process and avoid locally optimal solution. Experiments show that the proposed reward functions are able to accelerate the convergence rate by 38.4% on average with the mainstream DRL methods. The mean of convergence also increases by 9.5%, and the percentage of standard deviation decreases by 21.2%–23.3%. Results show that the proposed reward functions can significantly improve learning efficiency of DRL methods, and then provide practical possibility for automatic trajectory planning of rehabilitation robot.

Diabetic Retinopathy (DR) is a diabetic complication that predisposes patients to visual impairments that could lead to blindness. Lesion segmentation using deep learning algorithms is an effective measure to screen and prevent early DR. However, there are several types of DR with varying sizes and high inter-class similarity, making segmentation difficult. In this paper, we propose a supervised segmentation method (MSLF-Net) based on multi-scale–multi-level feature fusion to achieve accurate end-to-end DR lesion segmentation. MSLF-Net builds a Multi-Scale Feature Extraction (MSFE) module to extract multi-scale information and provide more comprehensive features for segmentation. This paper further introduces the Multi-Level Feature Fusion (MLFF) module to improve feature fusion using a cross-layer structure. This structure only fuses low- and high-level features of the same class based on category supervision, avoiding feature contamination. Moreover, this paper produces additional masked images for the dataset and performs image enhancement operations to ensure that the proposed method is trainable and functional on small datasets. The extensive experiments are conducted on public datasets IDRID and e_ophtha. The results showed that our proposed feature enhancement method can perform feature fusion more effectively. Therefore, In the end-to-end DR segmentation neural network model, MSLF Net is superior to other similar models in segmentation, and can effectively improve the DR lesion segmentation performance.

To improve the efficiency of deep reinforcement learning (DRL)-based methods for robotic trajectory planning in the unstructured working environment with obstacles. Different from the traditional sparse reward function, this paper presents two brand-new dense reward functions. First, the azimuth reward function is proposed to accelerate the learning process locally with a more reasonable trajectory by modeling the position and orientation constraints, which can reduce the blindness of exploration dramatically. To further improve the efficiency, a reward function at subtask-level is proposed to provide global guidance for the agent in the DRL. The subtask-level reward function is designed under the assumption that the task can be divided into several subtasks, which reduces the invalid exploration greatly. The extensive experiments show that the proposed reward functions are able to improve the convergence rate by up to three times with the state-of-the-art DRL methods. The percentage increase in convergence means is 2.25%-13.22% and the percentage decreases with respect to standard deviation by 10.8%-74.5%.

To improve the efficiency of surgical trajectory segmentation for surgical assessment and robot learning in robot-assisted minimally invasive surgery, this paper presents a fast unsupervised method using video and kinematic data, followed by a promoting procedure to address the over-segmentation issue. An unsupervised deep learning network called dense convolutional encoder-decoder network (DCED-Net) is first proposed to extract more discriminative features from videos in an effective way. DCED-Net has several advantages. It compresses the encoding-decoding structure, strengthens the feature propagation, and avoids the manual annotation. To further improve the accuracy of segmentation, on one hand, a modified transition state clustering model is employed with a strategy of reducing the redundancy of transition points. On the other hand, the segmentation results are promoted by identifying the over-segmented trajectories based on predefined similarity measurements. Extensive experiments on the public data set JIGSAWS show that with our method, the percentage increase in accuracy is 20.3% and the percentage decrease in time cost is 92.6%.