Search

In dynamic outdoor environments characterized by turbulent airflow and intermittent odor plumes, robotic odor plume tracking remains challenging, because existing algorithms heavily rely on manually tuning or learning from expert experience, which are hard to implement in an unknown environment. In this paper, a multi-continuous-output Takagi–Sugeno–Kang fuzzy system was designed and tuned with reinforcement learning to solve the robotic odor source localization problem in dynamic odor plumes. Based on the Lévy Taxis plume tracking controller, the proposed fuzzy system determined the parameters of the controller based on the robot's observation and guided the robot to turn and move towards the odor source at each searching step. The trained fuzzy system was tested in simulated filament-based odor plumes dispersed by a changing wind field. The results showed that the performance of the proposed fuzzy system-based controller trained with reinforcement learning can achieve a similar success rate and higher efficiency compared with a manually tuned and well-designed fuzzy system-based controller. The fuzzy system-based plume tracking controller was also validated through real robotic experiments. [ABSTRACT FROM AUTHOR]

Leg stiffness is considered a prevalent parameter used in data analysis of leg locomotion during different gaits, such as walking, running, and hopping. Quantification of the change in support leg stiffness during stair ascent and descent will enhance our understanding of complex stair climbing gait dynamics. The purpose of this study is to investigate a methodology to estimate leg stiffness during stair climbing and subdivide the stair climbing gait cycle. Leg stiffness was determined as the ratio of changes in ground reaction force in the direction of the support leg ${F}_{l}$ (leg force) to the respective changes in length ${L}_{l}$ during the entire stance phase. Eight subjects ascended and descended an instrumented staircase at different cadences. In this study, the changes of leg force and length (force–length curve) are described as the leg stiffness curve, the slope of which represents the normalized stiffness during stair climbing. The stair ascent and descent gait cycles were subdivided based on the negative and positive work fluctuations of the center-of-mass (CoM) work rate curve and the characteristics of leg stiffness. We found that the leg stiffness curve consists of several segments in which the force–length relationship was similarly linear and the stiffness value was relatively constant; the phase divided by the leg stiffness curve corresponds to the phase divided by the CoM work rate curve. The results of this study may guide biomimetic control strategies for a wearable lower-extremity robot for the entire stance phase during stair climbing. [ABSTRACT FROM AUTHOR]

Model-free reinforcement learning methods have successfully been applied to practical applications such as decision-making problems in Atari games. However, these methods have inherent shortcomings, such as a high variance and low sample efficiency. To improve the policy performance and sample efficiency of model-free reinforcement learning, we propose proximal policy optimization with model-based methods (PPOMM), a fusion method of both model-based and model-free reinforcement learning. PPOMM not only considers the information of past experience but also the prediction information of the future state. PPOMM adds the information of the next state to the objective function of the proximal policy optimization (PPO) algorithm through a model-based method. This method uses two components to optimize the policy: the error of PPO and the error of model-based reinforcement learning. We use the latter to optimize a latent transition model and predict the information of the next state. For most games, this method outperforms the state-of-the-art PPO algorithm when we evaluate across 49 Atari games in the Arcade Learning Environment (ALE). The experimental results show that PPOMM performs better or the same as the original algorithm in 33 games. [ABSTRACT FROM AUTHOR]

Fuzzy inference systems have been widely applied in robotic control. Previous studies proposed various methods to tune the fuzzy rules and the parameters of the membership functions (MFs). Training the systems with only supervised learning requires a large amount of input-output data, and the performance of the trained system is confined by that of the target system. Training the systems with only reinforcement learning (RL) does not require prior knowledge but is time-consuming, and the initialization of the system remains a problem. In this paper, a supervised-reinforced successive training framework is proposed for a multi-continuous-output fuzzy inference system (MCOFIS). The parameters of the fuzzy inference system are first tuned by a limited number of input-output data from an existing controller with supervised training and then are utilized to initialize the system in the reinforcement training stage. The proposed framework is applied in a robotic odor source searching task and the evaluation results demonstrate that the performance of the fuzzy inference system trained by the successive framework is superior to the systems trained by only supervised learning or RL. The system trained by the proposed framework can achieve around a 10% higher success rate compared to the systems trained by only supervised learning or RL. [ABSTRACT FROM AUTHOR]

Predicting the next foot placement of humans during walking can help improve compliant interactions between humans and walking aid robots. Previous studies have focused on foot placement estimation with wearable inertial sensors after heel-strike, but few have predicted foot placements in advance during the early swing phase. In this study, a Bayesian inference-based foot placement prediction approach was proposed. Possible foot placements were modeled as a probability distribution grid map. With selected foot motion feature events detected sequentially in the early swing phase, the foot placement probability map could be updated iteratively using the feature models we built. The weighted center of the probability distribution was regarded as the predicted foot placement. Prediction errors were evaluated with collected walking data sets. When testing with the data from inertial measurement units, the prediction errors were (5.46 cm ± 10.89 cm, −0.83 cm ± 10.56 cm) for cross-velocity walking data and (−4.99 cm ± 12.31 cm, −11.27 cm ± 7.74 cm) for cross-subject–cross-velocity walking data. The results were comparable to previous works yet the prediction could be made earlier. For the subject who walked with more stable gaits, the prediction error can be further decreased. The proposed foot placement prediction approach can be utilized to help walking aid robots adjust their pose before each heel-strike event during walking, which will make human–robot interactions more compliant. This study is also expected to inspire additional probabilistic gait analysis works. [ABSTRACT FROM AUTHOR]

Background: Leg muscle fatigue is the most important factor that affects walking endurance. Considering the legs act as actuators in alternate contact with the ground during walking, the ground reaction force (GRF) of each leg can indirectly reflect the strength of leg muscles. However, it is not clear how the elastically-suspended backpack (ESB) affects GRF of each leg during human level walking.Research Question: How is ESB related in GRF of each leg during walking, and how do multiple variables (stiffness and damping of ESB, load mass, walking speed) affect GRF?Methods: An extended bipedal walking model (EBW) with a spring-mass-damping system was proposed to predict the GRF of each leg. In order to evaluate the prediction effect of the model, seven healthy subjects were recruited to attend the experiments using our backpack prototype and the GRFs data was compared. Each subject walked under 12 conditions (load states: locked or unlocked, walking speed: 3.6 km/h, 4.0 km/h, 4.5 km/h, 5.0 km/h, 5.5 km/h, 6.0 km/h).Results: Results showed that the model could quantitatively predict experimental GRFs over the whole gait cycle (R2≥0.9628) and the characteristic forces (two peak forces and one trough force) were close to the experimental data (average predicted accuracy 93.7 %). The model can reflect relationships between variables and GRF. The relationships showed that an apparent tradeoff exists among the three characteristic forces, and the ESB can produce positive or negative effect under different variables.Significance: This work could help us understand the experimental GRF phenomena, especially the contradictory experimental phenomenon caused by the different parameters. It could also help designers optimize structural parameters of ESB for excellent effects on human. The ESBs with excellent performance can be wildly used in military and tourism. [ABSTRACT FROM AUTHOR]

An increasing number of robotic systems involving lots of robotic individuals are used to serve human, such as intelligent terminal, intelligent storage, intelligence factories, etc. It is a trend of robotics technology that robotics system will become huger with more individuals. In these systems, they form the robotic societies and need establish some computing rules and mechanisms to ensure the operation like all biological social systems. In this paper, a novel system architecture for swarm robotic system, including three layers: human-computer interaction layer, planning layer and execution layer, is put forward, which is effective for task-oriented swarm robotic system. Then, a hierarchical organizational model for the system is presented, which is used to establish management relationship between different layers and individuals. Because task-oriented characteristic is required, this paper elaborates task description knowledge to explain the relationship between tasks for task decomposition and task logic. In addition, a method of behavior generation based on proposition/transition Petri networks is designed, which would effectively assist the system to construct combined behavior using simple individual behavior to solve a variety of tasks. At last, Illustration is shown to prove effectiveness and an implementation of the method based on SociBuilder system is introduced. [ABSTRACT FROM AUTHOR]

Space manipulators are mainly used in the spatial loading task. According to problems of the spatial loading diversity, the testing loading installing position, and the utilization ratio of a test platform, the space manipulator is asked to evaluate the position and attitude of itself. This paper proposes the Point Omnidirectional Coefficient (POC) with unit attitude sphere/circle to describe attitude of the end-effector, which evaluates any points in the attainable space of the manipulators, in combination with the manipulation’s position message, and get relationships between its position and attitude of all points in the attainable space. It represents the mapping between sphere surface and plane for mission attitude constraints and the method for calculating volume of points space including attainable space, Omnidirectional space, and mission attitude space. Furthermore, the Manipulator Omnidirectional Coefficient based on mission or not is proposed for evaluating manipulator performance. Through analysis and simulation about 3D and 2D manipulators, the results show that the above theoretical approach is feasible and the relationships about link lengths, joints angles, attainable space, and Manipulator Omnidirectional Coefficient are drawn for guiding design. [ABSTRACT FROM AUTHOR]

Environmental information plays an important role in deep reinforcement learning (DRL). However, many algorithms do not pay much attention to environmental information. In multi-agent reinforcement learning decision-making, because agents need to make decisions combined with the information of other agents in the environment, this makes the environmental information more important. To prove the importance of environmental information, we added environmental information to the algorithm. We evaluated many algorithms on a challenging set of StarCraft II micromanagement tasks. Compared with the original algorithm, the standard deviation (except for the VDN algorithm) was smaller than that of the original algorithm, which shows that our algorithm has better stability. The average score of our algorithm was higher than that of the original algorithm (except for VDN and COMA), which shows that our work significantly outperforms existing multi-agent RL methods. [ABSTRACT FROM AUTHOR]

Planar surfaces are prevalent components of man-made indoor scenes, and plane extraction plays a vital role in practical applications of computer vision and robotics, such as scene understanding, and mobile manipulation. Nowadays, most plane extraction methods are based on reconstruction of the scene. In this paper, plane representation is formulated in inverse-depth images. Based on this representation, we explored the potential to extract planes in images directly. A fast plane extraction approach, which employs the region growing algorithm in inverse-depth images, is presented. This approach consists of two main components: seeding, and region growing. In the seeding component, seeds are carefully selected locally in grid cells to improve exploration efficiency. After seeding, each seed begins to grow into a continuous plane in succession. Both greedy policy and a normal coherence check are employed to find boundaries accurately. During growth, neighbor coplanar planes are checked and merged to overcome the over-segmentation problem. Through experiments on public datasets and generated saw-tooth images, the proposed approach achieves 80.2% CDR (Correct Detection Rate) on the ABW SegComp Dataset, which has proven that it has comparable performance with the state-of-the-art. The proposed approach runs at 5 Hz on typical 680 × 480 images, which has shown its potential in real-time practical applications in computer vision and robotics with further improvement. [ABSTRACT FROM AUTHOR]