Search

Abstract: In this work we propose a method for local feature subset selection, where we simultaneously partition the sample space into localities and select features for them. The partitions and the corresponding local features are represented using a novel notion of feature tree. The problem of finding an appropriate feature tree is then formulated as a reinforcement learning problem. A value-based Monte Carlo tree search with the corresponding credit assignment policy is devised to learn near-optimal feature trees. Furthermore, the Monte Carlo tree search is enhanced in a way to be applicable for large numbers of actions (i.e., features). This objective is achieved by taking into account a bandit-based explorative policy while having a soft exploitive estimation policy. The results for synthetic datasets show that when local features are present in data, the proposed method can outperform other feature selection methods. Furthermore, the results for microarray classification show that the method can obtain results comparable to the state of the art, using a simple KNN classifier. [Copyright &y& Elsevier]

Highlights: [•] An intelligent agent is proposed to negotiate for bilateral contracts. [•] The Stackelberg game is used to match the seller (load) and buyer (generator) agents. [•] A limited time is given to each pairs to negotiate and make an agreement. [•] The intelligent agent uses a Fuzzy SARSA learning method for the negotiation process. [•] Appropriate definitions of states & rewards play key roles in the learning process. [Copyright &y& Elsevier]

In this paper a new method for handling occlusion in face recognition is presented. In this method the faces are partitioned into blocks and a sequential recognition structure is developed. Then, a spatial attention control strategy over the blocks is learned using reinforcement learning. The outcome of this learning is a sorted list of blocks according to their average importance in the face recognition task. In the recall mode, the sorted blocks are employed sequentially until a confident decision is made. Obtained results of various experiments on the AR face database demonstrate the superior performance of proposed method as compared with that of the holistic approach in the recognition of occluded faces. [ABSTRACT FROM AUTHOR]

This paper presents a novel learning approach for Face Recognition by introducing Optimal Local Basis. Optimal local bases are a set of basis derived by reinforcement learning to represent the face space locally. The reinforcement signal is designed to be correlated to the recognition accuracy. The optimal local bases are derived then by finding the most discriminant features for different parts of the face space, which represents either different individuals or different expressions, orientations, poses, illuminations, and other variants of the same individual. Therefore, unlike most of the existing approaches that solve the recognition problem by using a single basis for all individuals, our proposed method benefits from local information by incorporating different bases for its decision. We also introduce a novel classification scheme that uses reinforcement signal to build a similarity measure in a non-metric space. Experiments on AR, PIE, ORL and YALE databases indicate that the proposed method facilitates robust face recognition under pose, illumination and expression variations. The performance of our method is compared with that of Eigenface, Fisherface, Subclass Discriminant Analysis, and Random Subspace LDA methods as well. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we aim at an integrated approach to Dynamic Biped Walking (DBW) and Dynamic Object Manipulation (DOM) at an abstract level. To this end, we offer a unified and abstract concept with a dual interpretation as a DOM and as a DBW system. We validate the proposed approach by using a set of simulations on an illustrative case study and show how it can be used in modeling as well as design of planning and control algorithms for DOM and DBW systems. In the case study, we describe the proposed approach and show its dual interpretation by identifying the relations between 2D dynamic object manipulation of a disc using two planar manipulators and 2D dynamic object locomotion of lower part of a biped robot. More specifically, having obtained the equations of DOM, we change the boundary conditions of the problem in such a way that both radius and mass of the disc tend to infinity. Simultaneously, both size and mass of the manipulators’ base, i.e. the planet earth, tend to some values in the order of human body mass and dimension. Regarding these changes, we can transform DOM into DBW and vice versa. To test the proposed approach, a simple control strategy is introduced to handle impact between the manipulators (legs) and the object (the earth). In addition, a motion planning system is designed in such a way that the manipulators (legs) catch and throw the manipulated object (the earth) in appropriate configurations. [Copyright &y& Elsevier]

Information is personally useless if its beholder cannot individually benefit from it further unless she shares it with those who can exploit that information to increase their mutual outcome. We study sharing such information anonymously in a non-strategic and non-competitive setting, where selfish and cooperative motives align. Although sharing information was cost-free and resulted in expected mutual payoff, almost all subjects showed some levels of hesitancy toward sharing information, and it was more severe in the introverts. According to our mechanistic model, this irrationality could arise because of the excessive subjective value of personally useless information and low other-regarding motives, that necessitated over-attainable personal benefit to drive sharing. Interestingly, other-regarding element correlated with the subjects' belief about how others are cooperative in general. In addition, sensitivity to the value of information correlated with their extraversion level. The results open a new window towards understanding inefficient motives that deprive people of collective benefit. [ABSTRACT FROM AUTHOR]

It has been argued that concepts can be perceived at three main levels of abstraction. Generally, in a recognition system, object categories can be viewed at three levels of taxonomic hierarchy which are known as superordinate, basic, and subordinate levels. For instance, “horse” is a member of subordinate level which belongs to basic level of “animal” and superordinate level of “natural objects.” Our purpose in this study is to take an investigation into visual features at each taxonomic level. We first present a recognition tree which is more general in terms of inclusiveness with respect to visual representation of objects. Then we focus on visual feature definition, that is, how objects from the same conceptual category can be visually represented at each taxonomic level. For the first level we define global features based on frequency patterns to illustrate visual distinctions among artificial and natural. In contrast, our approach for the second level is based on shape descriptors which are defined by recruiting moment based representation. Finally, we show how conceptual knowledge can be utilized for visual feature definition in order to enhance recognition of subordinate categories. [ABSTRACT FROM AUTHOR]

In an incomplete information game, a big challenge is to find the best way of exploiting available information for optimal decision making of the agents. In this paper, two decision making methods, namely model-based and learning-based bidding strategies, are proposed and compared, for repeated Cournot competition of the generators in a day-ahead electricity market. The sum of the rivals’ offered quantities (SROQ) is considered as the state of the agent and its value is estimated using an adaptive expectation method. In the model-based approach, the convergence of the agents’ strategies to the Nash equilibrium point is also studied in two different cases. In the learning-based approach, the optimal bidding strategy is learned through combination of state estimation and a reinforcement learning method. Using the estimated state (SROQ), the optimal decision is learned through a fuzzy Q-learning algorithm. Through a case study, which is performed on the three-bus benchmark Cournot model, the convergence of the generators’ bids to the Nash-Cournot equilibrium is examined. [ABSTRACT FROM AUTHOR]

Abstract: This paper offers a fuzzy balance management scheme between exploration and exploitation, which can be implemented in any critic-only fuzzy reinforcement learning method. The paper, however, focuses on a newly developed continuous reinforcement learning method, called fuzzy Sarsa learning (FSL) due to its advantages. Establishing balance greatly depends on the accuracy of action value function approximation. At first, the overfitting problem in approximating action value function in continuous reinforcement learning algorithms is discussed, and a new adaptive learning rate is proposed to prevent this problem. By relating the learning rate to the inverse of “fuzzy visit value” of the current state, the training data set is forced to have uniform effect on the weight parameters of the approximator and hence overfitting is resolved. Then, a fuzzy balancer is introduced to balance exploration vs. exploitation by generating a suitable temperature factor for the Softmax formula. Finally, an enhanced FSL (EFSL) is offered by integrating the proposed adaptive learning rate and the fuzzy balancer into FSL. Simulation results show that EFSL eliminates overfitting, well manages balance, and outperforms FSL in terms of learning speed and action quality. [Copyright &y& Elsevier]

Rationale: Brain catecholamines have long been implicated in reinforcement learning, exemplified by catecholamine drug and genetic effects on probabilistic reversal learning. However, the mechanisms underlying such effects are unclear. Objectives and methods: Here we investigated effects of an acute catecholamine challenge with methylphenidate (20 mg, oral) on a novel probabilistic reversal learning paradigm in a within-subject, double-blind randomised design. The paradigm was designed to disentangle effects on punishment avoidance from effects on reward perseveration. Given the known large individual variability in methylphenidate's effects, we stratified our effects by working memory capacity and trait impulsivity, putatively modulating the effects of methylphenidate, in a large sample (n = 102) of healthy volunteers. Results: Contrary to our prediction, methylphenidate did not alter performance in the reversal phase of the task. Our key finding is that methylphenidate altered learning of choice-outcome contingencies in a manner that depended on individual variability in working memory span. Specifically, methylphenidate improved performance by adaptively reducing the effective learning rate in participants with higher working memory capacity. Conclusions: This finding emphasises the important role of working memory in reinforcement learning, as reported in influential recent computational modelling and behavioural work, and highlights the dependence of this interplay on catecholaminergic function. [ABSTRACT FROM AUTHOR]

Magnetic flux leakage (MFL) method is the most widely used non-destructive testing techniques for detection and characterization of defects in product transmission pipelines. The maximum safe operating pressure is a crucial parameter in practice, which is predicted with respect to the length, width, and depth of defects. In a previous work, an algorithm is devised to detect and estimate the width of the defect based on MFL signals. In this paper, an efficient method based on axial MFL level contours is devised to estimate the length of the defect. This method uses the patterns of signal level contours in the region corresponding to the defect’s area. In addition, a Gaussian radial basis function neural network (NN) is trained to approximate the depth of the defect. The NN is fed with the estimated length, width, and signal peak-to-peak values, and the output of the network is the estimated depth. The proposed detection and estimation method is applied to MFL measurements to detect and determine the sizing of metal loss defects. The efficacy and accuracy of the proposed methods are examined through a rich set of sample defects that contains simulated defects, designed defects that are carved on a real pipe by milling, and actual pigging data validated by several dig up verifications. Obtained results confirm the effectiveness and accuracy of the proposed depth and length estimation method along with the previously devised detection and width estimation methods, in characterization of metal loss defects. [ABSTRACT FROM AUTHOR]

In our daily life, we continually exploit already learned multisensory associations and form new ones when facing novel situations. Improving our associative learning results in higher cognitive capabilities. We experimentally and computationally studied the learning performance of healthy subjects in a visual-auditory sensory associative learning task across active learning, attention cueing learning, and passive learning modes. According to our results, the learning mode had no significant effect on learning association of congruent pairs. In addition, subjects’ performance in learning congruent samples was not correlated with their vigilance score. Nevertheless, vigilance score was significantly correlated with the learning performance of the non-congruent pairs. Moreover, in the last block of the passive learning mode, subjects significantly made more mistakes in taking non-congruent pairs as associated and consciously reported lower confidence. These results indicate that attention and activity equally enhanced visual-auditory associative learning for non-congruent pairs, while false alarm rate in the passive learning mode did not decrease after the second block. We investigated the cause of higher false alarm rate in the passive learning mode by using a computational model, composed of a reinforcement learning module and a memory-decay module. The results suggest that the higher rate of memory decay is the source of making more mistakes and reporting lower confidence in non-congruent pairs in the passive learning mode. [ABSTRACT FROM AUTHOR]

Abstract: We compare the effects of linear and piecewise linear compliant spines on locomotion performance of quadruped robots in terms of energy efficiency and locomotion speed through a set of simulations and experiments. We first present a simple locomotion system that behaviorally resembles a bounding quadruped with flexible spine. Then, we show that robots with linear compliant spines have higher locomotion speed and lower cost of transportation in comparison with those with rigid spine. However, in linear case, optimal speed and minimum cost of transportation are attained at very different spine compliance values. Moreover, it is verified that fast and energy efficient locomotion can be achieved together when the spine flexibility is piecewise linear. Furthermore, it is shown that the robot with piecewise linear spine is more robust against changes in the load it carries. Superiority of piecewise linear spines over linear and rigid ones is additionally confirmed by simulating a quadruped robot in Webots and experiments on a crawling two-parts robot with flexible connection. [Copyright &y& Elsevier]