Search

In this paper, a novel metaheuristic algorithm called Social Network Search (SNS) is developed for solving optimization problems. The SNS algorithm simulates the attempts of users in social networks to gain more popularity by modeling the moods of users in expressing their opinions. These moods are named Imitation, Conversation, Disputation, and Innovation, which are real-world behaviors of users in social networks. These moods are used as optimization operators and model how users are affected and motivated to share their new views. To evaluate the performance of the SNS algorithm, two comparative studies with different properties were conducted. In the first step, 210 mathematical functions have been chosen, which include 120 fixed-dimension, 60 N-dimension, and 30 CEC 2014 problems. Seven metaheuristics are selected from the literature, and the statistical results of these methods are calculated and analyzed. Also, to provide a valid judgment about the performance of the new algorithm, four nonparametric statistical tests have been used. In the next step, the performance of the proposed algorithm is compared to some state-of-the-art algorithms in dealing with CEC 2017 problems. According to the performance of algorithms, the SNS method is capable of achieving better results compared to the other metaheuristics in 101 cases (48%) and performed the same or comparatively in dealing with the other problems.

Various communication systems have been developed to integrate the haptic channel in digital communication. Future directions of such haptic technologies are moving towards realistic virtual reality applications and human-robot social interaction. With the digitisation of touch, robots equipped with touch sensors and actuators can communicate with humans on a more emotional and intimate level, such as sharing a hug or kiss just like humans do. This paper presents the design guideline, implementation and evaluations of a novel haptic kissing machine for smart phones - the Kissenger machine. The key novelties and contributions of the paper are: (i) A novel haptic kissing device for mobile phones, which uses dynamic perpendicular force stimulation to transmit realistic sensations of kissing in order to enhance intimacy and emotional connection of digital communication; (ii) Extensive evaluations of the Kissenger machine, including a lab experiment that compares mediated kissing with Kissenger to real kissing, a unique haptic Turing test that involves the first academic study of human-machine kiss, and a field study of the effects of Kissenger on long distance relationships. The first experiment showed that mediated kissing with Kissenger elicited similar ratings for pleasure, arousal and user experience as real kissing. Experiment 2 confirmed our hypothesis that interrogators have a higher chance of winning the Imitation Game (Turing test) when Kissenger is used during the game. Results from experiment 3 showed that long relationship couples who used Kissenger for a week experienced increased relationship satisfaction and decreased perceived stress.

This paper proposes a new reinforcement learning approach for executing combat unmanned aerial vehicle (CUAV) missions. We consider missions with the following goals: guided missile avoidance, shortest-path flight and formation flight. For reinforcement learning, the representation of the current agent state is important. We propose a novel method of using the coordinates and angle of a CUAV to effectively represent its state. Furthermore, we develop a reinforcement learning algorithm with enhanced exploration through amplification of the imitation effect (AIE). This algorithm consists of self-imitation learning and random network distillation algorithms. We assert that these two algorithms complement each other and that combining them amplifies the imitation effect for exploration. Empirical results show that the proposed AIE approach is highly effective at finding a CUAV’s shortest-flight path while avoiding enemy missiles. Test results confirm that with our method, a single CUAV reaches its target from its starting point 95% of the time and a squadron of four simultaneously operating CUAVs reaches the target 70% of the time.

Recent developments in Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) have made them highly useful for various tasks. However, they both have their respective constraints that make them incapable of completing intricate tasks alone in many scenarios. For example, a UGV is unable to reach high places, while a UAV is limited by its power supply and payload capacity. In this paper, we propose an Imitation Augmented Deep Reinforcement Learning (IADRL) model that enables a UGV and UAV to form a coalition that is complementary and cooperative for completing tasks that they are incapable of achieving alone. IADRL learns the underlying complementary behaviors of UGVs and UAVs from a demonstration dataset that is collected from some simple scenarios with non-optimized strategies. Based on observations from the UGV and UAV, IADRL provides an optimized policy for the UGV-UAV coalition to work in an complementary way while minimizing the cost. We evaluate the IADRL approach in an visual game-based simulation platform, and conduct experiments that show how it effectively enables the coalition to cooperatively and cost-effectively accomplish tasks.

The growing production and usage of autonomous, cooperative mobile robots are going to be an integral part of our daily lives in the upcoming years. In this paper, a two agent master slave system has been explored wherein the slave imitates the master. Master slave robot communication form an important segment of swarm robotics and intelligence. Slaves generally follow the commands of the master. We particularly explore a domain wherein the slave imitates the motion of the master. We approach to demonstrate the master slave communication and eventual demonstration of imitation while keeping the whole arrangement and system’s cost low. We use low cost alternatives of other communication methodologies while essentially maintaining the reliability.

In smart homes, access-control policies increasingly depend on contexts, such as who is taking an action, whether there is an emergency, or whether an adult is nearby. The vast literature on context sensing could potentially be leveraged to support contextual access control, yet this literature mostly ignores attacks, adversaries, and privacy. In this paper, we reevaluate the literature on home context sensing through a security and privacy mindset. We first describe a novel threat model in smart homes focusing on the capabilities of non-technical adversaries. Replay, imitation, and shoulder-surfing attacks are much more likely in this model. We summarize contexts relevant to access control in homes, mapping them to existing sensors. We then systematize the sensing literature to construct a decision framework for home context sensing that considers security, privacy, and usability. Applying our framework, we find that current sensors do not fully mitigate likely threats in homes. Some sensors are susceptible to simple threats like physical denial-of-service attacks, making it easy to bypass policies relying on the absence of a characteristic. Many sensors collect more data than needed and are not effective for all groups of users or under all situations.

As driving task has been and still is the domain of humans, they are a relatively good model of how to behave on road. However, as we would like to constantly raise standards of safety and efficiency of transportation by automated driving, simple imitation of human driving seems not to be good enough. In this paper, we present a method for learning a baseline agent represented as a neural network, which is responsible for planning the behaviour of an automated car, based on a set of expert demonstrations. We show that imitation learning could be a promising way for achieving initial policy, which is almost as effective as human. The generated policy could be then fine-tuned by the reinforcement learning methods. We show that such an approach suits well for an automated driving task and allows faster training of the RL-based artificial driver.

This paper proposes an adaptive method to enable imitation learning from expert demonstrations in a multi-agent context. The proposed system employs the inverse reinforcement learning method to a coupled Dynamic Bayesian Network to facilitate dynamic learning in an interactive system. This method studies the interaction at both discrete and continuous levels by identifying inter-relationships between the objects to facilitate the prediction of an expert agent. We evaluate the learning procedure in the scene of learner agent based on probabilistic reward function. Our goal is to estimate policies that predict matched trajectories with the observed one by minimizing the Kullback-Leiber divergence. The reward policies provide a probabilistic dynamic structure to minimise the abnormalities.

In this paper, an end-to-end meta imitation learning method supplemented with target recognition (TaR-MIL) is proposed for one-shot learning. This approach divides the procedure of imitating from demonstrations into two parts: distinguishing the target object from distractors and executing the correct actions. Accordingly, the objective of imitation is defined as the combination of target recognition and behavior cloning. Specifically, a target recognition module is adopted in the model architecture, which helps to extract useful information about tasks from observations during training. After training with demonstrations of various tasks via meta learning, a policy capable of solving new tasks given one demonstration is obtained. The real-world experiments on a UR10e robot arm illustrate that, the derived policy manages to perform placing tasks in new scenarios or with new objects after one video demonstration is given, which verify the effectiveness of the proposed method.

Skill imitation has been an important ability in human-robot collaboration since it allows expeditious robot teaching of new tasks never seen before. To mimic the human pose, inverse kinematic solvers have been used to endow kinematic structures with human-like motion. Nevertheless, these solutions tend to be formulated for a specific robot or task. To address this generalization issue, this work presents the ICONS framework for imitation constraints, which proposes a general formulation for pose imitation, paired with a computationally efficient solver, inspired by the FABRIK algorithm. Three versions of the solver were developed to optimize the presented constraints. To assess the performance of ICONS, two tasks were evaluated, an incision task, and an assembly task. Fifty demonstrations were collected for each task. We compared the performance of our method, using pose accuracy and occlusion, against the numerical solver baseline (FABRIK). Notably, the ICONS framework improved the pose accuracy by 58% and reduced the environment occlusion by 38%. Moreover, the computational efficiency of the ICONS framework was assessed. Results show that the proposed algorithm maintains the efficiency of the baseline, finding the target solution under 10 iterations.

Post processing is usually necessary for photographs captured in undesired situations to improve their visual quality. As photo retouching requires operators to receive professional training, automatic photograph enhancing system would benefit amateur photographers and also save experts from tedious retouching tasks. In this paper, we present a deep learning framework based on imitation-to-innovation training scheme. Our method integrates the bilateral grid data structure and Generative Adversarial Network (GAN) to achieve high time-efficiency and appealing retouching output. Besides, we present bilateral loss function to maintain piecewise smoothness. The experimental results and user study demonstrate that our method produces improved results than other state-of-the-art methods.

The continuous opinion and discrete actions (CODA) model is a practical approach in the field of opinion dynamics to investigate how action observation can change an agent’s opinion and decision in a community. However, the original CODA model takes only a fixed number of agents’ action in the process of opinion updating when, in fact, the opinion of an individual in society will be influenced by a group of agents whose opinions are close to him/her. In this study, we develop a variant of the Hegelsman–Krause (HK) model that averages the opinions updated by the CODA rule after observing neighbors’ actions. Using this approach, we investigate the influence of social imitation on collective opinion evolution and decision making. We use a social network generated by the Barabasi–Albert(BA) algorithm to represent a social group. Simulation results show that agents’ opinions are more likely to be polarized after communicating with more neighbors. The impact of the level of biased-mindedness is also investigated. The results show that agents in a biased-minded social group could reach opinion stabilization faster than in a non-biased minded social group. In addition, the ratio of choice increases monotonically with the level of bias.

One-shot imitation learning is one of the crucial topics in robot learning with the pursuit of higher intelligence. Recently, conjugate task graph (CTG) network has been applied to generalize the imitation of homogeneous tasks based on a single video demonstration, where a standard optimization method is utilized to update the parameters of graph neural network. Nevertheless, when dealing with heterogeneous associated tasks, the standard algorithm needs to be improved to acquire higher learning accuracy. Given a set of heterogeneous tasks containing N sets of homogeneous tasks, we propose an N -Step Alternating Optimization in CTG (NSAO-CTG) to accomplish a superior learning, where each step incorporates the nodes and edges corresponding to a new set of homogeneous tasks. Furthermore, NSAO-CTG with a novel update rule for the node localizer and edge classifier (NSAO-CTG+) is proposed for execution based on the association information between tasks. Extensive experiments demonstrate the effectiveness of the proposed method in one-shot imitation learning of heterogeneous associated tasks.

Advances in machine learning, deep learning, and Artificial Intelligence(AI) allows people to exchange other people's faces and voices in videos to make it look like what they did or say whatever you want to say. These videos and photos are called “deepfake” and are getting more complicated every day and this has lawmakers worried. This technology uses machine learning technology to provide computers with real data about images, so that we can make forgeries. The creators of Deepfake use artificial intelligence and machine learning algorithms to mimic the work and characteristics of real humans. It differs from counterfeit traditional media because it is difficult to identify. As In the 2020 elections loomed, AI-generated deepfakes were hit the news cycle. DeepFakes threatens facial recognition and online content. This deception can be dangerous, because if used incorrectly, this technique can be abused. Fake video, voice, and audio clips can do enormous damage. This paper examines the algorithms used to generate deepfakes as well as the methods proposed to detect them. We go through the threats, research patterns, and future directions for deepfake technologies in detail. This research provides a detailed description of deep imitation technology and encourages the creation of new and more powerful methods to deal with increasingly severe deep imitation by studying the history of deep imitation.

Deep Imitation Learning requires a large number of expert demonstrations, which are not always easy to obtain, especially for complex tasks. A way to overcome this shortage of labels is through data augmentation. However, this cannot be easily applied to control tasks due to the sequential nature of the problem. In this work, we introduce a novel augmentation method which preserves the success of the augmented trajectories. To achieve this, we introduce a semi-supervised correction network that aims to correct distorted expert actions. To adequately test the abilities of the correction network, we develop an adversarial data augmented imitation architecture to train an imitation agent using synthetic experts. Additionally, we introduce a metric to measure diversity in trajectory datasets. Experiments show that our data augmentation strategy can improve accuracy and convergence time of adversarial imitation while preserving the diversity between the generated and real trajectories., Accepted in ICRA 2021

We present a hierarchical planning and control framework that enables an agent to perform various tasks and adapt to a new task flexibly. Rather than learning an individual policy for each particular task, the proposed framework, DISH, distills a hierarchical policy from a set of tasks by representation and reinforcement learning. The framework is based on the idea of latent variable models that represent high-dimensional observations using low-dimensional latent variables. The resulting policy consists of two levels of hierarchy: (i) a planning module that reasons a sequence of latent intentions that would lead to an optimistic future and (ii) a feedback control policy, shared across the tasks, that executes the inferred intention. Because the planning is performed in low-dimensional latent space, the learned policy can immediately be used to solve or adapt to new tasks without additional training. We demonstrate the proposed framework can learn compact representations (3- and 1-dimensional latent states and commands for a humanoid with 197- and 36-dimensional state features and actions) while solving a small number of imitation tasks, and the resulting policy is directly applicable to other types of tasks, i.e., navigation in cluttered environments. Video: https://youtu.be/HQsQysUWOhg, ICRA 2021, the first two authors contributed equally

The autonomous navigation in the crowded environment is a challenging task due to the sensor occlusion and the complex nature of the abstract social interactions. And yet, humans are capable of navigating in such complex environment. In this paper, we propose an effective navigation method that combines the learning-based and model-based methods in a way that a cost function that includes human imitation factor learned via deep learning is integrated into the dynamic window approach (DWA) [1]. The experiments conducted on simulations show that by training the robot to imitate the human trajectory, our navigation method is safer and more efficient than the state-of-the-art methods. Additionally, we successfully deployed a physical robot in an actual environment, and we validate that our navigation quality shares similar tendency with human in the path length, travel time, and the collision avoidance.

Complex organizational and technical systems in conflict are considered. The concept of a common undesirable goal for conflicting systems is introduced. Examples of its achievement are given. This state of systems and external environments is considered in which it is possible to achieve a common undesirable goal, despite the actions of decision makers using modern means of decision support. A formal model of conflicting systems is being built. The axioms of the functioning of systems are introduced, the implementation of which leads to the simultaneous achievement of undesirable goals by both systems. A possible algorithm for such an achievement is given. A simulation model is constructed based on the use of a modified Kaufman NK-automaton. The results of experiments are presented in which ensembles of modified Kaufman NK-automata participate, interacting with each other as conflicting systems. Various tactics of interactions of modified Kaufman NK-automata are considered. Conclusions are made about the results of experiments.

The creation of interactive VR applications from 3D scanned content usually includes a lot of manual and repetitive work. Our research aim is to develop a real-world, cross-domain, automatic, semantic segmentation system that enhances the creation of interactive VR applications. We trained segmentation agents in a superpoint growing environment that we extended with an expert function. This expert function solves the sparse reward signal problem of the previous approaches and enables to use a variant of imitation learning and deep reinforcement learning with dense feedback. Additionally, the function allows to calculate a performance metric for the degree of imitation for different segmentations. We trained our agents with 1182 scenes of the ScanNet data set. More specifically, we trained different neural network architectures with 1170 scenes and tested their performance with 12 scenes. Our intermediate results are promising such that our segmentation system might be able to assist the VR application development from 3D scanned content in near future.

In this work we analyse the use of malicious mimicry and cloning of darknet marketplaces and other ‘onion services’ as means for phishing, akin to traditional ‘typosquatting’ on the web. This phenomenon occurs due to the complex trust relationships in Tor’s onion services, and particularly the complex webs of trust enabled by darknet markets and similar services. To do so, we built a modular scraper tool to identify networks of maliciously cloned darknet marketplaces; in addition to other characteristics of onion services, in aggregate. The networks of phishing sites identified by this scraper were then subject to clustering and analysis to identify the method of phishing and the networks of ownership across these sites. We present a novel discovery mechanism for sites, means for clustering and analysis of onion service phishing and clone sites, and an analysis of their spectrum of sophistication.

Student ‘copying’ is often considered negatively as thoughts of plagiarism come to mind. Previously, we investigated the ways that instructors expect to use copying and imitation positively in their teaching. In this paper, we follow up that study by focusing on the student perspective and explore the ways in which students see copying and imitating as positive tools in learning to program (both at an introductory level and through more advanced learning of algorithms, etc.).In a qualitative research study, using semi-structured interviews, students were asked about how they use copying positively - their goals when they copy, how they go about it, how they view the experience of copying, and results beyond simply fulfilling their immediate goals.When comparing student results with the previous study, it was noted that there is some level of agreement between instructors and students about how copying can be useful for learning to program. There is also some degree of mismatch between instructor and student views. Students did not report imitating instructors’ approaches to learning and sometimes were unsure about whether they were supposed to copy the materials that they were given. This leads to some teaching suggestions in terms of instructors being more explicit in their attitude to this type of ‘legitimate’ copying and imitation.

The interactions between driver, vehicle and environment generate vehicle's behaviors, and the interactions of vehicle groups shape the traffic modes. Therefore, the conditionally or fully automated driving technologies should be developed and tested in the driver-vehicle-environment (DVE) system rather than being developed and tested individually. To build DVE system dynamics, firstly, we propose an architecture to cope with the complicated interactions in automated vehicles (AVs) and in mixed traffic situations. Then we summarize the driver behavior models and compare the differences of intelligence between human driver and automated vehicle. Finally, we summarize the feasible modeling approaches of DVE system into five categories. The primary distinctions are the modeling methods of human drivers' roles, which are realized by human driver per se, human driver's cognitive architecture, psychological motivation model, mechanism imitation, and specific mechanism transfer respectively. Taking the applications of human machine interface and AD strategy developments as examples, we analyze the benefits and drawbacks of these approaches.

The research objective is to develop a hybrid learning group dynamics subject with a Collaborative Problem Solving (CPS) model. The development step refers to the development formula proposed by Borg & Gall which is modified into potential and problem analysis, product development, product testing, product revision, product validation, and final product. The results of the study were in the form of three products, namely: (1) eight video lessons on group dynamics courses; (2) CPS syntax is phase 1 focusing attention, phase 2 project development, phase 3 imitation modification, phase 4 problem analysis, phase 5 problem solving, phase 6 report review, and phase 7 team assessment; and (3) one video implementation of hybrid learning in group dynamics courses with the CPS model.

For dynamic complex driving tasks of autonomous vehicle, traditional driving behavior learning methods still need to be improved in terms of learning accuracy and generalization performance. A convolutional neural network gaussian process regression (CNN-GPR) method for driving behavior learning is proposed to tackle these problems that full connection layers of end-to-end convolutional neural networks (CNN) have limited generalization ability and easily converge to local optimization. At the same time, the PilotNet-GPR algorithm based on the CNN-GPR method is designed. The gaussian process regression (GPR) method with global mapping capability and better generalization performance is used to improve fully connected layers of the end-to-end CNN in order to complete the mapping from features extracted to driving actions more efficiently for the proposed CNN-GPR method. In addition, the long short-term memory network (LSTM) is added into CNN-GPR method, and a convolutional long short-term memory network gaussian process regression (CNN-LSTM-GPR) method of driving behavior learning with time-sequential images is proposed in order to further improve the accuracy of driving behavior learning through time-sequential information. This method utilises the gaussian process regression method to improve structures of fully connected layers in the cascaded deep neural network (CNN-LSTM) in order to make a more efficient learning approximation between features extracted by the CNN-LSTM and driving actions. Verification experiments on the Apollo end-to-end dataset for autonomous driving show that the proposed CNN-GPR method can further improve the imitation accuracy for end-to-end driving behavior learning and promote generalization performance of learned models compared with the PilotNet method. compared with related end-to-end driving behavior learning methods based on single image under the same condition, the proposed CNN-LSTM-GPR method can make full use of time-sequential information of images, which leads to smaller imitation errors. Additionally, it can further enhance the learning accuracy and demonstrates a more satisfying generalization performance compared with the CNN-LSTM method.

This paper studies the competitive relationship between manufacturing companies and retailers in the recycling of waste electronic products. Based on the evolutionary game theory, the game model of both groups is constructed, and the replication dynamic equation is established. It analyzes the strategy choices of manufacturing companies and retailers in the recycling process, and uses simulation analysis to study the influencing factors of recycling strategy choices. The results show that the probability of the two groups choosing a cooperation strategy is positively related to the excess return and the penalty of breach of contract during cooperation; it is inversely related to the cost paid, the resources recovered from the other party during free-riding, and their own learning and imitation capabilities.

Nonindependent mate choice occurs when a female is influenced in her choice of a mate by her social environment. Mate-choice copying (MCC) is a typical example of nonindependent mate choice wherein mating decisions are influenced by observing the mating preferences of other individuals of the same species. It is an inheritable trait, and copiers and noncopiers exist in the same species. This means that MCC has both advantages and disadvantages. Furthermore, the advantages of MCC are confirmed, but the disadvantages are not. For this reason, we clarified the disadvantages of MCC and considered the nature of MCC. Since MCC causes the synchronization of mating preferences in a species, the diversity of the species decreases. Therefore, we used scenarios of different environmental changes to examine the disadvantages of MCC. In this work, we simulated MCC behavior using an agent-based model consisting of the imported MCC gene and female preferences influenced by the mate choices of other individuals as a meme. Only those agents with the MCC gene could imitate the mate choice of other females, and imitation of mate choice implied that a female imitates the gene trait of a male who is the most popular with her preference meme in our proposed model. We reproduced environmental changes in this model and explored the disadvantages of MCC. Experimental results revealed a disadvantage of MCC, and we discuss the trigger of MCC focusing on the experimental environment.

Motion control algorithms for quadruped robots undergo rapid development in recent years. Interactive quadruped robots have demonstrated they may positively enhance the effect of psychotherapy in the treatment of patients with cognitive impairment, which requires them to have more interactive capabilities than traditional quadruped robots. In this study, we focus on enabling interactive quadruped robots to imitate real animal motions extracted from videos, by which the design of robotic motion controllers can be simplified and the bionic degree and the interactive capabilities of the robots can be enhanced. The motion capture data, however, cannot be directly utilized by the motion controllers since the robots and the real animals differ in their respective body geometries, motion dynamics and the numbers of DOF. To address these differences, we propose two strategies for imitating two different kind of motions. For ordinary motions (head scratching, waving, etc.), we first apply a scaling method to motion captured data and then use an inverse kinematic algorithm for imitation. Furthermore, to minimize the error of motion trajectories between the real animals and the robots, we then transform motion trajectories into a nonlinear optimization problem. For walking motions, we first analyze a classical SLIP model-based walking control algorithm for quadruped robots, and then apply the parameters extracted from motion captured data to the walking control algorithm. Experiments based on an interactive quadruped robot we developed demonstrate that our proposed strategies have great potential in improving the imitation capability of robots on the motions of real animals.

End to end learning for autonomous navigation and driving has become a growing research trend in both industry and academia in recent years. Its promise is in treating the whole driving pipeline as the development of a deep neural network (DNN). Its Achilles’ heel is access to thousands of images required for training of the DNN. This paper comprehensively investigates the applicability of the deep transfer learning for the specific task of end to end learning of autonomous navigation. Five state of the art DNNs including ResNet, AlexNet, and Densenet are applied here for extracting features from images taken by the front-facing camera of a mobile robot. Extracted features have different information values as DNNs have different architectures and learning capabilities. These features are then processed by a multilayer fully connected neural network to estimate the robot angular velocity. Obtained results for different DNNs indicate that the transfer learning-based models show a promising performance for accurately estimating the angular velocity purely using visual information. According to obtained results, AlexNet-base model outperforms others in terms of the estimation accuracy and the performance consistency.

Fake portrait video generation techniques have been posing a new threat to the society with photorealistic deep fakes for political propaganda, celebrity imitation, forged evidences, and other identity related manipulations. Following these generation techniques, some detection approaches have also been proved useful due to their high classification accuracy. Nevertheless, almost no effort was spent to track down the source of deep fakes. We propose an approach not only to separate deep fakes from real videos, but also to discover the specific generative model behind a deep fake. Some pure deep learning based approaches try to classify deep fakes using CNNs where they actually learn the residuals of the generator. We believe that these residuals contain more information and we can reveal these manipulation artifacts by disentangling them with biological signals. Our key observation yields that the spatiotemporal patterns in biological signals can be conceived as a representative projection of residuals. To justify this observation, we extract PPG cells from real and fake videos and feed these to a state-of-the-art classification network for detecting the generative model per video. Our results indicate that our approach can detect fake videos with 97.29% accuracy, and the source model with 93.39% accuracy., Comment: To be published in the proceedings of 2020 IEEE/IAPR International Joint Conference on Biometrics (IJCB)

Games have played crucial role in advancing research in Artificial Intelligence and tracking its progress. In this article, a new proposal for game AI competition is presented. The goal is to create computer players which can learn and mimic the behavior of particular human players given access to their game records. We motivate usefulness of such an approach in various aspects, e.g., new ways of understanding what constitutes the human-like AI or how well it fits into the existing game production workflows. This competition may integrate many problems such as learning, representation, approximation and compression of AI, pattern recognition, knowledge extraction etc. This leads to multi-directional implications both on research and industry. In addition to the proposal, we include a short survey of the available game AI competitions.

In game AI research, most work aims at building a more powerful AI but few are reported on explaining AI’s behavior or revealing the characters of AI. Previous efforts in training a human-like AI via style learning implies that AI may behave with human characteristics. However, the early work treated human playstyle as a whole instead of identifying the difference among various AIs. In this paper, we focus on finding out manlike characters of individual AIs, and clustering AIs according to their characters. We propose a Neural Network based game AI imitator to imitate AIs’ behavior and find that some AIs are easier to imitate than others. Based on this observation we define the term imitability to describe the difficulty of imitation and cluster the AIs into two categories according to their imitability. Through statically analyzing, we find that AIs with lower imitability are generally farseeing with a global perspective while the other group are nearsighted and narrow-minded. The AIs hard to imitate also perform better when fighting with others. Upon the above semantic analysis of the clustering results, we conclude that the imitability can be used to identify AIs’ character.

Cavity filters are vital components of radio base stations and networks. After production, they need tuning, which has proven to be a difficult process to do manually and even more so to automate. Previous attempts to automate this process with Reinforcement Learning have failed to reach consistent performance on anything but the simplest filter models. In this paper, we build upon these results and aim to improve them. Multiple methods are tested and evaluated, including introducing a pre-processing step, tuning hyperparameters and dividing the problem into multiple sub-tasks. In particular, by using supervised Imitation Learning as an initial phase, a semi-realistic filter model with 13 tuning screws is tuned, fulfilling both insertion loss and return loss requirements. On this problem, this algorithm has a greater efficiency than any previously published results on Reinforcement Learning for Cavity filter tuning.

Bucket-filling is a repetitive task in earth-moving operations with wheel-loaders, which needs to be automated to enable efficient remote control and autonomous operation. Ideally, an automated bucket-filling solution should work for different machine-pile environments, with a minimum of manual retraining. It has been shown that for a given machine-pile environment, a time-delay neural network can efficiently fill the bucket after imitation-based learning from 100 examples by one expert operator. Can such a bucket-filling network be automatically adapted to different machine-pile environments without further imitation learning by optimization of a utility or reward function? This paper investigates the use of a deterministic actor-critic reinforcement learning algorithm for automatic adaptation of a neural network in a new pile environment. The algorithm is used to automatically adapt a bucket-filling network for medium coarse gravel to a cobble-gravel pile environment. The experiments presented are performed with a Volvo L180H wheel-loader in a real-world setting. We found that the bucket-weights in the novel pile environment can improve by five to ten percent within one hour of reinforcement learning with less than 40 bucket-filling trials. This result was obtained after investigating two different reward functions motivated by domain knowledge.

Consider a human who interacts with the physical world to autonomously learn in a task non-specific way through lifetime. It seems obvious that the fully autonomous learner does not have the luxury to have the mother to provide temporally dense state labels, but the context/state at every frame is beneficial (e.g., to generate attention for the next frame). How can we enable the learner to generate frame-wise contexts/states on the fly? Our past work on Developmental Network (DN-1) has shown that frame-wise state labels (e.g., stages within a phoneme) are useful to generate temporally sparse label (the type of the phoneme). However, such dense and sparse labels were handcrafted from a static data set, using human identified frame-wise equivalence. In this paper, we study a conceptually challenging problem — how to enable an autonomous learner to generate frame-wise states autonomously without human handcrafting dense labels at all. We propose that frame-wise muscle actions (e.g., producing a sound) are not only temporally dense and high-dimensional, but also natural as dense labels. However, it is unknown how a neural network can use such high-dimensional vectors as dense labels. In this work, we provide a model for this new issue and experiment with Developmental Network-2 (DN-2) for imitation of audio sequences. Our experimental results showed DN-2 can successfully emerge high-dimensional real-valued vector actions. These actions provide DN-2 with frame-wise temporal context information. This work corresponds to a key step toward our goal to enable the agent to fully autonomously generate frame-wise actions without human-provided dense labels and with only a few human-provided sparse labels.