Search

This chapter presents a vision-based face and gesture recognition system for human-robot interaction. By using subspace method, face and predefined hand poses are classified from the three largest skin-like regions that are segmented using YIQ color representation system. In the subspace method we consider separate eigenspaces for each class or pose. Face is recognized using pose specific subspace method and gesture is recognized using the rulebased approach whenever the combinations of three skin-like regions at a particular image frame satisfy a predefined condition. These gesture commands are sent to robot through TCP/IP wireless network for human-robot interaction. The effectiveness of this method has been demonstrated by interacting with an entertainment robot named AIBO and a humanoid robot Robovie. Human-robot symbiotic systems have been studied extensively in recent years, considering that robots will play an important role in the future welfare society [Ueno, 2001]. The use of intelligent robots encourages the view of the machine as a partner in communication rather than as a tool. In the near future, robots will interact closely with a group of humans in their everyday environment in the field of entertainment, recreation, health-care, nursing, etc. In human-human interaction, multiple communication modals such as speech, gestures and body movements are frequently used. The standard input methods, such as text input via the keyboard and pointer/location information from a mouse, do not provide a natural, intuitive interaction between humans and robots. Therefore, it is essential to create models for natural and intuitive communication between humans and robots. Furthermore, for intuitive gesture-based interaction between human and robot, the robot should understand the meaning of gesture with respect to society and culture. The ability to understand hand gestures will improve the naturalness and efficiency of human interaction with robot, and allow the user to communicate in complex tasks without using tedious sets of detailed instructions. This interactive system uses robot eye’s cameras or CCD cameras to identify humans and recognize their gestures based on face and hand poses. Vision-based face recognition systems have three major components: image processing or extracting important clues (face pose and position), tracking the facial features (related position or motion of face and hand poses), and face recognition. Vision-based face recognition system varies along a number of

The aim of this paper is to present the design and development of a safe and reliable autonomous cat toy for the purposes of entertaining pet cats when human interaction is not feasible. The proposed Autonomous Feline Entertainment Robot (AFER), an autonomous cat toy, is comprised of a table-mounted base, and a robotic arm with two linkages. The AFER Version 1(vl) has a cat-toy affixed to the end of a linkage as the end effector, while the AFER Version 2 (v2) has a passive joint with a string to which a cat toy end effector is attached. The AFER v1 has 2 Degrees of Freedom (DOF), while the AFER v2 has 3DOF, and both are actuated by servomotors interfaced with an Arduino microcontroller. In order to enable maximum functionality, the AFER is designed with multiple movement patterns or modes for play, as well as the ability to change the specific end effector of the toy. Position control was used for the motion and trajectory planning when developing the movement patterns. The interaction between AFER and two kittens has been evaluated in an experimental scenario, and the results show satisfactory and safe interactions. The AFER can be used as a model for studying effective animal-robot interactions and pet entertainment.

The application of humanoid robots has been popular in the field of healthcare and education. Amid the COVID-19 pandemic, most people are isolated at home. People in isolation are constantly looking for new entertainment activities. Therefore, entertaining robots would become people’s best partner in home isolation. In this article, the possibility of an entertainment robot for isolated people from the perspective of human factors is studied. NAO is an autonomous, programmable humanoid robot developed by the French Aldebaran Robot Company. Because of its cute appearance and ability to talk to people, we use NAO for this study. When isolated at home, parents might need to work at home and not have time to spend time with their children. A robot for children has entertaining capabilities such as playing soccer and dancing to teach soccer knowledge to children. Experimental results show that NAO can learn to search and kick a ball and dance to entertain children at home in the COVID-19 pandemic.

International audience; When we perform a collaborative task with another human, we are able to tell, to a certain extent, how things are going and more precisely if things are going well or not. This knowledge allows us to adapt our behavior. Therefore, we think it is desirable to provide robots with means to measure in real-time the Quality of the Interaction with their human partners. To make this possible, we propose a model and a set of metrics targeting the evaluation of the QoI in collaborative tasks through the measure of the human engagement and the online task effectiveness. These model and metrics have been implemented and tested within the high-level controller of an entertainment robot deployed in a mall. The first results show significant differences in the computed QoI when in interaction with a fully compliant human, a confused human and a non-cooperative one.

Robots for communication are developed extensively with an emphasis on sympathy. This study deals with the growth of character and the control of its operation. The child has time to be alone with the nature of his/her robot friend. That child can interact with other people&rsquo, s emotional expressions through a robot. Step by step, the robot character will grow as the child grows. Through design studies, qualitative processes such as customer experience audit, eye tracking, mental model diagrams, and semantic differences have been executed for the results. The participatory behavior research approach through user travel is mapped from the user&rsquo, s lead to the evidence-based design. This research considers how synthetic characteristics can be applied to the physical growth of robot toys through the product design process. With the development of robot toy &ldquo, Buddy&rdquo, two variations on the robot were made to achieve recognizable growth. (1) one-dimensional height scaling and (2) facial expression including the distance between two eyes on the screen. Observations represented children&rsquo, s reactions when "Buddy" was released with the children. As an independent synthetic character, the robot was recognized by children who had the designed function. Robots for training may require more experimentation.

Abstract This paper proposes a humanoid robot performance system for performing in public places, such as an event, exhibition, or street performance. The system of modular structures can be moved easily, and can be played by a module or a combination of modules. The system developed with open source-based software and hardware is easy to adapt and improve. The robot performance control program for controlling robots, displays, audios, videos,and performance instruments was developed using the open source language, Processing. The performance instrumentswere developed using the open source hardware, Arduino. The contents of the robot performance were composed of scene-specific image, background audio, computer graphics, and videos. For their control and synchronization, the performance control program communicates with the humanoid robots and the performance instruments. In addition,performance accessories required to represent the performance concepts are produced by 3D modeling and printing.In a public place, the robot performance is performed with the theme of celebrating a Halloween day. Keywords : Humanoids , Robot-Dance Entertainment-Robot Robot-Performance Robot-show

The automatic playing robot of saxophone is the robot that can control the reed pressing force by the artificial mouth system, control the blowing air pressure by the special pressure control valve and operate the fingering system using sheet music data just like a human player. Because the existing system uses desktop PC and PCI bass interface boards, it has problems of transportation and mobility at the situation of attending the concert with human players. Therefore, in this paper, compactification of the system is considered by using a note-PC as the controllers and using USB nterface of D/A and DIO for control devices. Furthermore, it is necessary for the robot to play with human players of a big band that it can use more sophisticated musical expressions such as sub-tones and over-tones besides attacks, envelope control and high level vibration techniques. Therefore improvements for more sophisticated performance abilities are proposed.

Navigation is the main issue for autonomous mobile robot due to its mobility in an unstructured environment. The autonomous object tracking and following robot has been applied in many places such as transport robot in industry and hospital, and as an entertainment robot. This kind of image processing based navigation requires more resources for computational time, however microcontroller currently applied to a robot has limited memory. Therefore, effective image processing from a vision sensor and obstacle avoidances from distance sensors need to be processed efficiently. The application of neural network can be an alternative to get a faster trajectory generation. This paper proposes a simple image processing and combines image processing result with distance information to the obstacles from distance sensors. The combination is conducted by the neural network to get the effective control input for robot motion in navigating through its assigned environment. The robot is deployed in three different environmental setting to show the effectiveness of the proposed method. The experimental results show that the robot can navigate itself effectively within reasonable time periods.

In 1999 Sony released the AlBO Entertainment Robot, selling more than 150,000 units worldwide until 2006. By 2014, Sony had stopped offering upgrades and maintenance for the product, and owners were faced with the fact their pet-like robots would “die”. Some shrines and temples in Japan hold ningyo kuyō or mass funerals for dolls and other toys. At the suggestion of a small Japanese tech-repair company called A-Fun, one temple began offering a Buddhist funeral ceremony for AIBOs. Approximately 700 AIBOs have so far received a funeral service. This paper surveys A-Fun‘s maintenance services for old AIBOs, the AIBO funerals, and Sony's new 2018 AIBO release, in the cross-disciplinary context of human-machine relations in Japan and elsewhere. Drawing on the author's interviews with key actors, it articulates links between philosophy and neuroscience to explain tendencies toward zoomorphism in robot design. Perceiving presence (sonzai kan) and sensibility (kansei) in objects is a culturally contingent phenomenon. Whereas ways of conceiving the partly animate are largely absent from Western philosophy, in the case of AIBO ownership in Japan there is a reverential mindfulness of the technology's inherent contradictions.

International audience; In this paper, a system to determine the emotion of a group of people via facial expression analysis is proposed for the Waseda Entertainment Robots. General models and standard methods for emotion definition and recognition are briefly described, as well as strategies for computing the group global emotion, knowing the individual emotions of group members. This work is based on Ekman's extended “Big Six” emotional model, popular in Computer Science and Affective Computing. Emotion recognition via facial expression analysis is performed with a cloud-computing based solution, using Microsoft Azure Cognitive services. First, the performances of both the Face API to detect faces, and Emotion API, to compute emotion via face expression analysis, are tested. After that, a solution to compute the emotion of a group of people has been implemented and its performances compared to human perceptions. This work presents concepts and strategies which can be generalized for applications within the scope of assistive robotics and, more broadly, affective computing, wherever it will be necessary to determine the emotion of a group of people

In recent years, there have been a number of popular robotics competitions whose intent is to advance the state of research by comparing embodied entries against one another in real time. The IEEE Humanoid application challenge is intended to broaden these by allowing more open-ended entries, with a general theme within which entrants are challenged to create the most effective application involving a humanoid robot. This year’s theme was Robot Magic, and this paper describes our first-place winning entry in the 2017 competition, running on a ROBOTIS OP2 humanoid robot. We describe the overall agent design and contributions to perception, learning, control, and representation, which together support a robust live robot magic performance.

We took a social robot Pepper to a shopping mall for one day to see what kind of initial responses it draws from people. We observed that the robot was quickly surrounded by children when there were others-especially adults-interacting with it. The children seemed to especially enjoy the activity-related applications, such as tickling the robot or giving a high-five. Adults were interested in hearing about useful applications and tended to talk to the robot as if it were any machine capable of speech recognition. These observations will help to design more interactive and entertaining applications for shopping mall robots.

In this paper, we propose a position estimation method of an entertainment robot in a performance stage. A position information is needed to produce contents of an entertainment robot performance. First, the performance stage is realized by a CD (cadmium sulfide) sensor matrix with a constant distance. Then the proposed position estimation uses ON/OFF information of a CD sensor in a performance stage. We confirmed that the position of the robot is detected with an maximum 4cm in position evaluation experiments. The robot was traveling to the final target position in the walking experiment with 75cm and 120cm path plan. Ultimately, the effectiveness of the proposed estimation is assessed by experimental results of a robot in performance stage. Also, in the proposed system installed by a robot performance contents, there is no necessity to mount a position estimation device on a robot; therefore an advantage of our system is that an entertainment robot commercialized by robot vendor can be utilized in our performance stage directly.

Unlike the market slowdown of industrial robots, service & entertainment robots have been highly regarded by most robotics reseach and market research agencies. In this study we developed a music playing robot (which can also work as a service robot) for public performance. The research is mainly focused on the mechanical and electrical control of piano-playing robot, the exploration of correlations among music theory, rhythm and piano keys, and eventually the research on playing skill of keyboard instrument. The piano-playing robot is capable of control linear motor, servo-motor and pneumatic devices in accordance with the notes and rhythm in order to drive the mechanical structure to proper positions for pressing the keys and generating music. The devices used for this robot are mainly crucial components produced by HIWIN Technology Corp. The design of robotic hand is based on the direction of anthropomorphic hand such that five fingers will be used for playing piano. The finger actuations include actions of finger rotation, finger pressing, and finger lifting; time required for these 3 stages must meet the requirement of rhythm. The purpose of entertainment robot can be achieved by playing electric piano with robotic hand, and we hope this research can contribute to the development of domestic entertainment music playing robots.

In this paper, we propose an emotion model with growth functions for robots. Many emotion models for robots have been developed using Neural Networks (NN), which focus on the functions of emotion recognition, control, and expression. One problem that affects these emotion models for robots is the development of a “simplified” emotion generation algorithm. Users readily lose interest in “simple” systems. Most models have attempted to generate complex emotional expressions, whereas no previous studies have considered the “growth of a robot.” Therefore, we propose a growth model for emotions based on changes in the network structure of a self-organizing map. We also applied a multilayer perceptron NN to generate more sophisticated expressions of emotion using growth functions. This model generated a similar behavior to the concept of affective change described in genetic psychology. Our results showed that this emotion model was more suitable for producing a robot with growth functions based on a psychological model.

Two high-speed video cameras are successfully used to detect the motion of a flying shuttlecock of badminton. The shuttlecock detection system is applied to badminton robots that play badminton fully autonomously. The detection system measures the three dimensional position and velocity of a flying shuttlecock, and predicts the position where the shuttlecock falls to the ground. The badminton robot moves quickly to the position where the shuttle-cock falls to, and hits the shuttlecock back into the opponent’s side of the court. In the game of badminton, there is a large audience, and some of them move behind a flying shuttlecock, which are a kind of background noise and makes it difficult to detect the motion of the shuttlecock. The present study demonstrates that such noises can be eliminated by the method of stereo imaging with two high-speed cameras.

This thesis shows a control algorithm for coping with a ball batting task for an entertainment robot. The robot is a three jointed robot with a redundant degree of freedom and its name is "Doggy". Doggy because of its dog-like costume. Design, mechanics and electronics were developed by us. DC-motors control the tooth belt driven joints, resulting in elasticities between the motor and link. Redundancy and elasticity have to be taken into account by our developed controller and are demanding control tasks. In this thesis we show the structure of the ball playing robot and how this structure can be described as a model. We distinguish two models: One model that includes a flexible bearing, the other does not. Both models are calibrated using the toolkit Sparse Least Squares on Manifolds (SLOM) - i.e. the parameters for the model are determined. Both calibrated models are compared to measurements of the real system. The model with the flexible bearing is used to implement a state estimator - based on a Kalman filter - on a microcontroller. This ensures real time estimation of the robot states. The estimated states are also compared with the measurements and are assessed. The estimated states represent the measurements well. In the core of this work we develop a Task Level Optimal Controller (TLOC), a model-predictive optimal controller based on the principles of a Linear Quadratic Regulator (LQR). We aim to play a ball back to an opponent precisely. We show how this task of playing a ball at a desired time with a desired velocity at a desired position can be embedded into the LQR principle. We use cost functions for the task description. In simulations, we show the functionality of the control concept, which consists of a linear part (on a microcontroller) and a nonlinear part (PC software). The linear part uses feedback gains which are calculated by the nonlinear part. The concept of the ball batting controller with precalculated feedback gains is evaluated on the robot. This shows successful batting motions. The entertainment aspect has been tested on the Open Campus Day at the University of Bremen and is summarized here shortly. Likewise, a jointly developed audience interaction by recognition of distinctive sounds is summarized herein. In this thesis we answer the question, if it is possible to define a rebound task for our robot within a controller and show the necessary steps for this.

The aim of this study was to determine how the violinist robot could produce a good quality of violin sounds. We began our study with the basic physics of producing sound with a violin. We found three parameters that influenced the quality of the sound produced by the violin; the bowing force, the bowing velocity and the sounding point. In particular, the bowing force was found to be the most important parameter in producing good sounds. Furthermore, to produce such sounds, a same amount of the bowing force must be applied on the contact point between a bow and a string. However, it is hard to keep a same amount of the bowing force on the contact point due to inherent characteristics of a bow. Thus, we primarily focused on the bowing force by considering bowing a string as a spring-mass system. Then, we devised a passive damper device to offset variables in the spring-mass system that may result in changing the bowing force on the contact point. We then validated our methodology with the violinist robot, a human-like torso robot.

By recognizing sensory information, through touch, vision, or voice sensory modalities, a robot can interact with people in a more intelligent manner. In human–robot interaction (HRI), emotion recognition has been a popular research topic in recent years. This paper proposes a method for emotion recognition, using a speech signal to recognize several basic human emotional states, for application in an entertainment robot. The proposed method uses voice signal processing and classification. Firstly, end-point detection and frame setting are accomplished in the pre-processing stage. Then, the statistical features of the energy contour are computed. Fisher’s linear discriminant analysis (FLDA) is used to enhance the recognition rate. In the final stage, a support vector machine (SVM) is used to complete the emotional state classification. In order to determine the effectiveness of emotional HRI, an embedded system was constructed and integrated with a self-built entertainment robot. The experimental results ...

The objectives of this study are to apply unexpectedness to the behavioral design of an entertainment robot and to evaluate the impression and satisfaction provided by the robot. Participants(N=44) observed four robot behaviors, which are distinguished by type of expectancy disconfirmation (positive disconfirmation, negative disconfirmation, simply confirmation, unexpected disconfirmation), and evaluated each behavior in terms of novelty, enjoyment, satisfaction, performance, and reliability. Participants perceived the unexpected disconfirmation behavior to be more novel and enjoyable such that they preferred this type over the other types. On the other hand, they evaluated the positive disconfirmation behavior as more intelligent and reliable than the other types. These findings will provide an essential basis for designing the behavior of an entertainment robot with the use of unexpectedness.

Music has long been used to strengthen bonds between humans. In our research, we develop musical coplayer robots with the hope that music may improve human–robot symbiosis as well. In this paper, we underline the importance of non-verbal, visual communication for ensemble synchronization at the start, during and end of a piece. We propose three cues for interplayer communication, and present a thereminplaying, singing robot that can detect them and adapt its play to a human flutist. Experiments with two naive flutists suggest that the system can recognize naturally occurring flutist gestures without requiring specialized user training. In addition, we show how the use of audio-visual aggregation can allow a robot to adapt to tempo changes quickly.

One of the most important issues in developing an entertainment robot is human-robot interaction, in which the robot is expected to learn new behaviors specified by the user. In this article we present an imitation-based mechanism to support robot learning, and use evolutionary computing to learn new behavior sequences. We also propose several advanced techniques at the task level and the computational level to evolve complex sequences. To evaluate our approach, we use it to evolve different behaviors for a humanoid robot. The results show the promise of our approach.

Deliberate control of an entertainment robot presents a special problem in balancing the requirement for intentional behavior with the existing mechanisms for autonomous action selection. It is proposed that the intentional biasing of activation in lower-level reactive behaviors is the proper mechanism for realizing such deliberative action. In addition, it is suggested that directed intentional bias can result in goal-oriented behavior without subsuming the underlying action selection used to generate natural behavior. This objective is realized through a structure called the intentional bus. The intentional bus serves as the interface between deliberative and reactive control by realizing high-level goals through the modulation of intentional signals sent to the reactive layer. A deliberative architecture that uses the intentional bus to realize planned behavior is described. In addition, it is shown how the intentional bus framework can be expanded to support the serialization of planned behavior by shifting from direct intentional influence for plan execution to attentional triggering of a learned action sequence. Finally, an implementation of this architecture, developed and tested on Sony’s humanoid robot QRIO, is described.

Robot Assisted Activity (RAA) and Robot Assisted Therapy (RAT) were put into practice and shown to be effective at the rehabilitation ward and elderly persons' nursing home using commercially available entertainment robots. Animal Assisted Activity (AAA) and Animal Assisted Therapy (AAT) are well known and modeled on this study. Remote control of the robot that is impossible in the case of an animal showed very effective in the rehabilitation practice and Robot Assisted Rehabilitation (RAR) was proposed. However there are several problems for the remote controlled RAR to receive wide recognition as the candidate of the complementary medicine. One of the problems is that the robot operator is needed besides intervener or therapist. To solve this problem a small and simple operating terminal with www browser was developed aiming it can be used by the intervener. In this paper, preliminary studies of RAA/RAT are reported laying stress on rehabilitation application and improvement of the robot remote control system.

In this paper we discuss two techniques to reduce the size of the acoustic model while maintaining or improving the accuracy of the recognition engine. The first technique, demiphone modeling, tries to reduce the redundancy existing in a context dependent state-clustered Hidden Markov Model (HMM). Three-state demiphones optimally designed from the triphone decision tree are introduced to drastically reduce the phone space of the acoustic model and to improve system accuracy. The second redundancy elimination technique is a more classical approach based on parameter tying. Similar vectors of variances in each HMM cluster are tied together to reduce the number of parameters. The closeness between the vectors of variances is measured using a Vector Quantizer (VQ) to maintain the information provided by the variances parameters. The paper also reports speech recognition improvements using assignment of variable number Gaussians per cluster and gender-based HMMs. The main motivation behind these techniques is to improve the acoustic model and at the same time lower its memory usage. These techniques may help in reducing memory and improving accuracy of an embedded Large Vocabulary Continuous Speech Recognition (LVCSR) application.

In order to build a heterogeneous multi-robot system that can be regarded as a primitive prototype of a future symbiotic autonomous human-robot system, this paper presents a knowledge model-based heterogeneous multi-robot system implemented by a software platform. With using frame-based knowledge representation, a knowledge model is constructed to describe the features of heterogeneous robots as well as their behaviors according to human requests. The required knowledge for constructing a heterogeneous multi-robot system can be therefore integrated in a single model and shared by robots and users. Based on the knowledge model, the heterogeneous multi-robot system is defined in the Software Platform for Agents and Knowledge Management (SPAK) by use of XML format. With the use of SPAK, the cooperative operation of heterogeneous multi-robot system can be flexibly carried out. The proposed system not only can integrate heterogeneous robots and various techniques for robots, but also can automatically perform human-robot interaction and plan robot behaviors taking into account different intelligence of robots corresponding to human requests. In this paper, an actual heterogeneous multi-robot system comprised by humanoid robots (Robovie, PINO), mobile robot (Scout) and entertainment robot dog (AIBO) is built and the effectiveness of the proposed system is verified by experiment.

The application of social robots as motivational tools and companions in education is increasingly being explored from a theoretical and practical point of view. In this paper, we examine the social robot NAO as a Persuasive Educational and Entertainment Robot (PEER) and present findings from a case study on the use of NAO to support learning environments in Danish primary schools.In the case study we focus on the children’s practice of articulation and embodied interaction with NAO and investigate the role of NAO as a ‘tool’, ‘social actor’ or ‘simulating medium’ in the learning designs. We examine whether this categorization is static or dynamic, i. e. develops and changes over the course of the interaction and explore how this relates to and affects the student’s motivation to engage in the NAO-supported learning activities.