Search

Visually impaired people are able to access the shape and texture of objects and recognize them by the sense of touch of hands. However, visual information such as color and pattern of clothing cannot be accessed. Therefore, it is difficult for visually impaired people to coordinate clothes without the assistance of a sighted person. In this study, we aim to develop a system to support visually impaired people to choose clothing. In this paper, we describe a method for converting visual information acquired from clothing into verbal representation using Deep Neural Networks (DNNs). In the experiment, we have developed a caption generating model using our dataset. As a result of try the computer experiment to generate caption for 100 images, we have observed subjectively acceptable captions for about 80% of the results.

For example, when sighted scholars study mathematics and physics etcetera, they need to access visual information, e.g., graphs and pictures. Furthermore, sighted people can express their own ideas and opinions visually. On the other hand, blind people can access visual information if it is expressed tactilely, but find it difficult to express their ideas and opinions visually. We are therefore developing a computer-aided system enabling blind people to draw their own figures on their own. This system consists of a matrix braille display to edit computer line drawings. The matrix braille display enables the blind to feel a tactile graphic during editing. After explaining two input methods for elementary plane shapes, we discuss two methods for scrolling tactile graphics to make the matrix braille display large enough to show tactile graphics in sufficient detail. We then show experimental results for using input and scrolling, and conclude with discussion on the usability of input and scrolling.

This manuscript describes a robot interaction for the driving assistance system of an Ultra-Compact Electric Vehicle (UCEV). Fun-to-drive and safety are important for improving the commercial value of UCEV. To improve fun-to-drive and safety, the improvement of the driving skills is important. However, the driving assistance system of an ordinary vehicle only considers the objective driving evaluation. Therefore, we propose an interactive driving assistance system that considers the relation between the subjective as well as the objective driving evaluation. Furthermore, we install a communicating robot within a UCEV to interact with human beings in real time. As a first step, we propose a driving evaluation system by applying a simplified fuzzy inference, and an interaction timing estimation method by applying a spiking neural network. Through an off-line simulation experiment, we verify the effectiveness of our proposal that is able to generate a robot utterances content as well as estimate reasonable timing.

This paper describes a presentation robot system that interacts with a large-group in a classroom. We are developing a presentation robot to give a lecture at the university. However, it is difficult to apply the robot interactions of previous researches to a large group. Furthermore, to continue the robot presentation for a long time, the interaction with a robot should be initiated not only by the robot but also by the participants. Therefore, our research target is the development of a presentation robot system for a large group, with robot interaction initiated by the participants. To interact with the participants, we developed the presentation robot system using a PC application. As the hardware, we use a humanoid robot PALRO named Masaru, and PCs for each participant. As the software, we designed a communication protocol for performing the lecture smoothly and steadily. For interacting with the participants, we made an answer button to answer the question from the robot and a query button for voluntary response from the participants. As an experiment, we performed a formal lecture using Masaru for one class. The result shows that the participants indicate their intentions actively by using the application. Furthermore, Masaru’s lecture involved the participants through interaction with the application, and the interest of the participants was increased by the robot presentation with interaction.

We have been developed P-CUBE2 which is a tangible programing tool intended to be suitable for beginners including visual impairments. This paper describes the system improvement and evaluation of P-CUBE2 using RFID system. P-CUBE2 enables users to make a program by an easy operation of set blocks on the mat. Since, we knew that visually impaired children felt difficulty on a process of transferring program to a controlled object, we adopt the Bluetooth module for improving the program transfer procedure easier for their.

We developed a tangible programming education tool “P-CUBE2” to aim at learning benefits of subroutine such as to create a function once and then reuse it. The target user of this tool are visual impairments and inexperienced persons who are not familiar with PC operation. We introduced the function mat, utterance function blocks and HIRAGANA (Japanese character) blocks so that user can learn subroutine concept. Users can create the function of utterance by combining and placing these blocks which on the function mat. The created function of utterance can be called on the main mat. By these operation, the user can control a robot which outputs sound as a controlled object. In this research, we introduce the system configuration of the P-CUBE2 and report the result of experiment for evaluation of the tool operability.

This paper describes the perceptual system based on invariant concept and the action system to aquire a better perception based on the perceiving-acting cycle to grasp an unknown object for a service robot. A service robot should take action in order to fulfill specific tasks in real time, even in an unknown environment. To work a service robot in an unknown environment, we focus on the perception system and the action system based on perceiving-acting cycle concept in ecological psychology. For the perception system, we focus on a direct perception concept. We have proposed the invariant detection based on the direct perception concept. To calculate the invariant information, we consider the relation with the principal moment of inertia of a detected object. For the action system, we focus on a perceiving-acting cycle concept. We have proposed the prediction method to acquire the better perception using the inertia tensor of a detected object. We have performed the experiments in an unknown environment which assumes to grasp an object. We discuss the cyclic system between perception and action for a robot.

An intelligent robot should take action in order to fulfill specific tasks in real time, even in a unknown environment. A robot perception to work in unknown environment is an important element. A recent robot has high performance sensors which can obtain a significant amount of data. However, it is difficult to recognize an accurate environmental situation in real time. On the other hand, human obtains a significant amount of data from sensory organ. However, they can perceive the suitable situation and take suitable action promptly. At that time, the accurate object size, posture and shape are not important if a person decides to grasp an object. There is a direct perception concept in ecological psychology. A direct perception insists that relevant information for action is perceived directly without integrating some physical information such as size, posture and so on. Furthermore, the concept of perceiving-acting cycle insists on the importance of perception and action for direct perception. In this paper, we discuss the concept of direct perception based on ecological psychology.

This paper discusses about the Multi-Layer Situation Map to estimate the relationship between the objects and the storage places in cleanup task. In recent years, a service robot has to estimate, decide and act automatically in various situations to provide services to human beings. Now, we are developing a robot integration system that estimates the storage place where a robot should carry the objects. An interaction robot talks a cleanup service, if a storage place is decided only one place. On the other hand, an interaction robot has to communicate with human about cleanup service in fuzzy situations. For an interaction robot talks suitable contents according to the fuzzy situations, we propose an estimation system which is called a Multi-Layer Situation Map, and express the relationship between the object and the storage place. The multi-layer situation map can move each component according to multi-layer spring model when an environmental situation is changed. We verify the multi-layer situation map is able to express the relationship of the components in the cleanup service situation.

This paper discusses the Multi-Layer Situation Map to estimate the relationship between the objects and the storage places for robot interaction. An interaction robot is required the many rules to take suitable action. But, It is difficult to provide rules based on all situations. Now, we focus on a cleanup service by a robot, and are developing an interaction robot which estimates the storage place where a robot should carry objects. In this system, an interaction robot has to know the relationship of the objects and the storage places in dynamically changing situation. Therefore, we propose the Multi-Layer Situation Map, and express the relationship between the object and the storage place. The multi-layer situation map can move each component using a spring model. To verify the effectivity of the multi-layer situation map, we have performed the simulation experiment. We discuss about the dynamically situation change when the same objects are exist on the table.

We developed a card-type programming tool "Pro-Tan" as an education tool for beginners in programming. This tool was designed to be usable without any instruction on its operations. In the experiment, we did not tell the users how to use any of the tools. The subjects were instructed to create a program by themselves. We found problems of Pro-Tan's design in the experiment. In this paper, we report the system configuration of Pro-Tan and problem of Pro-Tan's design.

We developed P-CUBE as a tangible programming education tool for visually impairments or inexperienced persons in PC operation. Users are able to control a mobile robot simply by positioning blocks on a mat. The fundamental programming concepts taught by P-CUBE consist of three elements: sequences, branches and loops. These systems uses radio frequency identification (RFID) systems to identify the position of blocks or cards on the mat. Blocks utilize RFID tags alone. P-CUBE is designed to operate via tactile information for visually impaired.

When sighted people, for example, study mathematics and physics, they need not only verbal information, but also non-verbal information such as graphs, pictures and so on. Blind people are able to receive non-verbal information if the information is expressed by tactile graphics, however, it is quite difficult for the blind to draw figures by themselves. Using refreshable braille displays is one of the possible methods when blind people draw figures by themselves. So, we discuss assistive systems with a refreshable braille display for blind people to draw figures by themselves. In this paper, we introduce two input methods of elementary plane shapes, which is called object input methods. Furthermore, two scrolling methods are also introduced, which are needed to show the drawing area where the user wants to touch. This paper describes experiments with blind participants, in these experiments, we have tested which object input method is efficient for blind users, and which scrolling method is easy to use. The results of the experiments are shown in this paper.

As a method of teaching fundamental programming concepts to visually impaired persons and novice programmers, we developed the P-CUBE algorithm education tool, with which users are able to control a mobile robot simply by positioning wooden blocks on a mat. The fundamental programming concepts taught by P-CUBE consist of three elements: sequences, branches and loops. The P-CUBE system consists of a mobile robot, a program mat, programming blocks, and a personal computer (PC). The programming blocks utilize radio frequency identification (RFID) tags alone, and thus require no precision equipment such as microcomputers. Furthermore, since P-CUBE is designed to be operated via tactile information, it can be utilized by visually impaired persons. In this paper, we report on the P-CUBE system configuration and a programming workshop held for visuarlly impaired persons. We then propose P-CUBE device improvements formulated through subjective assessments obtained from workshop participants.

We developed P-CUBE, which is an education tool for programming mobile robots, and confirmed the system operation. Elementary school and junior high school students have recently begun to learn programming using PCs. Students who are taught how to create programs are often required to use an interface of a personal computer. This method keeps visually impaired and younger children from learning programming. Furthermore, the cost associated with PCs is often prohibitive. In an attempt to address these problems, we have developed P-CUBE, which is a new programming education tool. P-CUBE consists of cubic blocks and a program mat. The user creates a program by laying out surface gap blocks on the program mat based on the structure of the algorithm. RFID readers are arranged in the programming mat, and the information of the RFID tags in the blocks is read in order to specify the location of each block. P-CUBE is considered to be nearly indestructible because of its simple structure and is expected to be used as an educational toy for students.

This study aims to develop a device to support the use of PCs by the elderly and visually impaired. The force display touch pad is a pointing device that presents the position of a window frame or buttons on a PC screen to the user using a force feedback function. We consider this device as a mockup, therefore we will perform demonstrations of the device, and then modify it according to surveyed user needs. Of the many users of PCs, visually impaired individuals typically make use of a screen reader, which is software that translates information on the computer screen into auditory information. In this paper, we evaluate the operability and accessibility of the force display touch pad with added auditory output.

This paper proposes a logical framework that visualizes worker's system awareness. Generally, this awareness is difficult to compile in a educational book, and is seldom communicated in verbally. To show broad utility of system awareness, we need to show operator's conceptual structure of system operating. We firstly carry out operation analysis of graph drawing task using a spreadsheet software. Based on it, we propose a logical representation of operator's system awareness using Formal Concept Analysis. Using this framework, we analyze the correspondence relations of difference of using functions for each operator and implication from formal concept. As a result, this representation shows the operators' implicit rules which become the selection criteria of system functions.

This paper proposes a novel strategy of designing play equipments. The strategy introduces two loose constraints as a guideline for designers. The first constraint is “describing unit of play action chain_ ⟨ 𝑂 , 𝑆 , 𝑉 ⟩ based on Barthes_ semiology, and the second is the infomorphism between designer, play equipment, and players based on channel theory. We provide detailed explanation of the strategy through an example of a designing process of playing environment where the players usage of the play equipment cannot be foreseen.