Search

Affective computing has recently become an important area in human-computer interaction research. Techniques have been developed to enable computers to understand human affects or emotions, in order to predict human intention more precisely and provide better service to user to enhance user experience. In this paper, we investigate into the detection of human attention level as a useful from of human affect, which could be influential in intelligent e-learning applications. We adopt ubiquitous hardware available in most computer systems, namely, webcam and mouse. Information from multiple input modalities is fused together for effective human attention detection. We invite human subjects to carry out experiments in reading articles when being imposed upon different kinds of distraction to induce them into different levels of attention. Machine-learning techniques are applied to identify useful features to recognize human attention level by building up user-independent models. Our result indicate performance improvement with multimodal inputs from webcam and mouse over that of a single device. We believe that our work has revealed and intersting affective computing direction with potential application in e-learning.

This paper presents Personalized Affect Detection with Minimal Annotation (PADMA), a user-dependent approach for identifying affective states from spontaneous facial expressions without the need for expert annotation. The conventional approach relies on the use of key frames in recorded affect sequences and requires an expert observer to identify and annotate the frames. It is susceptible to user variability and accommodating individual differences is difficult. The alternative is a user-dependent approach, but it would be prohibitively expensive to collect and annotate data for each user. PADMA uses a novel Association-based Multiple Instance Learning (AMIL) method, which learns a personal facial affect model through expression frequency analysis, and does not need expert input or frame-based annotation. PADMA involves a training/calibration phase in which the user watches short video segments and reports the affect that best describes his/her overall feeling throughout the segment. The most indicative facial gestures are identified and extracted from the facial response video, and the association between gesture and affect labels is determined by the distribution of the gesture over all reported affects. Hence both the geometric deformation and distribution of key facial gestures are specially adapted for each user. We show results that demonstrate the feasibility, effectiveness and extensibility of our approach.

Human-centered computing is rapidly becoming a major research direction as new developments in sensor technology make it increasingly feasible to obtain signals from human beings. At the same time, the pervasiveness of computing devices is also encouraging more research in human---computer interaction, especially in the direction of personalized and adaptive user interfaces. Among the various research issues, affective computing, or the ability of computers to understand and react according to what a user "feels," has been gaining in importance. In order to recognize the human affect (feeling), computers rely on the analysis of signal inputs captured by a multitude of means. This paper proposes the use of human physiological signals as a new form of modality in determining human affects, in a non-intrusive manner. The principle of non-invasiveness is very important, since it imposes no extra burden on the user, which improves user accessibility and encourages user adoption. This goal is realized via the physiological mouse, as a first step toward the support of affective computing. The conventional mouse is converted with a small optical component for capturing user photoplethysmographic (PPG) signal. With the PPG signal, it is possible to compute and derive human physiological signals. A prototype of the physiological mouse was built and raw PPG readings measured. The accuracy of the approach was evaluated through empirical studies to determine human physiological signals from the mouse PPG data. Finally, pilot experiments to correlate human physiological signals with various modes of human---computer interaction, namely gaming and video watching, were conducted. The trend in physiological signals could be used as feedback to the computer system which in turn adapts to the needs or the mood of the user, for instance change the volume and the light intensity when watching a video or playing a game based on current user emotion. The authors argue that this research will provide a new dimension for multimodal affective computing research, and the pilot study has already shed some light toward this research goal.

This paper presents an investigation into user intention prediction in two common web-based tasks: crowdsourcing annotation and web search, based on human-mouse interaction information. User experience is gaining importance within the research area of human-centered computing, and is particularly useful for complex, multi-step tasks. To enhance user experience, the computer should be intelligent enough to be able to predict the user intention. For instance, an intelligent agent might be able to anticipate when the user is about to press a button, and helpfully enlarge or highlight it in advance. In this paper, we propose two prediction models on user intention: a classical model that considers only historical mouse activity sequence, and a multimodal model that utilizes mouse interaction signals as well as features extracted from mouse trajectory and clicking events. We evaluate our models and find that they achieve reasonable accuracy. Our preliminary results indicate that we can dynamically learn a multimodal model that can effectively predict a user’s next activity from historical activity sequence and mouse interaction signals.

This article presents the design and development of i*CATch, a construction kit for physical and wearable computing that was designed to be scalable, plug-and-play, and to provide support for iterative and exploratory learning. It consists of a standardized construction interface that can be adapted for a wide range of soft textiles or electronic boards, a set of functional components, and an easy-to-use hybrid text-graphical integrated development environment. The objective was to design an easily usable, manufacturable and extensible construction kit that can be used in a wide range of teaching tasks for a wide variety of student demographic profiles. We present detailed specifications of our construction kit and explain some of the major design decisions. Experiences in using the kit in multiple teaching environments, ranging from elementary school to postgraduate, demonstrate that the design objectives have been achieved.

Systems that adapt to changes in human needs automatically are useful, built upon advancements in human-computer interaction research. In this paper, we investigate the problem of how well the eye movement of a user when reading an article can predict the level of reading comprehension, which could be exploited in intelligent adaptive e-learning systems. We characterize the eye movement pattern in the form of eye gaze signal. We invite human subjects in reading articles of different difficulty levels being induced to different comprehension levels. Machine-learning techniques are applied to identify useful features to recognize when readers are experiencing difficulties in understanding their reading material. Finally, a detection model that can identify different levels of user comprehension is built. We achieve a performance improvement of over 30% above the baseline, translating over 50% reduction in detection error.

Affective computing in human-computer interaction research enables computers to understand human affects or emotions to provide better service. In this paper, we investigate the detection of human attention useful in intelligent e-learning applications. Our principle is to use only ubiquitous hardware available in most computer systems, namely, webcam and mouse. Information from multiple modalities is fused together for effective human attention detection. We invite human subjects to carry out experiments in reading articles being subjected to different kinds of distraction to induce different attention levels. Machine-learning techniques are applied to identify useful features to recognize human attention level. Our results indicate improved performance with multimodal inputs, suggesting an interesting affective computing direction.

Many studies have attested to the efficacy of integrating innovative methods of teaching computing and engineering, especially for introductory students and at the K-12 level. As an example, robots have been successfully used to teach a wide range of subjects, from introductory programming to artificial intelligence. As a discipline, wearable computing is seen to be practical, yet futuristic and exciting, and it captures the attention and interest of people who might not otherwise be drawn to technology. Recent developments in this field have also raised the possibility of moving wearable computing construction within the reach of hobbyists and novices. However, there still exist substantial obstacles toward the adoption of wearable computing into education. This paper presents a framework with the objective of facilitating the integration of wearable computing into outreach and introductory computer science and engineering education. We also present a comprehensive evaluation of our platform, including a comparison with the current practice of sewing-based wearable computing.

The rapid development of Internet technologies in recent decades has imposed a heavy information burden on users. This has led to the popularity of recommender systems, which provide advice to users about items they may like to examine. Collaborative Filtering (CF) is the most promising technique in recommender systems, providing personalized recommendations to users based on their previously expressed preferences and those of other similar users. This paper introduces a CF framework based on Fuzzy Association Rules and Multiple-level Similarity (FARAMS). FARAMS extended existing techniques by using fuzzy association rule mining, and takes advantage of product similarities in taxonomies to address data sparseness and nontransitive associations. Experimental results show that FARAMS improves prediction quality, as compared to similar approaches.

The Wireless Online Tutoring System (WOTS) is a web‐based, synchronous groupware system that has been developed to facilitate student collaboration in group projects. It supports interactive and collaborative structured‐diagramming and is equipped with group awareness features such as remote cursors, locking, and text‐based instant messaging. Users can access WOTS from both personal computers and handheld devices, allowing it to be used in wireless classrooms or distributed environments. Tests on student software design projects validate the ability of WOTS to support student collaboration in group projects, and show that its architecture is scalable and reliable. Its performance is expected to improve as more powerful handheld devices and wireless technologies come on to the market.

Measuring the structural similarity among XML documents is the task of finding their semantic correspondence and is fundamental to many web-based applications. While there exist several methods to address the problem, the data mining approach seems to be a novel, interesting and promising one. It explores the idea of extracting paths from XML documents, encoding them as sequences and finding the maximal frequent sequences using the sequential pattern mining algorithms. In view of the deficiencies encountered by ignoring the hierarchical information in encoding the paths for mining, a new sequential pattern mining scheme for XML document similarity computation is proposed in this paper. It makes use of a preorder tree representation (PTR) to encode the XML tree’s paths so that both the semantics of the elements and the hierarchical structure of the document can be taken into account when computing the structural similarity among documents. In addition, it proposes a postprocessing step to reuse the mined patterns to estimate the similarity of unmatched elements so that another metric to qualify the similarity between XML documents can be introduced. Encouraging experimental results were obtained and reported.

Human-centered computing is rapidly becoming a major research direction in human-computer interaction research. Among the various research issues, we believe that affective computing, or the ability of computers to react according to what a user feels, is very important. In order to recognize the human affect (feeling), one can rely on the analysis of signal inputs captured by a multitude of means. In this paper, we propose to make use of human physiological signals as a new form of modality in determining human affects, in a non-intrusive manner. This is achieved via the physiological mouse, as a first step towards affective computing. We augment the mouse with a small optical component for capturing user photoplethysmographic (PPG) signal. With the PPG signal, we are able to compute and derive human physiological signals. We built a prototype of the physiological mouse and measured raw PPG readings. We performed experiments to study the accuracy of our approach in determining human physiological signals from the mouse PPG data. We believe that our research will provide a new dimension for multimodal affective computing research.

We present i*Chameleon, a configurable and extensible multimodal platform for developing highly interactive applications. The platform leverages a principled and comprehensive development cycle to systematically capture the multimodal interaction artifact. Importantly, by introducing MVC architectural pattern, it enforces the concept of separation-of-concerns to enable cross collaboration among device engineers, programmers, modality designers and interaction designers who are collectively working on different aspects of human computer interaction and programming. Collectively, the development efforts are combined, integrated and compiled by the i*Chameleon kernel to derive the multimodal interactive application. The i*Chameleon platform sets itself apart from previous works in that it advocates the need to engineer a software development approach that leverages a MVC software architectural pattern to promote ease of software development through division of responsibilities among engineers and HCI designers. To validate the usefulness of i*Chameleon, we describe several application case examples to demonstrate the ease of developing multimodal applications through systematic integration of the design models as described above.

Multimodality inputs are becoming increasingly popular in supporting pervasive applications, due to the demand for highly responsive and intuitive human control interfaces beyond the traditional keyboard and mouse. However, the heterogeneous nature of novel multimodal input devices and the tight coupling between input devices and applications complicate their deployment, rendering their dynamic integration to the intended applications rather difficult. i∗Chameleon exploits device abstraction in a web services-based framework to alleviate these problems. Developers can dynamically register new devices with the i∗Chameleon framework. They can also map specific device inputs to keyboard and mouse events efficiently. A number of input modalities such as tangible devices, speech, and finger gestures have been implemented to validate the feasibility of the i∗Chameleon framework in supporting multimodal input for pervasive applications.

i*Chameleon is a multimodal interaction framework that enables programmers to readily prototype and test new interactive devices or interaction modes. It allows users to customize their own desktop environment for interaction beyond the usual KVM devices, which would be particularly useful for users with difficulty using the keyboard and mouse, or for systems deployed in specialized environments. This is made possible with the engineering of an interaction framework that distills the complexity of control processing to a set of semantically-rich modal controls that are discoverable, composable and adaptable. The framework can also be used for developing new applications with multimodal interactions, for example, distributed applications in collaborative environments or robot control.

There has been much recent work in wearable computing that is directed at democratization of the field, to make it more accessible to the general public and more easily used by the hobbyist user. As the field becomes more diversified, there has also been a shift away from the highly specialized functionality of earlier applications towards aesthetics, creativity, design and self-expression, as well as a push towards using wearable computing as an outreach tool to broaden interest and exposure in engineering and computing. This paper presents the design and development of the i*CATch wearable computing framework, which was developed specifically for children and novices to the field. The i*CATch framework is based upon a bus-based architecture, and is more scalable than the current alternatives. It consists of a set of plug-and-play components, a construction platform with a standardized interface, and an easy-to-use hybrid text-graphical integrated development environment. We will also present results of the evaluation of the i*CATch framework in real teaching environments.

The field of wearable computing and e-textiles has recently attracted much interest from the research and general community. Recent developments in this field raises the possibility of e-textile construction kits for hobbyists and novices alike. The unique nature of wearable computing and e-textiles also gives it a lot of potential as an educational computing topic, as it allows students to exercise their creativity and imagination while learning about computing and technology. However, there are numerous difficulties involved in deploying existing technology in an educational environment. Current state of the art technology and techniques are not yet robust or reliable enough to stand up to the demands of educational computing, and they require a high level of skill from the user. In this paper, we present the TeeBoard, a constructive platform for e-textiles and wearable computing that is designed specifically to "lower the floor" for the integration of e-textiles into educational computing.

To address the unique demands and challenges of educational computing, various kinds of environments, including graphics-rich and textual environments, have been proposed for use in introductory courses to provide students with a rich and interesting learning environment. In our experience, students in Grade 7 and younger respond best to the graphics environments while senior high school students prefer a conventional textual programming environment. Clearly, this leaves a gap at Grade 11-13, with students often on the one hand finding the graphics-based environments too limited and on the other finding the textual environments too difficult. In this paper, we propose a text-enhanced graphical programming environment which is innovative and interactive, and designed for junior high students with no programming experience. This environment allows students to design their own creative stories or programs. They build their programs using drag-and-drop iconic blocks, but unlike other, similar icon-based programming languages, they are also presented with the syntax of the actual program they are constructing in real-time. Once a particular icon block has been dropped in the programming area, the syntax statements corresponding to that block is immediately generated and presented to the user. The environment also allows them to modify the code without any limitations. Our results show that our textual-graphical hybrid environment has a positive impact on the learning experience of the students.

This paper presents a design for a platform of collaborative robots and electronic devices. The platform design is based on the similarity between the type, functionality and characteristics of the robot or device. We categorize the participating devices by functionality rather than by architecture, therefore making it easy to support new robots or devices with similar functions but different architectures. This approach also allows users to develop, implement and port applications quickly and easily. We demonstrate the efficacy and correctness of our platform through a variety of robotic applications ranging from research to teaching.

Real-time collaborative design of complex objects across great distances has been made much easier through the World Wide Web. A number of issues, however, need to be resolved. These include the sharing and exchange of information on the designed objects, the coordination and control of the collaborative design activities, and the management of the different design versions. This paper explores these three issues and describes a prototype system that resolves the issues in the context of the collaborative design of complex geometrical objects.