Search

With an increasing number of people working with large music archives, advanced methods for automatic labeling and organization of music collections are required as these archives grow in size. Manual annotation and categorization is not feasible for massive collections of music. In the research domain of music information retrieval (MIR) a number of algorithms for the content-based description of music were developed, which perform the extraction of relevant features for the computation of similarity between pieces of music. This fundamental step enables a great range of applications for music retrieval and organization. With supervised machine learning, music can be classified into different kinds of categories, such as genres, artists or moods. Using unsupervised approaches such as the self-organizing map music can be clustered by similar style and visualized in a way that enables direct retrieval of similar music at a glance. In this chapter, we will review the most common audio feature extraction techniques, which serve as a basis for subsequent classification and clustering tasks. As an example, we will show how music is classified into a set of genres and how genre classification can be used for benchmarking. Moreover, the creation of the so-called "music maps" and their various visualizations is demonstrated, and an interactive application called "PlaySOM" is presented, with an interface which allows direct access to similar sounding pieces in a large music collection. Its mobile counterpart "PocketSOMPlayer" allows direct playback from a music map on a mobile device without having to browse lists. Both allow the convenient interactive creation of situation-based playlists. [ABSTRACT FROM AUTHOR]

We propose a generative statistical model for the classification of semi-structured multimedia documents. Its main originality is its ability to simultaneously take into account the structural and the content information present in a semi-structured document and also to cope with different types of content (text, image, etc.). We then present the results obtained on two sets of experiments: • One set concerns the filtering of pornographic Web pages • The second one concerns the thematic classification of Wikipedia documents. [ABSTRACT FROM AUTHOR]

Semantic labeling of large volumes of image and video archives is difficult, if not impossible, with the traditional methods due to the huge amount of human effort required for manual labeling used in a supervised setting. Recently, semi-supervised techniques which make use of annotated image and video collections are proposed as an alternative to reduce the human effort. In this direction, different techniques, which are mostly adapted from information retrieval literature, are applied to learn the unknown one-to-one associations between visual structures and semantic descriptions. When the links are learned, the range of application areas is wide including better retrieval and automatic annotation of images and videos, labeling of image regions as a way of large-scale object recognition and association of names with faces as a way of large-scale face recognition. In this chapter, after reviewing and discussing a variety of related studies, we present two methods in detail, namely, the so called "translation approach" which translates the visual structures to semantic descriptors using the idea of statistical machine translation techniques, and another approach which finds the densest component of a graph corresponding to the largest group of similar visual structures associated with a semantic description. [ABSTRACT FROM AUTHOR]

In comparison with the classic query-by-example paradigm, the "mental image search" paradigm lifts the strong assumption that the user has a relevant example at hand to start the search. In this chapter, we review different methods that implement this paradigm, originating from both the content-based image retrieval and the object recognition fields. In particular, we present two complementary methods. The first one allows the user to reach the target mental image by relevance feedback, using a Bayesian inference. The second one lets the user specify the mental image visual composition from an automatically generated visual thesaurus of segmented regions. In this scenario, the user formulates the query with an explicit representation of the image content, as opposed to the first scenario which accommodates an implicit representation. In terms of usage, we will show that the second approach is particularly suitable when the mental image has a well-defined visual composition. On the other hand, the Bayesian approach can handle more "semantic" queries, such as emotions for which the visual characterization is more implicit. [ABSTRACT FROM AUTHOR]

In recent years there has been a growing interest in improving all aspects of the interaction between humans and computers with the clear goal of achieving a natural interaction, similar to the way human-human interaction takes place. The most expressive way humans display emotions is through facial expressions. Humans detect and interpret faces and facial expressions in a scene with little or no effort. Still, development of an automated system that accomplishes this task is rather difficult. There are several related problems: detection of an image segment as a face, extraction of the facial expression information, and classification of the expression (e.g., in emotion categories). A system that performs these operations accurately and in real time would be a major step forward in achieving a human-like interaction between the man and machine. In this chapter, we present several machine learning algorithms applied to face analysis and stress the importance of learning the structure of Bayesian network classifiers when they are applied to face and facial expression analysis. [ABSTRACT FROM AUTHOR]

Bayesian methods are a class of statistical methods that have some appealing properties for solving problems in machine learning, particularly when the process being modelled has uncertain or random aspects. In this chapter we look at the mathematical and philosophical basis for Bayesian methods and how they relate to machine learning problems in multimedia. We also discuss the notion of decision theory, for making decisions under uncertainty, that is closely related to Bayesian methods. The numerical methods needed to implement Bayesian solutions are also discussed. Two specific applications of the Bayesian approach that are often used in machine learning - naïve Bayes and Bayesian networks - are then described in more detail. [ABSTRACT FROM AUTHOR]

Content-based image retrieval (CBIR) has attracted a lot of interest in recent years. When considering visual information retrieval in image databases, many difficulties arise. Learning is definitively considered as a very interesting issue to boost the efficiency of information retrieval systems. Different strategies, such as offline supervised learning or semi-supervised learning, have been proposed. Active learning methods have been considered with an increased interest in the statistical learning community. Initially developed in a classification framework, a lot of extensions are now proposed to handle multimedia applications. The purpose of this chapter is to present an overview of the online image retrieval systems based on supervised classification techniques. This chapter also provides algorithms in a statistical framework to extend active learning strategies for online content-based image retrieval. [ABSTRACT FROM AUTHOR]

When data objects that are the subject of analysis using machine learning techniques are described by a large number of features (i.e. the data are high dimension) it is often beneficial to reduce the dimension of the data. Dimension reduction can be beneficial not only for reasons of computational efficiency but also because it can improve the accuracy of the analysis. The set of techniques that can be employed for dimension reduction can be partitioned in two important ways; they can be separated into techniques that apply to supervised or unsupervised learning and into techniques that either entail feature selection or feature extraction. In this chapter an overview of dimension reduction techniques based on this organization is presented and the important techniques in each category are described. [ABSTRACT FROM AUTHOR]

Unsupervised learning is very important in the processing of multimedia content as clustering or partitioning of data in the absence of class labels is often a requirement. This chapter begins with a review of the classic clustering techniques of k-means clustering and hierarchical clustering. Modern advances in clustering are covered with an analysis of kernel-based clustering and spectral clustering. One of the most popular unsupervised learning techniques for processing multimedia content is the self-organizing map, so a review of self-organizing maps and variants is presented in this chapter. The absence of class labels in unsupervised learning makes the question of evaluation and cluster quality assessment more complicated than in supervised learning. So this chapter also includes a comprehensive analysis of cluster validity assessment techniques. [ABSTRACT FROM AUTHOR]

Supervised learning accounts for a lot of research activity in machine learning and many supervised learning techniques have found application in the processing of multimedia content. The defining characteristic of supervised learning is the availability of annotated training data. The name invokes the idea of a ‘supervisor' that instructs the learning system on the labels to associate with training examples. Typically these labels are class labels in classification problems. Supervised learning algorithms induce models from these training data and these models can be used to classify other unlabelled data. In this chapter we ground or analysis of supervised learning on the theory of risk minimization. We provide an overview of support vector machines and nearest neighbour classifiers~- probably the two most popular supervised learning techniques employed in multimedia research. [ABSTRACT FROM AUTHOR]

In this chapter we will introduce a new effective framework for learning an object detector. The main idea is to minimize the manual effort when learning a classifier and to combine the power of a discriminative classifier with the robustness of a generative model. Starting with motion detection an initial set of positive examples is obtained by analyzing the geometry (aspect ratio) of the motion blobs. Using these samples a discriminative classifier is trained using an online version of AdaBoost. In fact, applying this classifier nearly all objects are detected but there is a great number of false positives. Thus, we apply a generative classifier to verify the obtained detections and to decide if a detected patch represents the object of interest or not. As we have a huge amount of data (video stream) we can be very conservative and use only patches for (positive or negative) updates if we are very confident about our decision. Applying this update rules, an incrementally better classifier is obtained without any user interaction. Moreover, an already trained classifier can be retrained online and can therefore easily be adapted to a completely different scene. We demonstrate the framework on different scenarios including pedestrian and car detection. [ABSTRACT FROM AUTHOR]

This book presents biologically inspired walking machines (four- and six-legged walking machines) interacting with their real environmental stimuli as agent-environment interactions. Different reactive behaviors of animals were investigated for the behavior design of the walking machine(s). On the one hand, the obstacle avoidance behavior, in analogy to the obstacle avoidance and escape behavior of scorpions and cockroaches, was implemented in the walking machines as a negative tropism. On the other hand, the sound tropism which mimics prey capture behavior of spiders is represented as a positive tropism. It was simulated on the four-legged walking machine. [ABSTRACT FROM AUTHOR]

Where Chap. 2 investigated the biologically inspired perception-action systems, this chapter focuses on applying the principles of the biological domain to create artificial perception-action systems. First, several preprocessing units of different types of sensory signals are presented. They are used to filter and recognize the corresponding sensory signals and they can be described as perception parts. Second, the neural control of the four- and six-legged walking machines, which generates and controls the locomotion of the machines, is described. Third, the combination of the neural preprocessing and the neural control is explained. It gives rise to the ability of controlling reactive behaviors such as obstacle avoidance and sound tropism. Finally, both behavior controls are merged under a so-called behavior fusion controller by applying a sensor fusion technique to give a versatile perception-action system. [ABSTRACT FROM AUTHOR]

In order to test the capabilities of the artificial perception-action systems, several experiments were carried out. First, the signal processing networks were tested with the simulated signals and the real sensor signals. Afterwards the physical sensors, the neural preprocessing and the neural control were all together implemented on the physical walking machine(s) to demonstrate different reactive behaviors. [ABSTRACT FROM AUTHOR]

This chapter describes the development of the physical components that lead to the artificial perception-action systems. It begins with the descriptions of different physical sensors which are used to sense environmental information, followed by the details of the walking machines simulated in a physical simulation environment as well as the robots we have built. [ABSTRACT FROM AUTHOR]

This chapter presents methods and tools which are to be used throughout this book. It starts with a short introduction to a biological neuron together with an artificial neuron which is followed by the comparison of network structures between feedforward and recurrent neural networks. Then the discrete-time dynamical properties of the single neuron with a recurrent connection are described. Finally, artificial evolution is presented as a tool to develop and optimize neural structures as well as the strength of synapses. [ABSTRACT FROM AUTHOR]

Most of this book is devoted to creating and demonstrating so-called artificial perception-action systems inspired by biological sensing systems (perception) and animal behavior (action). Thus this chapter attempts to provide the biological background for understanding the approach taken in this book. It begins with a short introduction to some of the necessary principles of animal behavior. Then it concentrates on the obstacle avoidance and escape behavior of a scorpion and a cockroach, and continues with the prey capture behavior of a spider. Here, attention is given to the biological sensing systems used to trigger the described behaviors. Furthermore, different morphologies of walking animals are presented as inspiration for the design of walking machine platforms. Finally, a biologically inspired locomotion control, called a "central pattern generator" (CPG), is also discussed. This concept is later employed to generate the rhythmic leg movements of the machines. [ABSTRACT FROM AUTHOR]

Research in the domain of biologically inspired walking machines has been ongoing for over 20 years [59, 166, 190, 199, 207]. Most of it has focused on the construction of such machines [34, 47, 216, 223], on a dynamic gait control [43, 117, 201] and on the generation of an advanced locomotion control [30, 56, 104, 120], for instance on rough terrain [5, 66, 102, 180, 192]. In general, these walking machines were solely designed for the purpose of motion without responding to environmental stimuli. However, from this research area, only a few works have presented physical walking machines reacting to an environmental stimulus using different approaches [6, 36, 72, 95]. On the one hand, this shows that less attention has been paid to walking machines performing reactive behaviors. On the other hand, such complex systems can serve as a methodology for the study of embodied systems consisting of sensors and actuators for explicit agent-environment interactions. [ABSTRACT FROM AUTHOR]

When an application needs a fusion mechanism, the developer has to solve an essential problem: the construction of the appropriate model. This corresponds to (i) the selection of an aggregation operator and (ii) the determination of its parameters. This process should take into account several factors. Some of them are highlighted here. [ABSTRACT FROM AUTHOR]

This chapter reviews some of the existing tools for evaluating aggregation methods and their parameters. We focus on some indices for fuzzy measures (Shapley and Banzhaf), an interaction index, and the degree of disjunction. Other methods exist. The influence function and other tools such as grosserror sensitivity and local-shift sensitivity developed in robust statistics (see Section 2.2.6) are of interest here. The tools permit us to have some knowledge on how a particular estimator might behave when embedded in a real system. In particular, we have seen that the influence function of the arithmetic mean is unbounded while that of the median is bounded. [ABSTRACT FROM AUTHOR]