Search

Music can convey and evoke powerful emotions. But it is very challenging to recognize the music emotions accurately by computational models. The difficulty of the problem can exponentially increase when the music segments delivery multiple and complex emotions. This paper proposes a novel quantum convolutional neural network (QCNN) to learn music emotions. Inheriting the distinguished abstraction ability from deep learning, QCNN automatically extracts the music features that benefit emotion classification. The main contribution of this paper is that we utilize measurement postulate to simulate the human emotion awareness in music appreciation. Statistical experiments on the standard dataset shows that QCNN outperforms the classical algorithms as well as the state-of-the-art in the task of music emotion classification. Moreover, we provide demonstration experiment to explain the good performance of the proposed technique from the perspective of physics and psychology.

Recent studies have demonstrated that a well designed deep convolutional neural network (CNN) model achieves competitive performances on detecting the presence of secret message in digital images, compared with the classical rich model based steganalysis. In this paper, we propose to investigate a category of very deep CNN model–the deep residual network (DRN), for steganalysis. DRN is suitable for steganalysis from two aspects. For the first, the DRN model usually contains a large number of network layers, which proves to be effective to capture the complex statistics of digital images. For the second, DRN's residual learning (ResL) method actively strengthens the signal coming from secret messages, which is extremely beneficial for the discrimination between cover images and stego images. Comprehensive experiments on standard dataset show that the DRN model achieves very low detection error rates for the state of arts steganographic algorithms. It also outperforms the classical rich model method and several recently proposed CNN based methods.

Image and video classification tasks often suffer from the problem of high-dimensional feature space. How to discover the meaningful, low-dimensional representations of such high-order, high-dimensional observations remains a fundamental challenge. In this paper, we present a unified framework for tensor based dimensionality reduction including a new tensor distance (TD) metric and a novel multilinear globality preserving embedding (MGPE) strategy. Different with the traditional Euclidean distance, which is constrained by orthogonality assumption, TD measures the distance between data points by considering the relationships among different coordinates of high-order data. To preserve the natural tensor structure in low-dimensional space, MGPE directly works on the high-order form of input data and employs an iterative strategy to learn the transformation matrices. To provide faithful global representation for datasets, MGPE intends to preserve the distances between all pairs of data points. According to the proposed TD metric and MGPE strategy, we further derive two algorithms dubbed tensor distance based multilinear multidimensional scaling (TD-MMDS) and tensor distance based multilinear isometric embedding (TD-MIE). TD-MMDS finds the transformation matrices by keeping the TDs between all pairs of input data in the embedded space, while TD-MIE intends to preserve all pairwise distances calculated according to TDs along shortest paths in the neighborhood graph. By integrating tensor distance into tensor based embedding, TD-MMDS and TD-MIE perform tensor based dimensionality reduction through the whole learning procedure and achieve obvious performance improvement on various standard datasets.

Affective recognition is an important and challenging task for video content analysis. Affective information in videos is closely related to the viewer's feelings and emotions. Thus, video affective content analysis has a great potential value. However, most of the previous methods are focused on how to effectively extract features from videos for affective analysis. There are several issues are worth to be investigated. For example, what information is used to express emotions in videos, and which information is useful to affect audiences' emotions. Taking into account these issues, in this paper, we proposed a new video affective content analysis method based on protagonist information via Convolutional Neural Network CNN. The proposed method is evaluated on the largest video emotion dataset and compared with some previous work. The experimental results show that our proposed affective analysis method based on protagonist information achieves best performance in emotion classification and prediction.

Example learning-based super-resolution (SR) methods are effective to generate a high-resolution (HR) image from a single low-resolution (LR) input. And these SR methods have shown a great potential for many practical applications. Unfortunately, most of popular example learning-based approaches extract features from limited training images. These training images are insufficient for super resolution task. Our work is to transfer some supplemental information from other domains. Therefore, in this paper, a new algorithm Transfer Learning based on A+ (TLA) is proposed for image super-resolution task. First, we transfer supplemental information from other datasets to construct a new dictionary. Then, in sample selection, more training samples are supplemented to the basic training samples. In experiments, we seek to explore what types of images can provide more appropriate information for super-resolution task. Experimental results indicate that our approach is superior to A+ when transferring images containing similar content with original data.

Noise exists universally in multimedia data, especially in Internet era. For example, tags from web users are often incomplete, arbitrary, and low relevant with the visual information. Intuitively, noise in the dataset is harmful to learning tasks, which implies that huge volumes of image tags from social media can't be utilized directly. To collect the reliable training dataset, labor-intensive manual labeling and various learning based outlier detection techniques are widely used. This paper intends to discuss whether such kind of preprocessing is always needed. We focus on a very normal case in image classification that the available dataset includes a large amount of images weakly related to any target classes. We use deep models as the platform and design a series of experiments to compare the semi-supervised learning performance with/without weakly related unlabeled data. Fortunately, we validate that weakly related data is not always harmful, which is an encouraging finding for research on web image learning.

Steganography is the art of hiding information in ways that prevents the detection of hidden messages. Image steganography, which hides messages into a cover image for secret transmission, attracts increasing attention in social media era. Currently, most works focus on designing message embedding algorithms to avoid the stego images being distinguished from normal ones via visual observation or statistical analysis. This paper aims to make the detection of the stego images more difficult by selecting the suitable cover images. We propose a new measure to evaluate the hiding ability of the cover image based on Fisher Information Matrix and Gaussian Mixture Model. Experiments on standard dataset validate that the cover image with good hiding ability can improve the performance of various steganography algorithms obviously. Moreover, the proposed measure provides a statistical explanation of the existing cover image selection techniques and shows better performance against steganalysis.

The large-scale video data on the web contain a lot of semantics, which are an important part of semantic web. Video descriptors can usually represent somewhat the semantics. Thus, they play a very important role in web multimedia content analysis, such as Scale-invariant feature transform SIFT feature. In this paper, we proposed a new video descriptor, called a temporal-compress and shorter SIFTTC-S-SIFT which can efficiently and effectively represent the semantics of web videos. By omitting the least discriminability orientation in three stages of standard SIFT on every representative frame, the dimensions of the shorter SIFT are reduced from 128-dimension to 96-dimension to save space storage. Then, the SIFT can be compressed by tracing SIFT features on video temporal domain, which highly compress the quantity of local features to reduce visual redundancy, and keep basically the robustness and discrimination. Experimental results show our method can yield comparable accuracy and compact storage size.

Cerebellar ataxia is a progressive neuro-degenerative disease that has multiple genetic versions, each with a characteristic pattern of anatomical degeneration that yields distinctive motor and cognitive problems. Studying this pattern of degeneration can help with the diagnosis of disease subtypes, evaluation of disease stage, and treatment planning. In this work, we propose a learning framework using MR image data for discriminating a set of cerebellar ataxia types and predicting a disease related functional score. We address the difficulty in analyzing high-dimensional image data with limited training subjects by: 1) training weak classifiers/regressors on a set of image subdomains separately, and combining the weak classifier/regressor outputs to make the decision; 2) perturbing the image subdomain to increase the training samples; 3) using a deep learning technique called the stacked auto-encoder to develop highly representative feature vectors of the input data. Experiments show that our approach can reliably classify between one of four categories (healthy control and three types of ataxia), and predict the functional staging score for ataxia.

Water reflection, a typical imperfect reflection symmetry problem, plays an important role in image content analysis. Existing techniques of symmetry recognition, however, cannot recognize water reflection images correctly because of the complex and various distortions caused by the water wave. Hence, we propose a novel water reflection recognition technique to solve the problem. First, we construct a novel feature space composed of motion blur invariant moments in low-frequency curvelet space and of curvelet coefficients in high-frequency curvelet space. Second, we propose an efficient algorithm including two sub-algorithms: low-frequency reflection cost minimization and high-frequency curvelet coefficients discrimination to classify water reflection images and to determine the reflection axis. Through experimenting on authentic images in a series of tasks, the proposed techniques prove effective and reliable in classifying water reflection images and detecting the reflection axis, as well as in retrieving images with water reflection.

Water reflection, a kind of typical imperfect reflection symmetry problem, plays an important role in image content analysis. However, existing techniques of symmetry recognition cannot recognize water reflection images correctly because of the complex and various distortions caused by water wave. To address this difficulty, we construct a novel feature space which is composed of motion blur invariant moments. Moreover, we propose an efficient detection algorithm to determine the reflection axis in images with water reflection. By experimenting on real image dataset with different tasks, the proposed techniques demonstrate impressive results in the water reflection image classification, the reflection axis detection, and the retrieval of the images with water reflection.

Visual popout refers to searches in which one feature distinguishes a target and time to detection is relatively unaffected by number of nontargets. Rapid detection can make it seem like information about a target is obtained immediately and passively. We sought to investigate the acquisition of target-related information in the earliest moments of a search. In Experiment 1 a color-singleton display was masked after a 16.7-900 ms exposure, and the task was to click the position that the target occupied. We translated average response-distance-error into entropy, an information-theoretic measure of uncertainty. We could thus ask how much increasing exposure reduces uncertainty about a target's position. In the first 200ms, uncertainty declined linearly, then plateauing (reflecting perceptual and motor precision). We fit two continuous time models to the responses of each participant, one characterizing a passive strategy of estimating an item's position successively from received signals, the other, a more active strategy of querying whether a signal is present in successively smaller portions of an image. The active model fit significantly better. In Experiment 2, we scrutinized individual differences in rate of information gain. We replicated the results of Experiment 1, accompanied by color change detection to measure working memory capacity. Rate of information gain was significantly correlated with memory capacity (r=0.55). This suggests that rate of information acquisition may underwrite broad cognitive functioning as WM capacity is known to do. Additional experiments related these results to reaction times in detection search and other forms of popout. Overall, the results suggest a new way to characterize visual search at the algorithmic level. We discuss how this algorithmic theory is consistent with psychological theories such as 'guided search,' and how it can be extended to account for inefficient conjunction search and search asymmetries. Meeting abstract presented at VSS 2015. Language: en

Intuitively, extrapolating object trajectories should make visual tracking more accurate. This has proven to be true in many contexts that involve tracking a single item. But surprisingly, when tracking multiple identical items in what is known as "multiple object tracking," observers often appear to ignore direction of motion, relying instead on basic spatial memory. We investigated potential reasons for this behavior through probabilistic models that were endowed with perceptual limitations in the range of typical human observers, including noisy spatial perception. When we compared a model that weights its extrapolations relative to other sources of information about object position, and one that does not extrapolate at all, we found no reliable difference in performance, belying the intuition that extrapolation always benefits tracking. In follow-up experiments we found this to be true for a variety of models that weight observations and predictions in different ways; in some cases we even observed worse performance for models that use extrapolations compared to a model that does not at all. Ultimately, the best performing models either did not extrapolate, or extrapolated very conservatively, relying heavily on observations. These results illustrate the difficulty and attendant hazards of using noisy inputs to extrapolate the trajectories of multiple objects simultaneously in situations with targets and featurally confusable nontargets.

Construction of saliency map in multimedia data is useful for applications in multimedia like object segmentation, quality assessment, and object recognition. In this paper, we propose a novel saliency map model called Gabor & Curvelets based Saliency Map (GCSMP) relying on 2D Gabor and Curvelet transforms. Compared with the traditional model based on DOG and wavelets, our model takes advantage of Garbor transforms's spatial localization and Curvelet transform's edge and directional information. We also discuss the influence of center bias and object detectors in our model. Empirical validations on standard dataset demonstrate the effectiveness of the proposed technique.

Image recognition with incomplete data is a well-known hard problem in multimedia content analysis. This paper proposes a novel deep learning technique called semiconducting bilinear deep belief networks (SBDBN) by referencing human's visual cortex and intelligent perception. Inheriting from deep models, SBDBN simulates the laminar structure of human's cerebral cortex and the neural loop in human's visual areas. To address the special difficulties of image recognition with incomplete data, we design a novel second-order deep architecture with semiconducting restricted boltzmann machines. Moreover, two peaks activation of human's perception is implemented by three learning stages of semiconducting bilinear discriminant initialization, greedy layer-wise reconstruction, and global fine-tuning. Owing to exploiting the embedding information according to the reliable features rather than any completion of missing features, the proposed SBDBN has demonstrated outstanding recognition ability on two standard datasets and one constructed dataset, comparing with both incomplete image recognition techniques and existing deep learning models.

This paper proposes a novel framework to explore the motion trajectories of dynamic texture videos via manifold learning. First, we partition the high-dimensional dataset into a set of data clusters. Second, we construct intra-cluster neighborhood graphs using visible neighbors based on the individual character of each data cluster. Third, we construct the inter-cluster graph by analyzing the interrelation among these isolated data clusters. Then we compute the shortest paths according to the whole graph of the dataset. Finally, we embed the dataset to a unique low-dimensional space, trying to maintain the pairwise distances of all pairs of high-dimensional data points. Experiments on standard dynamic texture database show that the proposed framework can represent motion characters of dynamic textures very well. Index Terms—Dimensionality reduction, dynamic textures, manifold learning, trajectory representation