Search

This paper proposes a novel image set classification technique based on the concept of linear regression. Unlike most other approaches, the proposed technique does not require any training. We represent the gallery image sets as subspaces in a high dimensional space. Class specific gallery subspaces are used to estimate regression models for each image in the test image set. Images of the test set are then projected onto the gallery subspaces. The residuals, calculated using the Euclidean distance between the original and the projected test images, are used as the distance metric. Three different strategies are devised to decide on the final class of the test image set. We extensively evaluated the proposed technique using both low resolution and noisy images and with less gallery data to assess the suitability of our technique for the tasks of surveillance and video-based face recognition. The experiments show that the proposed technique achieves superior classification accuracy and has a faster execution time compared with existing techniques, especially under the challenging conditions of low resolution and a limited amount of gallery and test data.

Sound event detection (SED) has been widely applied in real world applications. Convolutional recurrent neural network based SED approaches have achieved state-of-the-art performance. However, the convolution process is typically performed by using a fixed sized kernel, which adversely affects the detection accuracy especially when the acoustic features of different event classes are characterized by high variations. To deal with this, this article proposes a sound event detection technique using a convolutional recurrent neural network framework with multiple convolutional kernels of different sizes. The top performing kernels are selected from a kernel pool based on the unsupervised clustering errors and the accuracies of the temporarily trained models. Afterwards, the selected kernels are fed to multiple convolution layers to deal with the acoustic feature variations. Experimental results on different subsets of AudioSet, namely the DCASE Challenge 2017 Task 4 and DCASE Challenge 2018 Task 4, demonstrate the performance of the proposed approach compared to state-of-the-art systems.

Acoustic event detection deals with the acoustic signals to determine the sound type and to estimate the audio event boundaries. Multi-label classification based approaches are commonly used to detect the frame wise event types with a median filter applied to determine the happening acoustic events. However, the multi-label classifiers are trained only on the acoustic event types ignoring the frame position within the audio events. To deal with this, this paper proposes to construct a joint learning based multi-task system. The first task performs the acoustic event type detection and the second task is to predict the frame position information. By sharing representations between the two tasks, we can enable the acoustic models to generalize better than the original classifier by averaging respective noise patterns to be implicitly regularized. Experimental results on the monophonic UPC-TALP and the polyphonic TUT Sound Event datasets demonstrate the superior performance of the joint learning method by achieving lower error rate and higher F-score compared to the baseline AED system.

A differential approach is proposed for tomographic rain field reconstruction using the estimated signal-to-noise ratio of microwave signals from low earth orbit satellites at the ground receivers, with the unknown baseline values eliminated before using least squares to reconstruct the attenuation field. Simulations are done when the baseline is modelled by an autoregressive process and when the baseline is assumed fixed. Comparisons between the reconstruction results for the differential and non-differential approaches suggest that the differential approach performs better in both scenarios. For high correlation coefficient and low model noise in the autoregressive process, the differential approach surpasses the non-differential approach significantly.

We propose a novel confidence estimation method for predictions from a multi-class classifier. Unlike existing methods, we learn a confidence-estimator on the basis of a held-out set from the training data. The predicted confidence values by the proposed system are used to improve the accuracy of multi-modal emotion and sentiment classification. The scores of different classes from the individual modalities are superposed on the basis of confidence values. Experimental results demonstrate that the accuracy of the proposed confidence based fusion method is significantly superior to that of the classifier trained on any modality separately, and achieves superior performance compared to other fusion methods.

Recently, neural network-based deep learning methods have been popularly applied to computer vision, speech signal processing and other pattern recognition areas. Remarkable success has been demonstrated by using the deep learning approaches. The purpose of this article is to provide a comprehensive survey for the neural network-based deep learning approaches on acoustic event detection. Different deep learning-based acoustic event detection approaches are investigated with an emphasis on both strongly labeled and weakly labeled acoustic event detection systems. This paper also discusses how deep learning methods benefit the acoustic event detection task and the potential issues that need to be addressed for prospective real-world scenarios.

Coral reefs are vital for marine ecosystem and fishing industry. Automatic classification of corals is essential for the preservation and study of coral reefs. However, significant intra-class variations and inter-class similarity among coral genera, as well as the challenges of underwater illumination present a great hindrance for the automatic classification. We propose an end-to-end trainable Deep Fusion Net for the classification of corals from two types of images. The network takes two simultaneous inputs of reflectance and fluorescence images. It is composed of three branches: Reflectance, Fluorescence and Integration. The branches are first trained individually and then fused together. Finally, the Deep Fusion Net is trained end-to-end for the classification of different coral genera and other non-coral classes. Experiments on the challenging Eliat Fluorescence Coral dataset show that the Deep Fusion net achieves superior classification accuracy compared to other methods.

With the progress in the field of computer vision, we are moving closer and closer towards the ultimate aim of human like vision for machines. Scene understanding is an essential part of this research. It seeks the goal that any image should be as understandable and decipherable for computers as it is for humans. The stall in the progress of the different components of scene understanding, due to the limitations of the traditional algorithms, has now been broken by the induction of neural networks for computer vision tasks. The advancements in parallel computational hardware has made it possible to train very deep and complex neural network architectures. This has vastly improved the performances of algorithms for all the different components of scene understanding. This chapter analyses these contributions of deep learning and also presents the advancements of high level scene understanding tasks, such as caption generation for images. It also sheds light on the need to combine these individual components into an integrated system.

In acoustic event detection, the training data size of some acoustic events are often small and imbalanced. To deal with these, this paper proposes to generate the virtual training data categorically using the auxiliary classifier generative adversarial networks. Soft labels of acoustic events are first calculated to represent the acoustic event localization information. The closer the current frame is to the middle of the manually labeled acoustic event, the higher the soft label will be, which makes the soft labels positively correlated to the acoustic event localization. Then the acoustic event class and the quantized soft labels are used as the input condition to the auxiliary classifier generative adversarial networks to generate arbitrary number of training samples. Experimental results on the TUT Sound Event 2016 under the home environment and TUT Sound Event 2017 under the street environment demonstrate the improved performance of the proposed technique compared to existing acoustic event detection systems.

In this paper we present an approach for acoustic scene classification, which aggregates spectral and temporal features. We do this by proposing the first use of the variable-Q transform (VQT) to generate the time-frequency representation for acoustic scene classification. The VQT provides finer control over the resolution compared to the constant-Q transform (CQT) or STFT and can be tuned to better capture acoustic scene information. We then adopt a variant of the local binary pattern (LBP), the Adjacent Evaluation Completed LBP (AECLBP), which is better suited to extracting features from acoustic time-frequency images. Our results yield a 5.2% improvement on the DCASE 2016 dataset compared to the application of standard CQT with LBP. Fusing our proposed AECLBP with HOG features we achieve a classification accuracy of 85.5% which outperforms one of the top performing systems.

The variety of event categories and event boundary information have resulted in limited success for acoustic event detection systems. To deal with this, we propose to utilize the long contextual information, low-dimensional discriminant global bottleneck features and category-specific bottleneck features. By concatenating several adjacent frames together, the use of contextual information makes it easier to cope with acoustic signals with long duration. Global and category-specific bottleneck features can extract the prior knowledge of the event category and boundary, which is ideally matched by the task of an event detection system. Evaluations on the UPC-TALP and ITC-IRST databases of highly variable acoustic events demonstrate the effectiveness of the proposed approaches by achieving a 5.30% and 4.44% absolute error rate improvement respectively compared to the state of art technique.

This paper proposes a novel image set classification technique based on the concept of linear regression. Unlike most other approaches, the proposed technique does not involve any training or feature extraction. The gallery image sets are represented as subspaces in a high dimensional space. Class specific gallery subspaces are used to estimate regression models for each image of the test image set. Images of the test set are then projected on the gallery subspaces. Residuals, calculated using the Euclidean distance between the original and the projected test images, are used as the distance metric. Three different strategies are devised to decide on the final class of the test image set. We performed extensive evaluations of the proposed technique under the challenges of low resolution, noise and less gallery data for the tasks of surveillance, video-based face recognition and object recognition. Experiments show that the proposed technique achieves a better classification accuracy and a faster execution time compared to existing techniques especially under the challenging conditions of low resolution and small gallery and test data.

The classification of acoustic scenes is important in emerging applications such as automatic audio surveillance, machine listening and multimedia content analysis. In this paper, we present an approach for acoustic scene classification by using joint time-frequency image-based feature representations. In acoustic scene classification, joint time-frequency representation (TFR) is shown to better represent important information across a wide range of low and middle frequencies in the audio signal. The audio signal is converted to Constant-Q Transform (CQT) and Mel-spectrum TFRs and local binary patterns (LBP) are used to extract the features from these TFRs. To ensure localized spectral information is not lost, the TFRs are divided into a number of zones. Then, we perform score level fusion to further improve the classification performance accuracy. Our technique achieves a competitive performance with a classification accuracy of 83.4% on the DCASE 2016 development dataset compared to the existing current state of the art

In this paper we introduce a novel multimodal biometric recognition system based on generalized sparse representations. In the recently proposed systems, with heterogeneous features (such as audio and video), the joint sparse optimization problem was addressed by bringing the features into the same dynamic range, such as normalizing the features into unitl 2 norm. This is however not optimal, and such normalization may decrease the performance of that modality unimodally. We propose to solve the original joint sparse optimization problem by introducing scaling factors for different modalities, such that the modalities interact efficiently at the feature level. The sequence-dependent scaling factors are automatically calculated so that the mismatch between the sparse representations of different modalities is accounted for. In the case of audiovisual recognition system, our experiments on the challenging MOBIO database show that the proposed method outperforms the original joint sparsity-based system (96.8% vs 94.3% recognition rate).

Class imbalance is a common problem in the case of real-world object detection and classification tasks. Data of some classes are abundant, making them an overrepresented majority, and data of other classes are scarce, making them an underrepresented minority. This imbalance makes it challenging for a classifier to appropriately learn the discriminating boundaries of the majority and minority classes. In this paper, we propose a cost-sensitive (CoSen) deep neural network, which can automatically learn robust feature representations for both the majority and minority classes. During training, our learning procedure jointly optimizes the class-dependent costs and the neural network parameters. The proposed approach is applicable to both binary and multiclass problems without any modification. Moreover, as opposed to data-level approaches, we do not alter the original data distribution, which results in a lower computational cost during the training process. We report the results of our experiments on six major image classification data sets and show that the proposed approach significantly outperforms the baseline algorithms. Comparisons with popular data sampling techniques and CoSen classifiers demonstrate the superior performance of our proposed method.

We propose a novel image set classification technique using linear regression models. Downsampled gallery image sets are interpreted as subspaces of a high dimensional space to avoid the computationally expensive training step. We estimate regression models for each test image using the class specific gallery subspaces. Images of the test set are then reconstructed using the regression models. Based on the minimum reconstruction error between the reconstructed and the original images, a weighted voting strategy is used to classify the test set. We performed extensive evaluation on the benchmark UCSD/Honda, CMU Mobo and YouTube Celebrity datasets for face classification, and ETH-80 dataset for object classification. The results demonstrate that by using only a small amount of training data, our technique achieved competitive classification accuracy and superior computational speed compared with the state-of-the-art methods.

Acoustic event detection, the determination of the acoustic event type and the localisation of the event, has been widely applied in many real-world applications. Many works adopt multi-label classification techniques to perform the polyphonic acoustic event detection with a global threshold to detect the active acoustic events. However, the global threshold has to be set manually and is highly dependent on the database being tested. To deal with this, we replaced the fixed threshold method with a frame-wise dynamic threshold approach in this paper. Two novel approaches, namely contour and regressor based dynamic threshold approaches are proposed in this work. Experimental results on the popular TUT Acoustic Scenes 2016 database of polyphonic events demonstrated the superior performance of the proposed approaches.

This paper deals with random forest regression based acoustic event detection (AED) by combining acoustic features with bottleneck features (BN). The bottleneck features have a good reputation of being inherently discriminative in acoustic signal processing. To deal with the unstructured and complex real-world acoustic events, an acoustic event detection system is constructed using bottleneck features combined with acoustic features. Evaluations were carried out on the UPC-TALP and ITC-Irst databases which consist of highly variable acoustic events. Experimental results demonstrate the usefulness of the low-dimensional and discriminative bottleneck features with relative 5.33% and 5.51% decreases in error rates respectively.

This paper deals with the acoustic event detection (AED) to improve the detection accuracy of acoustic events. Acoustic event detection task is performed by a regression via classification (RvC) based approach along with the random forest technique. A discretization process is used to convert the continuous frame positions within acoustic events into event duration class labels. Outputs of the category-specific random forest classifiers are then reversed back to the event boundary information. Evaluations on the UPC-TALP database which consists of highly variable acoustic events demonstrate the efficiency of the proposed approaches with improvements in detection error rate compared to the best baseline system.

This paper introduces the use of local binary patterns (LBP) extracted from a time-frequency representation (TFR) for acoustic scene classification. As LBP provides a description of the global TFR texture we propose a novel zoning mechanism that provides a simple solution to extract spectrally relevant local features which better characterize the audio TFRs. To further improve the classification performance, we perform feature and score level fusion of the proposed LBP (with zoning) with histogram of gradients (HOG) of the TFR images. Our technique demonstrates an improved performance by achieving a classification accuracy of 95.2% using a fusion of time-frequency derived features.

This chapter starts with a general and brief introduction of biometrics and audiovisual person recognition using mobile phone data. It begins with a discussion of what constitutes a biometric recognition system, and it then details the steps followed when audio-visual signals are used as inputs. This is followed by a review of the existing speaker and face recognition systems which have been evaluated on a mobile biometric database. We then discuss the key motivations of using deep neural network (DNN) for person recognition. We finally introduce a Deep Boltzmann Machine (DBM)- DNN, in short DBM-DNN, based framework for person recognition. An overview of the sections and sub-sections of this chapter is shown in Figure 4.1.

We propose an audio-visual person identification approach based on a joint deep Boltzmann machine (jDBM) model. The proposed jDBM model is trained in three steps: a) learning the unimodal DBM models corresponding to the speech and facial image modalities, b) learning the shared layer parameters using a joint Restricted Boltzmann Machine (jRBM) model and c) the fine-tuning of the jDBM model after the initialization with the parameters of the unimodal DBMs and the shared layer. The activation probabilities of the units of the shared layer are used as the joint features and a logistic regression classifier is used for the combined speech and facial image recognition. We show that by learning the shared layer parameters using a jRBM, a higher accuracy can be achieved compared to the greedy layer-wise initialization. The performance of our proposed model is also compared with state-of-the art support vector machine (SVM), deep belief network (DBN), and the deep auto-encoder (DAE) models. In addition, our experimental results show that the joint representations obtained from the proposed jDBM model are robust to noise and missing information. Experiments were carried out on the challenging MOBIO database, which includes audio-visual data captured using mobile phones.

Indoor scene recognition is a multi-faceted and challenging problem due to the diverse intra-class variations and the confusing inter-class similarities that characterize such scenes. This paper presents a novel approach that exploits rich mid-level convolutional features to categorize indoor scenes. Traditional convolutional features retain the global spatial structure, which is a desirable property for general object recognition. We, however, argue that the structure-preserving property of the convolutional neural network activations is not of substantial help in the presence of large variations in scene layouts, e.g., in indoor scenes. We propose to transform the structured convolutional activations to another highly discriminative feature space. The representation in the transformed space not only incorporates the discriminative aspects of the target data set but also encodes the features in terms of the general object categories that are present in indoor scenes. To this end, we introduce a new large-scale data set of 1300 object categories that are commonly present in indoor scenes. Our proposed approach achieves a significant performance boost over the previous state-of-the-art approaches on five major scene classification data sets.

In this paper we present a novel audio-visual (AV) person identification system based on joint sparse representation. Video features used were vectorized raw pixel values, while i-vectors were used as the audio features. Classification is performed by solving the joint sparsity optimization problem, and fusion is carried out by using the quality (confidence) assigned to each matcher. Our experimental results on the challenging MOBIO database using 100 subjects show that the system based on joint sparse representation outperforms the system based on separate sparse representations for each modality. Furthermore, we show that our newly introduced quality measure improves the system's performance, when compared to conventionally used quality measures for sparse representation - based systems.

Inexpensive structured light sensors can capture rich information from indoor scenes, and scene labeling problems provide a compelling opportunity to make use of this information. In this paper we present a novel conditional random field (CRF) model to effectively utilize depth information for semantic labeling of indoor scenes. At the core of the model, we propose a novel and efficient plane detection algorithm which is robust to erroneous depth maps. Our CRF formulation defines local, pairwise and higher order interactions between image pixels. At the local level, we propose a novel scheme to combine energies derived from appearance, depth and geometry-based cues. The proposed local energy also encodes the location of each object class by considering the approximate geometry of a scene. For the pairwise interactions, we learn a boundary measure which defines the spatial discontinuity of object classes across an image. To model higher-order interactions, the proposed energy treats smooth surfaces as cliques and encourages all the pixels on a surface to take the same label. We show that the proposed higher-order energies can be decomposed into pairwise sub-modular energies and efficient inference can be made using the graph-cuts algorithm. We follow a systematic approach which uses structured learning to fine-tune the model parameters. We rigorously test our approach on SUN3D and both versions of the NYU-Depth database. Experimental results show that our work achieves superior performance to state-of-the-art scene labeling techniques.

We present a framework to automatically detect and remove shadows in real world scenes from a single image. Previous works on shadow detection put a lot of effort in designing shadow variant and invariant hand-crafted features. In contrast, our framework automatically learns the most relevant features in a supervised manner using multiple convolutional deep neural networks (ConvNets). The features are learned at the super-pixel level and along the dominant boundaries in the image. The predicted posteriors based on the learned features are fed to a conditional random field model to generate smooth shadow masks. Using the detected shadow masks, we propose a Bayesian formulation to accurately extract shadow matte and subsequently remove shadows. The Bayesian formulation is based on a novel model which accurately models the shadow generation process in the umbra and penumbra regions. The model parameters are efficiently estimated using an iterative optimization procedure. Our proposed framework consistently performed better than the state-of-the-art on all major shadow databases collected under a variety of conditions.

This paper presents a confidence-based late fusion framework and its application to audio-visual biometric identification. We assign each biometric matcher a confidence value calculated from the matching scores it produces. Then a transformation of the matching scores is performed using a novel confidence-ratio (C-ratio) i.e., the ratio of a matcher confidence obtained at the test phase to the corresponding matcher confidence obtained at the training phase. We also propose modifications to the highest rank and Borda count rank fusion rules to incorporate the matcher confidence. We demonstrate by experiments that our proposed confidence-based fusion framework is more robust compared to the state-of-the-art late (score- and rank-level) fusion approaches.

We propose an approach using DBM-DNNs for i-vector based audio-visual person identification. The unsupervised training of two Deep Boltzmann Machines DBMspeech and DBMface is performed using unlabeled audio and visual data from a set of background subjects. The DBMs are then used to initialize two corresponding DNNs for classification, referred to as the DBM-DNNspeech and DBM-DNNface in this paper. The DBM-DNNs are discriminatively fine-tuned using the back-propagation on a set of training data and evaluated on a set of test data from the target subjects. We compared their performance with the cosine distance (cosDist) and the state-of-the-art DBN-DNN classifier. We also tested three different configurations of the DBM-DNNs. We show that DBM-DNNs with two hidden layers and 800 units in each hidden layer achieved best identification performance for 400 dimensional i-vectors as input. Our experiments were carried out on the challenging MOBIO dataset.

To follow...

Objects’ spatial layout estimation and clutter identification are two important tasks to understand indoor scenes. We propose to solve both of these problems in a joint framework using RGBD images of indoor scenes. In contrast to recent approaches which focus on either one of these two problems, we perform ‘fine grained structure categorization’ by predicting all the major objects and simultaneously labeling the cluttered regions. A conditional random field model is proposed to incorporate a rich set of local appearance, geometric features and interactions between the scene elements. We take a structural learning approach with a loss of 3D localisation to estimate the model parameters from a large annotated RGBD dataset, and a mixed integer linear programming formulation for inference. We demonstrate that our approach is able to detect cuboids and estimate cluttered regions across many different object and scene categories in the presence of occlusion, illumination and appearance variations.

© 2006-2015 Asian Research Publishing Network (ARPN). Satisfying customers' needs economically is one of the important aspects in mobile communication industry. Provider should cater a good and consistent quality of service as expected by the customers. Hence, it is amounts to controlling the speech quality perceived by the customers. However, to control the speech quality, the reliable measurement of the speech quality must be determined first, then exercising direct control of it. Traditionally, the quality of speech at the end user has been monitored and controlled based on radio link measurements such as the Signal Interference Ratio (SIR), Bit Error Rate (BER), or Frame Error Rate (FER). The truly perceive speech quality is measure through subjective listening tests. However, this method is not practical for real-time day to day applications. Newly, objective quality measurement algorithms have been developed to replace the subjective quality measurement with considerable accuracy. P.862 Perceptual Evaluation of Speech Quality (PESQ) model is the state of art on the referenced objective measurement method in the International Telecommunication Union's Telecommunication Standardization Sector (ITU-T). PESQ is calculated based on the so-called Frame Disturbance (FD). In this paper FD is investigated as a perceptual metric for control of speech quality in modern communication systems replacing the conventional metrics.

We present a discriminative graphical model which integrates geometrical information from RGBD images in its unary, pairwise and higher order components. We propose an improved geometry estimation scheme which is robust to erroneous sensor inputs. At the unary level, we combine appearance based beliefs defined on pixels and planes using a hybrid decision fusion scheme. Our proposed location potential gives an improved representation of the planar classes. At the pairwise level, we learn a balanced combination of various boundaries to consider the spatial discontinuity. Finally, we treat planar regions as higher order cliques and use graphcuts to make efficient inference. In our model based formulation, we use structured learning to fine tune the model parameters. We test our approach on two RGBD datasets and demonstrate significant improvements over the state-of-the-art scene labeling techniques.

A multibiometric identification system establishes the identity of a person based on the biometric data presented to its sub-systems. Each sub-system compares the features extracted from the input against the templates of all identities stored in its gallery. In rank-level fusion, ranked lists from different sub-systems are combined to reach the final decision about an identity. However, the state-of-art rank-level fusion methods consider that all sub-systems perform equally well in any conditions. In practice, the probe data may be affected by different degradations (e.g., illumination and pose variation on the face image, environmental noise etc.) and thus affect the overall recognition accuracy. In this paper, robust confidence-based rank-level fusion methods are proposed by using confidence measures for all participating sub-systems. Experimental results show that the confidence-based approach of rank-level fusion achieves higher recognition rates than the state-of-art.

We present a practical framework to automatically detect shadows in real world scenes from a single photograph. Previous works on shadow detection put a lot of effort in designing shadow variant and invariant hand-crafted features. In contrast, our framework automatically learns the most relevant features in a supervised manner using multiple convolutional deep neural networks (ConvNets). The 7-layer network architecture of each ConvNet consists of alternating convolution and sub-sampling layers. The proposed framework learns features at the super-pixel level and along the object boundaries. In both cases, features are extracted using a context aware window centered at interest points. The predicted posteriors based on the learned features are fed to a conditional random field model to generate smooth shadow contours. Our proposed framework consistently performed better than the state-of-the-art on all major shadow databases collected under a variety of conditions.

Geoscientific data interpretation is a challenging task, which requires the detection and synthesis of complex patterns within data. As a first step towards better understanding this interpretation process, our research focuses on quantitative monitoring of interpreters' brain responses associated with geoscientific target spotting. This paper presents a method that profiles brain responses using electroencephalography (EEG) to detect P300-like responses that are associated with target spotting for complex geoscientific data. In our experiment, eight interpreters with varying levels of expertise and experience were asked to detect features, which are likely to be copper-gold rich porphyry systems within magnetic geophysical data. The target features appear in noisy background and often have incomplete shape. Magnetic images with targets and without targets were shown to participants using the "oddball" paradigm. Event related potentials were obtained by averaging the EEG epochs across multiple trials and the results show delayed P3 response to the targets, likely due to the complexity of the task. EEG epochs were classified and the results show reliable single trial classification of EEG responses with an average accuracy of 83%. The result demonstrated the usability of the P300-like responses to quantify the geoscientific target spotting performances. © 2013 Elsevier Ltd.

We introduce a novel approach for source number estimation through an adaptive fuzzy c-means clustering. Spatial feature vectors are extracted from microphone observations, weighted for reliability and then clustered in a full-band manner using an adaptive variation on the fuzzy c-means. A number of quality measures are combined to produce a weighted sum which is used to find the optimal number of clusters at each iteration of the clustering algorithm. Experimental evaluations using real-world recordings from a reverberant room (RT60 = 390 ms) demonstrated encouraging performance in both even- and under-determined conditions.

In this paper we propose to incorporate contextual time frequency information for clustering-based blind source separation. Previous clustering-based approaches have successfully used clustering techniques to estimate time-frequency separation masks; however, these approaches generally do not consider the contextual information of each time-frequency slot. Motivated by the homogenous behavior of speech signals, we modify the fuzzy c-means clustering to bias the results in favor of cluster membership homogeneity within localized neighborhoods in the time-frequency space. Experimental evaluations in both simulated and real-world underdetermined environments demonstrate improvement in source separation performance over previous clustering approaches.

In this paper we investigate the use of observation weights and contextual time-frequency information for clustering-based blind source separation. Previous clustering-based approaches have successfully used clustering techniques to estimate time-frequency separationmasks; however, these approaches generally disregard the structured nature of speech signals. Motivated by the homogenous behaviour of speech signals, we propose to modify the established fuzzy cmeans algorithm to bias the clustering results in favor of cluster membership homogeneity within localized neighborhoods in the time-frequency space. This problem can be solved by using a two stage algorithm: firstly, the estimation of data weights to indicate the reliability of each data point, and secondly, the integration of local contextual information into the cluster update equations from neighboring time-frequency slots. The proposed algorithm is evaluated in a three-fold manner using simulated, real recordings and public benchmark data; notable improvement in source separation performance over previous clustering approaches was achieved.

In this paper, we investigate the application of a generative clustering technique for the estimation of time-frequency source separation masks. Recent advances in time-frequency clustering-based approaches to blind source separation have touched upon the Watson mixture model (WMM) as a tool for source separation. However, most methods have been frequency bin-wise and have thus required the additional permutation alignment stage, and previous full-band methods which employ the WMM have yet to be applied to the under-determined setting. We propose to evaluate the clustering ability of the WMM within the clustering-based source separation framework. Evaluations confirm the superiority of the WMM against other previously used clustering techniques such as the fuzzy c-means.

This paper presents a reliability estimation technique for an Audio-Visual (AV) person identification system. The estimated reliability measures are mapped to modality weights by using a mapping function. The weighted sum rule of fusion was used to fuse the audio and visual modalities before reaching a decision about the identity of a person. The proposed technique was tested on 88 subjects from the AusTalk AV database. Experimental results show that the proposed technique improves the identification accuracy of the min-max normalization, compared to when it is used without any reliability estimation. The identification accuracy of the proposed technique is also higher than the non-learned approach of modality weight adaptation.

The successful implementation of speech processing systems in the real world depends on its ability to handle adverse acoustic conditions with undesirable factors such as room reverberation and background noise. In this study, an extension to the established multiple sensors degenerate unmixing estimation technique (MENUET) algorithm for blind source separation is proposed based on the fuzzy c-means clustering to yield improvements in separation ability for underdetermined situations using a nonlinear microphone array. However, rather than test the blind source separation ability solely on reverberant conditions, this paper extends this to include a variety of simulated and real-world noisy environments. Results reported encouraging separation ability and improved perceptual quality of the separated sources for such adverse conditions. Not only does this establish this proposed methodology as a credible improvement to the system, but also implies further applicability in areas such as noise suppression in adverse acoustic environments.

Industrial automation with speech control functions is generally installed with a speech recognition sensor which is used as an interface for users to articulate speech commands. However, recognition errors are likely to be produced when background noise surrounds the command spoken into the speech recognition microcontrollers. In this paper, a speech enhancement strategy is proposed to develop noise suppression filters in order to improve the accuracy of speech recognition microcontrollers. It uses a universal estimator, namely a neural network, to enhance the recognition accuracy of microcontrollers by integrating better signals processed by various noise suppression filters, where a global optimization algorithm, namely an intelligent particle swarm optimization, is used to optimize the inbuilt parameters of the neural network in order to maximize accuracy of speech recognition microcontrollers working within noisy environments. The proposed approach overcomes the limitations of the existing noise suppression filters intended to improve recognition accuracy. The performance of the proposed approach was evaluated by a speech recognition microcontroller, which is used in electronic products with speech control functions. Results show that the accuracy of the speech recognition microcontroller can be improved using the proposed approach, when working under low signal to noise ratio conditions in the industrial environments of automobile engines and factory machines.

Considerable attention has been devoted to the reverberant blind source separation problem: in particular, the concept of time-frequency masking. However, realistic acoustic scenarios often comprise not only reverberation, but also additive noise due to factors such as non-ideal channels. This paper presents robust evaluations of a time-frequency masking approach for separation in such realistic conditions. The fuzzy c-means clustering algorithm is used to cluster spatial feature cues into a time-frequency mask. Experimental results demonstrated superiority in separation, with notable improvements in the SNR additionally observed. Not only does this establish the proposed scheme viable for reverberant blind source separation, but also as a credible means of speech enhancement in the presence of additive broadband noise.

The successful application of automatic speech recognition systems in the real world is conditional on its ability to handle realistic environments with unfavorable conditions such as reverberation and multiple sources of inteference. Previous research has identified time-frequency masking based approaches to blind source separation as a viable approach for multisource reverberant source separation. It is proposed the use of such separation techniques as a front-end to speech recognition will encourage greater recognition accuracy. Experimental evaluations confirmed the hypothesis with an improvement in recognition accuracy of over 20% at a reverberation time of RT60 = 300ms; this is indicative of the potential for future research in this field. © 2012 IEEE.

Accuracy and efficiency in geoscientific data interpretation is critically important for the resource industry, as based on these interpretations, significant financial decisions are made for exploration and extraction of resources. This study aims to understand image characteristics that impact interpreters' ability to detect geological targets within magnetic geophysics images. We use the Brain Computer Interface (BCI) approach to profile interpreter's neurological responses to geophysical target images using electroencephalography (EEG). These results are compared with image saliency analysis outputs, specifically the spatial distribution of visual attention peaks. Our experimental results showed that images with a wider dispersion of saliency lead to a lower rate of target detection based on EEG activities of interpreters.

Item does not contain fulltext, Under an ARC Linkage Infrastructure, Equipment and Facilities (LIEF) grant, speech science and technology experts from across Australia have joined forces to organise the recording of audio-visual (AV) speech data from representative speakers of Australian English in all capital cities and some regional centres. The Big Australian Speech Corpus (the Big ASC) will provide a standard recording setup and a collaboratively-designed elicitation protocol to create a corpus of AV speech data incorporating annotations and metadata, accessible via a centralised storage facility. The Big ASC infrastructure will provide a significant boost to research in speech science and human communication in Australia.