Search

This paper presents an algorithm for multichannel sound source separation using explicit modeling of level and time differences in source spatial covariance matrices (SCM). We propose a novel SCM model in which the spatial properties are modeled by the weighted sum of direction of arrival (DOA) kernels. DOA kernels are obtained as the combination of phase and level difference covariance matrices representing both time and level differences between microphones for a grid of predefined source directions. The proposed SCM model is combined with the NMF model for the magnitude spectrograms. Opposite to other SCM models in the literature, in this work, source localization is implicitly defined in the model and estimated during the signal factorization. Therefore, no localization preprocessing is required. Parameters are estimated using complex-valued nonnegative matrix factorization with both Euclidean distance and Itakura-Saito divergence. Separation performance of the proposed system is evaluated using the two-channel SiSEC development dataset and four channels signals recorded in a regular room with moderate reverberation. Finally, a comparison to other state-of-the-art methods is performed, showing better achieved separation performance in terms of SIR and perceptual measures.

In this article, we present an online score following framework designed to deal with automatic accompaniment. The proposed framework is based on spectral factorization and online Dynamic Time Warping (DTW) and has two separated stages: preprocessing and alignment. In the first one, we convert the score into a reference audio signal using a MIDI synthesizer software and we analyze the provided information in order to obtain the spectral patterns (i.e., basis functions) associated to each score unit. In this work, a score unit represents the occurrence of concurrent or isolated notes in the score. These spectral patterns are learned from the synthetic MIDI signal using a method based on Non-negative Matrix Factorization (NMF) with Beta-divergence, where the gains are initialized as the ground-truth transcription inferred from the MIDI. On the second stage, a non-iterative signal decomposition method with fixed spectral patterns per score unit is used over the magnitude spectrogram of the input signal resulting in a distortion matrix that can be interpreted as the cost of the matching for each score unit at each frame. Finally, the relation between the performance and the musical score times is obtained using a strategy based on online DTW, where the optimal path is biased by the speed of interpretation. Our system has been evaluated and compared to other systems, yielding reliable results and performance.