1. Phasebook and Friends: Leveraging Discrete Representations for Source Separation
- Author
-
Gordon Wichern, Shinji Watanabe, Andy M. Sarroff, John R. Hershey, and Jonathan Le Roux
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,Sound (cs.SD) ,Computer science ,Machine Learning (stat.ML) ,02 engineering and technology ,Computer Science - Sound ,Oracle ,Machine Learning (cs.LG) ,Statistics - Machine Learning ,Audio and Speech Processing (eess.AS) ,FOS: Electrical engineering, electronic engineering, information engineering ,0202 electrical engineering, electronic engineering, information engineering ,Source separation ,Electrical and Electronic Engineering ,Computer Science - Computation and Language ,Noise measurement ,business.industry ,Quantization (signal processing) ,Deep learning ,020206 networking & telecommunications ,Time–frequency analysis ,Speech enhancement ,Signal Processing ,Softmax function ,Artificial intelligence ,business ,Computation and Language (cs.CL) ,Algorithm ,Electrical Engineering and Systems Science - Audio and Speech Processing - Abstract
Speech enhancement and source separation systems based on deep learning have recently reached unprecedented levels of quality, to the point that performance is reaching a new ceiling. Most systems rely on estimating the magnitude of a target source by estimating a real-valued mask to be applied to a time-frequency representation of the mixture signal. A limiting factor in such approaches is a lack of phase estimation: the phase of the mixture is most often used when reconstructing the estimated time-domain signal. Here, we propose “magbook,” “phasebook,” and “combook,” three new types of layers based on discrete representations that can be used to estimate complex time-frequency masks. Magbook layers extend classical sigmoidal units and a recently introduced convex softmax activation for mask-based magnitude estimation. Phasebook layers use a similar structure to give an estimate of the phase mask without suffering from phase wrapping issues. Combook layers are an alternative to the magbook–phasebook combination that directly estimate complex masks. We present various training and inference schemes involving these representations, and explain in particular how to include them in an end-to-end learning framework. We also present an oracle study to assess upper bounds on performance for various types of masks using discrete phase representations. We evaluate the proposed methods on the wsj0-2mix dataset, a well-studied corpus for single-channel speaker-independent speaker separation, matching the performance of state-of-the-art mask-based approaches without requiring additional phase reconstruction steps.
- Published
- 2019
- Full Text
- View/download PDF