Search

Abstract: In this paper, we study the multiclass classification problem. We derive a framework to solve this problem by providing algorithms with the complexity of a single binary classifier. The resulting multiclass machines can be decomposed into two categories. The first category corresponds to vector-output machines, where we develop several algorithms. In the second category, we show that the least-squares classifier can be easily cast into a multiclass one-versus-all scheme, without the need to train multiple binary classifiers. The proposed framework shows that, while keeping the classification accuracy essentially unchanged, the computational complexity is orders of magnitude lower than those previously reported in the literature. This makes our approach extremely powerful and conceptually simple. Moreover, we study the coding of the multiclass labels, and demonstrate that several celebrated approaches are equivalent. These arguments are illustrated with experimentations on well-known benchmarks. [Copyright &y& Elsevier]

Abstract: Discovering structure within a collection of high-dimensional input vectors is a problem that often recurs in the area of machine learning. A very suitable and widely used algorithm for solving such tasks is Non-negative Matrix Factorization (NMF). The high-dimensional vectors are arranged as columns in a data matrix, which is decomposed into two non-negative matrix factors of much lower rank. Here, we adopt the NMF learning scheme proposed by Van hamme (2008) . It involves combining the training data with supervisory data, which imposes the low-dimensional structure known to be present. The reconstruction of such supervisory data on previously unseen inputs then reveals their underlying structure in an explicit way. It has been noted that for many problems, not all features of the training data correlate equally well with the underlying structure. In other words, some features are relevant for detecting patterns in the data, while others are not. In this paper, we propose an algorithm that builds upon the learning scheme of Van hamme (2008) , and automatically weights each input feature according to its relevance. Applications include both data improvement and feature selection. We experimentally show that our algorithm outperforms similar techniques on both counts. [Copyright &y& Elsevier]