Hierarchical Model for Multimedia Content Classification

An automatic content classification technique is an essential tool in multimedia applications. Present research for audio-based classifiers look at short- and long-term analysis of signals, using both temporal and spectral features as well as using auditory models. In this paper, we present a hierarchical scheme that maps familiar single-type streaming content sources (e.g., either music, movie, or voice) to processes in a JavaScript Object Notation (JSON) configuration file. For streaming sources containing mixed-type of content (e.g., movies and music) we employ a machine learning (ML) model to classify between the movie, music, and voice-based on metadata. Towards this end, statistical models of the various metadata are created since a large metadata dataset is not available. Subsequently, synthetic metadata are generated from these statistical models, and the synthetic metadata is input to the ML classifier as feature vectors. We demonstrate high discriminating capabilities (accuracy ≈ 90%), with very low latency (viz., ≈ on an average 7 ms), using real metadata from real-world content such as from YouTubeTM, Blu-rayTM, and user-generated content (UGC). The combined hierarchical system (comprising JSON file and ML-model) reliably identifies the content type with an accuracy greater than 90%.