Search

Action recognition in videos is one of the key problems in visual data interpretation. Despite intensive research, differencing and recognizing similar actions remains a challenge. This thesis deals with fine-grained classification of sport gestures from videos, with an application to table tennis.In this manuscript, we propose a method based on deep learning for automatically segmenting and classifying table tennis strokes in videos. Our aim is to design a smart system for students and teachers for analyzing their performances. By profiling the players, a teacher can therefore tailor the training sessions more efficiently in order to improve their skills. Players can also have an instant feedback on their performances.For developing such a system with fine-grained classification, a very specific dataset is needed to supervise the learning process. To that aim, we built the “TTStroke-21” dataset, which is composed of 20 stroke classes plus a rejection class. The TTStroke-21 dataset comprises video clips of recorded table tennis exercises performed by students at the sport faculty of the University of Bordeaux - STAPS. These recorded sessions were annotated by professional players or teachers using a crowdsourced annotation platform. The annotations consist in a description of the handedness of the player and information for each stroke performed (starting and ending frames, class of the stroke). Fine-grained action recognition has some notable differences with coarse-grained action recognition. In general, datasets used for coarse-grained action recognition, the background context often provides discriminative information that methods can use to classify the action, rather than focusing on the action itself. In fine-grained classification, where the inter-class similarity is high, discriminative visual features are harder to extract and the motion plays a key role for characterizing an action.In this thesis, we introduce a Twin Spatio-Temporal Convolutional Neural Network. This deep learning network takes as inputs an RGB image sequence and its computed Optical Flow. The RGB image sequence allows our model to capture appearance features while the optical flow captures motion features. Those two streams are processed in parallel using 3D convolutions, and fused at the last stage of the network. Spatio-temporal features extracted in the network allow efficient classification of video clips from TTStroke-21. Our method gets an average classification performance of 87.3% with a best run of 93.2% accuracy on the test set. When applied on joint detection and classification task, the proposed method reaches an accuracy of 82.6%.A systematic study of the influence of each stream and fusion types on classification accuracy has been performed, giving clues on how to obtain the best performances. A comparison of different optical flow methods and the role of their normalization on the classification score is also done. The extracted features are also analyzed by back-tracing strong features from the last convolutional layer to understand the decision path of the trained model. Finally, we introduce an attention mechanism to help the model focusing on particular characteristic features and also to speed up the training process. For comparison purposes, we provide performances of other methods on TTStroke-21 and test our model on other datasets. We notice that models performing well on coarse-grained action datasets do not always perform well on our fine-grained action dataset.The research presented in this manuscript was validated with publications in one international journal, five international conference papers, two international workshop papers and a reconductible task in MediaEval workshop in which participants can apply their action recognition methods to TTStroke-21. Two additional international workshop papers are in process along with one book chapter., La reconnaissance des actions à partir de vidéos est l'un des principaux problèmes de vision par ordinateur. Malgré des recherches intensives, la différenciation et la reconnaissance d'actions similaires restent un défi. Cette thèse porte sur la classification des gestes sportifs à partir de vidéos, avec comme cadre applicatif le tennis de table. Nous proposons une méthode d’apprentissage profond pour segmenter et classifier automatiquement les différents coup de Tennis de Table. Notre objectif est de concevoir un système intelligent permettant d'analyser les performances des élèves pongistes, et de donner la possibilité à l’entraîneur d'adapter ses séances d'entraînement pour améliorer leurs performances.Dans ce but, nous avons élaboré la base de données “TTStroke-21”, constituée de clips vidéo d'exercices de tennis de table, enregistrés par les étudiants de la faculté de sport de l'Université de Bordeaux – STAPS. Cette base de données a ensuite été annotée par des professionnels du domaine à l'aide d'une plateforme crowdsourcing. Les annotations consistent en une description des coups effectués (début, fin et type de coup). Au total, 20 différents coups de tennis de table sont considérés plus une classe de rejet. La reconnaissance des actions similaires présente des différences avec la reconnaissance d’actions classique. En effet, dans les bases de données classiques, le contexte de l’arrière plan fournit souvent des informations discriminantes que les méthodes peuvent utiliser pour classer l'action plutôt que de se concentrer sur l'action elle-même. Dans notre cas, la similarité entre classes est élevée, les caractéristiques visuelles discriminantes sont donc plus difficiles à extraire et le mouvement joue un rôle clef dans la caractérisation de l’action.Dans cette thèse, nous introduisons un réseau de neurones spatio-temporel convolutif avec une architecture Jumelle. Ce réseau d'apprentissage profond prend comme entrées une séquence d'images RVB et son flot optique estimé. Les données RVB permettent à notre modèle de capturer les caractéristiques d'apparence tandis que le flot optique capture les caractéristiques de mouvement. Ces deux flux sont traités en parallèle à l'aide de convolutions 3D, et sont fusionnés à la dernière étape du réseau. Les caractéristiques spatio-temporelles extraites dans le réseau permettent une classification efficace des clips vidéo de TTStroke-21. Notre méthode obtient une performance de classification de 93.2% sur l'ensemble des données tests. Appliquée à la tâche jointe de détection et de classification, notre méthode atteint une précision de 82.6%.Nous étudions les performances en fonction des types de données utilisés en entrée et la manière de les fusionner. Différents estimateurs de flot optique ainsi que leur normalisation sont testés afin d’améliorer la précision. Les caractéristiques de chaque branche de notre architecture sont également analysées afin de comprendre le chemin de décision de notre modèle. Enfin, nous introduisons un mécanisme d'attention pour aider le modèle à se concentrer sur des caractéristiques discriminantes et aussi pour accélérer le processus d’entraînement. Nous comparons notre modèle avec d'autres méthodes sur TTStroke-21 et le testons sur d'autres ensembles de données. Nous constatons que les modèles fonctionnant bien sur des bases de données d’actions classiques ne fonctionnent pas toujours aussi bien sur notre base de données d'actions similaires.Les travaux présentés dans cette thèse ont été validés par des publications dans une revue internationale, cinq papiers de conférences internationales, deux papiers d’un workshop international et une tâche reconductible dans le workshop MediaEval où les participants peuvent appliquer leurs méthodes de reconnaissance d'actions à notre base de données TTStroke-21. Deux autres papiers de workshop internationaux sont en cours de préparation, ainsi qu'un chapitre de livre.

Egocentric videos are becoming popular since the possibility to observe the scene flow from the user’s point of view (First Person Vision). Among the different applications of egocentric vision is the daily living monitoring of a user wearing the camera. We propose a system able to automatically organize egocentric videos acquired by the user over different days. Through an unsupervised temporal segmentation, each egocentric video is divided in chapters by considering the visual content. The obtained video segments related to the different days are hence connected according to the scene context in which the user acts. Experiments on a challenging egocentric video dataset demonstrate the effectiveness of the proposed approach that outperforms with a good margin the state of the art in accuracy and computational time.

International audience; This paper introduces a complete framework for temporal video segmentation. First, a computationally efficient shot extraction method is introduced, which adopts the normalized graph partition approach, enriched with a non-linear, multiresolution filtering of the similarity vectors involved. The shot boundary detection technique proposed yields high precision (90%) and recall (95%) rates, for all types of transitions, both abrupt and gradual. Next, for each detected shot we construct a static storyboard, by introducing a leap keyframe extraction method. The video abstraction algorithm is 23% faster than existing, state of the art techniques, for similar performances. Finally, we propose a shot grouping strategy that iteratively clusters visually similar shots, under a set of temporal constraints. Two different types of visual features are here exploited: HSV color histograms and interest points. In both cases, the precision and recall rates present average performances of 86%.

International audience; We propose in this paper an original method of camera motion classification based on Transferable Belief Model (TBM). It consists in locating in a video the motions of translation and zoom, and the absence of camera motion (i.e static camera). The classification process is based on a rule-based system that is divided into three stages. From a parametric motion model, the first stage consists in combining data to obtain frame-level belief masses on camera motions. To ensure the temporal coherence of motions, a filtering of belief masses according to TBM is achieved. The second stage carries out a separation between static and dynamic frames. In the third stage, a temporal integration allows the motion to be studied on a set of frames and to preserve only those with significant magnitude and duration. Then, a more detailed description of each motion is given. Experimental results obtained show the effectiveness of the method.

The goal of the project is to adapt the LIRE image retrieval system to videos. This project explores the expansion of Lucene Image Retrieval Engine (LIRE), an open-source Content-Based Image Retrieval (CBIR) system, for video retrieval on large scale video datasets. The fast growth of the need to store huge amounts of video in servers requires efficient, scalable search and indexing engines capable to assist users in their management and retrieval. In our tool, queries are formulated by visual examples allowing users to find the videos and the moment of time when the query image is matched with. The video dataset used on this scenario comprise over 1,000 hours of different news broadcast channels. This thesis presents an extension and adaptation of Lire and its plugin for Solr, an open-source enterprise search platform from the Apache Lucene project, for video retrieval based on visual features, as well as a web-interface for users from different devices. Este proyecto investiga el potencial de LIRE, una librería de código abierto para consulta de imágenes mediante ejemplo (CBIR), como herramienta de consulta e indexación para grandes bases de datos de vídeos. El rápido crecimiento de las necesidades de almacenar enormes cantidades de vídeos en diversos servidores requiere un motor de búsqueda e indexación escalable y eficiente, así como una herramienta que proporcione a los usuarios métodos para la gestión y consulta de contenido. En nuestra herramienta, las consultas se formulan mediante ejemplos visuales, permitiendo a los usuarios encontrar los vídeos y los segundos a los que pertenece un fotograma en concreto. La recopilación de vídeos utilizados en este escenario comprende más de 1000 horas de vídeo recopilados de diferentes canales de noticias internacionales. En esta tesis se presenta una extensión y adaptación de Lire y su plugin para Solr, una plataforma de búsqueda empresarial de código abierto del proyecto Apache Lucene, para la consulta e indexación de vídeos en servidores basado en características visuales, así como una interfaz web accesible para usuarios de diferentes dispositivos. Aquest projecte investiga el potencial de LIRE, una llibreria de codi obert per a consulta d'imatges mitjançant exemple (CBIR), com a eina de consulta i indexació per a grans bases de dades de vídeos. El ràpid creixement de les necessitats d'emmagatzemar enormes quantitats de vídeos en diversos servidors requereix un motor de cerca i indexació escalable i eficient, així com una eina que proporcioni als usuaris mètodes per a la gestió i consulta de contingut. En la nostra eina, les consultes es formulen mitjançant exemples visuals, permetent als usuaris trobar els vídeos i els segons als quals pertany un fotograma en concret. La recopilació de vídeos utilitzats en aquest escenari comprèn més de 1000 hores de vídeo recopilats de diferents canals de notícies internacionals. En aquesta tesi es presenta una extensió i adaptació de Lire i el seu plugin per Solr, una plataforma de cerca empresarial de codi obert del projecte Apache Lucene, per a la consulta i indexació de vídeos en servidors basat en característiques visuals, així com una interfície web accessible per a usuaris de diferents dispositius.

Recognizing human action in complex video sequences has always been challenging for researchers due to articulated movements, occlusion, background clutter, and illumination variation. Human action recognition has wide range of applications in surveillance, human computer interaction, video indexing and video annotation. In this paper, a discrete cosine transform based features have been exploited for action recognition. First, motion history image is computed for a sequence of images and then blocked-based truncated discrete cosine transform is computed for motion history image. Finally, K-Nearest Neighbor (K-NN) classifier is used for classification. This technique exhibits promising results for KTH and Weizmann dataset. Moreover, the proposed model appears to be computationally efficient and immune to illumination variations; however, this model is prone to viewpoint variations

This work addresses the development of a unified approach to content-based indexing and retrieval of digital videos from television archives. The proposed approach has been designed to deal with arbitrary television genres, making it suitable for various applications. To achieve this goal, the main steps of a content-based video retrieval system are addressed in this work, namely: video segmentation, key-frame extraction, content-based video indexing and the video retrieval operation itself. Video segmentation is addressed as a typical TV broadcast structuring problem, which consists in automatically determining the boundaries of each broadcasted program (like movies, news, among others) and inter-program (for instance, commercials). Specifically, to segment the videos, Electronic Program Guide (EPG) metadata is combined with the detection of two special cues, namely, audio cuts (silence) and dark monochrome frames. On the other hand, a color histogram-based approach performs key-frame extraction. Video indexing and retrieval are accomplished by using hashing and k-d tree methods, while visual signatures containing color, shape and texture information are estimated for the key-frames, by using image and frequency domain techniques. Experimental results with the dataset of a multimedia information system especially developed for managing television broadcast archives demonstrate that our approach works efficiently, retrieving videos in 0.16 seconds on average and achieving recall, precision and F1 measure values, as high as 0.76, 0.97 and 0.86 respectively.

In this work we present M-VIVIE, a system for video sequence indexing based on the identity of appearing subjects, which exploits a multithread architecture for fast processing. The system is composed of more conceptual component modules, each performing a specific kind of processing. Each module can possibly be substituted with a different one performing the same task. Special attention is devoted to classification and clustering activities. M-VIVIE presents a number of peculiarities, that distinguish it from existing systems with similar goals. Among these, the account for the concomitant appearance of two identities in the same clip, and the use of such information in the process of identity mapping. M-VIVIE was tested on different kinds of real video clips to assess both its efficacy and efficiency.

International audience; This paper presents a method for indexing activities of daily living in videos obtained from wearable cameras. In the context of dementia diagnosis by doctors, the videos are recorded at patients' houses and later visualized by the medical practitioners. The videos may last up to two hours, therefore a tool for an efficient navigation in terms of activities of interest is crucial for the doctors. The specific recording mode provides video data which are really difficult, being a single sequence shot where strong motion and sharp lighting changes often appear. Our work introduces an automatic motion based segmentation of the video and a video structuring approach in terms of activities by a hierarchical two-level Hidden Markov Model. We define our description space over motion and visual characteristics of video and audio channels. Experiments on real data obtained from the recording at home of several patients show the difficulty of the task and the promising results of our approach.

This paper presents a method for indexing human ac- tivities in videos captured from a wearable camera being worn by patients, for studies of progression of the dementia diseases. Our method aims to produce indexes to facilitate the navigation throughout the individual video recordings, which could help doctors search for early signs of the dis- ease in the activities of daily living. The recorded videos have strong motion and sharp lighting changes, inducing noise for the analysis. The proposed approach is based on a two steps analysis. First, we propose a new approach to segment this type of video, based on apparent motion. Each segment is characterized by two original motion de- scriptors, as well as color, and audio descriptors. Second, a Hidden-Markov Model formulation is used to merge the multimodal audio and video features, and classify the test segments. Experiments show the good properties of the ap- proach on real data., Comment: 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI), Madrid : Spain (2011)

I review my research activities in Video Indexing and Action Recognition and sketch a research agenda for bringing those two lines of research together to address the difficult problem of recognizing actions in movies. I first present a series of older projects in Video Indexing, starting with the DIVAN project at INA and the MPEG expert group (1998-2000), and continuing at INRIA under the VIBES project (2001-2004). This research falls under the general approach of "computational media aesthetics", where we attempt to recognize film events based on our knowledge of filming styles and conventions (cinematography and editing). This is illustrated with two applications - the automatic segmentation of TV news into topics; and the automatic indexing of movies with their scripts. I then present my more recent research in Action Recognition with the MOVI group at INRIA (2005-2008). Building upon the GRIMAGE infrastructure, I present experiments in (1) learning and recognizing a small repertoire of full body gestures in 3D using "motion history volumes"; (2) segmenting a raw stream of 3D image sequences into recognizable "primitive actions" ; and (3) using statistical models learned in 3D for recovering primitive actions and relative camera positions from a single 2D video recording of similar actions.; Je présente mes activités de recherche en indexation video et en reconnaissance d'actions, et je propose un programme de recherche permettant d'aborder ensemble ces deux questions au cours des prochaines années. Je décris d'abord une série de travaux réalisés dans le cadre du groupe MPEG et des projets DIVAN à l'INA (1998-2000), puis VIBES à l'INRIA (2001-2004), et qui visent à aborder l'indexation video à travers la reconnaissance des styles et conventions de la prise de vues et du montage. Cette première partie est illustrée par deux applications - le découpage d'un journal télévisé en sujets, et l'indexation d'un film de cinéma par son script. Je présente ensuite des travaux réalisés à l'INRIA en 2005-2008 au sein de l'équipe MOVI. Je montre comment nous avons utilisé l'infrastructure GRIMAGE pour (1) apprendre des modèles statistiques 3D d'un petit répertoire d'actions humaines permettant de les reconnaitre lorsqu'elles sont exécutées par d'autres acteurs, sous d'autres points de vue ; (2) découper une séquence d'images 3D en actions primitives reconnaissables; et (3) reconnaître ces mêmes actions selon le point de vue d'une seule caméra. Enfin, je propose quelques pistes pour étendre les résultats précédents afin d'aborder simultanément les deux problèmes de la reconnaissance des actions et des styles de mise en scène dans les films. Je présente les avantages et les difficultés d'une approche unifiée de ces deux problèmes, ainsi que des applications possibles dans les domaines de la fiction interactive, du jeu vidéo et du machinima.

I review my research activities in Video Indexing and Action Recognition and sketch a research agenda for bringing those two lines of research together to address the difficult problem of recognizing actions in movies. I first present a series of older projects in Video Indexing, starting with the DIVAN project at INA and the MPEG expert group (1998-2000), and continuing at INRIA under the VIBES project (2001-2004). This research falls under the general approach of "computational media aesthetics", where we attempt to recognize film events based on our knowledge of filming styles and conventions (cinematography and editing). This is illustrated with two applications - the automatic segmentation of TV news into topics; and the automatic indexing of movies with their scripts. I then present my more recent research in Action Recognition with the MOVI group at INRIA (2005-2008). Building upon the GRIMAGE infrastructure, I present experiments in (1) learning and recognizing a small repertoire of full body gestures in 3D using "motion history volumes"; (2) segmenting a raw stream of 3D image sequences into recognizable "primitive actions" ; and (3) using statistical models learned in 3D for recovering primitive actions and relative camera positions from a single 2D video recording of similar actions.; Je présente mes activités de recherche en indexation video et en reconnaissance d'actions, et je propose un programme de recherche permettant d'aborder ensemble ces deux questions au cours des prochaines années. Je décris d'abord une série de travaux réalisés dans le cadre du groupe MPEG et des projets DIVAN à l'INA (1998-2000), puis VIBES à l'INRIA (2001-2004), et qui visent à aborder l'indexation video à travers la reconnaissance des styles et conventions de la prise de vues et du montage. Cette première partie est illustrée par deux applications - le découpage d'un journal télévisé en sujets, et l'indexation d'un film de cinéma par son script. Je présente ensuite des travaux réalisés à l'INRIA en 2005-2008 au sein de l'équipe MOVI. Je montre comment nous avons utilisé l'infrastructure GRIMAGE pour (1) apprendre des modèles statistiques 3D d'un petit répertoire d'actions humaines permettant de les reconnaitre lorsqu'elles sont exécutées par d'autres acteurs, sous d'autres points de vue ; (2) découper une séquence d'images 3D en actions primitives reconnaissables; et (3) reconnaître ces mêmes actions selon le point de vue d'une seule caméra. Enfin, je propose quelques pistes pour étendre les résultats précédents afin d'aborder simultanément les deux problèmes de la reconnaissance des actions et des styles de mise en scène dans les films. Je présente les avantages et les difficultés d'une approche unifiée de ces deux problèmes, ainsi que des applications possibles dans les domaines de la fiction interactive, du jeu vidéo et du machinima.

Many algorithms for temporal video partitioning rely on the analysis of uncompressed video features. Since the information relevant to the partitioning process can be extracted directly from the MPEG compressed stream, higher efficiency can be achieved utilizing information from the MPEG compressed domain. This paper introduces a real-time algorithm for scene change detection that analyses the statistics of the macroblock features extracted directly from the MPEG stream. A method for extraction of the continuous frame difference that transforms the 3D video stream into a 1D curve is presented. This transform is then further employed to extract temporal units within the analysed video sequence. Results of computer simulations are reported.

Visual database systems require efficient indexing to enable fast access to the images and video. In addition, the large memory capacity and channel bandwidth requirements for the storage and transmission of visual data necessitate the use of compression techniques. Future multimedia applications are likely to increasingly store and transmit the visual information in compressed form. Hence indexing the visual content in compressed domain is expected to result in significant savings in computational complexity. Vector quantization (VQ) is an efficient technique for low bit rate image and video compression. In addition, the lower complexity of the decoder makes VQ attractive for low power systems and applications which require fast decoding. Most importantly, VQ is naturally an indexing technique, where a block of pixels is compactly represented using an index (label) which corresponds to a codebook. In this thesis, we propose the novel concept of using VQ for joint compression and indexing of images and video. The images/image frames are compressed using VQ and the labels and codewords are employed in indexing the visual content. First, we present a review of image/video compression and indexing. We then propose two techniques in the VQ compressed domain for image indexing. In the first technique, the histogram of codewords weighted by the number of labels is used as feature vector for indexing. In the second technique, the histogram of the labels, which are used to represent an image, is used as an index. We also propose a new technique based on adaptive wavelet VQ, which provides an improvement in coding and retrieval performance. Here, the images are decomposed using wavelet transform followed by VQ of the transform coefficients. A usage map of codewords is generated for each image and is stored along with the image. In the retrieval process, the usage map of the query image (VQ encoded) is compared with the corresponding usage maps of the target images in the database. Since video has both spatial and temporal dimensions, a straightforward extension of the image indexing techniques for video indexing is inefficient. We propose to employ both the spatial and temporal features for efficient indexing of video clips. The video sequence is partitioned into shots using the label maps of the individual frames and the camera operations and motion within each shot are then determined by further processing the label maps. Each shot is then represented using a spatio-temporal index. The spatial index represents the content of the key frame (image) of a shot, while the temporal index represents the motion and camera operations within the shot. Detailed simulations have been carried out using a large database of images and video sequences. Simulation results demonstrate the excellent retrieval performance of the proposed techniques at a significantly reduced computational complexity.

Action recognition in videos is one of the key problems in visual data interpretation. Despite intensive research, differencing and recognizing similar actions remains a challenge. This thesis deals with fine-grained classification of sport gestures from videos, with an application to table tennis.In this manuscript, we propose a method based on deep learning for automatically segmenting and classifying table tennis strokes in videos. Our aim is to design a smart system for students and teachers for analyzing their performances. By profiling the players, a teacher can therefore tailor the training sessions more efficiently in order to improve their skills. Players can also have an instant feedback on their performances.For developing such a system with fine-grained classification, a very specific dataset is needed to supervise the learning process. To that aim, we built the “TTStroke-21” dataset, which is composed of 20 stroke classes plus a rejection class. The TTStroke-21 dataset comprises video clips of recorded table tennis exercises performed by students at the sport faculty of the University of Bordeaux - STAPS. These recorded sessions were annotated by professional players or teachers using a crowdsourced annotation platform. The annotations consist in a description of the handedness of the player and information for each stroke performed (starting and ending frames, class of the stroke). Fine-grained action recognition has some notable differences with coarse-grained action recognition. In general, datasets used for coarse-grained action recognition, the background context often provides discriminative information that methods can use to classify the action, rather than focusing on the action itself. In fine-grained classification, where the inter-class similarity is high, discriminative visual features are harder to extract and the motion plays a key role for characterizing an action.In this thesis, we introduce a Twin Spatio-Temporal Convolutional Neural Network. This deep learning network takes as inputs an RGB image sequence and its computed Optical Flow. The RGB image sequence allows our model to capture appearance features while the optical flow captures motion features. Those two streams are processed in parallel using 3D convolutions, and fused at the last stage of the network. Spatio-temporal features extracted in the network allow efficient classification of video clips from TTStroke-21. Our method gets an average classification performance of 87.3% with a best run of 93.2% accuracy on the test set. When applied on joint detection and classification task, the proposed method reaches an accuracy of 82.6%.A systematic study of the influence of each stream and fusion types on classification accuracy has been performed, giving clues on how to obtain the best performances. A comparison of different optical flow methods and the role of their normalization on the classification score is also done. The extracted features are also analyzed by back-tracing strong features from the last convolutional layer to understand the decision path of the trained model. Finally, we introduce an attention mechanism to help the model focusing on particular characteristic features and also to speed up the training process. For comparison purposes, we provide performances of other methods on TTStroke-21 and test our model on other datasets. We notice that models performing well on coarse-grained action datasets do not always perform well on our fine-grained action dataset.The research presented in this manuscript was validated with publications in one international journal, five international conference papers, two international workshop papers and a reconductible task in MediaEval workshop in which participants can apply their action recognition methods to TTStroke-21. Two additional international workshop papers are in process along with one book chapter.; La reconnaissance des actions à partir de vidéos est l'un des principaux problèmes de vision par ordinateur. Malgré des recherches intensives, la différenciation et la reconnaissance d'actions similaires restent un défi. Cette thèse porte sur la classification des gestes sportifs à partir de vidéos, avec comme cadre applicatif le tennis de table. Nous proposons une méthode d’apprentissage profond pour segmenter et classifier automatiquement les différents coup de Tennis de Table. Notre objectif est de concevoir un système intelligent permettant d'analyser les performances des élèves pongistes, et de donner la possibilité à l’entraîneur d'adapter ses séances d'entraînement pour améliorer leurs performances.Dans ce but, nous avons élaboré la base de données “TTStroke-21”, constituée de clips vidéo d'exercices de tennis de table, enregistrés par les étudiants de la faculté de sport de l'Université de Bordeaux – STAPS. Cette base de données a ensuite été annotée par des professionnels du domaine à l'aide d'une plateforme crowdsourcing. Les annotations consistent en une description des coups effectués (début, fin et type de coup). Au total, 20 différents coups de tennis de table sont considérés plus une classe de rejet. La reconnaissance des actions similaires présente des différences avec la reconnaissance d’actions classique. En effet, dans les bases de données classiques, le contexte de l’arrière plan fournit souvent des informations discriminantes que les méthodes peuvent utiliser pour classer l'action plutôt que de se concentrer sur l'action elle-même. Dans notre cas, la similarité entre classes est élevée, les caractéristiques visuelles discriminantes sont donc plus difficiles à extraire et le mouvement joue un rôle clef dans la caractérisation de l’action.Dans cette thèse, nous introduisons un réseau de neurones spatio-temporel convolutif avec une architecture Jumelle. Ce réseau d'apprentissage profond prend comme entrées une séquence d'images RVB et son flot optique estimé. Les données RVB permettent à notre modèle de capturer les caractéristiques d'apparence tandis que le flot optique capture les caractéristiques de mouvement. Ces deux flux sont traités en parallèle à l'aide de convolutions 3D, et sont fusionnés à la dernière étape du réseau. Les caractéristiques spatio-temporelles extraites dans le réseau permettent une classification efficace des clips vidéo de TTStroke-21. Notre méthode obtient une performance de classification de 93.2% sur l'ensemble des données tests. Appliquée à la tâche jointe de détection et de classification, notre méthode atteint une précision de 82.6%.Nous étudions les performances en fonction des types de données utilisés en entrée et la manière de les fusionner. Différents estimateurs de flot optique ainsi que leur normalisation sont testés afin d’améliorer la précision. Les caractéristiques de chaque branche de notre architecture sont également analysées afin de comprendre le chemin de décision de notre modèle. Enfin, nous introduisons un mécanisme d'attention pour aider le modèle à se concentrer sur des caractéristiques discriminantes et aussi pour accélérer le processus d’entraînement. Nous comparons notre modèle avec d'autres méthodes sur TTStroke-21 et le testons sur d'autres ensembles de données. Nous constatons que les modèles fonctionnant bien sur des bases de données d’actions classiques ne fonctionnent pas toujours aussi bien sur notre base de données d'actions similaires.Les travaux présentés dans cette thèse ont été validés par des publications dans une revue internationale, cinq papiers de conférences internationales, deux papiers d’un workshop international et une tâche reconductible dans le workshop MediaEval où les participants peuvent appliquer leurs méthodes de reconnaissance d'actions à notre base de données TTStroke-21. Deux autres papiers de workshop internationaux sont en cours de préparation, ainsi qu'un chapitre de livre.