Search

International audience; In this paper, we introduce a conformal prediction method to construct prediction sets in a oneshot federated learning setting. More specifically, we define a quantile-of-quantiles estimator and prove that for any distribution, it is possible to output prediction sets with desired coverage in only one round of communication. To mitigate privacy issues, we also describe a locally differentially private version of our estimator. Finally, over a wide range of experiments, we show that our method returns prediction sets with coverage and length very similar to those obtained in a centralized setting. Overall, these results demonstrate that our method is particularly well-suited to perform conformal predictions in a one-shot federated learning setting.

We consider the problem of minimizing a convex function over a closed convex set, with Projected Gradient Descent (PGD). We propose a fully parameter-free version of AdaGrad, which is adaptive to the distance between the initialization and the optimum, and to the sum of the square norm of the subgradients. Our algorithm is able to handle projection steps, does not involve restarts, reweighing along the trajectory or additional gradient evaluations compared to the classical PGD. It also fulfills optimal rates of convergence for cumulative regret up to logarithmic factors. We provide an extension of our approach to stochastic optimization and conduct numerical experiments supporting the developed theory.

We consider contextual bandit problems with knapsacks [CBwK], a problem where at each round, a scalar reward is obtained and vector-valued costs are suffered. The learner aims to maximize the cumulative rewards while ensuring that the cumulative costs are lower than some predetermined cost constraints. We assume that contexts come from a continuous set, that costs can be signed, and that the expected reward and cost functions, while unknown, may be uniformly estimated -- a typical assumption in the literature. In this setting, total cost constraints had so far to be at least of order $T^{3/4}$, where $T$ is the number of rounds, and were even typically assumed to depend linearly on $T$. We are however motivated to use CBwK to impose a fairness constraint of equalized average costs between groups: the budget associated with the corresponding cost constraints should be as close as possible to the natural deviations, of order $\sqrt{T}$. To that end, we introduce a dual strategy based on projected-gradient-descent updates, that is able to deal with total-cost constraints of the order of $\sqrt{T}$ up to poly-logarithmic terms. This strategy is more direct and simpler than existing strategies in the literature. It relies on a careful, adaptive, tuning of the step size.

This work provides several fundamental characterizations of the optimal classification function under the demographic parity constraint. In the awareness framework, akin to the classical unconstrained classification case, we show that maximizing accuracy under this fairness constraint is equivalent to solving a corresponding regression problem followed by thresholding at level $1/2$. We extend this result to linear-fractional classification measures (e.g., ${\rm F}$-score, AM measure, balanced accuracy, etc.), highlighting the fundamental role played by the regression problem in this framework. Our results leverage recently developed connection between the demographic parity constraint and the multi-marginal optimal transport formulation. Informally, our result shows that the transition between the unconstrained problems and the fair one is achieved by replacing the conditional expectation of the label by the solution of the fair regression problem. Finally, leveraging our analysis, we demonstrate an equivalence between the awareness and the unawareness setups in the case of two sensitive groups.

We study the forecasting of the power consumptions of a population of households and of subpopulations thereof. These subpopulations are built according to location, to exogenous information and/or to profiles we determined from historical households consumption time series. Thus, we aim to forecast the electricity consumption time series at several levels of households aggregation. These time series are linked through some summation constraints which induce a hierarchy. Our approach consists in three steps: feature generation, aggregation and projection. Firstly (feature generation step), we build, for each considering group for households, a benchmark forecast (called features), using random forests or generalized additive models. Secondly (aggregation step), aggregation algorithms, run in parallel, aggregate these forecasts and provide new predictions. Finally (projection step), we use the summation constraints induced by the time series underlying hierarchy to re-conciliate the forecasts by projecting them in a well-chosen linear subspace. We provide some theoretical guaranties on the average prediction error of this methodology, through the minimization of a quantity called regret. We also test our approach on households power consumption data collected in Great Britain by multiple energy providers in the Energy Demand Research Project context. We build and compare various population segmentations for the evaluation of our approach performance.

In the context of the high-dimensional Gaussian linear regression for ordered variables, we study the variable selection procedure via the minimization of the penalized least-squares criterion. We focus on model selection where we propose to control predictive risk and False Discovery Rate simultaneously. For this purpose, we obtain a convenient trade-off thanks to a proper calibration of the hyperparameter K appearing in the penalty function. We obtain non-asymptotic theoretical bounds on the False Discovery Rate with respect to K. We then provide an algorithm for the calibration of K. It is based on completely observable quantities in view of applications. Our algorithm is validated by an extensive simulation study., 37 pages, 9 figures, 1 algorithm, 1 table

We consider the problem of unconstrained minimization of finite sums of functions. We propose a simple, yet, practical way to incorporate variance reduction techniques into SignSGD, guaranteeing convergence that is similar to the full sign gradient descent. The core idea is first instantiated on the problem of minimizing sums of convex and Lipschitz functions and is then extended to the smooth case via variance reduction. Our analysis is elementary and much simpler than the typical proof for variance reduction methods. We show that for smooth functions our method gives $\mathcal{O}(1 / \sqrt{T})$ rate for expected norm of the gradient and $\mathcal{O}(1/T)$ rate in the case of smooth convex functions, recovering convergence results of deterministic methods, while preserving computational advantages of SignSGD.

The increasing size of available data has led machine learning specialists to consider more complex models in order to achieve better performance. From a theoretical point of view, statistical learning under resource constraints has known a growing interest in the machine learning community. Many settings were developed to formalize budgeted limitations. In this thesis, we are motivated by these challenges, where we consider classical learning problems under the ``frugal lens". First, we tackle support recovery in a sparse linear regression problem, with one pass over data. We develop an online greedy algorithm named "online orthogonal matching pursuit" that actively selects covariates in a sequential way, with guarantees on its computational complexity that is adaptive to the unknown magnitude of the regression coefficients. Second, we consider the problem of model selection aggregation of experts. We present procedures that achieve fast rates under various budgeted settings and discuss the attainability of fast rates in different settings. Third, we tackle the problem of online prediction of individual sequences, where no distributional assumption is made in the process of generating data. We consider some natural budgeted constraints on the number of experts used for prediction and the number of observed feedbacks. We develop new strategies for each setting and discuss the attainability of constant regrets. Finally, we consider the problem of fixed confidence best arm identification. Given a confidence level, the learner wants to identify the arm with the largest mean using the least number of queries possible. We suppose that simultaneous queries are possible and prove that significant improvement can be made with respect to the BAI standard algorithms by taking the unknown covariance of the arms into consideration., Depuis le début du développement de la théorie de l'apprentissage statistique, un intérêt particulier a été porté aux méthodes efficaces en temps de calcul ainsi qu'en espace de stockage nécessaire, afin qu'elles soient utilisables en pratique. Ceci a motivé plusieurs théoriciens à formaliser différents problèmes d'apprentissage statistique sous contrainte d'accès aux données et aux ressources computationnelles. Dans cette thèse, nous avons considéré plusieurs problèmes d'apprentissage statistiques et d'apprentissage séquentiel, sous différents types de contraintes. Le premier problème traité concerne la régression parcimonieuse sous une contrainte de nature computationnelle. Nous développons un algorithme effectuant un seul passage sur les données (celles-ci sont supposées arriver en temps réel) avec une limitation sur l'espace mémoire disponible. Le deuxième problème traité concerne l'agrégation d'experts. Nous revisitons ce problème dans le cas où l'accès aux données est limité et développons des méthodes permettant d'atteindre des taux rapides pour l'excès de risques. Le problème suivant concerne l'agrégation d'experts pour la prédiction des suites individuelles fixes. Nous introduisant un formalisme similaire à celui utilisé dans le problème précédent: nous supposons que pour chaque tour, le joueur a une contrainte sur le nombre d'experts à utiliser pour la prédiction et une contrainte sur le nombre de pertes d'experts individuels observées après avoir fait une prédiction. Nous présentant des procédures pour chaque cas et développons des garanties théoriques sur le regret cumulé des stratégies présentées. Le dernier problème considéré est une instance du problème de l'identification du meilleur bras dans le cadre de la théorie des bandits stochastiques. Nous présentons une extension du formalisme standard en permettant le tirage de plusieurs bras simultanément. Dans ce cadre nous montrons que de nouvelles bornes, potentiellement meilleurs que les bornes classiques, sont possibles, et nous présentons des procédures permettant de les atteindre.

The reliability of vehicles is a major issue for automotive manufacturers. In particular, mechanical fatigue is an important preoccupation of the design office. Indeed, fatigue is a complex phenomenon that depends on the design of the part (geometry, materials used), the manufacturing and on the external loads it is subjected to. In order to design a safety part against fatigue, the part is numerically modeled and a deterministic fatigue criterion is applied to identify potential weaknesses. If these criteria prove to be effective when evaluated on experimental test data with standardized specimens, they are less effective for rig tests with prototypes. This results in an increase in development costs and duration. In order to remedy this issue, car manufacturers seek new digital tools to better predict the fatigue risks on new design proposals. In this thesis, we build a fatigue database, based on information provided by Stellantis, gathering numerical results along with fatigue test reports on prototypes. Unsupervised machine learning methods are applied offering a better understanding of the structure of the database and the relations between the available features. Then, the application of supervised machine learning methods (logistic regression, random forests, kernel SVM...) allows to estimate fatigue criteria offering better predictions than the standard fatigue criterion. However, the binary labels in this classification task are affected by a completely asymmetric label noise. This motivates an original approach to fatigue criteria estimation based on Positive-Unlabeled learning (PU learning). This problem is studied from all angles: theory, methodology and application. First, new risk bounds, adapted to this specific framework, are proved. Then, we develop a practical methodology to estimate a PU classifier. Finally, the methodology is evaluated on simulated data and on the fatigue database. The prediction performances confirm the interest of the methodology and its utility for car manufacturers., La fiabilité des véhicules est un enjeu majeur pour les constructeurs automobiles. En particulier, la fatigue mécanique est une préoccupation importante du bureau d'études. En effet, la fatigue est un phénomène complexe qui dépend du design de la pièce (géométrie, matériaux utilisés), des procédés de fabrication, et des chargements externes subis par la pièce. Le dimensionnement à la fatigue repose sur une modélisation numérique de la pièce et sur l'application de critères de fatigue déterministes afin d'identifier de potentielles faiblesses sur la conception. Ces critères, bien qu'efficaces sur des géométries simples, ne suffisent pas à prédire correctement les risques d'amorçage sur des composants complexes. Cela entraîne un allongement des temps de développement et une augmentation des coûts liés aux prototypes physiques. Pour y remédier, les constructeurs automobiles recherchent de nouvelles méthodes digitales, pour mieux identifier les zones critiques sur de nouvelles conceptions. Dans cette thèse, nous construisons une base de données fatigue, à partir d'informations mises à disposition par Stellantis, regroupant des résultats numériques et des comptes rendus d'essais de fatigue. Une analyse non supervisée du jeu de données est réalisée, permettant de mieux comprendre sa structure ainsi que les liens entre les covariables disponibles. Ensuite, l'application de méthodes d'apprentissage supervisé (régression logistique, forêts aléatoires, SVM à noyau...) permet d'estimer des critères de fatigue offrant de meilleures prédictions que le critère mécanique déterministe usuel. Une difficulté de l'analyse provient du fait que l'étiquetage des zones est affecté par un bruit asymétrique, ce qui motive une approche originale fondée sur l'apprentissage positif-non labellisé (PU learning). Cette approche est abordée suivant tous les angles: théorique, méthodologique et appliqué. De nouvelles bornes de risques adaptées à ce cadre spécifique sont démontrées. Une méthodologie est proposée pour l'estimation d'un classifieur PU à partir des données. Enfin, la méthodologie est évaluée sur des jeux de données simulés ainsi que sur les données de fatigue. Les performances obtenues confirment l'intérêt de la méthode et son utilité pour le constructeur automobile.

Thanks to its new connected trains, Transilien is now able to measure the number of passengers boardings and alightings per train door in real time.Our research uses this unique dataset to contribute to a better synchronization of train and passenger flows during the operational stage of railway operations.We first evaluate several statistical learning models to estimate dwell time as a function of railway operation and passenger flows variables. These models allow us to isolate critical situations where passenger flows significantly impact dwell time. Our research forecasts each of the railway operation and passenger flows variables, one stop ahead, from bidirectional autoregressive models exploiting their recent past. We then simplify those models’ using patterns derived from timetables. Finally, we estimate the occupancy rate by zone of the open gangways rolling stocks in order to inform passengers about the comfort on board and build two models of on-board passenger movements., Grâce à ses rames connectées, Transilien mesure en temps réel le nombre de montées et de descentes par porte du train. Nous contribuons à une meilleure synchronisation en phase opérationnelle des flux de trains et de voyageurs à l’aide de ces données uniques. Nous évaluons plusieurs modèles d’apprentissage statistique afin d’estimer les temps de stationnement en fonction des variables d’exploitation ferroviaire et des flux de voyageurs. Ces modèles permettent d’isoler des situations critiques où les flux de voyageurs impactent les temps de stationnement. Nous prévoyons chacune des variables, à l’horizon d’un arrêt, à partir de modèles autorégressifs bidirectionnels exploitant leur passé proche. Ces modèles se simplifient grâce aux motifs issus de la grille horaire. Nous estimons enfin des taux d’occupation par zone des rames traversantes, afin d’informer les voyageurs sur le confort à bord et proposons deux modèles de déplacement des voyageurs à bord.

We consider the idealized setting of gradient flow on the population risk for infinitely wide two-layer ReLU neural networks (without bias), and study the effect of symmetries on the learned parameters and predictors. We first describe a general class of symmetries which, when satisfied by the target function $ f^* $ and the input distribution, are preserved by the dynamics. We then study more specific cases. When $ f^* $ is odd, we show that the dynamics of the predictor reduces to that of a (non-linearly parameterized) linear predictor, and its exponential convergence can be guaranteed. When $ f^* $ has a low-dimensional structure, we prove that the gradient flow PDE reduces to a lower-dimensional PDE. Furthermore, we present informal and numerical arguments that suggest that the input neurons align with the lower-dimensional structure of the problem.

International audience; We lay the foundations of a non-parametric theory of best-arm identification in multi-armed bandits with a fixed budget T. We consider general, possibly non-parametric, models D for distributions over the arms; an overarching example is the model D = P(0,1) of all probability distributions over [0,1]. We propose upper bounds on the average log-probability of misidentifying the optimal arm based on information-theoretic quantities that correspond to infima over Kullback-Leibler divergences between some distributions in D and a given distribution. This is made possible by a refined analysis of the successive-rejects strategy of Audibert, Bubeck, and Munos (2010). We finally provide lower bounds on the same average log-probability, also in terms of the same new information-theoretic quantities; these lower bounds are larger when the (natural) assumptions on the considered strategies are stronger. All these new upper and lower bounds generalize existing bounds based, e.g., on gaps between distributions.

The development of electric vehicles (EV) is a major lever towards low-carbon transport. It comes with a growing number of charging infrastructures that can be used as flexible assets for the grid. To enable this smart-charging, an effective daily forecast of the charging behavior is necessary. In this context, the objective of this thesis is threefold: (a) to identify current modeling techniques and open data available (b) to propose new EV charging methodologies to characterize their charging behaviours (c) to specify innovative techniques for daily peak load forecasting. The first chapter of the manuscript presents the industrial issues and introduces the modeling framework for EV charging. Chapter 2 is a review of state of the art EV load models as well as an exploration of 8 open charging session datasets. Chapter 3 offers a comparative study of 14 EV load and occupancy models on the 8 datasets presented in the previous chapter. Chapter 4 introduces a model for EV arrivals as a non-homogeneous Poisson process with additive spline and wavelet effects. Finally, Chapter 5 introduces a model for daily electrical peaks with a multi-resolution approach. We show that the approaches proposed in our work are competitive with the best existing alternatives by evaluating their performance on real-world data.; Le développement des véhicules électriques (VEs) est un levier majeur vers un transport bas carbone. Il s'accompagne d'un nombre croissant d'infrastructures de recharge qui peuvent être utilisées comme actifs flexibles de gestion du réseau. Pour permettre cette recharge intelligente, une prévision journalière efficace des comportements de charge est nécessaire. Dans ce contexte, l'objectif de cette thèse est triple : (a) identifier les techniques de modélisation actuelles et les données ouvertes disponibles (b) proposer de nouvelles méthodologies de charge des VE pour caractériser leurs comportements de charge (c) spécifier des techniques innovantes pour la prévision des pics de consommation. Le premier chapitre du manuscrit présente les enjeux industriels et introduit le cadre de la modélisation de la charge des VE. Le chapitre 2 présente un examen approfondi des modèles de charge de VE à l'état de l'art ainsi qu'une exploration de 8 jeux de données ouverts de sessions de recharge trouvés dans cette recherche. Le chapitre 3 propose une étude comparative de 14 modèles de charge et d'occupation des VE sur les 8 jeux de données présentés au cours du chapitre précédent. Le chapitre 4 propose un modèle pour les arrivées des VE aux points de charge sous la forme d'un processus de Poisson non homogène avec des effets additifs projetés sur des bases de splines et d'ondelettes. Enfin, le chapitre 5 présente un modèle pour la prévision journalière des pics de demande électrique avec une approche multi-résolution. Nous montrons que les approches proposées dans nos travaux sont compétitives avec les meilleures alternatives existantes en évaluant leurs performances sur des données réelles.

National audience; Positive-Unlabeled learning (PU learning) is a binary classification task where only a subset of positive instances are labeled. The objective is then to find the correct classifier using positive labeled instances and unlabeled instances that contain a mixture of positive and negative data. In this paper, we illustrate some recent results about the convergence rates of PU learning under the general Selected At Random assumption, meaning that the labeled instances are not assumed to be a representative sample of the positive instances. We show that the simulations support the theoretical results highlighting the two regimes of convergence. We finally extend the simulations by relaxing some assumptions.

In this thesis, we first focus on two stochastic bandit problems. The first problem deals with the following resource allocation situation: an agent must sequentially choose T actions indexed by covariates among a larger set of N choices. Each time she chooses an action, she receives a payment depending on the action chosen in a non-parametric way, and this action is removed from the set of choices. In the second problem, we assume that the payment received is a linear function of a covariate describing the action, but that this payment is not observed by the agent. Instead, she observes an unfair evaluation of this payment, systematically biased against a set of actions. We design for each problem an algorithm, and we provide upper bounds on their regret. We also establish lower bounds on the regret, showing that these algorithms are optimal up to a (poly)logarithmic factor. In a second time, we address the problem of link prediction in a partially observed network. To do so, we study different methods for estimating the matrix of connection probabilities between the nodes of the network. First, we develop a mixed coordinate gradient descent algorithm to efficiently and robustly estimate the connection probabilities in a sparse network in the presence of missing links and outliers. Then, we study the maximum likelihood estimator in the stochastic block model. We show that it is optimal in the minimax sense under a homogeneity assumption on the connection probabilities of the nodes. Since this estimator cannot be computed in polynomial time, we also study its variational approximation. We show that these two estimators are asymptotically equivalent.; Dans cette thèse, nous nous intéressons dans un premier temps à deux problèmes de bandits stochastiques. Le premier problème étudié modélise la situation d'allocation de ressources suivantes : un agent doit choisir séquentiellement T actions indexées par des covariables parmi un ensemble plus grand de N choix. Chaque fois qu'il choisit une action, il reçoit un paiement dépendant de façon non paramétrique de l'action choisie.; puis cette action est retirée de l'ensemble des choix. Dans le second problème, on suppose que le paiement reçu est une fonction linéaire d'une covariable décrivant l'action choisie, mais que celui-ci n'est pas observé par l'agent ; à la place, celui-ci observe une évaluation injuste de ce paiement, systématiquement biaisée à l'encontre d'un groupe d'actions. Nous proposons pour chaque problème un algorithme dont nous bornons le regret. Nous établissons également des bornes inférieures sur le regret, montrant que ces algorithmes sont optimaux à un facteur (poly)logarithmique près. Dans un second temps, nous abordons le problème de prévision de lien manquant dans un réseau partiellement observé. Pour ce faire, nous étudions différentes méthodes pour estimer la matrice de probabilités de connexion entre les nœuds du réseau. Nous développons tout d'abord un algorithme de descente de gradient mixte par coordonnées pour estimer robustement et efficacement les probabilités de connexion dans un réseau creux en présence de liens manquants et d'intrus. Ensuite, nous étudions l'estimateur du maximum de vraisemblance dans le modèle à blocs stochastiques, et nous montrons que celui-ci est optimal au sens minimax sous une hypothèse d'homogénéité sur les probabilités de connexion des nœuds. Cet estimateur n'étant pas calculable en temps polynomial, on étudie également son approximation variationnelle, et nous montrons que ces deux derniers estimateurs sont asymptotiquement équivalents.

International audience; We consider a suburban railway network line in the greater Paris area, with sub-branches. Trains of the line are equipped both with automatic vehicle localization (AVL) and automatic passenger counting (APC) devices, leading to a rich data set with simultaneous measurements of variables related to railway operations (arrival delay, dwell time) and passenger flows (numbers of passengers alighting and boarding, total load at departure). We aim for one-station-ahead forecasting of each of these five variables independently from each other. To do so, we build a bi-auto-regressive approach consisting of using the past values of the variable of interest along a first dimension, given by past stations along the train ride, and along a second dimension, given by past trains at the station. A building block of this approach is a train-station representation that accommodates different types of train services. We identify repeated patterns in this representation and exploit this fact. Indeed, the proposed bi-auto-regressive models are based on linear regressions whose coefficients depend on the stations and possibly only on the location of the train ride within a repeated pattern. This results in models that have a smaller complexity than extremely local models tailored to the timetables, with no significant decrease in accuracy.

We consider contextual bandits with knapsacks, with an underlying structure between rewards generated and cost vectors suffered. We do so motivated by sales with commercial discounts. At each round, given the stochastic i.i.d.\ context $\mathbf{x}_t$ and the arm picked $a_t$ (corresponding, e.g., to a discount level), a customer conversion may be obtained, in which case a reward $r(a,\mathbf{x}_t)$ is gained and vector costs $c(a_t,\mathbf{x}_t)$ are suffered (corresponding, e.g., to losses of earnings). Otherwise, in the absence of a conversion, the reward and costs are null. The reward and costs achieved are thus coupled through the binary variable measuring conversion or the absence thereof. This underlying structure between rewards and costs is different from the linear structures considered by Agrawal and Devanur [2016] (but we show that the techniques introduced in the present article may also be applied to the case of these linear structures). The adaptive policies exhibited solve at each round a linear program based on upper-confidence estimates of the probabilities of conversion given $a$ and $\mathbf{x}$. This kind of policy is most natural and achieves a regret bound of the typical order (OPT/$B$) $\sqrt{T}$, where $B$ is the total budget allowed, OPT is the optimal expected reward achievable by a static policy, and $T$ is the number of rounds., Thirty-sixth Conference on Neural Information Processing Systems, 2022, New Orleans, United States

International audience; Identifying the relevant variables for a classification model with correct confidence levels is a central but difficult task in high-dimension. Despite the core role of sparse logistic regression in statistics and machine learning, it still lacks a good solution for accurate inference in the regime where the number of features p is as large as or larger than the number of samples n. Here we tackle this problem by improving the Conditional Randomization Test (CRT). The original CRT algorithm shows promise as a way to output p-values while making few assumptions on the distribution of the test statistics. As it comes with a prohibitive computational cost even in mildly high-dimensional problems, faster solutions based on distillation have been proposed. Yet, they rely on unrealistic hypotheses and result in low-power solutions. To improve this, we propose CRT-logit, an algorithm that combines a variable-distillation step and a decorrelation step that takes into account the geometry of $\ell_1$-penalized logistic regression problem. We provide a theoretical analysis of this procedure, and demonstrate its effectiveness on simulations, along with experiments on large-scale brain-imaging and genomics datasets.

We propose a theoretical framework for the problem of learning a real-valued function which meets fairness requirements. This framework is built upon the notion of $\alpha$-relative (fairness) improvement of the regression function which we introduce using the theory of optimal transport. Setting $\alpha = 0$ corresponds to the regression problem under the Demographic Parity constraint, while $\alpha = 1$ corresponds to the classical regression problem without any constraints. For $\alpha \in (0, 1)$ the proposed framework allows to continuously interpolate between these two extreme cases and to study partially fair predictors. Within this framework we precisely quantify the cost in risk induced by the introduction of the fairness constraint. We put forward a statistical minimax setup and derive a general problem-dependent lower bound on the risk of any estimator satisfying $\alpha$-relative improvement constraint. We illustrate our framework on a model of linear regression with Gaussian design and systematic group-dependent bias, deriving matching (up to absolute constants) upper and lower bounds on the minimax risk under the introduced constraint. We provide a general post-processing strategy which enjoys fairness, risk guarantees and can be applied on top of any black-box algorithm. Finally, we perform a simulation study of the linear model and numerical experiments of benchmark data, validating our theoretical contributions.

The current data revolution is characterized by the proliferation of massive data in all areas of economic activity and in sciences as well. In particular, this scientific data revolution concerns molecular biology. The study of gene expression in an organism is the key example in this thesis. Gene expression data are typically characterized by a high number of descriptive variables for a limited number of observations. Identifying the relevant variables is a crucial step for data exploitation and interpretation. This thesis focuses on the issue of variable selection in the statistical framework of high-dimensional Gaussian linear regression. The core of our analysis is based on introducing new penalty functions for the least squares adjustment criterion. These penalties depend on constants, considered as hyperparameters to be calibrated on the available data set. The originality of our approach lies in introducing the False Discovery Rate (FDR) to perform this calibration. First, we establish theoretical lower and upper bounds of the FDR when the variables are ordered. Then we set up a calibration algorithm of the hyperparameter to satisfy a trade-off between the predictive risk control and the FDR one. To select non-ordered variables in the high-dimension setting, we revisit the topic of variable selection via the minimization of a convex criterion such as the Lasso. We propose an approach to select the variables, which are ordered by the regularization path, via an adaptive penalization method. Intensive simulations show the interest in resampling and in non-asymptotic penalties. We generalize the adaptive penalty calibration method called "the slope heuristics" to calibrate two hyperparameters simultaneously and in the context of a random model collection. Finally, our new algorithm and some variable selection procedures are applied to a transcriptomic dataset of Arabidopsis thaliana. The biological problem here consists in identifying transcription factors of target genes., La révolution des données que nous connaissons aujourd'hui se caractérise par la prolifération de données massives dans tous les domaines d'activités économiques, mais aussi dans les sciences. Cette révolution des données scientifiques concerne en particulier la biologique moléculaire. L'étude de l'expression des gènes d'un organisme est l'exemple clé mis en avant dans cette thèse. Les données d'expression de gènes sont typiquement caractérisées par un nombre élevé de variables descriptives pour un nombre d'observations restant limité. Identifier les variables pertinentes constitue une étape cruciale pour l'exploitation des données ainsi que leur interprétation. Cette thèse est centrée sur la question de la sélection de variables dans le cadre statistique de la régression linéaire gaussienne en grande dimension. Le cœur de notre analyse repose sur l'introduction de nouvelles fonctions de pénalité pour le critère d'ajustement des moindres carrés. Celles-ci dépendent de constantes, que nous voyons comme des hyperparamètres à calibrer sur le jeu de données d'étude. L'originalité de notre approche réside en l'introduction du False Discovery Rate (FDR) pour réaliser cette calibration. Dans un premier temps, nous prouvons un encadrement théorique du FDR lorsque les variables sont ordonnées, puis nous mettons en place un algorithme de calibration de l'hyperparamètre pour satisfaire un compromis entre le contrôle du risque prédictif et celui du FDR. Pour sélectionner des variables non-ordonnées en grande dimension, nous revisitons le thème de la sélection de variables via la minimisation d'un critère convexe de type Lasso. Nous proposons une approche qui consiste à choisir les variables, ordonnées par le chemin de régularisation, via une méthode de pénalisation adaptative. Des simulations intensives mettent en évidence l'intérêt du ré-échantillonage et des pénalités non-asymptotiques. Nous généralisons la méthode de calibration adaptative de pénalité dite "de l'heuristique de pente" à la calibration de deux hyperparamètres simultanément ainsi qu'au contexte d'une collection de modèles aléatoires qui est ici le nôtre. Enfin, notre nouvel algorithme, ainsi que certaines procédures de sélection de variables, sont appliqués sur un jeu de données transcriptomiques d'Arabidopsis thaliana. L'identification des facteurs de transcription de gènes cibles constitue ici la problématique biologique.

We investigate the problem of cumulative regret minimization for individual sequence prediction with respect to the best expert in a finite family of size K under limited access to information. We assume that in each round, the learner can predict using a convex combination of at most p experts for prediction, then they can observe a posteriori the losses of at most m experts. We assume that the loss function is range-bounded and exp-concave. In the standard multi-armed bandits setting, when the learner is allowed to play only one expert per round and observe only its feedback, known optimal regret bounds are of the order O($\sqrt$ KT). We show that allowing the learner to play one additional expert per round and observe one additional feedback improves substantially the guarantees on regret. We provide a strategy combining only p = 2 experts per round for prediction and observing m $\ge$ 2 experts' losses. Its randomized regret (wrt. internal randomization of the learners' strategy) is of order O (K/m) log(K$\delta$ --1) with probability 1 -- $\delta$, i.e., is independent of the horizon T ("constant" or "fast rate" regret) if (p $\ge$ 2 and m $\ge$ 3). We prove that this rate is optimal up to a logarithmic factor in K. In the case p = m = 2, we provide an upper bound of order O(K 2 log(K$\delta$ --1)), with probability 1 -- $\delta$. Our strategies do not require any prior knowledge of the horizon T nor of the confidence parameter $\delta$. Finally, we show that if the learner is constrained to observe only one expert feedback per round, the worst-case regret is the "slow rate" $\Omega$($\sqrt$ KT), suggesting that synchronous observation of at least two experts per round is necessary to have a constant regret.

International audience; Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular , this includes a variant of the Lasso introduced by Zou, Hastié and Tibshirani. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.

With the increase in data acquisition and storage capabilities, developing efficient methods for processing graph-structured data has become a crucial issue in data science. We introduce and study new methods based on heat diffusion to compare graphs. The novelty of our approach essentially lies in the introduction of the concept of distance processes, where we consider the family of all distances computed over a continuous range of diffusion times for a given pair of graphs. This allows us to develop a multi-scale analysis of graphs. Moreover, by representing graphs via tools borrowed from topological data analysis, we are able to compare graphs of different sizes or unaligned graphs. The statistical properties of these processes are studied with the theory of empirical processes. We prove a functional central limit theorem (CLT), as well as a Gaussian approximation result allowing us to show that the convergence rate in the CLT is independent of the graphs' sizes. These results are general and can be applied to other processes. Moreover, they guarantee the asymptotic validity of resampling methods for constructing confidence bands and two-sample tests comparing graph populations. We study the performance of these tests on simulated data sets and apply them to the problem of distribution shift detection in the context of neural network learning., Avec l'augmentation des capacités d'acquisition et de stockage de données, le développement de méthodes efficaces pour le traitement de données à structure de graphe est devenu un point crucial pour les sciences des données. Nous introduisons et étudions de nouvelles méthodes de comparaison de graphes basées sur la diffusion de la chaleur. La nouveauté de notre approche réside essentiellement dans l'introduction du concept de processus de distances. Il s'agit de la famille de toutes les distances calculées sur une plage de temps de diffusion pour une paire de graphes donnée. Cela nous permet de développer une analyse multi-échelles des graphes. De plus, en représentant les graphes via des outils issus de l'analyse topologique des données, nous sommes en mesure de comparer des graphes de tailles différentes ou non alignés. L'étude des propriétés statistiques de ces processus se fait par la théorie des processus empiriques. Nous prouvons un théorème central limite (TCL) fonctionnel, ainsi qu'un résultat d'approximation gaussienne nous permettant de montrer que la vitesse de convergence dans le TCL est indépendante de la taille des graphes. Ces résultats sont généraux et peuvent être appliqués à d'autres processus. De plus, ils garantissent la validité asymptotique de méthodes de ré-échantillonage pour la construction de bandes de confiance et de tests à deux échantillons permettant de comparer des populations de graphes. Nous étudions les performances de ces tests sur des jeux de données simulés et nous les appliquons au problème de la détection de changement de distribution dans le cadre de l'apprentissage par réseaux de neurones.

International audience; We model dwell times for trains subject to a possibly dense timetable based on a rich data set containing both railway operations variables and passenger flows variables, which is rare in the literature. Another distinguishing feature of our modeling consists of building a single statistical model for actual dwell times at all stations and in all contexts, not just in constrained situations like late arrivals or not just a model for some minimum dwell time. These models are fully data-driven and stem from either linear regressions with multiplicative effects or machine-learning methods like random forests, both carefully tuned on training data sets. While railway operations variables remain key for the modeling of dwell time, we are able to characterize the added value of passenger flows variables. Overall, they lead to an average reduction of the global modeling error by about 0.5 s, with up to 5 s -- 10 s average improvements in challenging situations consisting, e.g., of late arrivals or associated with high affluence. We also study which are the most influential variables among the available operations and passenger flows variables, and we do so globally and by regime of punctuality: for instance, passenger flows variables, and in particular, the affluence at the critical door, are the most influential variables for trains suffering a late arrival, while the scheduled dwell time and the deviation to the scheduled arrival time are the most important variables for early trains.

This thesis deals with multiple testing problems in high-dimension, a regime which has become increasingly common nowadays in statistical inference. Its main goal is to provide efficient and reliable algorithms for multivariate inference, a hard problem that suffers from the curse of dimensionality. Our solutions improve on state-of-the-art methods, make them more stable and efficient while still maintaining their theoretical guarantees on controlling multiple testing metrics. Moreover, we show that our contributions perform reasonably well compared to the state-of-the-art in practical applications, considering analysis problems from life-sciences, such as neuroscience, medical imaging and genomics. In particular, we study the properties of knockoff filters, a method for controlling False Discovery Rate (FDR) that requires limited distribution assumptions. We then propose methods for aggregating several samplings to address knockoff filter's randomness, and prove non-asymptotic theoretical results on the aggregated knockoff, specifically guaranteed FDR control, which relies on some concentration inequalities. Furthermore, we extend the method, providing a version that can scale to extremely high dimensional regime. One of the key steps is to use randomized clustering to reduce the dimension to avoid the curse of dimensionality, and then to ensemble several runs to tame the bias from the selection of a fixed clustering. In order to take into account the compression of data that results from the clustering step, we introduce a spatial relaxation of the False Discovery Rate. Finally, we consider the problem of outputting p-values in the conditional inference for high-dimensional logistic regression. This method is a variant of the Conditional Randomization Test, with an additional decorrelation scheme that yields more accurate test statistics and more powerful than previous estimators. We conclude the thesis with a discussion about some open questions, which we believe are important and can serve as potential directions to further improve high-dimensional inference methods and their application to fields such as genomics or brain imaging.; Cette thèse traite des problèmes de tests multiples en grande dimension, un régime qui est devenu populaire dans l'inférence statistique moderne. Son objectif principal est de fournir des algorithmes efficaces et fiables pour l'inférence multivariée, un problème difficile qui souffre du fléau de la dimension. Nos solutions améliorent les méthodes de l'état de l'art, les rendent plus stables et efficaces tout en conservant leurs garanties théoriques sur le contrôle des métriques de tests multiples. De plus, nous montrons que nos contributions sont raisonnablement performantes par rapport à l'état de l'art dans des applications concrètes, à savoir des problèmes issus des sciences de la vie, comme les neurosciences, l'imagerie médicale et la génomique. En particulier, nous étudions les propriétés des filtres ''knockoff'', une méthode de contrôle du taux de fausses découvertes (False Discovery Rate -- FDR), qui nécessite peu d'hypothèses sur la loi des données. Nous proposons ensuite des méthodes d'agrégation de plusieurs échantillonnages pour traiter le caractère aléatoire du filtre knockoff, et prouvons des résultats théoriques non asymptotiques sur le knockoff agrégé, en particulier une garantie de contrôle du FDR, qui repose sur certaines inégalités de concentration. En outre, nous étendons la méthode, en fournissant une version qui peut s'adapter à un régime de dimension extrêmement élevée. L'une des étapes clés est l'utilisation d'un regroupement (clustering) aléatoire des covariables, afin d'éviter le fléau de la dimension, puis l'assemblage de plusieurs exécutions afin de limiter les biais causés par l'utilisation d'un seul regroupement. Afin de prendre en compte la compression des données qui résulte de l'étape de clustering, nous introduisons une relaxation spatiale du taux de fausses découvertes. Enfin, nous considérons le problème de construction de p-valeurs pour l'inférence conditionnelle avec la régression logistique en grande dimension. Cette méthode est une variante du test de randomisation conditionnel, avec un schéma de décorrélation supplémentaire qui donne des statistiques de test plus précises et plus puissantes que les estimateurs précédents. Nous concluons la thèse par une discussion sur certaines questions ouvertes, qui nous semblent importantes et peuvent servir de directions de travail pour améliorer les méthodes d'inférence en grande dimension et leur application à des domaines tels que la génomique ou l'imagerie cérébrale.

International audience; In the context of smart grids and load balancing, daily peak load forecasting has become a critical activity for stakeholders of the energy industry. An understanding of peak magnitude and timing is paramount for the implementation of smart grid strategies such as peak shaving. The modelling approach proposed in this paper leverages high-resolution and low-resolution information to forecast daily peak demand size and timing. The resulting multi-resolution modelling framework can be adapted to different model classes. The key contributions of this paper are a) a general and formal introduction to the multi-resolution modelling approach, b) a discussion on modelling approaches at different resolutions implemented via Generalised Additive Models and Neural Networks and c) experimental results on real data from the UK electricity market. The results confirm that the predictive performance of the proposed modelling approach is competitive with that of low-and high-resolution alternatives.

We consider the twin problems of estimating the effective rank and the Schatten norms $\|{\bf A}\|_{s}$ of a rectangular $p\times q$ matrix ${\bf A}$ from noisy observations. When $s$ is an even integer, we introduce a polynomial-time estimator of $\|{\bf A}\|_s$ that achieves the minimax rate $(pq)^{1/4}$. Interestingly, this optimal rate does not depend on the underlying rank of the matrix. When $s$ is not an even integer, the optimal rate is much slower. A simple thresholding estimator of the singular values achieves the rate $(q\wedge p)(pq)^{1/4}$, which turns out to be optimal up to a logarithmic multiplicative term. The tight minimax rate is achieved by a more involved polynomial approximation method. This allows us to build estimators for a class of effective rank indices. As a byproduct, we also characterize the minimax rate for estimating the sequence of singular values of a matrix., Comment: 67 pages

International audience; We study supervised and semi-supervised algorithms in the set-valued classification framework with controlled expected size. While the former methods can use only n labeled samples, the latter are able to make use of N additional unlabeled data. We obtain semi-supervised minimax rates of convergence under the α-margin assumption and a β-Hölder condition on the conditional distribution of labels. Our analysis implies that if no further assumption is made, there is no supervised method that outperforms the semi-supervised estimator proposed in this work – the best achievable rate for any supervised method is O(n−1/2), even if the margin assumption is extremely favorable; on the contrary, the developed semi-supervised estimator can achieve faster O((n/logn)−(1+α)β/(2β+d)) rate of convergence provided that sufficiently many unlabeled samples are available. We also show that under additional smoothness assumption, supervised methods are able to achieve faster rates and the unlabeled sample cannot improve the rate of convergence. Finally, a numerical study supports our theory and emphasizes the relevance of the assumptions we required from an empirical perspective.

International audience; In order to reliably design automotive structures, engineers need to determine and justify validation conditions and levels. These must stem from a thorough knowledge of structural damage induced by service loading conditions. From multi-input variable amplitude loading histories applied on a car's wheel axles, we propose a multidimensional pseudo-damage description for the design of car chassis weak points. We present a multivariate description of client loading histories. We use it in a statistical analysis of a labelled measurement campaign to explain the heterogeneity of car driver profiles. Finally, we explore the question of complex load damage reconstruction using proving ground reference loads.

International audience; Variational methods are extremely popular in the analysis of network data. Statistical guarantees obtained for these methods typically provide asymptotic normality for the problem of estimation of global model parameters under the stochastic block model. In the present work, we consider the case of networks with missing links that is important in application and show that the variational approximation to the maximum likelihood estimator converges at the minimax rate. This provides the first minimax optimal and tractable estimator for the problem of parameter estimation for the stochastic block model with missing links. We complement our results with numerical studies of simulated and real networks, which confirm the advantages of this estimator over current methods.

Variable selection methods are widely used in molecular biology to detect biomarkers or to infer gene regulatory networks from transcriptomic data. Methods are mainly based on the high-dimensional Gaussian linear regression model and we focus on this framework for this review. We propose a comparison study of variable selection procedures from regularization paths by considering three simulation settings. In the first one, the variables are independent allowing the evaluation of the methods in the theoretical framework used to develop them. In the second setting, two structures of the correlation between variables are considered to evaluate how biological dependencies usually observed affect the estimation. Finally, the third setting mimics the biological complexity of transcription factor regulations, it is the farthest setting from the Gaussian framework. In all the settings, the capacity of prediction and the identification of the explaining variables are evaluated for each method. Our results show that variable selection procedures rely on statistical assumptions that should be carefully checked. The Gaussian assumption and the number of explaining variables are the two key points. As soon as correlation exists, the regularization function Elastic-net provides better results than Lasso. LinSelect, a non-asymptotic model selection method, should be preferred to the eBIC criterion commonly used. Bolasso is a judicious strategy to limit the selection of non explaining variables., 15 pages, 5 tables

International audience; This work studies the problem of binary classification with the F-score as the performance measure. We propose a post-processing algorithm for this problem which fits a threshold for any score base classifier to yield high F-score. The post-processing step involves only unlabeled data and can be performed in logarithmic time. We derive a general finite sample post-processing bound for the proposed procedure and show that the procedure is minimax rate optimal, when the underlying distribution satisfies classical nonparametric assumptions. This result improves upon previously known rates for the F-score classification and bridges the gap between standard classification risk and the F-score. Finally, we discuss the generalization of this approach to the set-valued classification.

International audience; We provide a setting and a general approach to fair online learning with stochastic sensitive and non-sensitive contexts. The setting is a repeated game between the Player and Nature, where at each stage both pick actions based on the contexts. Inspired by the notion of unawareness, we assume that the Player can only access the non-sensitive context before making a decision, while we discuss both cases of Nature accessing the sensitive contexts and Nature unaware of the sensitive contexts. Adapting Blackwell's approachability theory to handle the case of an unknown contexts' distribution, we provide a general necessary and sufficient condition for learning objectives to be compatible with some fairness constraints. This condition is instantiated on (group-wise) no-regret and (group-wise) calibration objectives, and on demographic parity as an additional constraint. When the objective is not compatible with the constraint, the provided framework permits to characterise the optimal trade-off between the two.

This paper investigates robust versions of the general empirical risk minimization algorithm, one of the core techniques underlying modern statistical methods. Success of the empirical risk minimization is based on the fact that for a ‘well-behaved’ stochastic process $\left \{ f(X), \ f\in \mathscr F\right \}$ indexed by a class of functions $f\in \mathscr F$, averages $\frac{1}{N}\sum _{j=1}^N f(X_j)$ evaluated over a sample $X_1,\ldots ,X_N$ of i.i.d. copies of $X$ provide good approximation to the expectations $\mathbb E f(X)$, uniformly over large classes $f\in \mathscr F$. However, this might no longer be true if the marginal distributions of the process are heavy tailed or if the sample contains outliers. We propose a version of empirical risk minimization based on the idea of replacing sample averages by robust proxies of the expectations and obtain high-confidence bounds for the excess risk of resulting estimators. In particular, we show that the excess risk of robust estimators can converge to $0$ at fast rates with respect to the sample size $N$, referring to the rates faster than $N^{-1/2}$. We discuss implications of the main results to the linear and logistic regression problems and evaluate the numerical performance of proposed methods on simulated and real data.

International audience; The field of electric vehicle charging load modelling has been growing rapidly in the last decade. Various models have been applied to this new industrial problem. In light of the Paris Agreement, it is crucial to keep encouraging better modelling techniques for successful electric vehicle adoption. Additionally, numerous papers highlight the lack of charging station data available in order to build models that are consistent with reality. In this context, the purpose of this article is threefold. First, to provide the reader with an overview of the open datasets available and ready to be used in order to foster reproducible research in the field. Second, to review electric vehicle charging load models with their strengths and weaknesses. Third, to provide suggestions on matching the models reviewed to six datasets found in this research that have not previously been explored in the literature. The open data search covered more than 860 repositories and yielded around 60 datasets that are relevant for modelling electric vehicle charging load. These datasets include information on charging point locations, historical and real-time charging sessions, traffic counts, travel surveys and registered vehicles. The models reviewed range from statistical characterisation to stochastic processes and machine learning and the context of their application is assessed.

Multi-class classification problem is among the most popular and well-studied statistical frameworks. Modern multi-class datasets can be extremely ambiguous and single-output predictions fail to deliver satisfactory performance. By allowing predictors to predict a set of label candidates, set-valued classification offers a natural way to deal with this ambiguity. Several formulations of set-valued classification are available in the literature and each of them leads to different prediction strategies. The present survey aims to review popular formulations using a unified statistical framework. The proposed framework encompasses previously considered and leads to new formulations as well as it allows to understand underlying trade-offs of each formulation. We provide infinite sample optimal set-valued classification strategies and review a general plug-in principle to construct data-driven algorithms. The exposition is supported by examples and pointers to both theoretical and practical contributions. Finally, we provide experiments on real-world datasets comparing these approaches in practice and providing general practical guidelines.

International audience; Cross-validation is widely used for selecting among a family of learning rules. This paper studies a related method, called aggregated hold-out (Agghoo), which mixes cross-validation with aggregation; Agghoo can also be related to bagging. According to numerical experiments, Agghoo can improve significantly cross-validation's prediction error, at the same computational cost; this makes it very promising as a general-purpose tool for prediction. We provide the first theoretical guarantees on Agghoo, in the supervised classification setting, ensuring that one can use it safely: at worse, Agghoo performs like the hold-out, up to a constant factor. We also prove a non-asymptotic oracle inequality, in binary classification under the margin condition, which is sharp enough to get (fast) minimax rates.

The main objective of this thesis is to study methods for robust statistical learning. Traditionally, in statistics we use models or simplifying assumptions that allow us to represent the real world. However, some deviations from the hypotheses can strongly disrupt the statistical analysis of a database. By robust statistics, we mean methods that can handle on the one hand so-called abnormal data (sensor error, human error) but also data of a highly variable nature. We apply robust techniques to statistical learning, giving theoretical efficiency results of the proposed methods as well as illustrations on simulated and real data.; Le principal objectif de cette thèse est d'étudier des méthodes d'apprentissage statistique robuste. Traditionnellement, en statistique nous utilisons des modèles ou des hypothèses simplificatrices qui nous permettent de représenter le monde réel tout en sachant l'analyser convenablement. Cependant, certaines déviations des hypothèses peuvent fortement perturber l'analyse statistique d'une base de données. Par statistiques robuste, nous entendons ici des méthodes pouvant gérer d'une part des données dites anormales (erreur de capteur, erreur humaine) mais aussi des données de nature très variables. Nous appliquons ce genre de technique à l'apprentissage statistique, donnant ainsi des assurances théoriques d'efficacité des méthodes proposées ainsi que des illustrations sur des données simulées et réelles.

We propose two multiscale comparisons of graphs using heat diffusion, allowing to compare graphs without node correspondence or even with different sizes. These multiscale comparisons lead to the definition of Lipschitz-continuous empirical processes indexed by a real parameter. The statistical properties of empirical means of such processes are studied in the general case. Under mild assumptions, we prove a functional central limit theorem, as well as a Gaussian approximation with a rate depending only on the sample size. Once applied to our processes, these results allow to analyze data sets of pairs of graphs. We design consistent confidence bands around empirical means and consistent two-sample tests, using bootstrap methods. Their performances are evaluated by simulations on synthetic data sets., Comment: accepted in the Journal of Applied and Computational Topology, 24 pages : made changes suggested by referees

International audience; Aggregated hold-out (Agghoo) is a method which averages learning rules selected by hold-out (that is, cross-validation with a single split). We provide the first theoretical guarantees on Agghoo, ensuring that it can be used safely: Agghoo performs at worst like the hold-out when the risk is convex. The same holds true in classification with the 0-1 risk, with an additional constant factor. For the hold-out, oracle inequalities are known for bounded losses, as in binary classification. We show that similar results can be proved, under appropriate assumptions, for other risk-minimization problems. In particular, we obtain an oracle inequality for regularized kernel regression with a Lip-schitz loss, without requiring that the Y variable or the regressors be bounded. Numerical experiments show that aggregation brings a significant improvement over the hold-out and that Agghoo is competitive with cross-validation.

The recent abundance of data on electricity consumption at different scales opens new challenges and highlights the need for new techniques to leverage information present at finer scales in order to improve forecasts at wider scales. In this work, we take advantage of the similarity between this hierarchical prediction problem and multi-scale transfer learning. We develop two methods for hierarchical transfer learning, based respectively on the stacking of generalized additive models and random forests, and on the use of aggregation of experts. We apply these methods to two problems of electricity load forecasting at national scale, using smart meter data in the first case, and regional data in the second case. For these two usecases, we compare the performances of our methods to that of benchmark algorithms, and we investigate their behaviour using variable importance analysis. Our results demonstrate the interest of both methods, which lead to a significant improvement of the predictions.

International audience; Classification with abstention has gained a lot of attention in recent years as it allows to incorporate human decision-makers in the process. Yet, abstention can potentially amplify disparities and lead to discriminatory predictions. The goal of this work is to build a general purpose classification algorithm, which is able to abstain from prediction, while avoiding disparate impact. We formalize this problem as risk minimization under fairness and abstention constraints for which we derive the form of the optimal classifier. Building on this result, we propose a post-processing classification algorithm, which is able to modify any off-the-shelf score-based classifier using only unlabeled sample. We establish finite sample risk, fairness, and abstention guarantees for the proposed algorithm. In particular, it is shown that fairness and abstention constraints can be achieved independently from the initial classifier as long as sufficiently many unlabeled data is available. The risk guarantee is established in terms of the quality of the initial classifier. Our post-processing scheme reduces to a sparse linear program allowing for an efficient implementation, which we provide. Finally, we validate our method empirically showing that moderate abstention rates allow to bypass the risk-fairness trade-off.

L'électricité se stockant difficilement à grande échelle, l'équilibre entre la production et la consommation doit être rigoureusement maintenu. Une gestion par anticipation de la demande se complexifie avec l'intégration au mix de production des énergies renouvelables intermittentes. Parallèlement, le déploiement des compteurs communicants permet d'envisager un pilotage dynamique de la consommation électrique. Plus concrètement, l'envoi de signaux - tels que des changements du prix de l'électricité - permettrait d'inciter les usagers à moduler leur consommation afin qu'elle s'ajuste au mieux à la production d'électricité. Les algorithmes choisissant ces signaux devront apprendre la réaction des consommateurs face aux envois tout en les optimisant (compromis exploration-exploitation). Notre approche, fondée sur la théorie des bandits, a permis de formaliser ce problème d'apprentissage séquentiel et de proposer un premier algorithme pour piloter la demande électrique d'une population homogène de consommateurs. Une borne supérieure d'ordre $T^{2/3}$ a été obtenue sur le regret de cet algorithme. Des expériences réalisées sur des données de consommation de foyers soumis à des changements dynamiques du prix de l'électricité illustrent ce résultat théorique. Un jeu de données en « information complète » étant nécessaire pour tester un algorithme de bandits, un simulateur de données de consommation fondé sur les auto-encodeurs variationnels a ensuite été construit. Afin de s'affranchir de l'hypothèse d'homogénéité de la population, une approche pour segmenter les foyers en fonction de leurs habitudes de consommation est aussi proposée. Ces différents travaux sont finalement combinés pour proposer et tester des algorithmes de bandits pour un pilotage personnalisé de la consommation électrique.; As electricity is hard to store, the balance between production and consumption must be strictly maintained. With the integration of intermittent renewable energies into the production mix, the management of the balance becomes complex. At the same time, the deployment of smart meters suggests demand response. More precisely, sending signals - such as changes in the price of electricity - would encourage users to modulate their consumption according to the production of electricity. The algorithms used to choose these signals have to learn consumer reactions and, in the same time, to optimize them (exploration-exploration trade-off). Our approach is based on bandit theory and formalizes this sequential learning problem. We propose a first algorithm to control the electrical demand of a homogeneous population of consumers and offer $T^{2/3}$ upper bound on its regret. Experiments on a real data set in which price incentives were offered illustrate these theoretical results. As a “full information” dataset is required to test bandit algorithms, a consumption data generator based on variational autoencoders is built. In order to drop the assumption of the population homogeneity, we propose an approach to cluster households according to their consumption profile. These different works are finally combined to propose and test a bandit algorithm for personalized demand side management.

As electricity is hard to store, the balance between production and consumption must be strictly maintained. With the integration of intermittent renewable energies into the production mix, the management of the balance becomes complex. At the same time, the deployment of smart meters suggests demand response. More precisely, sending signals - such as changes in the price of electricity - would encourage users to modulate their consumption according to the production of electricity. The algorithms used to choose these signals have to learn consumer reactions and, in the same time, to optimize them (exploration-exploration trade-off). Our approach is based on bandit theory and formalizes this sequential learning problem. We propose a first algorithm to control the electrical demand of a homogeneous population of consumers and offer T⅔ upper bound on its regret. Experiments on a real data set in which price incentives were offered illustrate these theoretical results. As a “full information” dataset is required to test bandit algorithms, a consumption data generator based on variational autoencoders is built. In order to drop the assumption of the population homogeneity, we propose an approach to cluster households according to their consumption profile. These different works are finally combined to propose and test a bandit algorithm for personalized demand side management., L'électricité se stockant difficilement à grande échelle, l'équilibre entre la production et la consommation doit être rigoureusement maintenu. Une gestion par anticipation de la demande se complexifie avec l'intégration au mix de production des énergies renouvelables intermittentes. Parallèlement, le déploiement des compteurs communicants permet d'envisager un pilotage dynamique de la consommation électrique. Plus concrètement, l'envoi de signaux - tels que des changements du prix de l'électricité – permettrait d'inciter les usagers à moduler leur consommation afin qu'elle s'ajuste au mieux à la production d'électricité. Les algorithmes choisissant ces signaux devront apprendre la réaction des consommateurs face aux envois tout en les optimisant (compromis exploration-exploitation). Notre approche, fondée sur la théorie des bandits, a permis de formaliser ce problème d'apprentissage séquentiel et de proposer un premier algorithme pour piloter la demande électrique d'une population homogène de consommateurs. Une borne supérieure d'ordre T⅔ a été obtenue sur le regret de cet algorithme. Des expériences réalisées sur des données de consommation de foyers soumis à des changements dynamiques du prix de l'électricité illustrent ce résultat théorique. Un jeu de données en « information complète » étant nécessaire pour tester un algorithme de bandits, un simulateur de données de consommation fondé sur les auto-encodeurs variationnels a ensuite été construit. Afin de s'affranchir de l'hypothèse d'homogénéité de la population, une approche pour segmenter les foyers en fonction de leurs habitudes de consommation est aussi proposée. Ces différents travaux sont finalement combinés pour proposer et tester des algorithmes de bandits pour un pilotage personnalisé de la consommation électrique.

International audience; We study the problem of learning a real-valued function that satisfies the Demographic Parity constraint. It demands the distribution of the predicted output to be independent of the sensitive attribute. We consider the case that the sensitive attribute is available for prediction. We establish a connection between fair regression and optimal transport theory, based on which we derive a close form expression for the optimal fair predictor. Specifically, we show that the distribution of this optimum is the Wasserstein barycenter of the distributions induced by the standard regression function on the sensitive groups. This result offers an intuitive interpretation of the optimal fair prediction and suggests a simple post-processing algorithm to achieve fairness. We establish risk and distribution-free fairness guarantees for this procedure. Numerical experiments indicate that our method is very effective in learning fair models, with a relative increase in error rate that is inferior to the relative gain in fairness.

The main topics adressed in this thesis lie in the general domain of sequential learning, and in particular stochastic multi-armed bandits. The thesis is divided into four chapters and an introduction. In the first part of the main body of the thesis, we design a new algorithm achieving, simultaneously, distribution-dependent and distribution-free optimal guarantees. The next two chapters are devoted to adaptivity questions. First, in the context of continuum-armed bandits, we present a new algorithm which, for the first time, does not require the knowledge of the regularity of the bandit problem it is facing. Then, we study the issue of adapting to the unknown support of the payoffs in bounded K-armed bandits. We provide a procedure that (almost) obtains the same guarantees as if it was given the support in advance. In the final chapter, we study a slightly different bandit setting, designed to enforce diversity-preserving conditions on the strategies. We show that the optimal regert in this setting at a speed that is quite different from the traditional bandit setting. In particular, we observe that bounded regret is possible under some specific hypotheses.; Cette thèse s'inscrit dans le domaine des statistiques séquentielles. Le cadre principal étudié est celui des bandits stochastiques à plusieurs bras, cadre idéal qui modélise le dilemme exploration-exploitation face à des choix répétés. La thèse est composée de quatre chapitres, précédés d'une introduction. Dans la première partie du corps de la thèse, on présente un nouvel algorithme capable d'atteindre des garanties optimales à la fois d'un point de vue distribution-dépendent et distribution-free. Les deux chapitres suivants sont consacrés à des questions dites d'adaptation. D'abord, on propose un algorithme capable de s'adapter à la régularité inconnue dans des problèmes de bandits continus, mettant en évidence le coût polynomial de l'adaptation en bandits continus. Ensuite, on considère un problème d'adaptation au supports pour des problèmes de bandits à K bras, à distributions de paiements bornés dans des intervalles inconnus. Enfin, dans un dernier chapitre un peu à part, on étudie un cadre légèrement différent de bandits préservant la diversité. On montre que le regret optimal dans ce cadre croît à des vitesses différentes des vitesses classiques, avec notamment la possibilité d'atteindre un regret constant sous certaines hypothèses.

We consider joint selection of fixed and random effects in general mixed-effects models. The interpretation of estimated mixed-effects models is challenging since changing the structure of one set of effects can lead to different choices of important covariates in the model. We propose a stepwise selection algorithm to perform simultaneous selection of the fixed and random effects. It is based on Bayesian Information criteria whose penalties are adapted to mixed-effects models. The proposed procedure performs model selection in both linear and nonlinear models. It should be used in the low-dimension setting where the number of ovariates and the number of random effects are moderate with respect to the total number of observations. The performance of the algorithm is assessed via a simulation study, which includes also a comparative study with alternatives when available in the literature. The use of the method is illustrated in the clinical study of an antibiotic agent kinetics.

This thesis lies at the intersection of the theories of non-parametric statistics and statistical learning. Its goal is to provide an understanding of statistical problems in latent space random graphs. Latent space models have emerged as useful probabilistic tools for modeling large networks in various fields such as biology, marketing or social sciences.We first define an identifiable index of the dimension of the latent space and then a consistent estimator of this index. More generally, such identifiable and interpretable quantities alleviate the absence of identifiability of the latent space itself.We then introduce the pair-matching problem. From a non-observed graph, a strategy sequentially queries pairs of nodes and observes the presence/absence of edges. Its goal is to discover as many edges as possible with a fixed budget of queries. For this bandit type problem, we study optimal regrets in stochastic block models and random geometric graphs.Finally, we are interested in estimating the positions of the nodes in the latent space, in the particular situation where the space is a circle in the Euclidean plane.For each of the three problems, we obtain procedures that achieve the statistical optimal performance, as well as efficient procedures with theoretical guarantees. These algorithms are analysed from a non-asymptotic viewpoint, relying in particular on concentration inequalities.; Cette thèse s’inscrit dans les domaines de la statistique non-paramétrique et de la théorie statistique de l’apprentissage non-supervisé. Son objet est la compréhension et la mise en oeuvre de méthodes d’estimation et de décision pour des modèles de graphes aléatoires à espace latent. Ces outils probabilistes rencontrent un succès grandissant pour la modélisation de grands réseaux dans des domaines aussi différents que la biologie, le marketing ou les sciences sociales.Dans un premier temps, nous définissons un indice identifiable de la dimension de l’espace latent puis un estimateur consistant de cet indice. Plus généralement, ces quantités identifiables et interprétables permettent de palier l’absence d’identifiabilité de l’espace latent lui-même.Dans la suite, nous introduisons le problème de ‘pair-matching’. En partant d’un graphe non-observé, une stratégie choisit de façon séquentielle des paires de noeuds et observe la présence/absence d’arêtes. Son objectif est de découvrir le plus grand nombre possible d’arêtes avec un budget fixé. Pour ce problème de type bandit, nous étudions les regrets optimaux dans un modèle à blocs stochastiques puis dans un graphe aléatoire géométrique.Enfin, nous estimons les positions des noeuds dans l’espace latent, dans le cas particulier où l’espace est un cercle dans le plan euclidien.Pour chacun des trois problèmes, nous obtenons des procédures optimales au sens minimax, ainsi que des procédures efficaces satisfaisant certaines garanties théoriques. Ces algorithmes sont analysés d’un point de vue non-asymptotique en s’appuyant, entre autres, sur des inégalités de concentration.

Greedy algorithms for feature selection are widely used for recovering sparse high-dimensional vectors in linear models. In classical procedures, the main emphasis was put on the sample complexity, with little or no consideration of the computation resources required. We present a novel online algorithm: Online Orthogonal Matching Pursuit (OOMP) for online support recovery in the random design setting of sparse linear regression. Our procedure selects features sequentially, alternating between allocation of samples only as needed to candidate features, and optimization over the selected set of variables to estimate the regression coefficients. Theoretical guarantees about the output of this algorithm are proven and its computational complexity is analysed.

Let $(X, S, Y) \in \mathbb{R}^p \times \{1, 2\} \times \mathbb{R}$ be a triplet following some joint distribution $\mathbb{P}$ with feature vector $X$, sensitive attribute $S$ , and target variable $Y$. The Bayes optimal prediction $f^*$ which does not produce Disparate Treatment is defined as $f^*(x) = \mathbb{E}[Y | X = x]$. We provide a non-trivial example of a prediction $x \to f(x)$ which satisfies two common group-fairness notions: Demographic Parity \begin{align} (f(X) | S = 1) &\stackrel{d}{=} (f(X) | S = 2) \end{align} and Equal Group-Wise Risks \begin{align} \mathbb{E}[(f^*(X) - f(X))^2 | S = 1] = \mathbb{E}[(f^*(X) - f(X))^2 | S = 2]. \end{align} To the best of our knowledge this is the first explicit construction of a non-constant predictor satisfying the above. We discuss several implications of this result on better understanding of mathematical notions of algorithmic fairness., Presented at the NeurIPS 2020 Workshop on Algorithmic Fairness through the Lens of Causality and Interpretability