Search

The predominant paradigm for using machine learning models on a device is to train a model in the cloud and perform inference using the trained model on the device. However, with increasing number of smart devices and improved hardware, there is interest in performing model training on the device. Given this surge in interest, a comprehensive survey of the field from a device-agnostic perspective sets the stage for both understanding the state-of-the-art and for identifying open challenges and future avenues of research. However, on-device learning is an expansive field with connections to a large number of related topics in AI and machine learning (including online learning, model adaptation, one/few-shot learning, etc.). Hence, covering such a large number of topics in a single survey is impractical. This survey finds a middle ground by reformulating the problem of on-device learning as resource constrained learning where the resources are compute and memory. This reformulation allows tools, techniques, and algorithms from a wide variety of research areas to be compared equitably. In addition to summarizing the state-of-the-art, the survey also identifies a number of challenges and next steps for both the algorithmic and theoretical aspects of on-device learning., Comment: Edge Learning, TinyML, Resource Constrained Machine Learning, Deep learning on device, Statistical Learning Theory, 45 pages survey

Distributed optimization has attracted lots of attention in the operation of power systems in recent years, where a large area is decomposed into smaller control regions each solving a local optimization problem with periodic information exchange with neighboring regions. However, most distributed optimization methods are iterative and require synchronization of all regions at each iteration, which is hard to achieve without a centralized coordinator and might lead to under-utilization of computation resources due to the heterogeneity of the regions. To address such limitations of synchronous schemes, this paper investigates the applicability of asynchronous distributed optimization methods to power system optimization. Particularly, we focus on solving the AC Optimal Power Flow problem and propose an algorithmic framework based on the Alternating Direction Method of Multipliers (ADMM) method that allows the regions to perform local updates with information received from a subset of but not all neighbors. Through experimental studies, we demonstrate that the convergence performance of the proposed asynchronous scheme is dependent on the communication delay of passing messages among the regions. Under mild communication delays, the proposed scheme can achieve comparable or even faster convergence compared with its synchronous counterpart, which can be used as a good alternative to centralized or synchronous distributed optimization approaches., Comment: SmartGridComm 2017

Large scale, non-convex optimization problems arising in many complex networks such as the power system call for efficient and scalable distributed optimization algorithms. Existing distributed methods are usually iterative and require synchronization of all workers at each iteration, which is hard to scale and could result in the under-utilization of computation resources due to the heterogeneity of the subproblems. To address those limitations of synchronous schemes, this paper proposes an asynchronous distributed optimization method based on the Alternating Direction Method of Multipliers (ADMM) for non-convex optimization. The proposed method only requires local communications and allows each worker to perform local updates with information from a subset of but not all neighbors. We provide sufficient conditions on the problem formulation, the choice of algorithm parameter and network delay, and show that under those mild conditions, the proposed asynchronous ADMM method asymptotically converges to the KKT point of the non-convex problem. We validate the effectiveness of asynchronous ADMM by applying it to the Optimal Power Flow problem in multiple power systems and show that the convergence of the proposed asynchronous scheme could be faster than its synchronous counterpart in large-scale applications.

Recent studies have shown that multi-step optimization based on Model Predictive Control (MPC) can effectively coordinate the increasing number of distributed renewable energy and storage resources in the power system. However, the computation complexity of MPC is usually high which limits its use in practical implementation. To improve the efficiency of MPC, in this paper, we apply a distributed optimization method to MPC. The approach consists of a partitioning technique based on spectral clustering that determines the best system partition and an improved Optimality Condition Decomposition method that solves the optimization problem in a distributed manner. Results of simulations conducted on the IEEE 14-bus and 118-bus systems show that the distributed MPC problem can be solved significantly faster by using a good partition of the system and this partition is applicable to multiple time steps without frequent changes.

Distributed optimization for solving non-convex Optimal Power Flow (OPF) problems in power systems has attracted tremendous attention in the last decade. Most studies are based on the geographical decomposition of IEEE test systems for verifying the feasibility of the proposed approaches. However, it is not clear if one can extrapolate from these studies that those approaches can be applied to very large-scale real-world systems. In this paper, we show, for the first time, that distributed optimization can be effectively applied to a large-scale real transmission network, namely, the Polish 2383-bus system for which no pre-defined partitions exist, by using a recently developed partitioning technique. More specifically, the problem solved is the AC OPF problem with geographical decomposition of the network using the Alternating Direction Method of Multipliers (ADMM) method in conjunction with the partitioning technique. Through extensive experimental results and analytical studies, we show that with the presented partitioning technique the convergence performance of ADMM can be improved substantially, which enables the application of distributed approaches on very large-scale systems.