Search

There have been many recent advances in the Traffic Signal Control literature that use reinforcement learning, most of which is undertaken using the model-free approach. Approaches in the model-free domain, attempt to learn the value or the policy function directly without attempting to learn the environment transition dynamics. Therefore, training the value function under a specified dynamics fails to differentiate the value updates from the underlying dynamics, making these methods require much larger agent-environment interaction data to generalize over different scenarios. In contrast, approaches that optimize agent actions w.r.t. a learned dynamics model inherently avoid this tight coupling of dynamics and value, allowing for much faster adaptation as traffic scenarios change. For this work on single intersection control, we specifically adopt this latter model-based approach of learning a microscopic simulator model and then apply tree-search techniques to optimize control actions. This approach quickly generalizes to a diverse set of traffic demands, whereas the model-free method performs suboptimally in conditions unseen during training. Another benefit of model-based approaches is the ability to control new intersections with previously unseen topologies, which makes the method transferable in terms of both demand and intersection structure variation. Finally, we observe that pairing these control strategies with the learned model also makes our approach debuggable and explainable, which is a critical requirement for real-world deployment.

The growing use of information and communication technologies (ICT) in power grid operational environments has been essential for operators to improve the monitoring, maintenance and control of power generation, transmission and distribution, however, at the expense of an increased grid exposure to cyber threats. This paper considers cyberattack scenarios targeting substation protective relays that can form the most critical ingredient for the protection of power systems against abnormal conditions. Disrupting the relays operations may yield major consequences on the overall power grid performance possibly leading to widespread blackouts. We investigate methods for the enhancement of substation cybersecurity by leveraging the potential of machine learning for the detection of transformer differential protective relays anomalous behavior. The proposed method analyses operational technology (OT) data obtained from the substation current transformers (CTs) in order to detect cyberattacks. Power systems simulation using OPAL-RT HYPERSIM is used to generate training data sets, to simulate the cyberattacks and to assess the cybersecurity enhancement capability of the proposed machine learning algorithms.

The false tripping of circuit breakers initiated by cyberattacks on protective relays is a growing concern in power systems. This is of high importance because multiple false equipment tripping initiated by coordinated cyberattacks on protective relays can cause large scale disturbance in power systems and potentially lead to cascading failures and blackouts. In this paper, a deep learning based autoencoder is employed to identify anomalous voltage and current data injection to distance protection relays. The autoencoder is first trained with current and voltage data sets representing three-phase faults in zone 1 of a distance relay using a benchmark test system. The autoencoder is then employed to identify anomalies in voltage and current data to prevent false tripping commands by the distance relay. The simulation results verify the capability of the autoencoder model to extract signatures of three-phase faults in the intended zone of a protective relay and detect three-phase fault current and voltage data that do not contain these signatures with high accuracy.

Decision support systems for network security represent a critical element in the safe operation of computer networks. Unfortunately, due to their complexity, it can be difficult to implement and empirically assess novel techniques for displaying networks. This paper details an open source adaptive user interface that hopes to fill this gap. This system supports agile development and offers a wide latitude for human factors and machine learning design modifications. The intent of this system is to serve as an experimental testbed for determining the efficacy of different human factors and machine learning initiatives on operator performance in network monitoring.

The needs of vulnerable populations have been addressed, traditionally, by an ecosystem of private and public agencies that rely on donations (goods, money and services) which they re-distribute based on their perception of what is needed and where. This approach lacks a comprehensive understanding of the demand side as well as the ability to coordinate between various suppliers of goods and services, predict future demand and identify latent supply. We present a knowledge-based platform that utilizes advances in several Artificial Intelligence fields for a more efficient and effective way of provisioning goods and services to people in need.

As cybersecurity threats increasingly appear in news headlines, the security industry continues to build state of the art firewall and intrusion detection systems for monitoring activities in complex cyber networks. These systems generate millions of log files and continuous alerts. In order to make sense of cyber data, cyber security and system administrators review and analyze millions of logs using highly summarized views and manual cycles of click-intensive details-on-demand. This is laborious, induces cognitive overload, and is prone to errors resulting in important information and impacts not being seen when most needed. Our research focus is on developing “FocalPoint” a system that provides Adaptive Level of Detail (LOD) in user interfaces for cybersecurity operations. FocalPoint is a recommender system tailored for complex network information structures that reasons about contextual information associated with the network, user tasks, and cognitive load. This facilitates tuning cyber visualization displays thereby improving user performance in perception, comprehension and projection of current Cybersecurity Situational Awareness (Cyber SA). For cyber analysts, having the right information, in context, when most needed without cognitive overload could lead to effective decision making in cyber operations. We provide a use case scenario for FocalPoint with an in-progress prototype and highlight various challenges and potential considerations for building an effective adaptive system.

Applications of automated planning under uncertainty are often modelled as a discrete and continuous state Markov Decision Process (DC-MDP). Symbolic Dynamic Programming is the existent exact solution for DC-MDPs that uses the eXtended Algebraic Decision Diagrams (XADDs) to symbolically represent the state value function and that computes a complete state-space policy (which is very costly and limits solution to problems with small size and depth). Real-Time Dynamic Programming (RTDP) is an efficient solution method for discrete state MDPs that provides a partial solution for a known initial state. In this paper we combine the RTDP solution with XADD symbolic representation and computation of the value function to propose the Continuous Real Time Dynamic Programming (CRTDP) algorithm. This novel planner uses heuristic search and symbolic generalisation to efficiently update the value function by regions. We show that using the initial state information greatly reduces the number of regions in the value function, therefore allowing CRTDP to solve DC-MDPs more efficiently than standard symbolic dynamic programming both in time and space required for the solution.

We examine linear regression problems where some features may only be observable at a cost (e.g., in medical domains where features may correspond to diagnostic tests that take time and costs money). This can be important in the context of data mining, in order to obtain the best predictions from the data on a limited cost budget. We define a parsimonious linear regression objective criterion that jointly minimizes prediction error and feature cost. We modify least angle regression algorithms commonly used for sparse linear regression to produce the ParLiR algorithm, which not only provides an efficient and parsimonious solution as we demonstrate empirically, but it also provides formal guarantees that we prove theoretically. ispartof: pages:809-814 ispartof: Proceedings of the 8th IEEE International Conference on Data Mining pages:809-814 ispartof: ICDM location:Pisa, Italy date:15 Dec - 19 Dec 2008 status: published

We propose an occupancy grid mapping algorithm for mobile robots operating in environments where objects change their locations over time. Our approach uses a straightforward map differencing technique to detect changes in an environment over time. It employs the expectation maximization algorithm to learn models of non-stationary objects, and to determine the location of such objects in individual occupancy grid maps built at different points in time. By combining data from multiple maps when learning object models, the resulting models have higher fidelity than could be obtained from any single map. A Bayesian complexity measure is applied to determine the number of different objects in the model, making it possible to apply the approach to situations where not all objects are present at all times in the map.