Search

In their quest toward Exascale, High Performance Computing (HPC) systems are rapidly becoming larger and more complex, together with the issues concerning their maintenance. Luckily, many current HPC systems are endowed with data monitoring infrastructures that characterize the system state, and whose data can be used to train Deep Learning (DL) anomaly detection models, a very popular research area. However, the lack of labels describing the state of the system is a wide-spread issue, as annotating data is a costly task, generally falling on human system administrators and thus does not scale toward exascale. In this article we investigate the possibility to extract labels from a service monitoring tool (Nagios) currently used by HPC system administrators to flag the nodes which undergo maintenance operations. This allows to automatically annotate data collected by a fine-grained monitoring infrastructure; this labelled data is then used to train and validate a DL model for anomaly detection. We conduct the experimental evaluation on a tier-0 production supercomputer hosted at CINECA, Bologna, Italy. The results reveal that the DL model can accurately detect the real failures, and, moreover, it can predict the insurgency of anomalies, by systematically anticipating the actual labels (i.e., the moment when system administrators realize when an anomalous event happened); the average advance time computed on historical traces is around 45 minutes. The proposed technology can be easily scaled toward exascale systems to easy their maintenance. [ABSTRACT FROM AUTHOR]

We model portions of the search tree via so-called search constraints. We focus on a particular kind of search constraint, the k-discrepancy constraint appearing in discrepancy-based search. The property that […]

Constraint Logic Programming languages on Finite Domains, CLP(FD), provide a declarative framework for Artificial Intelligence problems. However, in many real life cases, domains are not known and must be acquired or computed. In systems that interact with the outer world, domain elements synthesize information on the environment, they are not all known at the beginning of the computation, and must be retrieved through an expensive acquisition process. In this article, we extend the CLP(FD) language by combining it with a new sort (called Incrementally specified Sets, I-Set). In the resulting language, CLP(FD + I-Set), FD variables can be defined on partially or fully unknown domains (I-Set). Domains can be linked each other through relations, and constraints can be imposed on them. We describe a propagation algorithm (called Known Arc Consistency (KAC)) based on known domain elements, and theoretically compare it with arc-consistency. The language can be implemented on top of different CLP systems, thus letting the user exploit different possible semantics for domains (e.g., lists, sets or streams). We state the specifications that the employed system should provide, and we show that two different CLP systems (Conjunto and {log}) can be effectively used. We provide motivating examples and describe promising applications. Categories and Subject Descriptors: D.1.6 [Programming Techniques]: Logic Programming; D.3.1 [Programming Languages]: Formal Definitions and Theory--Syntax; semantics General Terms: Algorithms, Languages, Performance, Theory Additional Key Words and Phrases: Constraints, domain acquisition, interaction, lazy evaluation, sets, streams

In this work, we introduce a muitiagent architecture called the MuitiAGent Metaheuristic Architecture (MAGMA) conceived as a conceptual and practical framework for metaheuristic algorithms. Metaheuristics can be seen as the result of the interaction among different kinds of agents: The basic architecture contains three levels, each hosting one or more agents. Level-0 agents build solutions, level-1 agents improve solutions, and level-2 agents provide the high level strategy. In this framework, classical metaheuristic algorithms can be smoothly accommodated and extended. The basic three level architecture can be enhanced with the introduction of a fourth level of agents (level-3 agents) coordinating lower level agents. With this additional level, MAGMA can also describe, in a uniform way, cooperative search and, in general, any combination of metaheuristics. We describe the entire architecture, the structure of agents in each level in terms of tuples, and the structure of their coordination as a labeled transition system. We propose this perspective with the aim to achieve a better and clearer understanding of metaheuristics, obtain hybrid algorithms, suggest guidelines for a software engineering-oriented implementation and for didactic purposes. Some specializations of the general architecture will be provided in order to show that existing metaheuristics [e.g., greedy randomized adaptive procedure (GRASP), ant colony optimization (ACO), iterated local search (ILS), memetic algorithms (MAs)] can be easily described in our framework. We describe cooperative search and large neighborhood search (LNS) in the proposed framework exploiting level-3 agents. We show also that a simple hybrid algorithm, called guided restart ILS, can be easily conceived as a combination of existing components in our framework. Index Terms--Agents, combinatorial optimization, constraint satisfaction, metaheuristics. Index Terms--Agents, combinatorial optimization, constraint satisfaction, metaheuristics.

The Traveling Salesman Problem with Time Windows (TSPTW) is the problem of finding a minimum-cost path visiting a set of cities exactly once, where each city must be visited within […]

Efforts aimed at combining operations research and constraint programming have become increasingly prominent and successful in the last few years. It is now widely recognized that integration, e.g., inference in […]

Intrusion detection systems (IDS) are discussed. PlanNet, a constraint-based planner, is configured to perform security management in computer networks.

This paper considers multi-stage optimization problems under uncertainty that involve distinct offline and online phases. In particular it addresses the issue of integrating these phases to show how the two are often interrelated in real-world applications. Our methods are applicable under two (fairly general) conditions: 1) the uncertainty is exogenous; 2) it is possible to define a greedy heuristic for the online phase that can be modeled as a parametric convex optimization problem. We start with a baseline composed by a two-stage offline approach paired with the online greedy heuristic. We then propose multiple methods to tighten the offline/online integration, leading to significant quality improvements, at the cost of an increased computation effort either in the offline or the online phase. Overall, our methods provide multiple options to balance the solution quality/time trade-off, suiting a variety of practical application scenarios. To test our methods, we ground our approaches on two real cases studies with both offline and online decisions: an energy management problem with uncertain renewable generation and demand, and a vehicle routing problem with uncertain travel times. The application domains feature respectively continuous and discrete decisions. An extensive analysis of the experimental results shows that indeed offline/online integration may lead to substantial benefits. [ABSTRACT FROM AUTHOR]

Energy efficiency is of paramount importance for the sustainability of high performance computing (HPC) systems. Energy consumption limits the peak performance of supercomputers and accounts for a large share of Total Cost of Ownership (TCO). Consequently, system owners and final users have started exploring mechanisms to trade off performance for power consumption, for example, through frequency and voltage scaling. However, only a limited number of studies have been devoted to explore the economic viability of performance scaling solutions and to devise pricing mechanisms fostering a more energy-conscious usage of resources, without adversely impacting return-of-investment on the HPC facility. How to foster green-computing to take advantage of power management knobs? Would today's accounting systems be sufficient or are novel pricing schemes needed? How to model the link between TCO and power management? To answer these questions, we present a parametrized model to analyze the impact of frequency scaling on energy consumption and to assess the potential cost benefits for the HPC facility and users. We evaluate four pricing schemes, considering both facility manager and the user perspectives. We then perform a design space exploration considering current and near-future HPC systems and technologies. [ABSTRACT FROM AUTHOR]

Designing and evaluating energy policies is a difficult challenge because the energy sector is a complex system that cannot be adequately understood without using models merging economic, social and individual perspectives. Appropriate models allow policy makers to assess the impact of policy measures, satisfy strategic objectives and develop sustainable policies. Often the implementation of a policy cannot be directly enforced by governments, but falls back to many stakeholders, such as private citizens and enterprises. We propose to integrate two basic cornerstones to devise realistic models: the self-reported behaviour, derived from surveys, and the observed behaviour, from historical data. The self-reported behaviour enables the identification of drivers and barriers pushing or limiting people in their decision making process, while the observed behaviour is used to tune these drivers/barriers in a model. We test our methodology on a case-study: the adoption of photovoltaic panels among private citizens in the Emilia–Romagna region, Italy. We propose an agent-based model devised using self-reported data and then empirically tuned using historical data. The results reveal that our model can predict with great accuracy the photovoltaic (PV) adoption rate and thus support the energy policy-making process. [ABSTRACT FROM AUTHOR]

Highlights • A HPC job dispatcher capable of bounding the power consumption is proposed. • The novel approach combines machine learning techniques and a constraint programming model. • Key idea: predict the power consumption of HPC application and use this knowledge to schedule them effectively. • A thorough comparison with other methods from the state-of-the-art is performed. Abstract Supercomputer installed capacity worldwide increased for many years and further growth is expected in the future. The next goal for high performance computing (HPC) systems is reaching Exascale. The increase in computational power threatens to lead to unacceptable power demands, if future machines will be built using current technology. Therefore reducing supercomputer power consumption has been the subject of intense research. A common approach to curtail the excessive power demands of supercomputers is to hard-bound their consumption, power capping. Power capping can be enforced by reactively throttling system performance when the power bound is hit, or by scheduling workload in a proactive fashion to avoid hitting the bound. In this paper we explore the second approach: our scheduler meets power capping constraints and minimizes quality-of-service (QoS) disruption through smart planning of the job execution order. The approach is based on constraint programming in conjunction with a machine learning module predicting the power consumptions of HPC applications. We evaluate our method on the Eurora supercomputer, using both synthetic workloads and historical traces. Our approach outperforms the state-of-the-art power capping techniques in terms of waiting time and QoS, while keeping schedule computation time under control. [ABSTRACT FROM AUTHOR]

This paper is motivated by the concept that the successful, effective, and sustainable implementation of the smart city paradigm requires a close cooperation among researchers with different, complementary interests and, in most cases, a multidisciplinary approach. It first briefly discusses how such a multidisciplinary methodology, transversal to various disciplines such as architecture, computer science, civil engineering, electrical, electronic and telecommunication engineering, social science and behavioral science, etc., can be successfully employed for the development of suitable modeling tools and real solutions of such sociotechnical systems. Then, the paper presents some pilot projects accomplished by the authors within the framework of some major European Union (EU) and national research programs, also involving the Bologna municipality and some of the key players of the smart city industry. Each project, characterized by different and complementary approaches/modeling tools, is illustrated along with the relevant contextualization and the advancements with respect to the state of the art. [ABSTRACT FROM PUBLISHER]

Recently, a number of noteworthy results have been achieved in various fields of artificial intelligence, and many aspects of the problem-solving process have received significant attention by the scientific community. In this context, the extraction of comprehensive knowledge suitable for problem solving and reasoning, from textual and pictorial problem descriptions, has been less investigated, but recognized as essential for autonomous thinking in artificial intelligence. In this work we present a challenge where methods and tools for deep understanding are strongly needed for enabling problem solving: we propose to solve mathematical puzzles by means of computers, starting from text and diagrams describing them, without any human intervention. We are aware that the proposed challenge is hard and difficult to solve nowadays (and in the foreseeable future), but even studying and solving only single parts of the proposed challenge would represent an important step forward for artificial intelligence. [ABSTRACT FROM AUTHOR]

Demand response mechanisms and load control in the electricity market represent an important area of research at the international level: the trend towards competition and market liberalization has led to the development of methodologies and tools to support energy providers. Demand side management helps energy suppliers to reduce the peak demand and remodel load profiles. This work is intended to support energy suppliers and policy makers in developing strategies to act on the behavior of energy consumers, with the aim to make a more efficient use of energy. We develop a non-linear optimization model for the dynamics of the electricity market, which can be used to obtain tariff recommendations or for setting the goals of a sensibilization campaign. The model comes in two variants: a stochastic version, designed for residential electricity consumption, and a deterministic version, suitable for large electricity users (e.g., public buildings, industrial users). We have tested ourmodel on data from the Italian energy market and performed an extensive analysis of different scenarios. We also tested the optimization model in a real setting in the context of the FP7 DAREED project (http://www.dareed.eu/), where the model has been employed to provide tariff recommendations or to help the identification of goals for local policies. [ABSTRACT FROM AUTHOR]

Scheduling and dispatching tools for high-performance computing (HPC) machines have the key role of mapping jobs to the available resources, trying to maximize performance and quality-of-service (QoS). Allocation and Scheduling in the general case are well-known NP-hard problems, forcing commercial schedulers to adopt greedy approaches to improve performance and QoS. Search-based approaches featuring the exploration of the solution space have seldom been employed in this setting, but mostly applied in off-line scenarios. In this paper, we present the first search-based approach to job allocation and scheduling for HPC machines, working in a production environment. The scheduler is based on Constraint Programming, an effective programming technique for optimization problems. The resulting scheduler is flexible, as it can be easily customized for dealing with heterogeneous resources, user-defined constraints and different metrics. We evaluate our solution both on virtual machines using synthetic workloads, and on the Eurora HPC with production workloads. Tests on a wide range of operating conditions show significant improvements in waitings and QoS in mid-tier HPC machines w.r.t state-of-the-art commercial rule-based dispatchers. Furthermore, we analyze the conditions under which our approach outperforms commercial approaches, to create a portfolio of scheduling algorithms that ensures robustness, flexibility and scalability. [ABSTRACT FROM AUTHOR]

Policy making is an extremely complex process occurring in changing environments and affecting the three pillars of sustainable development: society, economy and the environment. Each political decision in fact implies some form of social reactions, it affects economic and financial aspects and has substantial environmental impacts. Improving decision making in this context could have a huge beneficial impact on all these aspects. There are a number of Artificial Intelligence techniques that could play an important role in improving the policy-making process such as decision support and optimization techniques, game theory, data and opinion mining and agent-based simulation. We outline here some potential use of AI technology as it emerged by the European Union (EU) EU FP7 project ePolicy: Engineering the Policy Making Life Cycle, and we identify some potential research challenges. [ABSTRACT FROM AUTHOR]

Effective multicore computing requires to make efficient usage of the computational resources on a chip. Offline mapping and scheduling can be applied to improve the performance, but classical approaches require considerable a priori knowledge of the target application. In a practical setting, precise information is often unavailable; one can then resort to approximate time and resource usage figures, but this usually requires to make conservative assumptions. The issue is further stressed if real-time guarantees must be provided. We tackle predictable and efficient nonpreemptive scheduling of multitask applications in the presence of duration uncertainty. Hard real-time guarantees are provided with limited idle time insertion, by exploiting a hybrid offline/online technique known as Precedence Constraint Posting (PCP). Our approach does not require probability distributions to be specified, relying instead on simple and cheaper-to-obtain information (bounds, average values). The method has been tested on synthetic applications/platforms and compared with an offline optimized Fixed Priority Scheduling (FPS) approach and a pure online FIFO scheduler; the results are very promising, as the PCP schedules exhibit good stability and improved average execution time (14 percent on average, up to 30 percent versus FPS and up to 40 percent versus the FIFO scheduler). [ABSTRACT FROM AUTHOR]

We investigate the impact of information extracted from sampling and diving on the solution of Constraint Satisfaction Problems (CSP). A sample is a complete assignment of variables to values taken from their domain according to a given distribution. Diving consists in repeatedly performing depth first search attempts with random variable and value selection, constraint propagation enabled and backtracking disabled; each attempt is called a dive and, unless a feasible solution is found, it is a partial assignment of variables (whereas a sample is a -possibly infeasible- complete assignment). While the probability of finding a feasible solution via sampling or diving is negligible if the problem is difficult enough, samples and dives are very fast to generate and, intuitively, even when they are infeasible, they can provide some statistical information on search space structure. The aim of this paper is to understand to what extent it is possible to support the CSP solving process with information derived from sampling and diving. In particular, we are interested in extracting from samples and dives precise indications on the quality of individual variable-value assignments with respect to feasibility. We formally prove that even uniform sampling could provide precise evaluation of the quality of single variable-value assignments; as expected, this requires huge sample sizes and is therefore not useful in practice. On the contrary, diving is much better suited for assignment evaluation purposes. We undertake a thorough experimental analysis on a collection of Partial Latin Square and Car Sequencing instances to assess the quality of information provided by dives. Dive features are identified and their impact on search is evaluated. Results show that diving provides information that can be fruitfully exploited. [ABSTRACT FROM AUTHOR]