Search

With the rise of new technologies, such as Edge computing, Internet of Things, Smart Cities, and Smart Grids, there is a growing need for multi-agent systems (MAS) approaches. Designing multi-agent systems is challenging, and doing this in an automated way is even more so. To address this, we propose a new framework, Evolved Gossip Contracts (EGC). It builds on Gossip Contracts (GC), a decentralised cooperation protocol that is used as the communication mechanism to facilitate self-organisation in a cooperative MAS. GC has several methods that are implemented uniquely, depending on the goal the MAS aims to achieve. The EGC framework uses evolutionary computing to search for the best implementation of these methods. To evaluate EGC, it was used to solve a classical NP-hard optimisation problem, the Bin Packing Problem (BPP). The experimental results show that EGC successfully discovered a decentralised strategy to solve the BPP, which is better than two classical heuristics on test cases similar to those on which it was trained; the improvement is statistically significant. EGC is the first framework that leverages evolutionary computing to semi-automate the discovery of a communication protocol for a MAS that has been shown to be effective at solving an NP-hard problem.

Almost all state-of-the-art

ExSeisDat is designed using standard message passing interface (MPI) library for seismic data processing on high-performance super-computing clusters. These clusters are generally designed for efficient execution of complex tasks including large size IO. The IO performance degradation issues arise when multiple processes try accessing data from parallel networked storage. These complications are caused by restrictive protocols running by a parallel file system (PFS) controlling the disks and due to less advancement in storage hardware itself as well. This requires and leads to the tuning of specific configuration parameters to optimize the IO performance, commonly not considered by users focused on writing parallel application. Despite its consideration, the changes in configuration parameters are required from case to case. It adds up to further degradation in IO performance for a large SEG-Y format seismic data file scaling to petabytes. The SEG-Y IO and file sorting operations are the two of the main features of ExSeisDat. This research paper proposes technique to optimize these SEG-Y operations based on artificial neural networks (ANNs). The optimization involves auto-tuning of the related configuration parameters, using IO bandwidth prediction by the trained ANN models through machine learning (ML) process. Furthermore, we discuss the impact on prediction accuracy and statistical analysis of auto-tuning bandwidth results, by the variation in hidden layers nodes configuration of the ANNs. The results have shown the overall improvement in bandwidth performance up to 108.8% and 237.4% in the combined SEG-Y IO and file sorting operations test cases, respectively. Therefore, this paper has demonstrated the significant gain in SEG-Y seismic data bandwidth performance by auto-tuning the parameters settings on runtime by using an ML approach.

In many risk-aware and multi-objective reinforcement learning settings, the utility of the user is derived from a single execution of a policy. In these settings, making decisions based on the average future returns is not suitable. For example, in a medical setting a patient may only have one opportunity to treat their illness. Making decisions using just the expected future returns -- known in reinforcement learning as the value -- cannot account for the potential range of adverse or positive outcomes a decision may have. Therefore, we should use the distribution over expected future returns differently to represent the critical information that the agent requires at decision time by taking both the future and accrued returns into consideration. In this paper, we propose two novel Monte Carlo tree search algorithms. Firstly, we present a Monte Carlo tree search algorithm that can compute policies for nonlinear utility functions (NLU-MCTS) by optimising the utility of the different possible returns attainable from individual policy executions, resulting in good policies for both risk-aware and multi-objective settings. Secondly, we propose a distributional Monte Carlo tree search algorithm (DMCTS) which extends NLU-MCTS. DMCTS computes an approximate posterior distribution over the utility of the returns, and utilises Thompson sampling during planning to compute policies in risk-aware and multi-objective settings. Both algorithms outperform the state-of-the-art in multi-objective reinforcement learning for the expected utility of the returns., Comment: arXiv admin note: substantial text overlap with arXiv:2102.00966

Super-computing or HPC clusters are built to provide services to execute computationally complex applications. Generally, these HPC applications involve large scale IO (input/output) processing over the networked parallel file system disks. They are commonly developed on top of the C/C++ based MPI standard library. The HPC clusters MPI–IO performance significantly depends on the particular parameter value configurations, not generally considered when writing the algorithms or programs. Therefore, this leads to poor IO and overall program performance degradation. The IO is mostly left to individual practitioners to be optimised at code level. This usually leads to unexpected consequences due to IO bandwidth degradation which becomes inevitable as the file data scales in size to petabytes. To overcome the poor IO performance, this research paper presents an approach for auto-tuning of the configuration parameters by forecasting the MPI–IO bandwidth via artificial neural networks (ANNs), a machine learning (ML) technique. These parameters are related to MPI–IO library and lustre (parallel) file system. In addition to this, we have identified a number of common configurations out of numerous possibilities, selected in the auto-tuning process of READ/WRITE operations. These configurations caused an overall READ bandwidth improvement of 65.7% with almost 83% test cases improved. In addition, the overall WRITE bandwidth improved by 83% with number of test cases improved by almost 93%. This paper demonstrates that by using auto-tuning parameters via ANNs predictions, this can significantly impact overall IO bandwidth performance.

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems. Funding: Fonds voor Wetenschappelijk Onderzoek (FWO)FWO [1SA2820N]; Flemish GovernmentEuropean Commission; FWOFWO [iBOF/21/027]; National University of Ireland Galway Hardiman Scholarship; FAPERGSFundacao de Amparo a Ciencia e Tecnologia do Estado do Rio Grande do Sul (FAPERGS) [19/2551-0001277-2]; FAPESPFundacao de Amparo a Pesquisa do Estado de Sao Paulo (FAPESP) [2020/05165-1]; Swedish Governmental Agency for Innovation SystemsVinnova [NFFP7/2017-04885]; Wallenberg Artificial Intelligence, Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation; LIFT - Dutch Research Council (NWO) [019.011]; 2017 Microsoft Research PhD Scholarship Program; 2020 Microsoft Research EMEA PhD Award

Cloud computing has seen rapid growth in the last decade, as a result of this there is an increasing concern about the energy requirement of cloud data centres. One approach to tackle this issue is dynamic virtual machine consolidation (DVMC) where virtual machines (VMs) are packed into fewer hosts, while hosts which have no VMs are placed into a hibernate mode. DVMC is traditionally solved with strategies that build on centralised topologies. One disadvantage of these solutions is that they don ′ t scale well in very large data centres as the manager node becomes a bottleneck and a single point of failure. The nature of the problem lends itself to a decentralised solution which is preferred over a centralised one when a problem is: inherently modular, large in scale, in a very dynamic environment, and when communication channels can experience message delays and loss. We propose Gossip Contracts (GC), a new multi-agent framework for crafting decentralised co-operation strategies. GC is inspired by the Gossip and Contract Net protocols. Using GC we developed a GC-based DVMC strategy and compared it to two popular strategies: Sercon, a centralised strategy, and ecoCloud, a distributed strategy. One of the comparison experiments uses Google cluster-usage traces; a dataset of real usage data from a Google data centre. The GC-based strategy performs best with respect to SLA Violations, and is similar to or outperforms other strategies with respect to power consumption. By its decentralised nature, GC does not have a single point of failure.

Computational models can shape our understanding of cell and tissue remodelling, from cell spreading, to active force generation, adhesion, and growth. In this mini-review, we discuss recent progress in modelling of chemo-mechanical cell behaviour and the evolution of multicellular systems. In particular, we highlight recent advances in (i) free-energy based single cell models that can provide new fundamental insight into cell spreading, cancer cell invasion, stem cell differentiation, and remodelling in disease, and (ii) mechanical agent-based models to simulate large numbers of discrete interacting cells in proliferative tumours. We describe how new biological understanding has emerged from such theoretical models, and the trade-offs and constraints associated with current approaches. Ultimately, we aim to make a case for why theory should be integrated with an experimental workflow to optimise new in-vitro studies, to predict feedback between cells and their microenvironment, and to deepen understanding of active cell behaviour.

Proteomics aims to characterise system-wide protein expression and typically relies on mass-spectrometry and peptide fragmentation, followed by a database search for protein identification. It has wide ranging applications from clinical to environmental settings and virtually impacts on every area of biology. In that context

Energy related costs and environmental sustainability present a significant challenge for cloud computing practitioners and the development of next generation data centers. Virtual Machine (VM) placement provides a promising technique to save energy and improve resource management which is one of the greatest causes of high energy consumption in the operation of data centers today. A key challenge for VM placement algorithms is the ability to accurately forecast future resource demands due to the dynamic nature of cloud applications. Furthermore, the literature rarely considers placement strategies based on co-located resource consumption which has the potential to improve allocation decisions. Using real workload traces this work presents a comparative study of the most widely used prediction models and introduces our novel Predictive Anti-Correlated Placement Algorithm (PACPA) which considers both CPU and bandwidth resource consumption. Our empirical results demonstrate how the proposed approach reduces energy by 18% while also reducing service violations by over 34% compared to some of the most commonly used placement algorithms.

Energy awareness presents an immense challenge for cloud computing infrastructure and the development of next generation data centers. Virtual Machine (VM) consolidation is one technique that can be harnessed to reduce energy related costs and environmental sustainability issues of data centers. In recent times intelligent learning approaches have proven to be effective for managing resources in cloud data centers. In this paper we explore the application of Reinforcement Learning (RL) algorithms for the VM consolidation problem demonstrating their capacity to optimize the distribution of virtual machines across the data center for improved resource management. Determining efficient policies in dynamic environments can be a difficult task, however, the proposed RL approach learns optimal behavior in the absence of complete knowledge due to its innate ability to reason under uncertainty. Using real workload data we provide a comparative analysis of popular RL algorithms including SARSA and Q-learning. Our empirical results demonstrate how our approach improves energy efficiency by 25% while also reducing service violations by 63% over the popular Power-Aware heuristic algorithm.

Anaerobic Digestion (AD) is a waste treatment technology widely used for wastewater and solid waste treatment, with the advantage of being a source of renewable energy in the form of biogas. Anaerobic digestion model number 1 (ADM1) is the most common mathematical model available for AD modelling. Commercial software implementations of ADM1 are available but have limited flexibility and availability due to the closed sources and licensing fees. Python is the fastest growing programming language and is open source freely available. Python implementation of ADM1 makes this AD model available to the mass user base of the Python ecosystem and it’s libraries. The open easy to use implementation in PyADM1 makes it more accessible and provides possibilities for flexible direct use of the model linked to other software, e.g. machine learning libraries or Linux operating system on embedded hardware.

In many real-world scenarios, the utility of a user is derived from a single execution of a policy. In this case, to apply multi-objective reinforcement learning, the expected utility of the returns must be optimised. Various scenarios exist where a user’s preferences over objectives (also known as the utility function) are unknown or difficult to specify. In such scenarios, a set of optimal policies must be learned. However, settings where the expected utility must be maximised have been largely overlooked by the multi-objective reinforcement learning community and, as a consequence, a set of optimal solutions has yet to be defined. In this work, we propose first-order stochastic dominance as a criterion to build solution sets to maximise expected utility. We also define a new dominance criterion, known as expected scalarised returns (ESR) dominance, that extends first-order stochastic dominance to allow a set of optimal policies to be learned in practice. Additionally, we define a new solution concept called the ESR set, which is a set of policies that are ESR dominant. Finally, we present a new multi-objective tabular distributional reinforcement learning (MOTDRL) algorithm to learn the ESR set in multi-objective multi-armed bandit settings.