Search

We propose Scalable Message Passing Neural Networks (SMPNNs) and demonstrate that, by integrating standard convolutional message passing into a Pre-Layer Normalization Transformer-style block instead of attention, we can produce high-performing deep message-passing-based Graph Neural Networks (GNNs). This modification yields results competitive with the state-of-the-art in large graph transductive learning, particularly outperforming the best Graph Transformers in the literature, without requiring the otherwise computationally and memory-expensive attention mechanism. Our architecture not only scales to large graphs but also makes it possible to construct deep message-passing networks, unlike simple GNNs, which have traditionally been constrained to shallow architectures due to oversmoothing. Moreover, we provide a new theoretical analysis of oversmoothing based on universal approximation which we use to motivate SMPNNs. We show that in the context of graph convolutions, residual connections are necessary for maintaining the universal approximation properties of downstream learners and that removing them can lead to a loss of universality.

We propose a class of trainable deep learning-based geometries called Neural Spacetimes (NSTs), which can universally represent nodes in weighted directed acyclic graphs (DAGs) as events in a spacetime manifold. While most works in the literature focus on undirected graph representation learning or causality embedding separately, our differentiable geometry can encode both graph edge weights in its spatial dimensions and causality in the form of edge directionality in its temporal dimensions. We use a product manifold that combines a quasi-metric (for space) and a partial order (for time). NSTs are implemented as three neural networks trained in an end-to-end manner: an embedding network, which learns to optimize the location of nodes as events in the spacetime manifold, and two other networks that optimize the space and time geometries in parallel, which we call a neural (quasi-)metric and a neural partial order, respectively. The latter two networks leverage recent ideas at the intersection of fractal geometry and deep learning to shape the geometry of the representation space in a data-driven fashion, unlike other works in the literature that use fixed spacetime manifolds such as Minkowski space or De Sitter space to embed DAGs. Our main theoretical guarantee is a universal embedding theorem, showing that any $k$-point DAG can be embedded into an NST with $1+\mathcal{O}(\log(k))$ distortion while exactly preserving its causal structure. The total number of parameters defining the NST is sub-cubic in $k$ and linear in the width of the DAG. If the DAG has a planar Hasse diagram, this is improved to $\mathcal{O}(\log(k)) + 2)$ spatial and 2 temporal dimensions. We validate our framework computationally with synthetic weighted DAGs and real-world network embeddings; in both cases, the NSTs achieve lower embedding distortions than their counterparts using fixed spacetime geometries., Comment: 12 pages: main body and 19 pages: appendix

Clifford Group Equivariant Neural Networks (CGENNs) leverage Clifford algebras and multivectors as an alternative approach to incorporating group equivariance to ensure symmetry constraints in neural representations. In principle, this formulation generalizes to orthogonal groups and preserves equivariance regardless of the metric signature. However, previous works have restricted internal network representations to Euclidean or Minkowski (pseudo-)metrics, handpicked depending on the problem at hand. In this work, we propose an alternative method that enables the metric to be learned in a data-driven fashion, allowing the CGENN network to learn more flexible representations. Specifically, we populate metric matrices fully, ensuring they are symmetric by construction, and leverage eigenvalue decomposition to integrate this additional learnable component into the original CGENN formulation in a principled manner. Additionally, we motivate our method using insights from category theory, which enables us to explain Clifford algebras as a categorical construction and guarantee the mathematical soundness of our approach. We validate our method in various tasks and showcase the advantages of learning more flexible latent metric representations. The code and data are available at https://github.com/rick-ali/Metric-Learning-for-CGENNs, Comment: Workshop on Geometry-grounded Representation Learning and Generative Modeling (GRaM) at the ICML 2024

While 2D diffusion models generate realistic, high-detail images, 3D shape generation methods like Score Distillation Sampling (SDS) built on these 2D diffusion models produce cartoon-like, over-smoothed shapes. To help explain this discrepancy, we show that the image guidance used in Score Distillation can be understood as the velocity field of a 2D denoising generative process, up to the choice of a noise term. In particular, after a change of variables, SDS resembles a high-variance version of Denoising Diffusion Implicit Models (DDIM) with a differently-sampled noise term: SDS introduces noise i.i.d. randomly at each step, while DDIM infers it from the previous noise predictions. This excessive variance can lead to over-smoothing and unrealistic outputs. We show that a better noise approximation can be recovered by inverting DDIM in each SDS update step. This modification makes SDS's generative process for 2D images almost identical to DDIM. In 3D, it removes over-smoothing, preserves higher-frequency detail, and brings the generation quality closer to that of 2D samplers. Experimentally, our method achieves better or similar 3D generation quality compared to other state-of-the-art Score Distillation methods, all without training additional neural networks or multi-view supervision, and providing useful insights into relationship between 2D and 3D asset generation with diffusion models., Comment: NeurIPS 2024. 28 pages, 30 figures. Revision: additional comparisons and ablations studies

3D scene understanding for robotic applications exhibits a unique set of requirements including real-time inference, object-centric latent representation learning, accurate 6D pose estimation and 3D reconstruction of objects. Current methods for scene understanding typically rely on a combination of trained models paired with either an explicit or learnt volumetric representation, all of which have their own drawbacks and limitations. We introduce DreamUp3D, a novel Object-Centric Generative Model (OCGM) designed explicitly to perform inference on a 3D scene informed only by a single RGB-D image. DreamUp3D is a self-supervised model, trained end-to-end, and is capable of segmenting objects, providing 3D object reconstructions, generating object-centric latent representations and accurate per-object 6D pose estimates. We compare DreamUp3D to baselines including NeRFs, pre-trained CLIP-features, ObSurf, and ObPose, in a range of tasks including 3D scene reconstruction, object matching and object pose estimation. Our experiments show that our model outperforms all baselines by a significant margin in real-world scenarios displaying its applicability for 3D scene understanding tasks while meeting the strict demands exhibited in robotics applications.

Parameter-efficient fine-tuning optimizes large, pre-trained foundation models by updating a subset of parameters; in this class, Low-Rank Adaptation (LoRA) is particularly effective. Inspired by an effort to investigate the different roles of LoRA matrices during fine-tuning, this paper characterizes and leverages unexpected asymmetry in the importance of low-rank adapter matrices. Specifically, when updating the parameter matrices of a neural network by adding a product $BA$, we observe that the $B$ and $A$ matrices have distinct functions: $A$ extracts features from the input, while $B$ uses these features to create the desired output. Based on this observation, we demonstrate that fine-tuning $B$ is inherently more effective than fine-tuning $A$, and that a random untrained $A$ should perform nearly as well as a fine-tuned one. Using an information-theoretic lens, we also bound the generalization of low-rank adapters, showing that the parameter savings of exclusively training $B$ improves the bound. We support our conclusions with experiments on RoBERTa, BART-Large, LLaMA-2, and ViTs., Comment: 17 pages, 2 figures, 9 tables

Mixture-of-Experts (MoEs) can scale up beyond traditional deep learning models by employing a routing strategy in which each input is processed by a single "expert" deep learning model. This strategy allows us to scale up the number of parameters defining the MoE while maintaining sparse activation, i.e., MoEs only load a small number of their total parameters into GPU VRAM for the forward pass depending on the input. In this paper, we provide an approximation and learning-theoretic analysis of mixtures of expert MLPs with (P)ReLU activation functions. We first prove that for every error level $\varepsilon>0$ and every Lipschitz function $f:[0,1]^n\to \mathbb{R}$, one can construct a MoMLP model (a Mixture-of-Experts comprising of (P)ReLU MLPs) which uniformly approximates $f$ to $\varepsilon$ accuracy over $[0,1]^n$, while only requiring networks of $\mathcal{O}(\varepsilon^{-1})$ parameters to be loaded in memory. Additionally, we show that MoMLPs can generalize since the entire MoMLP model has a (finite) VC dimension of $\tilde{O}(L\max\{nL,JW\})$, if there are $L$ experts and each expert has a depth and width of $J$ and $W$, respectively.

In real-world scenarios, although data entities may possess inherent relationships, the specific graph illustrating their connections might not be directly accessible. Latent graph inference addresses this issue by enabling Graph Neural Networks (GNNs) to operate on point cloud data, dynamically learning the necessary graph structure. These graphs are often derived from a latent embedding space, which can be modeled using Euclidean, hyperbolic, spherical, or product spaces. However, currently, there is no principled differentiable method for determining the optimal embedding space. In this work, we introduce the Attentional Multi-Embedding Selection (AMES) framework, a differentiable method for selecting the best embedding space for latent graph inference through backpropagation, considering a downstream task. Our framework consistently achieves comparable or superior results compared to previous methods for latent graph inference across five benchmark datasets. Importantly, our approach eliminates the need for conducting multiple experiments to identify the optimal embedding space. Furthermore, we explore interpretability techniques that track the gradient contributions of different latent graphs, shedding light on how our attention-based, fully differentiable approach learns to choose the appropriate latent space. In line with previous works, our experiments emphasize the advantages of hyperbolic spaces in enhancing performance. More importantly, our interpretability framework provides a general approach for quantitatively comparing embedding spaces across different tasks based on their contributions, a dimension that has been overlooked in previous literature on latent graph inference.

The inductive bias of a graph neural network (GNN) is largely encoded in its specified graph. Latent graph inference relies on latent geometric representations to dynamically rewire or infer a GNN's graph to maximize the GNN's predictive downstream performance, but it lacks solid theoretical foundations in terms of embedding-based representation guarantees. This paper addresses this issue by introducing a trainable deep learning architecture, coined neural snowflake, that can adaptively implement fractal-like metrics on $\mathbb{R}^d$. We prove that any given finite weights graph can be isometrically embedded by a standard MLP encoder. Furthermore, when the latent graph can be represented in the feature space of a sufficiently regular kernel, we show that the combined neural snowflake and MLP encoder do not succumb to the curse of dimensionality by using only a low-degree polynomial number of parameters in the number of nodes. This implementation enables a low-dimensional isometric embedding of the latent graph. We conduct synthetic experiments to demonstrate the superior metric learning capabilities of neural snowflakes when compared to more familiar spaces like Euclidean space. Additionally, we carry out latent graph inference experiments on graph benchmarks. Consistently, the neural snowflake model achieves predictive performance that either matches or surpasses that of the state-of-the-art latent graph inference models. Importantly, this performance improvement is achieved without requiring random search for optimal latent geometry. Instead, the neural snowflake model achieves this enhancement in a differentiable manner., Comment: 9 Pages + Appendix, 2 Figures, 9 Tables

Score-based generative models (SGMs) sample from a target distribution by iteratively transforming noise using the score function of the perturbed target. For any finite training set, this score function can be evaluated in closed form, but the resulting SGM memorizes its training data and does not generate novel samples. In practice, one approximates the score by training a neural network via score-matching. The error in this approximation promotes generalization, but neural SGMs are costly to train and sample, and the effective regularization this error provides is not well-understood theoretically. In this work, we instead explicitly smooth the closed-form score to obtain an SGM that generates novel samples without training. We analyze our model and propose an efficient nearest-neighbor-based estimator of its score function. Using this estimator, our method achieves sampling times competitive with neural SGMs while running on consumer-grade CPUs., Comment: Under review

Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curvature, or combinations thereof, to better model the latent space and enhance model performance. However, little attention has been given to the problem of automatically identifying the optimal latent geometry for the downstream task. We mathematically define this novel formulation and coin it as neural latent geometry search (NLGS). More specifically, we introduce an initial attempt to search for a latent geometry composed of a product of constant curvature model spaces with a small number of query evaluations, under some simplifying assumptions. To accomplish this, we propose a novel notion of distance between candidate latent geometries based on the Gromov-Hausdorff distance from metric geometry. In order to compute the Gromov-Hausdorff distance, we introduce a mapping function that enables the comparison of different manifolds by embedding them in a common high-dimensional ambient space. We then design a graph search space based on the notion of smoothness between latent geometries and employ the calculated distances as an additional inductive bias. Finally, we use Bayesian optimization to search for the optimal latent geometry in a query-efficient manner. This is a general method which can be applied to search for the optimal latent geometry for a variety of models and downstream tasks. We perform experiments on synthetic and real-world datasets to identify the optimal latent geometry for multiple machine learning problems.

Recent studies propose enhancing machine learning models by aligning the geometric characteristics of the latent space with the underlying data structure. Instead of relying solely on Euclidean space, researchers have suggested using hyperbolic and spherical spaces with constant curvature, or their combinations (known as product manifolds), to improve model performance. However, there exists no principled technique to determine the best latent product manifold signature, which refers to the choice and dimensionality of manifold components. To address this, we introduce a novel notion of distance between candidate latent geometries using the Gromov-Hausdorff distance from metric geometry. We propose using a graph search space that uses the estimated Gromov-Hausdorff distances to search for the optimal latent geometry. In this work we focus on providing a description of an algorithm to compute the Gromov-Hausdorff distance between model spaces and its computational implementation., Comment: arXiv admin note: substantial text overlap with arXiv:2309.04810

We study the representation capacity of deep hyperbolic neural networks (HNNs) with a ReLU activation function. We establish the first proof that HNNs can $\varepsilon$-isometrically embed any finite weighted tree into a hyperbolic space of dimension $d$ at least equal to $2$ with prescribed sectional curvature $\kappa<0$, for any $\varepsilon> 1$ (where $\varepsilon=1$ being optimal). We establish rigorous upper bounds for the network complexity on an HNN implementing the embedding. We find that the network complexity of HNN implementing the graph representation is independent of the representation fidelity/distortion. We contrast this result against our lower bounds on distortion which any ReLU multi-layer perceptron (MLP) must exert when embedding a tree with $L>2^d$ leaves into a $d$-dimensional Euclidean space, which we show at least $\Omega(L^{1/d})$; independently of the depth, width, and (possibly discontinuous) activation function defining the MLP., Comment: 22 Pages + References, 1 Table, 4 Figures

Graph Neural Networks leverage the connectivity structure of graphs as an inductive bias. Latent graph inference focuses on learning an adequate graph structure to diffuse information on and improve the downstream performance of the model. In this work we employ stereographic projections of the hyperbolic and spherical model spaces, as well as products of Riemannian manifolds, for the purpose of latent graph inference. Stereographically projected model spaces achieve comparable performance to their non-projected counterparts, while providing theoretical guarantees that avoid divergence of the spaces when the curvature tends to zero. We perform experiments on both homophilic and heterophilic graphs., Comment: Accepted at the ICLR 2023 Workshop on Physics for Machine Learning

The typical size of computational meshes needed for realistic geometries and high-speed flow conditions makes Computational Fluid Dynamics (CFD) impractical for full-mission performance prediction and control. Reduced-Order Models (ROMs) in low-speed aerodynamics have come a long way in terms of reconstructing coherent flow patterns, thus enabling aerodynamic performance prediction. While many examples of ROMs exist for low-speed aerodynamics, there is no such broad literature for high-speed flows. We propose to use physics-enhanced neural fields for prediction of the steady, supersonic flow around a rocket for resolving: the bow shock profile, the boundary layer gradients and the wake over a range of incidences. This approach can lead to the construction of a computationally inexpensive, continuous aerothermodynamic model of a rocket at realistic flight conditions with applications to aerodynamic design and mission analysis and control. The use of neural fields allows to quickly and efficiently sweep the Angle of Attack (AoA) in a continuous manner, as compared to traditional CFD which requires running simulations for each discrete incidence value.

In this work, we present an efficient optic flow algorithm for the extraction of vertically resolved 3D atmospheric motion vector (AMV) fields from incomplete hyperspectral image data measures by infrared sounders. The model at the heart of the energy to be minimized is consistent with atmospheric dynamics, incorporating ingredients of thermodynamics, hydrostatic equilibrium and statistical turbulence. Modern optimization techniques are deployed to design a low-complexity solver for the energy minimization problem, which is non-convex, non-differentiable, high-dimensional and subject to physical constraints. In particular, taking advantage of the alternate direction of multipliers methods (ADMM), we show how to split the original high-dimensional problem into a recursion involving a set of standard and tractable optic-flow sub-problems. By comparing with the ground truth provided by the operational numerical simulation of the European Centre for Medium-Range Weather Forecasts (ECMWF), we show that the performance of the proposed method is superior to state-of-the-art optical flow algorithms in the context of real infrared atmospheric sounding interferometer (IASI) observations.

Graph Neural Networks usually rely on the assumption that the graph topology is available to the network as well as optimal for the downstream task. Latent graph inference allows models to dynamically learn the intrinsic graph structure of problems where the connectivity patterns of data may not be directly accessible. In this work, we generalize the discrete Differentiable Graph Module (dDGM) for latent graph learning. The original dDGM architecture used the Euclidean plane to encode latent features based on which the latent graphs were generated. By incorporating Riemannian geometry into the model and generating more complex embedding spaces, we can improve the performance of the latent graph inference system. In particular, we propose a computationally tractable approach to produce product manifolds of constant curvature model spaces that can encode latent features of varying structure. The latent representations mapped onto the inferred product manifold are used to compute richer similarity measures that are leveraged by the latent graph learning model to obtain optimized latent graphs. Moreover, the curvature of the product manifold is learned during training alongside the rest of the network parameters and based on the downstream task, rather than it being a static embedding space. Our novel approach is tested on a wide range of datasets, and outperforms the original dDGM model.

We demonstrate the applicability of model-agnostic algorithms for meta-learning, specifically Reptile, to GNN models in molecular regression tasks. Using meta-learning we are able to learn new chemical prediction tasks with only a few model updates, as compared to using randomly initialized GNNs which require learning each regression task from scratch. We experimentally show that GNN layer expressivity is correlated to improved meta-learning. Additionally, we also experiment with GNN emsembles which yield best performance and rapid convergence for k-shot learning.

Since 2009, Moving Target Defense (MTD) has become a new paradigm of defensive mechanism that frequently changes the state of the target system to confuse the attacker. This frequent change is costly and leads to a trade-off between misleading the attacker and disrupting the quality of service. Optimizing the MTD activation frequency is necessary to develop this defense mechanism when facing realistic, multi-step attack scenarios. Attack modeling formalisms based on DAG are prominently used to specify these scenarios. Our contribution is a new DAG-based formalism for MTDs and its translation into a Price Timed Markov Decision Process to find the best activation frequencies against the attacker's time/cost-optimal strategies. For the first time, MTD activation frequencies are analyzed in a state-of-the-art DAG-based representation. Moreover, this is the first paper that considers the specificity of MTDs in the automatic analysis of attack modeling formalisms. Finally, we present some experimental results using Uppaal Stratego to demonstrate its applicability and relevance.

A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear maps between these spaces. SNNs have been shown to have useful theoretical properties that help tackle issues arising from heterophily and over-smoothing. One complication intrinsic to these models is finding a good sheaf for the task to be solved. Previous works proposed two diametrically opposed approaches: manually constructing the sheaf based on domain knowledge and learning the sheaf end-to-end using gradient-based methods. However, domain knowledge is often insufficient, while learning a sheaf could lead to overfitting and significant computational overhead. In this work, we propose a novel way of computing sheaves drawing inspiration from Riemannian geometry: we leverage the manifold assumption to compute manifold-and-graph-aware orthogonal maps, which optimally align the tangent spaces of neighbouring data points. We show that this approach achieves promising results with less computational overhead when compared to previous SNN models. Overall, this work provides an interesting connection between algebraic topology and differential geometry, and we hope that it will spark future research in this direction., Comment: Presented at the ICML 2022 Workshop on Topology, Algebra, and Geometry in Machine Learning

Memory replay may be key to learning in biological brains, which manage to learn new tasks continually without catastrophically interfering with previous knowledge. On the other hand, artificial neural networks suffer from catastrophic forgetting and tend to only perform well on tasks that they were recently trained on. In this work we explore the application of latent space based memory replay for classification using artificial neural networks. We are able to preserve good performance in previous tasks by storing only a small percentage of the original data in a compressed latent space version.

The Reynolds-averaged Navier-Stokes (RANS) equations require accurate modeling of the anisotropic Reynolds stress tensor. Traditional closure models, while sophisticated, often only apply to restricted flow configurations. Researchers have started using machine learning approaches to tackle this problem by developing more general closure models informed by data. In this work we build upon recent convolutional neural network architectures used for turbulence modeling and propose a multi-task learning-based fully convolutional neural network that is able to accurately predict the normalized anisotropic Reynolds stress tensor for turbulent duct flows. Furthermore, we also explore the application of curriculum learning to data-driven turbulence modeling.

Hot Jupiters are very good targets for transmission spectroscopy analysis. Their atmospheres have a large scale height implying a high signal to noise ratio. As these planets orbit close to their stars, they often present strong thermal and chemical hetereogeneities between the day and the night side of their atmosphere. For the hottest ones, the thermal dissociation of several species occurs in their atmospheres which leads to a stronger chemical dichotomy between the two hemispheres. It has already been shown that the current retrieval algorithms, which are using 1D forward models, find biased molecular abundances in ultra hot Jupiters. Here, we quantify the effective temperature domain over which these biases are present. We use a set of 12 simulations of typical Hot Jupiters from Teq = 1000 K to Teq = 2100 K performed with the Substellar and Planetary Atmospheric Radiation and Circulation global climate model and generate transmission spectra that fully account for the 3D structure of the atmosphere with Pytmosph3R. These spectra are then analyzed using the 1D TauREx retrieval code. We find that for JWST-like data, accounting for non-isothermal vertical temperature profiles is required over the whole temperature range. We further find that 1D retrieval codes start to estimate wrong parameter values for planets with equilibrium temperatures greater than 1400 K if there are absorbers in the visible able to create a hot stratosphere. The high temperatures at low pressures indeed entail a thermal dissociation of species which creates a strong chemical day-night dichotomy. As a by-product, we demonstrate that when using synthetic observations to assess the detectability of a given feature or process using a Bayesian framework, it is valid to use non-randomized input data, as long as the anticipated observational uncertainties are correctly taken into account., Comment: 18 pages, 13 figures, 3 tables, 1 appendix. Received 3 August 2021; revised 17 August 2021; accepted 14 October 2021 in A&A

The Reynolds-averaged Navier-Stokes (RANS) equations are widely used in turbulence applications. They require accurately modeling the anisotropic Reynolds stress tensor, for which traditional Reynolds stress closure models only yield reliable results in some flow configurations. In the last few years, there has been a surge of work aiming at using data-driven approaches to tackle this problem. The majority of previous work has focused on the development of fully-connected networks for modeling the anisotropic Reynolds stress tensor. In this paper, we expand upon recent work for turbulent channel flow and develop new convolutional neural network (CNN) models that are able to accurately predict the normalized anisotropic Reynolds stress tensor. We apply the new CNN model to a number of one-dimensional turbulent flows. Additionally, we present interpretability techniques that help drive the model design and provide guidance on the model behavior in relation to the underlying physics.

Deep learning is a powerful tool whose applications in quantitative finance are growing every day. Yet, artificial neural networks behave as black boxes and this hinders validation and accountability processes. Being able to interpret the inner functioning and the input-output relationship of these networks has become key for the acceptance of such tools. In this paper we focus on the calibration process of a stochastic volatility model, a subject recently tackled by deep learning algorithms. We analyze the Heston model in particular, as this model's properties are well known, resulting in an ideal benchmark case. We investigate the capability of local strategies and global strategies coming from cooperative game theory to explain the trained neural networks, and we find that global strategies such as Shapley values can be effectively used in practice. Our analysis also highlights that Shapley values may help choose the network architecture, as we find that fully-connected neural networks perform better than convolutional neural networks in predicting and interpreting the Heston model prices to parameters relationship.

In the fields of co-simulation and component-based modelling, designers import models as building blocks to create a composite model that provides more complex functionalities. Modelling tools perform instantaneous cycle detection (ICD) on the composite models having feedback loops to reject the models if the loops are mathematically unsound and to improve simulation performance. In this case, the analysis relies heavily on the availability of dependency information from the imported models. However, the cycle detection problem becomes harder when the model's input to output dependencies are mode-dependent, i.e. changes for certain events generated internally or externally as inputs. The number of possible modes created by composing such models increases significantly and unknown factors such as environmental inputs make the offline (statical) ICD a difficult task. In this paper, an online ICD method is introduced to address this issue for the models used in cyber-physical systems. The method utilises an oracle as a central source of information that can answer whether the individual models can make mode transition without creating instantaneous cycles. The oracle utilises three types of data-structures created offline that are adaptively chosen during online (runtime) depending on the frequency as well as the number of models that make mode transitions. During the analysis, the models used online are stalled from running, resulting in the discrepancy with the physical system. The objective is to detect an absence of the instantaneous cycle while minimising the stall time of the model simulation that is induced from the analysis. The benchmark results show that our method is an adequate alternative to the offline analysis methods and significantly reduces the analysis time., Comment: \c{opyright} 2021. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/

Artificial Intelligence (AI) and Internet of Things (IoT) applications are rapidly growing in today's world where they are continuously connected to the internet and process, store and exchange information among the devices and the environment. The cloud and edge platform is very crucial to these applications due to their inherent compute-intensive and resource-constrained nature. One of the foremost challenges in cloud and edge resource allocation is the efficient management of computation and communication resources to meet the performance and latency guarantees of the applications. The heterogeneity of cloud resources (processors, memory, storage, bandwidth), variable cost structure and unpredictable workload patterns make the design of resource allocation techniques complex. Numerous research studies have been carried out to address this intricate problem. In this paper, the current state-of-the-art resource allocation techniques for the cloud continuum, in particular those that consider time-sensitive applications, are reviewed. Furthermore, we present the key challenges in the resource allocation problem for the cloud continuum, a taxonomy to classify the existing literature and the potential research gaps., Comment: 15 pages. A version submitted to Information Technology | De Gruyter

Cyber Physical Systems are systems controlled or monitored by computer-based programs, tightly integrated networks, sensors, and actuators. Software development of CPS has become so difficult that it represents most of the cost of CPS production. In addition, it is interesting to note that the integration, verification and validation of software in CPS require more efforts than the analysis, design, and implementation activities. The main reason is that these activities are conducted late in the development process and issues discovered at this stage of the process will require to rework artifacts produced in the previous activities (i.e. analysis, design and/or implementation). In this document, we present our work aiming to improve the reliability of software development in the domain of CPS. In this context, we define the reliability of the development process as its capacity to deliver intermediate artifacts for which the rework effort would be as small as possible. This problem is very difficult for general purpose software (i.e. used on desktop computers or servers), and even more difficult for software in CPS. The main reason is that software in CPS is often critical, real-time and embedded on domain specific execution platforms. As a consequence, non-functional properties (also called quality attributes) of software applications in CPS are often as important and difficult to satisfy as the logical correctness of these applications. In order to the improve the reliability of software development in the domain of CPS, we propose a Model Driven Engineering (MDE) method based on step-wise refinements of software architecture descriptions. The results obtained with this method are summarized in this habilitation thesis.

Transmission spectroscopy provides us with information on the atmospheric properties at the limb, which is often intuitively assumed to be a narrow annulus around the planet. Consequently, studies have focused on the effect of atmospheric horizontal heterogeneities along the limb. Here we demonstrate that the region probed in transmission -- the limb -- extends significantly toward the day and night sides of the planet. We show that the strong day-night thermal and compositional gradients expected on synchronous exoplanets create sufficient heterogeneities across the limb to result in important systematic effects on the spectrum and bias its interpretation. To quantify these effects, we developed a 3D radiative transfer model able to generate transmission spectra of atmospheres based on 3D atmospheric structures. We first produce synthetic JWST observations from a simulation of GJ 1214b and demonstrate the necessity of a real 3D approach to model data for such precise observatories. Second, we investigate how day-night temperature gradients cause a systematic bias in retrieval analysis performed with 1D forward models. For that purpose we synthesize a large set of forward spectra for prototypical HD 209458 b and GJ 1214 b type planets varying the temperatures of the day and night sides as well as the width of the transition region. We then perform typical retrieval analyses and compare the retrieved parameters to the ground truth of the input model. This study reveals systematic biases on the retrieved temperature (found to be higher than the terminator temperature) and abundances. This is due to the fact that the hotter dayside is more extended vertically and screens the nightside. These biases are difficult to detect as the 1D profiles used in the retrieval are found to provide an excellent match to the observed spectra. This needs to be kept in mind when interpreting real data., Comment: Accepted for publication at Astronomy and Astrophysics. Abstract abridged to meet ArXiv size limit

We provide the catalogue of all transit-like features, including false alarms, detected by the CoRoT exoplanet teams in the 177 454 light curves of the mission. All these detections have been re-analysed with the same softwares so that to ensure their homogeneous analysis. Although the vetting process involves some human evaluation, it also involves a simple binary flag system over basic tests: detection significance, presence of a secondary, difference between odd and even depths, colour dependence, V-shape transit, and duration of the transit. We also gathered the information from the large accompanying ground-based programme carried out on the planet candidates and checked how useful the flag system could have been at the vetting stage of the candidates. In total, we identified and separated 824 false alarms of various kind, 2269 eclipsing binaries among which 616 are contact binaries and 1653 are detached ones, 37 planets and brown dwarfs, and 557 planet candidates. For the planet candidates, the catalogue gives not only their transit parameters but also the products of their light curve modelling, together with a summary of the outcome of follow-up observations when carried out and their current status. Among the planet candidates whose nature remains unresolved, we estimate that 8 +/- 3 planets are still to be identified. We derived planet and brown dwarf occurrences and confirm disagreements with Kepler estimates: small-size planets with orbital period less than ten days are underabundant by a factor of three in the CoRoT fields whereas giant planets are overabundant by a factor of two. These preliminary results would however deserve further investigations using the recently released CoRoT light curves that are corrected of the various instrumental effects and a homogeneous analysis of the stellar populations observed by the two missions., Comment: Full tables will be provided online at CDS

CoRoT-9b is one of the rare long-period (P=95.3 days) transiting giant planets with a measured mass known to date. We present a new analysis of the CoRoT-9 system based on five years of radial-velocity (RV) monitoring with HARPS and three new space-based transits observed with CoRoT and Spitzer. Combining our new data with already published measurements we redetermine the CoRoT-9 system parameters and find good agreement with the published values. We uncover a higher significance for the small but non-zero eccentricity of CoRoT-9b ($e=0.133^{+0.042}_{-0.037}$) and find no evidence for additional planets in the system. We use simulations of planet-planet scattering to show that the eccentricity of CoRoT-9b may have been generated by an instability in which a $\sim 50~M_\oplus$ planet was ejected from the system. This scattering would not have produced a spin-orbit misalignment, so we predict that CoRoT-9b orbit should lie within a few degrees of the initial plane of the protoplanetary disk. As a consequence, any significant stellar obliquity would indicate that the disk was primordially tilted., Comment: 11 pages, 8 figures, accepted for publication in Astronomy and Astrophysics

Stellar activity and convection-related surface structures might cause bias in planet detection and characterization that use these transits. Surface convection simulations help to quantify the granulation signal. We used realistic three-dimensional radiative hydrodynamical simulations from the Stagger grid and synthetic images computed with the radiative transfer code Optim3D to model the transits of three prototype planets: a hot Jupiter, a hot Neptune, and a terrestrial planet. We computed intensity maps from RHD simulations of the Sun and a K-dwarf star at different wavelength bands from optical to far-infrared. We modeled the transit using synthetic stellar-disk images and emulated the temporal variation of the granulation intensity. We identified two types of granulation noise that act simultaneously during the planet transit: (i) the intrinsic change in the granulation pattern with timescales smaller than the usual planet transit, and (ii) the fact that the transiting planet occults isolated regions of the photosphere that differ in local surface brightness. Our modeling approach shows that the granulation pattern has a non-negligible effect on the light curve depth during the transit, and, consequentially on the determination of the radius of the planet transiting. The granulation noise appears to be correlated among the different wavelength ranges either in the visible or in the infrared regions. The granulation has to be considered as an intrinsic uncertainty (as a result of stellar variability) on the precise measurements of exoplanet transits of planets. The full characterization of the granulation is essential for determining the degree of uncertainty on the planet parameters. In this context, the use of 3D RHD simulations is important to measure the convection-related fluctuations., Comment: Accepted for publication in Section 10. Planets and planetary systems of Astronomy and Astrophysics

Clouds have been shown to be present in many exoplanetary atmospheres. Cloud formation modeling predicts considerable inhomogeneities of cloud cover, consistent with optical phase curve observations. However, optical phase curves cannot resolve some existing degeneracies between cloud location and cloud optical properties. We present a conceptually simple technique to detect inhomogeneous cloud cover on exoplanets. Such an inhomogeneous cloud cover produces an asymmetric primary transit of the planet in front of the host star. Asymmetric transits produce characteristic residuals compared to a standard symmetric model. Furthermore, bisector spans can be used to determine asymmetries in the transit light curve. We apply a model of asymmetric transits to the light curves of HAT-P-7b, Kepler-7b and HD209458b and search for possible cloud signatures. The nearly uninterrupted Kepler photometry is particularly well-suited for this method since it allows for a very high time resolution. We do not find any statistically sound cloud signature in the data of the considered planets. For HAT-P-7b, a tentative detection of an asymmetric cloud cover is found, consistent with analysis of the optical phase curve. Based on Bayesian probability arguments, a symmetric model with an offset in the transit ephemeris remains, however, the most viable model. Still, this work demonstrates that for suitable targets, namely low-gravity planets around bright stars, the method can be used to constrain cloud cover characteristics and is thus a helpful additional tool to study exoplanetary atmospheres., Comment: accepted for publication in Astronomy&Astrophysics on Feb. 12th, 11 pages, 20 figures, 2 tables, + Appendix

Our purpose is to offer a logical analysis of the system of units and to explore possible paths towards a consistent and unified system with an original perspective. The path taken here builds on the fact that, thanks to optical or matter-wave interferometry, any measurement can be reduced to a dimensionless phase measurement and we follow this simple guiding line. We finally show how one could progress even further on the path of a synthetic framework for fundamental metrology based upon pure geometry in five dimensions.

Basic atmospheric properties such as albedo and heat redistribution between day and nightside have been inferred for a number of planets using observations of secondary eclipses and thermal phase curves. Optical phase curves have not yet been used to constrain these atmospheric properties consistently. We re-model previously published phase curves of CoRoT-1b, TrES-2b and HAT-P-7b and infer albedos and recirculation efficiencies. These are then compared to previous estimates based on secondary eclipse data. We use a physically consistent model to construct optical phase curves. This model takes Lambertian reflection, thermal emission, ellipsoidal variations and Doppler boosting into account. CoRoT-1b shows a non-negligible scattering albedo (0.11

We report the detection of a rare transiting brown dwarf with a mass of 59 M_Jup and radius of 1.1 R_Jup around the metal-rich, [Fe/H] = +0.44, G9V star CoRoT-33. The orbit is eccentric (e = 0.07) with a period of 5.82 d. The companion, CoRoT-33b, is thus a new member in the so-called brown dwarf desert. The orbital period is within 3% to a 3:2 resonance with the rotational period of the star. CoRoT-33b may be an important test case for tidal evolution studies. The true frequency of brown dwarfs close to their host stars (P < 10 d) is estimated to be approximately 0.2% which is about six times smaller than the frequency of hot Jupiters in the same period range. We suspect that the frequency of brown dwarfs declines faster with decreasing period than that of giant planets., Comment: 13 pages, 8 figures, accepted for publication in A&A

Accurate molecular spectroscopy in the mid-infrared region allows precision measurements of fundamental constants. For instance, measuring the linewidth of an isolated Doppler-broadened absorption line of ammonia around 10 $\mu$m enables a determination of the Boltzmann constant k B. We report on our latest measurements. By fitting this lineshape to several models which include Dicke narrowing or speed-dependent collisional effects, we find that a determination of k B with an uncertainty of a few ppm is reachable. This is comparable to the best current uncertainty obtained using acoustic methods and would make a significant contribution to any new value of k B determined by the CODATA. Furthermore, having multiple independent measurements at these accuracies opens the possibility of defining the kelvin by fixing k B, an exciting prospect considering the upcoming redefinition of the International System of Units., Comment: proceedings of the ICAP 2012 conference (23rd International Conference on Atomic Physics). arXiv admin note: text overlap with arXiv:1309.4549

We present constraints on neutrino masses, the primordial fluctuation spectrum from inflation, and other parameters of the $\Lambda$CDM model, using the one-dimensional Ly$\alpha$-forest power spectrum measured by Palanque-Delabrouille et al. (2013) from SDSS-III/BOSS, complemented by Planck 2015 cosmic microwave background (CMB) data and other cosmological probes. This paper improves on the previous analysis by Palanque-Delabrouille et al. (2015) by using a more powerful set of calibrating hydrodynamical simulations that reduces uncertainties associated with resolution and box size, by adopting a more flexible set of nuisance parameters for describing the evolution of the intergalactic medium, by including additional freedom to account for systematic uncertainties, and by using Planck 2015 constraints in place of Planck 2013. Fitting Ly$\alpha$ data alone leads to cosmological parameters in excellent agreement with the values derived independently from CMB data, except for a weak tension on the scalar index $n_s$. Combining BOSS Ly$\alpha$ with Planck CMB constrains the sum of neutrino masses to $\sum m_\nu < 0.12$ eV (95\% C.L.) including all identified systematic uncertainties, tighter than our previous limit (0.15 eV) and more robust. Adding Ly$\alpha$ data to CMB data reduces the uncertainties on the optical depth to reionization $\tau$, through the correlation of $\tau$ with $\sigma_8$. Similarly, correlations between cosmological parameters help in constraining the tensor-to-scalar ratio of primordial fluctuations $r$. The tension on $n_s$ can be accommodated by allowing for a running ${\mathrm d}n_s/{\mathrm d}\ln k$. Allowing running as a free parameter in the fits does not change the limit on $\sum m_\nu$. We discuss possible interpretations of these results in the context of slow-roll inflation., Comment: 24 pages, 10 figures, change in notation for running of scalar index to make it compatible with that of Planck, article accepted for publication in JCAP

Context. We present the discovery of two transiting extrasolar planets by the satellite CoRoT. Aims. We aim at a characterization of the planetary bulk parameters, which allow us to further investigate the formation and evolution of the planetary systems and the main properties of the host stars. Methods. We used the transit light curve to characterize the planetary parameters relative to the stellar parameters. The analysis of HARPS spectra established the planetary nature of the detections, providing their masses. Further photometric and spectroscopic ground-based observations provided stellar parameters (log g,Teff,v sin i) to characterize the host stars. Our model takes the geometry of the transit to constrain the stellar density into account, which when linked to stellar evolutionary models, determines the bulk parameters of the star. Because of the asymmetric shape of the light curve of one of the planets, we had to include the possibility in our model that the stellar surface was not strictly spherical. Results. We present the planetary parameters of CoRoT-28b, a Jupiter-sized planet (mass 0.484+/-0.087MJup; radius 0.955+/-0.066RJup) orbiting an evolved star with an orbital period of 5.208 51 +/- 0.000 38 days, and CoRoT-29b, another Jupiter-sized planet (mass 0.85 +/- 0.20MJup; radius 0.90 +/- 0.16RJup) orbiting an oblate star with an orbital period of 2.850 570 +/- 0.000 006 days. The reason behind the asymmetry of the transit shape is not understood at this point. Conclusions. These two new planetary systems have very interesting properties and deserve further study, particularly in the case of the star CoRoT-29., Comment: 19 pages, 21 figures. Revised version to update affiliations and title, according to published version

The discovery of almost 2000 exoplanets has revealed an unexpectedly diverse planet population. Observations to date have shown that our Solar System is certainly not representative of the general population of planets in our Milky Way. The key science questions that urgently need addressing are therefore: What are exoplanets made of? Why are planets as they are? What causes the exceptional diversity observed as compared to the Solar System? EChO (Exoplanet Characterisation Observatory) has been designed as a dedicated survey mission for transit and eclipse spectroscopy capable of observing a large and diverse planet sample within its four-year mission lifetime. EChO can target the atmospheres of super-Earths, Neptune-like, and Jupiter-like planets, in the very hot to temperate zones (planet temperatures of 300K-3000K) of F to M-type host stars. Over the next ten years, several new ground- and space-based transit surveys will come on-line (e.g. NGTS, CHEOPS, TESS, PLATO), which will specifically focus on finding bright, nearby systems. The current rapid rate of discovery would allow the target list to be further optimised in the years prior to EChO's launch and enable the atmospheric characterisation of hundreds of planets. Placing the satellite at L2 provides a cold and stable thermal environment, as well as a large field of regard to allow efficient time-critical observation of targets randomly distributed over the sky. A 1m class telescope is sufficiently large to achieve the necessary spectro-photometric precision. The spectral coverage (0.5-11 micron, goal 16 micron) and SNR to be achieved by EChO, thanks to its high stability and dedicated design, would enable a very accurate measurement of the atmospheric composition and structure of hundreds of exoplanets., Comment: 50 pages, 30 figures. Experimental Astronomy

We present constraints on the parameters of the $\Lambda$CDM cosmological model in the presence of massive neutrinos, using the one-dimensional Ly$\alpha$ forest power spectrum obtained with the Baryon Oscillation Spectroscopic Survey (BOSS) of the Sloan Digital Sky Survey (SDSS) by Palanque-Delabrouille et al. (2013), complemented by additional cosmological probes. The interpretation of the measured Ly$\alpha$ spectrum is done using a second-order Taylor expansion of the simulated power spectrum. BOSS Ly$\alpha$ data alone provide better bounds than previous Ly$\alpha$ results, but are still poorly constraining, especially for the sum of neutrino masses $\sum m_\nu$, for which we obtain an upper bound of 1.1~eV (95\% CL), including systematics for both data and simulations. Ly$\alpha$ constraints on $\Lambda$CDM parameters and neutrino masses are compatible with CMB bounds from the Planck collaboration. Interestingly, the combination of Ly$\alpha$ with CMB data reduces the uncertainties significantly, due to very different directions of degeneracy in parameter space, leading to the strongest cosmological bound to date on the total neutrino mass, $\sum m_\nu < 0.15$~eV at 95\% CL (with a best-fit in zero). Adding recent BAO results further tightens this constraint to $\sum m_\nu < 0.14$~eV at 95\% CL. This bound is nearly independent of the statistical approach used, and of the different combinations of CMB and BAO data sets considered in this paper in addition to Ly$\alpha$. Given the measured values of the two squared mass differences $\Delta m^2$, this result tends to favor the normal hierarchy scenario against the inverted hierarchy scenario for the masses of the active neutrino species., Comment: 38 pages, 14 figures, version accepted for publication in JCAP

The CoRoT satellite has provided high-precision photometric light curves for more than 163,000 stars and found several hundreds of transiting systems compatible with a planetary scenario. If ground-based velocimetric observations are the best way to identify the actual planets among many possible configurations of eclipsing binary systems, recent transit surveys have shown that it is not always within reach of the radial-velocity detection limits. In this paper, we present a transiting exoplanet candidate discovered by CoRoT whose nature cannot be established from ground-based observations, and where extensive analyses are used to validate the planet scenario. They are based on observing constraints from radial-velocity spectroscopy, adaptive optics imaging and the CoRoT transit shape, as well as from priors on stellar populations, planet and multiple stellar systems frequency. We use the fully Bayesian approach developed in the PASTIS analysis software, and conclude that the planet scenario is at least 1400 times more probable than any other false positive scenario. The primary star is a metallic solar-like dwarf, with Ms = 1.099+-0.049 Msun and Rs = 1.136 (+0.038,-0.090) Rsun . The validated planet has a radius of Rp = 4.88 (+0.17,-0.39) RE and mass less than 49 ME. Its mean density is smaller than 2.56 g/cm^3 and orbital period is 9.7566+-0.0012 days. This object, called CoRoT-22 b, adds to a large number of validated Kepler planets. These planets do not have a proper measurement of the mass but allow statistical characterization of the exoplanet population.