Search

We propose two methods for the unsupervised detection of communities in undirected multiplex networks. These networks consist of multiple layers that record different relationships between the same entities or incorporate data from different sources. Both methods are formulated as gradient flows of suitable energy functionals: the first (MPBTV) builds on the minimization of a balanced total variation functional, which we show to be equivalent to multiplex modularity maximization, while the second (DGFM3) directly maximizes multiplex modularity. The resulting non-linear matrix-valued ordinary differential equations (ODEs) are solved efficiently by a graph Merriman--Bence--Osher (MBO) scheme. Key to the efficiency is the approximate integration of the discrete linear differential operators by truncated eigendecompositions in the matrix exponential function. Numerical experiments on several real-world multiplex networks show that our methods are competitive with the state of the art with respect to various metrics. Their major benefit is a significant reduction of computational complexity leading to runtimes that are orders of magnitude faster for large multiplex networks.

Isogeometric Analysis (IgA) is a versatile method for the discretization of partial differential equations on complex domains, which arise in various applications of science and engineering. Some complex geometries can be better described as a computational domain by a multi-patch approach, where each patch is determined by a tensor product Non-Uniform Rational Basis Splines (NURBS) parameterization. This allows on the one hand to consider the problem of the complex assembly of mass or stiffness matrices (or tensors) over the whole geometry locally on the individual smaller patches, and on the other hand it is possible to perform local mesh refinements independently on each patch, allowing efficient local refinement in regions of high activity where higher accuracy is required, while coarser meshes can be used elsewhere. Furthermore, the information about differing material models or properties that are to apply in a subdomain of the geometry can be included in the patch in which this subdomain is located. For this it must be ensured that the approximate solution is continuous over the entire computational domain and therefore at the interfaces of two (or more) patches. The most promising approach for this problem, which transfers the idea of Finite Element Tearing and Interconnecting (FETI) methods into the isogeometric setup, was the IsogEometric Tearing and Interconnecting (IETI) method, where by introducing a constraints matrix and associated Lagrange multipliers and formulating it into a dual problem, depending only on the Lagrange multipliers, continuity at the interfaces was ensured in solving the resulting system. In this paper we illustrate that low-rank methods based on the tensor-train format can be generalised for a multi-patch IgA setup, which follows the IETI idea.

One of the main computational bottlenecks when working with kernel based learning is dealing with the large and typically dense kernel matrix. Techniques dealing with fast approximations of the matrix vector product for these kernel matrices typically deteriorate in their performance if the feature vectors reside in higher-dimensional feature spaces. We here present a technique based on the non-equispaced fast Fourier transform (NFFT) with rigorous error analysis. We show that this approach is also well suited to allow the approximation of the matrix that arises when the kernel is differentiated with respect to the kernel hyperparameters; a problem often found in the training phase of methods such as Gaussian processes. We also provide an error analysis for this case. We illustrate the performance of the additive kernel scheme with fast matrix vector products on a number of data sets. Our code is available at https://github.com/wagnertheresa/NFFTAddKer, Comment: Official code https://github.com/wagnertheresa/NFFTAddKer

Real-world autonomous driving systems must make safe decisions in the face of rare and diverse traffic scenarios. Current state-of-the-art planners are mostly evaluated on real-world datasets like nuScenes (open-loop) or nuPlan (closed-loop). In particular, nuPlan seems to be an expressive evaluation method since it is based on real-world data and closed-loop, yet it mostly covers basic driving scenarios. This makes it difficult to judge a planner's capabilities to generalize to rarely-seen situations. Therefore, we propose a novel closed-loop benchmark interPlan containing several edge cases and challenging driving scenarios. We assess existing state-of-the-art planners on our benchmark and show that neither rule-based nor learning-based planners can safely navigate the interPlan scenarios. A recently evolving direction is the usage of foundation models like large language models (LLM) to handle generalization. We evaluate an LLM-only planner and introduce a novel hybrid planner that combines an LLM-based behavior planner with a rule-based motion planner that achieves state-of-the-art performance on our benchmark.

In this paper, we focus on efficient methods to solve discretized linear systems obtained from eddy current optimal control problems in an all-at-once approach. We construct a new low-rank matrix equation method based on a special splitting of the coefficient matrix and the Krylov-plus-inverted-Krylov (KPIK) algorithm. Firstly, we rewrite the resulting discretized linear system in a matrix-equation form. Then using the KPIK algorithm, we can obtain the low-rank approximation solution. The new method is named the splitting-based Krylov-plus-inverted-Krylov (SKPIK) method. The SKPIK method can not only solve the large and sparse discretized systems fast but also overcomes the storage problem. Theoretical results about the existence of the low-rank solutions are given. Numerical experiments are used to illustrate the performance of the new low-rank matrix equation method by compared with some existing classical efficient methods., Comment: 18

To determine the optimal set of hyperparameters of a Gaussian process based on a large number of training data, both a linear system and a trace estimation problem must be solved. In this paper, we focus on establishing numerical methods for the case where the covariance matrix is given as the sum of possibly multiple Kronecker products, i.e., can be identified as a tensor. As such, we will represent this operator and the training data in the tensor train format. Based on the AMEn method and Krylov subspace methods, we derive an efficient scheme for computing the matrix functions required for evaluating the gradient and the objective function in hyperparameter optimization.

In this paper we consider the numerical solution to the soft-margin support vector machine optimization problem. This problem is typically solved using the SMO algorithm, given the high computational complexity of traditional optimization algorithms when dealing with large-scale kernel matrices. In this work, we propose employing an NFFT-accelerated matrix-vector product using an ANOVA decomposition for the feature space that is used within an interior point method for the overall optimization problem. As this method requires the solution of a linear system of saddle point form we suggest a preconditioning approach that is based on low-rank approximations of the kernel matrix together with a Krylov subspace solver. We compare the accuracy of the ANOVA-based kernel with the default LIBSVM implementation. We investigate the performance of the different preconditioners as well as the accuracy of the ANOVA kernel on several large-scale datasets., Comment: Official Code https://github.com/wagnertheresa/NFFTSVMipm

Core-periphery detection aims to separate the nodes of a complex network into two subsets: a core that is densely connected to the entire network and a periphery that is densely connected to the core but sparsely connected internally. The definition of core-periphery structure in multiplex networks that record different types of interactions between the same set of nodes on different layers is nontrivial since a node may belong to the core in some layers and to the periphery in others. We propose a nonlinear spectral method for multiplex networks that simultaneously optimises a node and a layer coreness vector by maximising a suitable nonconvex homogeneous objective function by a provably convergent alternating fixed point iteration. We derive a quantitative measure for the quality of a given multiplex core-periphery structure that allows the determination of the optimal core size. Numerical experiments on synthetic and real-world networks illustrate that our approach is robust against noisy layers and significantly outperforms baseline methods while improving the latter with our novel optimised layer coreness weights. As the runtime of our method depends linearly on the number of edges of the network it is scalable to large-scale multiplex networks.

Automated driving has the potential to revolutionize personal, public, and freight mobility. Beside accurately perceiving the environment, automated vehicles must plan a safe, comfortable, and efficient motion trajectory. To promote safety and progress, many works rely on modules that predict the future motion of surrounding traffic. Modular automated driving systems commonly handle prediction and planning as sequential, separate tasks. While this accounts for the influence of surrounding traffic on the ego vehicle, it fails to anticipate the reactions of traffic participants to the ego vehicle's behavior. Recent methods increasingly integrate prediction and planning in a joint or interdependent step to model bidirectional interactions. To date, a comprehensive overview of different integration principles is lacking. We systematically review state-of-the-art deep learning-based planning systems, and focus on how they integrate prediction. Different facets of the integration ranging from system architecture to high-level behavioral aspects are considered and related to each other. Moreover, we discuss the implications, strengths, and limitations of different integration principles. By pointing out research gaps, describing relevant future challenges, and highlighting trends in the research field, we identify promising directions for future research.

The operation of machine tools often demands a highly accurate knowledge of the tool center point's (TCP) position. The displacement of the TCP over time can be inferred from thermal models, which comprise a set of geometrically coupled heat equations. Each of these equations represents the temperature in part of the machine, and they are often formulated on complicated geometries. The accuracy of the TCP prediction depends highly on the accuracy of the model parameters, such as heat exchange parameters, and the initial temperature. Thus it is of utmost interest to determine the influence of these parameters on the TCP displacement prediction. In turn, the accuracy of the parameter estimate is essentially determined by the measurement accuracy and the sensor placement. Determining the accuracy of a given sensor configuration is a key prerequisite of optimal sensor placement. We develop here a thermal model for a particular machine tool. On top of this model we propose two numerical algorithms to evaluate any given thermal sensor configuration with respect to its accuracy. We compute the posterior variances from the posterior covariance matrix with respect to an uncertain initial temperature field. The full matrix is dense and potentially very large, depending on the model size. Thus, we apply a low-rank method to approximate relevant entries, i.e. the variances on its diagonal. We first present a straightforward way to compute this approximation which requires computation of the model sensitivities with with respect to the initial values. Additionally, we present a low-rank tensor method which exploits the underlying system structure. We compare the efficiency of both algorithms with respect to runtime and memory requirements and discuss their respective advantages with regard to optimal sensor placement problems.

Predicting the future motion of observed vehicles is a crucial enabler for safe autonomous driving. The field of motion prediction has seen large progress recently with state-of-the-Art (sotA) models achieving impressive results on large-scale public benchmarks. However, recent work revealed that learning-based methods are prone to predict off-road trajectories in challenging scenarios. These can be created by perturbing existing scenarios with additional turns in front of the target vehicle while the motion history is left unchanged. We argue that this indicates that SotA models do not consider the map information sufficiently and demonstrate how this can be solved by representing model inputs and outputs in a Frenet frame defined by lane centreline sequences. To this end, we present a general wrapper that leverages a Frenet representation of the scene, and that can be applied to SotA models without changing their architecture. We demonstrate the effectiveness of this approach in a comprehensive benchmark using two SotA motion prediction models. Our experiments show that this reduces the off-road rate on challenging scenarios by more than 90% without sacrificing average performance.

We challenge the perceived consensus that the application of deep learning to solve the automated driving planning task necessarily requires huge amounts of real-world data or highly realistic simulation. Focusing on a roundabout scenario, we show that this requirement can be relaxed in favour of targeted, simplistic simulated data. A benefit is that such data can be easily generated for critical scenarios that are typically underrepresented in realistic datasets. By applying vanilla behavioural cloning almost exclusively to lightweight simulated data, we achieve reliable and comfortable driving in a real-world test vehicle. We leverage an incremental development approach that includes regular in-vehicle testing to identify sim-to-real gaps, targeted data augmentation, and training scenario variations. In addition to a detailed description of the methodology, we share our lessons learned, touching upon scenario generation, simulation features, and evaluation metrics.

We consider the solution of large stiff systems of ordinary differential equations with explicit exponential Runge--Kutta integrators. These problems arise from semi-discretized semi-linear parabolic partial differential equations on continuous domains or on inherently discrete graph domains. A series of results reduces the requirement of computing linear combinations of $\varphi$-functions in exponential integrators to the approximation of the action of a smaller number of matrix exponentials on certain vectors. State-of-the-art computational methods use polynomial Krylov subspaces of adaptive size for this task. They have the drawback that the required number of Krylov subspace iterations to obtain a desired tolerance increase drastically with the spectral radius of the discrete linear differential operator, e.g., the problem size. We present an approach that leverages rational Krylov subspace methods promising superior approximation qualities. We prove a novel a-posteriori error estimate of rational Krylov approximations to the action of the matrix exponential on vectors for single time points, which allows for an adaptive approach similar to existing polynomial Krylov techniques. We discuss pole selection and the efficient solution of the arising sequences of shifted linear systems by direct and preconditioned iterative solvers. Numerical experiments show that our method outperforms the state of the art for sufficiently large spectral radii of the discrete linear differential operators. The key to this are approximately constant numbers of rational Krylov iterations, which enable a near-linear scaling of the runtime with respect to the problem size.

High-dimensional data in the form of tensors are challenging for kernel classification methods. To both reduce the computational complexity and extract informative features, kernels based on low-rank tensor decompositions have been proposed. However, what decisive features of the tensors are exploited by these kernels is often unclear. In this paper we propose a novel kernel that is based on the Tucker decomposition. For this kernel the Tucker factors are computed based on re-weighting of the Tucker matrices with tuneable powers of singular values from the HOSVD decomposition. This provides a mechanism to balance the contribution of the Tucker core and factors of the data. We benchmark support tensor machines with this new kernel on several datasets. First we generate synthetic data where two classes differ in either Tucker factors or core, and compare our novel and previously existing kernels. We show robustness of the new kernel with respect to both classification scenarios. We further test the new method on real-world datasets. The proposed kernel has demonstrated a higher test accuracy than the state-of-the-art tensor train multi-way multi-level kernel, and a significantly lower computational time.

The field of motion prediction for automated driving has seen tremendous progress recently, bearing ever-more mighty neural network architectures. Leveraging these powerful models bears great potential for the closely related planning task. In this letter we propose a novel goal-conditioning method and show its potential to transform a state-of-the-art prediction model into a goal-directed planner. Our key insight is that conditioning prediction on a navigation goal at the behaviour level outperforms other widely adopted methods, with the additional benefit of increased model interpretability. We train our model on a large open-source dataset and show promising performance in a comprehensive benchmark.

The accurate prediction of physicochemical properties of chemical compounds in mixtures (such as the activity coefficient at infinite dilution $\gamma_{ij}^\infty$) is essential for developing novel and more sustainable chemical processes. In this work, we analyze the performance of previously-proposed GNN-based models for the prediction of $\gamma_{ij}^\infty$, and compare them with several mechanistic models in a series of 9 isothermal studies. Moreover, we develop the Gibbs-Helmholtz Graph Neural Network (GH-GNN) model for predicting $\ln \gamma_{ij}^\infty$ of molecular systems at different temperatures. Our method combines the simplicity of a Gibbs-Helmholtz-derived expression with a series of graph neural networks that incorporate explicit molecular and intermolecular descriptors for capturing dispersion and hydrogen bonding effects. We have trained this model using experimentally determined $\ln \gamma_{ij}^\infty$ data of 40,219 binary-systems involving 1032 solutes and 866 solvents, overall showing superior performance compared to the popular UNIFAC-Dortmund model. We analyze the performance of GH-GNN for continuous and discrete inter/extrapolation and give indications for the model's applicability domain and expected accuracy. In general, GH-GNN is able to produce accurate predictions for extrapolated binary-systems if at least 25 systems with the same combination of solute-solvent chemical classes are contained in the training set and a similarity indicator above 0.35 is also present. This model and its applicability domain recommendations have been made open-source at https://github.com/edgarsmdn/GH-GNN., Comment: Code available at: https://github.com/edgarsmdn/GH-GNN

In this paper we focus on comparing machine learning approaches for quantum graphs, which are metric graphs, i.e., graphs with dedicated edge lengths, and an associated differential operator. In our case the differential equation is a drift-diffusion model. Computational methods for quantum graphs require a careful discretization of the differential operator that also incorporates the node conditions, in our case Kirchhoff-Neumann conditions. Traditional numerical schemes are rather mature but have to be tailored manually when the differential equation becomes the constraint in an optimization problem. Recently, physics informed neural networks (PINNs) have emerged as a versatile tool for the solution of partial differential equations from a range of applications. They offer flexibility to solve parameter identification or optimization problems by only slightly changing the problem formulation used for the forward simulation. We compare several PINN approaches for solving the drift-diffusion on the metric graph.

The Cahn--Hilliard equations are a versatile model for describing the evolution of complex morphologies. In this paper we present a computational pipeline for the numerical solution of a ternary phase-field model for describing the nanomorphology of donor--acceptor semiconductor blends used in organic photovoltaic devices. The model consists of two coupled fourth-order partial differential equations that are discretized using a finite element approach. In order to solve the resulting large-scale linear systems efficiently, we propose a preconditioning strategy that is based on efficient approximations of the Schur-complement of a saddle point system. We show that this approach performs robustly with respect to variations in the discretization parameters. Finally, we outline that the computed morphologies can be used for the computation of charge generation, recombination, and transport in organic solar cells.

Kernel matrices are crucial in many learning tasks such as support vector machines or kernel ridge regression. The kernel matrix is typically dense and large-scale. Depending on the dimension of the feature space even the computation of all of its entries in reasonable time becomes a challenging task. For such dense matrices the cost of a matrix-vector product scales quadratically with the dimensionality N , if no customized methods are applied. We propose the use of an ANOVA kernel, where we construct several kernels based on lower-dimensional feature spaces for which we provide fast algorithms realizing the matrix-vector products. We employ the non-equispaced fast Fourier transform (NFFT), which is of linear complexity for fixed accuracy. Based on a feature grouping approach, we then show how the fast matrix-vector products can be embedded into a learning method choosing kernel ridge regression and the conjugate gradient solver. We illustrate the performance of our approach on several data sets., Comment: Official Code https://github.com/wagnertheresa/NFFT4ANOVA

This work addresses optimal control problems governed by a linear time-dependent partial differential equation (PDE) as well as integer constraints on the control. Moreover, partial observations are assumed in the objective function. The resulting problem poses several numerical challenges due to the mixture of combinatorial aspects, induced by integer variables, and large scale linear algebra issues, arising from the PDE discretization. Since classical solution approaches such as the branch-and-bound framework are typically overwhelmed by such large-scale problems, this work extends an improved penalty algorithm proposed by the authors, to the time-dependent setting. The main contribution is a novel combination of an interior point method, preconditioning, and model order reduction yielding a tailored local optimization solver at the heart of the overall solution procedure. A thorough numerical investigation is carried out both for a Poisson problem as well as a convection-diffusion problem demonstrating the versatility of the approach., Comment: arXiv admin note: text overlap with arXiv:1907.06462

In recent years, graph neural networks (GNNs) have gained increasing popularity and have shown very promising results for data that are represented by graphs. The majority of GNN architectures are designed based on developing new convolutional and/or pooling layers that better extract the hidden and deeper representations of the graphs to be used for different prediction tasks. The inputs to these layers are mainly the three default descriptors of a graph, node features $(X)$, adjacency matrix $(A)$, and edge features $(W)$ (if available). To provide a more enriched input to the network, we propose a random walk data processing of the graphs based on three selected lengths. Namely, (regular) walks of length 1 and 2, and a fractional walk of length $\gamma \in (0,1)$, in order to capture the different local and global dynamics on the graphs. We also calculate the stationary distribution of each random walk, which is then used as a scaling factor for the initial node features ($X$). This way, for each graph, the network receives multiple adjacency matrices along with their individual weighting for the node features. We test our method on various molecular datasets by passing the processed node features to the network in order to perform several classification and regression tasks. Interestingly, our method, not using edge features which are heavily exploited in molecular graph learning, let a shallow network outperform well known deep GNNs.

When working with PDEs the reconstruction of a previous state often proves difficult. Good prior knowledge and fast computational methods are crucial to build a working reconstruction. We want to identify the heat sources on a three dimensional domain from later measurements under the assumption of small, distinct sources, such as hot chippings from a milling tool. This leads us to the need for a Prior reflecting this a priori information. Sparsity-inducing hyperpriors have proven useful for similar problems with sparse signal or image reconstruction. We combine the method of using a hierarchical Bayesian model with gamma hyperpriors to promote sparsity with low-rank computations for PDE systems in tensor train format., Comment: 24 pages, 7 figures

We study urban public transport systems by means of multiplex networks in which stops are represented as nodes and each line is represented by a layer. We determine and visualize public transport network orientations and compare them with street network orientations of the $36$ largest German as well as $18$ selected major European cities. We find that German urban public transport networks are mainly oriented in a direction close to the cardinal east-west axis, which usually coincides with one of two orthogonal preferential directions of the corresponding street network. While this behavior is present in only a subset of the considered European cities it remains true that none but one considered public transport network has a distinct north-south-like preferential orientation. Furthermore, we study the applicability of the class of matrix function-based centrality measures, which has recently been generalized from single-layer networks to layer-coupled multiplex networks, to our more general urban multiplex framework. Numerical experiments based on highly efficient and scalable methods from numerical linear algebra show promising results, which are in line with previous studies. The centrality measures allow detailed insights into geometrical properties of urban systems such as the spatial distribution of major transport axes, which can not be inferred from orientation plots. We comment on advantages over existing methodology, elaborate on the comparison of different measures and weight models, and present detailed hyper-parameter studies. All results are illustrated by demonstrative graphical representations.

Centrality measures identify and rank the most influential entities of complex networks. In this paper, we generalize matrix function-based centrality measures, which have been studied extensively for single-layer and temporal networks in recent years to layer-coupled multiplex networks. The layers of these networks can reflect different relationships and interactions between entities or changing interactions over time. We use the supra-adjacency matrix as network representation, which has already been used to generalize eigenvector centrality to temporal and multiplex networks. With a suitable choice of edge weights, the definition of single-layer matrix function-based centrality measures in terms of walks on networks carries over naturally to the multilayer case. In contrast to other walk-based centralities, matrix function-based centralities are parameterized measures, which have been shown to interpolate between (local) degree and (global) eigenvector centrality in the single-layer case. As the explicit evaluation of the involved matrix function expressions becomes infeasible for medium to large-scale networks, we present highly efficient approximation techniques from numerical linear algebra, which rely on Krylov subspace methods, Gauss quadrature, and stochastic trace estimation. We present extensive numerical studies on synthetic and real-world multiplex transportation, communication, and collaboration networks. The comparison with established multilayer centrality measures shows that our framework produces meaningful rankings of nodes, layers, and node-layer pairs. Furthermore, our experiments corroborate the theoretically indicated linear computational complexity of the employed numerical methods for sparse supra-adjacency matrices, which allows the efficient treatment of large-scale networks with the number of node-layer pairs of order $10^7$ or higher.

Time series data play an important role in many applications and their analysis reveals crucial information for understanding the underlying processes. Among the many time series learning tasks of great importance, we here focus on semi-supervised learning based on a graph representation of the data. Two main aspects are involved in this task. A suitable distance measure to evaluate the similarities between time series, and a learning method to make predictions based on these distances. However, the relationship between the two aspects has never been studied systematically in the context of graph-based learning. We describe four different distance measures, including (Soft) DTW and MPDist, a distance measure based on the Matrix Profile, as well as four successful semi-supervised learning methods, including the graph Allen--Cahn method and a Graph Convolutional Neural Network. We then compare the performance of the algorithms on binary classification data sets. In our findings we compare the chosen graph-based methods using all distance measures and observe that the results vary strongly with respect to the accuracy. As predicted by the ``no free lunch'' theorem, no clear best combination to employ in all cases is found. Our study provides a reproducible framework for future work in the direction of semi-supervised learning for time series with a focus on graph representations., Comment: 28 pages

Graph Convolutional Networks (GCNs) have proven to be successful tools for semi-supervised classification on graph-based datasets. We propose a new GCN variant whose three-part filter space is targeted at dense graphs. Examples include Gaussian graphs for 3D point clouds with an increased focus on non-local information, as well as hypergraphs based on categorical data. These graphs differ from the common sparse benchmark graphs in terms of the spectral properties of their graph Laplacian. Most notably we observe large eigengaps, which are unfavorable for popular existing GCN architectures. Our method overcomes these issues by utilizing the pseudoinverse of the Laplacian. Another key ingredient is a low-rank approximation of the convolutional matrix, ensuring computational efficiency and increasing accuracy at the same time. We outline how the necessary eigeninformation can be computed efficiently in each applications and discuss the appropriate choice of the only metaparameter, the approximation rank. We finally showcase our method's performance regarding runtime and accuracy in various experiments with real-world datasets.

We generalize a graph-based multiclass semi-supervised classification technique based on diffuse interface methods to multilayer graphs. Besides the treatment of various applications with an inherent multilayer structure, we present a very flexible approach that interprets high-dimensional data in a low-dimensional multilayer graph representation. Highly efficient numerical methods involving the spectral decomposition of the corresponding differential graph operators as well as fast matrix-vector products based on the nonequispaced fast Fourier transform (NFFT) enable the rapid treatment of large and high-dimensional data sets. We perform various numerical tests putting a special focus on image segmentation. In particular, we test the performance of our method on data sets with up to 10 million nodes per layer as well as up to 104 dimensions resulting in graphs with up to 52 layers. While all presented numerical experiments can be run on an average laptop computer, the linear dependence per iteration step of the runtime on the network size in all stages of our algorithm makes it scalable to even larger and higher-dimensional problems.

PDE-constrained optimization problems arise in a broad number of applications such as hyperthermia cancer treatment or blood flow simulation. Discretization of the optimization problem and using a Lagrangian approach result in a large-scale saddle-point system, which is challenging to solve, and acquiring a full space-time solution is often infeasible. We present a new framework to efficiently compute a low-rank approximation to the solution by reformulating the KKT system into a Sylvester-like matrix equation. This matrix equation is subsequently projected onto a small subspace via an iterative rational Krylov method and we obtain a reduced problem by imposing a Galerkin condition on its residual. In our work we discuss implementation details and dependence on the various problem parameters. Numerical experiments illustrate the performance of the new strategy also when compared to other low-rank approaches., Comment: 18 pages, 8 figures

An increasing amount of collected data are high-dimensional multi-way arrays (tensors), and it is crucial for efficient learning algorithms to exploit this tensorial structure as much as possible. The ever-present curse of dimensionality for high dimensional data and the loss of structure when vectorizing the data motivates the use of tailored low-rank tensor classification methods. In the presence of small amounts of training data, kernel methods offer an attractive choice as they provide the possibility for a nonlinear decision boundary. We develop the Tensor Train Multi-way Multi-level Kernel (TT-MMK), which combines the simplicity of the Canonical Polyadic decomposition, the classification power of the Dual Structure-preserving Support Vector Machine, and the reliability of the Tensor Train (TT) approximation. We show by experiments that the TT-MMK method is usually more reliable computationally, less sensitive to tuning parameters, and gives higher prediction accuracy in the SVM classification when benchmarked against other state-of-the-art techniques., Comment: 20 pages, 5 figures, 2 table, 2 Algorithm

Efficient numerical linear algebra is a core ingredient in many applications across almost all scientific and industrial disciplines. With this survey we want to illustrate that numerical linear algebra has played and is playing a crucial role in enabling and improving data science computations with many new developments being fueled by the availability of data and computing resources. We highlight the role of various different factorizations and the power of changing the representation of the data as well as discussing topics such as randomized algorithms, functions of matrices, and high-dimensional problems. We briefly touch upon the role of techniques from numerical linear algebra used within deep learning.

Differential algebraic Riccati equations are at the heart of many applications in control theory. They are time-depent, matrix-valued, and in particular nonlinear equations that require special methods for their solution. Low-rank methods have been used heavily computing a low-rank solution at every step of a time-discretization. We propose the use of an all-at-once space-time solution leading to a large nonlinear space-time problem for which we propose the use of a Newton-Kleinman iteration. Approximating the space-time problem in low-rank form requires fewer applications of the discretized differential operator and gives a low-rank approximation to the overall solution., Comment: 20 pages

Differential equations on metric graphs can describe many phenomena in the physical world but also the spread of information on social media. To efficiently compute the solution is a hard task in numerical analysis. Solving a design problem, where the optimal setup for a desired state is given, is even more challenging. In this work, we focus on the task of solving an optimization problem subject to a differential equation on a metric graph with the control defined on a small set of Dirichlet nodes. We discuss the discretization by finite elements and provide rigorous error bounds as well as an efficient preconditioning strategy to deal with the large-scale case. We show in various examples that the method performs very robustly., Comment: 20 pages, 4 figures

Optimal control problems including partial differential equation (PDE) as well as integer constraints merge the combinatorial difficulties of integer programming and the challenges related to large-scale systems resulting from discretized PDEs. So far, the Branch-and-Bound framework has been the most common solution strategy for such problems. In order to provide an alternative solution approach, especially in a large-scale context, this article investigates penalization techniques. Taking inspiration from a well-known family of existing exact penalty algorithms, a novel improved penalty algorithm is derived, whose key ingredients are a basin hopping strategy and an interior point method, both of which are specialized for the problem class. A thorough numerical investigation is carried out for a standard stationary test problem. Extensions to a convection-diffusion as well as a nonlinear test problem finally demonstrate the versatility of the approach.

Graph Convolutional Networks (GCNs) have proven to be successful tools for semi-supervised learning on graph-based datasets. For sparse graphs, linear and polynomial filter functions have yielded impressive results. For large non-sparse graphs, however, network training and evaluation becomes prohibitively expensive. By introducing low-rank filters, we gain significant runtime acceleration and simultaneously improved accuracy. We further propose an architecture change mimicking techniques from Model Order Reduction in what we call a reduced-order GCN. Moreover, we present how our method can also be applied to hypergraph datasets and how hypergraph convolution can be implemented efficiently.

Isogeometric analysis (IGA) has become one of the most popular methods for the discretization of partial differential equations motivated by the use of NURBS for geometric representations in industry and science. A crucial challenge lies in the solution of the discretized equations, which we discuss in this talk with a particular focus on PDE-constrained optimization discretized using IGA. The discretization results in a system of large mass and stiffness matrices, which are typically very costly to assemble. To reduce the computation time and storage requirements, low-rank tensor methods have become a promising tool. We present a framework for the assembly of these matrices in low-rank form as the sum of a small number of Kronecker products. For assembly of the smaller matrices only univariate integration is required. The resulting low rank Kronecker product structure of the mass and stiffness matrices can be used to solve a PDE-constrained optimization problem without assembling the actual system matrices. We present a framework which preserves and exploits the low-rank Kronecker product format for both the matrices and the solution. We use the block AMEn method to efficiently solve the corresponding KKT system of the optimization problem. We show several numerical experiments with 3D geometries to demonstrate that the low-rank assembly and solution drastically reduces the memory demands and computing times, depending on the approximation ranks of the domain., Comment: 23 pages, 9 figures

In this note, we consider the existence and uniqueness of the solution of a time-dependent optimal control problem constrained by a partial differential equation with uncertain inputs. Relying on the Lions' Lemma for deterministic problems, we furthermore characterize the optimal control of the stochastic problem.

Signed networks contain both positive and negative kinds of interactions like friendship and enmity. The task of node classification in non-signed graphs has proven to be beneficial in many real world applications, yet extensions to signed networks remain largely unexplored. In this paper we introduce the first analysis of node classification in signed social networks via diffuse interface methods based on the Ginzburg-Landau functional together with different extensions of the graph Laplacian to signed networks. We show that blending the information from both positive and negative interactions leads to performance improvement in real signed social networks, consistently outperforming the current state of the art., Comment: Accepted at ECML-PKDD 2019

The graph Laplacian is a standard tool in data science, machine learning, and image processing. The corresponding matrix inherits the complex structure of the underlying network and is in certain applications densely populated. This makes computations, in particular matrix-vector products, with the graph Laplacian a hard task. A typical application is the computation of a number of its eigenvalues and eigenvectors. Standard methods become infeasible as the number of nodes in the graph is too large. We propose the use of the fast summation based on the nonequispaced fast Fourier transform (NFFT) to perform the dense matrix-vector product with the graph Laplacian fast without ever forming the whole matrix. The enormous flexibility of the NFFT algorithm allows us to embed the accelerated multiplication into Lanczos-based eigenvalues routines or iterative linear system solvers and even consider other than the standard Gaussian kernels. We illustrate the feasibility of our approach on a number of test problems from image segmentation to semi-supervised learning based on graph-based PDEs. In particular, we compare our approach with the Nystr\"om method. Moreover, we present and test an enhanced, hybrid version of the Nystr\"om method, which internally uses the NFFT., Comment: 28 pages, 9 figures

We study the modeling and simulation of gas pipeline networks, with a focus on fast numerical methods for the simulation of transient dynamics. The obtained mathematical model of the underlying network is represented by a nonlinear differential algebraic equation (DAE). With our modeling, we reduce the number of algebraic constraints, which correspond to the $(2,2)$ block in our semi-explicit DAE model, to the order of junction nodes in the network, where a junction node couples at least three pipelines. We can furthermore ensure that the $(1, 1)$ block of all system matrices including the Jacobian is block lower triangular by using a specific ordering of the pipes of the network. We then exploit this structure to propose an efficient preconditioner for the fast simulation of the network. We test our numerical methods on benchmark problems of (well-)known gas networks and the numerical results show the efficiency of our methods., Comment: 26 pages, 22 figures

PDE-constrained optimization problems with control or state constraints are challenging from an analytical as well as numerical perspective. The combination of these constraints with a sparsity-promoting $\rm L^1$ term within the objective function requires sophisticated optimization methods. We propose the use of an Interior Point scheme applied to a smoothed reformulation of the discretized problem, and illustrate that such a scheme exhibits robust performance with respect to parameter changes. To increase the potency of this method we introduce fast and efficient preconditioners which enable us to solve problems from a number of PDE applications in low iteration numbers and CPU times, even when the parameters involved are altered dramatically.

Optimal transport problems pose many challenges when considering their numerical treatment. We investigate the solution of a PDE-constrained optimisation problem subject to a particular transport equation arising from the modelling of image metamorphosis. We present the nonlinear optimisation problem, and discuss the discretisation and treatment of the nonlinearity via a Gauss--Newton scheme. We then derive preconditioners that can be used to solve the linear systems at the heart of the (Gauss--)Newton method. With the optical flow in mind, we further propose the reduction of dimensionality by choosing a radial basis function discretisation that uses the centres of superpixels as the collocation points. Again, we derive suitable preconditioners that can be used for this formulation., Comment: 28 pages, 5 figures, submitted manuscript

This paper aims at the efficient numerical solution of stochastic eigenvalue problems. Such problems often lead to prohibitively high dimensional systems with tensor product structure when discretized with the stochastic Galerkin method. Here, we exploit this inherent tensor product structure to develop a globalized low-rank inexact Newton method with which we tackle the stochastic eigenproblem. We illustrate the effectiveness of our solver with numerical experiments.

Many problems in computational science and engineering are simultaneously characterized by the following challenging issues: uncertainty, nonlinearity, nonstationarity and high dimensionality. Existing numerical techniques for such models would typically require considerable computational and storage resources. This is the case, for instance, for an optimization problem governed by time-dependent Navier-Stokes equations with uncertain inputs. In particular, the stochastic Galerkin finite element method often leads to a prohibitively high dimensional saddle-point system with tensor product structure. In this paper, we approximate the solution by the low-rank Tensor Train decomposition, and present a numerically efficient algorithm to solve the optimality equations directly in the low-rank representation. We show that the solution of the vorticity minimization problem with a distributed control admits a representation with ranks that depend modestly on model and discretization parameters even for high Reynolds numbers. For lower Reynolds numbers this is also the case for a boundary control. This opens the way for a reduced-order modeling of the stochastic optimal flow control with a moderate cost at all stages., Comment: 29 pages

We consider the problem of estimating the uncertainty in statistical inverse problems using Bayesian inference. When the probability density of the noise and the prior are Gaussian, the solution of such a statistical inverse problem is also Gaussian. Therefore, the underlying solution is characterized by the mean and covariance matrix of the posterior probability density. However, the covariance matrix of the posterior probability density is full and large. Hence, the computation of such a matrix is impossible for large dimensional parameter spaces. It is shown that for many ill-posed problems, the Hessian matrix of the data misfit part has low numerical rank and it is therefore possible to perform a low-rank approach to approximate the posterior covariance matrix. For such a low-rank approximation, one needs to solve a forward partial differential equation (PDE) and the adjoint PDE in both space and time. This in turn gives $\mathcal{O}(n_x n_t)$ complexity for both, computation and storage, where $n_x$ is the dimension of the spatial domain and $n_t$ is the dimension of the time domain. Such computations and storage demand are infeasible for large problems. To overcome this obstacle, we develop a new approach that utilizes a recently developed low-rank in time algorithm together with the low-rank Hessian method. We reduce both the computational complexity and storage requirement from $\mathcal{O}(n_x n_t)$ to $\mathcal{O}(n_x + n_t)$. We use numerical experiments to illustrate the advantages of our approach.

PDE-constrained optimization aims at finding optimal setups for partial differential equations so that relevant quantities are minimized. Including sparsity promoting terms in the formulation of such problems results in more practically relevant computed controls but adds more challenges to the numerical solution of these problems. The needed $\rm L^1$-terms as well as additional inclusion of box control constraints require the use of semismooth Newton methods. We propose robust preconditioners for different formulations of the Newton's equation. With the inclusion of a line-search strategy and an inexact approach for the solution of the linear systems, the resulting semismooth Newton's method is feasible for practical problems. Our results are underpinned by a theoretical analysis of the preconditioned matrix. Numerical experiments illustrate the robustness of the proposed scheme.

Diffuse interface methods have recently been introduced for the task of semi-supervised learning. The underlying model is well-known in materials science but was extended to graphs using a Ginzburg--Landau functional and the graph Laplacian. We here generalize the previously proposed model by a non-smooth potential function. Additionally, we show that the diffuse interface method can be used for the segmentation of data coming from hypergraphs. For this we show that the graph Laplacian in almost all cases is derived from hypergraph information. Additionally, we show that the formerly introduced hypergraph Laplacian coming from a relaxed optimization problem is well suited to be used within the diffuse interface method. We present computational experiments for graph and hypergraph Laplacians., Comment: 13 pages, 11 figures

Recently, Garcke et al.[Garcke, Hinze, Kahle, A stable and linear time discretization for a thermodynamically consistent model for two-phase incompressible flow, Applied Numerical Mathematics 99, pp. 151-171, 2016] developed a consistent discretization scheme for a thermodynamically consistent diffuse interface model for incompressible two-phase flows with different densities. At the heart of this method lies the solution of large and sparse linear systems that arise in a semismooth Newton method. We propose the use of preconditioned Krylov subspace solvers using effective Schur complement approximations. Numerical results illustrate the efficiency of our approach. In particular, our preconditioner is shown to be robust with respect to parameter changes.

The meshless/meshfree radial basis function (RBF) method is a powerful technique for interpolating scattered data. But, solving large RBF interpolation problems without fast summation methods is computationally expensive. For RBF interpolation with $N$ points, using a direct method requires $\mathcal{O}(N^2)$ operations. As a fast summation method, the fast multipole method (FMM) has been implemented in speeding up the matrix-vector multiply, which reduces the complexity from $\mathcal{O}(N^2)$ to $\mathcal{O}(N^{1.5})$ and even to $\mathcal{O}(NlogN)$ for the multilevel fast multipole method (MLFMM). In this paper, we present a novel kernel-independent fast multipole method for RBF interpolation, which is used in combination with the evaluation of point-to-point interactions by RBF and the fast matrix-vector multiplication. This approach is based on band-limited approximation and quadrature rules, which extends the range of applicability of FMM.