Search

This paper proposes a novel method, Explicit Flow Matching (ExFM), for training and analyzing flow-based generative models. ExFM leverages a theoretically grounded loss function, ExFM loss (a tractable form of Flow Matching (FM) loss), to demonstrably reduce variance during training, leading to faster convergence and more stable learning. Based on theoretical analysis of these formulas, we derived exact expressions for the vector field (and score in stochastic cases) for model examples (in particular, for separating multiple exponents), and in some simple cases, exact solutions for trajectories. In addition, we also investigated simple cases of diffusion generative models by adding a stochastic term and obtained an explicit form of the expression for score. While the paper emphasizes the theoretical underpinnings of ExFM, it also showcases its effectiveness through numerical experiments on various datasets, including high-dimensional ones. Compared to traditional FM methods, ExFM achieves superior performance in terms of both learning speed and final outcomes.

We develop a new method HTBB for the multidimensional black-box approximation and gradient-free optimization, which is based on the low-rank hierarchical Tucker decomposition with the use of the MaxVol indices selection procedure. Numerical experiments for 14 complex model problems demonstrate the robustness of the proposed method for dimensions up to 1000, while it shows significantly more accurate results than classical gradient-free optimization methods, as well as approximation and optimization methods based on the popular tensor train decomposition, which represents a simpler case of a tensor network.

We developed a new method PROTES for black-box optimization, which is based on the probabilistic sampling from a probability density function given in the low-parametric tensor train format. We tested it on complex multidimensional arrays and discretized multivariable functions taken, among others, from real-world applications, including unconstrained binary optimization and optimal control problems, for which the possible number of elements is up to $2^{100}$. In numerical experiments, both on analytic model functions and on complex problems, PROTES outperforms existing popular discrete optimization methods (Particle Swarm Optimization, Covariance Matrix Adaptation, Differential Evolution, and others).

Tensor train (TT) format is a common approach for computationally efficient work with multidimensional arrays, vectors, matrices, and discretized functions in a wide range of applications, including computational mathematics and machine learning. In this work, we propose a new algorithm for TT-tensor optimization, which leads to very accurate approximations for the minimum and maximum tensor element. The method consists in sequential tensor multiplications of the TT-cores with an intelligent selection of candidates for the optimum. We propose the probabilistic interpretation of the method, and make estimates on its complexity and convergence. We perform extensive numerical experiments with random tensors and various multivariable benchmark functions with the number of input dimensions up to $100$. Our approach generates a solution close to the exact optimum for all model problems, while the running time is no more than $50$ seconds on a regular laptop., Comment: 16 page, 3 figures, 3 tables

Surrogate models can reduce computational costs for multivariable functions with an unknown internal structure (black boxes). In a discrete formulation, surrogate modeling is equivalent to restoring a multidimensional array (tensor) from a small part of its elements. The alternating least squares (ALS) algorithm in the tensor train (TT) format is a widely used approach to effectively solve this problem in the case of non-adaptive tensor recovery from a given training set (i.e., tensor completion problem). TT-ALS allows obtaining a low-parametric representation of the tensor, which is free from the curse of dimensionality and can be used for fast computation of the values at arbitrary tensor indices or efficient implementation of algebra operations with the black box (integration, etc.). However, to obtain high accuracy in the presence of restrictions on the size of the train data, a good choice of initial approximation is essential. In this work, we construct the ANOVA representation in the TT-format and use it as an initial approximation for the TT-ALS algorithm. The performed numerical computations for a number of multidimensional model problems, including the parametric partial differential equation, demonstrate a significant advantage of our approach for the commonly used random initial approximation. For all considered model problems we obtained an increase in accuracy by at least an order of magnitude with the same number of requests to the black box. The proposed approach is very general and can be applied in a wide class of real-world surrogate modeling and machine learning problems., Comment: 17 pages, 3 figures, 4 tables

This paper presents a method to build explicit tensor-train (TT) representations. We show that a wide class of tensors can be explicitly represented with sparse TT-cores, obtaining, in many cases, optimal TT-ranks. Numerical experiments show that our method outperforms the existing ones in several practical applications, including game theory problems. Theoretical estimations of the number of operations show that in some problems, such as permanent calculation, our methods are close to the known optimal asymptotics, which are obtained by a completely different type of methods.

Diffusion models have recently outperformed alternative approaches to model the distribution of natural images, such as GANs. Such diffusion models allow for deterministic sampling via the probability flow ODE, giving rise to a latent space and an encoder map. While having important practical applications, such as estimation of the likelihood, the theoretical properties of this map are not yet fully understood. In the present work, we partially address this question for the popular case of the VP SDE (DDPM) approach. We show that, perhaps surprisingly, the DDPM encoder map coincides with the optimal transport map for common distributions; we support this claim theoretically and by extensive numerical experiments.

Spatial soil sampling is an integral part of a soil survey aimed at creating a soil map. We propose considering the soil sampling procedure as a task of optimal design. In practical terms, optimal experiments can reduce experimentation costs, as they allow the researcher to obtain one optimal set of points. We present a sampling design, based on the fundamental idea of selecting sample locations by performing an optimal design method called the maxvol algorithm. It is shown that the maxvol-base algorithm has a high potential for practical usage. Our method outperforms popular sampling methods in soil taxa prediction based on topographical features of the site and deals with massive agricultural datasets in a reasonable time.

Model-based experimental design is attracting increasing attention in chemical process engineering. Typically, an iterative procedure is pursued: an approximate model is devised, prescribed experiments are then performed and the resulting data is exploited to refine the model. To help to reduce the cost of trial-and-error approaches, strategies for model-based design of experiments suggest experimental points where the expected gain in information for the model is the largest. It requires the resolution of a large nonlinear, generally nonconvex, optimization problem, whose solution may greatly depend on the starting point. We present two discretization strategies that can assist the experimenter in setting the number of relevant experiments and performing an optimal selection, and we compare them against two pattern-based strategies that are independent of the problem. The validity of the approaches is demonstrated on an academic example and two test problems from chemical engineering including a vapor liquid equilibrium and reaction kinetics.

In the paper we consider the problem of multivariate function approximation in polynomial basis. In order to solve this problem, we adjust the least squares method (LSM) by adding information about derivatives of the function. This modification allows reducing the number of evaluations of approximating function while keeping the accuracy at the appropriate level. We propose several techniques for time-efficient calculation of derivatives in various applications. Numerical examples are given for comparison between the standard LSM and the proposed approach., Comment: 17 pages, 7 figures