Search

Independent component analysis (ICA), as a statistical and computational approach, has been successfully applied to digital signal processing. Performance analysis for the ICA approach is perceived as a challenging task to work on. This contribution concerns the complex-valued FastICA algorithm in the range of ICA over the complex number domain. The focus is on the robust and equivariant behavior analysis of the complex-valued FastICA estimator. Although the complex-valued FastICA algorithm as well as its derivatives have been widely used methods for approaching the complex blind signal separation problem, rigorous mathematical treatments of the robust measurement and equivariance for the complex-valued FastICA estimator are still missing. This paper strictly analyzes the robustness against outliers and separation performance depending on the global system. We begin with defining the influence function (IF) of complex-valued FastICA functional and followed by deriving its closed-form expression. Then, we prove that the complex-valued FastICA algorithm based on the optimizing cost function is linear-equivariant, depending only on the source signals. [ABSTRACT FROM AUTHOR]

The existing methods for addressing visual navigation employ deep reinforcement learning as the standard tool for the task. However, they tend to be vulnerable to statistical shifts between the training and test data, resulting in poor generalization over novel environments that are out-of-distribution from the training data. In this study, we attempt to improve the generalization ability by utilizing the inductive biases available for the task. Employing the active neural SLAM that learns policies with the advantage actor-critic method as the base framework, we first point out that the mappings represented by the actor and the critic should satisfy specific symmetries. We then propose a network design for the actor and the critic to inherently attain these symmetries. Specifically, we use G-convolution instead of the standard convolution and insert the semi-global polar pooling layer, which we newly design in this study, in the last section of the critic network. Our method can be integrated into existing methods that utilize intermediate goals and 2D occupancy maps. Experimental results show that our method improves generalization ability by a good margin over visual exploration and object goal navigation, which are two main embodied visual navigation tasks. [ABSTRACT FROM AUTHOR]

Deep neural networks achieve state-of-the-art results in many different problem settings by exploiting vast amounts of training data. However, collecting, storing and—in the case of supervised learning—labelling the data is expensive and time-consuming. Additionally, assessing the networks’ generalization abilities or predicting how the inferred output changes under input transformations is complicated since the networks are usually treated as a black box. Both of these problems can be mitigated by incorporating prior knowledge into the neural network. One promising approach, inspired by the success of convolutional neural networks in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations of the problem to solve that affect the output in a predictable way. This promises an increased data efficiency and more interpretable network outputs. In this survey, we try to give a concise overview about different approaches that incorporate geometrical prior knowledge into neural networks. Additionally, we connect those methods to 3D object detection for autonomous driving, where we expect promising results when applying those methods. [ABSTRACT FROM AUTHOR]

We extend the equivariance and invariance conditions for construction of optimal designs to multiple-group mixed models and, hence, derive the support of optimal designs for first- and second-order models on a symmetric square. Moreover, we provide a tool for computation of D- and L-efficient exact designs in multiple-group mixed models by adapting the algorithm of Harman et al. (Appl Stoch Models Bus Ind, 32:3–17, 2016). We show that this algorithm can be used both for size-constrained problems and also in settings that require multiple resource constraints on the design, such as cost constraints or marginal constraints. [ABSTRACT FROM AUTHOR]

This paper presents a comparative analysis of the performance of Equivariant Quantum Neural Networks (EQNNs) and Quantum Neural Networks (QNNs), juxtaposed against their classical counterparts: Equivariant Neural Networks (ENNs) and Deep Neural Networks (DNNs). We evaluate the performance of each network with three two-dimensional toy examples for a binary classification task, focusing on model complexity (measured by the number of parameters) and the size of the training dataset. Our results show that the Z 2 × Z 2 EQNN and the QNN provide superior performance for smaller parameter sets and modest training data samples. [ABSTRACT FROM AUTHOR]

Machine learning algorithms are heavily relied on to understand the vast amounts of data from high-energy particle collisions at the CERN Large Hadron Collider (LHC). The data from such collision events can naturally be represented with graph structures. Therefore, deep geometric methods, such as graph neural networks (GNNs), have been leveraged for various data analysis tasks in high-energy physics. One typical task is jet tagging, where jets are viewed as point clouds with distinct features and edge connections between their constituent particles. The increasing size and complexity of the LHC particle datasets, as well as the computational models used for their analysis, have greatly motivated the development of alternative fast and efficient computational paradigms such as quantum computation. In addition, to enhance the validity and robustness of deep networks, we can leverage the fundamental symmetries present in the data through the use of invariant inputs and equivariant layers. In this paper, we provide a fair and comprehensive comparison of classical graph neural networks (GNNs) and equivariant graph neural networks (EGNNs) and their quantum counterparts: quantum graph neural networks (QGNNs) and equivariant quantum graph neural networks (EQGNN). The four architectures were benchmarked on a binary classification task to classify the parton-level particle initiating the jet. Based on their area under the curve (AUC) scores, the quantum networks were found to outperform the classical networks. However, seeing the computational advantage of quantum networks in practice may have to wait for the further development of quantum technology and its associated application programming interfaces (APIs). [ABSTRACT FROM AUTHOR]

Analyzing 3D objects without pose priors using neural networks is challenging. In view of the shortcoming that spherical convolutional networks lack the construction of a part–whole hierarchy with rotation equivariance for 3D object recognition with unknown poses, which generates whole rotation-equivariant features that cannot be effectively preserved, a rotation-equivariant part–whole hierarchy spherical vector network is proposed in this paper. In our experiments, we map a 3D object onto the unit sphere, construct an ordered list of vectors from the convolutional layers of the rotation-equivariant spherical convolutional network and then construct a part–whole hierarchy to classify the 3D object using the proposed rotation-equivariant routing algorithm. The experimental results show that the proposed method improves not only the recognition of 3D objects with known poses, but also the recognition of 3D objects with unknown poses compared to previous spherical convolutional neural networks. This finding validates the construction of the rotation-equivariant part–whole hierarchy. [ABSTRACT FROM AUTHOR]

Measuring the effect of peers on individuals' outcomes is a challenging problem, in part because individuals often select peers who are similar in both observable and unobservable ways. Group formation experiments avoid this problem by randomly assigning individuals to groups and observing their responses; for example, do first‐year students have better grades when they are randomly assigned roommates who have stronger academic backgrounds? In this paper, we propose randomization‐based permutation tests for group formation experiments, extending classical Fisher Randomization Tests to this setting. The proposed tests are justified by the randomization itself, require relatively few assumptions, and are exact in finite samples. This approach can also complement existing strategies, such as linear‐in‐means models, by using a regression coefficient as the test statistic. We apply the proposed tests to two recent group formation experiments. [ABSTRACT FROM AUTHOR]

Given a prime number p let Cp be the topological completion of the algebraic closure of the field of p-adic numbers. Let O(T) be the Galois orbit of a transcendental element T of Cp with respect to the absolute Galois group. Our aim is to study the class of Galois equivariant functions defined on O(T) with values in Cp. We show that each function from this class is continuous and we characterize the class of Lipschitz functions, respectively the class of differentiable functions, with respect to a new orthonormal basis. Then we discuss some aspects related to analytic continuation for the functions of this class. [ABSTRACT FROM AUTHOR]

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Independent component analysis (ICA), as a statistical and computational approach, has been successfully applied to digital signal processing. Performance analysis for the ICA approach is perceived as a challenging task to work on. This contribution concerns the complex-valued FastICA algorithm in the range of ICA over the complex number domain. The focus is on the robust and equivariant behavior analysis of the complex-valued FastICA estimator. Although the complex-valued FastICA algorithm as well as its derivatives have been widely used methods for approaching the complex blind signal separation problem, rigorous mathematical treatments of the robust measurement and equivariance for the complex-valued FastICA estimator are still missing. This paper strictly analyzes the robustness against outliers and separation performance depending on the global system. We begin with defining the influence function (IF) of complex-valued FastICA functional and followed by deriving its closed-form expression. Then, we prove that the complex-valued FastICA algorithm based on the optimizing cost function is linear-equivariant, depending only on the source signals.

Capsule networks exhibit the potential to enhance computer vision tasks through their utilization of equivariance for capturing spatial relationships. However, the broader adoption of these networks has been impeded by the computational complexity of their routing mechanism and shallow backbone model. To address these challenges, this paper introduces an innovative hybrid architecture that seamlessly integrates a pretrained backbone model with a task-specific capsule head (CapsHead). Our methodology is extensively evaluated across a range of classification and segmentation tasks, encompassing diverse datasets. The empirical findings robustly underscore the efficacy and practical feasibility of our proposed approach in real-world vision applications. Notably, our approach yields substantial 3.45 % and 6.24 % enhancement in linear evaluation on the CIFAR10 dataset and segmentation on the VOC2012 dataset, respectively, compared to baselines that do not incorporate the capsule head. This research offers a noteworthy contribution by not only advancing the application of capsule networks, but also mitigating their computational complexities. The results substantiate the feasibility of our hybrid architecture, thereby paving the way for a wider integration of capsule networks into various computer vision tasks. [ABSTRACT FROM AUTHOR]

Sensitivity testing often involves sequential design strategies in small-sample settings that provide binary data which are then used to develop generalized linear models. Model parameters are usually estimated via maximum likelihood methods. Often, confidence bounds relating to model parameters and quantiles are based on the likelihood ratio. In this paper, it is demonstrated how the bias-corrected parametric bootstrap used in conjunction with approximate pivotal quantities can be used to provide an alternative means for constructing bounds when using a location-scale model. In small-sample settings, the coverage of bounds based on the likelihood ratio is often anticonservative due to bias in estimating the scale parameter. In contrast, bounds produced by the bias-corrected parametric bootstrap can provide accurate levels of coverage in such settings when both the sequential strategy and method for parameter estimation effectively adapt (are approximately equivariant) to the location and scale. A series of simulations illustrate this contrasting behavior in a small-sample setting when assuming a normal/probit model in conjunction with a popular sequential design strategy. In addition, it is shown how a high-fidelity assessment of performance can be attained with reduced computational effort by using the nonparametric bootstrap to resample pivotal quantities obtained from a small-scale set of parametric bootstrap simulations. [ABSTRACT FROM AUTHOR]

Symmetries in the data and how they constrain the learned weights of modern deep networks is still an open problem. In this work we study the simple case of fully connected shallow non-linear neural networks and consider two types of symmetries: full dataset symmetries where the dataset $X$ is mapped into itself by any transformation $g$ , i.e. $gX=X$ or single data point symmetries where $gx=x$ , $x\in X$ . We prove and experimentally confirm that symmetries in the data are directly inherited at the level of the network’s learned weights and relate these findings with the common practice of data augmentation in modern machine learning. Finally, we show how symmetry constraints have a profound impact on the spectrum of the learned weights, an aspect of the so-called network implicit bias.

Transformation-robustness is an important feature for machine learning models that perform image classification. Many methods aim to bestow this property to models by the use of data augmentation strategies, while more formal guarantees are obtained via the use of equivariant models. We recognise that compositional, or part-whole structure is also an important aspect of images that has to be considered for building transformation-robust models. Thus, we propose a capsule network model that is, at once, equivariant and compositionality aware. Equivariance of our capsule network model comes from the use of equivariant convolutions in a carefully-chosen novel architecture. The awareness of compositionality comes from the use of our proposed novel, iterative, graph-based routing algorithm, termed Iterative collaborative routing (ICR). ICR, the core of our contribution, weights the predictions made for capsules based on an iteratively averaged score of the degree-centralities of its nearest neighbours. Experiments on transformed image classification on FashionMNIST, CIFAR-10, and CIFAR-100 show that our model that uses ICR outperforms convolutional and capsule baselines to achieve state-of-the-art performance.

We give a complete description of the correspondence between the reversibility and equivariance properties of evolution families (so for dynamical systems with continuous time, possibly nonautonomous) and their evolution groups. To the best of our knowledge, no similar result appeared before in the literature for dynamical systems with continuous time, even autonomous. Moreover, based on these results, we describe the faithful correspondence between the reversibility properties of stable and unstable invariant manifolds of evolution families and of their associated evolution groups. Finally, we construct center invariant manifolds for an equation that has a line of equilibria generated by a group of symmetries for which the equation is equivariant. [ABSTRACT FROM AUTHOR]

This paper presents a comparative analysis of the performance of Equivariant Quantum Neural Networks (EQNNs) and Quantum Neural Networks (QNNs), juxtaposed against their classical counterparts: Equivariant Neural Networks (ENNs) and Deep Neural Networks (DNNs). We evaluate the performance of each network with three two-dimensional toy examples for a binary classification task, focusing on model complexity (measured by the number of parameters) and the size of the training dataset. Our results show that the Z2×Z2 EQNN and the QNN provide superior performance for smaller parameter sets and modest training data samples.

Machine learning algorithms are heavily relied on to understand the vast amounts of data from high-energy particle collisions at the CERN Large Hadron Collider (LHC). The data from such collision events can naturally be represented with graph structures. Therefore, deep geometric methods, such as graph neural networks (GNNs), have been leveraged for various data analysis tasks in high-energy physics. One typical task is jet tagging, where jets are viewed as point clouds with distinct features and edge connections between their constituent particles. The increasing size and complexity of the LHC particle datasets, as well as the computational models used for their analysis, have greatly motivated the development of alternative fast and efficient computational paradigms such as quantum computation. In addition, to enhance the validity and robustness of deep networks, we can leverage the fundamental symmetries present in the data through the use of invariant inputs and equivariant layers. In this paper, we provide a fair and comprehensive comparison of classical graph neural networks (GNNs) and equivariant graph neural networks (EGNNs) and their quantum counterparts: quantum graph neural networks (QGNNs) and equivariant quantum graph neural networks (EQGNN). The four architectures were benchmarked on a binary classification task to classify the parton-level particle initiating the jet. Based on their area under the curve (AUC) scores, the quantum networks were found to outperform the classical networks. However, seeing the computational advantage of quantum networks in practice may have to wait for the further development of quantum technology and its associated application programming interfaces (APIs).

Machine-learning interatomic potentials (MLIPs) have made a significant contribution to the recent progress in the fields of computational materials and chemistry due to the MLIPs’ ability of accurately approximating energy landscapes of quantum-mechanical models while being orders of magnitude more computationally efficient. However, the computational cost and number of parameters of many state-of-the-art MLIPs increases exponentially with the number of atomic features. Tensor (non-neural) networks, based on low-rank representations of high-dimensional tensors, have been a way to reduce the number of parameters in approximating multidimensional functions, however, it is often not easy to encode the model symmetries into them. In this work we develop a formalism for rank-efficient equivariant tensor networks (ETNs), i.e. tensor networks that remain invariant under actions of SO(3) upon contraction. All the key algorithms of tensor networks like orthogonalization of cores and DMRG-based algorithms carry over to our equivariant case. Moreover, we show that many elements of modern neural network architectures like message passing, pulling, or attention mechanisms, can in some form be implemented into the ETNs. Based on ETNs, we develop a new class of polynomial-based MLIPs that demonstrate superior performance over existing MLIPs for multicomponent systems.

The importance of symmetries has recently been recognized in quantum machine learning from the simple motto: if a task exhibits a symmetry (given by a group $\mathfrak{G}$ ), the learning model should respect said symmetry. This can be instantiated via $\mathfrak{G}$ -equivariant quantum neural networks (QNNs), i.e. parametrized quantum circuits whose gates are generated by operators commuting with a given representation of $\mathfrak{G}$ . In practice, however, there might be additional restrictions to the types of gates one can use, such as being able to act on at most k qubits. In this work we study how the interplay between symmetry and k -bodyness in the QNN generators affect its expressiveness for the special case of $\mathfrak{G} = S_n$ , the symmetric group. Our results show that if the QNN is generated by one- and two-body S _n -equivariant gates, the QNN is semi-universal but not universal. That is, the QNN can generate any arbitrary special unitary matrix in the invariant subspaces, but has no control over the relative phases between them. Then, we show that in order to reach universality one needs to include n -body generators (if n is even) or $(n-1)$ -body generators (if n is odd). As such, our results brings us a step closer to better understanding the capabilities and limitations of equivariant QNNs.

Geometric transformations of the training data as well as the test data present challenges to the use of deep neural networks to vision-based learning tasks. To address this issue, we present a deep neural network model that exhibits the desirable property of transformationrobustness. Our model, termed RobustCaps, uses group-equivariant convolutions in an improved capsule network model. RobustCaps uses a global context-normalised procedure in its routing algorithm to learn transformation-invariant part-whole relationships within image data. This learning of such relationships allows our model to outperform both capsule and convolutional neural network baselines on transformation-robust classification tasks. Specifically, RobustCaps achieves state-of-the-art accuracies on CIFAR-10, FashionMNIST, and CIFAR-100 when the images in these datasets are subjected to train and test-time rotations and translations. [ABSTRACT FROM AUTHOR]

Solving an ill-posed linear inverse problem requires knowledge about the underlying signal model. In many applications, this model is a priori unknown and has to be learned from data. However, it is impossible to learn the model using observations obtained via a single incomplete measurement operator, as there is no information about the signal model in the nullspace of the operator, resulting in a chicken-and-egg problem: to learn the model we need reconstructed signals, but to reconstruct the signals we need to know the model. Two ways to overcome this limitation are using multiple measurement operators or assuming that the signal model is invariant to a certain group action. In this paper, we present necessary and sufficient sensing conditions for learning the signal model from measurement data alone which only depend on the dimension of the model and the number of operators or properties of the group action that the model is invariant to. As our results are agnostic of the learning algorithm, they shed light into the fundamental limitations of learning from incomplete data and have implications in a wide range set of practical algorithms, such as dictionary learning, matrix completion and deep neural networks. [ABSTRACT FROM AUTHOR]

In recent years, algorithms and neural architectures based on the Weisfeiler-Leman algorithm, a well-known heuristic for the graph isomorphism problem, have emerged as a powerful tool for machine learning with graphs and relational data. Here, we give a comprehensive overview of the algorithm's use in a machine-learning setting, focusing on the supervised regime. We discuss the theoretical background, show how to use it for supervised graph and node representation learning, discuss recent extensions, and outline the algorithm's connection to (permutation-)equivariant neural architectures. Moreover, we give an overview of current applications and future directions to stimulate further research. [ABSTRACT FROM AUTHOR]

Composite materials with different microstructural material symmetries are common in engineering applications where grain structure, alloying and particle/fiber packing are optimized via controlled manufacturing. In fact these microstructural tunings can be done throughout a part to achieve functional gradation and optimization at a structural level. To predict the performance of particular microstructural configuration and thereby overall performance, constitutive models of materials with microstructure are needed. In this work we provide neural network architectures that provide effective homogenization models of materials with anisotropic components. These models satisfy equivariance and material symmetry principles inherently through a combination of equivariant and tensor basis operations. We demonstrate them on datasets of stochastic volume elements with different textures and phases where the material undergoes elastic and plastic deformation, and show that the these network architectures provide significant performance improvements. [Display omitted] [ABSTRACT FROM AUTHOR]

Underwater imaging faces challenges for enhancing object visibility and restoring true colors due to the absorptive and scattering characteristics of water. Underwater image restoration (UIR) seeks solutions to restore clean images from degraded ones, providing significant utility in downstream tasks. Recently, data-driven UIR has garnered much attention due to the potent expressive capabilities of deep neural networks (DNNs). These DNNs are supervised, relying on a large amount of labeled training samples. However, acquiring such data is expensive or even impossible in real-world underwater scenarios. While recent researches suggest that unsupervised learning is effective in UIR, none of these frameworks consider signal physical priors. In this work, we present a novel physics-inspired unsupervised UIR framework empowered by equivariance and unbiased estimation techniques. Specifically, equivariance stems from the invariance, inherent in natural signals to enhance data-efficient learning. Given that degraded images invariably contain noise, we propose a noise-tolerant loss for unsupervised UIR based on the Stein unbiased risk estimator to achieve an accurate estimation of the data consistency. Extensive experiments on the benchmark UIR datasets, including the UIEB and RUIE datasets, validate the superiority of the proposed method in terms of quantitative scores, visual outcomes, and generalization ability, compared to state-of-the-art counterparts. Moreover, our method demonstrates even comparable performance with the supervised model. • Introduction of UIR-ES, a fully unsupervised learning framework for UIR. • Proposing scene radiance equivariant learning and observation equivariant learning strategies. • Using SURE to estimate data consistency for improved performance. • Superior UIR performance over previous unsupervised learning methods on two UIR datasets. [ABSTRACT FROM AUTHOR]

Robustness of deep learning segmentation models is crucial for their safe incorporation into clinical practice. However, these models can falter when faced with distributional changes. This challenge is evident in magnetic resonance imaging (MRI) scans due to the diverse acquisition protocols across various domains, leading to differences in image characteristics such as textural appearances. We posit that the restricted anatomical differences between subjects could be harnessed to refine the latent space into a set of shape components. The learned set then aims to encompass the relevant anatomical shape variation found within the patient population. We explore this by utilising multiple MRI sequences to learn texture invariant and shape equivariant features which are used to construct a shape dictionary using vector quantisation. We investigate shape equivariance to a number of different types of groups. We hypothesise and prove that the greater the group order, i.e., the denser the constraint, the better becomes the model robustness. We achieve shape equivariance either with a contrastive based approach or by imposing equivariant constraints on the convolutional kernels. The resulting shape equivariant dictionary is then sampled to compose the segmentation output. Our method achieves state-of-the-art performance for the task of single domain generalisation for prostate and cardiac MRI segmentation. Code is available at https://github.com/AinkaranSanthi/A_Geometric_Perspective_For_Robust_Segmentation. • Geometric constraints in the latent space of a deep learning for Robust Segmentation. • We hypothesis and prove group equivariant constraints in the latent space improves robustness. • A discrete equivariant shape latent space is sampled to construct the segmentation map. • Method demonstrated in the task of single domain generalisation. [ABSTRACT FROM AUTHOR]

It is time-consuming and expensive to take high-quality or high-resolution electron microscopy (EM) and fluorescence microscopy (FM) images. Taking these images could be even invasive to samples and may damage certain subtleties in the samples after long or intense exposures, often necessary for achieving high-quality or high-resolution in the first place. Advances in deep learning enable us to perform various types of microscopy image-to-image transformation tasks such as image denoising, super-resolution, and segmentation that computationally produce high-quality images from the physically acquired low-quality ones. When training image-to-image transformation models on pairs of experimentally acquired microscopy images, prior models suffer from performance loss due to their inability to capture inter-image dependencies and common features shared among images. Existing methods that take advantage of shared features in image classification tasks cannot be properly applied to image transformation tasks because they fail to preserve the equivariance property under spatial permutations, something essential in image-to-image transformation. To address these limitations, we propose the augmented equivariant attention networks (AEANets) with better capability to capture inter-image dependencies, while preserving the equivariance property. The proposed AEANets captures inter-image dependencies and shared features via two augmentations on the attention mechanism, which are the shared references and the batch-aware attention during training. We theoretically derive the equivariance property of the proposed augmented attention model and experimentally demonstrate its consistent superiority in both quantitative and visual results over the baseline methods. [ABSTRACT FROM AUTHOR]

Transformation-robustness is an important feature for machine learning models that perform image classification. Many methods aim to bestow this property to models by the use of data augmentation strategies, while more formal guarantees are obtained via the use of equivariant models. We recognise that compositional, or part-whole structure is also an important aspect of images that has to be considered for building transformation-robust models. Thus, we propose a capsule network model that is, at once, equivariant and compositionality aware. Equivariance of our capsule network model comes from the use of equivariant convolutions in a carefully-chosen novel architecture. The awareness of compositionality comes from the use of our proposed novel, iterative, graph-based routing algorithm, termed Iterative collaborative routing (ICR). ICR, the core of our contribution, weights the predictions made for capsules based on an iteratively averaged score of the degree-centralities of its nearest neighbours. Experiments on transformed image classification on FashionMNIST, CIFAR-10, and CIFAR-100 show that our model that uses ICR outperforms convolutional and capsule baselines to achieve state-of-the-art performance. [ABSTRACT FROM AUTHOR]