Search

Traditional mathematical models used in designing next-generation communication systems often fall short due to inherent simplifications, narrow scope, and computational limitations. In recent years, the incorporation of deep learning (DL) methodologies into communication systems has made significant progress in system design and performance optimisation. Autoencoders (AEs) have become essential, enabling end-to-end learning that allows for the combined optimisation of transmitters and receivers. Consequently, AEs offer a data-driven methodology capable of bridging the gap between theoretical models and real-world complexities. The paper presents a comprehensive survey of the application of AEs within communication systems, with a particular focus on their architectures, associated challenges, and future directions. We examine 120 recent studies across wireless, optical, semantic, and quantum communication fields, categorising them according to transceiver design, channel modelling, digital signal processing, and computational complexity. This paper further examines the challenges encountered in the implementation of AEs, including the need for extensive training data, the risk of overfitting, and the requirement for differentiable channel models. Through data-driven approaches, AEs provide robust solutions for end-to-end system optimisation, surpassing traditional mathematical models confined by simplifying assumptions. This paper also summarises the computational complexity associated with AE-based systems by conducting an in-depth analysis employing the metric of floating-point operations per second (FLOPS). This analysis encompasses the evaluation of matrix multiplications, bias additions, and activation functions. This survey aims to establish a roadmap for future research, emphasising the transformative potential of AEs in the formulation of next-generation communication systems.

Synthetic data has gained attention for training large language models, but poor-quality data can harm performance (see, e.g., Shumailov et al. (2023); Seddik et al. (2024)). A potential solution is data pruning, which retains only high-quality data based on a score function (human or machine feedback). Previous work Feng et al. (2024) analyzed models trained on synthetic data as sample size increases. We extend this by using random matrix theory to derive the performance of a binary classifier trained on a mix of real and pruned synthetic data in a high dimensional setting. Our findings identify conditions where synthetic data could improve performance, focusing on the quality of the generative model and verification strategy. We also show a smooth phase transition in synthetic label noise, contrasting with prior sharp behavior in infinite sample limits. Experiments with toy models and large language models validate our theoretical results.

We demonstrate that preference optimization methods can effectively enhance LLM safety. Applying various alignment techniques to the Falcon 11B model using safety datasets, we achieve a significant boost in global safety score (from $57.64\%$ to $99.90\%$) as measured by LlamaGuard 3 8B, competing with state-of-the-art models. On toxicity benchmarks, average scores in adversarial settings dropped from over $0.6$ to less than $0.07$. However, this safety improvement comes at the cost of reduced general capabilities, particularly in math, suggesting a trade-off. We identify noise contrastive alignment (Safe-NCA) as an optimal method for balancing safety and performance. Our study ultimately shows that alignment techniques can be sufficient for building safe and robust models.

Anomaly detection in videos is challenging due to the complexity, noise, and diverse nature of activities such as violence, shoplifting, and vandalism. While deep learning (DL) has shown excellent performance in this area, existing approaches have struggled to apply DL models across different anomaly tasks without extensive retraining. This repeated retraining is time-consuming, computationally intensive, and unfair. To address this limitation, a new DL framework is introduced in this study, consisting of three key components: transfer learning to enhance feature generalization, model fusion to improve feature representation, and multi-task classification to generalize the classifier across multiple tasks without training from scratch when new task is introduced. The framework's main advantage is its ability to generalize without requiring retraining from scratch for each new task. Empirical evaluations demonstrate the framework's effectiveness, achieving an accuracy of 97.99% on the RLVS dataset (violence detection), 83.59% on the UCF dataset (shoplifting detection), and 88.37% across both datasets using a single classifier without retraining. Additionally, when tested on an unseen dataset, the framework achieved an accuracy of 87.25%. The study also utilizes two explainability tools to identify potential biases, ensuring robustness and fairness. This research represents the first successful resolution of the generalization issue in anomaly detection, marking a significant advancement in the field.

Adversarial attacks are a potential threat to machine learning models by causing incorrect predictions through imperceptible perturbations to the input data. While these attacks have been extensively studied in unstructured data like images, applying them to tabular data, poses new challenges. These challenges arise from the inherent heterogeneity and complex feature interdependencies in tabular data, which differ from the image data. To account for this distinction, it is necessary to establish tailored imperceptibility criteria specific to tabular data. However, there is currently a lack of standardised metrics for assessing the imperceptibility of adversarial attacks on tabular data. To address this gap, we propose a set of key properties and corresponding metrics designed to comprehensively characterise imperceptible adversarial attacks on tabular data. These are: proximity to the original input, sparsity of altered features, deviation from the original data distribution, sensitivity in perturbing features with narrow distribution, immutability of certain features that should remain unchanged, feasibility of specific feature values that should not go beyond valid practical ranges, and feature interdependencies capturing complex relationships between data attributes. We evaluate the imperceptibility of five adversarial attacks, including both bounded attacks and unbounded attacks, on tabular data using the proposed imperceptibility metrics. The results reveal a trade-off between the imperceptibility and effectiveness of these attacks. The study also identifies limitations in current attack algorithms, offering insights that can guide future research in the area. The findings gained from this empirical analysis provide valuable direction for enhancing the design of adversarial attack algorithms, thereby advancing adversarial machine learning on tabular data., Comment: 36 pages

The stochastic reaction-diffusion model driven by a multiplicative noise is examined. We construct the gradient discretisation method (GDM), an abstract framework combining several numerical method families. The paper provides the discretisation and proves the convergence of the approximate schemes using a compactness argument that works under natural assumptions on data. We also investigate, using a finite volume method, known as the hybrid mixed mimetic (HMM) approach, the effects of multiplicative noise on the dynamics of the travelling waves in the excitable media displayed by the model. Particularly, we consider how sufficiently high noise can cause waves to backfire or fail to propagate.

Deep learning has significantly advanced automatic medical diagnostics and released the occupation of human resources to reduce clinical pressure, yet the persistent challenge of data scarcity in this area hampers its further improvements and applications. To address this gap, we introduce a novel ensemble framework called `Efficient Transfer and Self-supervised Learning based Ensemble Framework' (ETSEF). ETSEF leverages features from multiple pre-trained deep learning models to efficiently learn powerful representations from a limited number of data samples. To the best of our knowledge, ETSEF is the first strategy that combines two pre-training methodologies (Transfer Learning and Self-supervised Learning) with ensemble learning approaches. Various data enhancement techniques, including data augmentation, feature fusion, feature selection, and decision fusion, have also been deployed to maximise the efficiency and robustness of the ETSEF model. Five independent medical imaging tasks, including endoscopy, breast cancer, monkeypox, brain tumour, and glaucoma detection, were tested to demonstrate ETSEF's effectiveness and robustness. Facing limited sample numbers and challenging medical tasks, ETSEF has proved its effectiveness by improving diagnostics accuracies from 10\% to 13.3\% when compared to strong ensemble baseline models and up to 14.4\% improvements compared with published state-of-the-art methods. Moreover, we emphasise the robustness and trustworthiness of the ETSEF method through various vision-explainable artificial intelligence techniques, including Grad-CAM, SHAP, and t-SNE. Compared to those large-scale deep learning models, ETSEF can be deployed flexibly and maintain superior performance for challenging medical imaging tasks, showing the potential to be applied to more areas that lack training data, Comment: 64 pages, 20 figures

Recently, transfer learning and self-supervised learning have gained significant attention within the medical field due to their ability to mitigate the challenges posed by limited data availability, improve model generalisation, and reduce computational expenses. Transfer learning and self-supervised learning hold immense potential for advancing medical research. However, it is crucial to recognise that transfer learning and self-supervised learning architectures exhibit distinct advantages and limitations, manifesting variations in accuracy, training speed, and robustness. This paper compares the performance and robustness of transfer learning and self-supervised learning in the medical field. Specifically, we pre-trained two models using the same source domain datasets with different pre-training methods and evaluated them on small-sized medical datasets to identify the factors influencing their final performance. We tested data with several common issues in medical domains, such as data imbalance, data scarcity, and domain mismatch, through comparison experiments to understand their impact on specific pre-trained models. Finally, we provide recommendations to help users apply transfer learning and self-supervised learning methods in medical areas, and build more convenient and efficient deployment strategies., Comment: 37 pages, 8 figures

We modify the axioms of triangulated categories to include both higher triangles and distinguished maps of higher triangles. The distinguished maps are specializations of Neeman's ``good'' maps of $2$-triangles. The axioms both simplify Neeman's axioms in his 1991 paper and generalize them to higher triangles in the way proposed by Balmer et al. We provide a geometric formulation via directed truncated simplices to enable a more concrete approach. The axioms are modelled by homotopy and derived categories. We look at a number of key theorems including the fact that sums of maps of $2$-triangles are distinguished if and only if the maps are distinguished and a strong version of the $3$x$3$ lemma. These illustrate some key proof methods. We also show that maps of faces of distinguished triangles are distinguished. We construct some useful distinguished $5$-triangles., Comment: 23 pages

Exact travelling wave solutions to the two-dimensional stochastic Allen-Cahn equation with multiplicative noise are obtained through the hyperbolic tangent (tanh) method. This technique limits the solutions to travelling wave profiles by representing them with a finite tanh power series. This study focuses on how multiplicative noise affects the dynamics of these travelling waves, in particular, occurring of wave propagation failure due to high levels of noise., Comment: Need to be revised in section of the derivation of exact travelling wave solutions

The recent advancement of Blockchain technology consolidates its status as a viable alternative for various domains. However, evaluating the performance of blockchain applications can be challenging due to the underlying infrastructure's complexity and distributed nature. Therefore, a reliable modelling approach is needed to boost Blockchain-based applications' development and evaluation. While simulation-based solutions have been researched, machine learning (ML) model-based techniques are rarely discussed in conjunction with evaluating blockchain application performance. Our novel research makes use of two ML model-based methods. Firstly, we train a $k$ nearest neighbour ($k$NN) and support vector machine (SVM) to predict blockchain performance using predetermined configuration parameters. Secondly, we employ the salp swarm optimization (SO) ML model which enables the investigation of optimal blockchain configurations for achieving the required performance level. We use rough set theory to enhance SO, hereafter called ISO, which we demonstrate to prove achieving an accurate recommendation of optimal parameter configurations; despite uncertainty. Finally, statistical comparisons indicate that our models have a competitive edge. The $k$NN model outperforms SVM by 5\% and the ISO also demonstrates a reduction of 4\% inaccuracy deviation compared to regular SO.

The impact of wettability on the co-moving velocity of two-fluid flow in porous media is analyzed herein. The co-moving velocity, developed by Roy et al. (2022), is a novel representation of the flow behavior of two fluids through porous media. Our study aims to better understand the behavior of the co-moving velocity by analyzing simulation data under various wetting conditions. The simulations were conducted using the Lattice-Boltzmann color-fluid model and evaluated the relative permeability for different wetting conditions on the same rock. The analysis of the simulation data followed the methodology proposed by Roy et al. (2022) to reconstruct a constitutive equation for the co-moving velocity. Surprisingly, it was found that the coefficients of the constitutive equation were nearly the same for all wetting conditions. Based on these results, a simple approach was proposed to reconstruct the oil phase relative permeability using only the co-moving velocity relationship and water phase relative permeability. This proposed method provides new insights into the dependency of relative permeability curves, which has implications for the history matching of production data and solving the associated inverse problem. The research findings contribute to a better understanding of the impact of wettability on fluid flow in porous media and provide a practical approach for estimating relative permeability based on the co-moving velocity relationship, which has never been shown before., Comment: 14 pages, 6 figures

Considerable recent work has focused on methods for analyzing experiments which exhibit treatment interference -- that is, when the treatment status of one unit may affect the response of another unit. Such settings are common in experiments on social networks. We consider a model of treatment interference -- the K-nearest neighbors interference model (KNNIM) -- for which the response of one unit depends not only on the treatment status given to that unit, but also the treatment status of its $K$ ``closest'' neighbors. We derive causal estimands under KNNIM in a way that allows us to identify how each of the $K$-nearest neighbors contributes to the indirect effect of treatment. We propose unbiased estimators for these estimands and derive conservative variance estimates for these unbiased estimators. We then consider extensions of these estimators under an assumption of no weak interaction between direct and indirect effects. We perform a simulation study to determine the efficacy of these estimators under different treatment interference scenarios. We apply our methodology to an experiment designed to assess the impact of a conflict-reducing program in middle schools in New Jersey, and we give evidence that the effect of treatment propagates primarily through a unit's closest connection.

Our recent intensive study has found that physics-informed neural networks (PINN) tend to be local approximators after training. This observation leads to this novel physics-informed radial basis network (PIRBN), which can maintain the local property throughout the entire training process. Compared to deep neural networks, a PIRBN comprises of only one hidden layer and a radial basis "activation" function. Under appropriate conditions, we demonstrated that the training of PIRBNs using gradient descendent methods can converge to Gaussian processes. Besides, we studied the training dynamics of PIRBN via the neural tangent kernel (NTK) theory. In addition, comprehensive investigations regarding the initialisation strategies of PIRBN were conducted. Based on numerical examples, PIRBN has been demonstrated to be more effective and efficient than PINN in solving PDEs with high-frequency features and ill-posed computational domains. Moreover, the existing PINN numerical techniques, such as adaptive learning, decomposition and different types of loss functions, are applicable to PIRBN. The programs that can regenerate all numerical results can be found at https://github.com/JinshuaiBai/PIRBN., Comment: 48 pages, 26 figures