Search

It is projected that 10 million deaths could be attributed to drug-resistant bacteria infections in 2050. To address this concern, identifying new-generation antibiotics is an effective way. Antimicrobial peptides (AMPs), a class of innate immune effectors, have received significant attention for their capacity to eliminate drug-resistant pathogens, including viruses, bacteria, and fungi. Recent years have witnessed widespread applications of computational methods especially machine learning (ML) and deep learning (DL) for discovering AMPs. However, existing methods only use features including compositional, physiochemical, and structural properties of peptides, which cannot fully capture sequence information from AMPs. Here, we present SAMP, an ensemble random projection (RP) based computational model that leverages a new type of feature called proportionalized split amino acid composition (PSAAC) in addition to conventional sequence-based features for AMP prediction. With this new feature set, SAMP captures the residue patterns like sorting signals at both the N-terminal and the C-terminal, while also retaining the sequence order information from the middle peptide fragments. Benchmarking tests on different balanced and imbalanced datasets demonstrate that SAMP consistently outperforms existing state-of-the-art methods, such as iAMPpred and AMPScanner V2, in terms of accuracy, Matthews correlation coefficient (MCC), G-measure, and F1-score. In addition, by leveraging an ensemble RP architecture, SAMP is scalable to processing large-scale AMP identification with further performance improvement, compared to those models without RP. To facilitate the use of SAMP, we have developed a Python package that is freely available at https://github.com/wan-mlab/SAMP. [ABSTRACT FROM AUTHOR]

The multiple correlation coefficient (MCC) quantifies the maximum correlation between a variable and a linear combination of a set of variables. Within the context of multiple regression and correlation analysis, testing for the null hypothesis of zero MCC has garnered significant attention. However, in high-dimensional data settings, where the data dimension (p) far exceeds the number of observations (n), this testing problem poses a challenging task that renders classical testing methodologies practically infeasible. This study proposes three testing procedures to assess the null hypothesis of zero MCC under the assumption of multivariate normality. Simulations are conducted to evaluate the performance of the proposed tests and compare them with an existing competitor. The simulation results are highly encouraging. Finally, the proposed tests are applied to two publicly available datasets to demonstrate their practical utility. [ABSTRACT FROM AUTHOR]

We propose a new method for dimension reduction of high-dimensional spherical data based on the nonlinear projection of sphere-valued data to a randomly chosen subsphere. The proposed method, spherical random projection, leads to a probabilistic lower-dimensional mapping of spherical data into a subsphere of the original. In this paper, we investigate some properties of spherical random projection, including expectation preservation and distance concentration, from which we derive an analogue of the Johnson–Lindenstrauss Lemma for spherical random projection. Clustering model selection is discussed as an application of spherical random projection, and numerical experiments are conducted using real and simulated data. [ABSTRACT FROM AUTHOR]

High dimensional learning is a perennial problem due to challenges posed by the "curse of dimensionality"; learning typically demands more computing resources as well as more training data. In differentially private (DP) settings, this is further exacerbated by noise that needs adding to each dimension to achieve the required privacy. In this paper, we present a surprisingly simple approach to address all of these concerns at once, based on histograms constructed on a low-dimensional random projection (RP) of the data. Our approach exploits RP to take advantage of hidden low-dimensional structures in the data, yielding both computational efficiency, and improved error convergence with respect to the sample size—whereby less training data suffice for learning. We also propose a variant for efficient differentially private (DP) classification that further exploits the data-oblivious nature of both the histogram construction and the RP based dimensionality reduction, resulting in an efficient management of the privacy budget. We present a detailed and rigorous theoretical analysis of generalisation of our algorithms in several settings, showing that our approach is able to exploit low-dimensional structure of the data, ameliorates the ill-effects of noise required for privacy, and has good generalisation under minimal conditions. We also corroborate our findings experimentally, and demonstrate that our algorithms achieve competitive classification accuracy in both non-private and private settings. [ABSTRACT FROM AUTHOR]

Cancelable template protection techniques are indispensable to provide essential security and privacy privileges in biometric systems. This paper introduces an efficient cancelable palmprint recognition technique based on multi-level transformations. Gabor filtering with feature remapping is introduced for extracting highly discriminative features. Histogram remapping is applied to nonlinearly transform the downsampled Gabor features to be normally distributed. This feature-remapping step is proposed to enhance the discriminatory power and alleviate the effect of feature variability and image artifacts. Comb filtering is applied to the mapped features as a first protection layer. To provide security guarantees against linkability attacks, index-based locality-sensitive hashing (LSH) is introduced as a second protection layer to transform the comb-filtered mapped real-valued features into maximum-ranked indices. Recognition is performed in the secured domain to accelerate matching and to preserve the user's privacy. Results show that the proposed scheme provides a large re-issuance ability, protects templates from being inverted, and demonstrates strong unlinkability across different databases. In addition, favorable verification/identification accuracy is obtained and the system satisfies the needs for real-time applications. A global measure D ↔ sys value of 0.01 is obtained, and thus, correlation attacks are mitigated. The recognition results for the legitimate scenario are 100% identification accuracy and 0% EER. For the worst case (same token scenario), the corresponding results are 99.752% and 0.31%, respectively. Highlights: • Multi-level transformations are proposed for efficient cancelable palmprints. • Histogram-remapped downsampled Gabor features are subjected to comb filtering. • Locality-sensitive hashing follows comb filtering to obtain maximum-ranked indices. • The algorithm provides significant capability for re-issuance. • The system protects templates from inversion and ensures unlinkability across databases. [ABSTRACT FROM AUTHOR]

Abstract Cancelable template protection techniques are indispensable to provide essential security and privacy privileges in biometric systems. This paper introduces an efficient cancelable palmprint recognition technique based on multi-level transformations. Gabor filtering with feature remapping is introduced for extracting highly discriminative features. Histogram remapping is applied to nonlinearly transform the downsampled Gabor features to be normally distributed. This feature-remapping step is proposed to enhance the discriminatory power and alleviate the effect of feature variability and image artifacts. Comb filtering is applied to the mapped features as a first protection layer. To provide security guarantees against linkability attacks, index-based locality-sensitive hashing (LSH) is introduced as a second protection layer to transform the comb-filtered mapped real-valued features into maximum-ranked indices. Recognition is performed in the secured domain to accelerate matching and to preserve the user’s privacy. Results show that the proposed scheme provides a large re-issuance ability, protects templates from being inverted, and demonstrates strong unlinkability across different databases. In addition, favorable verification/identification accuracy is obtained and the system satisfies the needs for real-time applications. A global measure $${D}_{\leftrightarrow }^{sys}$$ D ↔ sys value of 0.01 is obtained, and thus, correlation attacks are mitigated. The recognition results for the legitimate scenario are 100% identification accuracy and 0% EER. For the worst case (same token scenario), the corresponding results are 99.752% and 0.31%, respectively.

Biometric template protection schemes are designed to provide specific security guarantees to biometric traits. However, most of these schemes do not address the unlinkability requirement (viz., multiple biometric templates should be independent of each other) from the theoretical vantage point. This study proposes a mechanism for storing fingerprint templates in an unlikable manner by hiding them within fixed-sized groups. The core of our model is based on the notion of k-anonymity, which guarantees that the biometric template of a particular subject remains concealed among the templates of other k - 1 subjects. The resulting anonymized features ensure that the identity of the subject does not get revealed under any circumstance, thereby preventing cross-matching or linking-based attacks within different applications. We have formally analyzed our model by quantifying the degree of anonymization of the fingerprint templates in the form of a metric having range [0, 1]. The entire scheme remains non-invertible due to the generalization of the fingerprint features. We have performed extensive empirical evaluations of our model over four benchmark fingerprint databases, for which we obtained EERs that are comparable to that in the stolen-token scenario. Hence our work introduces an approach for securing biometric databases in a provable manner. [ABSTRACT FROM AUTHOR]

In this work, we develop two stable estimators for solving linear functional regression problems. It is well known that such a problem is an ill-posed stochastic inverse problem. Hence, a special interest has to be devoted to the stability issue in the design of an estimator for solving such a problem. Our proposed estimators are based on combining a stable least-squares technique and a random projection of the slope function β 0 (·) ∈ L 2 (J) , where J is a compact interval. Moreover, these estimators have the advantage of having a fairly good convergence rate with reasonable computational load, since the involved random projections are generally performed over a fairly small dimensional subspace of L 2 (J). More precisely, the first estimator is given as a least-squares solution of a regularized minimization problem over a finite dimensional subspace of L 2 (J). In particular, we give an upper bound for the empirical risk error as well as the convergence rate of this estimator. The second proposed stable LFR estimator is based on combining the least-squares technique with a dyadic decomposition of the i.i.d. samples of the stochastic process, associated with the LFR model. In particular, we provide an L 2 -risk error of this second LFR estimator. Finally, we provide some numerical simulations on synthetic as well as on real data that illustrate the results of this work. These results indicate that our proposed estimators are competitive with some existing and popular LFR estimators. [ABSTRACT FROM AUTHOR]

High dimensional learning is data-hungry in general; however, many natural data sources and real-world learning problems posses some hidden low-complexity structure that permit effective learning from relatively small sample sizes. We are interested in the general question of how to discover and exploit such hidden benign traits when problem-specific prior knowledge is insufficient. In this work, we address this question through random projection's ability to expose structure. We study both compressive learning and high dimensional learning from this angle by introducing the notions of compressive distortion and compressive complexity. We give user-friendly PAC bounds in the agnostic setting that are formulated in terms of these quantities, and we show that our bounds can be tight when these quantities are small. We then instantiate these quantities in several examples of particular learning problems, demonstrating their ability to discover interpretable structural characteristics that make high dimensional instances of these problems solvable to good approximation in a random linear subspace. In the examples considered, these turn out to resemble some familiar benign traits such as the margin, the margin distribution, the intrinsic dimension, the spectral decay of the data covariance, or the norms of parameters—while our general notions of compressive distortion and compressive complexity serve to unify these, and may be used to discover benign structural traits for other PAC-learnable problems. [ABSTRACT FROM AUTHOR]

Nonnegative tensor factorizations (NTF) have applications in statistics, computer vision, exploratory multi-way data analysis, and blind source separation. This paper studies randomized multiplicative updating algorithms for symmetric NTF via random projections and random samplings. For random projections, we consider two methods to generate the random matrix and analyze the computational complexity, while for random samplings the uniform sampling strategy and its variants are examined. The mixing of these two strategies is then considered. Some theoretical results are presented based on the bounds of the singular values of sub-Gaussian matrices and the fact that randomly sampling rows from an orthogonal matrix results in a well-conditioned matrix. These algorithms are easy to implement, and their efficiency is verified via test tensors from both synthetic and real datasets, such as for clustering facial images. [ABSTRACT FROM AUTHOR]

AbstractA simple test is proposed to test the independence of high-dimensional random normal vectors. The method consists of two steps. First, the primary high-dimensional data is projected onto a low-dimensional subspace multiple times using random projection matrices. Second, the test statistic is constructed by utilizing the classical statistics obtained from the projected low-dimensional datasets. Simulations are performed to compare the performance of the proposed test with existing state-of-the-art tests, in terms of test sizes and powers. Finally, the proposed methodology is illustrated using two gene datasets, namely the Colon and Leukemia cancer datasets. [ABSTRACT FROM AUTHOR]

In this article, we consider the problem of hypothesis testing in high-dimensional single-index models. First, we study the feasibility of applying the classical F-test to a single-index model when the dimension of covariate vector and sample size are of the same order, and derive its asymptotic null distribution and asymptotic local power function. For the ultrahigh-dimensional single-index model, we construct F-statistics based on lower-dimensional random projections of the data, and establish the asymptotic null distribution and the asymptotic local power function of the proposed test statistics for the hypothesis testing of global and partial parameters. The new proposed test possesses the advantages of having a simple structure as well as being easy to compute. We compare the proposed test with other high-dimensional tests and provide sufficient conditions under which the proposed tests are more efficient. We conduct simulation studies to evaluate the finite-sample performances of the proposed tests and demonstrate that it has higher power than some existing methods in the models we consider. The application of real high-dimensional gene expression data is also provided to illustrate the effectiveness of the method. for this article are available online. [ABSTRACT FROM AUTHOR]

We address the problem of testing for the invariance of a probability measure under the action of a group of linear transformations. We propose a procedure based on consideration of one-dimensional projections, justified using a variant of the Cramér–Wold theorem. Our test procedure is powerful, computationally efficient, and dimension-independent, extending even to the case of infinite-dimensional spaces (multivariate functional data). It includes, as special cases, tests for exchangeability and sign-invariant exchangeability. We compare our procedure with some previous proposals in these cases, in a small simulation study. The paper concludes with two real-data examples. [ABSTRACT FROM AUTHOR]

In high-dimensional data modeling, variable selection plays a crucial role in improving predictive accuracy and enhancing model interpretability through sparse representation. Unfortunately, certain variable selection methods encounter challenges such as insufficient model sparsity, high computational overhead, and difficulties in handling large-scale data. Recently, axis-aligned random projection techniques have been applied to address these issues by selecting variables. However, these techniques have seen limited application in handling complex data within the regression framework. In this study, we propose a novel method, sparse partial least squares via axis-aligned random projection, designed for the analysis of high-dimensional data. Initially, axis-aligned random projection is utilized to obtain a sparse loading vector, significantly reducing computational complexity. Subsequently, partial least squares regression is conducted within the subspace of the top-ranked significant variables. The submatrices are iteratively updated until an optimal sparse partial least squares model is achieved. Comparative analysis with some state-of-the-art high-dimensional regression methods demonstrates that the proposed method exhibits superior predictive performance. To illustrate its effectiveness, we apply the method to four cases, including one simulated dataset and three real-world datasets. The results show the proposed method’s ability to identify important variables in all four cases. [ABSTRACT FROM AUTHOR]

In unsupervised learning, clustering is a common starting point for data processing. The convex or concave fusion clustering method is a novel approach that is more stable and accurate than traditional methods such as k-means and hierarchical clustering. However, the optimization algorithm used with this method can be slowed down significantly by the complexity of the fusion penalty, which increases the computational burden. This paper introduces a random projection ADMM algorithm based on the Bernoulli distribution and develops a double random projection ADMM method for high-dimensional fusion clustering. These new approaches significantly outperform the classical ADMM algorithm due to their ability to significantly increase computational speed by reducing complexity and improving clustering accuracy by using multiple random projections under a new evaluation criterion. We also demonstrate the convergence of our new algorithm and test its performance on both simulated and real data examples. [ABSTRACT FROM AUTHOR]

AbstractWe propose a new method for high-dimensional semi-supervised learning problems based on the careful aggregation of the results of a low-dimensional procedure applied to many axis-aligned random projections of the data. Our primary goal is to identify important variables for distinguishing between the classes; existing low-dimensional methods can then be applied for final class assignment. To this end, we score projections according to their class-distinguishing ability; for instance, motivated by a generalized Rayleigh quotient, we can compute the traces of estimated whitened between-class covariance matrices on the projected data. This enables us to assign an importance weight to each variable for a given projection, and to select our signal variables by aggregating these weights over high-scoring projections. Our theory shows that the resulting Sharp-SSL algorithm is able to recover the signal coordinates with high probability when we aggregate over sufficiently many random projections and when the base procedure estimates the diagonal entries of the whitened between-class covariance matrix sufficiently well. For the Gaussian EM base procedure, we provide a new analysis of its performance in semi-supervised settings that controls the parameter estimation error in terms of the proportion of labeled data in the sample. Numerical results on both simulated data and a real colon tumor dataset support the excellent empirical performance of the method. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. [ABSTRACT FROM AUTHOR]

The projected gradient method for matrix completion is generalized towards the higher-dimensional case of low-rank Tucker tensors. It is shown that an operation order rearrangement in the common projected gradient approach provides a complexity improvement. An even better algorithm complexity can be obtained by replacing the completion operator by a general operator that satisfies restricted isometry property; however, such a replacement transforms the completion algorithm into an approximation algorithm. [ABSTRACT FROM AUTHOR]

We present a general framework for dealing with set heterogeneity in data and learning problems, which is able to exploit low complexity components. The main ingredients are (i) A definition of complexity for elements of a convex union that takes into account the complexities of their individual composition – this is used to cover the heterogeneous convex union; and (ii) Upper bounds on the complexities of restricted subsets. We demonstrate this approach in two different application areas, highlighting their conceptual connection. (1) In random projection based dimensionality reduction, we obtain improved bounds on the uniform preservation of Euclidean norms and distances when low complexity components are present in the union. (2) In statistical learning, our generalisation bounds justify heterogeneous ensemble learning methods that were incompletely understood before. We exemplify empirical results with boosting type random subspace and random projection ensembles that implement our bounds. [ABSTRACT FROM AUTHOR]

Several problems in stochastic analysis are defined through their geometry, and preserving that geometric structure is essential to generating meaningful predictions. Nevertheless, how to design principled deep learning (DL) models capable of encoding these geometric structures remains largely unknown. We address this open problem by introducing a universal causal geometric DL framework in which the user specifies a suitable pair of metric spaces X$\mathcal {X}$ and Y$\mathcal {Y}$ and our framework returns a DL model capable of causally approximating any "regular" map sending time series in XZ$\mathcal {X}^{\mathbb {Z}}$ to time series in YZ$\mathcal {Y}^{\mathbb {Z}}$ while respecting their forward flow of information throughout time. Suitable geometries on Y$\mathcal {Y}$ include various (adapted) Wasserstein spaces arising in optimal stopping problems, a variety of statistical manifolds describing the conditional distribution of continuous‐time finite state Markov chains, and all Fréchet spaces admitting a Schauder basis, for example, as in classical finance. Suitable spaces X$\mathcal {X}$ are compact subsets of any Euclidean space. Our results all quantitatively express the number of parameters needed for our DL model to achieve a given approximation error as a function of the target map's regularity and the geometric structure both of X$\mathcal {X}$ and of Y$\mathcal {Y}$. Even when omitting any temporal structure, our universal approximation theorems are the first guarantees that Hölder functions, defined between such X$\mathcal {X}$ and Y$\mathcal {Y}$ can be approximated by DL models. [ABSTRACT FROM AUTHOR]

This paper presents an ensemble method for binary classification, where each base model is based on an extended neighbourhood rule (ExNRule). The ExNRule identifies the neighbours of an unseen observation in a stepwise manner. This rule first selects the sample point closest to the experimental observation and then selects the second observation nearest to the previously chosen one. To find the required data points in the neighbourhood, this search is repeated up to k steps. The test sample point is predicted using majority voting in the class labels of the k chosen neighbours. In the proposed method, a large number of ExNRule based models are constructed on randomly projected bootstrap samples. The error rates of these models are computed using out-of-bag data points. The models are then ranked according to their out-of-bag errors, and a proportion of the most accurate models are selected. The final ensemble is constructed by combining the selected models. The proposed method is compared with other classical procedures on 15 benchmark datasets in terms of classification accuracy, Kohen’s kappa and Brier score (BS) as performance metrics. Boxplots of the results are also constructed. The proposed ensemble is outperforming the existing methods on almost all the benchmark datasets. For further evaluation, the proposed method is compared with other kNN based classifiers on 3 datasets using different k values. Furthermore, the performance of the proposed method is also evaluated using simulated data under different scenarios.

In many real-world applications, data are described by high-dimensional feature spaces, posing new challenges for current ensemble clustering methods. The goal is to combine sets of base clusters to enhance clustering accuracy, but this makes them susceptible to low quality. However, the reliability of present ensemble clustering in high-dimensional data still needs improvement. In this context, we propose a new fuzzy divergence-weighted ensemble clustering based on random projection and spectral learning. Firstly, random projection (RP) is used to create various dimensional data and find membership matrices via fuzzy c-means (FCM). Secondly, fuzzy partitions of random projections are ranked using entropy-based local weighting along with Kullback-Leibler (KL) divergence to detect any uncertainty. Then it used to evaluate the weight of each cluster. Finally, we create regularized graphs from these membership matrices and use spectral matrices to estimate the affinity matrices of these graphs using fuzzy KL divergence anchor graphs. Subsequently, obtaining the final clustering results is considered as an optimization problem, and the ensemble clustering results are obtained. The experimental results on high-dimensional data demonstrate the efficiency of our method compared to state-of-the-art methods.

Cancelable biometrics serves as an effective countermeasure against various template attacks launched by intruders, safeguarding the biometric system. This paper proposes a cancelable approach with a novel feature extraction technique for iris recognition, known as the hybrid architecture of the convolutional neural network (CNN) and GRU (gated recurrent unit). To provide cancelability to the system, the paper makes use of a random projection technique. The proposed method has the best outcome in terms of accurate identification. The method is validated on two Iris datasets IITD and MMU, which show promising results on the equal error rate (EER) and accuracy. The proposed model provides 0.02 and 0.045 EER for IITD and MMU, respectively, and accuracy 0.98 and 0.933%, for IITD and MMU Iris dataset, respectively, which is very high compared to other methodologies. The proposed hybrid architecture is being used for a cancelable biometric system for the first time based on literature review. The efficiency of the proposed method is high when validated on the datasets. [ABSTRACT FROM AUTHOR]

The stacking pseudoinverse learning algorithm can effectively improve the classification accuracy of the model and reduce the training time. However, the effect of random projection blocks on the neural network is often ignored, which reduces the performance of stacked generalization. To improve the generalization performance of neural networks and obtain high-quality classification results, we propose a random projection stacking pseudoinverse learning algorithm based on attention mechanism named AM-RP stacking PILers. Firstly, the input weight matrices with specific distributions are randomly generated, and then different random projection blocks are obtained through a pseudoinverse learning algorithm. Secondly, the training results of different random projection blocks are taken as a new input dataset, and the idea of attention mechanism is introduced to assign different weights to different random projection blocks. Finally, we use the stacking pseudoinverse learning algorithm to train a single hidden layer neural network and obtain the classification results with high generalization performance. The experimental results on a total of 76788 images in three public datasets of Salinas, MNIST and COIL-20 show that our algorithm achieves better performance in accuracy, precision, recall, F1 score and training time. [ABSTRACT FROM AUTHOR]

Our research aims to improve automated intrusion detection by developing a highly accurate classifier with minimal false alarms. The motivation behind our work is to tackle the challenges of high dimensionality in intrusion detection and enhance the classification performance of classifiers, ultimately leading to more accurate and efficient detection of intrusions. To achieve this, we conduct experiments using the NSL-KDD data set, a widely used benchmark in this domain. This data set comprises approximately 126,000 samples of normal and abnormal network traffic for training and 23,000 samples for testing. Initially, we employ the entire feature set to train classifiers, and the outcomes are promising. Among the classifiers tested, the J48 tree achieves the highest reported accuracy of 79.1 percent. To enhance classifier performance, we explore two projection approaches: Random Projection and PCA. Random Projection yields notable improvements, with the PART algorithm achieving the best-reported accuracy of 82.0 %, outperforming the original feature set. Moreover, random projection proves to be more time-efficient than PCA across most classifiers. Our findings demonstrate the effectiveness of random projection in improving intrusion detection accuracy while reducing training time. This research contributes valuable insights to the cybersecurity field and fosters potential advancements in intrusion detection systems. [ABSTRACT FROM AUTHOR]

Bearings are critical components in rotating machinery, and their failure can lead to costly repairs and downtime. To prevent such failures, it is important to detect and diagnose bearing faults early. In recent years, deep-learning techniques have shown promise for detecting and diagnosing bearing faults automatically. While these algorithms can all achieve diagnostic accuracy of over 90%, their generalizability and robustness in complex, extreme variable loading conditions have not been thoroughly validated. In this paper, a feature extraction method based on Synchro-squeezing Wavelet Transform (SSWT), Random projection (RP), and deep learning (DL) is presented. To fulfill the data requirements of neural networks, data augmentation is initially utilized to augment the size of the original data. Subsequently, the SSWT technique is employed to convert the signals from the Time domain to the Time-Frequency domain, resulting in the conversion of the 1-D signal to a 2-D feature image. To decrease the complexity of deep learning computation, data preprocessing involves utilizing Random projection to reduce feature dimensionality. The final step involves constructing a Convolutional Neural Network (CNN) model that can identify fault features from the obtained Time-Frequency images and perform accurate fault classification. By utilizing the CWRU and IMS datasets to evaluate the method, the study demonstrates that the suggested approach outperforms advanced techniques in terms of both diagnostic accuracy and robustness.

Our research aims to improve automated intrusion detection by developing a highly accurate classifier with minimal false alarms. The motivation behind our work is to tackle the challenges of high dimensionality in intrusion detection and enhance the classification performance of classifiers, ultimately leading to more accurate and efficient detection of intrusions. To achieve this, we conduct experiments using the NSL-KDD data set, a widely used benchmark in this domain. This data set comprises approximately 126,000 samples of normal and abnormal network traffic for training and 23,000 samples for testing. Initially, we employ the entire feature set to train classifiers, and the outcomes are promising. Among the classifiers tested, the J48 tree achieves the highest reported accuracy of 79.1 percent. To enhance classifier performance, we explore two projection approaches: Random Projection and PCA. Random Projection yields notable improvements, with the PART algorithm achieving the best-reported accuracy of 82.0 %, outperforming the original feature set. Moreover, random projection proves to be more time-efficient than PCA across most classifiers. Our findings demonstrate the effectiveness of random projection in improving intrusion detection accuracy while reducing training time. This research contributes valuable insights to the cybersecurity field and fosters potential advancements in intrusion detection systems.

Momentum Iterative Hessian Sketch (▪) techniques, a group of solvers for large scale regularized linear Least Squares (LS) problems, are proposed and analyzed in detail. The proposed ▪ techniques are obtained by incorporating Polyak's heavy ball acceleration into the Iterative Hessian Sketch algorithm and they provide significant improvements over the randomized preconditioning techniques. By solving the linear systems arising in the sub-problems during the iterations approximately, the proposed techniques are capable of avoiding all matrix decompositions and inversions, which is one of the main advantages over the alternative solvers such as the Blendenpik and the LSRN. Similar to the Chebyshev semi-iterations, the ▪ variants do not use any inner products and eliminate the corresponding synchronization steps in hierarchical or distributed memory systems, yet the ▪ converges faster than the Chebyshev semi-iteration based solvers. Lower bounds on the required sketch size for various randomized distributions are established through the error analyses. Unlike the previously proposed approaches to produce a solution approximation, the proposed ▪ techniques can use sketch sizes that are proportional to the statistical dimension which is always smaller than the rank of the coefficient matrix. • A group of novel least squares solvers based on the random projections is proposed for large-scale linear systems. • The proposed techniques provide significant improvements over the existing randomized preconditioning methods. • The presented analysis shows that the sketch size can be chosen proportional to the statistical dimension of the problem. • An heuristic to estimate the statistical dimension is proposed. • An efficient linear system solver for problems of the form A T A x = b that arise during the iterations is proposed. [ABSTRACT FROM AUTHOR]

We present some randomized algorithms for computing multilinear rank- (μ 1 , μ 2 , μ 3) approximations of tensors by combining the sparse subspace embedding and the singular value decomposition. The error bound for this algorithm with the high probability is obtained by the properties of sparse subspace embedding. Furthermore, combining the power scheme and the proposed randomized algorithm, we derive a three-stage randomized algorithm and make a probabilistic analysis for its error bound. The efficiency of the proposed algorithms is illustrated via numerical examples. [ABSTRACT FROM AUTHOR]

Nowadays, enormous amounts of data are produced every second. These data also contain private information from sources including media platforms, the banking sector, finance, healthcare, and criminal histories. Data mining is a method for looking through and analyzing massive volumes of data to find usable information. Preserving personal data during data mining has become difficult, thus privacy-preserving data mining (PPDM) is used to do so. Data perturbation is one of the several tactics used by the PPDM data privacy protection mechanism. In perturbation, datasets are perturbed in order to preserve personal information. Both data accuracy and data privacy are addressed by it. This paper will explore and compare several hybrid perturbation strategies that may be used to protect data privacy. For this, two perturbation-based techniques named improved random projection perturbation (IRPP) and enhanced principal component analysis-based technique (EPCAT) were used. These methods are employed to assess the precision, run time, and accuracy of the experimental results. This paper provides the impacts of perturbation-based privacy preserving techniques. It is observed that hybrid approaches are more efficient than the traditional approach. [ABSTRACT FROM AUTHOR]

In unsupervised learning, clustering is a common starting point for data processing. The convex or concave fusion clustering method is a novel approach that is more stable and accurate than traditional methods such as k-means and hierarchical clustering. However, the optimization algorithm used with this method can be slowed down significantly by the complexity of the fusion penalty, which increases the computational burden. This paper introduces a random projection ADMM algorithm based on the Bernoulli distribution and develops a double random projection ADMM method for high-dimensional fusion clustering. These new approaches significantly outperform the classical ADMM algorithm due to their ability to significantly increase computational speed by reducing complexity and improving clustering accuracy by using multiple random projections under a new evaluation criterion. We also demonstrate the convergence of our new algorithm and test its performance on both simulated and real data examples.

Online learning has gained immense popularity, especially since the COVID-19 pandemic. However, it has also brought its own set of challenges. One of the critical challenges in online learning is the ability to evaluate students' concentration levels during virtual classes. Unlike traditional brick-and-mortar classrooms, teachers do not have the advantage of observing students' body language and facial expressions to determine whether they are paying attention. To address this challenge, this study proposes utilizing facial and body gestures to evaluate students' concentration levels. Common gestures such as yawning, playing with fingers or objects, and looking away from the screen indicate a lack of focus. A dataset containing images of students performing various actions and gestures representing different concentration levels is collected. We propose an enhanced model based on a vision transformer (RViT) to classify the concentration levels. This model incorporates a majority voting feature to maintain real-time prediction accuracy. This feature classifies multiple frames, and the final prediction is based on the majority class. The proposed method yields a promising 92% accuracy while maintaining efficient computational performance. The system provides an unbiased measure for assessing students' concentration levels, which can be useful in educational settings to improve learning outcomes. It enables educators to foster a more engaging and productive virtual classroom environment. [ABSTRACT FROM AUTHOR]

Multivariate time‐series (MTS) data are prevalent in diverse domains and often high dimensional. We propose new random projection ensemble classifiers with high‐dimensional MTS. The method first applies dimension reduction in the time domain via randomly projecting the time‐series variables into some low‐dimensional space, followed by measuring the disparity via some novel base classifier between the data and the candidate generating processes in the projected space. Our contributions are twofold: (i) We derive optimal weighted majority voting schemes for pooling information from the base classifiers for multiclass classification and (ii) we introduce new base frequency‐domain classifiers based on Whittle likelihood (WL), Kullback‐Leibler (KL) divergence, eigen‐distance (ED), and Chernoff (CH) divergence. Both simulations for binary and multiclass problems, and an Electroencephalogram (EEG) application demonstrate the efficacy of the proposed methods in constructing accurate classifiers with high‐dimensional MTS. [ABSTRACT FROM AUTHOR]

The problem that the randomly generated random projection matrix will lead to unstable classification results is addressed in this paper. To this end, a Tighter Random Projection-oriented entropy-weighted ensemble algorithm is proposed for classifying hyperspectral remote sensing images. In particular, this paper presents a random projection matrix selection strategy based on the separable information of a single class able to project the features of a certain class of objects. The projection result is measured by the degree of separability, thereby obtaining the low-dimensional image with optimal separability of the class. After projecting samples with the same random projection matrix, to calculate the distance matrix, the Minimum Distance classifier is devised, repeating for all classes. Finally, the weight of the distance matrix is considered in ensemble classification by using the information entropy. The proposed algorithm is tested on real hyperspectral remote sensing images. The experiments show an increase in both stability and performance. [ABSTRACT FROM AUTHOR]

The Johnson-Lindenstrauss Lemma states that there exist linear maps that project a set of points of a vector space into a space of much lower dimension such that the Euclidean distance between these points is approximately preserved. This lemma has been previously used to prove that we can randomly aggregate, using a random matrix whose entries are drawn from a zero-mean sub-Gaussian distribution, the equality constraints of an Linear Program (LP) while preserving approximately the value of the problem. In this paper we extend these results to the inequality case by introducing a random matrix with non-negative entries that allows to randomly aggregate inequality constraints of an LP while preserving approximately the value of the problem. By duality, the approach we propose allows to reduce both the number of constraints and the dimension of the problem while obtaining some theoretical guarantees on the optimal value. We will also show an extension of our results to certain semidefinite programming instances. [ABSTRACT FROM AUTHOR]

In this paper, we consider the estimation algorithm of tensor response on vector covariate regression model. Based on projection theory of tensor and the idea of randomized algorithm for tensor decomposition, three new algorithms named SHOLRR, RHOLRR and RSHOLRR are proposed under the low‐rank Tucker decomposition and some theoretical analyses for two randomized algorithms are also provided. To explore the nonlinear relationship between tensor response and vector covariate, we develop the KRSHOLRR algorithm based on kernel trick and RSHOLRR algorithm. Our proposed algorithms can not only guarantee high estimation accuracy but also have the advantage of fast computing speed, especially for higher‐order tensor response. Through extensive synthesized data analyses and applications to two real datasets, we demonstrate the outperformance of our proposed algorithms over the stat‐of‐art. [ABSTRACT FROM AUTHOR]

Cancellable biometrics is the solution for the trade-off between two concepts: Biometrics for Security and Security for Biometrics. The cancelable template is stored in the authentication system's database rather than the original biometric data. In case of the database is compromised, it is easy for the template to be canceled and regenerated from the same biometric data. Recoverability of the cancelable template comes from the diversity of the cancelable transformation parameters (cancelable key). Therefore, the cancelable key must be secret to be used in the system authentication process as a second authentication factor in conjunction with the biometric data. The main contribution of this paper is to tackle the risks of stolen/lost/shared cancelable keys by using biometric trait (in different feature domains) as the only authentication factor, in addition to achieving good performance with high security. The standard Generative Adversarial Network (GAN) is proposed as an encryption tool that needs the cancelable key during the training phase, and the testing phase depends only on the biometric trait. Additionally, random projection transformation is employed to increase the proposed system's security and performance. The proposed transformation system is tested using the standard ORL face database, and the experiments are done by applying different features domains. Moreover, a security analysis for the proposed transformation system is presented. [ABSTRACT FROM AUTHOR]

Inspired by the definition of tensor-tensor product and tensor tubal rank, a randomized singular value decomposition of tensor is presented in this paper. Based on tensor singular value decomposition (t-SVD) and tensor randomized singular value decomposition (t-RSVD), we obtain two efficient algorithms to solve tensor completion problem. We also propose the adaptive rank method to adjust the tubal rank of tensor. The main advantage of the random projection-based t-RSVD is to cut down the computing time in consideration of large-scale problems. In the optimization process, the alternating minimization algorithm is employed to solve the tensor completion problem. Finally, numerical experiment results indicate that the t-RSVD is competitive and consumes less time than the t-SVD. The efficiency and feasibility of our methods are illustrated by the image and video recovery. [ABSTRACT FROM AUTHOR]

This work proposes a Tensor Train Random Projection (TTRP) method for dimension reduction, where pairwise distances can be approximately preserved. Our TTRP is systematically constructed through a Tensor Train (TT) representation with TT-ranks equal to one. Based on the tensor train format, this random projection method can speed up the dimension reduction procedure for high-dimensional datasets and requires fewer storage costs with little loss in accuracy, comparedwith existingmethods. We provide a theoretical analysis of the bias and the variance of TTRP, which shows that this approach is an expected isometric projectionwith bounded variance, and we show that the scaling Rademacher variable is an optimal choice for generating the corresponding TT-cores. Detailed numerical experiments with synthetic datasets and theMNIST dataset are conducted to demonstrate the efficiency of TTRP. [ABSTRACT FROM AUTHOR]

There is an increasing body of work exploring the integration of random projection into algorithms for numerical linear algebra. The primary motivation is to reduce the overall computational cost of processing large datasets. A suitably chosen random projection can be used to embed the original dataset in a lower-dimensional space such that key properties of the original dataset are retained. These algorithms are often referred to as sketching algorithms, as the projected dataset can be used as a compressed representation of the full dataset. We show that random matrix theory, in particular the Tracy–Widom law, is useful for describing the operating characteristics of sketching algorithms in the tall-data regime when the sample size n is much greater than the number of variables d. Asymptotic large sample results are of particular interest as this is the regime where sketching is most useful for data compression. In particular, we develop asymptotic approximations for the success rate in generating random subspace embeddings and the convergence probability of iterative sketching algorithms. We test a number of sketching algorithms on real large high-dimensional datasets and find that the asymptotic expressions give accurate predictions of the empirical performance. [ABSTRACT FROM AUTHOR]

The study of disease-gene associations is an important topic in the field of computational biology. The accumulation of massive amounts of biomedical data provides new possibilities for exploring potential relations between diseases and genes through computational strategy, but how to extract valuable information from the data to predict pathogenic genes accurately and rapidly is currently a challenging and meaningful task. Therefore, we present a novel computational method called PGAGP for inferring potential pathogenic genes based on an adaptive network embedding algorithm. The PGAGP algorithm is to first extract initial features of nodes from a heterogeneous network of diseases and genes efficiently and effectively by Gaussian random projection and then optimize the features of nodes by an adaptive refining process. These low-dimensional features are used to improve the disease-gene heterogenous network, and we apply network propagation to the improved heterogenous network to predict pathogenic genes more effectively. By a series of experiments, we study the effect of PGAGP’s parameters and integrated strategies on predictive performance and confirm that PGAGP is better than the state-of-the-art algorithms. Case studies show that many of the predicted candidate genes for specific diseases have been implied to be related to these diseases by literature verification and enrichment analysis, which further verifies the effectiveness of PGAGP. Overall, this work provides a useful solution for mining disease-gene heterogeneous network to predict pathogenic genes more effectively. [ABSTRACT FROM AUTHOR]

Directed networks are broadly used to represent asymmetric relationships among units. Co-clustering aims to cluster the senders and receivers of directed networks simultaneously. In particular, the well-known spectral clustering algorithm could be modified as the spectral co-clustering to co-cluster directed networks. However, large-scale networks pose great computational challenges to it. In this paper, we leverage sketching techniques and derive two randomized spectral co-clustering algorithms, one random-projection-based and the other random-sampling-based, to accelerate the co-clustering of large-scale directed networks. We theoretically analyze the resulting algorithms under two generative models - the stochastic co-block model and the degree-corrected stochastic co-block model, and establish their approximation error rates and misclustering error rates, indicating better bounds than the state-of-the-art results of co-clustering literature. Numerically, we design and conduct simulations to support our theoretical results and test the efficiency of the algorithms on real networks with up to millions of nodes. A publicly available R package RandClust is developed for better usability and reproducibility of the proposed methods. [ABSTRACT FROM AUTHOR]

In this short note, we extend to the continuous case a mean projection theorem for discrete determinantal point processes associated with a finite range projection, thus strengthening a known result in random linear algebra due to Ermakov and Zolotukhin. We also give a new formula for the variance of the exterior power of the random projection. [ABSTRACT FROM AUTHOR]

It is known that a reduced matrix obtained by Gaussian random projections preserves the stability of an original matrix in a probabilistic manner. However, due to unknown constants appearing in a probability concentration inequality, the exact value of the probability when stability is preserved, or the condition for the dimension of the reduced matrix to preserve stability with high probability, is not known. In this study, we derive the stable probability of the reduced matrix obtained by Gaussian random projections by presenting a variant of the theorem used for the existing research without unknown constants. [ABSTRACT FROM AUTHOR]

In a projection-based model reduction, a Galerkin-type projection is frequently used to generate a reduced matrix. However, the stability may not be preserved and the computational effort for generating the projection is not negligible. In this study, we use random projections to reduce an original large-scale matrix. We show that the stability of the reduced matrix is guaranteed with high probability and that the matrix can be obtained efficiently. [ABSTRACT FROM AUTHOR]

We consider the problem of detecting significance in high-dimensional linear models, in which the dimension of the regression coefficient is greater than the sample size. We propose novel test statistics for hypothesis tests of the global significance of the linear model, as well as for the significance of part of the regression coefficients. The new tests are based on randomly projecting the high-dimensional data onto a low-dimensional space, and then working with the classical F-test using the projected data. An appealing feature of the proposed tests is that they have a simple form and are computationally easy to implement. We derive the asymptotic local power functions of the proposed tests and compare them with the existing methods for hypothesis testing in high-dimensional linear models. We also provide a sufficient condition under which our proposed tests have higher asymptotic relative efficiency. Simulation studies evaluate the finite-sample performance of the proposed tests and demonstrate that it outperforms existing tests in the models considered. Lastly, we illustrate the proposed tests by applying them to real high-dimensional gene expression data. [ABSTRACT FROM AUTHOR]