Search

Traditional methods for identifying hit molecules from a large collection of potential drug-like candidates rely on biophysical theory to compute approximations to the Gibbs free energy of the binding interaction between the drug and its protein target. These approaches have a significant limitation in that they require exceptional computing capabilities for even relatively small collections of molecules. Increasingly large and complex state-of-the-art deep learning approaches have gained popularity with the promise to improve the productivity of drug design, notorious for its numerous failures. However, as deep learning models increase in their size and complexity, their acceleration at the hardware level becomes more challenging. Hyperdimensional Computing (HDC) has recently gained attention in the computer hardware community due to its algorithmic simplicity relative to deep learning approaches. The HDC learning paradigm, which represents data with high-dimension binary vectors, allows the use of low-precision binary vector arithmetic to create models of the data that can be learned without the need for the gradient-based optimization required in many conventional machine learning and deep learning methods. This algorithmic simplicity allows for acceleration in hardware that has been previously demonstrated in a range of application areas (computer vision, bioinformatics, mass spectrometery, remote sensing, edge devices, etc.). To the best of our knowledge, our work is the first to consider HDC for the task of fast and efficient screening of modern drug-like compound libraries. We also propose the first HDC graph-based encoding methods for molecular data, demonstrating consistent and substantial improvement over previous work. We compare our approaches to alternative approaches on the well-studied MoleculeNet dataset and the recently proposed LIT-PCBA dataset derived from high quality PubChem assays. We demonstrate our methods on multiple target hardware platforms, including Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs), showing at least an order of magnitude improvement in energy efficiency versus even our smallest neural network baseline model with a single hidden layer. Our work thus motivates further investigation into molecular representation learning to develop ultra-efficient pre-screening tools. We make our code publicly available at https://github.com/LLNL/hdbind .

The rapid expansion of the Internet of Things (IoT) has facilitated the interconnection of numerous ubiquitous and heterogeneous devices within networks. Intrusion detection system (IDS) is crucial for ensuring the security of the IoT, particularly in detecting unknown attacks. Most existing studies are based on supervised IDSs, requiring the labor-intensive labeling of large amounts of data. Thus, it is essential to implement unsupervised IDSs that do not rely on prior knowledge of cyberattacks and eliminate the need for labeling. However, such IDSs may suffer from a high false alarm rate (FAR). This study presents a Memory Module-assisted Convolutional Autoencoder-based (MACAE) model for unsupervised intrusion detection in IoT networks. Specifically, we convert raw network traffic data into images, avoiding manually designing features in large-scale samples. A Convolutional Neural Network (CNN) is then employed to learn spatial structure for representation, capturing high-dimensional features. Subsequently, a memory module is integrated into the latent space of Autoencoder to enhance the model’s ability to remember prototypical normal patterns, thereby mitigating the issue of a high FAR. Experiments were conducted using a real smart grid dataset, and results show that MACAE achieves the lowest FAR of 0.0511, which is a 90.80%, 89.92%, and 83.92% reduction compared to the unsupervised methods DAGMM, VAE, and Deep SVDD, respectively. Furthermore, the proposed method has been verified to have good generalization, adaptability, and unknown attack detection capability for the CICIDS2017 and MedBIoT datasets. [ABSTRACT FROM AUTHOR]

Temporal knowledge graph completion (TKGC) focuses on inferring missing facts from temporal knowledge graphs (TKGs) and has been widely studied. While previous models based on graph neural networks (GNNs) have shown noteworthy outcomes, they tend to focus on designing complex modules to learn contextual representations. These complex solutions require a large number of parameters and heavy memory consumption. Additionally, existing TKGC approaches focus on exploiting static feature representation for entities and relationships, which fail to effectively capture the semantic information of contexts. In this paper, we propose a global and local information-aware relational graph convolutional neural network (GLARGCN) model to address these issues. First, we design a sampler, which captures significant neighbors by combining global historical event frequencies with local temporal relative displacements and requires no additional learnable parameters. We then employ a time-aware encoder to model timestamps, relations, and entities uniformly. We perform a graph convolution operation to learn a global graph representation. Finally, our method predicts missing entities using a scoring function. We evaluate the model on four benchmark datasets and one specific dataset with unseen timestamps. The experimental results demonstrate that our proposed GLARGCN model not only outperforms contemporary models but also shows robust performance in scenarios with unseen timestamps. [ABSTRACT FROM AUTHOR]

Fine-grained visual classification (FGVC) seeks to identify sub-classes within the same meta-class. Prior efforts mainly mine the features of discriminative parts to enhance classification performance. However, we argue that most of these works ignore the spatial details inside each part and the spatial correlations between parts when extracting local features and fusing global features, inhibiting the further improvement of feature quality, especially for the irregular discriminative parts. To alleviate this issue, we rethink the feature generation route from pixels to parts and to objects, and propose a novel graph-in-graph discriminative feature enhancement network (G 2 DFE-Net). Specifically, the G 2 DFE-Net consists of two nested graph convolutional networks, where an internal graph is first developed based on the spatial attention strategy to highlight details of the irregular discriminative regions. Then, a KNN-based external graph is introduced to capture the spatial context correlation among independent discriminative parts. With the collaboration of internal and external graph, G 2 DFE-Net boosts the class separability and compactness of global feature representation, thereby benefiting the accurate FGVC. We conduct thorough experiments on five benchmark datasets, and both quantitative and qualitative results confirm the superior accuracy of our G 2 DFE-Net compared to previous state-of-the-art algorithms. The code is available at [ABSTRACT FROM AUTHOR]

Graph neural networks (GNNs) have gained prominence as an effective technique for representation learning and have found wide application in tag recommendation tasks. Existing approaches aim to encode the hidden collaborative information among entities into embedding representations by propagating node information between connected nodes. However, in sparse observable graph structures, a significant number of connections are missing, leading to incomplete and biased propagation. To address these issues, we propose a novel model called Low-frequency Spectral Graph Convolution Networks with one-hop connections information for Personalized Tag Recommendation (LSGCNT). This model utilizes graph convolution in the spectral domain and incorporates a graph structure comprising two bipartite graphs, the user–tag interaction graph and the item–tag interaction graph. Our model aims to reduce information loss caused by propagation by utilizing graph convolution networks with trainable convolution kernels to recover preference information. In order to preserve useful low-frequency signals, we couple graph convolution with low-pass filters in the frequency domain. Through reconstructing the true rating tensor and ranking the tag scores within the tensor, we can achieve top-N recommendations. Furthermore, to preserve the one-hop connection information of the bipartite graphs, we treat the observed two bipartite graphs as two homogeneous graphs, where both users and tags contribute to the convolution of a node in the user–tag graph, and both items and tags contribute to the convolution of a node in the item–tag graph. Lastly, we analyze the impact of different internal components, pooling methods, parameter choices, and prediction approaches of LSGCNT on recommendation performance. Experimental results on two real-world datasets demonstrate that LSGCNT achieves superior recommendation performance compared with eight other state-of-the-art recommendation models. [ABSTRACT FROM AUTHOR]

Data augmentation methods offer a cost-effective and efficient alternative to the acquisition of additional data, significantly enhancing data diversity and model generalization, making them particularly favored in object detection tasks. However, existing data augmentation techniques primarily focus on the visible spectrum and are directly applied to RGB-T object detection tasks, overlooking the inherent differences in image data between the two tasks. Visible images capture rich color and texture information during the daytime, while infrared images are capable of imaging under low-light complex scenarios during the nighttime. By integrating image information from both modalities, their complementary characteristics can be exploited to improve the overall effectiveness of data augmentation methods. To address this, we propose a cross-modality data augmentation method tailored for RGB-T object detection, leveraging masked image modeling within representation learning. Specifically, we focus on the temporal consistency of infrared images and combine them with visible images under varying lighting conditions for joint data augmentation, thereby enhancing the realism of the augmented images. Utilizing the masked image modeling method, we reconstruct images by integrating multimodal features, achieving cross-modality data augmentation in feature space. Additionally, we investigate the differences and complementarities between data augmentation methods in data space and feature space. Building upon existing theoretical foundations, we propose an integrative framework that combines these methods for improved augmentation effectiveness. Furthermore, we address the slow convergence observed with the existing Mosaic method in aerial imagery by introducing a multi-scale training strategy and proposing a full-scale Mosaic method as a complement. This optimization significantly accelerates network convergence. The experimental results validate the effectiveness of our proposed method and highlight its potential for further advancements in cross-modality object detection tasks. [ABSTRACT FROM AUTHOR]

Federated learning ensures the privacy of clients by conducting distributed training on individual client devices and sharing only the model weights with a central server. However, in real-world scenarios, especially in IoT scenarios where devices have varying capabilities and data heterogeneity exists among IoT clients, appropriate personalization methods are necessary. In this paper, this work aims to address this heterogeneity using a form of parameter decoupling known as representation learning. Representation learning divides deep learning models into 'base' and 'head' components. The base component, capturing common features across all clients, is shared with the server, while the head component, capturing unique features specific to individual clients, remains local. This work proposes a new representation learning-based approach, named FedSeq, that suggests decoupling the entire deep learning model into more densely divided parts with the application of suitable scheduling methods, which can benefit not only data heterogeneity but also class heterogeneity. FedSeq has two different layer scheduling approaches, namely forward (Vanilla) and backward (Anti), in the context of data and class heterogeneity among clients. Our experimental results show that FedSeq, when compared to existing personalized federated learning algorithms, achieves increased accuracy, especially under challenging conditions, while reducing computation costs. The study introduces a novel personalized federated learning approach that integrates sequential layer expansion and dynamic scheduling methods, demonstrating a 7.31% improvement in classification accuracy on the CIFAR-100 dataset and a 4.1% improvement on the Tiny-ImageNet dataset compared to existing methods, while also reducing computation costs by up to 15%. Furthermore, Anti Scheduling achieves a computational efficiency improvement of 3.91% compared to FedAvg and 3.06% compared to FedBABU, while Vanilla Scheduling achieves a significant efficiency improvement of 63.93% compared to FedAvg and 63.61% compared to FedBABU. [ABSTRACT FROM AUTHOR]

The continuous combination of digital network technology and traditional financial services has given birth to digital financial networks, which explore massive economic data under the AI-driven models to achieve intelligent connections among financial institutions, markets, transactions, and instruments. Empirical asset pricing is a challenging task in financial analysis, which has attracted research attention. However, existing studies only focus on tackling the challenges of equity risk premium in the single stock market. Considering multiple economic linkages between the two countries, the transaction history of the US stock market as empirical knowledge is a powerful supplement to improve the prediction of equity risk premium in the China market. In this paper, we aim to fully leverage the prior information in two stock markets for empirical asset pricing models. Due to the rich financial domain knowledge, there may be various characteristic signals that partially overlap in different periods. To address these issues, we propose a framework based on long-short dual-mode knowledge distillation, termed as LSDM-KD, which incorporates US and China stock market models, and a shared characteristic signals model. The method effectively understands the relationships between assets and market behaviour, reducing reliance on expensive correlation databases and professional knowledge. Extensive experiments conducted on US and China stock market datasets demonstrate that our LSDM-KD can significantly improve the performance of empirical asset pricing. [ABSTRACT FROM AUTHOR]

Understanding the interactions between a ligand and its molecular target is crucial in guiding the optimization of molecules for any in silico drug design workflow. Multiple experimental and computational methods have been developed to better understand these intermolecular interactions. With the availability of a large number of structural datasets, there is a need for developing statistical frameworks that improve upon existing physics-based solutions. Here, we report a method based on geometric deep learning that is capable of predicting the binding conformations of ligands to protein targets. A technique to generate graphical representations of proteins was developed to exploit the topological and electrostatic properties of the binding region. The developed framework, based on graph neural networks, learns a statistical potential based on the distance likelihood, which is tailor-made for each ligand–target pair. This potential can be coupled with global optimization algorithms such as differential evolution to reproduce the experimental binding conformations of ligands. We show that the potential based on distance likelihood, described here, performs similarly or better than well-established scoring functions for docking and screening tasks. Overall, this method represents an example of how artificial intelligence can be used to improve structure-based drug design. [ABSTRACT FROM AUTHOR]

Although it requires simple computations, provides good performance on linear classification tasks and offers a suitable environment for active learning strategies, the Hebbian learning rule is very sensitive to how the training data relate to each other in the input space. Since this spatial arrangement is inherent to each set of samples, the practical application of this learning paradigm is limited. Thus, representation learning may play an important role in projecting the input data into a new space where linear separability is improved. Earlier methods based on orthogonal coding addressed this issue but presented many side effects, impoverishing the generalization of the model. Hence, this paper considers a recently proposed method based on kernel density estimators, which performs a likelihood-based projection where linear separability and generalization capacity are enhanced in an autonomous fashion. Results show that this novel method allows one to use linear classifiers to solve many binary classification problems and overcome the performance of well-established classifiers. [ABSTRACT FROM AUTHOR]

Structural deep document clustering methods, which leverage both structural information and inherent data properties to learn document representations using deep neural networks for clustering, have recently garnered increased research interest. However, the structural information used in these methods is usually static and remains unchanged during the clustering process. This can negatively impact the clustering results if the initial structural information is inaccurate or noisy. In this paper, we present an adaptive structural enhanced representation learning network for document clustering. This network can adjust the structural information with the help of clustering partitions and consists of two components: an adaptive structure learner, which automatically evaluates and adjusts structural information at both the document and term levels to facilitate the learning of more effective structural information, and a structural enhanced representation learning network. The latter incorporates integrates this adjusted structural information to enhance text document representations while reducing noise, thereby improving the clustering results. The iterative process between clustering results and the adaptive structural enhanced representation learning network promotes mutual optimization, progressively enhancing model performance. Extensive experiments on various text document datasets demonstrate that the proposed method outperforms several state-of-the-art methods. The overall framework of adaptive structural enhanced representation learning network [ABSTRACT FROM AUTHOR]

Accomplishment in various aspects of life is associated with the balanced intelligence of human beings. Intelligence can be classified into several types, each of them has an influence on one's individual and activity-related outcomes. Depending on the previous literature on intelligent systems, this article intends to trace the evolution of diverging types of multifaceted and multiple computational intelligence in the context of multimedia information framework; right from cognitive intelligence to naturalistic intelligence. We have illustrated a systematic schema of how the multiple and mutually exclusive intelligence categories affect the structural information processing phenomena. We have depicted a set of contextual case studies over the multimedia information patterns, such as images, linguistics, and music to enhance the capability of learning and representation. Thereby, distinctive characteristics of intelligence suggestively influence the entire multimedia information fusion schema in terms of multimedia pattern analysis, information retrieval, and recommendations. We have incorporated three diverge and semantic case studies for demonstrating the multifaceted intelligent information fusion on the multimedia contents such as (a) facial recognition system for auto-generated response management, (b) speech recognition module for response management and interactive system demonstration, and (c) music processing schema for illustrating pervasive music teaching–learning framework. We have evaluated the performance metrics for each of the three demonstrated case studies. The appraised outcome shows that our projected multimedia module-based information processing paradigm provides efficient system manifestation and is effectively capable of signifying multifaceted intelligent computing systems. [ABSTRACT FROM AUTHOR]

Acupuncture therapy, rooted in traditional Chinese medicine (TCM), plays a pivotal role in both disease treatment and preventive health care. A significant challenge within this realm is precise acupoint recommendations tailored to specific symptoms, with consideration of the intricate inherent relationships between the symptoms and acupoints. Traditional recommendation methods encounter another difficulty in grappling with the sparse nature of TCM data. To address these issues, we present a novel approach called the enhanced graph convolutional network with property fusion (PEGCN), which consists of two key components, the property feature graph encoder module and the enhanced graph convolutional network module. The former extracts property knowledge of acupoints to enrich their representations. The latter integrates the GCN structure and an attention mechanism to efficiently capture the underlying relationships between symptoms and acupoints. In this paper, we apply the PEGCN model to a real-world dataset related to acupuncture therapy, and the experimental results demonstrate its superiority over the baseline models in terms of the evaluation metrics, which include Precision@K, Recall@K, and NDCG@K. This finding suggests that our model effectively addresses the challenges associated with acupoint recommendations, offering an improved method for personalized treatments in the TCM context. [ABSTRACT FROM AUTHOR]

While encryption enhances data security, it also presents significant challenges for network traffic analysis, especially in detecting malicious activities. To tackle this challenge, this paper introduces combined Attention-aware Feature Fusion and Communication Graph Embedding Learning (AFF_CGE), an advanced representation learning framework designed for detecting encrypted malicious traffic. By leveraging an attention mechanism and graph neural networks, AFF_CGE extracts rich semantic information from encrypted traffic and captures complex relations between communicating nodes. Experimental results reveal that AFF_CGE substantially outperforms traditional methods, improving F1-scores by 5.3% through 22.8%. The framework achieves F1-scores ranging from 0.903 to 0.929 across various classifiers, exceeding the performance of state-of-the-art techniques. These results underscore the effectiveness and robustness of AFF_CGE in detecting encrypted malicious traffic, demonstrating its superior performance. [ABSTRACT FROM AUTHOR]

To tackle the challenges of cold start and data sparsity in recommendation systems, an increasing number of researchers are integrating item features, resulting in the emergence of multimodal recommendation systems. Although graph convolutional network-based approaches have achieved significant success, they still face two limitations: (1) Users have different preferences for various types of features, but existing methods often treat these preferences equally or fail to specifically address this issue. (2) They do not effectively distinguish the similarity between different modality item features, overlook the unique characteristics of each type, and fail to fully exploit their complementarity. To solve these issues, we propose the user perception-guided graph convolutional network for multimodal recommendation (UPGCN). This model consists of two main parts: the user perception-guided representation enhancement module (UPEM) and the multimodal two-step enhanced fusion method, which are designed to capture user preferences for different modalities to enhance user representation. At the same time, by distinguishing the similarity between different modalities, the model filters out noise and fully leverages their complementarity to achieve more accurate item representations. We performed comprehensive experiments on the proposed model, and the results indicate that it outperforms other baseline models in recommendation performance, strongly demonstrating its effectiveness. [ABSTRACT FROM AUTHOR]

Knowledge Graph Embedding aims to encode both entities and relations into a continuous low-dimensional vector space, which is crucial for knowledge-driven application scenarios. As abstract entities in knowledge graphs, concepts inherently possess unique hierarchical structures and encompass rich semantic information. Although existing methods for jointly embedding concepts and instances achieve promising performance, they still face two issues: (1) They fail to explicitly reconstruct the hierarchical tree structure of concepts in the embedding space; (2) They ignore disjoint concept pairs and overlapping concept pairs derived from concepts. In this paper, we propose a novel concept representation approach, called Hyper Spherical Cone Concept Embedding (HCCE), to explicitly model the hierarchical tree structure of concepts in the embedding space. Specifically, HCCE represents each concept as a hyperspherical cone and each instance as a vector, maintaining the anisotropy of concept embeddings. We propose two variant methods to explore the impact of embedding concepts and instances in the same or different spaces. Moreover, we design score functions for disjoint concept pairs and overlapping concept pairs, using relative position relations to incorporate them seamlessly into our geometric models. Experimental results on three benchmark datasets show that HCCE outperforms most existing state-of-the-art methods on concept-related triples and achieves competitive results on instance-related triples. The visualization of embedding results intuitively shows the hierarchical tree structure of concepts in the embedding space. [ABSTRACT FROM AUTHOR]

Self-supervised learning has become the cornerstone of building generalizable and transferable artificial intelligence systems in medical imaging. In particular, contrastive representation learning techniques trained on large multi-modal datasets have demonstrated impressive capabilities of producing highly transferable representations for different downstream tasks. In ophthalmology, large multi-modal datasets are abundantly available and conveniently accessible as modern retinal imaging scanners acquire both 2D fundus images and 3D optical coherence tomography (OCT) scans to assess the eye. In this context, we introduce a novel multi-modal contrastive learning-based pipeline to facilitate learning joint representations for the two retinal imaging modalities. After self-supervised pre-training on 153,306 scan pairs, we show that such a pre-training framework can provide both a retrieval system and encoders that produce comprehensive OCT and fundus image representations that generalize well for various downstream tasks on three independent external datasets, explicitly focusing on clinically pertinent prediction tasks. In addition, we show that interchanging OCT with lower-cost fundus imaging can preserve the predictive power of the trained models. [ABSTRACT FROM AUTHOR]

Detecting mastitis on time in dairy cows is crucial for maintaining milk production and preventing significant economic losses, and machine learning has recently gained significant attention as a promising solution to address this issue. Most studies have detected mastitis on time series data using a supervised learning model, which requires the scale of labeled data; however, annotating the onset of mastitis in milking data from dairy cows is very difficult and costly, while supervised learning relies on accurate labels for ensuring the performance. Therefore, this study proposed a mastitis classification based on weakly supervised representation learning using an autoencoder on time series milking data, which allows for concurrent milking representation learning and weakly supervision with low-cost labels. The proposed method employed a structure where the classifier branches from the latent space of a 1D-convolutional autoencoder, enabling representation learning of milking data to be conducted from the perspective of reconstructing the original information and detecting mastitis. The branched classifier backpropagate the mastitis symptoms, which are less costly than mastitis diagnosis, during the encoder's representation learning. The results showed that the proposed method achieved an F1-score of 0.6 that demonstrates performance comparable to previous studies despite using low-cost labels. Our method has the advantage of being easily reproducible across various data domains through low-cost annotation for supervised learning and is practical as it can be implemented with just milking data and weak labels, which can be collected in the field. [ABSTRACT FROM AUTHOR]

Temporal Knowledge Graph Completion (TKGC) aims to infer missing facts in Temporal Knowledge Graphs (TKGs), where facts are stored along with significant temporal information. However, existing TKGC methods only consider message passing on pairwise relations and fail to capture the complex temporal structural dependencies at the levels of time, predicate and entity in TKGs. To fill this gap, we collect high-frequency patterns in TKGs using mathematical statistics and propose a Temporal-Structural Adaptation Network that is equipped with three specific components, time-component, pred-component, and ent-component, as well as one general component, res-component. Concretely, specific components utilize the time consistency pattern to capture facts with significant regularity in time, and complex structural dependencies in TKGs are handled through predicate concurrency and entity collaboration. Moreover, considering low-frequency and nonoccurrence facts, an additional general component is introduced to make predictions on all entities. The outputs of different components are adaptively fused to vote for the final result. Extensive experiments on six benchmarks demonstrate that our method outperforms state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Visual reinforcement learning (RL) enables agents to develop optimal control strategies directly from image data. However, most existing research primarily concentrates on numerical simulations for learning algorithms, often neglecting the challenges encountered in real-world scenarios. To address this gap, this study introduces a semi-real environment that combines MuJoCo Gym simulation with a real camera sensor, aiming to create a more realistic augmented simulation for state-of-the-art visual RL algorithms. The usefulness of this semi-real environment was initially demonstrated through conventional camera-free learning, revealing that general RL experiences substantial performance degradation, especially with fast-moving objects, due to motion blur effects. Building on this semi-real environment, the study also presents the deceleration visual RL (DVRL) algorithm, which incorporates a novel deep learning-based image quality assessment to evaluate the suitability of the acquired data for learning policies. The DVRL algorithm performs real-time image quality assessment and manages fast-moving targets by adjusting their speed, thereby balancing speed and image quality to optimize policy learning and achieve superior performance compared to baseline models. [ABSTRACT FROM AUTHOR]

This paper explores the integration of advanced machine learning (ML) techniques within simulation-based design optimization (SBDO) processes for naval applications, focusing on the hydrodynamic shape optimization of the DTMB 5415 destroyer model. The use of unsupervised learning for design-space dimensionality reduction, combined with supervised learning through active learning-based multi-fidelity surrogate modeling, allows for significant improvements in computational efficiency while addressing complex, high-dimensional design spaces. By applying these ML techniques to both single- and multi-objective optimizations, aimed at minimizing resistance and enhancing seakeeping performance, the proposed framework demonstrates its practical value in hydrodynamic design. This approach provides a scalable and efficient solution, reducing the reliance on high-fidelity simulations while accelerating the optimization process, without substantial modifications to existing toolchains. A design-space dimensionality reduction of approximately 70% is achieved, reducing the design variables from 22 to 7 while retaining 95% of the original geometric variance. Additionally, computational cost reductions of 65% to 98% are observed, compared to using the full design space and high-fidelity simulations only. [ABSTRACT FROM AUTHOR]

With Company2Vec, the paper proposes a novel application in representation learning. The model analyzes business activities from unstructured company website data using Word2Vec and dimensionality reduction. Company2Vec maintains semantic language structures and thus creates efficient company embeddings in fine-granular industries. These semantic embeddings can be used for various applications in banking. Direct relations between companies and words allow semantic business analytics (e.g., top-n words for a company). Furthermore, industry prediction is presented as a supervised learning application and evaluation method. The vectorized structure of the embeddings allows measuring companies' similarities with the cosine distance. Company2Vec hence offers a more fine-grained comparison of companies than the standard industry labels (NACE). This property is relevant for unsupervised learning tasks, such as clustering. An alternative industry segmentation is shown with k-means clustering on the company embeddings. Finally, this paper proposes three algorithms for (1) firm-centric, (2) industry-centric and (3) portfolio-centric peer-firm identification. [ABSTRACT FROM AUTHOR]

This paper focuses on the implementation of the Whisper architecture to create an automatic speech recognition (ASR) system optimized for the Turkish language, which is considered a low-resource language in terms of speech recognition technologies. Whisper is a transformer-based model known for its high performance across numerous languages. However, its performance in Turkish, a language with unique linguistic features and limited labeled data, has yet to be fully explored. To address this, we conducted a series of experiments using five different Turkish speech datasets to assess the model's baseline performance. Initial evaluations revealed a range of word error rates (WERs) between 4.3% and 14.2%, reflecting the challenges posed by Turkish. To improve these results, we applied the low-rank adaptation (LoRA) technique, which is designed to fine-tune large-scale models efficiently by introducing a reduced set of trainable parameters. After fine-tuning, significant performance improvements were observed, with WER reductions of up to 52.38%. This study demonstrates that fine-tuned Whisper models can be successfully adapted for Turkish, resulting in a robust and accurate end-to-end ASR system. This research highlights the applicability of Whisper in low-resource languages and provides insights into the challenges of and strategies for improving speech recognition performance in Turkish. [ABSTRACT FROM AUTHOR]

Rehabilitation training is essential for the recovery of patients with conditions such as stroke and Parkinson’s disease. However, traditional skeletal-based assessments often fail to capture the subtle movement qualities necessary for personalized care and are not optimized for scoring tasks. To address these limitations, we propose a hierarchical contrastive learning framework that integrates multi-view skeletal data, combining both positional and angular joint information. This integration enhances the framework’s ability to detect subtle variations in movement during rehabilitation exercises. In addition, we introduce a novel contrastive loss function specifically designed for regression tasks. This new approach yields substantial improvements over existing state-of-the-art models, achieving over a 30% reduction in mean absolute deviation on both the KIMORE and UIPRMD datasets. The framework demonstrates robustness in capturing both global and local movement characteristics, which are critical for accurate clinical evaluations. By precisely quantifying action quality, the framework supports the development of more targeted, personalized rehabilitation plans and shows strong potential for broad application in rehabilitation practices as well as in a wider range of motion assessment tasks.

Abstract Traditional methods for identifying “hit” molecules from a large collection of potential drug-like candidates rely on biophysical theory to compute approximations to the Gibbs free energy of the binding interaction between the drug and its protein target. These approaches have a significant limitation in that they require exceptional computing capabilities for even relatively small collections of molecules. Increasingly large and complex state-of-the-art deep learning approaches have gained popularity with the promise to improve the productivity of drug design, notorious for its numerous failures. However, as deep learning models increase in their size and complexity, their acceleration at the hardware level becomes more challenging. Hyperdimensional Computing (HDC) has recently gained attention in the computer hardware community due to its algorithmic simplicity relative to deep learning approaches. The HDC learning paradigm, which represents data with high-dimension binary vectors, allows the use of low-precision binary vector arithmetic to create models of the data that can be learned without the need for the gradient-based optimization required in many conventional machine learning and deep learning methods. This algorithmic simplicity allows for acceleration in hardware that has been previously demonstrated in a range of application areas (computer vision, bioinformatics, mass spectrometery, remote sensing, edge devices, etc.). To the best of our knowledge, our work is the first to consider HDC for the task of fast and efficient screening of modern drug-like compound libraries. We also propose the first HDC graph-based encoding methods for molecular data, demonstrating consistent and substantial improvement over previous work. We compare our approaches to alternative approaches on the well-studied MoleculeNet dataset and the recently proposed LIT-PCBA dataset derived from high quality PubChem assays. We demonstrate our methods on multiple target hardware platforms, including Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs), showing at least an order of magnitude improvement in energy efficiency versus even our smallest neural network baseline model with a single hidden layer. Our work thus motivates further investigation into molecular representation learning to develop ultra-efficient pre-screening tools. We make our code publicly available at https://github.com/LLNL/hdbind .

Abstract Graph neural networks (GNNs) have gained prominence as an effective technique for representation learning and have found wide application in tag recommendation tasks. Existing approaches aim to encode the hidden collaborative information among entities into embedding representations by propagating node information between connected nodes. However, in sparse observable graph structures, a significant number of connections are missing, leading to incomplete and biased propagation. To address these issues, we propose a novel model called Low-frequency Spectral Graph Convolution Networks with one-hop connections information for Personalized Tag Recommendation (LSGCNT). This model utilizes graph convolution in the spectral domain and incorporates a graph structure comprising two bipartite graphs, the user–tag interaction graph and the item–tag interaction graph. Our model aims to reduce information loss caused by propagation by utilizing graph convolution networks with trainable convolution kernels to recover preference information. In order to preserve useful low-frequency signals, we couple graph convolution with low-pass filters in the frequency domain. Through reconstructing the true rating tensor and ranking the tag scores within the tensor, we can achieve top-N recommendations. Furthermore, to preserve the one-hop connection information of the bipartite graphs, we treat the observed two bipartite graphs as two homogeneous graphs, where both users and tags contribute to the convolution of a node in the user–tag graph, and both items and tags contribute to the convolution of a node in the item–tag graph. Lastly, we analyze the impact of different internal components, pooling methods, parameter choices, and prediction approaches of LSGCNT on recommendation performance. Experimental results on two real-world datasets demonstrate that LSGCNT achieves superior recommendation performance compared with eight other state-of-the-art recommendation models.

Abstract Self-supervised learning has become the cornerstone of building generalizable and transferable artificial intelligence systems in medical imaging. In particular, contrastive representation learning techniques trained on large multi-modal datasets have demonstrated impressive capabilities of producing highly transferable representations for different downstream tasks. In ophthalmology, large multi-modal datasets are abundantly available and conveniently accessible as modern retinal imaging scanners acquire both 2D fundus images and 3D optical coherence tomography (OCT) scans to assess the eye. In this context, we introduce a novel multi-modal contrastive learning-based pipeline to facilitate learning joint representations for the two retinal imaging modalities. After self-supervised pre-training on 153,306 scan pairs, we show that such a pre-training framework can provide both a retrieval system and encoders that produce comprehensive OCT and fundus image representations that generalize well for various downstream tasks on three independent external datasets, explicitly focusing on clinically pertinent prediction tasks. In addition, we show that interchanging OCT with lower-cost fundus imaging can preserve the predictive power of the trained models.

Deep clustering has gained prominence due to its impressive capability to handle high-dimensional real-world data. However, in the absence of ground-truth labels, existing clustering methods struggle to discern false positives that resemble the target cluster and false negatives that visually differ but maintain semantic consistency. The unreliable projections caused by visual ambiguity disrupt representation learning, leading to sub-optimal clustering outcomes. To address this challenge, we propose a novel method called uncertainty-based learning for deep clustering (ULDC), which aims to discover more optimal cluster structures within data from an uncertainty perspective. Specifically, we utilize the Dirichlet distribution to quantify the uncertainty of feature projections in the latent space, providing a probabilistic framework for modeling uncertainty during the clustering process. We then develop uncertainty-based learning to mitigate the interference caused by false positives and negatives in the clustering tasks. Additionally, a semantic calibration module is introduced to achieve a global alignment of cross-instance semantics, facilitating the learning of clustering-favorite representations. Extensive experiments on five widely-used benchmarks demonstrate the effectiveness of ULDC. The source code is available from . [ABSTRACT FROM AUTHOR]