Search

General matrix multiplication (gemm) is a fundamental kernel in scientific computing and current frameworks for deep learning. Modern realisations of gemm are mostly written in C, on top of a small, highly tuned micro-kernel that is usually encoded in assembly. The high performance realisation of gemm in linear algebra libraries in general include a single micro-kernel per architecture, usually implemented by an expert. In this paper, we explore a couple of paths to automatically generate gemm micro-kernels, either using C++ templates with vector intrinsics or high-level Python scripts that directly produce assembly code. Both solutions can integrate high performance software techniques, such as loop unrolling and software pipelining, accommodate any data type, and easily generate micro-kernels of any requested dimension. The performance of this solution is tested on three ARM-based cores and compared with state-of-the-art libraries for these processors: BLIS, OpenBLAS and ArmPL. The experimental results show that the auto-generation approach is highly competitive, mainly due to the possibility of adapting the micro-kernel to the problem dimensions. [ABSTRACT FROM AUTHOR]

With the continuous progress of technology, the subject of life science plays an increasingly important role, among which the application of artificial intelligence in the medical field has attracted more and more attention. Bell facial palsy, a neurological ailment characterized by facial muscle weakness or paralysis, exerts a profound impact on patients' facial expressions and masticatory abilities, thereby inflicting considerable distress upon their overall quality of life and mental well-being. In this study, we designed a facial attribute recognition model specifically for individuals with Bell's facial palsy. The model utilizes an enhanced SSD network and scientific computing to perform a graded assessment of the patients' condition. By replacing the VGG network with a more efficient backbone, we improved the model's accuracy and significantly reduced its computational burden. The results show that the improved SSD network has an average precision of 87.9% in the classification of light, middle and severe facial palsy, and effectively performs the classification of patients with facial palsy, where scientific calculations also increase the precision of the classification. This is also one of the most significant contributions of this article, which provides intelligent means and objective data for future research on intelligent diagnosis and treatment as well as progressive rehabilitation. [ABSTRACT FROM AUTHOR]

Deep neural networks are extensively used in text mining and Natural Language Processing is to enable computers to understand, analyze, and generate natural language data, such as text or speech, but semantic resources, such as taxonomies and ontologies, are not fully included in deep learning. In this paper, we use Deep Convolutional Neural Network (Deep CNN) to classify research papers using the Computer Science Ontology, an ontology of research areas in the field of computer science. It takes as input the abstract and keywords of a particular research paper and returns the relevant research topic. To evaluate our ontology, we used a gold standard dataset that includes research articles. To further improve text classification results, we propose to design a Deep CNN model. We then used ontology matching to reduce the classes and get better results. Experimental results show that the proposed approach outperforms the one with the highest precision, recall, and F1-score. [ABSTRACT FROM AUTHOR]

Simulating solute transport in heterogeneous porous media poses computational challenges due to the high-resolution meshing required for traditional solvers. To overcome these challenges, this study explores a mesh-free method based on deep learning to accelerate solute transport simulation. We employ Physics-informed Neural Networks (PiNN) with a periodic activation function to solve solute transport problems in both homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that rely on large training datasets, PiNNs use strong-form mathematical models to constrain the network in the training phase and simultaneously solve for multiple dependent or independent field variables, such as pressure and solute concentration fields. To demonstrate the effectiveness of using PiNNs with a periodic activation function to resolve solute transport in porous media, we construct PiNNs using two activation functions, sin and tanh, for seven case studies, including 1D and 2D scenarios. The accuracy of the PiNNs' predictions is then evaluated using absolute point error and mean square error metrics and compared to the ground truth solutions obtained analytically or numerically. Our results demonstrate that the PiNN with sin activation function, compared to tanh activation function, is up to two orders of magnitude more accurate and up to two times faster to train, especially in heterogeneous porous media. Moreover, PiNN's simultaneous predictions of pressure and concentration fields can reduce computational expenses in terms of inference time by three orders of magnitude compared to FEM simulations for two-dimensional cases. [ABSTRACT FROM AUTHOR]

Multiscale is a hallmark feature of complex systems, presenting challenges for traditional numerical methods due to their reliance on local Taylor series constraints. Further, multiscale techniques often face limitations in identifying appropriate heuristic closures used to handle unresolved scales or processes within the system being modeled. In this study, we develop an efficient time-stepping strategy for multiscale problems. The proposed method's novelty lies in synergizing the hierarchical deep learning formalism with an adaptive step-size estimation process to approximate dynamical system flow maps across timescales efficiently. The model is purely data-driven and provides improved forecasting accuracy compared to fixed-step neural network time steppers. To demonstrate the effectiveness of the proposed scheme, we provide numerical simulations for several canonical nonlinear systems and source codes for their implementation. We believe the method has the potential to benefit multiscale analysis of complex systems and encourage further investigation in this area. • Identifies potential drawbacks in the current multiscale methods. • Introduces efficient time-stepping algorithm for multiscale problems using hierarchical deep learning. • Demonstrates the application of adaptive time-stepping on canonical ODEs and PDEs with noisy measurements. [ABSTRACT FROM AUTHOR]

Multiscale is a hallmark feature of complex systems, presenting challenges for traditional numerical methods due to their reliance on local Taylor series constraints. Further, multiscale techniques often face limitations in identifying appropriate heuristic closures used to handle unresolved scales or processes within the system being modeled. In this study, we develop an efficient time-stepping strategy for multiscale problems. The proposed method's novelty lies in synergizing the hierarchical deep learning formalism with an adaptive step-size estimation process to approximate dynamical system flow maps across timescales efficiently. The model is purely data-driven and provides improved forecasting accuracy compared to fixed-step neural network time steppers. To demonstrate the effectiveness of the proposed scheme, we provide numerical simulations for several canonical nonlinear systems and source codes for their implementation. We believe the method has the potential to benefit multiscale analysis of complex systems and encourage further investigation in this area. • Identifies potential drawbacks in the current multiscale methods. • Introduces efficient time-stepping algorithm for multiscale problems using hierarchical deep learning. • Demonstrates the application of adaptive time-stepping on canonical ODEs and PDEs with noisy measurements. [ABSTRACT FROM AUTHOR]

"Deep Learning and Scientific Computing with R torch" by Sigrid Keydana is a book that introduces an R interface for PyTorch, a popular deep learning framework. The book is well-written and easy to follow, with plenty of illustrations and examples. It is divided into three parts: the basics of torch, the implementation of deep neural networks, and the utility of torch beyond deep learning. The book covers topics such as tensors, automatic differentiation, neural network architecture, optimization algorithms, and various applications of torch. Overall, it is a valuable resource for R users interested in deep learning and scientific computing. [Extracted from the article]

The article introduces a new section structure for the SIAM Journal on Scientific Computing (SISC), including a focus on Scientific Machine Learning as a major development in the field of scientific computing. Topics include the development of accurate, efficient, and scalable machine learning algorithms for solving mathematical modeling problems, emphasizing the importance of reproducibility in the review process.

Deep neural networks have been shown to provide accurate function approximations in high dimensions. However, fitting network parameters requires informative training data that are often challenging to collect in science and engineering applications. This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time, which enables adaptively collecting new training data in a self-informed manner that is guided by the dynamics described by the partial differential equations. This is in contrast to other machine learning methods that aim to fit network parameters globally in time without taking into account training data acquisition. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions. Numerical experiments demonstrate that Neural Galerkin schemes have the potential to enable simulating phenomena and processes with many variables for which traditional and other deep-learning-based solvers fail, especially when features of the solutions evolve locally such as in high-dimensional wave propagation problems and interacting particle systems described by Fokker-Planck and kinetic equations. • Coupling training of deep networks with adaptive data acquisition to solve high-dimensional evolution equations. • Enables predicting local features in high dimensions for which training with classical, uniform data acquisition fails. • Sequentially collects data over time and so avoids requirement of having rich initial data set over the time-space domain. • Suited for transport problems, for which collocation-based deep-network methods with uniform data sampling are inefficient. [ABSTRACT FROM AUTHOR]