Search

We present ExaDigiT, an open-source framework for developing comprehensive digital twins of liquid-cooled supercomputers. It integrates three main modules: (1) a resource allocator and power simulator, (2) a transient thermo-fluidic cooling model, and (3) an augmented reality model of the supercomputer and central energy plant. The framework enables the study of "what-if" scenarios, system optimizations, and virtual prototyping of future systems. Using Frontier as a case study, we demonstrate the framework's capabilities by replaying six months of system telemetry for systematic verification and validation. Such a comprehensive analysis of a liquid-cooled exascale supercomputer is the first of its kind. ExaDigiT elucidates complex transient cooling system dynamics, runs synthetic or real workloads, and predicts energy losses due to rectification and voltage conversion. Throughout our paper, we present lessons learned to benefit HPC practitioners developing similar digital twins. We envision the digital twin will be a key enabler for sustainable, energy-efficient supercomputing., Comment: 14 pages, 9 figures, To be published in the Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2024

In a post-ChatGPT world, this paper explores the potential of leveraging scalable artificial intelligence for scientific discovery. We propose that scaling up artificial intelligence on high-performance computing platforms is essential to address such complex problems. This perspective focuses on scientific use cases like cognitive simulations, large language models for scientific inquiry, medical image analysis, and physics-informed approaches. The study outlines the methodologies needed to address such challenges at scale on supercomputers or the cloud and provides exemplars of such approaches applied to solve a variety of scientific problems., Comment: 17 pages, 5 figures

We develop time-series machine learning (ML) methods for closure modeling of the Unsteady Reynolds Averaged Navier Stokes (URANS) equations applied to stably stratified turbulence (SST). SST is strongly affected by fine balances between forces and becomes more anisotropic in time for decaying cases. Moreover, there is a limited understanding of the physical phenomena described by some of the terms in the URANS equations. Rather than attempting to model each term separately, it is attractive to explore the capability of machine learning to model groups of terms, i.e., to directly model the force balances. We consider decaying SST which are homogeneous and stably stratified by a uniform density gradient, enabling dimensionality reduction. We consider two time-series ML models: Long Short-Term Memory (LSTM) and Neural Ordinary Differential Equation (NODE). Both models perform accurately and are numerically stable in a posteriori tests. Furthermore, we explore the data requirements of the ML models by extracting physically relevant timescales of the complex system. We find that the ratio of the timescales of the minimum information required by the ML models to accurately capture the dynamics of the SST corresponds to the Reynolds number of the flow. The current framework provides the backbone to explore the capability of such models to capture the dynamics of higher-dimensional complex SST flows.

The applicability of computational fluid dynamics (CFD) based design tools depend on the accuracy and complexity of the physical models, for example turbulence models, which remains an unsolved problem in physics, and rotor models that dictates the computational cost of rotorcraft and wind/hydro turbine farm simulations. The research focuses on investigation of the ability of neural networks to learn correlation between desired modeling variables and flow parameters, thereby providing surrogate models. For the turbulence modeling, the machine learned turbulence model is developed for unsteady boundary layer flow, and the predictions are validated against DNS data and compared with one-equation unsteady Reynolds Averaged NavierStokes (URANS) predictions. The machine-learned model performs much better than the URANS model due to its ability to incorporate the non-linear correlation between turbulent stresses and rate-of-strain. The development of the surrogate rotor model builds on the hypothesis that if a model can mimic the axial and tangential momentum deficit generated by a blade resolved model, then it should produce a qualitatively and quantitatively similar wake recovery. An initial validation of the hypothesis was performed, which showed encouraging results.

A machine-learned (ML) model is developed to enhance the accuracy of turbulence transport equations of Reynolds Averaged Navier Stokes (RANS) solver and applied for periodic hill test case, which involves complex flow regimes, such as attached boundary layer, shear-layer, and separation and reattachment. The accuracy of the model is investigated in extrapolation modes, i.e., the test case has much larger separation bubble and higher turbulence than the training cases. A parametric study is also performed to understand the effect of network hyperparameters on training and model accuracy and to quantify the uncertainty in model accuracy due to the non-deterministic nature of the neural network training. The study revealed that, for any network, less than optimal mini-batch size results in overfitting, and larger than optimal batch size reduces accuracy. Data clustering is found to be an efficient approach to prevent the machine-learned model from over-training on more prevalent flow regimes, and results in a model with similar accuracy using almost one-third of the training dataset. Feature importance analysis reveals that turbulence production is correlated with shear strain in the free-shear region, with shear strain and wall-distance and local velocity-based Reynolds number in the boundary layer regime, and with streamwise velocity gradient in the accelerating flow regime. The flow direction is found to be key in identifying flow separation and reattachment regime. Machine-learned models perform poorly in extrapolation mode, wherein the prediction shows less than 10% correlation with Direct Numerical Simulation (DNS). A priori tests reveal that model predictability improves significantly as the hill dataset is partially added during training in a partial extrapolation model, e.g., with the addition of only 5% of the hill data increases correlation with DNS to 80%., Comment: 50 pages, 18 figures