Search

Physics-informed neural networks (PINNs) have shown remarkable prospects in solving forward and inverse problems involving partial differential equations (PDEs). However, PINNs still face the challenge of high computational cost in solving strongly nonlinear PDEs involving fluid dynamics. In this study, inspired by the input design in surrogate modeling, we propose a feature-enhanced neural network. By introducing geometric features including distance and angle or physical features including the solution of the potential flow equation in the inputs of PINNs, FENN can more easily learn the flow, resulting in better performance in terms of both accuracy and efficiency. We establish the feature networks in advance to avoid the invalid PDE loss in FENN caused by neglecting the partial derivatives of the features with respect to space-time coordinates. Through five numerical experiments involving forward, inverse, and parametric problems, we verify that FENN generally reduces the computational cost of PINNs by approximately four times. In addition, the numerical experiments also demonstrate that the proposed method can reduce the number of observed data for inverse problem and successfully solve the parametric problem where PINNs fail., Comment: 24 pages, 20 figures

Recent years have witnessed increasing research interests of physics-informed neural networks (PINNs) in solving forward, inverse, and parametric problems governed by partial differential equations (PDEs). Despite their promise, PINNs still face significant challenges in many scenarios due to ill-conditioning. Time-stepping-oriented neural network (TSONN) addresses this by reformulating the ill-conditioned optimization problem into a series of well-conditioned sub-problems, greatly improving its ability to handle complex scenarios. This paper presents a new solver for laminar flow around airfoils based on TSONN and mesh transformation, validated across various test cases. Specifically, the solver achieves mean relative errors of approximately 3.6% for lift coefficients and 1.4% for drag coefficients. Furthermore, this paper extends the solver to parametric problems involving flow conditions and airfoil shapes, covering nearly all laminar flow scenarios in engineering. The shape parameter space is defined as the union of 30% perturbations applied to each airfoil in the UIUC airfoil database, with Reynolds numbers ranging from 100 to 5000 and angles of attack spanning from -5{\deg} to 15{\deg}. The parametric solver solves all laminar flows within the parameter space in just 4.6 day, at approximately 40 times the computational cost of solving a single flow. The model training involves hundreds of millions of flow conditions and airfoil shapes, ultimately yielding a surrogate model with strong generalization capability that does not require labeled data. Specifically, the surrogate model achieves average errors of 4.6% for lift coefficients and 1.1% for drag coefficients, demonstrating its potential for high generalizability, cost-effectiveness, and efficiency in addressing high-dimensional parametric problems and surrogate modeling.

We uncover a phenomenon largely overlooked by the scientific community utilizing AI: neural networks exhibit high susceptibility to minute perturbations, resulting in significant deviations in their outputs. Through an analysis of five diverse application areas -- weather forecasting, chemical energy and force calculations, fluid dynamics, quantum chromodynamics, and wireless communication -- we demonstrate that this vulnerability is a broad and general characteristic of AI systems. This revelation exposes a hidden risk in relying on neural networks for essential scientific computations, calling further studies on their reliability and security., Comment: 26 pages, 6 figures

Selfie taking is a popular social pastime, and is an important part of socialising online. This activity is popular with young people but is also becoming more prevalent with older generations. Despite this, there are a number of accessibility issues when taking selfies. In this research, we investigate preferences from elderly citizens when taking a selfie, to understand the current challenges. As a potential solution to address the challenges identified, we propose the use of drones and present a novel concept for hands free selfie taking. With this work, we hope to trigger conversation around how such a technology can be utilised to enable elderly citizens, and more broadly people with physical disabilities, the ability to easily take part in this social pastime., Comment: CHI2020: Workshop of the CHI Conference on Human Factors in Computing Systems, April 25-30, 2020, Honolulu, HI, USA

Immigration and aging have always been significant topics of discussion in society, concerning the stability and future development of a country and its people. Research in the field of HCI on immigration and aging has primarily focused on their practical needs but has paid less attention to the adaptability issues of older internal migrants moving with their families. In this study, we investigate the challenges older internal migrants face in adapting socially, using metadata surveys and semi-structured interviews to delve into their life struggles and resilience sources. Our findings highlight the older internal migrants' remarkable resilience, particularly evident in their reminiscences. We explore the integration of reminiscences with the metaverse, identifying the necessary conditions to create a "Metamemory". We introduce a novel design for a metaverse scene that bridges past and present experiences. This aims to encourage discussions on enhancing older internal migrants' reminiscence, leveraging the metaverse's positive potential, and devising strategies to more effectively address older internal migrants' concerns in the future.

In recent years, Physics-Informed Neural Networks (PINNs) have become a representative method for solving partial differential equations (PDEs) with neural networks. PINNs provide a novel approach to solving PDEs through optimization algorithms, offering a unified framework for solving both forward and inverse problems. However, some limitations in terms of solution accuracy and generality have also been revealed. This paper systematically summarizes the limitations of PINNs and identifies three root causes for their failure in solving PDEs: (1) Poor multiscale approximation ability and ill-conditioning caused by PDE losses; (2) Insufficient exploration of convergence and error analysis, resulting in weak mathematical rigor; (3) Inadequate integration of physical information, causing mismatch between residuals and iteration errors. By focusing on addressing these limitations in PINNs, we outline the future directions and prospects for the intelligent computing of PDEs: (1) Analysis of ill-conditioning in PINNs and mitigation strategies; (2) Improvements to PINNs by enforcing temporal causality; (3) Empowering PINNs with classical numerical methods., Comment: 30 pages, 9 figures, 3 tables

Surrogate model can replace the parametric full-order model (FOM) by an approximation model, which can significantly improve the efficiency of optimization design and reduce the complexity of engineering systems. However, due to limitations in efficiency and accuracy, the applications of high-dimensional surrogate models are still challenging. In the present study, we propose a method for extracting hidden features to simplify high-dimensional problems, thereby improving the accuracy and robustness of surrogate models. We establish a goal-oriented feature extraction (GFE) neural network through indirect supervised learning. We constrained the distance between hidden features based on the differences in the target output. This means that in the hidden feature space, cases that are closer in distance output approximately the same, and vice versa. The proposed hidden feature learning method can significantly reduce the dimensionality and nonlinearity of the surrogate model, thereby improving modeling accuracy and generalization capability. To demonstrate the efficiency of our proposed ideas, We conducted numerical experiments on three popular surrogate models. The modeling results of typical high-dimensional mathematical cases and aerodynamic performance cases of ONERA M6 wings show that goal-oriented feature extraction significantly improves the modeling accuracy. Goal-oriented feature extraction can effectively reduce the error distribution of predicting cases and reduce the convergence and robustness differences caused by various data-driven surrogate models.

Accurate and complete aerodynamic data sets are the basis for comprehensive and accurate evaluation of the overall performance of aircraft. However, the sampling cost of full-state aerodynamic data is extremely high, and there are often differences between wind tunnel conditions and actual flight conditions. Conventional scaling parameter extraction methods can solve the problem of aerodynamic state extrapolation, but hard to achieve data migration and shape generalization. In order to realize the low-cost construction of a full-state nonlinear aerodynamic database, this research proposes the Scaling Function Learning (SFL) method. In SFL method, symbolic regression is used to mine the composite function expression of aerodynamic force coefficient for a relatively complete aerodynamic data set of typical aircraft. The inner layer of the function represents a scaling function. The SFL method was validated on the HB-2 by extracting scaling parameters for axial force coefficients and generalizing the scaling function by releasing its constants. The effectiveness and accuracy of the scaling function are verified using different hypersonic aircraft configurations, such as HBS, double ellipsoid, sharp cone, and double cone missile. The results show that the extracted scaling function has the ability to generalize across states and configurations. With only 3-4 state samples, the aerodynamic database construction of variable Mach number, angle of attack and Reynolds number can be realized, which shows great state extrapolation ability with a relative error of about 1-5%. This research also lays a methodological foundation for parameter space dimensionality reduction and small sample modeling of other complex high-dimensional engineering problems., Comment: This paper needs to be retracted due to methodological flaws found in Section 2. After rigorous reexamination, equations (1) and (2) in Section 2 contain errors in their statements, which fundamentally undermine the validity of the conclusions

The increasing use of smart devices has emphasized the critical role of maintenance in production activities. Interactive Electronic Technical Manuals (IETMs) are vital tools that support the maintenance of smart equipment. However, traditional IETMs face challenges such as transitioning from Graphical User Interfaces (GUIs) to natural Language User Interfaces (LUIs) and managing complex logical relationships. Additionally, they must meet the current demands for higher intelligence. This paper proposes a Maintenance Scheme Generation Method based on Large Language Models (LLM-R). The proposed method includes several key innovations: We propose the Low Rank Adaptation-Knowledge Retention (LORA-KR) loss technology to proportionally adjust mixed maintenance data for fine-tuning the LLM. This method prevents knowledge conflicts caused by mixed data, improving the model's adaptability and reasoning ability in specific maintenance domains, Besides, Hierarchical Task-Based Agent and Instruction-level Retrieval-Augmented Generation (RAG) technologies are adopted to optimize the generation steps and mitigate the phenomenon of hallucination caused by the model's Inability to access contextual information. This enhancement improves the model's flexibility and accuracy in handling known or unknown maintenance objects and maintenance scheme scenarios. To validate the proposed method's effectiveness in maintenance tasks, a maintenance scheme dataset was constructed using objects from different fields. The experimental results show that the accuracy of the maintenance schemes generated by the proposed method reached 91.59%, indicating which improvement enhances the intelligence of maintenance schemes and introduces novel technical approaches for equipment maintenance., Comment: 30 pages, 7 figures

Iterative methods are widely used for solving partial differential equations (PDEs). However, the difficulty in eliminating global low-frequency errors significantly limits their convergence speed. In recent years, neural networks have emerged as a novel approach for solving PDEs, with studies revealing that they exhibit faster convergence for low-frequency components. Building on this complementary frequency convergence characteristics of iterative methods and neural networks, we draw inspiration from multigrid methods and propose a hybrid solving framework that combining iterative methods and neural network-based solvers, termed PINN-MG (PMG). In this framework, the iterative method is responsible for eliminating local high-frequency oscillation errors, while Physics-Informed Neural Networks (PINNs) are employed to correct global low-frequency errors. Throughout the solving process, high- and low-frequency components alternately dominate the error, with each being addressed by the iterative method and PINNs respectively, thereby accelerating the convergence. We tested the proposed PMG framework on the linear Poisson equation and the nonlinear Helmholtz equation, and the results demonstrated significant acceleration of the PMG when built on Gauss-Seidel, pseudo-time, and GMRES methods. Furthermore, detailed analysis of the convergence process further validates the rationality of the framework. We proposed that the PMG framework is a hybrid solving approach that does not rely on training data, achieving an organic integration of neural network methods with iterative methods., Comment: 29 pages, 22figures

Neural-based multi-task learning (MTL) has been successfully applied to many recommendation applications. However, these MTL models (e.g., MMoE, PLE) did not consider feature interaction during the optimization, which is crucial for capturing complex high-order features and has been widely used in ranking models for real-world recommender systems. Moreover, through feature importance analysis across various tasks in MTL, we have observed an interesting divergence phenomenon that the same feature can have significantly different importance across different tasks in MTL. To address these issues, we propose Deep Multiple Task-specific Feature Interactions Network (DTN) with a novel model structure design. DTN introduces multiple diversified task-specific feature interaction methods and task-sensitive network in MTL networks, enabling the model to learn task-specific diversified feature interaction representations, which improves the efficiency of joint representation learning in a general setup. We applied DTN to our company's real-world E-commerce recommendation dataset, which consisted of over 6.3 billion samples, the results demonstrated that DTN significantly outperformed state-of-the-art MTL models. Moreover, during online evaluation of DTN in a large-scale E-commerce recommender system, we observed a 3.28% in clicks, a 3.10% increase in orders and a 2.70% increase in GMV (Gross Merchandise Value) compared to the state-of-the-art MTL models. Finally, extensive offline experiments conducted on public benchmark datasets demonstrate that DTN can be applied to various scenarios beyond recommendations, enhancing the performance of ranking models.

Prognosis and Health Management (PHM), critical for ensuring task completion by complex systems and preventing unexpected failures, is widely adopted in aerospace, manufacturing, maritime, rail, energy, etc. However, PHM's development is constrained by bottlenecks like generalization, interpretation and verification abilities. Presently, generative artificial intelligence (AI), represented by Large Model, heralds a technological revolution with the potential to fundamentally reshape traditional technological fields and human production methods. Its capabilities, including strong generalization, reasoning, and generative attributes, present opportunities to address PHM's bottlenecks. To this end, based on a systematic analysis of the current challenges and bottlenecks in PHM, as well as the research status and advantages of Large Model, we propose a novel concept and three progressive paradigms of Prognosis and Health Management Large Model (PHM-LM) through the integration of the Large Model with PHM. Subsequently, we provide feasible technical approaches for PHM-LM to bolster PHM's core capabilities within the framework of the three paradigms. Moreover, to address core issues confronting PHM, we discuss a series of technical challenges of PHM-LM throughout the entire process of construction and application. This comprehensive effort offers a holistic PHM-LM technical framework, and provides avenues for new PHM technologies, methodologies, tools, platforms and applications, which also potentially innovates design, research & development, verification and application mode of PHM. And furthermore, a new generation of PHM with AI will also capably be realized, i.e., from custom to generalized, from discriminative to generative, and from theoretical conditions to practical applications.

With the development of high performance computer and experimental technology, the study of turbulence has accumulated a large number of high fidelity data. However, few general turbulence knowledge has been found from the data. So we use the symbolic regression (SR) method to find a new mixing length formula which is generally valid in wall-bounded turbulence, and this formula has physical interpretation that it has correct asymptotic relationships in viscous sublayer,buffer layer, log-law region and outer region. Coupled with Reynolds averaged Navier-Stokes (RANS) solver, we test several classic cases. The prediction results fully demonstrate the accuracy and generalization of the formula. So far, we have found that SR method can help us find general laws from complex turbulent systems, and it is expected that through this 'white box' machine learning method, more turbulence knowledge with physical interpretation can be found in the future., Comment: 13 pages, 12 figures

While in-context Learning (ICL) has proven to be an effective technique to improve the performance of Large Language Models (LLMs) in a variety of complex tasks, notably in translating natural language questions into Structured Query Language (NL2SQL), the question of how to select the most beneficial demonstration examples remains an open research problem. While prior works often adapted off-the-shelf encoders to retrieve examples dynamically, an inherent discrepancy exists in the representational capacities between the external retrievers and the LLMs. Further, optimizing the selection of examples is a non-trivial task, since there are no straightforward methods to assess the relative benefits of examples without performing pairwise inference. To address these shortcomings, we propose DeTriever, a novel demonstration retrieval framework that learns a weighted combination of LLM hidden states, where rich semantic information is encoded. To train the model, we propose a proxy score that estimates the relative benefits of examples based on the similarities between output queries. Experiments on two popular NL2SQL benchmarks demonstrate that our method significantly outperforms the state-of-the-art baselines on one-shot NL2SQL tasks.

Federated learning (FL) based on the centralized design faces both challenges regarding the trust issue and a single point of failure. To alleviate these issues, blockchain-aided decentralized FL (BDFL) introduces the decentralized network architecture into the FL training process, which can effectively overcome the defects of centralized architecture. However, deploying BDFL in wireless networks usually encounters challenges such as limited bandwidth, computing power, and energy consumption. Driven by these considerations, a dynamic stochastic optimization problem is formulated to minimize the average training delay by jointly optimizing the resource allocation and client selection under the constraints of limited energy budget and client participation. We solve the long-term mixed integer non-linear programming problem by employing the tool of Lyapunov optimization and thereby propose the dynamic resource allocation and client scheduling BDFL (DRC-BDFL) algorithm. Furthermore, we analyze the learning performance of DRC-BDFL and derive an upper bound for convergence regarding the global loss function. Extensive experiments conducted on SVHN and CIFAR-10 datasets demonstrate that DRC-BDFL achieves comparable accuracy to baseline algorithms while significantly reducing the training delay by 9.24% and 12.47%, respectively., Comment: 14 pages, 4 figures

Artificial intelligence techniques are considered an effective means to accelerate flow field simulations. However, current deep learning methods struggle to achieve generalization to flow field resolutions while ensuring computational efficiency. This paper presents a deep learning approach for rapid prediction of two types of subsonic flow fields with different resolutions. Unlike convolutional neural networks, the constructed feature extractor integrates features of different spatial scales along the channel dimension, reducing the sensitivity of the deep learning model to resolution while improving computational efficiency. Additionally, to ensure consistency between the input and output resolutions of the deep learning model, a memory pooling strategy is proposed, which ensures accurate reconstruction of flow fields at any resolution. By conducting extensive qualitative and quantitative analyses on a given test dataset, it is demonstrated that the proposed deep learning model can achieve a three-order-of-magnitude speedup compared to CPU-based solvers while adapting to flow fields of arbitrary resolutions. Moreover, the prediction accuracy for pressure exceeds 99\%, laying the foundation for the development of large-scale models in the field of aerodynamics.

As an emerging artificial intelligence technology, graph neural networks (GNNs) have exhibited promising performance across a wide range of graph-related applications. However, information exchanges among neighbor nodes in GNN pose new challenges in the resource-constrained scenario, especially in wireless systems. In practical wireless systems, the communication links among nodes are usually unreliable due to wireless fading and receiver noise, consequently resulting in performance degradation of GNNs. To improve the learning performance of GNNs, we aim to maximize the number of long-term average (LTA) communication links by the optimized power control under energy consumption constraints. Using the Lyapunov optimization method, we first transform the intractable long-term problem into a deterministic problem in each time slot by converting the long-term energy constraints into the objective function. In spite of this non-convex combinatorial optimization problem, we address this problem via equivalently solving a sequence of convex feasibility problems together with a greedy based solver. Simulation results demonstrate the superiority of our proposed scheme over the baselines., Comment: 5 pages,3 figures

In light of the challenges surrounding convergence and error propagation encountered in Reynolds-averaged Navier-Stokes (RANS) equations with data-driven Reynolds stress closures, researchers commonly attribute these issues to ill-conditioning through conditional number analysis. This paper delves into an additional factor, numerical instability, contributing to these challenges. We conduct global stability analysis for the RANS equations, closed by the Reynolds stress of direct numerical simulation (DNS), with the time-averaged solution of DNS as the base flow. Our findings reveal that, for turbulent channel flow at high Reynolds numbers, significant ill-conditioning exists, yet the system remains stable. Conversely, for separated flow over periodic hills, notable ill-conditioning is absent, but unstable eigenvalues are present, indicating that error propagation arises from the mechanism of numerical instability. Furthermore, the effectiveness of the decomposition method employing eddy viscosity is compared, results show that the spatial distribution and amplitude of eddy viscosity influences the numerical stability., Comment: 11 pages, 5 figures

Physics-informed neural networks (PINNs) have recently emerged as a novel and popular approach for solving forward and inverse problems involving partial differential equations (PDEs). However, achieving stable training and obtaining correct results remain a challenge in many cases, often attributed to the ill-conditioning of PINNs. Nonetheless, further analysis is still lacking, severely limiting the progress and applications of PINNs in complex engineering problems. Drawing inspiration from the ill-conditioning analysis in traditional numerical methods, we establish a connection between the ill-conditioning of PINNs and the ill-conditioning of the Jacobian matrix of the PDE system. Specifically, for any given PDE system, we construct its controlled system. This controlled system allows for adjustment of the condition number of the Jacobian matrix while retaining the same solution as the original system. Our numerical findings suggest that the ill-conditioning observed in PINNs predominantly stems from the Jacobian matrix. As the condition number of the Jacobian matrix decreases, PINNs exhibit faster convergence rates and higher accuracy. Building upon this understanding and the natural extension of controlled systems, we present a general approach to mitigate the ill-conditioning of PINNs, leading to successful simulations of the three-dimensional flow around the M6 wing at a Reynolds number of 5,000. To the best of our knowledge, this is the first time that PINNs have been successful in simulating such complex systems, offering a promising new technique for addressing industrial complexity problems. Our findings also offer valuable insights guiding the future development of PINNs.