Search

Metamorphic testing has become one mainstream technique to address the notorious oracle problem in software testing, thanks to its great successes in revealing real-life bugs in a wide variety of software systems. Metamorphic relations, the core component of metamorphic testing, have continuously attracted research interests from both academia and industry. In the last decade, a rapidly increasing number of studies have been conducted to systematically generate metamorphic relations from various sources and for different application domains. In this article, based on the systematic review on the state of the art for metamorphic relations' generation, we summarize and highlight visions for further advancing the theory and techniques for identifying and constructing metamorphic relations, and discuss potential research trends in related areas., Comment: Accepted by International Workshop on Software Engineering in 2030

While ChatGPT is a well-known artificial intelligence chatbot being used to answer human's questions, one may want to discover its potential in advancing software testing. We examine the capability of ChatGPT in advancing the intelligence of software testing through a case study on metamorphic testing (MT), a state-of-the-art software testing technique. We ask ChatGPT to generate candidates of metamorphic relations (MRs), which are basically necessary properties of the object program and which traditionally require human intelligence to identify. These MR candidates are then evaluated in terms of correctness by domain experts. We show that ChatGPT can be used to generate new correct MRs to test several software systems. Having said that, the majority of MR candidates are either defined vaguely or incorrect, especially for systems that have never been tested with MT. ChatGPT can be used to advance software testing intelligence by proposing MR candidates that can be later adopted for implementing tests; but human intelligence should still inevitably be involved to justify and rectify their correctness., Comment: 4 pages (short communications), 2 figures, 2 tables

Advancing ocean science has a significant impact to the development of the world, from operating a safe navigation for vessels to maintaining a healthy and diverse ocean ecosystem. Various ocean software systems have been extensively adopted for different purposes, for instance, predicting hourly sea level elevation across shorelines, simulating large-scale ocean circulations, as well as integrating into Earth system models for weather forecasts and climate projections. Regardless of their significance, guaranteeing the trustworthiness of ocean software and modelling systems is a long-standing challenge. The testing of ocean software suffers a lot from the so-called oracle problem, which refers to the absence of test oracles mainly due to the nonlinear interactions of multiple physical variables and the high complexity in computation. In the ocean, observed tidal signals are distorted by non-deterministic physical variables, hindering us from knowing the "true" astronomical tidal constituents existing in the timeseries. In this paper, we present how to test tidal analysis and prediction (TAP) software based on metamorphic testing (MT), a simple yet effective testing approach to the oracle problem. In particular, we construct metamorphic relations from the periodic property of astronomical tide, and then use them to successfully detect a real-life defect in an open-source TAP software. We also conduct a series of experiments to further demonstrate the applicability and effectiveness of MT in the testing of TAP software. Our study not only justifies the potential of MT in testing more complex ocean software and modelling systems, but also can be expanded to assess and improve the quality of a broader range of scientific simulation software systems., Comment: 7 pages, 3 tables

Automated driving systems (ADS) are expected to be reliable and robust against a wide range of driving scenarios. Their decisions, first and foremost, must be well understood. Understanding a decision made by ADS is a great challenge, because it is not straightforward to tell whether the decision is correct or not, and how to verify it systematically. In this paper, a Sequential MetAmoRphic Testing Smart framework is proposed based on metamorphic testing, a mainstream software testing approach. In metamorphic testing, metamorphic groups are constructed by selecting multiple inputs according to the so-called metamorphic relations, which are basically the system's necessary properties; the violation of certain relations by some corresponding metamorphic groups implies the detection of erroneous system behaviors. The proposed framework makes use of sequences of metamorphic groups to understand ADS behaviors, and is applicable without the need of ground-truth datasets. To demonstrate its effectiveness, the framework is applied to test three ADS models that steer an autonomous car in different scenarios with another car either leading in front or approaching in the opposite direction. The conducted experiments reveal a large number of undesirable behaviors in these top-ranked deep learning models in the scenarios. These counter-intuitive behaviors are associated with how the core models of ADS respond to different positions, directions and properties of the other car in its proximity. Further analysis of the results helps identify critical factors affecting ADS decisions and thus demonstrates that the framework can be used to provide a comprehensive understanding of ADS before their deployment, Comment: 11 pages, 6 figures, 3 tables

Understanding the thermodynamic phase transition of black holes can provide the deep insight into the fundamental properties of black hole gravity to establish the quantum gravity. In this work, we investigate the phase transition and its dynamics for the charged EPYM AdS black hole. Through reconstructing the Maxwell's equal-area law, we find there exists the high-/low-potential black hole (HPBH/LPBL) phase transition, not only the pure large/small one. The Gibbs free energy landscape ($G_L$) is treated as a function of the black hole horizon, which is the order parameter of the phase transition due to the thermal fluctuation. From the view point of $G_L$, the stable HPBH/LPBL states are corresponding to two wells of $G_L$, which have the same depth. The unstable intermediate-potential black hole state corresponds to the local maximum of $G_L$. Then we focus on the probability evolution governed by the Fokker-Planck equation. Through solving the Fokker-Planck equating with different reflection/aborption boundary conditions and initial conditions, the dynamics of switching between the coexistent HPBH and LPBL phases is probed within the first passage time. Furthermore, the effect of temperature on the dynamic property of phase transition is also investigated., Comment: arXiv admin note: substantial text overlap with arXiv:2104.10309

Deep learning is being used extensively in a variety of software engineering tasks, e.g., program classification and defect prediction. Although the technique eliminates the required process of feature engineering, the construction of source code model significantly affects the performance on those tasks. Most recent works was mainly focused on complementing AST-based source code models by introducing contextual dependencies extracted from CFG. However, all of them pay little attention to the representation of basic blocks, which are the basis of contextual dependencies. In this paper, we integrated AST and CFG and proposed a novel source code model embedded with hierarchical dependencies. Based on that, we also designed a neural network that depends on the graph attention mechanism.Specifically, we introduced the syntactic structural of the basic block, i.e., its corresponding AST, in source code model to provide sufficient information and fill the gap. We have evaluated this model on three practical software engineering tasks and compared it with other state-of-the-art methods. The results show that our model can significantly improve the performance. For example, compared to the best performing baseline, our model reduces the scale of parameters by 50\% and achieves 4\% improvement on accuracy on program classification task., Comment: 17pages, published on Journal of Systems and Software

Autonomous driving has gained much attention from both industry and academia. Currently, Deep Neural Networks (DNNs) are widely used for perception and control in autonomous driving. However, several fatal accidents caused by autonomous vehicles have raised serious safety concerns about autonomous driving models. Some recent studies have successfully used the metamorphic testing technique to detect thousands of potential issues in some popularly used autonomous driving models. However, prior study is limited to a small set of metamorphic relations, which do not reflect rich, real-world traffic scenarios and are also not customizable. This paper presents a novel declarative rule-based metamorphic testing framework called RMT. RMT provides a rule template with natural language syntax, allowing users to flexibly specify an enriched set of testing scenarios based on real-world traffic rules and domain knowledge. RMT automatically parses human-written rules to metamorphic relations using an NLP-based rule parser referring to an ontology list and generates test cases with a variety of image transformation engines. We evaluated RMT on three autonomous driving models. With an enriched set of metamorphic relations, RMT detected a significant number of abnormal model predictions that were not detected by prior work. Through a large-scale human study on Amazon Mechanical Turk, we further confirmed the authenticity of test cases generated by RMT and the validity of detected abnormal model predictions.

Internet of Things (IoT) systems continuously collect a large amount of data from heterogeneous "smart objects" through standardised service interfaces. A key challenge is how to use these data and relevant event logs to construct continuously adapted usage profiles and apply them to enhance testing methods, i.e., prioritization of tests for the testing of continuous integration of an IoT system. In addition, these usage profiles provide relevance weightings to analyse architecture and behaviour of the system. Based on the analysis, testing methods can predict specific system locations that are susceptible to error, and therefore suggest where expanded runtime monitoring is necessary. Furthermore, IoT aims to connect billions of "smart devices" over the network. Testing even a small IoT system connecting a few dozens of smart devices would require a network of test Virtual Machines (VMs) possibly spreading across the Fog and the Cloud. In this paper we propose a framework for testing of each IoT layer in a separate VM environment, and discuss potential difficulties with optimal VM allocation., Comment: PREPRINT. Paper accepted to the 25th Australasian Software Engineering Conference (ASWEC 2018)

The personalized health care service utilizes the relational patient data and big data analytics to tailor the medication recommendations. However, most of the health care data are in unstructured form and it consumes a lot of time and effort to pull them into relational form. This study proposes a novel data lake architecture to reduce the data ingestion time and improve the precision of healthcare analytics. It also removes the data silos and enhances the analytics by allowing the connectivity to the third-party data providers (such as clinical lab results, chemist, insurance company,etc.). The data lake architecture uses the Hadoop Distributed File System (HDFS) to provide the storage for both structured and unstructured data. This study uses K-means clustering algorithm to find the patient clusters with similar health conditions. Subsequently, it employs a support vector machine to find the most successful healthcare recommendations for the each cluster. Our experiment results demonstrate the ability of data lake to reduce the time for ingesting data from various data vendors regardless of its format. Moreover, it is evident that the data lake poses the potential to generate clusters of patients more precisely than the existing approaches. It is obvious that the data lake provides a unified storage location for the data in its native format. It can also improve the personalized healthcare medication recommendations by removing the data silos., Comment: 15 Pages, 3 figures, 1 table, Presented in Sixth Australasian Symposium on Service Research and Innovation 2017

The Time-Of-Flight system consisting of plastic scintillation counters plays an important role for particle identification in the BESIII experiment at the BEPCII double ring $e^+e^-$ collider. Degradation of the detection efficiency of the barrel TOF system has been observed since the start of physical data taking and this effect has triggered intensive and systematic studies about aging effects of the detector. The aging rates of the attenuation lengths and relative gains are obtained based on the data acquired in past several years. This study is essential for ensuring an extended operation of the barrel TOF system in optimal conditions., Comment: 14 pages, 8 figures

The discovery of the neutrino mixing angle $\theta_{13}$ opens new opportunities for the discovery of the leptonic CP violation for high intensity neutrino beams. MOMENT a future neutrino facility with a high-power proton beam of 15 MW from a continuous-wave linac is focused on that discovery. The high power of the proton beam causes extreme radiation conditions for the facility and especially for the target station where the pion capture system of five superconducting solenoids is located. In this paper initial studies are performed for the effects of the radiation on the solenoid structure and the area surrounding it. A concept cooling system is also proposed., Comment: 7 pages, 24 figures, to be submitted to Chinese Physics C

The relative differences in coordinates of Cylindrical-Gas-Electron-Multiplier-Detector-based Inner Tracker (CGEM-IT) clusters are studied to search for track segments in CGEM-IT. With the full simulation of single muon track samples, clear patterns are found and parameterized for the correct cluster combinations. The cluster combinations satisfying the patterns are selected as track segment candidates in CGEM-IT with an efficiency higher than 99%. The parameters of the track segments are obtained by a helix fitting. Some chi-squared quantities, evaluating the differences in track parameters between the track segments in CGEM-IT and the tracks found in Outer-Drift-Chamber (ODC), are calculated and used to match them. Proper chi-squared requirements are determined as a function of transverse momentum and the matching efficiency is found reasonable.

Considering the aging effects of existing Inner Drift Chamber (IDC) of BES\uppercase\expandafter{\romannumeral3}, a GEM based inner tracker is proposed to be designed and constructed as an upgrade candidate for IDC. This paper introduces a full simulation package of CGEM-IT with a simplified digitization model, describes the development of the softwares for cluster reconstruction and track fitting algorithm based on Kalman filter method for CGEM-IT. Preliminary results from the reconstruction algorithms are obtained using a Monte Carlo sample of single muon events in CGEM-IT.

In this paper, we present our vision of the integration of human factors engineering into the software development process. The aim of this approach is to improve the quality of software and to deal with human errors in a systematic way., Comment: Preprint, Workshop on Applications of Human Error Research to Improve Software Engineering (WAHESE) at ICSE 2015