Search

Testing a program to confirm whether it consistently implements its requirements specification is a necessary but time-consuming activity in software development. Automatic testing based on specifications can significantly alleviate the workload and cost, but faces a challenge of how to ensure that both the user's concerns in the specification and possible execution paths in the program are all covered. In this paper, we describe a new method, called "Vibration-Method" or simply "V-Method", for automatic generation of test cases and test oracle from model-based formal specifications, aiming to address this challenge. The proposed method is suitable for testing information systems in which rich data types are used. Supporting the principle of "divide and conquer", the method provides a specific technique for generating test cases based on functional scenarios defined in the specification, test case generation criteria, automatic test case generation algorithms, and a well-defined mechanism for deriving test oracle. We elaborate on the method by discussing how initial test cases can be automatically generated, how additional necessary test cases are produced using the "vibration" technique, and how a test oracle can be automatically derived for a group of test cases. We also describe a controlled experiment to evaluate the effectiveness of the method and discuss the important issues in relation to the performance and applicability of the method.

Medical Apps are increasingly gaining in importance. But comprehensive testing of the app is difficult and there is no single method. For example, compliance has been ignored so far in most cases. A general remote method has been developed to test the compliance behavior when using medical apps, as well as to take them into account in later treatment. With the help of timestamps at appropriate locations, frequency, times and periods of use can be measured and (automatically) compared with predefined values. Furthermore, this method provides information about the efficiency and the learning phase of the users, as well as the usability, which is of great importance for the evaluation of compliance. With the help of different smartphone sensors, it is possible to evaluate the data quality. By taking other factors into consideration, such as the quality of experience and external circumstances, as well as the determination of data quality, the medical app can be evaluated comprehensively. This allows user profiles to be created . This resulting information can be entered into the treatment and the quality of the treatment can be increased .

To find why we cannot see obvious relationships between entrance examination scores and academic records in universities in mathematics subjects, we have investigated three testing records of the placement test, the learning check test, and term examinations. First, irreducible probabilistic fluctuations of academic scores are analyzed by using the placement test and term examinations. As a result, we have noticed the magnitude of irreducible probabilistic fluctuations. Next, we have compared the score distributions of the three testing scores in equally split classes using the placement test scores. In testings in this paper, unlike the ordinary testing style of the description type in mathematics subjects, the multiple choice type testings were performed. Then, we have found two important things. One is that we can eliminate teacher’s evaluation biases. The other is that the student’s academic growth could be seen more clearly using the multiple choice type testing than using the description type testing. In addition, we have proposed a fundamental equation on student’s ability including irreducible probabilistic fluctuations.

Grid codes around the world are requiring grid-support functions (GSFs) and standardized interoperability interfaces for distributed energy resources (DERs) to address the rapid increase of renewable energy. However, these new GSFs need to be assessed to ensure the desired power and communication capabilities exist in the field. The IEEE 1547.1 standard outlines the conformance test procedures for DER devices and is currently undergoing a major revision to align it with IEEE 1547–2018. Once it is published (anticipated in mid-2020), GSFs in commercial PV inverters in USA and Canada will be certified to the IEEE 1547.1 conformance test procedures. Several international research laboratories are collaborating to develop a versatile open-source DER testing platform that performs automated testing of DER devices. This community of laboratories is developing open-source IEEE Std. 1547.1 test scripts to lower barriers to DER vendor internal equipment evaluations, ease product compliance testing at certification laboratories, and provide research institutions a tool to study DER behaviors. In this work, test scripts were used for test verification of GSFs, including limit active power, constant reactive power, active power-reactive power (watt-var), and prioritization of GSF response for several DER devices. Sample test results for these DER GSFs and test protocol recommendations are presented in this paper.

Protocol conformance test plays a necessary and important role in guaranteeing that Long Term Evolution-Vehicle to Everything (LTE-V2X) devices to meet the standards and achieve interoperability. Based on in-depth analyzing 3rd Generation Partnership Project (3GPP) release 14 and China LTE-V2X standards, in this paper, a LTE-V2X Conformance Testing Scheme is designed, while corresponding test process and message sets are defined. Test suite are designed according to message layer and network layer requirements. Finally, a test tool, VeTest, is proposed to realize conformance test. The work is significance for promoting the LTE-V2X standardization and interoperability.

Objectives. To effectively evaluate the compliance degree between the electronic medical records of Traditional Chinese Medicine (TCM) hospitals, as well as the information platform, and the related information standards of electronic medical records, a standard compliance testing scheme based on electronic medical records of TCM outpatients is proposed. Methods. This research selected the data of clinical outpatients accumulated in 10 years by the Digital Medicine Institute of Chengdu University of TCM and processed the data through security check and desensitization process. And then 28348 cases of processed electronic medical records of TCM outpatients were inputted into the standard compliance testing platform for assessment. The result was then outputted. Results. There are 924 cases among the 28348 that can be rated as five-star medical records, 84 cases four-star, 132 cases three-star, 12460 cases two-star, 13488 one-star, and 1260 cases zero-star through the integrity and standardization test. Conclusion. By the way of assessing the integrity and standardization of data, the standard compliance test algorithm scheme for electronic medical records of TCM outpatients introduced in this paper can solve the problems such as data unavailability caused by ununified codes and incomplete data in the data-sharing process and provides technical support for the construction of data standardization testing in electronic medical records of TCM outpatients.

International audience; In the setting of ioco-based conformance testing with test purposes, we propose an automatic approach to generate a test plan (set of test purposes) with its associated test suite (set of test cases) covering all transitions of the IOLTS model of the system. The approach can also be applied to improve an existing test plan, by both, completing the coverage and eliminating redundancies. Implementing our approach on top of the CADP toolbox, we report on experiments with several examples of concurrent systems and discuss possible variants and heuristics to fine-tune the overall performance of the approach, as well as the quality of the computed test plan.

The paper covers theoretical and practical works on conformance testing performed in ISP RAS since 1994 till now. The conformance theory development was done in various directions and, in the whole, was characterized by generalization of the interaction semantics, models and conformances in use. The necessity of such generalization was imposed, first of all, by requirements of testing practice. It is true for such system properties as nondeterminism, partial specified, asynchronous behavior, diversity of test stimuli and observations of the implementation behavior etc. It was always focused on testing effectiveness defined both by optimization of tests suites and by test generation algorithms including on-the-fly. We consider the main milestones on this way in a brief and informal discussion, paying attention not to details, but to the main problems and their solutions trying to reveal the common tendency of the development

Complete conformance testing is a model-based test technique for programmable controllers. It checks whether an implementation conforms to its specifications with regard to all possible combinations of input signals, which is useful for small scale and safety critical systems. However, the state space explosion issue limits its application to large scale systems. This paper presents a method for reducing state space in generation of test cases by utilizing not only specification models but also features extracted from plant models. The application on a benchmark case study shows that the number of test cases is reduced significantly.

This paper first introduces the basic concepts of communication protocol conformance testing, analyzes and summarizes the content of communication protocol consistency test, the principle of communication protocol consistency test, the process of communication protocol consistency test and the method of communication protocol consistency test. Based on the above research, , design a conformance test model combined with the characteristics of OpenADR communication protocol to ensure interoperability between different interactive entities, improve the efficiency of implementation of interconnection services, ensure the normal development of demand-side systems and equipment interconnection, reduces the cost of power demand side management, and contributes to the development of demand-responsive communication protocols.

In this paper, an approach based on the IOCOS testing relation to test safety PLC code using the tool QuickCheck is presented. Testing and validation of the safety PLC code is typically carried out on a physical system using checklists. These checklists are developed by engineers using system specification. However, due to the manual nature of checklist generation and execution, certain test cases can be overlooked and can lead to human accidents. The presented approach allows on-the-fly generation and execution of test cases, which expands the scope of testing by including test cases unconceived during checklist generation. Furthermore, it is demonstrated how errors in the safety PLC code are uncovered based on the IOCOS relation.

Electromagnetic compatibility (EMC) compliance testing is mandatory for most electronic products slated for sale throughout the world. Therefore, EMC should be one of the main objectives of designing any electronic product, at the same level as electrical, mechanical, or even software design. However, the training that students receive in relation to EMC is usually scarce, mainly because the specific equipment that this subject requires is rarely present in educational laboratories. This paper presents an affordable EMC pre-compliance test lab for educational purposes, together with lab activities related to four of the most important EMC topics. The aim is to raise the awareness of electronic engineering students, who are the future designers of electronics products, of the importance of considering EMC techniques from the beginning of the design process.

In this paper we propose a new approach to conformance testing based on Abstract State Machine (ASM) model refinement. It consists in generating test sequences from ASM models and checking the conformance between code and models in multiple iterations. This process is applied at different models, starting from the more abstract model to the one that is very close to the code. The process consists of the following steps: (1) model the system as an Abstract State Machine, (2) generate test sequences based on the ASM model, (3) compute the code coverage using generated tests, (4) if the coverage is low refine the Abstract State Machine and return to step 2. We have applied the proposed approach to Antidote, an open-source implementation of IEEE 11073-20601 Personal Health Device (PHD) protocol which allows personal healthcare devices to exchange data with other devices such as small computers and smartphones.

In the context of distributed test, the activity of the test is performed by a set of parallel testers called PTCs (Parallel Test Components). The difficulty is in writing communication procedures of coordination and cooperation between the PTCs. The coordination problems in the distributed test context have been a research issue for many years. In this paper, we discuss some of these coordination problems and we propose a solution using an Ontology-based approach for modeling such situations. We present a model of inter-agent communications based ontology for the distributed test. The proposed approach leads us to introduce a common language that is comprehensible by all the agent testers. The objective of this work is to eventually have a platform for compliance testing of distributed applications.

Model checking of real-time systems has evolved throughout the years. Recently, the model checker Ecdar, using timed I/O automata, was used to perform compositional verification. However, in order to fully integrate model checking of real-time systems into industrial development, we need a productive and reliable way to test if such a system conforms to its corresponding model. Hence, we present an extension of Ecdar that integrates conformance testing into a new IDE that now features modelling, verification, and testing. The new tool uses model-based mutation testing, requiring only the model and the system under test, to locate faults and to prove the absence of certain types of faults. It supports testing using either real-time or simulated time. It parallelises test-case generation and test execution to provide a significant speed-up. We also introduce new mutation operators that improve the ability to detect and locate faults. Finally, we conduct a case study with 140 faulty systems, where Ecdar detects all faults., In Proceedings GandALF 2018, arXiv:1809.02416

Partial observability and controllability are two well-known issues in test-case synthesis for interactive systems. We address the problem of partial control in the synthesis of test cases from timed-automata specifications. Building on the tioco timed testing framework, we extend a previous game interpretation of the test-synthesis problem from the untimed to the timed setting. This extension requires a deep reworking of the models, game interpretation and test-synthesis algorithms. We exhibit strategies of a game that tries to minimize both control losses and distance to the satisfaction of a test purpose, and prove they are winning under some fairness assumptions. This entails that when turning those strategies into test cases, we get properties such as soundness and exhaustiveness of the test synthesis method., Comment: Augmented version with proofs. Reference to be added later

In the current Web Service Description Language (WSDL), only the interface information of a web service is provided without any indication on its behavior logic. Naturally, it is difficult for the service user and developer to achieve a shared understanding of the service behavior through such a description. A particular challenge is how to make explicit the various behavior assumptions and restrictions of a service (for the user), and make sure that the service implementation conforms to them (for the developer). In order to improve the behavior conformance of services, in this paper we propose a constraint-based model-driven testing approach for web services. In our approach, constraints are introduced in an extended WSDL, called CxWSDL, to formally and explicitly express the implicit restrictions and assumptions on the behavior of web services, and then the predefined constraints are used to derive test cases in a model-driven manner to test the service implementation’s conformance to these behavior constraints from the user’s perspective. We have conducted an empirical study with three real-life web services as subject programs, and the experimental results have shown that our approach can effectively validate the service’s conformance to the behavior constraints.

Mil-Std-461 defines emission and susceptibility requirements for electronic, electrical and electromechanical equipment and subsystems designed or procured by agencies of Department of Defense (DoD), USA. Latest version of this standard, Mil-Std-461G, was released on 11th December 2015. The same standard is used by Indian defense sector to test its equipment and subsystems for their EMC compliance. Latest version brought many changes as well as added new test requirements which are not present in earlier versions. This paper presents a study report on the latest version which discusses about new test requirements, general changes and modifications made in individual tests. The requirement of new equipment to upgrade test facility is also discussed. This study report will help test labs and engineers in upgrading test facilities as well as conducting compliance testing in accordance with latest version.

Through the recent decades, the technical area of antenna testing has undergone disruptive changes, since modern telecommunication systems demand ever increasing test cases. OTA (Over the Air) testing initially mostly concerned making sure that cellular phones complied with radiation pattern and efficiency requirements [1]. For future BTS (Base Transceiver Station) systems, OTA testing will cover a much larger proportion of the testing needed for a transceiver product. Testing of multi-antenna devices will further require more advanced test methods, in many times involving spatial channel models [2]. This paper presents different existing OTA test methods of wireless transceivers, and what adaptations of these which may be needed for testing future 5G BTS systems. Aspects of both conformance testing and performance testing are discussed.

Objective The NIST Syndromic Surveillance Test Suite for 2015 Edition ONC certification testing was published in February 2016. Key information related to the purpose, development, and use of this conformance test tool is provided via snapshots on a poster. Introduction Details about the ONC 2015 Edition certification criteria for Syndromic Surveillance and the related NIST Test Suite were explained previously. We now provide an overview and key information regarding updates to the Test Suite and how it is designed to be used. Methods Snapshots are provided on a poster and are used by the presenter to explain the steps involved in developing the NIST Syndromic Surveillance Test Suite 2015 Edition, to show key features of and updates to the Test Suite, and to illustrate the relationship of the Test Suite to various releases of the PHIN Messaging Guide for Syndromic Surveillance. Results The NIST Syndromic Surveillance Test Suite for 2015 Edition ONC certification testing was published in February 2016. As the target stakeholders began using it and providing feedback, this tool and associated documentation were updated. The Test Suite is being used by test labs for ONC certification testing of health information technologies, by developers in preparation for certification testing, and ultimately by public health jurisdictions for on-boarding of provider organizations that need to submit surveillance data. Conclusions Automated conformance test tools enable validation of health information technologies’ ability to support the requirements published in the PHIN Messaging Guide for Syndromic Surveillance. Having this standard and the means to validate conformance helps drive the industry toward the level of interoperability needed to promote efficient reporting and utilization of syndrome-based public health surveillance information.

The process of compliance testing for electrical equipment can be performed in a multitude of environments possessing various unique conditions that are potentially critical influences to the proper results of each test. This is particularly true when testing is performed outside of the controlled environment of a laboratory as it may be in field evaluation. The variations in these environmental conditions raise concerns as to the requirement for the compensation of results and their documentation based solely upon the severity of their influences. By researching the general effects of environmental factors, including ambient temperature, atmospheric pressure, and humidity, on the various electrical properties of different electrical components and equipment, and applying such considerations to common compliance testing methods, the requirement for the technical documentation of these factors will be determined. The data analyzed suggested that each environmental factor exhibits similar effects to varying degrees when applied to the common compliance testing methods consisting of the dielectric strength test, leakage current test, ratings test, and temperature test. Ultimately, due to the variations in their significance when applied to common compliance testing methods, the consensus for evaluators of equipment to document environmental conditions in addition to compensating results is deemed necessary only while operating in environments to which specific conditions are severe enough to distort results.

Model-based testing (MBT) is typically a black-box testing technique. Therefore, generated test suites may leave some untested gaps in a given implementation under test (IUT). We propose an approach to use the structural and behavioural information exploited from the implementation domain to generate effective and efficient test suites. Our approach considers both specification models and implementation models, and generates an enriched test model which is used to automatically generate test suites. We show that the proposed approach is sound and exhaustive and cover both the specification and the implementation. We examine the applicability and the effectiveness of our approach by applying it to a well-known example from the railway domain.

Functional failures in Safety Instrumented System (SIS) of offshore platforms may have catastrophic consequences for the production, facility, environment and health. This work presents a method for automatic conformance testing of safety specifications represented in a Cause and Effect Matrix (CEM) for Programmable Logic Controllers (PLC) in charge of SIS. Test cases are automatically designed from the CEM using a CEG-BOR strategy to enhance coverage of black box test. Petri Net models support the automated oracle creation and test result evaluation. An experimental tool has been developed to edit the CEM, to generate and execute test cases on a PLC simulator, to generate and execute the Petri Net oracles and to present the verdict. The method has been applied to test the SIS of an offshore oil platform.

Accurate and unique sensor node identifier encoding will be an important factor to ensure the application of Wireless Sensor Network (WSN) identifier. Therefore, in order to detect the conformity of the WSN identifier encoding, the design and implementation of a WSN identifier encoding conformance test system based on web are presented. The WSN identifier encoding conformance test system consists of test method, system function, and test architecture. System function includes executable test suite (ETS) generation function, the test preparation function, the test execution function, the test response information analysis function and the test analysis function. For rapid conformance and credibility of the test system, we designed the conformance test software which consists of test server and test router. The design of test server includes man-machine interface, application service components, supporting service components and system management. The design of test router includes data communication layer management, test execution management. The implementation of test system mainly including the dataflow of test system and implementation process of test server. Verification of the test system comprises ETS verification and the test execution result verification. Additionally, the WSN identifier encoding test specifications was formulated according to the design of test system. The test results demonstrate the effectiveness of the test system by testing the sensor nodes in WSN.

IEC has developed 61968 specifications for interoperability in distribution network area, provides the possibility for solving isolated island of information. This paper designed the conformance test system for information exchange bus, used to verify the information bus meets IEC 61968 interoperability requirements. First introduced the information bus and message transmission mode based on topic. Then it describes the conformance test system structure, testing method and the testing process, as well as visualization of the message exchange process. The paper summarizes domestic and foreign information interoperability testing experience and future planning.

International audience; In recent years, Extensible Messaging and Presence Protocol (XMPP) is gaining momentum in Internet of Things (IoT). It has been widely used in chatting, message exchanging and unique addressing. As a matter of course, it raises an interesting issue: how to formally test the conformance and performance of XMPP in IoT environment. While conformance testing of communicating protocols is a functional test that verifies whether the behaviors of the protocol satisfy defined requirements, performance testing is a qualitative and quantitative test that aims at checking whether the performance requirements of the protocol are satisfied under certain conditions. In this paper, we present a logic-based passive testing approach that can test both the conformance and the performance of XMPP protocol through real execution traces and formally specified properties. To evaluate and assess our methodology, we present a developed prototype and the experiments with a set of XMPP properties. Finally, the relevant verdicts and conclusions are provided

This article summarizes the experience gained while developing test suites for conformance testing of implementations of Internet protocols. The projects described in this article used the UniTESK technology as a base for constructing tests. During the development of test suites we identified certain features of the protocols that make it difficult to apply UniTESK unmodified to protocol conformance testing, as well as some limitations of UniTESK toolkits. Nevertheless all these obstacles were successfully overcome without going beyond UniTESK concepts.

In automation domain, the conformance of field bus devices in regard to the corresponding bus system standard is ensured by deploying conformance tests. The correctness of the tests is therefore decisive. A common practice to develop conformance test manually is fault-prone due to the fact that the test engineer can misinterpret the standard or the standard itself has defects. Therefore, in this present work we argue that a conformance test development should be based on formally verified requirements and propose an approach to automatically generate conformance tests based on the verified model of a bus system standard in order to assure the test correctness.

This paper simply introduces a smart substation monitoring system which is designed based on the IEC61850 communication standard and uniform background. The project is the integration of online monitoring unit and the intelligent components. It uploads the information of the integrated intelligent components and integrates the online monitoring system based on IEC61850.The paper also analyzes testing basic principles of the substation equipment online monitoring system, designs the basic program of the conformance testing of smart substation equipment online monitoring system, thus illustrate the feasibility of the test program.

With more than 10 years of maturing through industrial use and standardization, the Testing and Test Control Notation (TTCN-3) has become a widely used technology that many businesses depend upon for ensuring their product quality. With the rising number of supporters and tools, the demand for a means to assess the standards compliance of TTCN-3 tools has increased. In this article, we describe the motivation, approach, methodology, and results of the still ongoing project to develop a standardized conformance test suite for TTCN-3 tools. We discuss the challenges involved in creating such a test suite, the way to deal with imposed resource limitations of the project, and where we think the effort is heading.

The paper presents some technical issues for testing distributed systems. The proposed approach consists on exploring the temporal properties that specify the time required to exchange messages between the various components of the distributed test application. Thereby taking into consideration the temporal properties in the specification of the behavior of conformance testing, it will provide a higher degree of expressiveness and flexibility. In other side, the paper show how to cope with problems encountered in distributed testing using timed distributed testing rules. Afterwards, we present our algorithm allowing the generation of such rules and a prototype of test using a Distributed Algorithmic and Rule-Based System (DARBS).

Access control mechanisms demand rigorous software testing approaches, otherwise they can end up with security flaws. Finite state machines (FSM) have been used for testing Role-Based Access Control (RBAC) mechanisms and complete, but significantly large, test suites can be obtained. Experimental studies have shown that recent FSM testing methods can reduce the overall test suite length for random FSMs. However, since the similarity between random FSMs and these specifying RBAC mechanisms is unclear, these outcomes cannot be necessarily generalized to RBAC. In this paper, we compare the characteristics and effectiveness of test suites generated by traditional and recent FSM testing methods for RBAC policies specified as FSM models. The methods W, HSI and SPY were applied on RBAC policies specified as FSMs and the test suites obtained were evaluated considering test characteristics (number of resets, average test case length, and test suite length) and effectiveness on the RBAC fault domain. Our results corroborate some outcomes of previous investigations in which test suites presented different characteristics. On average, the SPY method generated test suites with 32% less resets, average test case length 78% greater than W and HSI, and overall length 46% lower. There were no differences among FSM testing methods for RBAC regarding effectiveness. However, the SPY method significantly reduced the overall test suite length and the number of resets.

Conformance testing is a formal and structured approach to verifying system correctness. We propose a conformance testing algorithm for cyber-physical systems, based on the notion of hybrid conformance by Abbas and Fainekos. We show how the dynamics of system specification and the sampling rate play an essential role in making sound verdicts. We specify and prove error bounds that lead to sound test-suites for a given specification and a given sampling rate. © 2016 IEEE. VR Project EFFEMBAC and KK Project AUTO-CAAS

Software Defined Networking has become extremely hot both in academia and industry. Most of the SDN protocol is the OpenFlow protocol in the southbound interface. But bue to different manufacturers may have different understanding to the OpenFlow protocol and lead to the implementation may be different. So OpenFlow protocol conformance testing is becoming more important. This paper first introduces the concept of sdn and the current research situation of openflow protocol conformance testing, and then refer to the open source framework OFTest, use API of OFTest to write test cases for conformance testing. Finally, analyze the relationship between the number of test steps and test time according to the results of the test.

The electric vehicle market is rapidly growing due to its environmental friendliness and governmental support. As electric vehicles are powered by electricity, the interoperability between the vehicles and the chargers made by multiple vendors is crucial for the success of the technology. Relevant standards are being published, but the methods for conformance testing need to be developed. In this paper, we present our conformance test system for the electric vehicle charger in accordance with the standards ISO/IEC 15118, IEC 61851 and IEC 61850-90-8. Our test system leverages the TTCN-3 framework for its flexibility and productivity. We evaluate the test system by lab tests with two reference chargers that we built. We also present the test results in two international testival events for the ISO/IEC 15118 interoperability. We confirmed that our test system is robust, efficient and practical.

This session describes the Syndromic Surveillance Messaging Validation Suite being developed collaboratively by experts from NIST, ISDS, and CDC. The session explains the features and functions of the validation suite; describes how it will be used for ONC HIT certification testing in 2016; provides details about the test scenarios (including test stories, test data, and example conformant messages); and explains how using the validation suite for local implementation can enhance standard conformance and improve the quality of SyS data provided to public health agencies by hospitals, emergency departments, and urgent care centers.

This paper addresses conformance testing of communication protocols. The Session Initiation Protocol (SIP) is often considered "the standard protocol" for deploying VoIP (Voice over IP) solutions. We implemented a popular SIP distributed scenario on an industrial test bed. Active testing of SIP maintains our attention, while we kept on experimenting with the test cases dependencies architecture. Finally this work, presents new results from our use case in the European Telecommunication Standards Institute, Test Description Language standard (TDL).

This chapter focuses on the fundamental principles of conformance testing. Although the concepts of conformance testing are the key focal points addressed here, they can’t be discussed in isolation; thus, we include related concepts when appropriate. A testing life cycle and process are presented along with their relationship to the standards development life cycle. An important principle in standards development is to integrate testing early in the process in order to obtain feedback for the authors of the standard. A testing methodology framework is introduced that provides a process for developing, organizing, and managing tests, as well as conducting testing and analyzing the results. A detailed description and example of a Test Plan is given. Since our focus is on communication between distributed applications, we discuss how sending and receiving applications can be tested, using a laboratory test results case study as the context. Next, we offer a set of basic principles for developing Test Plans and Test Cases. Finally, a comparison between capability testing and site-testing is presented. The conformance testing principles discussed in this chapter are applied in Chap. 13 and are examined in relation to test tool implementations in Chap. 14.

The case study in this article is temperature condition modeling between a temperature artifact and a black test corner measuring instrument. The black test corner is an instrument which consists of two wooden walls and a floor, with build-in thermocouples fixed on the back side of the copper disks. The front of the disk is flush with the surface of the board. The black test corner is used for measuring how the temperature of a household appliance is influencing the surroundings in the real environment, e.g., in the kitchen, the living room, etc. The temperature artifact as presented in this article is a specially developed heating plate which is very stable and can be set to different temperatures. Technical standards for conformity assessment usually describe only what should be measured, in some cases also how accurate the measurement should be, but not what kind of measuring instrument should be used. Therefore, it sometimes happens that measurements are performed with improper equipment or in an improper way. For the same level of appliance conformance testing, laboratories shall use the same testing procedures and comparable measuring instruments. This article deals with the analysis of influencing parameters when measuring the temperature rise using the black test corner. Modeling of temperature conditions between a temperature artifact and a black test corner, using commercial modeling software, was performed to find out whether this modeling can be used for detailed evaluation of all possible influencing parameters of the mentioned testing procedure. A scheme and a list of influencing parameters that has to be modeled in the following research is prepared to arrange an optimal experiment.

The Ada Conformity Assessment Test Suite (ACATS) includes both positive and negative tests. The negative tests have intentional errors that a compiler is intended to diagnose with a compile-time error message. Interestingly, the negative tests also include numerous "OK" lines that the compiler must not reject. But the absence of an error message on the "OK" lines is not always a convincing demonstration that the "OK" lines were correctly compiled, since the negative tests are not executable. By removing the intentional errors from a copy of each negative test, we are able to compile and run the resulting modified tests, demonstrating more convincingly that the "OK" lines are correctly compiled.

International audience; Conformance test is a black-box test technique aiming at checking whether an implementation conforms to its specification. Numerous results have been already obtained in this field for specifications expressed in a formal language. However, these results cannot be applied for conformance test of industrial logic controllers whose specifications are given in standardized specification languages. To contribute to solve this issue, this paper proposes a method to obtain, from a Grafcet specification, an equivalent Mealy machine, without semantics loss. This method permits to describe explicitly and formally all the states and transitions that are implicitly represented in a Grafcet model.

Standards are developed by highly reputed organizations with a genuine intention that the document acts as a reference requirement and specification record for all manufacturers, and in this process the consumer is presented with a list of conforming products that are all interoperable. Active RFID tags and readers are designed with commercial intent by different manufacturers according to the ISO 18000-7 standard. Through intensive research and tests, it has come to our attention that it is possible to design two active RFID systems that in their entirety conform to the ISO 18000-7 standard and yet be not interoperable with each other. From this statement it can be inferred that conformance is not the sufficient condition for interoperability as is popularly believed, but only the necessary or minimum condition to satisfy interoperability. Therefore apart from preliminary conformance testing, it becomes absolutely essential to include additional and supplemental interoperability tests into the verification process of the production cycle. This research primarily establishes the requirement for interoperability testing giving instances where the standard fails to insure interoperable products. The traditional method to test interoperability currently in practice are investigated and their limitations are exposed. Further this research paper introduces an innovative and ingenious methodology to test active RFID systems for interoperability at the physical layer.

This article examines the potential of application profiles and domain profiles as means to adapt technical specifications of data structures to particular needs. The authors argue that application profiling is better suited to increase the use of formal specifications than the creation of new specifications. The authors also describe a method to generate specific conformance test systems for machine-readable application profiles. The authors describe the respective tool set of the SchemaProf Application Profiling Tool and the Generic Test System and report on the experience of their usage in developing and introducing the IMS Common Cartridge domain profile.

Substation Automation System is basically operating throughout communicative services among IEC 61850 based devices. So, minimum requirement in order to install the system in the substation is to verify server and client conformance of IEDs and HMIs according to the IEC 61850 standard. Even though UCA IUG issued IEC 61850 conformance test procedures for both the server and the client, Much of the test is limited to IEDs as server and there is no client-only testing tool except KEMA server test tool sets. this study aims to develop IEC 61850 client testing system for verifying the full communication conformance of the substation automation system and this paper describes the design and implementation of the testing system.

Existing methods for testing an SDL specification mainly allow for either black box simulation or conformance testing to verify that the behavior of an implementation matches its corresponding model. However, this relies on the potentially hazardous assumption that the model is completely correct. We propose a test generation method that can accomplish conformance verification as well as coverage criteria-driven white box testing of the specification itself. We first reformat a set of EFSMs equivalent to the processes in an SDL specification and identify ''hot spots'' - nodes or edges in the EFSM which should be prioritized during testing to effectively increase coverage. Then, we generate test sequences intended to cover selected hot spots; we address the possible infeasibility of such a test sequence by allowing for its rejection decided by a constraint solver and re-generation of an alternate test sequence to the hot spot. In this paper, we present our test generation method and tool, and provide case studies on five SDL processes demonstrating the effectiveness of our coverage-based test sequence selection.

Embedded systems are of growing importance in industry. For example, in a today's vehicle a huge number of embedded and communicating systems can be found. Exhaustive testing of such systems is a requirement, because changes after delivery and use are expensive and sometimes even impossible. In this paper we propose the use of qualitative models, which are an abstraction of quantitative physical models, for test case generation and test execution. In particular, we show how Simulink models from which control programs are automatically extracted can be tested with respect to qualitative models. Since Simulink models are heavily used in industry, the approach is of practical interest.

An approach to modeling the components of distributed systems whose interaction is based on handling events with regard for their priorities is considered. Although the priority-based servicing of requests or messages is widely used in practice, the mathematical models of the interaction of such programs often neglect the priorities thus introducing extra nondeterminism in the description of their behavior. The proposed approach attempts to avoid this drawback by defining the parallel composition that provides a model for the interaction of this kind. The subject matter of this paper is the development of a formal theory of testing the components that use priorities. Within this theory, the concept of a safe execution of the model and the conformance relation between the models are introduced, and the generation of test suites that check conformity is considered.

Fault-based conformance testing is a conformance testing strategy that relies on specific fault models. Previously, this mutation testing technique has been applied to protocol specifications. Although a practical case study of web-server testing has been conducted, we observed several issues when applying this method in a large industrial project. In this paper, we discuss the foundations, techniques and tools to overcome these shortcomings. More specifically, we show a solution to the problem of state-space explosion in generating mutation tests for industrial scale applications. Furthermore, the previous approach used the counterexamples of a bisimulation check (between the original and the mutant) as test purposes. With respect to input-output conformance (ioco), this is an over-approximation resulting in more tests than are necessary. Hence, we propose to use an ioco-checker in order to generate less test cases. An industrial case study demonstrates these improvements.