Search

This chapter deals with the importance of SOC design verification, plan and strategies adopted for verification. It defines functional simulation, functional coverage, code coverage, and other important terms used in verification. Importance of FPGA validation and how it complements the SOC design verification is explained in this chapter. Most of the simulation concept of SOC design verification explained in the chapter can be seen in the verification of the reference designs provided in Chapter 12.

This chapter reviews a vast amount of neuroimaging studies of Chinese, Japanese, and Korean in comparison to L2 English, using the neuonral recycling hypothesis (Dehaene, 2009) and the (writing) system accommodation hypothesis (Perfetti & Liu, 2005) as theoretical frameworks. In order to understand the basic brain network associated with reading, the major reading circuits found among typical readers are first reviewed. The findings of neuroimaging studies of reading in alphabetic scripts are reviewed and then moved on to the nonalphabetic Chinese and Japanese scripts, compared to L2 English. Although reviewed studies were not carried out to directly test script relativity, evidence converges on biological unity, script diversity, and cognitive diversity, which points toward script relativity.

Using theuniversal grammar of readingand thesystem accommodation hypothesis(Perfetti, 2003) as theoretical frameworks, this chapter reviews a wide range of linguistic evidence that supportsscript relativity. Universality and specificity found according to script features are discussed with respect to the operating principle (alphabet vs. logography), psycholinguistic gran size (phoneme vs. syllable), graph configuration (linearity vs. block), symbolic representation (arbitrariness vs. iconic quality), graph complexity (traditional characters vs. simplified characters), and multi-script representation (phonogram Kana vs. Ideogram Kanji). Linguistic skills associated with reading in terms of orthography, phonology, morphology as well as cross-linguistic and cross-scriptal transfer are reviewed. Next, based on the reviewed literature, each criterion for causality from script to cognition through reading as a multifaceted cognitive activity is checked. Although the existing literature did not aim to directly test script relativity, research findings collectively suggest script effects on readers’ thought and cognition.

HPC support teams are often tasked with installing scientific software for their user community and the complexity of managing a large software stack gets very challenging. Software installation brings forth many challenges that requires a team of domain expertise and countless hours troubleshooting to build an optimal software state that is tuned to the architecture. In the past decade, two software build tools (Easybuild, Spack) have emerged that are widely accepted in HPC community to accelerate building a complete software stack for HPC systems. The support team are constantly involved in fulfilling software request for end-users which leads to an ever-growing software ecosystem. Once a software is installed, the support team hands it off to the user without any testing because scientific software requires domain expertise in order to test software. Some software packages are shipped with a test suite that can be run at post build while many software have no mechanism for testing. This poses a knowledge gap between HPC support team and end-users on the type of testing to do. Some HPC centers may have developed in-house test scripts that are suitable for testing their software, but these tests are not portable due to hardcoded paths and are often site dependent. In addition, there is no collaboration between HPC sites in building a test repository that will benefit the community. This paper presents buildtest, a framework to automate software testing for a software stack along with several module operations that would be of interest to the HPC support team.

If a web application is migrated or the version of the system is changed then the test scripts becomes incapable of being validated because of the differences in the platform being used in testing. Regression tests need to be applied at that application after migration or version update are required to get written again in order to resolve compatibility issues with the new technologies. A problematic scenario occurs for the automation of script-level test when an application is updated or migrated just to include the technology which is not supported by the tool in use of testing. In this case, the purpose of test automation is eradicated because the scripts for regression testing are rewritten with the scripting language of the new testing tool being utilized. In this paper, a Model-Driven Approach is presented to develop the automated testing scripts for the validation of the FinTech Web Applications. A Meta-Model is proposed that depicts the scenario of regression testing of a FinTech system. A case study of small e-commerce platform known as Liberty Books is used for validation.

The analysis of past disaster response operations led to recommendations for improving disaster management, namely on the areas of situational awareness, decision support, and organizational agility. Information technologies offer solutions with the potential to answer these recommendations. This was the reason for launching the THEMIS (disTributed Holistic Emergency Management Intelligent System) project, which followed the UCD approach. THEMIS is an Intelligent System that applies Artificial Intelligence methodologies in support of disaster managers and first responders. The paper addresses the activities regarding designing and testing the user interaction of a digital prototype developed for disaster managers, describing the procedures adopted, the test conditions, test population, and the evolution of prototypes based on test findings. The paper also addresses the use of eye-tracking to support the analysis of tests, helping on the identification of user errors or problems faced while performing the tasks established in a validated test script.

Many modern software projects use bug-tracking systems (e.g., Bugzilla, Google Code Issue Tracker) to track software issues and help developers reproduce these issues. There has been recent work on automatically translating the natural language text (i.e., steps to reproduce) of bug reports to reproducing scripts, targeted at Android apps, to facilitate app debugging process. The scripts describe the event sequences leading to the app issues and thus can be reused for testing newer versions of the apps. However, existing techniques require manually providing the text description of steps to reproduce for generating reproducing scripts, which is a non-trivial task because natural language text in bug reports can be complex and contain much information irrelevant for bug reproduction. In this paper, we propose an approach that can automatically extract the text description of steps to reproduce (S2R) from bug reports to advance automated software issue diagnosis and test script reuse. The approach is implemented as a tool, called S2RMiner, which combines HTML parsing, natural language processing, and machine learning techniques. We have evaluated S2RMiner on 1000 original Android bug reports. The results show that S2RMiner can extract S2R with high accuracy.

The Mann-Whitney test for the two-sample location problem is considered. We adopt this nonparametric test to interval-valued data perceived from the epistemic perspective, where the available observations are just interval-valued perceptions of the unknown true outcomes of the experiment. Unlike typical generalizations of statistical procedures into the interval-valued framework, the proposed test entails very low computational costs. However, the presence of interval-valued data results in set-valued p-value which leads no longer to a definite binary decision (reject or not reject the null hypothesis) but may indicate the abstention from making a final decision if the information is too vague.

Due to copyright restrictions, the access to the full text of this article is only available via subscription. Model-based testing relies on models of the system under test to automatically generate test cases. Consequently, the effectiveness of the generated test cases depends on models. In general, these models are created manually, and as such, they are subject to errors like omission of certain system usage behavior. Such omitted behaviors are also omitted by the generated test cases. In practice, these faults are usually detected with exploratory testing. However, exploratory testing mainly relies on the knowledge and manual activities of experienced test engineers. In this paper, we introduce an approach and a toolset, ARME, for automatically refining system models based on recorded testing activities of these engineers. ARME compares the recorded execution traces with respect to the possible execution paths in test models. Then, these models are automatically refined to incorporate any omitted system behavior and update model parameters to focus on the mostly executed scenarios. The refined models can be used for generating more effective test cases. We applied our approach in the context of 3 industrial case studies to improve the models for model-based testing of a digital TV system. In all of these case studies, several critical faults were detected after generating test cases based on the refined models. These faults were not detected by the initial set of test cases. They were also missed during the exploratory testing activities. Vestel Electronics ; the Turkish Ministry of Science, Industry and Technology

Complete and correct requirements specification is the foundation for developing high-quality Automatic Train Protection (ATP) software. Requirements validation aims at facilitating the completeness and correctness of the specification. In this paper, we propose a novel requirements validation approach combining diagram-guided specification review and scenario-based specification testing for ATP software. The specification is transformed into an executable prototype. Diagrams are generated from the prototype to visualize the interactions between variables for an effective review. To check whether the specification conforms to the user’s concerned scenarios of train operation, the scenarios are specified as test cases for testing the prototype. The conformance is then determined via test analysis. Through the review and the testing, the requirements specification is validated. The case study and experiments show that the approach achieves a higher error detection rate and while it reduces the time costs comparing to the traditional review method used by our industrial partner.

Design for Testability (DFT) is a subject covering a huge amount material. The 1983 survey by Williams and Parker [Will83] is still remarkably current in its enumeration of DFT techniques (it lacks Boundary-Scan of course), but many of the contexts have changed. For example, signature analysis [Nadi77] testing is now conducted on-chip, though it started as a board-level technique. This reflects the incredible increase in the density of Integrated Circuit (IC) components. In 1983, the 1149.1 Standard would have been largely impractical because the logic needed to implement it would have been a large fraction of an IC. Today, we are seeing ICs designed with significant amounts of on-chip testing circuitry, including 1149.1. Without DFT, a VLSI component might not be economical to produce in volume.

KeYTestGen is a white-box test generator for Java methods based on KeY's program analysis and symbolic execution. KeYTestGen generates a JUnit test harness (test driver) which does not only initialize method parameters but also the global state that is defined by the (potentially private) fields of objects and classes. For example, a complex linked data structure may be created as test input. The tests can satisfy different test criteria such as branch coverage, path coverage, and MC/DC coverage. The user may also provide a specification in the Java Modeling Language (JML) from which a test oracle can be generated or which can be used as an abstraction for a loop or method call. KeYTestGen can be used either as a simple stand-alone tool not requiring expert knowledge or it can be used in an advanced way to support and complement formal verification.

At Daimler, a HiL for the high power net of an electrical vehicle was built. The rest-bus simulation was planned to be implemented as a script that simply pushed constant values to the bus. Later on, the script was to be extended by further logic to become more realistic. At the same time for another team of Daimler, a SiL was developed to carry out pretesting of code and data changes of some ECUs. The SiL contained three virtual ECUs that used the real in-house software, a plant model and of course some rest-bus simulation. By chance an engineer of the HiL team met and discussed about their current work. A synergy was detected: when the SiL would be put into HiL context, it would be a perfect rest-bus simulation since the original ECU code could be used to calculate all signals needed. Also the rest-bus simulation was coming almost for free because the SiL was ready to use. The virtual ECU tool Silver was then integrated into the HiL and further synergy effects were found. For the price of a powerful desktop PC, the plant model of the SiL, which was more detailed than needed, could be reused for the HiL. At last, the Silver could be extended to remotely control the hardware of the HiL. Thus, the SiL could even drive test scripts on the HiL. Integrating the SiL into the HiL improved its quality and sped up its build. Last but not least, the SiL helped to reduce costs because the three virtual ECUs replaced the three real ones for which special HiL hardware would have been required as well.

When developing a computer (mobile) game, testing the game is an important task and takes a large share of the development cost. So far, testing a game’s functional features relies mainly on human testers, who personally plays the game, and is thus labor intensive. This paper proposes a method that automates game testing. Since games are usually built on top of game frameworks, the idea is to enhance a game framework with a testing layer, which can execute (playback) test scripts that perform user events and assert the correctness of the game. We design an HTML5 game framework with such a support. In addition, a case study is performed to compare the testing cost of three different methods: writing a test script directly, recording a test script, and testing the game directly by a human tester. The results showed that when repeated testings are necessary, an automatic testing (either writing or recording test scripts) can reduce human cost. Among these three testing methods, recording scripts was the most favored method.

In order to remain useful test scripts must evolve parallel to the test objects they are intended to test. In the approach described here the test objects are web services whose test script is derived from the web service interface definition. The test script structure is automatically generated from the WSDL structure with tags and attributes, however, the content, i.e. the test data has to be inserted by hand. From this script service requests are automatically generated and service responses automatically validated. As with other generated software artifacts, once the structure of the interface or the logic of the targeted service is changed, the content of the test script is no longer valid. It has to be altered and/or enhanced to fit the new interface structure and/or the altered service logic. In this paper the author proposes a semi-automated approach to solving this test maintenance problem and explains how it has been implemented in a web service testing tool by employing data reverse engineering techniques. The author also report on his experience with the approach when maintaining a test in the field.

In the automated testing, finding an optimal architecture of the scripts and level of structuring to reusable objects from an economic point of view could become a challenging task. The optimality of the architecture of automated test scripts is context dependent and many factors are playing role in the final economics of the test automation project. To support this task with an exact method, we propose a model for evaluation of the automated testing architecture. This model is based on principal structural elements, which can be identified in the automated test scripts. Elements are then composed the particular architecture and for each of them, a set of properties and metrics are defined. Using the proposed metrics, more accurate estimations of implementation and maintenance costs can be performed for the particular architecture and test automation case, especially reflecting duplicity and reuse ratio of the code.

This paper presents a tool to filter/configure the test cases generated within the Model-Based Testing project PBGT. The models are written in a Domain Specific Language called PARADIGM and are composed by User Interface Test Patterns (UITP) describing the testing goals. To generate test cases, the tester has to provide test input data for each UITP in the model. After that, it is possible to generate test cases. However, without a filter/configuration of the test case generation algorithm, the number of test cases can be so huge that becomes unfeasible. So, this paper presents an approach to define parameters for the test case generation in order to generate a feasible number of test cases. The approach is evaluated by comparing the different test strategies and measuring the performance of the modeling tool against a capture-replay tool used for web testing.

Mobile Test Automation is gaining significant importance for an app-tester because it helps to alleviate the voluminous effort and time associated in thoroughly testing an application. Challenges like diversity in mobile hardware, multiple operating systems, ever-increasing application complexity and high volume of test cases etc. reiterate the importance of exploiting automation techniques for mobile application testing. In order to exhaustively capture user actions during the record-phase, faithfully reproduce those actions during playback-phase and also to capture the relevant metrics while playing back, instrumentation of the Application-under-test (AUT) becomes an imperative process. However, the type and level of instrumentation is different and is very specific to the category of testing which has to be automated. This paper presents Appstrument, a unified framework for instrumenting mobile applications to make them ready for functional, performance and accessibility testing. This framework allows instrumenting the application to get it ready for either a single category of testing or a combination of two or more of these categories, with multiple optional features for each category. In addition to this, given a test script, the framework also supports automated playback of instrumented applications. Appstrument has been deployed and tested against some popular applications from Google Play (Android apps) and some IBM in-house iOS applications. Results indicate that this framework is able to successfully instrument a sizeable number of applications and effectively playback user-defined test cases automatically to collect relevant metrics/results corresponding to each category of testing.

The purpose of the system integration test is to test the operation of the business system across and between application systems. Business system testing emphasizes on a common planning approach for all types of testing and advocates the reuse of deliverable components to test successively larger aspects of the application system.