Search

Pokritost programske kode s testi je mera, ki pove, kolikšen delež izvorne kode izbranega programa oziroma poti skozi program je bilo testiranih pri izvajanju testne zbirke. V diplomskem delu smo razvili javansko orodje za analizo pokritosti kode s principi mutacijskega testiranja. Mutacije so majhne modifikacije izvorne kode, ki jih namerno povzročamo v testirani programski kodi z namenom ugotavljanja kvalitete obstoječih testov ter njihove izboljšave. Uporabili smo mutacijski operator BSR ter v javansko izvorno kodo sistematično vstavljali tako imenovane bombe, javanske stavke, ki pri izvajanju povzročijo napako. Na mutirani programski kodi smo poganjali testno zbirko in spremljali, ali se testi uspešno izvedejo. Na osnovi tega smo sklepali na pokritost mesta v kodi, kamor je bila postavljena bomba. Pri tem smo veliko pozornosti posvetili optimizaciji algoritma, ker je mutacijsko testiranje že v osnovi zelo časovno zahteven proces. Aplikacijo smo razvili kot vtičnik za Maven, orodje za upravljanje javanskih projektov, kar omogoča enostavno uporabo na poljubnem projektu. Za cilj smo si postavili razviti orodje, ki je po funkcionalnosti in načinu merjenja pokritosti podobno orodju EMMA, enemu od javanskih orodij za merjenje pokritosti. Pri testiranju aplikacije na nekaj realnih projektih se je pokazalo, da so rezultati precej natančni in primerljivi z orodjem EMMA. Code coverage is a measure used to describe the degree to which the source code of a program or paths through the program has been tested by a particular test suite. In this thesis we have designed and implemented a Java tool for code coverage analysis which is based on principles of mutation testing. Mutations are small modifications of source code which are deliberately seeded into the source code under testing in order to evaluate the quality of existing test suite and improve it. We used the mutation operator BSR to systematically seed so-called bombs into Java source code. In mutation testing a bomb is a statement which causes an exception when executed. After that we ran the test suite against the mutated code and checked whether it finished successfully. Based on the test suite results it is possible to assess the code coverage at the position where the bomb was placed. We put a lot of effort in algorithm optimization because mutation testing is known to be extremely time-consuming process. We designed the application as a plugin for Maven, software project management and comprehension tool, which makes it simple to use on any Maven based Java project. Our goal was to develop a tool similar to EMMA by functionality and code coverage measurement methodology. By running the code coverage analysis on a few real-world open source projects, the application proved to be pretty accurate and comparable to EMMA.

Diplomska naloga zajema področje testiranja programske opreme z izvajanjem mutacijske analize. Mutacijska analiza oziroma testiranje se uporablja za ustvarjanje novih ali za ovrednotenje že obstoječih programskih testov. Kakovost testov se določi na način, da se v program vstavijo vnaprej definirane napake. V kolikor testi v programu odkrijejo vse napake, so učinkoviti, v nasprotnem primeru pa so testi neučinkoviti in jih je potrebno ustrezno dopolniti. V diplomski nalogi je tako podrobneje predstavljen koncept mutacijskega testiranja, postopek generiranja in testiranja mutantov z uporabo sistema MuJava in eksperiment, ki se osredotoča na aritmetični mutacijski operator. Eksperiment naslavlja problem časovne zahtevnosti mutacijskega testiranja in s pomočjo analize učinkovitosti posameznih mutacij mutacijskega operatorja AOR predlaga možne rešitve na področju optimizacije postopka generiranja mutantov. The thesis covers the field of software testing with the use of mutation testing. Mutation testing or mutation analysis is used to create new software tests or to evaluate the efficiency of preexisting tests. The quality of a given test is determined by injecting predefined faults into a program's source code. If the test detects faults in the program it is considered to be adequate. In contrast, if the test does not detect any inappropriate behavior it is considered to be inadequate and must therefore be adjusted. The thesis describes the concept of mutation testing in detail, demonstrates the process of generating and testing mutants with the use of the mutation testing system called MuJava and carries out an experiment that focuses on the arithmetic mutation operator. The experiment addresses the problem of runtime complexity of mutation testing and proposes possible optimization solutions in the process of generating mutants. The solutions are based on the analysis that determines the efficiency of individual mutations that are applied as part of the AOR mutation operator.