Comparisons of metaheuristic algorithms and fitness functions on software test data generation

Graphical abstractDisplay Omitted HighlightsThis paper applies meta-heuristic algorithms to software testing problem.Different meta-heuristics were employed to analyze their performances on test data generation.A control parameter sensitivity analysis was performed on the algorithms.Various fitness functions based on different coverage approaches were compared. Cost of testing activities is a major portion of the total cost of a software. In testing, generating test data is very important because the efficiency of testing is highly dependent on the data used in this phase. In search-based software testing, soft computing algorithms explore test data in order to maximize a coverage metric which can be considered as an optimization problem. In this paper, we employed some meta-heuristics (Artificial Bee Colony, Particle Swarm Optimization, Differential Evolution and Firefly Algorithms) and Random Search algorithm to solve this optimization problem. First, the dependency of the algorithms on the values of the control parameters was analyzed and suitable values for the control parameters were recommended. Algorithms were compared based on various fitness functions (path-based, dissimilarity-based and approximation level+branch distance) because the fitness function affects the behaviour of the algorithms in the search space. Results showed that meta-heuristics can be effectively used for hard problems and when the search space is large. Besides, approximation level+branch distance based fitness function is generally a good fitness function that guides the algorithms accurately.