Search

Software refactoring plays an important role in software engineering. Developers often turn to refactoring when they want to restructure software to improve its quality without changing its external behavior. Studies show that small-scale (floss) refactoring is common in industry and can often be performed by a single developer in short sessions, even though developers do much of this work manually instead of using refactoring tools. However, some refactoring efforts are much larger in scale, requiring entire teams and months of effort, and the role of tools in these efforts is not as well studied. In this paper, we report on a survey we conducted with developers to understand large-scale refactoring, its prevalence, and how tools support it. Our results from 107 industry developers demonstrate that projects commonly go through multiple large-scale refactorings, each of which requires considerable effort. While there is often a desire to refactor, other business concerns such as developing new features often take higher priority. Our study finds that developers use several categories of tools to support large-scale refactoring and rely more heavily on general-purpose tools like IDEs than on tools designed specifically to support refactoring. Tool support varies across the different activities, with some particularly challenging activities seeing little use of tools in practice. Our study demonstrates a clear need for better large-scale refactoring tools and an opportunity for refactoring researchers to make a difference in industry. The results we summarize in this paper is one concrete step towards this goal., Comment: 10 pages, 7 figures, 4 tables

Due to the growing complexity of software systems, there has been a dramatic increase and industry demand for tools and techniques on software refactoring in the last ten years, defined traditionally as a set of program transformations intended to improve the system design while preserving the behavior. Refactoring studies are expanded beyond code-level restructuring to be applied at different levels (architecture, model, requirements, etc.), adopted in many domains beyond the object-oriented paradigm (cloud computing, mobile, web, etc.), used in industrial settings and considered objectives beyond improving the design to include other non-functional requirements (e.g., improve performance, security, etc.). Thus, challenges to be addressed by refactoring work are, nowadays, beyond code transformation to include, but not limited to, scheduling the opportune time to carry refactoring, recommendations of specific refactoring activities, detection of refactoring opportunities, and testing the correctness of applied refactorings. Therefore, the refactoring research efforts are fragmented over several research communities, various domains, and objectives. To structure the field and existing research results, this paper provides a systematic literature review and analyzes the results of 3183 research papers on refactoring covering the last three decades to offer the most scalable and comprehensive literature review of existing refactoring research studies. Based on this survey, we created a taxonomy to classify the existing research, identified research trends, and highlighted gaps in the literature and avenues for further research., Comment: 23 pages

CSSI Element: Software: Software Health Monitoring and Improvement Framework

Background. Refactoring is a critical task in software maintenance and is generally performed to enforce the best design and implementation practices or to cope with design defects. Several studies attempted to detect refactoring activities through mining software repositories allowing to collect, analyze and get actionable data-driven insights about refactoring practices within software projects. Aim. We aim at identifying, among the various quality models presented in the literature, the ones that are more in-line with the developer's vision of quality optimization, when they explicitly mention that they are refactoring to improve them. Method. We extract a large corpus of design-related refactoring activities that are applied and documented by developers during their daily changes from 3,795 curated open source Java projects. In particular, we extract a large-scale corpus of structural metrics and anti-pattern enhancement changes, from which we identify 1,245 quality improvement commits with their corresponding refactoring operations, as perceived by software engineers. Thereafter, we empirically analyze the impact of these refactoring operations on a set of common state-of-the-art design quality metrics. Results. The statistical analysis of the obtained results shows that (i) a few state-of-the-art metrics are more popular than others; and (ii) some metrics are being more emphasized than others. Conclusions. We verify that there are a variety of structural metrics that can represent the internal quality attributes with different degrees of improvement and degradation of software quality. Most of the metrics that are mapped to the main quality attributes do capture developer intentions of quality improvement reported in the commit messages., technical paper accepted in 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

Many model transformation problems require different intermediate transformation steps. For example, platform-specific models (PSM) are often generated from platform-independent models (PIM) by chains of model transformations. This requires the presence of several intermediate meta-models between those of the PIM and the PSM. Thus, most of the effort is needed to define a transformation mechanism for each intermediate step. The solution proposed in this paper is to investigate whether it is possible to generate a single transformation from a chain of transformations, solely involving the initial PIM and final PSM meta-models. The presented work focuses on the composition of transformations at the rule level. We apply the automatic procedure for composing rules in the context of the evolution of the Enterprise Java Beans (EJB) language, transforming UML models into EJB 2.0 models and then to EJB 3.0 models., Electronic Communications of the EASST, Volume 42: Multi-Paradigm Modeling 2010

La transformation de modèles consiste à transformer un modèle source en un modèle cible conformément à des méta-modèles source et cible. Nous distinguons deux types de transformations. La première est exogène où les méta-modèles source et cible représentent des formalismes différents et où tous les éléments du modèle source sont transformés. Quand elle concerne un même formalisme, la transformation est endogène. Ce type de transformation nécessite généralement deux étapes : l’identification des éléments du modèle source à transformer, puis la transformation de ces éléments. Dans le cadre de cette thèse, nous proposons trois principales contributions liées à ces problèmes de transformation. La première contribution est l’automatisation des transformations des modèles. Nous proposons de considérer le problème de transformation comme un problème d'optimisation combinatoire où un modèle cible peut être automatiquement généré à partir d'un nombre réduit d'exemples de transformations. Cette première contribution peut être appliquée aux transformations exogènes ou endogènes (après la détection des éléments à transformer). La deuxième contribution est liée à la transformation endogène où les éléments à transformer du modèle source doivent être détectés. Nous proposons une approche pour la détection des défauts de conception comme étape préalable au refactoring. Cette approche est inspirée du principe de la détection des virus par le système immunitaire humain, appelée sélection négative. L’idée consiste à utiliser de bonnes pratiques d’implémentation pour détecter les parties du code à risque. La troisième contribution vise à tester un mécanisme de transformation en utilisant une fonction oracle pour détecter les erreurs. Nous avons adapté le mécanisme de sélection négative qui consiste à considérer comme une erreur toute déviation entre les traces de transformation à évaluer et une base d’exemples contenant des traces de transformation de bonne qualité. La fonction oracle calcule cette dissimilarité et les erreurs sont ordonnées selon ce score. Les différentes contributions ont été évaluées sur d’importants projets et les résultats obtenus montrent leurs efficacités., Model transformations take as input a source model and generate as output a target model. The source and target models conform to given meta-models. We distinguish between two transformation categories. Exogenous transformations are transformations between models expressed using different languages, and the whole source model is transformed. Endogenous transformations are transformations between models expressed in the same language. For endogenous transformations, two steps are needed: identifying the source model elements to transform and then applying the transformation on them. In this thesis, we propose three principal contributions. The first contribution aims to automate model transformations. The process is seen as an optimization problem where different transformation possibilities are evaluated and, for each possibility, a quality is associated depending on its conformity with a reference set of examples. This first contribution can be applied to exogenous as well as endogenous transformation (after determining the source model elements to transform). The second contribution is related precisely to the detection of elements concerned with endogenous transformations. In this context, we present a new technique for design defect detection. The detection is based on the notion that the more a code deviates from good practice, the more likely it is bad. Taking inspiration from artificial immune systems, we generate a set of detectors that characterize the ways in which a code can diverge from good practices. We then use these detectors to determine how far the code in the assessed systems deviates from normality. The third contribution concerns transformation mechanism testing. The proposed oracle function compares target test cases with a base of examples containing good quality transformation traces, and assigns a risk level based on the dissimilarity between the two. The traces help the tester understand the origin of an error. The three contributions are evaluated with real software projects and the obtained results confirm their efficiencies.

Orientadora: Silvia Regina Vergilio Coorientador: Marouane Kessentini Tese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 24/04/2020 Inclui referências: p. 61-66 Área de concentração: Ciência da Computação Resumo: A atividade de refatoração tem como principal objetivo aplicar um conjunto de transformações em um artefato de software para melhorar sua estrutura sem alterar sua funcionalidade. Alguns estudos recentes, apresentam bons resultados ao gerarem modelos de predição de refatorações. Além disso, os estudos mostram que refatorações similares são aplicadas em diferentes contextos e podem ser aprendidas. Neste sentido, a maioria dos trabalhos existentes utiliza técnicas de aprendizado de máquina para gerar modelos que predizem se um dado trecho de código deve ser refatorado. Entretanto, essas abordagens possuem limitações. Elas buscam por refatorações específicas e exatamente como aplicadas por desenvolvedores, o que limita que outras refatorações sejam encontradas. Dada a natureza subjetiva da atividade de refatoração de software, a exploração por refatorações com base em outros critérios também é vantajosa. Existem trabalhos na área conhecida como Refatoração de Software Baseada em Busca (SBR) (do inglês, Search Based Software Refactoring), em que algoritmos de busca são utilizados para encontrar refatorações em um grande espaço de busca e visando a melhorar diversos aspectos. Recentemente, trabalhos em SBR começaram a utilizar exemplos de refatorações já aplicadas por desenvolvedores para incorporar aprendizado na busca. Entretanto, essas abordagens são limitadas em termos de generalização dos resultados, uma vez que não geram um modelo que possa ser utilizado para diferentes programas. Desse modo, abordagens existentes de SBR devem ser configuradas e executadas a cada novo programa. Neste contexto, este trabalho visa a incorporar os benefícios encontrados na área de aprendizado de máquina e na área de SBR, apresentando uma abordagem chamada Gorgeous (do inglês, Generation of Refactoring Algorithms through Grammatical Evolution). Gorgeous tem como objetivo gerar algoritmos de refatoração compostos por regras, que quando executados, determinam trechos de código que devem ser refatorados e refatorações a serem aplicadas. Os algoritmos são criados de forma que as refatorações sugeridas sejam similares a refatorações aplicadas na prática e que também melhorem a qualidade do software. Os algoritmos são criados utilizando um processo de aprendizado que primeiro extrai padrões de refatoração de programas agrupando elementos que foram refatorados de maneira similar. Após isso, uma evolução gramatical é executada para gerar algoritmos de refatoração com base nos padrões extraídos. Gorgeous é avaliada utilizando dados de refatoração extraídos de 40 programas Java do repositório GitHub. Como resultado, os algoritmos gerados foram capazes de obter bons resultados para diferentes programas, melhorando em média 60% a qualidade do programa e obtendo 50% de similaridade com refatorações aplicadas na prática. Palavras-chave: Refatoração, Engenharia de Software Baseada em Busca,Agrupamento, Evolução Gramatical Abstract: The refactoring activity addresses the application of a set of transformations in software artifacts to improve their structure while preserving their functionality. Recent studies present promising results generating prediction models for refactoring. Furthermore, they provide evidences that similar refactoring operations are applied in different contexts and they can be learned using Machine Learning (ML). Most works on ML based refactoring generate models to predict if a piece of code should be refactored. Despite the capability of prediction, existing works are limited to learn specific refactoring operations as applied by developers. However, to explore refactoring operations possibilities based on other criteria is also beneficial, mainly by the subjective context of refactoring. In this context, the Search-Based Software Refactoring (SBR) area addresses studies using search algorithms to find refactoring operations in a huge search space, aiming at improving several other aspects. However, existing SBR approaches do not support generalization of results since they do not generate a model as ML studies. In this way, a SBR approach needs to be configured and executed for each program in need of refactoring. In this context, this work introduces a SBR learning approach aiming at taking most advantage of both fields. Gorgeous (Generation of Refactoring Algorithms through Grammatical Evolution) generates refactoring algorithms composed by several rules determining pieces of code that should be refactored and the refactoring types to be used. A refactoring algorithm provides as solution a set of refactoring operations to be applied in a program. In this respect, the algorithm is generated with the goal of increasing similarity of the refactoring operations with the ones applied in practice, and also improving program quality. To do this, a learning process first extracts refactoring patterns from programs by grouping their elements that were refactored in similar ways. After that, a Grammatical Evolution (GE) is executed to generate the algorithms based on the extracted patterns. Gorgeous is evaluated using refactoring data from 40 Java programs of GitHub repository. The refactoring algorithms are capable of obtaining good results to different programs, obtaining around 60% of program quality improvement and 50% of similarity with real refactoring applications. Keywords: Refactoring, Search-Based Software Engineering, Clustering, Grammatical Evolution

Orientador: Silvia Regina Vergilio Coorientador: Marouane Kessentini Tese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 07/11/2019 Inclui referências: p. 100-106 Área de concentração: Ciência da Computação Resumo: Algoritmos evolutivos para multiplos e muitos objetivos tem sido aplicados para selecionar produtos para o teste de variabilidade de Linhas de Produtos de Software (LPS). Esse problema refere-se a selecao de um conjunto adequado de produtos para testar uma LPS, pois testar todos os existentes e inviavel. O problema e impactado por muitos fatores, como numero de produtos a serem testados, criterios de cobertura a serem satisfeitos e eficacia em revelar defeitos. E possivel notar que muitas funcoes objetivo conflitantes precisam ser otimizadas ao mesmo tempo. No entanto, alguns problemas surgem quando o numero de objetivos a serem otimizados aumenta, por exemplo, as solucoes geradas pelos algoritmos de otimizacao se tornam incomparaveis, projetar uma frente de Pareto requer um grande numero de solucoes e a visualizacao de tais solucoes exige tecnicas especiais. Varias tecnicas sao propostas na literatura para resolver esse problema, como tecnicas de decomposicao e algoritmos baseados em indicadores. Entre eles, os algoritmos baseados na reducao de dimensionalidade e algoritmos baseados nas preferencias do usuario sao amplamente utilizados. Embora utilizados em diferentes abordagens de maneira separada, nao ha estudos na literatura que abordam o uso de reducao de dimensionalidade e algoritmos baseados em preferencias de forma combinada. Diante disso, este trabalho propoe uma abordagem chamada MaDRUP, Otimizacao de Muitos Objetivos com Reducao de Dimensionalidade baseada em Preferencias dos Usuarios. Esta abordagem visa a reduzir o numero de objetivos a serem otimizados com base nas preferencias declaradas durante o processo de geracao da solucao, e tem como objetivo principal a geracao de um conjunto reduzido de solucoes que leva menos tempo de execucao, mas mantem competitivamente os atributos de qualidade em comparacao com outros algoritmos. Para avaliar a aplicabilidade da abordagem proposta, MaDRUP foi instanciada com o algoritmo NSGA-II. Essa instanciacao e chamada COR-NSGA-II (NSGA-II com Reducao de Objetivos baseada em Confianca). O COR-NSGA-II define para cada funcao objetivo um nivel de confianca calculado pelas preferencias do usuario fornecidas interativamente. Os objetivos com valores mais altos de confianca sao removidos da proxima execucao do algoritmo. Para avaliar a viabilidade do COR-NSGA-II, tambem foi implementada uma ferramenta chamada Nautilus, e foram realizados experimentos usando seis LPSs diferentes, dois tipos de pontos de referencia representando as preferencias do usuario, cinco algoritmos e dois cenarios para simular diferentes perfis de usuario. Os resultados mostram que o COR-NSGA-II supera os algoritmos avaliados na maioria dos casos, gerando um numero menor de solucoes, com um menor tempo de execucao. Uma analise qualitativa tambem foi realizada com um conjunto de 12 usuarios que, respondendo a um questionario, indicaram ser mais facil escolher uma solucao gerada pelo COR-NSGA-II do que escolher uma solucao gerada pelos outros algoritmos. Palavras-chave: linha de produto de software. engenharia de software baseada em busca. algoritmos baseados em preferencias. reducao da dimensionalidade. Abstract: Multi- and Many-Evolutionary Algorithms have been applied to derive products for the variability testing of Software Product Lines (SPLs). This problem refers to the selection of an adequate set of products to test a SPL, since to test all the existing products is infeasible. The problem is impacted by many factors, such as the number of products to be tested, testing criteria to be satisfied, and efficacy to reveal faults. We can see that many conflicting objective functions need to be optimized at the same time. However, some problems emerge when the number of objectives to be optimized increases, for example, the solutions generated by the optimization algorithms become incomparable, designing a Pareto-front in this context requires a large number of solutions, and the visualization of such solutions requires special techniques. Several techniques are proposed in the literature to tackle this problem, such as decomposition and algorithms based on indicators. Among them, the algorithms based on dimensionality reduction and algorithms based on the user preferences are widely used. Even though used in different approaches in a separated way, there are no studies in the literature that investigate the usage of dimensionality reduction and preference-based algorithms in a combined way. In light of this, this work proposes an approach called MaDRUP, a Many-objective Optimization with Dimensionality Reduction based on User Preferences for reducing the number of objectives to be optimized based on preferences stated during the solution generation process. This approach has as main goal the generation of a reduced set of solutions taking less execution time but competitively maintaining the quality attributes in comparison with other algorithms. To evaluate the applicability of the proposed approach, we instantiated MaDRUP with the NSGA-II algorithm. Such instantiation is called COR-NSGA-II (Confidence-based Objective Reduction NSGA-II). COR-NSGA-II defines for each objective function a confidence-level calculated with the user preferences provided interactively. The objectives with higher values of confidence are removed from the next algorithm execution. For assessing the feasibility of COR-NSGA-II, a tool called Nautilus was also implemented, and experiments were conducted by using six different SPLs, two types of reference points representing the user preferences, five algorithms, and two scenarios to simulate different user profiles. The results show COR-NSGA-II outperforms most of the evaluated algorithms generating a lower number of solutions with a lower execution time. A qualitative analysis was also performed with a set of 12 potential users. The results show that for such users, the task of selecting a solution generated by COR-NSGA-II is easier than to select a solution generated by other algorithms. Keywords: software product Line testing. search-based software engineering. preference-based algorithms. dimensionality reduction.

Les systèmes logiciels sont devenus de plus en plus répondus et importants dans notre société. Ainsi, il y a un besoin constant de logiciels de haute qualité. Pour améliorer la qualité de logiciels, l’une des techniques les plus utilisées est le refactoring qui sert à améliorer la structure d'un programme tout en préservant son comportement externe. Le refactoring promet, s'il est appliqué convenablement, à améliorer la compréhensibilité, la maintenabilité et l'extensibilité du logiciel tout en améliorant la productivité des programmeurs. En général, le refactoring pourra s’appliquer au niveau de spécification, conception ou code. Cette thèse porte sur l'automatisation de processus de recommandation de refactoring, au niveau code, s’appliquant en deux étapes principales: 1) la détection des fragments de code qui devraient être améliorés (e.g., les défauts de conception), et 2) l'identification des solutions de refactoring à appliquer. Pour la première étape, nous traduisons des régularités qui peuvent être trouvés dans des exemples de défauts de conception. Nous utilisons un algorithme génétique pour générer automatiquement des règles de détection à partir des exemples de défauts. Pour la deuxième étape, nous introduisons une approche se basant sur une recherche heuristique. Le processus consiste à trouver la séquence optimale d'opérations de refactoring permettant d'améliorer la qualité du logiciel en minimisant le nombre de défauts tout en priorisant les instances les plus critiques. De plus, nous explorons d'autres objectifs à optimiser: le nombre de changements requis pour appliquer la solution de refactoring, la préservation de la sémantique, et la consistance avec l’historique de changements. Ainsi, réduire le nombre de changements permets de garder autant que possible avec la conception initiale. La préservation de la sémantique assure que le programme restructuré est sémantiquement cohérent. De plus, nous utilisons l'historique de changement pour suggérer de nouveaux refactorings dans des contextes similaires. En outre, nous introduisons une approche multi-objective pour améliorer les attributs de qualité du logiciel (la flexibilité, la maintenabilité, etc.), fixer les « mauvaises » pratiques de conception (défauts de conception), tout en introduisant les « bonnes » pratiques de conception (patrons de conception)., Software systems have become prevalent and important in our society. There is a constant need for high-quality software. Hence, to improve software quality, one of the most-used techniques is the refactoring which improves design structure while preserving the external behavior. Refactoring has promised, if applied well, to improve software readability, maintainability and extendibility while increasing the speed at which programmers can write and maintain their code. In general, refactoring can be performed in various levels such as the requirement, design, or code level. In this thesis, we mainly focus on the source code level where automated refactoring recommendation can be performed through two main steps: 1) detection of code fragments that need to be improved/fixed (e.g., code-smells), and 2) identification of refactoring solutions to achieve this goal. For the code-smells identification step, we translate regularities that can be found in such code-smell examples into detection rules. To this end, we use genetic programming to automatically generate detection rules from examples of code-smells. For the refactoring identification step, a search-based approach is used. The process aims at finding the optimal sequence of refactoring operations that improve software quality by minimizing the number of detected code-smells while prioritizing the most critical ones. In addition, we explore other objectives to optimize using a multi-objective approach: the code changes needed to apply refactorings, semantics preservation, and the consistency with development change history. Hence, reducing code changes allows us to keep as much as possible the initial design. On the other hand, semantics preservation insures that the refactored program is semantically coherent, and that it models correctly the domain-semantics. Indeed, we use knowledge from historical code change to suggest new refactorings in similar contexts. Furthermore, we introduce a novel multi-objective approach to improve software quality attributes (i.e., flexibility, maintainability, etc.), fix “bad” design practices (i.e., code-smells) while promoting “good” design practices (i.e., design patterns).

Context: Despite model querying being an important practical problem, existing solutions lack either usability, expressiveness, or generality. The Visual Model Query Language (VMQL) is a query byexample solution created to satisfy these requirements simultaneously. Objective: In the present paper we study whether VMQL queries can be executed in an e_cient way, such that VMQL is suitable for ad-hoc model querying in practical settings involving large models. Method: We study VMQL query execution performance on sets of models ranging over a broad spectrum of sizes and degrees of complexity. The models are based on large and realistic case studies. Results: We observe that our approach exhibits competitive performance, while providing superior usability and generality.