Search

Motivation. Large language models (LLMs) have exhibited remarkable proficiency in diverse software engineering (SE) tasks. Handling such tasks typically involves acquiring foundational coding knowledge on large, general-purpose datasets during a pre-training phase, and subsequently refining on smaller, task-specific datasets as part of a fine-tuning phase. Problem statement. Data leakage is a well-known issue in training of machine learning models. A manifestation of this issue is the intersection of the training and testing splits. While intra-dataset code duplication examines this intersection within a given dataset and has been addressed in prior research, inter-dataset code duplication, which gauges the overlap between different datasets, remains largely unexplored. If this phenomenon exists, it could compromise the integrity of LLM evaluations because of the inclusion of fine-tuning test samples that were already encountered during pre-training, resulting in inflated performance metrics. Contribution. This paper explores the phenomenon of inter-dataset code duplication and its impact on evaluating LLMs across diverse SE tasks. Study design. We conduct an empirical study using the CSN dataset, a widely adopted pre-training dataset, and five fine-tuning datasets used for various SE tasks. We first identify the intersection between the pre-training and fine-tuning datasets using a deduplication process. Then, we fine-tune four models pre-trained on CSN to evaluate their performance on samples encountered during pre-training and those unseen during that phase. Results. Our findings reveal a potential threat to the evaluation of various LLMs across multiple SE tasks, stemming from the inter-dataset code duplication phenomenon. Moreover, we demonstrate that this threat is accentuated by factors like the LLM's size and the chosen fine-tuning technique.

'El 23 de julio de 1923 tuvo lugar la fundación del Banco de la Republica. Su diseño siguió las recomendaciones de la misión dirigida por Edwin Kemmerer, el asesor norteamericano que por esa misma época lidero el establecimiento de bancos centrales con características similares en todos los países andinos de Suramérica: Ecuador, Perú, Bolivia y Chile. El centenario de esa fecha constituye una gran oportunidad para reflexionar sobre el papel que ha cumplido el Banco en la historia económica colombiana durante el siglo transcurrido desde su fundación. Gracias al apoyo del Banco de Desarrollo de América Latina-CAF, y de su presidente Sergio Díaz-Granados, hemos podido publicar en este libro cinco ensayos de gran relevancia para dicha reflexión. Los primeros cuatro fueron elaborados por autores que aparte de ser ampliamente reconocidos como historiadores, han sido también actores importantes de la política económica de Colombia y han estado vinculados directamente al Banco y a su Junta Directiva. José Antonio Ocampo presidió la Junta en dos oportunidades como ministro de Hacienda, entre agosto de 1996 y noviembre de 1997 y entre agosto de 2022 y abril de 2023, y fue codirector entre 2018 y 2019. Miguel Urrutia fue gerente general durante doce años a partir de 1993 y había sido previamente codirector desde la creación de la Junta Independiente por parte de la Constitución de 1991. Antonio Hernández Gamarra, por su parte, fue codirector entre 1996 y 2001 y Carlos Caballero, quien en este articulo esta acompañado de Pilar Esguerra como coautora, lo fue entre 2001 y 2003 […] El quinto ensayo presenta una visión más global sobre la evolución de la independencia de los bancos centrales de América Latina y su impacto sobre la inflación. Sus autores son Luis Jácome, quien fue presidente del Banco Central del Ecuador y actualmente está vinculado al Fondo Monetario Internacional y a la Universidad de Georgetown, y Samuel Pienknagura, economista sénior del Banco Mundial.' Tomado d, Leonardo Villar Gómez Pilar Esguerra Umaña (editores) José Antonio Ocampo Gaviria Antonio Hernández Gamarra Carlos Caballero Argáez Miguel Urrutia Montoya Samuel Pienknagura Loor Luis Jácome Hidalgo (autores) Ensayos de historia económica Cien años del Banco de la República Ensayos de historia económica Cien años del Banco de la República Ensayos de historia económica Cien años del Banco de la República Leornando Villar Gómez Pilar Esguerra Umaña (editores) Ensayos de historia económica. Cien años del Banco de la República / José Antonio Ocampo Gaviria [y otros seis]. -- Edición Leonardo Villar Gómez, Pilar Esguerra Umaña. -- Bogotá: Banco de la República, 2023. 502 páginas; 24 cm. Incluye bibliografía. 1. Banco de la República (Bogotá) - Historia - 1923-2023 2. Bancos centrales - Historia -Colombia 4. Historia económica - Colombia 5. Crisis económica - Historia - Colombia 6. Inflación - América Latina 7. Colombia - Política monetaria I. Ocampo Gaviria, José Antonio, 1952- , autor II. Villar Gómez, Leonardo, editor III. Esguerra Umaña, Pilar, Editora 330.9861 cd 22 ed. Noviembre de 2023 ISBN impreso 978-958-664-473-0 https://doi.org/10.32468/Ebook.664-473-0 Fotografía: cortesía revista Cromos Bogotá, marzo de 1923 Diseño de portada e interiores Adrián Díaz Corrección de estilo Fernando Carretero Revisión de pruebas Luis Eduardo Castellanos Carolina Torres Carol Alvear Vega Sección de Gestión de Publicaciones Departamento de Servicios Administrativos Banco de la República Armada electrónica y finalización de arte Asesores Culturales S. A. S. Impresión La Imprenta Editores S. A. Las opiniones expresadas en los capítulos de este libro no comprometen al Banco de la República ni a su Junta Directiva. Contenido xi Prólogo Leonardo Villar Gómez 1 Capítulo 1 Una historia de las crisis económicas de Colombia José Antonio Ocampo 1 1.1 Introducción 3 1.2 Las crisis económicas desde comienzos del siglo XX 10 1.3 Los antecedentes: las crisis durante la era de las exportaciones 1

The objective of pre-trained language models is to learn contextual representations of textual data. Pre-trained language models have become mainstream in natural language processing and code modeling. Using probes, a technique to study the linguistic properties of hidden vector spaces, previous works have shown that these pre-trained language models encode simple linguistic properties in their hidden representations. However, none of the previous work assessed whether these models encode the whole grammatical structure of a programming language. In this paper, we prove the existence of a syntactic subspace, lying in the hidden representations of pre-trained language models, which contain the syntactic information of the programming language. We show that this subspace can be extracted from the models' representations and define a novel probing method, the AST-Probe, that enables recovering the whole abstract syntax tree (AST) of an input code snippet. In our experimentations, we show that this syntactic subspace exists in five state-of-the-art pre-trained language models. In addition, we highlight that the middle layers of the models are the ones that encode most of the AST information. Finally, we estimate the optimal size of this syntactic subspace and show that its dimension is substantially lower than those of the models' representation spaces. This suggests that pre-trained language models use a small part of their representation spaces to encode syntactic information of the programming languages.

The availability of shared software models provides opportunities for reusing, adapting and learning from them. Public models are typically stored in a variety of locations, including model repositories, regular source code repositories, web pages, etc. To profit from them developers need effective search mechanisms to locate the models relevant for their tasks. However, to date, there has been little success in creating a generic and efficient search engine specially tailored to the modelling domain. In this paper we present MAR, a search engine for models. MAR is generic in the sense that it can index any type of model if its meta-model is known. MAR uses a query-by-example approach, that is, it uses example models as queries. The search takes the model structure into account using the notion of bag of paths, which encodes the structure of a model using paths between model elements and is a representation amenable for indexing. MAR is built over HBase using a specific design to deal with large repositories. Our benchmarks show that the engine is efficient and has fast response times in most cases. We have also evaluated the precision of the search engine by creating model mutants which simulate user queries. A REST API is available to perform queries and an Eclipse plug-in allows end users to connect to the search engine from model editors. We have currently indexed more than 50.000 models of different kinds, including Ecore meta-models, BPMN diagrams and UML models. MAR is available at http://mar-search.org.

Max Andrews (1975) and Mariana Cánepa Luna (1977) are the co-founders of Latitudes, an independent curatorial office based in Barcelona, which works internationally within the field of contemporary art. Informed by an initial focus on ecology, Latitudes is especially interested in site, process and context. In this text, originally written in English, Latitudes gives us the clues to understand the first experimental videos by the artist José Antonio Hernández-Díez (b. Caracas, Venezuela, 1964), along with his other early works using screens and vitrines. The exhibition shows what the artist termed as a “New Christian Iconography,” and as the curators claim, “neither ironic nor profane, this notion led Hernández-Díez to create a series of macabre works that hinge on death, consciousness and resurrection.” This Quadern portàtil is published on the occasion of the exhibition I will fear no evil, by José Antonio Hernández-Díez at MACBA’s Convent dels Àngels from 18 March to 26 June 2016., https://www.librarystack.org/jose-antonio-hernandez%e2%80%90diez-no-temere-mal-alguno/?ref=unknown

Max Andrews (1975) and Mariana Cánepa Luna (1977) are the co-founders of Latitudes, an independent curatorial office based in Barcelona, which works internationally within the field of contemporary art. Informed by an initial focus on ecology, Latitudes is especially interested in site, process and context. In this text, originally written in English, Latitudes gives us the clues to understand the first experimental videos by the artist José Antonio Hernández-Díez (b. Caracas, Venezuela, 1964), along with his other early works using screens and vitrines. The exhibition shows what the artist termed as a “New Christian Iconography,” and as the curators claim, “neither ironic nor profane, this notion led Hernández-Díez to create a series of macabre works that hinge on death, consciousness and resurrection.” This Quadern portàtil is published on the occasion of the exhibition I will fear no evil, by José Antonio Hernández-Díez at MACBA’s Convent dels Àngels from 18 March to 26 June 2016., https://www.librarystack.org/jose-antonio-hernandez%e2%80%90diez-no-temere-mal-alguno/?ref=unknown