Search

This study critically examines the resource efficiency and performance of Shallow Machine Learning (SML) methods versus Large Language Models (LLMs) in text classification tasks by exploring the balance between accuracy and environmental sustainability. We introduce a novel optimization strategy that prioritizes computational efficiency and ecological impact alongside traditional performance metrics leveraging Automated Machine Learning (AutoML). Our analysis reveals that while the pipelines we developed did not surpass state-of-the-art (SOTA) models regarding raw performance, they offer a significantly reduced carbon footprint. We discovered SML optimal pipelines with competitive performance and up to 70 times less carbon emissions than hybrid or fully LLM pipelines, such as standard BERT and DistilBERT variants. Similarly, we obtain hybrid pipelines (using SML and LLMs) with between 20% and 50% reduced carbon emissions compared to fine-tuned alternatives and only a marginal decrease in performance. This research challenges the prevailing reliance on computationally intensive LLMs for NLP tasks and underscores the untapped potential of AutoML in sculpting the next wave of environmentally conscious AI models., Este estudio analiza críticamente la eficiencia de recursos y el rendimiento de los métodos de Aprendizaje Automático Superficial (SML) frente a los Grandes Modelos de Lenguaje (LLM) en tareas de clasificación de texto explorando el equilibrio entre precisión y sostenibilidad medioambiental. Se introduce una novedosa estrategia de optimización que prioriza la eficiencia computacional y el impacto ecológico junto con las métricas de rendimiento tradicionales aprovechando el Aprendizaje Automático de Máquinas (AutoML). El análisis revela que, si bien los pipelines desarrollados no superan a los modelos SOTA m´as avanzados en cuanto a rendimiento bruto, reducen significativamente la huella de carbono. Se descubrieron pipelines óptimos de SML con un rendimiento competitivo y hasta 70 veces menos emisiones de carbono que pipelines híbridos o totalmente LLM, como las variantes estándar de BERT y DistilBERT. Del mismo modo, obtenemos pipelines híbridos (que incorporan SML y LLM) con entre un 20% y un 50% menos de emisiones de carbono en comparación con las alternativas fine-tuneadas y sólo una disminución marginal del rendimiento. Esta investigación pone en cuestión la dependencia predominante de los LLM de alta carga computacional para tareas de PLN y subraya el potencial sin explotar de AutoML para esculpir la próxima oleada de modelos de IA con conciencia medioambiental.

Las redes sociales han revolucionado la creación y difusión de contenido, permitiendo a los usuarios no solo consumir sino también producir y compartir información a una escala sin precedentes. Esto ha resultado en una enorme cantidad de datos disponibles en la web, que incluyen no solo hechos y datos objetivos, sino también opiniones, emociones y valoraciones personales. Esta información subjetiva es de gran valor, pero su aprovechamiento efectivo no es sencillo debido a la escala y diversidad de los datos generados en las redes sociales, lo que hace prácticamente imposible un análisis manual. En este contexto, surge la necesidad de utilizar herramientas automatizadas para la monitorización, extracción, procesamiento y análisis de información subjetiva a gran escala en tiempo real. Este es el objetivo del análisis de sentimientos, un área dentro del procesamiento del lenguaje natural (PLN) que se enfoca en identificar y clasificar opiniones expresadas en textos de forma automática. Sin embargo, el trabajo con información subjetiva en redes sociales a gran escala presenta varios desafíos que subrayan la importancia de investigar y desarrollar nuevas metodologías que sean eficientes en el uso de recursos computacionales, para hacer el análisis de sentimientos más accesible y democratizado. Los objetivos de investigación de este trabajo se centran en dos aspectos principales. El primero consiste en diseñar y evaluar una metodología de análisis de sentimientos que sea eficaz y al mismo tiempo no consuma excesivos recursos computacionales. Se busca determinar si es posible obtener una efectividad aceptable con una utilización reducida de recursos y evaluar en qué contextos es más eficaz. Para ello se propone una propuesta que utiliza aprendizaje automático supervisado donde las unidades de información son skipgrams, términos de más de una palabra que no aparecen adyacentes explícitamente en los textos. La innovación de la propuesta reside en el método de generación cont

This article argues that digital educational content should be structured as knowledge graphs (KGs). Unlike traditional repositories such as Moodle, a KG offers a more flexible representation of the relationships between concepts, facilitating intuitive navigation and discovery of connections. In addition, it integrates effectively with Large Language Models, enhancing personalized explanations, answers, and recommendations. This article studies different proposals based on semantics and knowledge modelling to determine the most appropriate ways to strengthen intelligent educational technologies.

The incorporation of machine learning algorithms into high-risk decision-making tasks has raised some alarms in the scientific community. Research shows that machine learning-based technologies can contain biases that cause unfair decisions for certain population groups. The fundamental danger of ignoring this problem is that machine learning methods can not only reflect the biases present in our society but could also amplify them. This article presents the design and validation of a technology to assist the fair automation of classification problems. In essence, the proposal is based on taking advantage of the intermediate solutions generated during the resolution of classification problems through using Auto-ML tools, in particular, AutoGOAL, to create unbiased/fair classifiers. The technology employs a multi-objective optimization search to find the collection of models with the best trade-offs between performance and fairness. To solve the optimization problem, we introduce a combination of Probabilistic Grammatical Evolution Search and NSGA-II. The technology was evaluated using the Adult dataset from the UCI repository, a common benchmark in related research. Results were compared with other published results in scenarios with single and multiple fairness definitions. Our experiments demonstrate the technology's ability to automate classification tasks while incorporating fairness constraints. Additionally, our method achieves competitive results against other bias mitigation techniques. A notable advantage of our approach is its minimal requirement for machine learning expertise, thanks to its Auto-ML foundation. This makes the technology accessible and valuable for advancing fairness in machine learning applications. The source code is available online for the research community.

This paper presents KD SENSO-MERGER, a novel Knowledge Discovery (KD) architecture that is capable of semantically integrating heterogeneous data from various sources of structured and unstructured data (i.e. geolocations, demographic, socio-economic, user reviews, and comments). This goal drives the main design approach of the architecture. It works by building internal representations that adapt and merge knowledge across multiple domains, ensuring that the knowledge base is continuously updated. To deal with the challenge of integrating heterogeneous data, this proposal puts forward the corresponding solutions: (i) knowledge extraction, addressed via a plugin-based architecture of knowledge sensors; (ii) data integrity, tackled by an architecture designed to deal with uncertain or noisy information; (iii) scalability, this is also supported by the plugin-based architecture as only relevant knowledge to the scenario is integrated by switching-off non-relevant sensors. Also, we minimize the expert knowledge required, which may pose a bottleneck when integrating a fast-paced stream of new sources. As proof of concept, we developed a case study that deploys the architecture to integrate population census and economic data, municipal cartography, and Google Reviews to analyze the socio-economic contexts of educational institutions. The knowledge discovered enables us to answer questions that are not possible through individual sources. Thus, companies or public entities can discover patterns of behavior or relationships that would otherwise not be visible and this would allow extracting valuable information for the decision-making process.

The spread of fake news (FN) has attracted attention from disciplines ranging from social sciences to Artificial Intelligence. This work is novel because it explores the news-sharing behaviour of social-media users, focussing on those that spread FN, rather than the psychological motivations behind them. The 14-item Risky News-Sharing Quotient (RNSQ) was developed and Exploratory Factor Analysis discovered three relevant factors: (i) news-sharing behaviour that contributes to debunking FN; (ii) news-sharing frequency and attitudes to sharing; and (iii) news-sharing behaviour that contributes to the spread of FN. The study, conducted among university students, found that 75% reported risky news-sharing behaviour that spreads FN. No link was found between perceiving FN as a problem and debunking it. Moreover, 83% of survey participants were unable to identify a FN story. Overall, the findings suggest an inability to apply knowledge of the relevant FN detection strategies to debunk FN, but importantly an apparent lack of motivation to check the veracity of a news story. From these conclusions, better-informed educational intervention strategies can be implemented to address the FN problem in-situ, such as promoting the importance of responsible news-sharing by raising awareness of how the spread of FN can impede the proper functioning of societies.

Recent research has shown that the unaware automation of high-risk decision-making tasks can result in unfair decisions being made. The most common approaches to address this problem adopt definitions of fairness based on protected attributes. Precise annotation of protected attributes enables the application of bias mitigation techniques to commonly unlabeled kinds of data (e.g., images, text, etc.). This paper proposes a framework to automatically annotate protected attributes in data collections. The framework focuses on providing a single interface to annotate protected attributes of different types (e.g., gender, race, etc.) and from different kinds of data. Internally, the framework coordinates multiple sensors to produce the final annotation. Several sensors for textual data are proposed. An optimization search technique is designed to tune the framework to specific domains. Additionally, a small dataset of movie reviews —annotated with gender and sentiment— was created. The evaluation in datasets of texts from diverse domains shows the quality of the annotations and their effectiveness to be used as a proxy to estimate fairness in datasets and machine learning models. The source code is available online for the research community.

The objective of Geo-IA is to research, design and implement a Geo-Smart Artificial Intelligence technology platform for public and private business organizations. The GeoIA project presents a geolocation platform that integrates technological innovation to support a strategy for the creation of a Smart Territories. To do this, Text Mining, Machine Learning (including deep learning) and Natural Language Processing technologies are deployed. The functionality of the geolocation platform is to analyze, integrate, share data, visualize and represent territorial indicators, with the aim of facilitating the monitoring and fulfillment of territorial strategies. In short, GeoIA promotes interoperability between public administration bodies and also provides citizens with mechanisms to access information of interest, where the magnitude of the integrated and interrelated data permits. GeoIA also provides digital knowledge (tools, linked information, semantics, virtual assistants) for use by public administrations to enhance their decision making through greater knowledge of the environment and to improve services to citizens.

Los modelos de aprendizaje automático están siendo ampliamente utilizados en múltiples áreas de la vida humana. Tradicionalmente, se han aplicado en reconocimiento de voz, detección de rostros, clasificación de imágenes, sistemas de recomendación, etc. Con la reciente revolución de los modelos generativos, la popularidad de los chatbots conversacionales se ha disparado. Esto ha dado lugar a que los modelos de aprendizaje automático se utilicen cada vez más para abordar tareas para las que no estaban específicamente capacitados. El prompt engineering ha permitido que personas no expertas en aprendizaje automático (que comúnmente tampoco están familiarizadas con los problemas subyacentes al uso de modelos de aprendizaje automático para hacer predicciones) automaticen ciertas tareas. La incorporación de algoritmos de aprendizaje automático en tareas de toma de decisiones de alto riesgo ha levantado algunas alertas en la comunidad científica. Las tareas de toma de decisiones de alto riesgo denotan aquellas tareas que pueden tener un gran impacto en las vidas de las personas sobre quienes se toman las decisiones. Por ejemplo, se han utilizado modelos para decidir si una persona es contratada o no, si se le concede un préstamo, si se acepta una solicitud de cobertura ampliada de seguridad sanitaria y para predecir la probabilidad de reincidencia en un delito. Estudios han demostrado que la automatización inconsciente de este tipo de tareas contiene sesgos, lo cual provoca que decisiones injustas sean tomadas sobre determinados grupos de población. El peligro fundamental de ignorar este problema es que los métodos de aprendizaje automático podrían no sólo reflejar los sesgos presentes en nuestra sociedad, sino que también amplificarlos. Esta tesis presenta el diseño y validación de una tecnología para asistir la automatización justa de problemas de clasificación. En esencia, la propuesta se basa en diseñar una tecnología que saque provecho de las soluciones intermedias gen

This work presents a standardised approach to create datasets for Science and Technology Parks (STPs), facilitating future analysis of STP characteristics, trends and performance. STPs are the most representative examples of innovation ecosystems. The ETL (extraction-transformation-load) structure was adapted to a global field study of STPs. A selection stage and quality check were incorporated, and the methodology was applied to Spanish STPs. This study applies diverse techniques such as expert labelling and information extraction which uses language technologies. A novel methodology for building quality and standardised STP datasets was designed and applied to a Spanish STP case study with 49 STPs. An updatable dataset and a list of the main features impacting STPs are presented. Twenty-one (n = 21) core features were refined and selected, with fifteen of them (71.4 %) being robust enough for developing further quality analysis. The methodology presented integrates different sources with heterogeneous information that is often decentralised, disaggregated and in different formats: excel files, and unstructured information in HTML or PDF format. The existence of this updatable dataset and the defined methodology will enable powerful AI tools to be applied that focus on more sophisticated analysis, such as taxonomy, monitoring, and predictive and prescriptive analytics in the innovation ecosystems field.

This study contributes to English for Specific Purposes (ESP) pedagogy by providing an updated examination of conceptual metaphor types (CMTs) employed in financial markets reporting. The investigation delves into the prevalence of CMTs in both social media and mainstream media contexts. Robust patterns were identified to distinguish CMT usage between mainstream and social media by leveraging big-scale data analysis, encompassing 38.6 million documents from The Financial Times, The Wall Street Journal, Twitter and Reddit. The data collection spans fifteen months during the COVID-19 pandemic (January 2020–March 2021), marked by socioeconomic upheaval and coinciding with a surge in retail investors using low or no-cost mobile financial trading applications. Examining the proportion of CMTs reveals that war/combat, markets animate, markets inanimate, and health metaphor types have strong associations with both social and mainstream media. The gambling CMT is predominantly linked to social media. In terms of metaphor density, the results indicate a higher concentration in social media compared to mainstream media. Texts sourced from social media, characterized by greater conciseness, emerge as a potential communication barrier. The findings underscore the importance of incorporating authentic texts from social media into specialized language courses, thus enhancing language learning experiences in the domain of financial markets reporting.

Large language models have shown impressive performance in Natural Language Processing tasks, but their black box characteristics render the explainability of the model’s decision difficult to achieve and the integration of semantic knowledge. There has been a growing interest in combining external knowledge sources with language models to address these drawbacks. This paper, OntoLM, proposes a novel architecture combining an ontology with a pre-trained language model to classify biomedical entities in text. This approach involves constructing and processing graphs from ontologies and then using a graph neural network to contextualize each entity. Next, the language model and the graph neural network output are combined into a final classifier. Results show that OntoLM improves the classification of entities in medical texts using a set of categories obtained from the Unified Medical Language System. We can create more traceable natural language processing architectures using ontology graphs and graph neural networks., Los grandes modelos de lenguaje han mostrado un rendimiento impresionante en tareas de Procesamiento del Lenguaje Natural, pero su condición de caja negra hace difícil explicar las decisiones del modelo e integrar conocimiento semántico. Existe un interés creciente en combinar fuentes de conocimiento externas con LLMs para solventar estos inconvenientes. En este artículo, proponemos OntoLM, una arquitectura novedosa que combina una ontología con un modelo de lenguaje pre-entrenado para clasificar entidades biomédicas en texto. El enfoque propuesto consiste en construir y procesar grafos provenientes de una ontología utilizando una red neuronal de grafos para contextualizar cada entidad. A continuación, combinamos los resultados del modelo de lenguaje y la red neuronal de grafos en un clasificador final. Los resultados muestran que OntoLM mejora la clasificación de entidades en textos médicos utilizando un conjunto de categorías obtenidas de Unified Medical Language System. Utilizando grafos de ontologías y redes neuronales de grafos podemos crear arquitecturas de procesamiento de lenguaje natural más rastreables.

En la actualidad existen múltiples modelos de inteligencia artificial centrados en la detección de entidades nombradas, que son capaces de detectar una amplia variedad de aspectos. En este trabajo, se centran esos aspectos a enfermedades raras, detectándolas en textos del ámbito clínico. Todos esos textos clínicos son resúmenes de documentos científicos publicados en PubMed. De las enfermedades, no solo se detectarán sus nombres en sí, sino que se quieren detectar una amplia variedad de aspectos relacionados con esas enfermedades, como por ejemplo, sus causas, tratamientos, diagnósticos... Todos esos aspectos se clasificarán en una serie de categorías. Las anotaciones del modelo se generarán, en primera instancia, de forma automática, usando la herramienta Metathesaurus, contenida dentro de UMLS, un sistema de lenguaje médico. Metathesaurus contiene más de 3 millones de conceptos, siendo la inmensa mayoría del ámbito clínico. Además, cuenta con una serie de categorías ya definidas, y con los conceptos clasificados en estas categorías. Para cada texto, se cuenta con un archivo txt que contiene el texto y un archivo ann que contiene sus anotaciones. Esas anotaciones se encuentran definidas en formato BRAT, un formato de anotación que permite después visualizarlas de forma fácil, modificarlas y crear nuevas. Para cada anotación, se especifica el inicio, final, la categoría a la que pertenece y las palabras o grupos de palabras sobre las que se aplica. Una vez se cuenta con esas anotaciones, es posible revisarlas manualmente para que el corpus sea de la mayor calidad posible, pero al tener una base ya de anotaciones, esta tarea será más ágil. La clasificación que se debe de realizar es compleja, ya que contiene bastantes categorías, además de que cada palabra (o grupos de palabras) pueden pertenecer a la vez a varias clases, por lo que las anotaciones se pueden superponer tanto de forma estricta (mismo inicio y final) como de forma parcial. Para la obtención del modelo

The large amount of information in digital format that exists today makes it unfeasible to use manual means to acquire the knowledge contained in these documents. Therefore, it is necessary to develop tools that allow us to incorporate this knowledge into a structure that is easy to use by both machines and humans. This paper presents a system that can incorporate the relevant information from a document in any format, structured or unstructured, into a semantic network that represents the existing knowledge in the document. The system independently processes from structured documents based on its annotation scheme to unstructured documents, written in natural language, for which it uses a set of sensors that identifies the relevant information and subsequently incorporates it to enrich the semantic network that is created by linking all the information based on the knowledge discovered.

The role played by science and technology parks (STPs) in technology transfer, industrial innovation, and economic growth is examined in this paper. The accurate monitoring of their evolution and impact is hindered by the lack of uniformity in STP models or goals, and the scarcity of high-quality datasets. This work uses existing terminologies, definitions, and core features of STPs to conduct a multidimensional data analysis that explores and evaluates the 21 core features which describe the key internal factors of an STP. The core features are gathered from a reliable and updatable dataset of Spanish STPs. The methodological framework can be replicated for other STP contexts and is based on descriptive techniques and machine-learning tools. The results of the study provide an overview of the general situation of STPs in Spain, validate the existence and characteristics of three types of STPs, and identify the typical features of STPs. Moreover, the prototype STP can be used as a benchmark so that other STPs can identify the features that need to be improved. Finally, this work makes it possible to carry out classifications of STPs, in addition to prediction and decision making for innovation ecosystems.

The project T2Know presents the use of natural language processing technologies for the creation of a semantic platform of scientific documents via knowledge graphs. This knowledge graph will link relevant parts of each document with those of other documents in such a way that trend analysis and recommendations can be achieved. The goals addressed within the scope of this project include entity recognizers development, profile definition and documents linkage through the use of transformers technologies. As a result, the relevant parts of the documents to be extracted are related not only to the title and affiliation of the authors, but also to article topics such as references, which are also considered relevant parts of the scientific article.

Generative language models achieve the state of the art in many tasks within natural language processing (NLP). Although these models correctly capture syntactic information, they fail to interpret knowledge (semantics). Moreover, the lack of interpretability of these models promotes the use of other technologies as a replacement or complement to generative language models. This is the case with research focused on incorporating knowledge by resorting to knowledge bases mainly in the form of graphs. The generation of large knowledge graphs is carried out with unsupervised or semi-supervised techniques, which promotes the validation of this knowledge with the same type of techniques due to the size of the generated databases. In this review, we will explain the different techniques used to test and infer knowledge from graph structures with machine learning algorithms. The motivation of validating and inferring knowledge is to use correct knowledge in subsequent tasks with improved embeddings.

El modelado de skipgrams es una técnica para la generación de términos multi-palabra que conserva parte de la secuencialidad y flexibilidad del lenguaje. Sin embargo, en algunos casos el número de skipgrams generados puede ser excesivo a medida que se aumenta la distancia entre palabras. Además, esta distancia no suele ser tenida en cuenta a la hora de valorar los términos que se generan. En este trabajo proponemos una técnica para la generación y filtrado eficientes de skipgrams y un esquema de pesado que tiene en cuenta la distancia entre los términos, dando más importancia a aquellos más cercanos. Aplicaremos y evaluaremos estas propuestas en la tarea de análisis de sentimientos., Skipgram modelling is a technique for generating multi-word terms that preserves some of the sequentiality and flexibility of the language. However, in some cases the number of skipgrams generated may become excessive as the distance between words increases. Moreover, this distance is often not taken into account when evaluating the terms that are generated. In this paper we propose a technique for efficient skipgram generation and filtering, and a weighing scheme that takes into account the distance between terms, giving more importance to those closer. We will apply and evaluate these proposals in the task of sentiment analysis.

Discovering the main features of virality patterns in Twitter is the focus of this research. Five trending topics related to the COVID-19 pandemic were selected for the study, with Spanish as the target language. To carry out the discovery of virality patterns, we applied opinion mining techniques that enable us to structure the information based on the polarity of the messages and the emotions they contain. After transforming the information from an unstructured textual representation to a structured one, data mining techniques were applied, specifically association rules mining. Message patterns with the highest virality (high shares and high likes), and at the same time the most relevant characteristics of the patterns with less impact were extracted. After an exhaustive analysis of the most relevant non-redundant rules, it can be concluded that messages with a high-negative polarity and a very high emotional charge, especially emotions that have intensified with the COVID-19 pandemic, such as fear, sadness, anger and surprise are more likely to go viral in social media. By contrast, messages with little news coverage in the media, few authors, and the absence of surprise are relevant features when it comes to seeing messages with very low dissemination in social media.

Automatic Machine Learning (Auto-ML) tools enable the automatic solution of real-world problems through machine learning techniques. These tools tend to be more time consuming than standard machine learning libraries, therefore, exploiting all the available resources to the full is a valuable feature. This paper presents a two-phase optimization system for solving classification problems. The system is designed to produce more robust classifiers by exploiting the different architectures that are generated while solving classification problems with Auto-ML tools, particularly AutoGOAL. In the first phase, the system follows a probabilistic strategy to find the best combination of algorithms and hyperparameters to generate a collection of base models according to certain diversity criteria; and in the second, it follows similar Auto-ML strategies to ensemble those models. The HAHA 2019 challenge corpus and the Adult dataset were used to evaluate the system. The experimental results show that: i) a better solution can be built by ensembling a subset of the already tested models; ii) the performance of ensemble methods depends on the collection of base models used; and, iii) ensuring diversity using the double-fault measure produces better results than the disagreement measure. The source code is available online for the research community.

El aprendizaje automático es un campo de la inteligencia artificial que ha ganado un reciente interés en todas las áreas de la industria, motivado fundamentalmente por el acelerado crecimiento de las capacidades de cómputo y la disponibilidad de datos. Sin embargo, una de las principales dificultades para su aplicación es la necesidad de expertos que conozcan los detalles internos de los múltiples modelos que pueden ser utilizados. En este contexto ha surgido un nuevo campo de estudio, denominado AutoML (Automated Machine Learning), que facilita la utilización de estas técnicas por expertos de otros dominios. Este artículo presenta una propuesta concreta de un sistema —AutoGOAL— que ha sido diseñada para resolver problemas de aprendizaje automático de variada naturaleza. Además, se realiza una breve comparación entre sistemas existentes de relevancia en el campo. La propuesta es competitiva con herramientas del estado del arte en problemas clásicos de aprendizaje, a la vez que puede desplegarse, sin esfuerzo adicional, en dominios más complejos, como el procesamiento de lenguaje natural. AutoGOAL constituye un paso más hacia la democratización del aprendizaje automático para usuarios no expertos en el tema., Machine Learning is a field of Artificial Intelligence that has gained recent interest in all areas of the industry, motivated primarily by the accelerated growth of computer capabilities and data availability. However, one of the main difficulties for its application is the need for experts who know the internal details of the multiple models that can be used. In this context, a new field of study has emerged, AutoML (Automated Machine Learning), which facilitates the use of these techniques by experts from other domains. This paper presents a concrete proposal of a system —AutoGOAL— which has been designed to solve machine learning problems of various kinds. In addition, a brief comparison is made between relevant existing systems in the field. The proposal is competitive with state-of-the-art tools in classic machine learning problems, and it can be seamlessly deployed in more complex domains, such as natural language processing. AutoGOAL is another step towards the democratization of machine learning for non-expert users.