Search

Large Language Models (LLMs), characterized by being trained on broad amounts of data in a self-supervised manner, have shown impressive performance across a wide range of tasks. Indeed, their generative abilities have aroused interest on the application of LLMs across a wide range of contexts. However, neural networks in general, and LLMs in particular, are known to be vulnerable to adversarial attacks, where an imperceptible change to the input can mislead the output of the model. This is a serious concern that impedes the use of LLMs on high-stakes applications, such as healthcare, where a wrong prediction can imply serious consequences. Even though there are many efforts on making LLMs more robust to adversarial attacks, there are almost no works that study \emph{how} and \emph{where} these vulnerabilities that make LLMs prone to adversarial attacks happen. Motivated by these facts, we explore how to localize and understand vulnerabilities, and propose a method, based on Mechanistic Interpretability (MI) techniques, to guide this process. Specifically, this method enables us to detect vulnerabilities related to a concrete task by (i) obtaining the subset of the model that is responsible for that task, (ii) generating adversarial samples for that task, and (iii) using MI techniques together with the previous samples to discover and understand the possible vulnerabilities. We showcase our method on a pretrained GPT-2 Small model carrying out the task of predicting 3-letter acronyms to demonstrate its effectiveness on locating and understanding concrete vulnerabilities of the model.

Background: Attention-Deficit/Hyperactivity Disorder (ADHD) is a prevalent neurodevelopmental disorder that significantly impacts various key aspects of life, requiring accurate diagnostic methods. Electroencephalogram (EEG) signals are used in diagnosing ADHD, but proper preprocessing is crucial to avoid noise and artifacts that could lead to unreliable results. Method: This study utilized a public EEG dataset from children diagnosed with ADHD and typically developing (TD) children. Four preprocessing techniques were applied: no preprocessing (Raw), Finite Impulse Response (FIR) filtering, Artifact Subspace Reconstruction (ASR), and Independent Component Analysis (ICA). EEG recordings were segmented, and features were extracted and selected based on statistical significance. Classification was performed using Machine Learning models, as XGBoost, Support Vector Machine, and K-Nearest Neighbors. Results: The absence of preprocessing leads to artificially high classification accuracy due to noise. In contrast, ASR and ICA preprocessing techniques significantly improved the reliability of results. Segmenting EEG recordings revealed that later segments provided better classification accuracy, likely due to the manifestation of ADHD symptoms over time. The most relevant EEG channels were P3, P4, and C3. The top features for classification included Kurtosis, Katz fractal dimension, and power spectral density of Delta, Theta, and Alpha bands. Conclusions: Effective preprocessing is essential in EEG-based ADHD diagnosis to prevent noise-induced biases. This study identifies crucial EEG channels and features, providing a foundation for further research and improving ADHD diagnostic accuracy. Future work should focus on expanding datasets, refining preprocessing methods, and enhancing feature interpretability to improve diagnostic accuracy and model robustness for clinical use.

Transformer-based language models are treated as black-boxes because of their large number of parameters and complex internal interactions, which is a serious safety concern. Mechanistic Interpretability (MI) intends to reverse-engineer neural network behaviors in terms of human-understandable components. In this work, we focus on understanding how GPT-2 Small performs the task of predicting three-letter acronyms. Previous works in the MI field have focused so far on tasks that predict a single token. To the best of our knowledge, this is the first work that tries to mechanistically understand a behavior involving the prediction of multiple consecutive tokens. We discover that the prediction is performed by a circuit composed of 8 attention heads (~5% of the total heads) which we classified in three groups according to their role. We also demonstrate that these heads concentrate the acronym prediction functionality. In addition, we mechanistically interpret the most relevant heads of the circuit and find out that they use positional information which is propagated via the causal mask mechanism. We expect this work to lay the foundation for understanding more complex behaviors involving multiple-token predictions.

Information visualization plays a key role in business intelligence analytics. With ever larger amounts of data that need to be interpreted, using the right visualizations is crucial in order to understand the underlying patterns and results obtained by analysis algorithms. Despite its importance, defining the right visualization is still a challenging task. Business users are rarely experts in information visualization, and they may not exactly know the most adequate visualization tools or patterns for their goals. Consequently, misinterpreted graphs and wrong results can be obtained, leading to missed opportunities and significant losses for companies. The main problem underneath is a lack of tools and methodologies that allow non-expert users to define their visualization and data analysis goals in business terms. In order to tackle this problem, we present an iterative goal-oriented approach based on the i* language for the automatic derivation of data visualizations. Our approach links non-expert user requirements to the data to be analyzed, choosing the most suited visualization techniques in a semi-automatic way. The great advantage of our proposal is that we provide non-expert users with the best suited visualizations according to their information needs and their data with little effort and without requiring expertise in information visualization.

Choosing the right Visualization techniques is critical in Big Data Analytics. However, decision makers are not experts on visualization and they face up with enormous difficulties in doing so. There are currently many different (i) Big Data sources and also (ii) many different visual analytics to be chosen. Every visualization technique is not valid for every Big Data source and is not adequate for every context. In order to tackle this problem, we propose an approach, based on the Model Driven Architecture (MDA) to facilitate the selection of the right visual analytics to non-expert users. The approach is based on three different models: (i) a requirements model based on goal-oriented modeling for representing information requirements, (ii) a data representation model for representing data which will be connected to visualizations and, (iii) a visualization model for representing visualization details regardless of their implementation technology. Together with these models, a set of transformations allow us to semi-automatically obtain the corresponding implementation avoiding the intervention of the non-expert users. In this way, the great advantage of our proposal is that users no longer need to focus on the characteristics of the visualization, but rather, they focus on their information requirements and obtain the visualization that is better suited for their needs. We show the applicability of our proposal through a case study focused on a tax collection organization from a real project developed by the Spin-off company Lucentia Lab.

The identification of molecules in exoplanetary atmospheres is only possible thanks to the availability of high-resolution molecular spectroscopic data. However, due to its intensive and time-consuming generation process, at present, only on order 100 molecules have high-resolution spectroscopic data available, limiting new molecular detections. Using routine quantum chemistry calculations (i.e., scaled harmonic frequency calculations using the B97-1/def2-TZVPD model chemistry with median errors of 10cm-1), here we present a complementary high-throughput approach to rapidly generate approximate vibrational spectral data for 2743 molecules made from the biologically most important elements C, H, N, O, P and S. Though these data are not accurate enough to enable definitive molecular detections and does not seek to replace the need for high-resolution data, it has powerful applications in identifying potential molecular candidates responsible for unknown spectral features. We explore this application for the 4.1 micron (2439cm-1) feature in the atmospheric spectrum of WASP-39b, listing potential alternative molecular species responsible for this spectral line, together with SO2. Further applications of this big data compilation also include identifying molecules with strong absorption features that are likely detectable at quite low abundances, and training set for machine learning predictions of vibrational frequencies. Characterising exoplanetary atmospheres through molecular spectroscopy is essential to understand the planet's physico-chemical processes and likelihood of hosting life. Our rapidly generated quantum chemistry big data set will play a crucial role in supporting this understanding by giving directions into possible initial identifications of the more unusual molecules to emerge.

Despite the popularity of scaled harmonic frequency calculations in chemistry, sparse benchmarking is available to guide users on appropriate level of theory and basis set choices (model chemistry). Here, we assess the performance of over 600 model chemistries in scaled harmonic frequencies calculations evaluating different scaling factors types. We can summarise our results into three main findings: (1) using model chemistry-specific scaling factors optimised for three different frequency regions (low (below 1,000cm-1), mid (1,000-2,000cm-1), and high (above 2,000cm-1)) results in better agreement between the scaled harmonic and experimental frequencies compared to other choices; (2) larger basis sets and more robust levels of theory generally lead to superior performance; however, the particular model chemistry choice matters; and (3) outliers should be expected in routine calculations. In terms of specific recommendations, overall, the highest accuracy model chemistries are double-hybrid functionals with non-Pople augmented triple-zeta basis sets, which can produce median frequency errors of down to 7.6cm-1 (DSD-PBEP86/def2-TZVPD) lying close to the error in the harmonic approximation. Double-zeta basis sets should not be used with double-hybrid functionals as there is no improvement compared to hybrid functionals. Note that the 6-311G* and 6-311+G* bases perform like double-zeta basis sets for vibrational frequencies. After scaling, all studied hybrid functionals with non-Pople triple-zeta basis sets will produce median errors of less than 15cm-1, with the best result of 9.9cm-1 with B97-1/def2-TZVPD. Appropriate matching of double-zeta basis sets with hybrid functionals can produce high quality results. The B97-1, TPSS0-D3(BJ) or wB97X-D hybrid functionals with 6-31G*, pc-1 or pcseg-1 are recommended for fast routine calculations, all delivering median errors of 11-12cm-1.

High-throughput approaches for producing approximate vibrational spectral data for molecules of astrochemistry interest rely on scaled harmonic frequency calculations. However, level of theory and basis set pair recommendations for these calculations are not yet available and thus benchmarking against comprehensive benchmark databases is needed. Here, we present a new database for vibrational frequency calculations (VIBFREQ1295) storing 1,295 experimental fundamental frequencies and CCSD(T)(F12*)/cc-pVDZ-F12 ab initio harmonic frequencies from 141 molecules. VIBFREQ1295's experimental data was complied through a comprehensive review of contemporary experimental data while the ab initio data was computed here. The chemical space spanned by the molecules chosen is considered in depth and shown to have good representation of common organic functional groups and vibrational modes. Scaling factors are routinely used to approximate the effect of anharmonicity and convert computed harmonic frequencies to predicted fundamental frequencies. With our experimental and high-level ab initio data, we find that a global scaling factor of 0.9617 results in median errors of 15.9 cm-1. Far superior performance with a median error of 7.5 cm-1 can be obtained, however, by using separate scaling factors for three regions: frequencies less than 1000 cm-1 (0.987), between 1000 and 2000 cm-1 (0.9727) and above 2000 cm-1 (0.9564). This sets a lower bound for the performance of level of theory and basis set pairs in scaled harmonic frequency calculations. VIBFREQ1295's most important purpose is to provide a robust benchmarking database for vibrational frequency calculations. The database can be found as part of the supplemental material for this paper, or in the Harvard DataVerse at https://doi.org/10.7910/DVN/VLVNU7.

A new silicon monoxide ($^{28}$Si$^{16}$O) line list covering infrared, visible and ultraviolet regions called SiOUVenIR is presented. This line list extends the infrared EBJT ExoMol line list by including vibronic transitions to the $A\,{}^{1}\Pi$ and $E\,{}^{1}\Sigma^{+}$ electronic states. Strong perturbations to the $A\,{}^{1}\Pi$ band system are accurately modelled through the treatment of 6 dark electronic states: $C\,{}^{1}\Sigma^{-}$, $D\,{}^{1}\Delta$, $a\,{}^{3}\Sigma^{+}$, $b\,{}^{3}\Pi$, $e\,{}^{3}\Sigma^{-}$ and $d\,{}^{3}\Delta$. Along with the $X\,{}^{1}\Sigma^{+}$ ground state, these 9 electronic states were used to build a comprehensive spectroscopic model of SiO using a combination of empirical and ab initio curves, including the potential energy (PE), spin-orbit (SO), electronic angular momentum (EAM) and (transition) dipole moment curves. The ab initio PE and coupling curves, computed at the multireference configuration interaction (MRCI) level of theory, were refined by fitting their analytical representations to 2617 experimentally derived SiO energy levels determined from 97 vibronic bands belonging to the $X$-$X$, $E$-$X$ and $A$-$X$ electronic systems through the MARVEL procedure. 112 observed forbidden transitions from the $C$-$X$, $D$-$X$, $e$-$X$, and $d$-$X$ bands were assigned using our predictions, and these could be fed back into the MARVEL procedure. The SiOUVenIR line list was computed using published ab initio transition dipole moments for the $E$-$X$ and $A$-$X$ bands; the line list is suitable for temperatures up to 10,000 K and for wavelengths longer than 140 nm. SiOUVenIR is available from www.exomol.com and the CDS database.

Vibrational frequency calculations performed under the harmonic approximation are widespread across chemistry. However, it is well-known that the calculated harmonic frequencies tend to systematically overestimate experimental fundamental frequencies; a limitation commonly overcome with multiplicative scaling factors. In practice, multiplicative scaling factors are derived for each individual model chemistry choice (i.e., a level of theory and basis set pair), where performance is judged by, for example, the root-mean square error (RMSE) between the predicted scaled and experimental frequencies. However, despite the overwhelming number of scaling factors reported in the literature and model chemistry approximations available, there is little guidance for users on appropriate model chemistry choices for harmonic frequency calculations. Here, we compile and analyse the data for 1495 scaling factors calculated using 141 levels of theory and 109 basis sets. Our meta-analysis of this data shows that scaling factors and RMSE approach convergence with only hybrid functionals and double-zeta basis sets, with anharmonicity error already dominating model chemistry errors. Noting inconsistent data and the lack of independent testing, we can nevertheless conclude that a minimum error of 25cm-1 -- arising from insufficiently accurate treatment of anharmonicity -- is persistent regardless of the model chemistry choice. Based on the data we compiled and cautioning the need for a future systematic benchmarking study, we recommend wB97X-D/def2-TZVP for most applications and B2PLYP/def2-TZVPD for superior intensity predictions. With a smaller benchmark set, direct comparison strongly prefers wB97X-D/6-31G* to B3LYP/6-31G*., Comment: 6 figures, 5 tables, 13 pages