Search

Multilingual language models have pushed state-of-the-art in cross-lingual NLP transfer. The majority of zero-shot cross-lingual transfer, however, use one and the same massively multilingual transformer (e.g., mBERT or XLM-R) to transfer to all target languages, irrespective of their typological, etymological, and phylogenetic relations to other languages. In particular, readily available data and models of resource-rich sibling languages are often ignored. In this work, we empirically show, in a case study for Faroese -- a low-resource language from a high-resource language family -- that by leveraging the phylogenetic information and departing from the 'one-size-fits-all' paradigm, one can improve cross-lingual transfer to low-resource languages. In particular, we leverage abundant resources of other Scandinavian languages (i.e., Danish, Norwegian, Swedish, and Icelandic) for the benefit of Faroese. Our evaluation results show that we can substantially improve the transfer performance to Faroese by exploiting data and models of closely-related high-resource languages. Further, we release a new web corpus of Faroese and Faroese datasets for named entity recognition (NER), semantic text similarity (STS), and new language models trained on all Scandinavian languages.

The impact of geohazards on the mental health of the local populations, is well recognized but understudied. We used natural language processing (NLP) of Twitter posts to analyse the sentiments expressed in relation to a pre-eruptive seismic unrest, and a subsequent volcanic eruption in Iceland 2021. We show that despite the small size and negligible material damage, these geohazards were associated with a measurable change in expressed emotions in the local populations. The seismic unrest was associated with predominantly negative sentiments, but the eruption with predominantly positive. We demonstrate a cost-effective tool for gauging public discourse that could be used in risk management.

Recent advances in text-to-image diffusion models have enabled the generation of diverse and high-quality images. However, generated images often fall short of depicting subtle details and are susceptible to errors due to ambiguity in the input text. One way of alleviating these issues is to train diffusion models on class-labeled datasets. This comes with a downside, doing so limits their expressive power: (i) supervised datasets are generally small compared to large-scale scraped text-image datasets on which text-to-image models are trained, and so the quality and diversity of generated images are severely affected, or (ii) the input is a hard-coded label, as opposed to free-form text, which limits the control over the generated images. In this work, we propose a non-invasive fine-tuning technique that capitalizes on the expressive potential of free-form text while achieving high accuracy through discriminative signals from a pretrained classifier, which guides the generation. This is done by iteratively modifying the embedding of a single input token of a text-to-image diffusion model, using the classifier, by steering generated images toward a given target class. Our method is fast compared to prior fine-tuning methods and does not require a collection of in-class images or retraining of a noise-tolerant classifier. We evaluate our method extensively, showing that the generated images are: (i) more accurate and of higher quality than standard diffusion models, (ii) can be used to augment training data in a low-resource setting, and (iii) reveal information about the data used to train the guiding classifier. The code is available at \url{https://github.com/idansc/discriminative_class_tokens}

Background and aims:The causal contribution of apolipoprotein B (apoB) particles to coronary artery disease (CAD) is established. We examined whether this atherogenic contribution is better reflected by non-high-density lipoprotein cholesterol (non-HDL-C) or apoB particle concentration.Method and results:We performed Mendelian randomization (MR) analysis using 235 variants as genetic instruments; testing the relationship between their effects on the exposures, non-HDL-C and apoB, and on the outcome CAD using weighted regression. Variant effect estimates on the exposures came from the UK Biobank (N = 376 336) and on the outcome from a meta-analysis of five CAD datasets (187 451 cases and 793 315 controls). Subsequently, we carried out sensitivity and replication analyses.In univariate MR analysis, both exposures associated with CAD (βnon-HDL-C = 0.40, P = 2.8 × 10−48 and βapoB = 0.38, P = 1.3 × 10−44). Adding effects on non-HDL-C into a model that already included those on apoB significantly improved the genetically predicted CAD effects (P = 3.9 × 10−5), while adding apoB into the model including non-HDL-C did not (P = 0.69). Thirty-five per cent (82/235) of the variants used as genetic instruments had discordant effects on the exposures, associating with non-HDL-C/apoB ratio at P Conclusion:Many sequence variants have discordant effects on non-HDL-C and apoB. These variants allowed us to show that the causal mechanism underlying the relationship between apolipoprotein B particles and CAD is more associated with non-HDL-C than apoB particle concentration.

Nonalcoholic fatty liver (NAFL) and its sequelae are growing health problems. We performed a genome-wide association study of NAFL, cirrhosis and hepatocellular carcinoma, and integrated the findings with expression and proteomic data. For NAFL, we utilized 9,491 clinical cases and proton density fat fraction extracted from 36,116 liver magnetic resonance images. We identified 18 sequence variants associated with NAFL and 4 with cirrhosis, and found rare, protective, predicted loss-of-function variants in MTARC1 and GPAM, underscoring them as potential drug targets. We leveraged messenger RNA expression, splicing and predicted coding effects to identify 16 putative causal genes, of which many are implicated in lipid metabolism. We analyzed levels of 4,907 plasma proteins in 35,559 Icelanders and 1,459 proteins in 47,151 UK Biobank participants, identifying multiple proteins involved in disease pathogenesis. We show that proteomics can discriminate between NAFL and cirrhosis. The present study provides insights into the development of noninvasive evaluation of NAFL and new therapeutic options.

The ability to assess the robustness of image classifiers to a diverse set of manipulations is essential to their deployment in the real world. Recently, semantic manipulations of real images have been considered for this purpose, as they may not arise using standard adversarial settings. However, such semantic manipulations are often limited to style, color or attribute changes. While expressive, these manipulations do not consider the full capacity of a pretrained generator to affect adversarial image manipulations. In this work, we aim at leveraging the full capacity of a pretrained image generator to generate highly detailed, diverse and photorealistic image manipulations. Inspired by recent GAN-based image inversion methods, we propose a method called Adversarial Pivotal Tuning (APT). APT first finds a pivot latent space input to a pretrained generator that best reconstructs an input image. It then adjusts the weights of the generator to create small, but semantic, manipulations which fool a pretrained classifier. Crucially, APT changes both the input and the weights of the pretrained generator, while preserving its expressive latent editing capability, thus allowing the use of its full capacity in creating semantic adversarial manipulations. We demonstrate that APT generates a variety of semantic image manipulations, which preserve the input image class, but which fool a variety of pretrained classifiers. We further demonstrate that classifiers trained to be robust to other robustness benchmarks, are not robust to our generated manipulations and propose an approach to improve the robustness towards our generated manipulations. Code available at: https://captaine.github.io/apt/

It can be challenging to build effective open question answering (open QA) systems for languages other than English, mainly due to a lack of labeled data for training. We present a data efficient method to bootstrap such a system for languages other than English. Our approach requires only limited QA resources in the given language, along with machine-translated data, and at least a bilingual language model. To evaluate our approach, we build such a system for the Icelandic language and evaluate performance over trivia style datasets. The corpora used for training are English in origin but machine translated into Icelandic. We train a bilingual Icelandic/English language model to embed English context and Icelandic questions following methodology introduced with DensePhrases (Lee et al., 2021). The resulting system is an open domain cross-lingual QA system between Icelandic and English. Finally, the system is adapted for Icelandic only open QA, demonstrating how it is possible to efficiently create an open QA system with limited access to curated datasets in the language of interest.

We present Mi{\dh}eind's submission for the English$\to$Icelandic and Icelandic$\to$English subsets of the 2021 WMT news translation task. Transformer-base models are trained for translation on parallel data to generate backtranslations iteratively. A pretrained mBART-25 model is then adapted for translation using parallel data as well as the last backtranslation iteration. This adapted pretrained model is then used to re-generate backtranslations, and the training of the adapted model is continued.

We present a new Icelandic-English parallel corpus, the Icelandic Parallel Abstracts Corpus (IPAC), composed of abstracts from student theses and dissertations. The texts were collected from the Skemman repository which keeps records of all theses, dissertations and final projects from students at Icelandic universities. The corpus was aligned based on sentence-level BLEU scores, in both translation directions, from NMT models using Bleualign. The result is a corpus of 64k sentence pairs from over 6 thousand parallel abstracts.

State-of-the-art machine translation (MT) systems rely on the availability of large parallel corpora, containing millions of sentence pairs. For the Icelandic language, the parallel corpus ParIce exists, consisting of about 3.6 million English-Icelandic sentence pairs. Given that parallel corpora for low-resource languages typically contain sentence pairs in the tens or hundreds of thousands, we classify Icelandic as a medium-resource language for MT purposes. In this paper, we present on-going experiments with different MT models, both statistical and neural, for translating English to Icelandic based on ParIce. We describe the corpus and the filtering process used for removing noisy segments, the different models used for training, and the preliminary automatic and human evaluation. We find that, while using an aggressive filtering approach, the most recent neural MT system (Transformer) performs best, obtaining the highest BLEU score and the highest fluency and adequacy scores from human evaluation for in-domain translation. Our work could be beneficial to other languages for which a similar amount of parallel data is available.

The first main theme of this thesis is the approximation of holomorphic functions on Stein manifolds by polynomials. In our setting a polynomial is an entire function whose absolute value is bounded from above by $\exp(a\psi+b)$ on the complement of a compact set, where $a,b$ are positive constants and $\psi$ is a given plurisubharmonic exhaustion function. In particular we generalize the Bernstein-Walsh-Siciak theorem to a certain class of Stein manifolds, which describes the equivalence between the possible holomorphic continuation of a function $f$ defined on a compact set $K$ in $\C^n$ to the rapidity of the best uniform approximation of $f$ on $K$ by polynomials. We also generalize Winiarski's theorem, which relates the growth rate of an entire function on $\C^n$ to its best uniform approximation by polynomials on a compact set. If a plurisubharmonic exhaustion function $\psi$ on a Stein manifold $X$ satisfies the homogeneous Monge-Amp\`ere equation outside a compact subset of $X$, i.e.\ if $\psi$ is a parabolic potential on $X$, then the polynomial spaces defined by $\psi$ are finite dimensional. Therefore the problem of constructing parabolic potentials arises naturally in the theory of polynomial approximation on Stein manifolds. The second main theme of this thesis is the complex Monge-Amp\`ere operator. In particular we derive formulas for the Monge-Ampère measures of functions of the form $\log |\Phi|_c$ , where $\Phi$ is a holomorphic map on a complex manifold of dimension $n$ with values in $\C^{n+1}\setminus \{0\}$ and $|\cdot|_c$ is the Lie-norm in $\C^{n+1}$., Fyrsta meginþema þessarar ritgerðar er margliðunálganir á fáguðum föllum á Stein víðáttum. Hér lítum við á margliðu sem heilt fágað fall hvers algildi er takmarkað að ofan af falli af gerðinni exp(aψ + b) á fyllimengi þjappaðs mengis, þar sem a, b eru jákvæðir fastar og ψ er einhver ge n fjölundirþýð tæming. Sér í lagi þá alhæfum við Bernstein-Walsh-Siciak setninguna þannig að hún nái y r ákveðinn nýjan okk af Stein víðáttum. Rifjum upp að Bernstein-Walsh-Siciak setningin lýsir venslunum á milli mögulegrar fágaðrar framlengingar á falli f sem er skilgreint á þjöppuðu mengi K í C n og hraðanum á bestu nálgun á f í jöfnum mæli á K með margliðum. Við alhæfum líka setningu Winiarski sem lýsir venslunum á milli vaxtahraða heils fágaðs falls á C n og bestu nálgun f í jöfnum mæli á þjöppuðu mengi með margliðum. Ef fjölundirþýð tæming ψ á Stein víðáttu X uppfyllir óhliðruðu Monge-Ampère jöfnuna á fyllimengi þjappaðs mengis í X, þ.e. ef ψ er eyggert mætti á X, þá eru margliðurúmin sem ψ skilgreinir af endanlegri vídd. Þess vegna framlengjast rannsóknir á margliðunálgunum á Stein víðáttum náttúrulega y r í rannsóknir á MongeAmpère virkjanum. Seinna meginþema þessarar ritgerðar er Monge-Ampère virkinn. Sér í lagi leiðum við út jöfnur fyrir Monge-Ampère mál falla af gerðinni log |Φ|c þar sem að Φ er fáguð vörpun á fágaðri víðáttu af vídd n sem tekur gildi sín í C n \{0} og |·|c er Lie-staðallinn á Cn+1., This work was done at the Science Institute of the University of Iceland and supported by the Research Fund of the University of Iceland and by The Icelandic Centre for Research grant no. 152572-052.

In this paper we derive formulas for the Monge-Amp\`ere measures of functions of the form $\log|\Phi|_c$, where $\Phi$ is a holomorphic map on a complex manifold $X$ of dimension $n$ with values in $\mathbb{C}^{n+1}\setminus\{0\}$ and $|\cdot|_c$ is the Lie norm on $\mathbb{C}^{n+1}$.

In this paper we generalize to a certain class of Stein manifolds the Bernstein-Walsh-Siciak theorem which describes the equivalence between possible holomorphic continuation of a function $f$ defined on a compact set $K$ in $\mathbb{C}^N$ to the rapidity of the best uniform approximation of $f$ on $K$ by polynomials. We also generalize Winiarski's theorem which relates the growth rate of an entire function $f$ on $\mathbb{C}^N$ to its best uniform approximation by polynomials on a compact set.

We train several language models for Icelandic, including IceBERT, that achieve state-of-the-art performance in a variety of downstream tasks, including part-of-speech tagging, named entity recognition, grammatical error detection and constituency parsing. To train the models we introduce a new corpus of Icelandic text, the Icelandic Common Crawl Corpus (IC3), a collection of high quality texts found online by targeting the Icelandic top-level-domain (TLD). Several other public data sources are also collected for a total of 16GB of Icelandic text. To enhance the evaluation of model performance and to raise the bar in baselines for Icelandic, we translate and adapt the WinoGrande dataset for co-reference resolution. Through these efforts we demonstrate that a properly cleaned crawled corpus is sufficient to achieve state-of-the-art results in NLP applications for low to medium resource languages, by comparison with models trained on a curated corpus. We further show that initializing models using existing multilingual models can lead to state-of-the-art results for some downstream tasks.