Search

Transformer-based language models have taken many fields in NLP by storm. BERT and its derivatives dominate most of the existing evaluation benchmarks, including those for Word Sense Disambiguation (WSD), thanks to their ability in capturing context-sensitive semantic nuances. However, there is still little knowledge about their capabilities and potential limitations in encoding and recovering word senses. In this article, we provide an in-depth quantitative and qualitative analysis of the celebrated BERT model with respect to lexical ambiguity. One of the main conclusions of our analysis is that BERT can accurately capture high-level sense distinctions, even when a limited number of examples is available for each word sense. Our analysis also reveals that in some cases language models come close to solving coarse-grained noun disambiguation under ideal conditions in terms of availability of training data and computing resources. However, this scenario rarely occurs in real-world settings and, hence, many practical challenges remain even in the coarse-grained setting. We also perform an in-depth comparison of the two main language model-based WSD strategies, namely, fine-tuning and feature extraction, finding that the latter approach is more robust with respect to sense bias and it can better exploit limited available training data. In fact, the simple feature extraction strategy of averaging contextualized embeddings proves robust even using only three training sentences per word sense, with minimal improvements obtained by increasing the size of this training data. [ABSTRACT FROM AUTHOR]

In this paper, we consider the operation of a thermal-aware task scheduler, dispatching tasks from an arrival queue as well as setting the voltage and frequency of the processing cores to optimize the mean temperature margin of the entire chip (i.e., cores as well as the NoC routers). We model the decision process of the task scheduler as a semi-Markov decision problem (SMDP) to account for the most common uncertainties prevalent in MPSoC systems (including: the stochastic nature of the workload inter-arrival times, time-varying workload characteristics, the uncertain chip thermal profile as well as random inter-task communications). SMDP is among the fairly general variants of continuous-time optimization frameworks from the stochastic control theory and is a much more efficient choice compared to discrete-time formalisms (which would lead to an increase in task waiting times and degraded system performance). To solve the formulated SMDP, we propose two reinforcement learning (RL) algorithms that are capable of computing the optimal task assignment policy without requiring the statistical knowledge of the stochastic dynamics underlying the system states. The proposed algorithms also rely on function approximation techniques to handle the infinite length of the task queue as well as the continuous nature of temperature readings. Compared to previous work, the simulations demonstrate nearly 6 Kelvin reduction in system average peak temperature and 66 milliseconds decrease in mean task service time. • In MPSoCs, task scheduler faces several uncertainties in task-to-core mappings. • The random task pairings involve the NoC routers. • A thermal-aware continuous-time task assignment scheme is presented. • Capturing uncertainties in MPSoCs leads to lower peak temperature and service time. [ABSTRACT FROM AUTHOR]