Search

Chemical mechanical planarization (CMP) is vital for ensuring chip fabrication uniformity at nanometer scales. The emergence of a series of phenomenological CMP process models (Stine et al., 1997; Gbondo-Tugbawa, 2002; Xie, 2007; Vasilev, 2011) suggests that the existing model upgrade approach is largely based on a change in phenomenological model assumptions, demanding deep insights into complex process mechanisms and protracted period for accuracy improvements. To tackle this issue, this paper proposes a decoupling regression strategy for model upgrades. This strategy employs a data-driven approach to enhance the coupling relationships within the model, facilitating continuous improvement of simulation accuracy based on the existing model. It is capable of achieving improvements in model accuracy even in scenarios where modelers lack insight into complex process mechanisms. We validate our method by upgrading the Density Step Height (DSH) model to the Extend-DSH model to address poor erosion predictions at the 28nm node. Comparing model predictions with silicon data reveals that the Extend-DSH model aligns better with the measured data, reducing the root mean square error from 159.31Å to 6.89Å and increasing the coefficient of determination from -0.83561 to 0.6058, showcasing the effectiveness of the proposed chip-level CMP model upgrade method grounded in the decoupling regression strategy. [ABSTRACT FROM AUTHOR]

Chemical mechanical polishing/planarization (CMP) combines physical grinding and chemical reactions to planarize the wafer surface. The complex mechanism of CMP brings great challenges to the mechanism-based modeling process. The data-driven CMP modeling process is limited by insufficient datasets. At the same time, these two types of models generally have high computational complexity. In this paper, we introduce the group method of data handling (GMDH)-type polynomial network to build the CMP model to address the above challenges. We designed and manufactured the test chip using a 28nm process. The measurement data from the test chip shows that compared with the mechanism-based CMP model, the trained CMP model based on GMDH-type polynomial network has higher accuracy and lower computational complexity, with the average simulation speed being 115x faster. Experiments based on silicon data show that this modeling method has a small demand for data, and 20 randomly selected sets of data can meet the needs for modeling the current CMP process. [ABSTRACT FROM AUTHOR]

This paper proposes a linear regression model for scalar-valued responses and two-way functional (bivariate) predictors. Our motivation stems from the quality evaluation of products based on optical emission spectroscopy data from virtual metrology of semiconductor manufacturing. We focus on multivariate cases where the smoothness and shapes of the data vary significantly across variables. We propose a two-step solution to this problem, consisting of decomposition and prediction. First, we decompose the two-way functional data into pairs of component functions using functional singular value decomposition. Next, we build functional linear models for the decomposed functional variables and obtain the final predictor by averaging the models. Results from numerical studies, including simulation studies and real data analysis, demonstrate the promising empirical properties of the proposed approach, especially when the number of predictors is large. [ABSTRACT FROM AUTHOR]

Video games are one of the richest and most popular forms of human-computer interaction and, hence, their role is critical for our understanding of human behaviour and affect at a large scale. As artificial intelligence (AI) tools are gradually adopted by the game industry a series of ethical concerns arise. Such concerns, however, have so far not been extensively discussed in a video game context. Motivated by the lack of a comprehensive review on the ethics of AI as applied to games, we survey the current state of the art in this area and discuss ethical considerations of these systems from the holistic perspective of the affective loop. Through the components of this loop, we study the ethical challenges that AI faces in video game development. Elicitation highlights the ethical boundaries of artificially induced emotions; sensing showcases the trade-off between privacy and safe gaming spaces; and detection, as utilised during in-game adaptation, poses challenges to transparency and ownership. This paper calls for an open dialogue and action for the games of today and the virtual spaces of the future. By setting an appropriate framework we aim to protect users and to guide developers towards safer and better experiences for their customers. [ABSTRACT FROM AUTHOR]

With the proliferation of user-generated online videos, Multimodal Sentiment Analysis (MSA) has attracted increasing attention recently. Despite significant progress, there are still two major challenges on the way towards robust MSA: 1) inefficiency when modeling cross-modal interactions in unaligned multimodal data; and 2) vulnerability to random modality feature missing which typically occurs in realistic settings. In this paper, we propose a generic and unified framework to address them, named Efficient Multimodal Transformer with Dual-Level Feature Restoration (EMT-DLFR). Concretely, EMT employs utterance-level representations from each modality as the global multimodal context to interact with local unimodal features and mutually promote each other. It not only avoids the quadratic scaling cost of previous local-local cross-modal interaction methods but also leads to better performance. To improve model robustness in the incomplete modality setting, on the one hand, DLFR performs low-level feature reconstruction to implicitly encourage the model to learn semantic information from incomplete data. On the other hand, it innovatively regards complete and incomplete data as two different views of one sample and utilizes siamese representation learning to explicitly attract their high-level representations. Comprehensive experiments on three popular datasets demonstrate that our method achieves superior performance in both complete and incomplete modality settings. [ABSTRACT FROM AUTHOR]

In recent years, many agent-based models of human groups have implemented a mechanism of emotion contagion, yet empirical validation is lagging behind. The aim of the present paper is to validate an agent-based model of emotion contagion at the level of group emotion, by comparing simulations against the emotional development of real people in small groups. To study the effect of emotion contagion, the participants interacted via a video call, where they were virtually placed in different social environments while they played a quiz. This allowed the exchange of emotion among all, some or none of the participants. The patterns of emotional development in the empirical results supported our hypotheses based on literature of emotion contagion and social norms. Further, the simulations with the complete model resembled many of these patterns. When emotion contagion was disabled in the model, the resemblance decreased. These results give a first indication that emotion contagion occurs in groups that meet via video calls, and can in-part be predicted by the proposed model of emotion contagion. Yet, further study with a larger and more diverse empirical sample is needed, as well as comparisons across contagion mechanisms, to draw stronger conclusions and ultimately justify societal application. [ABSTRACT FROM AUTHOR]

Edge devices are increasingly utilized for deploying deep learning applications on embedded systems. The real-time nature of many applications and the limited resources of edge devices necessitate latency-targeted neural network compression. However, measuring latency on real devices is challenging and expensive. Therefore, this letter presents a novel and efficient framework, named EvoLP, to accurately predict the inference latency of models on edge devices. This predictor can evolve to achieve higher latency prediction precision during the network compression process. Experimental results demonstrate that EvoLP outperforms previous state-of-the-art approaches by being evaluated on three edge devices and four model variants. Moreover, when incorporated into a model compression framework, it effectively guides the compression process for higher model accuracy while satisfying strict latency constraints. We open-source EvoLP at https://github.com/ntuliuteam/EvoLP. [ABSTRACT FROM AUTHOR]

Speech emotion recognition (SER) promotes the development of intelligent devices, which enable natural and friendly human-computer interactions. However, the recognition performance of existing approaches is significantly reduced on unseen datasets, and the lack of sufficient training data limits the generalizability of deep learning models. In this article, we analyze the impact of the domain generalization method on cross-corpus SER and propose an adversarial domain generalized transformer (ADoGT), which is aimed at learning a shared feature distribution for the source and target domains. Specifically, we investigate the effect of domain adversarial learning by eliminating nonaffective information. We also combine the center loss with the softmax function as joint supervision to learn discriminative features. Moreover, we introduce unsupervised transfer learning to extract additional features, and incorporate a gated fusion model to learn the complementary information of the features learned by the supervised feature extractor and pretrained model. The proposed transformer based domain generalization method is evaluated using four emotional datasets. We also provide an ablation study of different domain adversarial model structures and feature fusion models. The results of comparative experiments demonstrate the effectiveness of the proposed ADoGT. [ABSTRACT FROM AUTHOR]

How can we reliably transfer affect models trained in controlled laboratory conditions (in-vitro) to uncontrolled real-world settings (in-vivo)? The information gap between in-vitro and in-vivo applications defines a core challenge of affective computing. This gap is caused by limitations related to affect sensing including intrusiveness, hardware malfunctions and availability of sensors. As a response to these limitations, we introduce the concept of privileged information for operating affect models in real-world scenarios (in the wild). Privileged information enables affect models to be trained across multiple modalities available in a lab, and ignore, without significant performance drops, those modalities that are not available when they operate in the wild. Our approach is tested in two multimodal affect databases one of which is designed for testing models of affect in the wild. By training our affect models using all modalities and then using solely raw footage frames for testing the models, we reach the performance of models that fuse all available modalities for both training and testing. The results are robust across both classification and regression affect modeling tasks which are dominant paradigms in affective computing. Our findings make a decisive step towards realizing affect interaction in the wild. [ABSTRACT FROM AUTHOR]

A succession of accidents shows that production vehicles with autonomous driving systems do not work safely in real-world environments, especially when facing unseen scenarios. Therefore, how to ensure that autonomous systems drive more safely becomes a challenge. Thanks to the self-learning ability of human beings, human drivers can gradually learn how to drive from a driving test with typical and finite scenarios to the real world with infinite ones. Analogically, it is believed that accidents can be largely reduced once the designed autonomous vehicles are endowed with a self-learning ability to adapt to the unseen and then to infinite scenarios in the real world. Accordingly, this work proposes a principle to design autonomous systems with a self-evolution feature not just for a single vehicle but for a group. In addition, it describes our development of a self-evolution autonomous system as an illustrative case study of implementing such principles in practice. The ultimate aim is to propose a feasible solution to speed up the design process of a fully safe autonomous system. [ABSTRACT FROM AUTHOR]

Graph serialization is a critical aspect of advancing graph-oriented systems and applications. Despite the importance of standardized serialization for property graphs, there is a lack of a universal format encompassing all essential features of graph database systems. This study introduces YARS-PG, a simple, extensible, and platform-independent serialization format tailored for property graphs. YARS-PG supports all the features permitted by current property graph-based database systems and is compatible with other graph-oriented databases and tools. We delineate the design requirements of YARS-PG by detailing both functional and non-functional aspects. Besides the basic features of property graph data, YARS-PG supports schema definition, metadata, metaproperties, variables, and graph definitions. Moreover, we discuss extensions of YARS-PG, demonstrating its flexibility through canonicalization techniques. Our comparative analyses with existing formats provide valuable insights, emphasizing the unique strengths that distinguish YARS-PG in the realm of graph data interchange. This paper serves as a definitive guide to YARS-PG, unraveling its complexities and showcasing its potential as a communication protocol, a data storage format, and a messaging specification.

Change detection (CD) using deep learning techniques is a prominent topic in the field of remote sensing (RS). However, the existing methods require large amounts of labeled samples for supervised learning, which is time-consuming and labor-intensive. To address this challenge, semi-supervised learning methods that utilize a limited number of labeled samples along with a large pool of unlabeled samples have emerged as a compelling solution. We propose a novel semi-supervised CD (SSCD) network that combines self-training and consistency regularization, namely STCRNet. During the self-training phase, STCRNet selects unlabeled samples with reliable pseudolabels based on their prediction stability across different training epochs and the consistency between class activation maps and prediction results within the model. Then, we apply data augmentation to the reliable samples and enforce consistency regularization on the augmented samples using the pseudolabels to enhance the network's robustness. Moreover, feature consistency regularization is applied to the remaining unlabeled samples with image perturbations, thereby broadening the feature space and improving the model's generalization performance. The experimental results on two widely used datasets demonstrate that STCRNet achieves state-of-the-art performance, especially with a significantly small amount (5%–10%) of labeled samples. STCRNet presents a promising solution for SSCD. [ABSTRACT FROM AUTHOR]

Contrastive learning has revolutionized the field of computer vision, learning rich representations from unlabeled data, which generalize well to diverse vision tasks. Consequently, it has become increasingly important to explain these approaches and understand their inner workings mechanisms. Given that contrastive models are trained with interdependent and interacting inputs and aim to learn invariance through data augmentation, the existing methods for explaining single-image systems (e.g., image classification models) are inadequate as they fail to account for these factors and typically assume independent inputs. Additionally, there is a lack of evaluation metrics designed to assess pairs of explanations, and no analytical studies have been conducted to investigate the effectiveness of different techniques used to explaining contrastive learning. In this work, we design visual explanation methods that contribute towards understanding similarity learning tasks from pairs of images. We further adapt existing metrics, used to evaluate visual explanations of image classification systems, to suit pairs of explanations and evaluate our proposed methods with these metrics. Finally, we present a thorough analysis of visual explainability methods for contrastive learning, establish their correlation with downstream tasks and demonstrate the potential of our approaches to investigate their merits and drawbacks. [ABSTRACT FROM AUTHOR]

This research investigates the potential of in vivo learning to enhance visual representation learning for image-based person re-identification (re-ID). Compared to traditional self-supervised learning (which require external data), the introduced in vivo learning utilizes supervisory labels generated from pedestrian images to improve re-ID accuracy without relying on external data sources. Three carefully designed in vivo learning tasks, leveraging statistical regularities within images, are proposed without the need for laborious manual annotations. These tasks enable feature extractors to learn more comprehensive and discriminative person representations by jointly modeling various aspects of human biological structure information, contributing to enhanced re-ID performance. Notably, the method seamlessly integrates with existing re-ID frameworks, requiring minimal modifications and no additional data beyond the existing training set. Extensive experiments on diverse datasets, including Market1501, CUHK03-NP, Celeb-reID, Celeb-reid-light, PRCC, and LTCC, demonstrate substantial enhancements in rank-1 precision compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Private Set Intersection (PSI) protocols can securely compute the intersection of the private sets on the server and the client without revealing additional data. This work introduces the concept of Privacy-Preserving Feature Retrieved Private Set Intersection ($\mathsf {P^{2}FRPSI}$). In $\mathsf {P^{2}FRPSI}$ protocols, the client can obtain the intersection that satisfies a given predicate without revealing the predicate and additional data. We formally define the $\mathsf {P^{2}FRPSI}$ protocol, including its inputs, outputs, functionality, and security. To achieve the privacy guarantee in $\mathsf {P^{2}FRPSI}$ protocols, a new two-party protocol is designed, namely Secure Secret Shared Retrieval ($\mathsf {S^{3}R}$), which can be used to securely determine whether each item on the server satisfies the predicate. We construct an $\mathsf {S^{3}R}$ protocol and prove its security in the semi-honest model. On the basis of this, we design an efficient OT-based $\mathsf {P^{2}FRPSI}$ protocol and an easy-to-implement DH-based $\mathsf {P^{2}FRPSI}$ protocol and prove that they are secure in the semi-honest model. Our implementation shows that the OT-based $\mathsf {P^{2}FRPSI}$ protocol can perform the matching for about 1000K items in 3.8 seconds with a single thread. Moreover, the DH-based $\mathsf {P^{2}FRPSI}$ can perform the matching for about 7000K items in one hour with four threads, with communication totaling 1456 MB, while the OT-based $\mathsf {P^{2}FRPSI}$ protocol requires 1673 MB. [ABSTRACT FROM AUTHOR]

Ethical affective computing (AC) requires maximizing the benefits to users while minimizing its harm to obtain trust from users. This requires responsible development and deployment to ensure fairness, bias mitigation, privacy preservation, and accountability. To obtain this, we require methodologies that can quantify, visualize, analyze, and mine insights from affect data. Hence, in this article, we propose a spatio-temporal model for representing secondary affect data from network sciences’ perspective. We propose a network science-based model to represent spatio-temporal data, e.g., action units’ sequences, and continuous affect reports. In particular, the proposed model captures the spatial and temporal strength of the relationship among essential variables in the data. The proposed model allows to analyze data as a whole system. We also demonstrated the use case of the model for graph analytics on secondary affect data that can assist to measure and quantify several issues that can be originated from the study setup, data recording devices, and the influences/biases that can originate from the perspective of the affect reporters. We also demonstrated the use cases of the proposed method on ethical trustworthy emotional AI via measuring biases from de-identified data and how it contributes towards ethics, transparency, value alignment, and governance. [ABSTRACT FROM AUTHOR]

Recently, facial expression recognition techniques have made significant progress on high-resolution web images. However, in real-world applications, the obtained images are often with low resolution since they are mostly captured in a wide range of public spaces. As a result, the ambiguity of the expression labels hinders recognition performance due to not only subjective emotion annotations but also ambiguous images. Existing approaches tend to perform poorly when the resolution of face images decreases. In this work, we aim to model the aleatoric uncertainty induced by low-image-resolution and label ambiguity for robust facial expression recognition. We propose probabilistic data uncertainty learning to capture the ambiguity induced by poor image resolution. Additionally, we introduce the emotion wheel to learn the label-uncertainty-aware embedding. Moreover, we exploit the ambiguous nature of neutrality and propose a neutral expression constraint to learn more robust features for facial expression recognition. To the best of our knowledge, this is the first work utilizing the intrinsic nature of neutrality as a regularization to benefit model training. Extensive experimental results show the effectiveness and robustness of our approach. Under low-resolution conditions, our proposed method outperforms the state-of-the-art approaches by 3.02% and 3.16% in terms of accuracy on RAF-DB and FERPlus, respectively. [ABSTRACT FROM AUTHOR]

Micro-expressions have drawn increasing interest lately due to various potential applications. The task is, however, difficult as it incorporates many challenges from the fields of computer vision, machine learning and emotional sciences. Due to the spontaneous and subtle characteristics of micro-expressions, the available training and testing data are limited, which make evaluation complex. We show that data leakage and fragmented evaluation protocols are issues among the micro-expression literature. We find that fixing data leaks can drastically reduce model performance, in some cases even making the models perform similarly to a random classifier. To this end, we go through common pitfalls, propose a new standardized evaluation protocol using facial action units with over 2000 micro-expression samples, and provide an open source library that implements the evaluation protocols in a standardized manner. Code is publicly available in https://github.com/tvaranka/meb. [ABSTRACT FROM AUTHOR]

Recently, wearable emotion recognition based on peripheral physiological signals has drawn massive attention due to its less invasive nature and its applicability in real-life scenarios. However, how to effectively fuse multimodal data remains a challenging problem. Moreover, traditional fully-supervised based approaches suffer from overfitting given limited labeled data. To address the above issues, we propose a novel self-supervised learning (SSL) framework for wearable emotion recognition, where efficient multimodal fusion is realized with temporal convolution-based modality-specific encoders and a transformer-based shared encoder, capturing both intra-modal and inter-modal correlations. Extensive unlabeled data is automatically assigned labels by five signal transforms, and the proposed SSL model is pre-trained with signal transformation recognition as a pretext task, allowing the extraction of generalized multimodal representations for emotion-related downstream tasks. For evaluation, the proposed SSL model was first pre-trained on a large-scale self-collected physiological dataset and the resulting encoder was subsequently frozen or fine-tuned on three public supervised emotion recognition datasets. Ultimately, our SSL-based method achieved state-of-the-art results in various emotion classification tasks. Meanwhile, the proposed model was proved to be more accurate and robust compared to fully-supervised methods on low data regimes. [ABSTRACT FROM AUTHOR]

Multimodal Emotion Recognition in Conversation (ERC) plays an influential role in the field of human-computer interaction and conversational robotics since it can motivate machines to provide empathetic services. Multimodal data modeling is an up-and-coming research area in recent years, which is inspired by human capability to integrate multiple senses. Several graph-based approaches claim to capture interactive information between modalities, but the heterogeneity of multimodal data makes these methods prohibit optimal solutions. In this article, we introduce a multimodal fusion approach named Graph and Attention based Two-stage Multi-source Information Fusion (GA2MIF) for emotion detection in conversation. Our proposed method circumvents the problem of taking heterogeneous graph as input to the model while eliminating complex redundant connections in the construction of graph. GA2MIF focuses on contextual modeling and cross-modal modeling through leveraging Multi-head Directed Graph ATtention networks (MDGATs) and Multi-head Pairwise Cross-modal ATtention networks (MPCATs), respectively. Extensive experiments on two public datasets (i.e., IEMOCAP and MELD) demonstrate that the proposed GA2MIF has the capacity to validly capture intra-modal long-range contextual information and inter-modal complementary information, as well as outperforms the prevalent State-Of-The-Art (SOTA) models by a remarkable margin. [ABSTRACT FROM AUTHOR]

Smart healthcare has achieved significant progress in recent years. Emerging artificial intelligence (AI) technologies enable various smart applications across various healthcare scenarios. As an essential technology powered by AI, natural language processing (NLP) plays a key role in smart healthcare due to its capability of analysing and understanding human language. In this work, we review existing studies that concern NLP for smart healthcare from the perspectives of technique and application. We first elaborate on different NLP approaches and the NLP pipeline for smart healthcare from the technical point of view. Then, in the context of smart healthcare applications employing NLP techniques, we introduce representative smart healthcare scenarios, including clinical practice, hospital management, personal care, public health, and drug development. We further discuss two specific medical issues, i.e., the coronavirus disease 2019 (COVID-19) pandemic and mental health, in which NLP-driven smart healthcare plays an important role. Finally, we discuss the limitations of current works and identify the directions for future works. [ABSTRACT FROM AUTHOR]