Search

Speech Emotion Recognition (SER) aims to endow machines with the intelligence in perceiving latent affective components from speech. However, the existing works on deep-learning-based SER make it difficult to jointly consider time-frequency and sequential information in speech due to their structures, which may lead to deficiencies in exploring reasonable local emotional representations. In this regard, we propose a Convolutional-Recurrent Neural Network with Multiple Attention mechanisms (CRNN-MA) for SER in this paper, including paralleled Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) modules, using extracted Mel-spectrums and frame-level features respectively, in order to acquire time-frequency and sequential information simultaneously. Further, we set three strategies for the proposed CRNN-MA: A multiple self-attention layer in the CNN module on frame-level weights, a multi-dimensional attention layer as the input features of the LSTM, and a fusion layer summarizing the features of the two modules. Experimental results on three conventional SER corpora demonstrate the effectiveness of the proposed approach through using the convolutional-recurrent and multiple-attention modules, compared with other related models and existing state-of-the-art approaches.

As a common mental disorder, depression becomes a major threat to human health and may even heavily influence one’s daily life. Considering this background, it is necessary to investigate strategies for automatically detecting depression, especially through the audio modality represented by speech segments, mainly due to the efficient latent information included in speech when describing depression. However, most of the existing works focus on stacking deep networks in audio-based depression detection, which may lead to insufficient knowledge for representing depression in speech. In this regard, we propose a deep learning model based on a parallel convolutional neural network and a transformer in order to mine effective information with an acceptable complexity. The proposed approach consists of a parallel convolutional neural network (parallel-CNN) module used to focus on local knowledge, while a transformer module is employed as the other parallel stream to perceive temporal sequential information using linear attention mechanisms with kernel functions. Then, we performed experiments on two datasets of Distress Analysis Interview Corpus-Wizard of OZ (DAIC-WOZ) and Multi-modal Open Dataset for Mental-disorder Analysis (MODMA). The experimental results indicate that the proposed approach achieves a better performance compared with the state-of-the-art strategies.

The severe acute respiratory syndrome coronavirus 2 (SARS-COV-2) antigenic profile evolves in response to the vaccine and natural infection-derived immune pressure, resulting in immune escape and threatening public health. Exploring the possible antigenic evolutionary potentials improves public health preparedness, but it is limited by the lack of experimental assays as the sequence space is exponentially large. Here we introduce the Machine Learning-guided Antigenic Evolution Prediction (MLAEP), which combines structure modeling, multi-task learning, and genetic algorithm to model the viral fitness landscape and explore the antigenic evolution viain silicodirected evolution. As demonstrated by existing SARS-COV-2 variants, MLAEP can infer the order of variants along antigenic evolutionary trajectories, which is also strongly correlated with their sampling time. The novel mutations predicted by MLAEP are also found in immunocompromised covid patients and newly emerging variants, like XBB1.5. The predictions of MLAEP were validated by conducting in vitro neutralizing antibody binding assay, which demonstrated that the model-generated variants displayed significantly increased immune evasion ability compared with the controls. In sum, our approach enables profiling existing variants and forecasting prospective antigenic variants, thus may help guide the development of vaccines and increase preparedness against future variants. Our model is available athttps://mlaep.cbrc.kaust.edu.sa.

ObjectivesTo analyse the current curative expenditure (CCE) of NCDs in China from 2017 to 2019.DesignA cross-sectional study. Medical institutions were collected by multistage stratified random sampling from 2017 to 2019.SettingDalian, ChinaParticipants408 institutions and 8 104 233 valid items were included in the study. NCDs patients were selected according to International Classification of Diseases-10.Primary and secondary outcome measuresCCE for NCDs was measured based on the System of Health Accounts 2011. Influenced factors were analysed by linear regression. All analyses and calculations were performed by STATA V.15.0.ResultsThe CCE of NCDs was ¥14.929 billion in 2017, ¥16.377 billion in 2018 and ¥18.055 billion in 2019, which accounted for more than 65% of total expenditure spent each year. More than 60% came from public financing. The proportion of family health financing continued to decline, reaching 31.16% in 2019. The expenditures were mainly in general hospitals, above 70%. Elderly patients account for the majority. Diseases of the circulatory system, diseases of the digestive system and neoplasms were the main NCDs. Year, age, gender, length of stay, surgery, insurance and institution level affected hospitalisation expenses.ConclusionsNCDs are the main CCE of diseases in China, and their resources are not allocated reasonably. To reduce the CCE of NCDs, the government needs to optimise resource allocation and rationalise institutional flows and functions.

Speech Emotion Recognition (SER) makes it possible for machines to perceive affective information. Our previous research differed from conventional SER endeavours in that it focused on recognising unseen emotions in speech autonomously through machine learning. Such a step would enable the automatic leaning of unknown emerging emotional states. This type of learning framework, however, still relied on manual annotations to obtain multiple samples of each emotion. In order to reduce this additional workload, herein, we propose a zero-shot SER framework employing a per-emotion semantic-embedding paradigm to describe emotions in zero-shot SER, instead of using the sample-wise descriptors. Aiming to optimise the relationship between emotions, prototypes, and speech samples, this framework includes two types of learning strategies: Sample-wise learning and emotion-wise learning. These strategies apply a novel learning process to speech samples and emotions, respectively, via specifically designed semantic-embedding prototypes. We verify the utility of these approaches by performing an extensive experimental evaluation on two corpora on three aspects, namely the influence of different types of learning strategies, emotional-pair comparison, and the selections of semantic-embedding prototypes and paralinguistic features. The experimental results indicate that it is applicable to use semantic-embedding prototypes for zero-shot emotion recognition in speech, despite the influence of choosing optimal strategies and prototypes.