Search

This paper proposes a hybrid deep learning emotion classification system (HDECS), a hybrid multimodal deep learning system designed for emotion classification in a specific national language. Emotion classification is important in diverse fields, including tailored corporate services, AI advancement, and more. Additionally, most sentiment classification techniques in speaking situations are based on a single modality: voice, conversational text, vital signs, etc. However, analyzing these data presents challenges because of the variations in vocal intonation, text structures, and the impact of external stimuli on physiological signals. Korean poses challenges in natural language processing, including subject omission and spacing issues. To overcome these challenges and enhance emotion classification performance, this paper presents a case study using Korean multimodal data. The case study model involves retraining two pretrained models, LSTM and CNN, until their predictions on the entire dataset reach an agreement rate exceeding 0.75. Predictions are used to generate emotional sentences appended to script data, which are further processed using BERT for final emotion prediction. The research result is evaluated by using categorical cross-entropy (CCE) to measure the difference between the model’s predictions and actual labels, F1 score, and accuracy. According to the evaluation, the case model outperforms the existing KLUE/roBERTa model with improvements of 0.5 in CCE, 0.09 in accuracy, and 0.11 in F1 score. As a result, the HDECS is expected to perform well not only on Korean multimodal datasets but also on sentiment classification considering the speech characteristics of various languages and regions.

This paper proposes a recommendation system based on a hybrid learning approach for a personal deep sleep service, called the Customized Deep Sleep Recommender System (CDSRS). Sleep is one of the most important factors for human life in modern society. Optimal sleep contributes to increasing work efficiency and controlling overall well-being. Therefore, a sleep recommendation service is considered a necessary service for modern individuals. Accurate sleep analysis and data are required to provide such a personalized sleep service. However, given the variations in sleep patterns between individuals, there is currently no international standard for sleep. Additionally, service platforms face a cold start problem when dealing with new users. To address these challenges, this study utilizes K-means clustering analysis to define sleep patterns and employs a hybrid learning algorithm to evaluate recommendations by combining user-based and collaborative filtering methods. It also incorporates feedback top-N classification processing for user profile learning and recommendations. The behavior of the study model is as follows. Using personal information received through mobile devices and data, such as snoring, sleep time, movement, and noise collected through AI motion beds, we recommend sleep and receive user evaluations of recommended sleep. This assessment reconstructs the profile and, finally, makes recommendations using top-N classification. The experimental results were evaluated using two absolute error measurement methods: mean squared error (MSE) and mean absolute percentage error (MAPE). The research results regarding the hybrid learning methods show 13.2% fewer errors than collaborative filtering (CF) and 10.2% fewer errors than content-based filtering (CBF) on an MSE basis. According to the MAPE, the methods are 14.7% more accurate than the CF model and 9.2% better than the CBF model. These results demonstrate that CDSRS systems can provide more accurate recommendations and customized sleep services to users than CF, CBF, and combination models. As a result, CDSRS, a hybrid learning method, can better reflect a user’s evaluation than traditional methods and can increase the accuracy of recommendations as the number of users increases.