Search

Emotion Support Conversation (ESC) is a crucial application, which aims to reduce human stress, offer emotional guidance, and ultimately enhance human mental and physical well-being. With the advancement of Large Language Models (LLMs), many researchers have employed LLMs as the ESC models. However, the evaluation of these LLM-based ESCs remains uncertain. Inspired by the awesome development of role-playing agents, we propose an ESC Evaluation framework (ESC-Eval), which uses a role-playing agent to interact with ESC models, followed by a manual evaluation of the interactive dialogues. In detail, we first re-organize 2,801 role-playing cards from seven existing datasets to define the roles of the role-playing agent. Second, we train a specific role-playing model called ESC-Role which behaves more like a confused person than GPT-4. Third, through ESC-Role and organized role cards, we systematically conduct experiments using 14 LLMs as the ESC models, including general AI-assistant LLMs (ChatGPT) and ESC-oriented LLMs (ExTES-Llama). We conduct comprehensive human annotations on interactive multi-turn dialogues of different ESC models. The results show that ESC-oriented LLMs exhibit superior ESC abilities compared to general AI-assistant LLMs, but there is still a gap behind human performance. Moreover, to automate the scoring process for future ESC models, we developed ESC-RANK, which trained on the annotated data, achieving a scoring performance surpassing 35 points of GPT-4. Our data and code are available at https://github.com/haidequanbu/ESC-Eval., Comment: Pre-print

Powered by remarkable advancements in Large Language Models (LLMs), Multimodal Large Language Models (MLLMs) demonstrate impressive capabilities in manifold tasks. However, the practical application scenarios of MLLMs are intricate, exposing them to potential malicious instructions and thereby posing safety risks. While current benchmarks do incorporate certain safety considerations, they often lack comprehensive coverage and fail to exhibit the necessary rigor and robustness. For instance, the common practice of employing GPT-4V as both the evaluator and a model to be evaluated lacks credibility, as it tends to exhibit a bias toward its own responses. In this paper, we present MLLMGuard, a multidimensional safety evaluation suite for MLLMs, including a bilingual image-text evaluation dataset, inference utilities, and a lightweight evaluator. MLLMGuard's assessment comprehensively covers two languages (English and Chinese) and five important safety dimensions (Privacy, Bias, Toxicity, Truthfulness, and Legality), each with corresponding rich subtasks. Focusing on these dimensions, our evaluation dataset is primarily sourced from platforms such as social media, and it integrates text-based and image-based red teaming techniques with meticulous annotation by human experts. This can prevent inaccurate evaluation caused by data leakage when using open-source datasets and ensures the quality and challenging nature of our benchmark. Additionally, a fully automated lightweight evaluator termed GuardRank is developed, which achieves significantly higher evaluation accuracy than GPT-4. Our evaluation results across 13 advanced models indicate that MLLMs still have a substantial journey ahead before they can be considered safe and responsible.

Although the known maximum total generalized correntropy (MTGC) and generalized maximum blakezisserman total correntropy (GMBZTC) algorithms can maintain good performance under the errors-in-variables (EIV) model disrupted by generalized Gaussian noise, their requirement for manual ad-justment of parameters is excessive, greatly increasing the practical difficulty of use. To solve this problem, the total arctangent based on logical distance metric (TACLDM) algo-rithm is proposed by utilizing the advantage of few parameters in logical distance metric (LDM) theory and the convergence behavior is improved by the arctangent function. Compared with other competing algorithms, the TACLDM algorithm not only has fewer parameters, but also has better robustness to generalized Gaussian noise and significantly reduces the steady-state error. Furthermore, the analysis of the algorithm in the generalized Gaussian noise environment is analyzed in detail in this paper. Finally, computer simulations demonstrate the outstanding performance of the TACLDM algorithm and the rigorous theoretical deduction in this paper., Comment: 10 page

In recent years, multi-modal entity linking (MEL) has garnered increasing attention in the research community due to its significance in numerous multi-modal applications. Video, as a popular means of information transmission, has become prevalent in people's daily lives. However, most existing MEL methods primarily focus on linking textual and visual mentions or offline videos's mentions to entities in multi-modal knowledge bases, with limited efforts devoted to linking mentions within online video content. In this paper, we propose a task called Online Video Entity Linking OVEL, aiming to establish connections between mentions in online videos and a knowledge base with high accuracy and timeliness. To facilitate the research works of OVEL, we specifically concentrate on live delivery scenarios and construct a live delivery entity linking dataset called LIVE. Besides, we propose an evaluation metric that considers timelessness, robustness, and accuracy. Furthermore, to effectively handle OVEL task, we leverage a memory block managed by a Large Language Model and retrieve entity candidates from the knowledge base to augment LLM performance on memory management. The experimental results prove the effectiveness and efficiency of our method., Comment: 13 pages, 6 figures

The minimum error entropy (MEE) has been extensively used in unscented Kalman filter (UKF) to handle impulsive noises or abnormal measurement data in non-Gaussian systems. However, the MEE-UKF has poor numerical stability due to the inverse operation of singular matrix. In this paper, a novel UKF based on minimum error entropy with fiducial points (MEEF) is proposed \textcolor{black}{to improve the problem of non-positive definite key matrix. By adding the correntropy to the error entropy, the proposed algorithm further enhances the ability of suppressing impulse noise and outliers. At the same time, considering the uncertainty of noise distribution, the modified Sage-Husa estimator of noise statistics is introduced to adaptively update the noise covariance matrix. In addition, the convergence analysis of the proposed algorithm provides a guidance for the selection of kernel width. The robustness and estimation accuracy of the proposed algorithm are manifested by the state tracking examples under complex non-Gaussian noises., Comment: 29 pages,6 figures

The conventional Minimum Error Entropy criterion (MEE) has its limitations, showing reduced sensitivity to error mean values and uncertainty regarding error probability density function locations. To overcome this, a MEE with fiducial points criterion (MEEF), was presented. However, the efficacy of the MEEF is not consistent due to its reliance on a fixed Gaussian kernel. In this paper, a generalized minimum error with fiducial points criterion (GMEEF) is presented by adopting the Generalized Gaussian Density (GGD) function as kernel. The GGD extends the Gaussian distribution by introducing a shape parameter that provides more control over the tail behavior and peakedness. In addition, due to the high computational complexity of GMEEF criterion, the quantized idea is introduced to notably lower the computational load of the GMEEF-type algorithm. Finally, the proposed criterions are introduced to the domains of adaptive filter, kernel recursive algorithm, and multilayer perceptron. Several numerical simulations, which contain system identification, acoustic echo cancellation, times series prediction, and supervised classification, indicate that the novel algorithms' performance performs excellently., Comment: 12 pages, 9 figures

Limited by fixed step-size and sparsity penalty factor, the conventional sparsity-aware normalized subband adaptive filtering (NSAF) type algorithms suffer from trade-off requirements of high filtering accurateness and quicker convergence behavior for sparse system identification. To deal with this problem, this paper proposes variable step-size L0-norm constraint NSAF algorithms (VSS-L0-NSAFs). We first analyze mean-square-deviation (MSD) statistics behavior of the L0-NSAF innovatively in according to novel weight recursion form and arrive at corresponding expressions for the cases that background noise variance is available and unavailable, where correlation degree of system input is indicated by scaling parameter r. Based on derivations, we develop an effective variable step-size scheme through minimizing the upper bounds of the MSD under some reasonable assumptions and lemma. Furthermore, an effective reset strategy is incorporated into presented algorithms to tackle with non-stationary situations. Finally, numerical simulations corroborate that the proposed algorithms achieve better performance in terms of estimation accurateness and tracking capability in comparison with existing related algorithms in sparse system identification and adaptive echo cancellation circumstances., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

When the input signal is correlated input signals, and the input and output signal is contaminated by Gaussian noise, the total least squares normalized subband adaptive filter (TLS-NSAF) algorithm shows good performance. However, when it is disturbed by impulse noise, the TLS-NSAF algorithm shows the rapidly deteriorating convergence performance. To solve this problem, this paper proposed the robust total minimum mean M-estimator normalized subband filter (TLMM-NSAF) algorithm. In addition, this paper also conducts a detailed theoretical performance analysis of the TLMM-NSAF algorithm and obtains the stable step size range and theoretical steady-state mean squared deviation (MSD) of the algorithm. To further improve the performance of the algorithm, we also propose a new variable step size (VSS) method of the algorithm. Finally, the robustness of our proposed algorithm and the consistency of theoretical and simulated values are verified by computer simulations of system identification and echo cancellation under different noise models.

In recent years, the multitask diffusion least mean square (MD-LMS) algorithm has been extensively applied in the distributed parameter estimation and target tracking of multitask network. However, its performance is mainly limited by two aspects, i.e, the correlated input signal and impulsive noise interference. To overcome these two limitations simultaneously, this paper firstly introduces the subband adaptive filter (SAF) into the multitask network. Then, a robust multitask diffusion normalized M-estimate subband adaptive filtering (MD-NMSAF) algorithm is proposed by solving the modified Huber function based global network optimization problem in a distributed manner, which endows the multitask network strong decorrelation ability for correlated inputs and robustness to impulsive noise interference, and accelerates the convergence of the algorithm significantly. Compared with the robust multitask diffusion affine projection M-estimate (MD-APM) algorithm, the computational complexity of the proposed MD-NMSAF is greatly reduced. In addition, the stability condition, the analytical expressions of the theoretical transient and steady-state network mean square deviation (MSD) of the MD-NMSAF are also provided and verified through computer simulations. Simulation results under different input signals and impulsive noise environment fully demonstrate the performance advantages of the MD-NMSAF algorithm over some other competitors in terms of steady-state accuracy and tracking speed.

The bias-compensated set-membership normalised LMS (BCSMNLMS) algorithm is proposed based on the concept of set-membership filtering, which incorporates the bias-compensation technique to mitigate the negative effect of noisy inputs. Moreover, an efficient regression noise variance estimation method is developed by taking the iterative-shrinkage method. Simulations in the context of system identification demonstrate that the misalignment of the proposed BCSM-NLMS algorithm is low for noisy inputs.

Recently, the leaky diffusion least-mean-square (DLMS) algorithm has obtained much attention because of its good performance for high input eigenvalue spread and low signal-to-noise ratio (SNR). However, the leaky DLMS algorithm may suffer from performance deterioration in the sparse system. To overcome this drawback, the leaky zero attracting DLMS (LZA-DLMS) algorithm is developed in this paper, which adds an l1-norm penalty to the cost function to exploit the property of sparse system. The leaky reweighted zero attracting DLMS (LRZA-DLMS) algorithm is also put forward, which can improve the estimation performance in the presence of time-varying sparsity. Instead of using the l1-norm penalty, in the reweighted version, a log-sum function is employed as the substitution. Based on the weight error variance relation and several common assumptions, we analyze the transient behavior of our findings and determine the stability bound of the step-size. Moreover, we implement the steady state theoretical analysis for the proposed algorithms. Simulations in the context of distributed network system identification illustrate that the proposed schemes outperform various existing algorithms and validate the accuracy of the theoretical results.

This paper studies the statistical models of the noise-robust normalized subband adaptive filter (NR-NSAF) algorithm in the mean and mean square deviation senses involving transient-state and steady-state behavior by resorting to the method of the vectorization operation and the Kronecker product. The analysis method does not require the Gaussian input signal. Moreover, the proposed analysis removes the paraunitary assumption imposed on the analysis filter banks as in the existing analyses of subband adaptive algorithms. Simulation results in various conditions demonstrate the effectiveness of our theoretical analysis. For a special form of the algorithm, the proposed steady-state expression is also better accurate than the previous analysis., Comment: 7 pages,8 figures

The maximum correntropy criterion (MCC) has recently been successfully applied in robust regression, classification and adaptive filtering, where the correntropy is maximized instead of minimizing the well-known mean square error (MSE) to improve the robustness with respect to outliers (or impulsive noises). Considerable efforts have been devoted to develop various robust adaptive algorithms under MCC, but so far little insight has been gained as to how the optimal solution will be affected by outliers. In this work, we study this problem in the context of parameter estimation for a simple linear errors-in-variables (EIV) model where all variables are scalar. Under certain conditions, we derive an upper bound on the absolute value of the estimation error and show that the optimal solution under MCC can be very close to the true value of the unknown parameter even with outliers (whose values can be arbitrarily large) in both input and output variables. Illustrative examples are presented to verify and clarify the theory., Comment: 8 pages, 7 figures

The normalized subband adaptive filter (NSAF) is widely accepted as a preeminent adaptive filtering algorithm because of its efficiency under the colored excitation. However, the convergence rate of NSAF is slow. To address this drawback, in this paper, a variant of the NSAF, called the differential evolution (DE)-NSAF (DE-NSAF), is proposed for channel estimation based on DE strategy. It is worth noticing that there are several papers concerning designing DE strategies for adaptive filter. But their signal models are still the single adaptive filter model rather than the fullband adaptive filter model considered in this paper. Thus, the problem considered in our work is quite different from those. The proposed DE-NSAF algorithm is based on real-valued manipulations and has fast convergence rate for searching the global solution of optimized weight vector. Moreover, a design step of new algorithm is given in detail. Simulation results demonstrate the improved performance of the proposed DE-NSAF algorithm in terms of the convergence rate., Comment: 7 pages, 4 figures

As a well-established adaptation criterion, the maximum correntropy criterion (MCC) has been receiving increasing attention due to its robust against outliers. In this paper, a new complex recursive maximum correntropy (CRMC) algorithm without any priori information on the noise characteristics, is proposed under the MCC. The proposed algorithm is useful for adaptive beamforming, when the desired signal is contaminated by the impulsive noises. Moreover, the analysis of convergence property of the CRMC algorithm is performed. The results obtained from simulation study establish the effectiveness of this new beamformer., Comment: arXiv admin note: substantial text overlap with arXiv:1701.08407

Recently, the normalized subband adaptive filter (NSAF) algorithm has attracted much attention for handling the colored input signals. Based on the first-order Markov model of the optimal tap-weight vector, this paper provides a convergence analysis of the standard NSAF. Following the analysis, both the step size and the regularization parameter in the NSAF are jointly optimized in such a way that minimizes the mean square deviation. The resulting joint-optimization step size and regularization parameter (JOSR-NSAF) algorithm achieves a good tradeoff between fast convergence rate and low steady-state error. Simulation results in the context of acoustic echo cancellation demonstrate good features of the proposed algorithm., Comment: 8 pages, 4 figures, accepted by Signal, Image and Video Processing on 18-Sep-2016

Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher-order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non-Gaussian signal processing and machine learning. In this work, we propose a new similarity measure in kernel space, called the kernel risk-sensitive loss (KRSL), and provide some important properties. We apply the KRSL to adaptive filtering and investigate the robustness, and then develop the MKRSL algorithm and analyze the mean square convergence performance. Compared with correntropy, the KRSL can offer a more efficient performance surface, thereby enabling a gradient based method to achieve faster convergence speed and higher accuracy while still maintaining the robustness to outliers. Theoretical analysis results and superior performance of the new algorithm are confirmed by simulation., Comment: 12 pages,8 figures

In this Letter, we present a novel class of diffusion algorithms that can be used to estimate the coefficients of sparse Volterra network (SVN). The development of the algorithms is based on the logarithmic cost and l0-norm constraint. Simulations for Gaussian and impulsive scenarios are conducted to demonstrate the superior performance of the proposed algorithms as compared with the existing algorithms.