Search

Lowering blood cholesterol levels is crucial for reducing the risk of cardiovascular disease in patients with familial hypercholesterolemia. To develop Perilla frutescens (L.) Britt. leaves as a functional food with a cholesterol-lowering effect, in this study, we collected P. frutescens (L.) Britt. leaves from different regions of China and Republic of Korea. On the basis of the extraction yield (all components; g/kg), we selected P. frutescens (L.) Britt. leaves from Hebei Province, China with an extract yield of 60.9 g/kg. After evaluating different concentrations of ethanol/water solvent for P. frutescens (L.) Britt. leaves, with luteolin 7-glucuronide as the indicator component, we selected a 30% ethanol/water solvent with a high luteolin 7-glucuronide content of 0.548 mg/g in Perilla. frutescens (L.) Britt. leaves. Subsequently, we evaluated the cholesterol-lowering effects of P. frutescens (L.) Britt. leaf extract and luteolin 7-glucuronide by detecting total cholesterol in HepG2 cells. The 30% ethanol extract lowered cholesterol levels significantly by downregulating 3-hydroxy-3-methyl-glutaryl-coenzyme A reductase expression. This suggests that P. frutescens (L.) Britt leaves have significant health benefits and can be explored as a potentially promising food additive for the prevention of hypercholesterolemia-related diseases. [ABSTRACT FROM AUTHOR]

Most facial analysis methods perform well in standardized testing but not in real-world testing. The main reason is that training models cannot easily learn various human features and background noise, especially for facial landmark detection and head pose estimation tasks with limited and noisy training datasets. To alleviate the gap between standardized and real-world testing, we propose a pseudo-labeling technique using a face recognition dataset consisting of various people and background noise. The use of our pseudo-labeled training dataset can help to overcome the lack of diversity among the people in the dataset. Our integrated framework is constructed using complementary multitask learning methods to extract robust features for each task. Furthermore, introducing pseudo-labeling and multitask learning improves the face recognition performance by enabling the learning of pose-invariant features. Our method achieves state-of-the-art (SOTA) or near-SOTA performance on the AFLW2000-3D and BIWI datasets for facial landmark detection and head pose estimation, with competitive face verification performance on the IJB-C test dataset for face recognition. We demonstrate this through a novel testing methodology that categorizes cases as soft, medium, and hard based on the pose values of IJB-C. The proposed method achieves stable performance even when the dataset lacks diverse face identifications. [ABSTRACT FROM AUTHOR]

Video-based person re-identification (ReID) aims to exploit relevant features from spatial and temporal knowledge. Widely used methods include the part- and attention-based approaches for suppressing irrelevant spatial–temporal features. However, it is still challenging to overcome inconsistencies across video frames due to occlusion and imperfect detection. These mismatches make temporal processing ineffective and create an imbalance of crucial spatial information. To address these problems, we propose the Spatiotemporal Multi-Granularity Aggregation (ST-MGA) method, which is specifically designed to accumulate relevant features with spatiotemporally consistent cues. The proposed framework consists of three main stages: extraction, which extracts spatiotemporally consistent partial information; augmentation, which augments the partial information with different granularity levels; and aggregation, which effectively aggregates the augmented spatiotemporal information. We first introduce the consistent part-attention (CPA) module, which extracts spatiotemporally consistent and well-aligned attentive parts. Sub-parts derived from CPA provide temporally consistent semantic information, solving misalignment problems in videos due to occlusion or inaccurate detection, and maximize the efficiency of aggregation through uniform partial information. To enhance the diversity of spatial and temporal cues, we introduce the Multi-Attention Part Augmentation (MA-PA) block, which incorporates fine parts at various granular levels, and the Long-/Short-term Temporal Augmentation (LS-TA) block, designed to capture both long- and short-term temporal relations. Using densely separated part cues, ST-MGA fully exploits and aggregates the spatiotemporal multi-granular patterns by comparing relations between parts and scales. In the experiments, the proposed ST-MGA renders state-of-the-art performance on several video-based ReID benchmarks (i.e., MARS, DukeMTMC-VideoReID, and LS-VID). [ABSTRACT FROM AUTHOR]

Most studies on convolutional Neural Network (CNN) based image processing have proposed networks that can be optimized for a single level. Here, the term "level" refers to the specific objective defined for each task, such as the degree of noise in denoising tasks. Hence, they underperform on other levels and must be retrained to deliver optimal performance. Using multiple models to cover multiple levels involves very high computational costs. To solve these problems, recent approaches train the networks on two different levels and propose their own modulation methods to enable the arbitrary intermediate levels. However, many of them 1) have difficulty adapting from one level to the other, 2) suffer from unintended artifacts in the intermediate levels, or 3) require large memory and computational cost. In this paper, we propose a novel framework using Filter Transition Network (FTN), which is a non-linear module that easily adapts to new levels, is regularized to prevent undesirable side-effects, and extremely lightweight being a data-independent module. Additionally, for stable learning of FTN, we newly propose a method to initialize nonlinear CNNs with identity mappings. Extensive results for various image processing tasks indicate that the performance of FTN is stable regarding adaptation and modulation and is comparable to that of the other heavy frameworks. [ABSTRACT FROM AUTHOR]

The decision tree is one of the most effective tools for deriving meaningful outcomes from image data acquired from the visual sensors. Owing to its reliability, superior generalization abilities, and easy implementation, the tree model has been widely used in various applications. However, in image classification problems, conventional tree methods use only a few sparse attributes as the splitting criterion. Consequently, they suffer from several drawbacks in terms of performance and environmental sensitivity. To overcome these limitations, this paper introduces a new tree induction algorithm that classifies images on the basis of local area learning. To train our predictive model, we extract a random local area within the image and use it as a feature for classification. In addition, the self-organizing map, which is a clustering technique, is used for node learning.We also adopt a random sampled optimization technique to search for the optimal node. Finally, each trained node stores the weights that represent the training data and class probabilities. Thus, a recursively trained tree classifies the data hierarchically based on the local similarity at each node. The proposed tree is a type of predictive model that offers benefits in terms of image's semantic energy conservation compared with conventional tree methods. Consequently, it exhibits improved performance under various conditions, such as noise and illumination changes. Moreover, the proposed algorithm can improve the generalization ability owing to its randomness. In addition, it can be easily applied to ensemble techniques. To evaluate the performance of the proposed algorithm, we perform quantitative and qualitative comparisons with various tree-based methods using four image datasets. The results show that our algorithm not only involves a lower classification error than the conventional methods but also exhibits stable performance even under unfavorable conditions such as noise and illumination changes. [ABSTRACT FROM AUTHOR]

Multi-object tracking (MOT), which is crucial for computer vision and video processing, has immense potential for improvement. Traditional tracking-by-detection approaches include feature-based object re-identification methods that use trained features, but these methods suffer from a lack of suitable training data. In training datasets used for MOT, every object in a video sequence must have its own location and ID. However, assigning IDs to each object in every sequence is considerably labor-intensive, and hence current MOT datasets are unsuitable for training re-identification networks. To resolve this issue, this paper proposes a novel self-supervised learning method using several short videos that contain no human-added labels, based on the idea that each video is a set of temporally corresponding image frames. We then describe how to improve tracking performance using a re-identification network trained in a self-supervised manner. In addition, ablation studies were conducted in order to define the optimal parameters, such as number of clips, data augmentation, and appropriate matching algorithms. The proposed approach achieved competitive performance compared with current best-practice methods including supervised methods, achieving MOT accuracy = 62.0% and ID F1-score = 62.7% on the MOT17 benchmark. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a novel thermal three-dimensional (3D) modeling system that includes 3D shape, visual, and thermal infrared information and solves a registration problem among these three types of information. The proposed system consists of a projector, a visual camera and, a thermal camera (PVT). To generate 3D shape information, we use a structured light technique, which consists of a visual camera and a projector. A thermal camera is added to the structured light system in order to provide thermal information. To solve the correspondence problem between the three sensors, we use three-view geometry. Finally, we obtain registered PVT data, which includes visual, thermal, and 3D shape information. Among various potential applications such as industrial measurements, biological experiments, military usage, and so on, we have adapted the proposed method to biometrics, particularly for face recognition. With the proposed method, we obtain multi-modal 3D face data that includes not only textural information but also data regarding head pose, 3D shape, and thermal information. Experimental results show that the performance of the proposed face recognition system is not limited by head pose variation which is a serious problem in face recognition. [Copyright &y& Elsevier]

Abstract: The receiver operating characteristics (ROC) curve has been extensively used for performance evaluation in multimodal biometrics fusion. However, the processes of fusion classifier design and the final ROC performance evaluation are usually conducted separately. This has been inevitable because the ROC, when taken from the error counting point of view, does not have a well-posed structure linking to the fusion classifier of interest. In this work, we propose to optimize the ROC performance directly according to the fusion classifier design. The area under the ROC curve (AUC) will be used as the optimization objective since it provides a good representation of the ROC performance. Due to the piecewise cumulative structure of the AUC, a smooth approximate formulation is proposed. This enables a direct optimization of the AUC with respect to the classifier parameters. When a fusion classifier has linear parameters, computation of the solution to optimize a quadratic AUC approximation is surprisingly simple and yet effective. Our empirical experiments on biometrics fusion show strong evidences regarding the potential of the proposed method. [Copyright &y& Elsevier]

Abstract: Lately, the once powerful one-factor authentication which is based solely on either password, token or biometric approach, appears to be insufficient in addressing the challenges of identity frauds. For example, the sole biometric approach suffers from the privacy invasion and non-revocable issues. Passwords and tokens are easily forgotten and lost. To address these issues, the notion of cancellable biometrics was introduced to denote biometric templates that can be cancelled and replaced with the inclusion of another independent authentication factor. BioHash is a form of cancellable biometrics which mixes a set of user-specific random vectors with biometric features. In verification setting, BioHash is able to deliver extremely low error rates as compared to the sole biometric approach when a genuine token is used. However, this raises the possibility of two identity theft scenarios: (i) stolen-biometrics, in which an impostor possesses intercepted biometric data of sufficient high quality to be considered genuine and (ii) stolen-token, in which an impostor has access to the genuine token and used by the impostor to claim as the genuine user. We found that the recognition rate for the latter case is poorer. In this paper, the quantised random projection ensemble based on the Johnson–Lindenstrauss Lemma is used to establish the mathematical foundation of BioHash. Based on this model, we elucidate the characteristics of BioHash in pattern recognition as well as security view points and propose new methods to rectify the stolen-token problem. [Copyright &y& Elsevier]

Abstract: This paper proposes a weighted power series model for face verification scores fusion. Essentially, a linear parametric power series model is adopted to directly minimize an approximated total error rate for fusion of multi-modal face verification scores. Unlike the conventional least-squares error minimization approach which involves fitting of a learning model to data density and then perform a threshold process for error counting, this work directly formulates the required target error count rate in terms of design model parameters with a closed-form solution. The solution is found to belong to a specific setting of the weighted least squares. Our experiments on fusing scores from visual and infra-red face images as well as on public data sets show promising results. [Copyright &y& Elsevier]

Abstract: This paper addresses the biometric scores fusion problem from the error rate minimization point of view. Comparing to the conventional approach which treats fusion classifier design and performance evaluation as a two-stage process, this work directly optimizes the target performance with respect to fusion classifier design. Based on a smooth approximation to the total error rate of identity verification, a deterministic solution is proposed to solve the fusion optimization problem. The proposed method is applied to a face and iris verification fusion problem addressing the demand for high security in the modern networked society. Our empirical evaluations show promising potential in terms of decision accuracy and computing efficiency. [Copyright &y& Elsevier]

• This paper explores the potential of spatial feature dimensions to seek an effective feature representation for point clouds. • The dimension expansion of MKConv enables spatially correlated feature representation and discrete convolutions for point cloud data without information loss. • For stable and effective learning, kernel weight normalization and multidimensional attention are proposed. • The effectiveness of multidimensional feature representation is demonstrated by the superior performance on three different point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation. Despite the remarkable success of deep learning, an optimal convolution operation on point clouds remains elusive owing to their irregular data structure. Existing methods mainly focus on designing an effective continuous kernel function that can handle an arbitrary point in continuous space. Various approaches exhibiting high performance have been proposed, but we observe that the standard pointwise feature is represented by 1D channels and can become more informative when its representation involves additional spatial feature dimensions. In this paper, we present Multidimensional Kernel Convolution (MKConv), a novel convolution operator that learns to transform the point feature representation from a vector to a multidimensional matrix. Unlike standard point convolution, MKConv proceeds via two steps. (i) It first activates the spatial dimensions of local feature representation by exploiting multidimensional kernel weights. These spatially expanded features can represent their embedded information through spatial correlation as well as channel correlation in feature space, carrying more detailed local structure information. (ii) Then, discrete convolutions are applied to the multidimensional features which can be regarded as a grid-structured matrix. In this way, we can utilize the discrete convolutions for point cloud data without voxelization that suffers from information loss. Furthermore, we propose a spatial attention module, Multidimensional Local Attention (MLA), to provide comprehensive structure awareness within the local point set by reweighting the spatial feature dimensions. We demonstrate that MKConv has excellent applicability to point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation with superior results. [ABSTRACT FROM AUTHOR]

Ever since the availability of real-time three-dimensional (3D) data acquisition sensors such as time-of-flight and Kinect depth sensor, the performance of gesture recognition can be largely enhanced. However, since conventional two-dimensional (2D) image based feature extraction methods such as local binary pattern (LBP) generally use texture information, they cannot be applied to depth or range image which does not contain texture information. In this paper, we propose an adaptive local binary pattern (ALBP) for effective depth images based applications. Contrasting to the conventional LBP which is only rotation invariant, the proposed ALBP is invariant to both rotation and the depth distance in range images. Using ALBP, we can extract object features without using texture or color information. We further apply the proposed ALBP for hand tracking using depth images to show its effectiveness and its usefulness. Our experimental results validate the proposal. [ABSTRACT FROM AUTHOR]

• Surveillance anomaly detection is critical in our daily life that replaces inefficient human monitoring with an automated system and provides various pattern recognition applications. • In this paper, novel architecture ITAE learns normal appearance and motion patterns by implicitly capturing static and dynamic features. • By utilizing normalizing flow generative model, we are the first to estimate the distribution of appearance and motion surveillance video features. • The proposed approach achieves superior performance on six surveillance anomaly detection benchmarks and demonstrates its effectiveness of generalization ability which is crucial issue in real-world scenarios. In contemporary society, surveillance anomaly detection, i.e., spotting anomalous events such as crimes or accidents in surveillance videos, is a critical task. As anomalies occur rarely, most training data consists of unlabeled videos without anomalous events, which makes the task challenging. Most existing methods use an autoencoder (AE) to learn to reconstruct normal videos; they then detect anomalies based on their failure to reconstruct the appearance of abnormal scenes. However, because anomalies are distinguished by appearance as well as motion, many previous approaches have explicitly separated appearance and motion informationfor example, using a pre-trained optical flow model. This explicit separation restricts reciprocal representation capabilities between two types of information. In contrast, we propose an implicit two-path AE (ITAE), a structure in which two encoders implicitly model appearance and motion features, along with a single decoder that combines them to learn normal video patterns. For the complex distribution of normal scenes, we suggest normal density estimation of ITAE features through normalizing flow (NF)-based generative models to learn the tractable likelihoods and identify anomalies using out-of-distribution detection. NF models intensify ITAE performance by learning normality through implicitly learned features. Finally, we demonstrate the effectiveness of ITAE and its feature distribution modeling on six benchmarks, including databases that contain various anomalies in real-world scenarios. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) is developed for next-generation video coding, which achieves significant improvements in coding efficiency compared with H.264/Advanced Video Coding by adopting various tools including a quadtree-based block partitioning structure. However, this causes high encoding complexity for exhaustive rate-distortion (RD) cost computation of the extended prediction unit (PU) searching. In this paper, a fast PU skip and split termination algorithm is proposed. The proposed method consists of three algorithms: 1) early skip; 2) PU skip; and 3) PU split termination. The early skip algorithm allows immediate skipping of the RD cost computation for large PUs according to the neighboring PUs. Based on Bayes’s rule, the PU skip algorithm allows skipping of the full RD cost computation, and the split termination algorithm terminates further PU splitting using the RD cost of rough mode decision (RMD). The decision parameter for the PU skip and the split termination is presented as the ratio of the RMD RD costs between the current PU and the spatially adjacent or upper depth PU. The simulation results show that the proposed algorithm achieves a saving of 53.52% encoding time while maintaining almost the same RD performances as the HEVC reference software. [ABSTRACT FROM PUBLISHER]

High Efficiency Video Coding (HEVC) employs a coding unit (CU), prediction unit (PU), and transform unit (TU) based on the quadtree coding tree unit (CTU) structure to improve coding efficiency. However, the computational complexity increases greatly because the rate-distortion (RD) optimization process should be performed for all CUs, PUs, and TUs to obtain the optimal CTU partition. In this paper, a fast CU size decision algorithm is proposed to reduce the encoder complexity of HEVC. Based on the statistical analysis, three approaches with SKIP mode decision (SMD), CU skip estimation (CUSE), and early CU termination (ECUT) are considered. In SMD, it is determined that the remaining modes except for SKIP mode are preformed or not. CUSE and ECUT determine that larger CU sizes and smaller CU sizes are coded or not, respectively. Thresholds for SMD, CUSE, and ECUT are designed based on Bayes’ rule with a complexity factor. Update process is performed to estimate the statistical parameters for SMD, CUSE, and ECUT considering the characteristic of RD cost. The experimental results demonstrate that the proposed CU size decision algorithm significantly reduces computational complexity by 69% on average with 2.99% Bjøntegaard difference bitrate (BDBR) increase for random access. The complexity reduction and BDBR increase for low delay are 68% and 2.46%, respectively. The experimental results also show that our proposed scheme performs well for various characteristic of sequences and outperforms the two previous state-of-the-art works. [ABSTRACT FROM PUBLISHER]

Abstract: We propose a face detection method based on skin color likelihood via a boosting algorithm which emphasizes skin color information while deemphasizing non-skin color information. A stochastic model is adapted to compute the similarity between a color region and the skin color. Both Haar-like features and Local Binary Pattern (LBP) features are utilized to build a cascaded classifier. The boosted classifier is implemented based on skin color emphasis to localize the face region from a color image. Based on our experiments, the proposed method shows good tolerance to face pose variation and complex background with significant improvements over classical boosting-based classifiers in terms of total error rate performance. [Copyright &y& Elsevier]

The latest video coding standard H.264/AVC achieves significant coding efficiency by adopting variable block sizes for inter mode decision. However, this technique brings about high computational complexity since the optimal mode is determined by exhaustively calculating rate-distortion cost (RDcost) of all modes. To solve this problem, fast inter mode decision algorithm is proposed, as follows. Based on high correlation between current and previous frames, the motion vector based-moving window (MVMW) scheme predicts the RDcost for each mode, and the predicted RDcosts are used as thresholds for selecting the optimal mode. The experimental results show that the proposed algorithm achieves 71.78% time saving on average with negligible PSNR loss and bit rate increment. This feature will help to develop real time video applications such as video conference and telephone. [ABSTRACT FROM PUBLISHER]

Abstract: The primary goal of linear discriminant analysis (LDA) in face feature extraction is to find an effective subspace for identity discrimination. The introduction of kernel trick has extended the LDA to nonlinear decision hypersurface. However, there remained inherent limitations for the nonlinear LDA to deal with physical applications under complex environmental factors. These limitations include the use of a common covariance function among each class, and the limited dimensionality inherent to the definition of the between-class scatter. Since these problems are inherently caused by the definition of the Fisher''s criterion itself, they may not be solvable under the conventional LDA framework. This paper proposes to adopt a margin-based between-class scatter and a regularization process to resolve the issue. Essentially, we redesign the between-class scatter matrix based on the SVM margins to facilitate an effective and reliable feature extraction. This is followed by a regularization of the within-class scatter matrix. Extensive empirical experiments are performed to compare the proposed method with several other variants of the LDA method using the FERET, AR, and CMU-PIE databases. [Copyright &y& Elsevier]

Multi-person pose estimation has been gaining considerable interest due to its use in several real-world applications, such as activity recognition, motion capture, and augmented reality. Although the improvement of the accuracy and speed of multi-person pose estimation techniques has been recently studied, limitations still exist in balancing these two aspects. In this paper, a novel knowledge distilled lightweight top-down pose network (KDLPN) is proposed that balances computational complexity and accuracy. For the first time in multi-person pose estimation, a network that reduces computational complexity by applying a "Pelee" structure and shuffles pixels in the dense upsampling convolution layer to reduce the number of channels is presented. Furthermore, to prevent performance degradation because of the reduced computational complexity, knowledge distillation is applied to establish the pose estimation network as a teacher network. The method performance is evaluated on the MSCOCO dataset. Experimental results demonstrate that our KDLPN network significantly reduces 95% of the parameters required by state-of-the-art methods with minimal performance degradation. Moreover, our method is compared with other pose estimation methods to substantiate the importance of computational complexity reduction and its effectiveness. [ABSTRACT FROM AUTHOR]

In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods. [ABSTRACT FROM AUTHOR]

The H.264/AVC standard achieves high compression performance compared to existing video coding standards by employing the rate distortion optimization (RDO) technique. All possible combinations of modes are calculated to find the minimum rate distortion cost which results in significant computational complexity. In this paper, we propose a fast block size and mode decision method for intra prediction, which can effectively reduce computational burden. For fast block size decision, intra16x16 is selectively searched based on the homogeneity of inner-8x8 block of macroblock (MB). For fast mode decision, the search of intra16x16 and intra4x4 prediction modes is restricted by the similarity of the reference pixels. Moreover, by considering the similarities of the upper reference pixels and left reference pixels separately, considerable reduction of computational complexity is achieved. The results of simulations demonstrate maxima of 79% and 77% average time savings with negligible loss in PSNRY and bitrate. [ABSTRACT FROM PUBLISHER]