Search

Most facial analysis methods perform well in standardized testing but not in real-world testing. The main reason is that training models cannot easily learn various human features and background noise, especially for facial landmark detection and head pose estimation tasks with limited and noisy training datasets. To alleviate the gap between standardized and real-world testing, we propose a pseudo-labeling technique using a face recognition dataset consisting of various people and background noise. The use of our pseudo-labeled training dataset can help to overcome the lack of diversity among the people in the dataset. Our integrated framework is constructed using complementary multitask learning methods to extract robust features for each task. Furthermore, introducing pseudo-labeling and multitask learning improves the face recognition performance by enabling the learning of pose-invariant features. Our method achieves state-of-the-art (SOTA) or near-SOTA performance on the AFLW2000-3D and BIWI datasets for facial landmark detection and head pose estimation, with competitive face verification performance on the IJB-C test dataset for face recognition. We demonstrate this through a novel testing methodology that categorizes cases as soft, medium, and hard based on the pose values of IJB-C. The proposed method achieves stable performance even when the dataset lacks diverse face identifications. [ABSTRACT FROM AUTHOR]

Video-based person re-identification (ReID) aims to exploit relevant features from spatial and temporal knowledge. Widely used methods include the part- and attention-based approaches for suppressing irrelevant spatial–temporal features. However, it is still challenging to overcome inconsistencies across video frames due to occlusion and imperfect detection. These mismatches make temporal processing ineffective and create an imbalance of crucial spatial information. To address these problems, we propose the Spatiotemporal Multi-Granularity Aggregation (ST-MGA) method, which is specifically designed to accumulate relevant features with spatiotemporally consistent cues. The proposed framework consists of three main stages: extraction, which extracts spatiotemporally consistent partial information; augmentation, which augments the partial information with different granularity levels; and aggregation, which effectively aggregates the augmented spatiotemporal information. We first introduce the consistent part-attention (CPA) module, which extracts spatiotemporally consistent and well-aligned attentive parts. Sub-parts derived from CPA provide temporally consistent semantic information, solving misalignment problems in videos due to occlusion or inaccurate detection, and maximize the efficiency of aggregation through uniform partial information. To enhance the diversity of spatial and temporal cues, we introduce the Multi-Attention Part Augmentation (MA-PA) block, which incorporates fine parts at various granular levels, and the Long-/Short-term Temporal Augmentation (LS-TA) block, designed to capture both long- and short-term temporal relations. Using densely separated part cues, ST-MGA fully exploits and aggregates the spatiotemporal multi-granular patterns by comparing relations between parts and scales. In the experiments, the proposed ST-MGA renders state-of-the-art performance on several video-based ReID benchmarks (i.e., MARS, DukeMTMC-VideoReID, and LS-VID). [ABSTRACT FROM AUTHOR]

Lowering blood cholesterol levels is crucial for reducing the risk of cardiovascular disease in patients with familial hypercholesterolemia. To develop Perilla frutescens (L.) Britt. leaves as a functional food with a cholesterol-lowering effect, in this study, we collected P. frutescens (L.) Britt. leaves from different regions of China and Republic of Korea. On the basis of the extraction yield (all components; g/kg), we selected P. frutescens (L.) Britt. leaves from Hebei Province, China with an extract yield of 60.9 g/kg. After evaluating different concentrations of ethanol/water solvent for P. frutescens (L.) Britt. leaves, with luteolin 7-glucuronide as the indicator component, we selected a 30% ethanol/water solvent with a high luteolin 7-glucuronide content of 0.548 mg/g in Perilla. frutescens (L.) Britt. leaves. Subsequently, we evaluated the cholesterol-lowering effects of P. frutescens (L.) Britt. leaf extract and luteolin 7-glucuronide by detecting total cholesterol in HepG2 cells. The 30% ethanol extract lowered cholesterol levels significantly by downregulating 3-hydroxy-3-methyl-glutaryl-coenzyme A reductase expression. This suggests that P. frutescens (L.) Britt leaves have significant health benefits and can be explored as a potentially promising food additive for the prevention of hypercholesterolemia-related diseases. [ABSTRACT FROM AUTHOR]

Multi-object tracking (MOT), which is crucial for computer vision and video processing, has immense potential for improvement. Traditional tracking-by-detection approaches include feature-based object re-identification methods that use trained features, but these methods suffer from a lack of suitable training data. In training datasets used for MOT, every object in a video sequence must have its own location and ID. However, assigning IDs to each object in every sequence is considerably labor-intensive, and hence current MOT datasets are unsuitable for training re-identification networks. To resolve this issue, this paper proposes a novel self-supervised learning method using several short videos that contain no human-added labels, based on the idea that each video is a set of temporally corresponding image frames. We then describe how to improve tracking performance using a re-identification network trained in a self-supervised manner. In addition, ablation studies were conducted in order to define the optimal parameters, such as number of clips, data augmentation, and appropriate matching algorithms. The proposed approach achieved competitive performance compared with current best-practice methods including supervised methods, achieving MOT accuracy = 62.0% and ID F1-score = 62.7% on the MOT17 benchmark. [ABSTRACT FROM AUTHOR]

Depth completion is the task of reconstructing dense depth images from sparse LiDAR data. LiDAR depth completion, for which LiDAR data is the only input, is an ill-posed and challenging problem owing to the underlying properties of LiDAR data: extremely few points, presence of discontinuities, and absence of texture information. Accordingly, most approaches are heavily dependent on guided color images, which leads to unsatisfactory results when the color images are degraded. To alleviate the dependency on color images but leverage this information during training, we present a deep convolutional neural network (CNN) consisting of depth and edge CNNs via transferring of knowledge. In order to compensate for the limitations of LiDAR data, we design the edge CNN to learn a gradient depth image from a powerful teacher network through the Knowledge-Distillation method. Since the teacher network is trained with color images, color-embedded information can be obtained in the test phase even if color images are not used as an input. We further propose a Self-Distillation method for transferring the color-embedded features from the edge CNN to the depth CNN. Enforcing the depth features to contain edge information hardly observed in LiDAR data enables the depth CNN to generate more edge-attentive and structure-preserving results. Our novel methods show remarkable results in outdoor and indoor environments for KITTI and NYU-Depth-V2 datasets. Experiments performed with low-channel LiDAR data in KITTI and few depth points in the NYU-Depth-V2 dataset show that our method is robust to data sparsity and applicable in various scenarios. [ABSTRACT FROM AUTHOR]

Various sensors can be attached and added to autonomous vehicles, included visual cameras, radar, LiDAR (Light Detection And Ranging), and GNSS (Global Navigation Satellite System). These sensors have been studied in many research areas, in particular studies on building precise 3D maps. It is essential for autonomous driving to create an accurate 3D map of the surrounding scene. However, creating an accurate static 3D map is difficult due to changes in moving objects or dynamic environments. Spurious objects on the 3D map can be handled by removing or ignoring them for 3D mapping. Following this idea, we propose an object segmentation and inpainting network. The proposed network called SAM-Net, addresses the object duplication issue by segmenting the objects and inpainting them with the segmentation results. Conventional inpainting research has dealt with RGB images. No matter how well such approaches reconstruct holes or corrupted images, they do not establish 3D points’ relationship with the point cloud frame. Therefore, we suggest a depth inpainting method for outdoor object segmentation and inpainting tasks that utilizes a high-precision depth range sensor (Velodyne HDL-64E), which is not suggested before. Unfortunately, no dataset exists for the outdoor depth inpainting task. Thus, to train our model, we generate a new dataset by locating objects on a clean static background. Moreover, our proposed method shows outstanding depth performance compared to the previous visual inpainting method. Our dataset will be available at: “ https://github.com/JunhyeopLee/lidar%5finpainting ”. [ABSTRACT FROM AUTHOR]

Detecting moving objects has been widely studied since it plays vital many applications, such as video surveillance and intelligent transportation systems. It is necessary to accurately differentiate the foreground and the background in this technology to analyze object motions in the scene. Conventional detection methods use many reference frames to model the background to detect moving objects; however, the detection is inaccurate when immediate changes occur in the scene because the instant update of the background model is impossible. To be robust illumination changes and dynamic backgrounds, we propose an accurate and instant background modeling (AIBM) method that inpaints the background with superpixels and enhances it in detail with pixel-levels. Unlike the previous approaches, the proposed AIBM method utilizes spatio-temporal information of only two consecutive frames to eliminate the lengthy initialization and update period of the model. In this paper, we illustrate the importance of accurate and instant background modeling in detecting moving objects. The performance of our method is evaluated with three benchmark datasets (CDnet2014, LASIESTA, and SBI). The experimental results show that our AIBM method is robust to sudden changes in the scene and outperforms the other conventional methods with F-measure of 0.8911 and 0.9682 in detection accuracy in CDNet2014 and LASIESTA datasets, respectively. The accuracy of the generated background is also measured on the SBI dataset, which demonstrates the importance of high-quality background modeling. [ABSTRACT FROM AUTHOR]

To replace compromised biometric templates, cancelable biometrics has recently been introduced. The concept is to transform a biometric signal or feature into a new one for enrollment and matching. For making cancelable fingerprint templates, previous approaches used either the relative position of a minutia to a core point or the absolute position of a minutia in a given fingerprint image. Thus, a query fingerprint is required to be accurately aligned to the enrolled fingerprint in order to obtain identically transformed minutiae. In this paper, we propose a new method for making cancelable fingerprint templates that do not require alignment. For each minutia, a rotation and translation invariant value is computed from the orientation information of neighboring local regions around the minutia. The invariant value is used as the input to two changing functions that output two values for the translational and rotational movements of the original minutia, respectively, in the cancelable template. When a template is compromised, it is replaced by a new one generated by different changing functions. Our approach preserves the original geometric relationships (translation and rotation) between the enrolled and query templates after they are transformed. Therefore, the transformed templates can be used to verify a person without requiring alignment of the input fingerprint images. In our experiments, we evaluated the proposed method in terms of two criteria: performance and changeability. When evaluating the performance, we examined how verification accuracy varied as the transformed templates were used for matching. When evaluating the changeability, we measured the dissimilarities between the original and transformed templates, and between two differently transformed templates, which were obtained from the same original fingerprint. The experimental results show that the two criteria mutually affect each other and can be controlled by varying the control parameters of the changing functions. Index Terms--Alignment, cancelable biometrics, fingerprint, privacy concerns, security.

We describe a face recognition system based on two different three-dimensional (3D) sensors. We use 3D sensors to overcome the pose-variation problems that cannot be effectively solved in two-dimensional images. We acquire input data based on a structured-light system and compare it with 3D faces that are obtained from a 3D laser scanner. Owing to differences in structure between the input data and the 3D faces, we can generate the range images of the probe and stored images. For estimating the head pose of input data, we propose a novel error-compensated singular-value decomposition that geometrically estimates the rotation angle. Face recognition rates obtained with principal component analysis on various range images of 35 people in different poses show promising results.

Most standards provide a generalized syntax and semantics framework for video coders, leaving the selection and optimization of the right parameter set (and lookup tables) to the implementation. The choice of the right parameter set that is suitable for a rich enough class of input sequences is, however, quite difficult. This difficulty is particularly amplified in the low-bit-rate video coding arena, where robust parameter sets are very important. We propose that robust parameter estimation, using the Taguchi methods, when applied to low-bit-rate video coding allows effective (near optimal) performance over a wide variety of input data streams. A number of experimental results confirm the improvement (via robustness) vis-a-vis conventional parameter estimation methods, and these methods promise a solution to the design of efficient parameter sets that support standards. Index Terms - H.263, parameter optimization, rate control, robust design, video coding.

Heterogeneous Face Recognition (HFR) is a task that matches faces across two different domains such as visible light (VIS), near-infrared (NIR), or the sketch domain. Due to the lack of databases, HFR methods usually exploit the pre-trained features on a large-scale visual database that contain general facial information. However, these pre-trained features cause performance degradation due to the texture discrepancy with the visual domain. With this motivation, we propose a graph-structured module called Relational Graph Module (RGM) that extracts global relational information in addition to general facial features. Because each identity’s relational information between intra-facial parts is similar in any modality, the modeling relationship between features can help cross-domain matching. Through the RGM, relation propagation diminishes texture dependency without losing its advantages from the pre-trained features. Furthermore, the RGM captures global facial geometrics from locally correlated convolutional features to identify long-range relationships. In addition, we propose a Node Attention Unit (NAU) that performs node-wise recalibration to concentrate on the more informative nodes arising from relation-based propagation. Furthermore, we suggest a novel conditional-margin loss function ($C$ -softmax) for the efficient projection learning of the embedding vector in HFR. The proposed method outperforms other state-of-the-art methods on five HFR databases. Furthermore, we demonstrate performance improvement on three backbones because our module can be plugged into any pre-trained face recognition backbone to overcome the limitations of a small HFR database. [ABSTRACT FROM AUTHOR]

The decision tree is one of the most effective tools for deriving meaningful outcomes from image data acquired from the visual sensors. Owing to its reliability, superior generalization abilities, and easy implementation, the tree model has been widely used in various applications. However, in image classification problems, conventional tree methods use only a few sparse attributes as the splitting criterion. Consequently, they suffer from several drawbacks in terms of performance and environmental sensitivity. To overcome these limitations, this paper introduces a new tree induction algorithm that classifies images on the basis of local area learning. To train our predictive model, we extract a random local area within the image and use it as a feature for classification. In addition, the self-organizing map, which is a clustering technique, is used for node learning.We also adopt a random sampled optimization technique to search for the optimal node. Finally, each trained node stores the weights that represent the training data and class probabilities. Thus, a recursively trained tree classifies the data hierarchically based on the local similarity at each node. The proposed tree is a type of predictive model that offers benefits in terms of image's semantic energy conservation compared with conventional tree methods. Consequently, it exhibits improved performance under various conditions, such as noise and illumination changes. Moreover, the proposed algorithm can improve the generalization ability owing to its randomness. In addition, it can be easily applied to ensemble techniques. To evaluate the performance of the proposed algorithm, we perform quantitative and qualitative comparisons with various tree-based methods using four image datasets. The results show that our algorithm not only involves a lower classification error than the conventional methods but also exhibits stable performance even under unfavorable conditions such as noise and illumination changes. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) is developed for next-generation video coding, which achieves significant improvements in coding efficiency compared with H.264/Advanced Video Coding by adopting various tools including a quadtree-based block partitioning structure. However, this causes high encoding complexity for exhaustive rate-distortion (RD) cost computation of the extended prediction unit (PU) searching. In this paper, a fast PU skip and split termination algorithm is proposed. The proposed method consists of three algorithms: 1) early skip; 2) PU skip; and 3) PU split termination. The early skip algorithm allows immediate skipping of the RD cost computation for large PUs according to the neighboring PUs. Based on Bayes’s rule, the PU skip algorithm allows skipping of the full RD cost computation, and the split termination algorithm terminates further PU splitting using the RD cost of rough mode decision (RMD). The decision parameter for the PU skip and the split termination is presented as the ratio of the RMD RD costs between the current PU and the spatially adjacent or upper depth PU. The simulation results show that the proposed algorithm achieves a saving of 53.52% encoding time while maintaining almost the same RD performances as the HEVC reference software. [ABSTRACT FROM PUBLISHER]

High Efficiency Video Coding (HEVC) employs a coding unit (CU), prediction unit (PU), and transform unit (TU) based on the quadtree coding tree unit (CTU) structure to improve coding efficiency. However, the computational complexity increases greatly because the rate-distortion (RD) optimization process should be performed for all CUs, PUs, and TUs to obtain the optimal CTU partition. In this paper, a fast CU size decision algorithm is proposed to reduce the encoder complexity of HEVC. Based on the statistical analysis, three approaches with SKIP mode decision (SMD), CU skip estimation (CUSE), and early CU termination (ECUT) are considered. In SMD, it is determined that the remaining modes except for SKIP mode are preformed or not. CUSE and ECUT determine that larger CU sizes and smaller CU sizes are coded or not, respectively. Thresholds for SMD, CUSE, and ECUT are designed based on Bayes’ rule with a complexity factor. Update process is performed to estimate the statistical parameters for SMD, CUSE, and ECUT considering the characteristic of RD cost. The experimental results demonstrate that the proposed CU size decision algorithm significantly reduces computational complexity by 69% on average with 2.99% Bjøntegaard difference bitrate (BDBR) increase for random access. The complexity reduction and BDBR increase for low delay are 68% and 2.46%, respectively. The experimental results also show that our proposed scheme performs well for various characteristic of sequences and outperforms the two previous state-of-the-art works. [ABSTRACT FROM PUBLISHER]

Multi-person pose estimation has been gaining considerable interest due to its use in several real-world applications, such as activity recognition, motion capture, and augmented reality. Although the improvement of the accuracy and speed of multi-person pose estimation techniques has been recently studied, limitations still exist in balancing these two aspects. In this paper, a novel knowledge distilled lightweight top-down pose network (KDLPN) is proposed that balances computational complexity and accuracy. For the first time in multi-person pose estimation, a network that reduces computational complexity by applying a "Pelee" structure and shuffles pixels in the dense upsampling convolution layer to reduce the number of channels is presented. Furthermore, to prevent performance degradation because of the reduced computational complexity, knowledge distillation is applied to establish the pose estimation network as a teacher network. The method performance is evaluated on the MSCOCO dataset. Experimental results demonstrate that our KDLPN network significantly reduces 95% of the parameters required by state-of-the-art methods with minimal performance degradation. Moreover, our method is compared with other pose estimation methods to substantiate the importance of computational complexity reduction and its effectiveness. [ABSTRACT FROM AUTHOR]

In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods. [ABSTRACT FROM AUTHOR]

The energy cost of producing steel in an electric arc furnace (EAF) has a sizable influence on the prices of natural gas and electricity. Therefore, it is important to use these energies efficiently via a tailored oxy-fuel combustion burner and oxygen lance. In this study, an important modification of the side-wall injector system in the EAF at Hyundai Steel Incheon works was implemented to reduce electrical energy consumption and improve productivity. A protruding water-cooled copper jacket, including a newly designed burner, was developed to reduce the distance between the jet nozzle and the molten steel. In addition, the jet angles for the burner and lance were separately set for each scrap melting and refining mode. The modifications led to a reduction in electrical energy consumption of 5 kWh/t and an increase in productivity of approximately 3.1 t/h. Consequently, total energy cost savings of 0.3 USD/t and a corresponding annual cost savings of approximately 224,000 USD/year were achieved. [ABSTRACT FROM AUTHOR]

The H.264/AVC standard achieves high compression performance compared to existing video coding standards by employing the rate distortion optimization (RDO) technique. All possible combinations of modes are calculated to find the minimum rate distortion cost which results in significant computational complexity. In this paper, we propose a fast block size and mode decision method for intra prediction, which can effectively reduce computational burden. For fast block size decision, intra16x16 is selectively searched based on the homogeneity of inner-8x8 block of macroblock (MB). For fast mode decision, the search of intra16x16 and intra4x4 prediction modes is restricted by the similarity of the reference pixels. Moreover, by considering the similarities of the upper reference pixels and left reference pixels separately, considerable reduction of computational complexity is achieved. The results of simulations demonstrate maxima of 79% and 77% average time savings with negligible loss in PSNRY and bitrate. [ABSTRACT FROM PUBLISHER]

The latest video coding standard H.264/AVC achieves significant coding efficiency by adopting variable block sizes for inter mode decision. However, this technique brings about high computational complexity since the optimal mode is determined by exhaustively calculating rate-distortion cost (RDcost) of all modes. To solve this problem, fast inter mode decision algorithm is proposed, as follows. Based on high correlation between current and previous frames, the motion vector based-moving window (MVMW) scheme predicts the RDcost for each mode, and the predicted RDcosts are used as thresholds for selecting the optimal mode. The experimental results show that the proposed algorithm achieves 71.78% time saving on average with negligible PSNR loss and bit rate increment. This feature will help to develop real time video applications such as video conference and telephone. [ABSTRACT FROM PUBLISHER]

Abstract: The receiver operating characteristics (ROC) curve has been extensively used for performance evaluation in multimodal biometrics fusion. However, the processes of fusion classifier design and the final ROC performance evaluation are usually conducted separately. This has been inevitable because the ROC, when taken from the error counting point of view, does not have a well-posed structure linking to the fusion classifier of interest. In this work, we propose to optimize the ROC performance directly according to the fusion classifier design. The area under the ROC curve (AUC) will be used as the optimization objective since it provides a good representation of the ROC performance. Due to the piecewise cumulative structure of the AUC, a smooth approximate formulation is proposed. This enables a direct optimization of the AUC with respect to the classifier parameters. When a fusion classifier has linear parameters, computation of the solution to optimize a quadratic AUC approximation is surprisingly simple and yet effective. Our empirical experiments on biometrics fusion show strong evidences regarding the potential of the proposed method. [Copyright &y& Elsevier]

Abstract: Lately, the once powerful one-factor authentication which is based solely on either password, token or biometric approach, appears to be insufficient in addressing the challenges of identity frauds. For example, the sole biometric approach suffers from the privacy invasion and non-revocable issues. Passwords and tokens are easily forgotten and lost. To address these issues, the notion of cancellable biometrics was introduced to denote biometric templates that can be cancelled and replaced with the inclusion of another independent authentication factor. BioHash is a form of cancellable biometrics which mixes a set of user-specific random vectors with biometric features. In verification setting, BioHash is able to deliver extremely low error rates as compared to the sole biometric approach when a genuine token is used. However, this raises the possibility of two identity theft scenarios: (i) stolen-biometrics, in which an impostor possesses intercepted biometric data of sufficient high quality to be considered genuine and (ii) stolen-token, in which an impostor has access to the genuine token and used by the impostor to claim as the genuine user. We found that the recognition rate for the latter case is poorer. In this paper, the quantised random projection ensemble based on the Johnson–Lindenstrauss Lemma is used to establish the mathematical foundation of BioHash. Based on this model, we elucidate the characteristics of BioHash in pattern recognition as well as security view points and propose new methods to rectify the stolen-token problem. [Copyright &y& Elsevier]

Abstract: This paper proposes a weighted power series model for face verification scores fusion. Essentially, a linear parametric power series model is adopted to directly minimize an approximated total error rate for fusion of multi-modal face verification scores. Unlike the conventional least-squares error minimization approach which involves fitting of a learning model to data density and then perform a threshold process for error counting, this work directly formulates the required target error count rate in terms of design model parameters with a closed-form solution. The solution is found to belong to a specific setting of the weighted least squares. Our experiments on fusing scores from visual and infra-red face images as well as on public data sets show promising results. [Copyright &y& Elsevier]

Abstract: This paper addresses the biometric scores fusion problem from the error rate minimization point of view. Comparing to the conventional approach which treats fusion classifier design and performance evaluation as a two-stage process, this work directly optimizes the target performance with respect to fusion classifier design. Based on a smooth approximation to the total error rate of identity verification, a deterministic solution is proposed to solve the fusion optimization problem. The proposed method is applied to a face and iris verification fusion problem addressing the demand for high security in the modern networked society. Our empirical evaluations show promising potential in terms of decision accuracy and computing efficiency. [Copyright &y& Elsevier]

Abstract: The primary goal of linear discriminant analysis (LDA) in face feature extraction is to find an effective subspace for identity discrimination. The introduction of kernel trick has extended the LDA to nonlinear decision hypersurface. However, there remained inherent limitations for the nonlinear LDA to deal with physical applications under complex environmental factors. These limitations include the use of a common covariance function among each class, and the limited dimensionality inherent to the definition of the between-class scatter. Since these problems are inherently caused by the definition of the Fisher''s criterion itself, they may not be solvable under the conventional LDA framework. This paper proposes to adopt a margin-based between-class scatter and a regularization process to resolve the issue. Essentially, we redesign the between-class scatter matrix based on the SVM margins to facilitate an effective and reliable feature extraction. This is followed by a regularization of the within-class scatter matrix. Extensive empirical experiments are performed to compare the proposed method with several other variants of the LDA method using the FERET, AR, and CMU-PIE databases. [Copyright &y& Elsevier]