Search

Most facial analysis methods perform well in standardized testing but not in real-world testing. The main reason is that training models cannot easily learn various human features and background noise, especially for facial landmark detection and head pose estimation tasks with limited and noisy training datasets. To alleviate the gap between standardized and real-world testing, we propose a pseudo-labeling technique using a face recognition dataset consisting of various people and background noise. The use of our pseudo-labeled training dataset can help to overcome the lack of diversity among the people in the dataset. Our integrated framework is constructed using complementary multitask learning methods to extract robust features for each task. Furthermore, introducing pseudo-labeling and multitask learning improves the face recognition performance by enabling the learning of pose-invariant features. Our method achieves state-of-the-art (SOTA) or near-SOTA performance on the AFLW2000-3D and BIWI datasets for facial landmark detection and head pose estimation, with competitive face verification performance on the IJB-C test dataset for face recognition. We demonstrate this through a novel testing methodology that categorizes cases as soft, medium, and hard based on the pose values of IJB-C. The proposed method achieves stable performance even when the dataset lacks diverse face identifications. [ABSTRACT FROM AUTHOR]

Video-based person re-identification (ReID) aims to exploit relevant features from spatial and temporal knowledge. Widely used methods include the part- and attention-based approaches for suppressing irrelevant spatial–temporal features. However, it is still challenging to overcome inconsistencies across video frames due to occlusion and imperfect detection. These mismatches make temporal processing ineffective and create an imbalance of crucial spatial information. To address these problems, we propose the Spatiotemporal Multi-Granularity Aggregation (ST-MGA) method, which is specifically designed to accumulate relevant features with spatiotemporally consistent cues. The proposed framework consists of three main stages: extraction, which extracts spatiotemporally consistent partial information; augmentation, which augments the partial information with different granularity levels; and aggregation, which effectively aggregates the augmented spatiotemporal information. We first introduce the consistent part-attention (CPA) module, which extracts spatiotemporally consistent and well-aligned attentive parts. Sub-parts derived from CPA provide temporally consistent semantic information, solving misalignment problems in videos due to occlusion or inaccurate detection, and maximize the efficiency of aggregation through uniform partial information. To enhance the diversity of spatial and temporal cues, we introduce the Multi-Attention Part Augmentation (MA-PA) block, which incorporates fine parts at various granular levels, and the Long-/Short-term Temporal Augmentation (LS-TA) block, designed to capture both long- and short-term temporal relations. Using densely separated part cues, ST-MGA fully exploits and aggregates the spatiotemporal multi-granular patterns by comparing relations between parts and scales. In the experiments, the proposed ST-MGA renders state-of-the-art performance on several video-based ReID benchmarks (i.e., MARS, DukeMTMC-VideoReID, and LS-VID). [ABSTRACT FROM AUTHOR]

Most studies on convolutional Neural Network (CNN) based image processing have proposed networks that can be optimized for a single level. Here, the term "level" refers to the specific objective defined for each task, such as the degree of noise in denoising tasks. Hence, they underperform on other levels and must be retrained to deliver optimal performance. Using multiple models to cover multiple levels involves very high computational costs. To solve these problems, recent approaches train the networks on two different levels and propose their own modulation methods to enable the arbitrary intermediate levels. However, many of them 1) have difficulty adapting from one level to the other, 2) suffer from unintended artifacts in the intermediate levels, or 3) require large memory and computational cost. In this paper, we propose a novel framework using Filter Transition Network (FTN), which is a non-linear module that easily adapts to new levels, is regularized to prevent undesirable side-effects, and extremely lightweight being a data-independent module. Additionally, for stable learning of FTN, we newly propose a method to initialize nonlinear CNNs with identity mappings. Extensive results for various image processing tasks indicate that the performance of FTN is stable regarding adaptation and modulation and is comparable to that of the other heavy frameworks. [ABSTRACT FROM AUTHOR]

Lowering blood cholesterol levels is crucial for reducing the risk of cardiovascular disease in patients with familial hypercholesterolemia. To develop Perilla frutescens (L.) Britt. leaves as a functional food with a cholesterol-lowering effect, in this study, we collected P. frutescens (L.) Britt. leaves from different regions of China and Republic of Korea. On the basis of the extraction yield (all components; g/kg), we selected P. frutescens (L.) Britt. leaves from Hebei Province, China with an extract yield of 60.9 g/kg. After evaluating different concentrations of ethanol/water solvent for P. frutescens (L.) Britt. leaves, with luteolin 7-glucuronide as the indicator component, we selected a 30% ethanol/water solvent with a high luteolin 7-glucuronide content of 0.548 mg/g in Perilla. frutescens (L.) Britt. leaves. Subsequently, we evaluated the cholesterol-lowering effects of P. frutescens (L.) Britt. leaf extract and luteolin 7-glucuronide by detecting total cholesterol in HepG2 cells. The 30% ethanol extract lowered cholesterol levels significantly by downregulating 3-hydroxy-3-methyl-glutaryl-coenzyme A reductase expression. This suggests that P. frutescens (L.) Britt leaves have significant health benefits and can be explored as a potentially promising food additive for the prevention of hypercholesterolemia-related diseases. [ABSTRACT FROM AUTHOR]

Multi-object tracking (MOT), which is crucial for computer vision and video processing, has immense potential for improvement. Traditional tracking-by-detection approaches include feature-based object re-identification methods that use trained features, but these methods suffer from a lack of suitable training data. In training datasets used for MOT, every object in a video sequence must have its own location and ID. However, assigning IDs to each object in every sequence is considerably labor-intensive, and hence current MOT datasets are unsuitable for training re-identification networks. To resolve this issue, this paper proposes a novel self-supervised learning method using several short videos that contain no human-added labels, based on the idea that each video is a set of temporally corresponding image frames. We then describe how to improve tracking performance using a re-identification network trained in a self-supervised manner. In addition, ablation studies were conducted in order to define the optimal parameters, such as number of clips, data augmentation, and appropriate matching algorithms. The proposed approach achieved competitive performance compared with current best-practice methods including supervised methods, achieving MOT accuracy = 62.0% and ID F1-score = 62.7% on the MOT17 benchmark. [ABSTRACT FROM AUTHOR]

Depth completion is the task of reconstructing dense depth images from sparse LiDAR data. LiDAR depth completion, for which LiDAR data is the only input, is an ill-posed and challenging problem owing to the underlying properties of LiDAR data: extremely few points, presence of discontinuities, and absence of texture information. Accordingly, most approaches are heavily dependent on guided color images, which leads to unsatisfactory results when the color images are degraded. To alleviate the dependency on color images but leverage this information during training, we present a deep convolutional neural network (CNN) consisting of depth and edge CNNs via transferring of knowledge. In order to compensate for the limitations of LiDAR data, we design the edge CNN to learn a gradient depth image from a powerful teacher network through the Knowledge-Distillation method. Since the teacher network is trained with color images, color-embedded information can be obtained in the test phase even if color images are not used as an input. We further propose a Self-Distillation method for transferring the color-embedded features from the edge CNN to the depth CNN. Enforcing the depth features to contain edge information hardly observed in LiDAR data enables the depth CNN to generate more edge-attentive and structure-preserving results. Our novel methods show remarkable results in outdoor and indoor environments for KITTI and NYU-Depth-V2 datasets. Experiments performed with low-channel LiDAR data in KITTI and few depth points in the NYU-Depth-V2 dataset show that our method is robust to data sparsity and applicable in various scenarios. [ABSTRACT FROM AUTHOR]

Various sensors can be attached and added to autonomous vehicles, included visual cameras, radar, LiDAR (Light Detection And Ranging), and GNSS (Global Navigation Satellite System). These sensors have been studied in many research areas, in particular studies on building precise 3D maps. It is essential for autonomous driving to create an accurate 3D map of the surrounding scene. However, creating an accurate static 3D map is difficult due to changes in moving objects or dynamic environments. Spurious objects on the 3D map can be handled by removing or ignoring them for 3D mapping. Following this idea, we propose an object segmentation and inpainting network. The proposed network called SAM-Net, addresses the object duplication issue by segmenting the objects and inpainting them with the segmentation results. Conventional inpainting research has dealt with RGB images. No matter how well such approaches reconstruct holes or corrupted images, they do not establish 3D points’ relationship with the point cloud frame. Therefore, we suggest a depth inpainting method for outdoor object segmentation and inpainting tasks that utilizes a high-precision depth range sensor (Velodyne HDL-64E), which is not suggested before. Unfortunately, no dataset exists for the outdoor depth inpainting task. Thus, to train our model, we generate a new dataset by locating objects on a clean static background. Moreover, our proposed method shows outstanding depth performance compared to the previous visual inpainting method. Our dataset will be available at: “ https://github.com/JunhyeopLee/lidar%5finpainting ”. [ABSTRACT FROM AUTHOR]

Detecting moving objects has been widely studied since it plays vital many applications, such as video surveillance and intelligent transportation systems. It is necessary to accurately differentiate the foreground and the background in this technology to analyze object motions in the scene. Conventional detection methods use many reference frames to model the background to detect moving objects; however, the detection is inaccurate when immediate changes occur in the scene because the instant update of the background model is impossible. To be robust illumination changes and dynamic backgrounds, we propose an accurate and instant background modeling (AIBM) method that inpaints the background with superpixels and enhances it in detail with pixel-levels. Unlike the previous approaches, the proposed AIBM method utilizes spatio-temporal information of only two consecutive frames to eliminate the lengthy initialization and update period of the model. In this paper, we illustrate the importance of accurate and instant background modeling in detecting moving objects. The performance of our method is evaluated with three benchmark datasets (CDnet2014, LASIESTA, and SBI). The experimental results show that our AIBM method is robust to sudden changes in the scene and outperforms the other conventional methods with F-measure of 0.8911 and 0.9682 in detection accuracy in CDNet2014 and LASIESTA datasets, respectively. The accuracy of the generated background is also measured on the SBI dataset, which demonstrates the importance of high-quality background modeling. [ABSTRACT FROM AUTHOR]

Heterogeneous Face Recognition (HFR) is a task that matches faces across two different domains such as visible light (VIS), near-infrared (NIR), or the sketch domain. Due to the lack of databases, HFR methods usually exploit the pre-trained features on a large-scale visual database that contain general facial information. However, these pre-trained features cause performance degradation due to the texture discrepancy with the visual domain. With this motivation, we propose a graph-structured module called Relational Graph Module (RGM) that extracts global relational information in addition to general facial features. Because each identity’s relational information between intra-facial parts is similar in any modality, the modeling relationship between features can help cross-domain matching. Through the RGM, relation propagation diminishes texture dependency without losing its advantages from the pre-trained features. Furthermore, the RGM captures global facial geometrics from locally correlated convolutional features to identify long-range relationships. In addition, we propose a Node Attention Unit (NAU) that performs node-wise recalibration to concentrate on the more informative nodes arising from relation-based propagation. Furthermore, we suggest a novel conditional-margin loss function ($C$ -softmax) for the efficient projection learning of the embedding vector in HFR. The proposed method outperforms other state-of-the-art methods on five HFR databases. Furthermore, we demonstrate performance improvement on three backbones because our module can be plugged into any pre-trained face recognition backbone to overcome the limitations of a small HFR database. [ABSTRACT FROM AUTHOR]

In video coding, the sum of squared differences (SSD) is traditionally used for rate-distortion optimization (RDO). However, SSD has been known that has low correlation on subjective quality. In particular, film grain noise (FGN)-synthesized video sequence is a very good example of subjective quality degradation with SSD-based RDO. Therefore, structural similarity (SSIM) has been considered for RDO owing to its simplicity and high correlation with subjective quality. The SSIM metric was not designed to be used for previous RDO framework; additional processing, such as content analysis or adaptive Lagrangian multipliers, was required in previous studies. Based on analyzing cases of degradation in SSIM-based coding, this study proposes a novel SSIM-like distortion measure. In this paper, two objectives are considered. First one is FGN-synthesized video coding using the SSIM-like distortion measure to preserve noise pattern. Seconds, the proposed metric is designed for direct application in previously developed RDO frameworks without scene-analysis-based RDO. The experimental results demonstrate that the proposed method reduces erroneous prediction blocks and the Bjøntegaard delta rate by 67.46% on average compared to original SSIM-based RDO for FGN-synthesized video sequences. The results show the proposed metric is effective for film grain noise in similar bit rate, compared to a high-efficiency video coding test model (HM16.6) and the original SSIM metric. [ABSTRACT FROM AUTHOR]

The decision tree is one of the most effective tools for deriving meaningful outcomes from image data acquired from the visual sensors. Owing to its reliability, superior generalization abilities, and easy implementation, the tree model has been widely used in various applications. However, in image classification problems, conventional tree methods use only a few sparse attributes as the splitting criterion. Consequently, they suffer from several drawbacks in terms of performance and environmental sensitivity. To overcome these limitations, this paper introduces a new tree induction algorithm that classifies images on the basis of local area learning. To train our predictive model, we extract a random local area within the image and use it as a feature for classification. In addition, the self-organizing map, which is a clustering technique, is used for node learning.We also adopt a random sampled optimization technique to search for the optimal node. Finally, each trained node stores the weights that represent the training data and class probabilities. Thus, a recursively trained tree classifies the data hierarchically based on the local similarity at each node. The proposed tree is a type of predictive model that offers benefits in terms of image's semantic energy conservation compared with conventional tree methods. Consequently, it exhibits improved performance under various conditions, such as noise and illumination changes. Moreover, the proposed algorithm can improve the generalization ability owing to its randomness. In addition, it can be easily applied to ensemble techniques. To evaluate the performance of the proposed algorithm, we perform quantitative and qualitative comparisons with various tree-based methods using four image datasets. The results show that our algorithm not only involves a lower classification error than the conventional methods but also exhibits stable performance even under unfavorable conditions such as noise and illumination changes. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) is developed for next-generation video coding, which achieves significant improvements in coding efficiency compared with H.264/Advanced Video Coding by adopting various tools including a quadtree-based block partitioning structure. However, this causes high encoding complexity for exhaustive rate-distortion (RD) cost computation of the extended prediction unit (PU) searching. In this paper, a fast PU skip and split termination algorithm is proposed. The proposed method consists of three algorithms: 1) early skip; 2) PU skip; and 3) PU split termination. The early skip algorithm allows immediate skipping of the RD cost computation for large PUs according to the neighboring PUs. Based on Bayes’s rule, the PU skip algorithm allows skipping of the full RD cost computation, and the split termination algorithm terminates further PU splitting using the RD cost of rough mode decision (RMD). The decision parameter for the PU skip and the split termination is presented as the ratio of the RMD RD costs between the current PU and the spatially adjacent or upper depth PU. The simulation results show that the proposed algorithm achieves a saving of 53.52% encoding time while maintaining almost the same RD performances as the HEVC reference software. [ABSTRACT FROM PUBLISHER]

High Efficiency Video Coding (HEVC) employs a coding unit (CU), prediction unit (PU), and transform unit (TU) based on the quadtree coding tree unit (CTU) structure to improve coding efficiency. However, the computational complexity increases greatly because the rate-distortion (RD) optimization process should be performed for all CUs, PUs, and TUs to obtain the optimal CTU partition. In this paper, a fast CU size decision algorithm is proposed to reduce the encoder complexity of HEVC. Based on the statistical analysis, three approaches with SKIP mode decision (SMD), CU skip estimation (CUSE), and early CU termination (ECUT) are considered. In SMD, it is determined that the remaining modes except for SKIP mode are preformed or not. CUSE and ECUT determine that larger CU sizes and smaller CU sizes are coded or not, respectively. Thresholds for SMD, CUSE, and ECUT are designed based on Bayes’ rule with a complexity factor. Update process is performed to estimate the statistical parameters for SMD, CUSE, and ECUT considering the characteristic of RD cost. The experimental results demonstrate that the proposed CU size decision algorithm significantly reduces computational complexity by 69% on average with 2.99% Bjøntegaard difference bitrate (BDBR) increase for random access. The complexity reduction and BDBR increase for low delay are 68% and 2.46%, respectively. The experimental results also show that our proposed scheme performs well for various characteristic of sequences and outperforms the two previous state-of-the-art works. [ABSTRACT FROM PUBLISHER]

Multi-person pose estimation has been gaining considerable interest due to its use in several real-world applications, such as activity recognition, motion capture, and augmented reality. Although the improvement of the accuracy and speed of multi-person pose estimation techniques has been recently studied, limitations still exist in balancing these two aspects. In this paper, a novel knowledge distilled lightweight top-down pose network (KDLPN) is proposed that balances computational complexity and accuracy. For the first time in multi-person pose estimation, a network that reduces computational complexity by applying a "Pelee" structure and shuffles pixels in the dense upsampling convolution layer to reduce the number of channels is presented. Furthermore, to prevent performance degradation because of the reduced computational complexity, knowledge distillation is applied to establish the pose estimation network as a teacher network. The method performance is evaluated on the MSCOCO dataset. Experimental results demonstrate that our KDLPN network significantly reduces 95% of the parameters required by state-of-the-art methods with minimal performance degradation. Moreover, our method is compared with other pose estimation methods to substantiate the importance of computational complexity reduction and its effectiveness. [ABSTRACT FROM AUTHOR]

In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods. [ABSTRACT FROM AUTHOR]

The energy cost of producing steel in an electric arc furnace (EAF) has a sizable influence on the prices of natural gas and electricity. Therefore, it is important to use these energies efficiently via a tailored oxy-fuel combustion burner and oxygen lance. In this study, an important modification of the side-wall injector system in the EAF at Hyundai Steel Incheon works was implemented to reduce electrical energy consumption and improve productivity. A protruding water-cooled copper jacket, including a newly designed burner, was developed to reduce the distance between the jet nozzle and the molten steel. In addition, the jet angles for the burner and lance were separately set for each scrap melting and refining mode. The modifications led to a reduction in electrical energy consumption of 5 kWh/t and an increase in productivity of approximately 3.1 t/h. Consequently, total energy cost savings of 0.3 USD/t and a corresponding annual cost savings of approximately 224,000 USD/year were achieved. [ABSTRACT FROM AUTHOR]

The H.264/AVC standard achieves high compression performance compared to existing video coding standards by employing the rate distortion optimization (RDO) technique. All possible combinations of modes are calculated to find the minimum rate distortion cost which results in significant computational complexity. In this paper, we propose a fast block size and mode decision method for intra prediction, which can effectively reduce computational burden. For fast block size decision, intra16x16 is selectively searched based on the homogeneity of inner-8x8 block of macroblock (MB). For fast mode decision, the search of intra16x16 and intra4x4 prediction modes is restricted by the similarity of the reference pixels. Moreover, by considering the similarities of the upper reference pixels and left reference pixels separately, considerable reduction of computational complexity is achieved. The results of simulations demonstrate maxima of 79% and 77% average time savings with negligible loss in PSNRY and bitrate. [ABSTRACT FROM PUBLISHER]

The latest video coding standard H.264/AVC achieves significant coding efficiency by adopting variable block sizes for inter mode decision. However, this technique brings about high computational complexity since the optimal mode is determined by exhaustively calculating rate-distortion cost (RDcost) of all modes. To solve this problem, fast inter mode decision algorithm is proposed, as follows. Based on high correlation between current and previous frames, the motion vector based-moving window (MVMW) scheme predicts the RDcost for each mode, and the predicted RDcosts are used as thresholds for selecting the optimal mode. The experimental results show that the proposed algorithm achieves 71.78% time saving on average with negligible PSNR loss and bit rate increment. This feature will help to develop real time video applications such as video conference and telephone. [ABSTRACT FROM PUBLISHER]