Search

Object detection is a critical technology for the intelligent analytical processing of images captured by drones. The objects usually come in various scales and can be extremely small. Existing detection methods are inherently based on pyramid hierarchy architectures to extract multi-scale features and provide better feature representation for small objects. Nevertheless, they inevitably dilute the representation of details in low-level features during top-down feature fusion and are totally unconcerned with whether the fused feature fits the objects of specific scales within a layer. Moreover, the pyramid can only implicitly fuse the spatial context, which makes the fused features cannot receive fine spatial location information for object localization. In this work, we propose an effective boundary-aware network with attention refinement and spatial interaction to tackle the above challenges. Specifically, we first present a highly effective yet simple boundary-aware detection head (BAH), which directly guides representation learning of object structure semantics in the prediction layer to preserve object-related boundary semantics. Additionally, the attentional feature parallel fusion (AFPF) module offers multi-scale feature encoding capability in a parallel triple fusion fashion and adaptively selects features appropriate for objects of certain scales. Furthermore, we design a spatial interactive module (SIM) to preserve fine spatial detail through cross-spatial feature association. Extensive experiments prove that the proposed network significantly outperforms the state-of-the-art methods, in which we achieve 33.1 mAP and 56.5 AP50 on the VisDrone benchmark, 63.4 mAP and 94 AP50 on the NWPU VHR-10 benchmark. The source code will be released. [ABSTRACT FROM AUTHOR]

Facial action unit (AU) recognition and optical flow estimation are two highly correlated tasks, since optical flow can provide motion information of facial muscles to facilitate AU recognition. However, most existing AU recognition methods handle the two tasks independently by offline extracting optical flow as auxiliary information or directly ignoring the use of optical flow. In this paper, we propose a novel end-to-end joint framework of AU recognition and optical flow estimation, in which the two tasks contribute to each other. Moreover, due to the lack of optical flow annotations in AU datasets, we propose to estimate optical flow in a self-supervised manner. To regularize the self-supervised estimation of optical flow, we propose an identical mapping constraint for the optical flow guided image warping process, in which the estimated optical flow between two same images is required to not change the image during warping. Experiments demonstrate that our framework (i) outperforms most of the state-of-the-art AU recognition methods on the challenging BP4D and GFT benchmarks, and (ii) also achieves competitive self-supervised optical flow estimation performance. • An end-to-end joint framework of AU recognition and self-supervised optical flow estimation. • An identity mapping constraint to ensure the reliability of self-supervised optical flow estimation. • An AU occurrence probability map regression loss for exploiting AU location information and facial spatial information. • Our approach outperforms most of the state-of-the-art AU recognition methods. • Our approach achieves comparable self-supervised optical flow estimation performance. [ABSTRACT FROM AUTHOR]

Correlations in facial action units (AUs) convey significant information for AU detection yet have not been thoroughly exploited. Most existing methods learn the regional correlation distribution of each AU, or reason the dependencies among AUs. However, these methods typically either predefine the correlations based on prior knowledge, which often ignores useful information, or directly learn the correlations guided by AU detection, which often includes irrelevant information. To resolve these limitations, we propose a novel hybrid relational reasoning framework for AU detection. In particular, we propose to adaptively reason pixel-level correlations of each AU, under the constraint of predefined regional correlations by facial landmarks, as well as the supervision of AU detection. Moreover, we propose to adaptively reason AU-level correlations using a graph convolutional network, by considering both predefined AU relationships and learnable relationship weights. Our framework is beneficial for integrating the advantages of correlation predefinition and correlation learning. Extensive experiments demonstrate that our approach (i) soundly outperforms the state-of-the-art AU detection methods on the challenging BP4D, DISFA, and GFT benchmarks, and (ii) can precisely reason the regional correlation distribution of each AU. [ABSTRACT FROM AUTHOR]

Facial expression transfer between two unpaired images is a challenging problem, as fine-grained expression is typically tangled with other facial attributes. Most existing methods treat expression transfer as an application of expression manipulation, and use predicted global expression, landmarks or action units (AUs) as a guidance. However, the prediction may be inaccurate, which limits the performance of transferring fine-grained expression. Instead of using an intermediate estimated guidance, we propose to explicitly transfer facial expression by directly mapping two unpaired input images to two synthesized images with swapped expressions. Specifically, considering AUs semantically describe fine-grained expression details, we propose a novel multi-class adversarial training method to disentangle input images into two types of fine-grained representations: AU-related feature and AU-free feature. Then, we can synthesize new images with preserved identities and swapped expressions by combining AU-free features with swapped AU-related features. Moreover, to obtain reliable expression transfer results of the unpaired input, we introduce a swap consistency loss to make the synthesized images and self-reconstructed images indistinguishable. Extensive experiments show that our approach outperforms the state-of-the-art expression manipulation methods for transferring fine-grained expressions while preserving other attributes including identity and pose. [ABSTRACT FROM AUTHOR]

Facial action unit (AU) detection and face alignment are two highly correlated tasks, since facial landmarks can provide precise AU locations to facilitate the extraction of meaningful local features for AU detection. However, most existing AU detection works handle the two tasks independently by treating face alignment as a preprocessing, and often use landmarks to predefine a fixed region or attention for each AU. In this paper, we propose a novel end-to-end deep learning framework for joint AU detection and face alignment, which has not been explored before. In particular, multi-scale shared feature is learned firstly, and high-level feature of face alignment is fed into AU detection. Moreover, to extract precise local features, we propose an adaptive attention learning module to refine the attention map of each AU adaptively. Finally, the assembled local features are integrated with face alignment feature and global feature for AU detection. Extensive experiments demonstrate that our framework (i) significantly outperforms the state-of-the-art AU detection methods on the challenging BP4D, DISFA, GFT and BP4D+ benchmarks, (ii) can adaptively capture the irregular region of each AU, (iii) achieves competitive performance for face alignment, and (iv) also works well under partial occlusions and non-frontal poses. The code for our method is available at https://github.com/ZhiwenShao/PyTorch-JAANet. [ABSTRACT FROM AUTHOR]

Facial landmarks are highly correlated with each other since a certain landmark can be estimated by its neighboring landmarks. Most of the existing deep learning methods only use one fully-connected layer called shape prediction layer to estimate the locations of facial landmarks. In this paper, we propose a novel deep learning framework named Multi-Center Learning with multiple shape prediction layers for face alignment. In particular, each shape prediction layer emphasizes on the detection of a certain cluster of semantically relevant landmarks respectively. Challenging landmarks are focused firstly, and each cluster of landmarks is further optimized respectively. Moreover, to reduce the model complexity, we propose a model assembling method to integrate multiple shape prediction layers into one shape prediction layer. Extensive experiments demonstrate that our method is effective for handling complex occlusions and appearance variations with real-time performance. The code for our method is available at https://github.com/ZhiwenShao/MCNet-Extension. [ABSTRACT FROM AUTHOR]

Highlights: [•] The hot deformation behavior of a newly Mg–9d–3Y–2Zn–0.5Zr alloy was studied. [•] The form of the flow stress curves was sensitive to temperature and strain rate. [•] The processing map at the strain of 0.5 representing steady-state flow was developed. [•] The microstructures evolution and dynamic recrystallized (DRX) were analyzed. [•] The optimum deformation condition was at 450°C with the strain rate of 0.01s−1. [ABSTRACT FROM AUTHOR]

Abstract: The influence of low frequency electromagnetic field and power ultrasonic field on the microstructure of AZ80Mg alloy billets was studied. The magnetic flux density and time average electromagnetic volume force density were obtained by numerical simulation. The acoustic pressure distribution was also numerically calculated in order to characterize the ultrasonic field propagation. After comparison of the different effects of grain refinement by low frequency electromagnetic casting (LFEC) and ultrasonic casting (UC), a new method (LFEC+UC) was developed by simultaneously applying ultrasonic vibration to the melt during low frequency electromagnetic semi-continuous casting of AZ80Mg alloy billets. With the application of the compound field under the optimum conditions, the as-cast macrostructure and microstructure were refined and homogeneously distributed. An obvious improvement of mechanical properties was finally obtained. The mechanisms of grain refinement and interaction of low frequency electromagnetic field and power ultrasonic field with the melt were discussed. [Copyright &y& Elsevier]

Due to the subjective nature of people's aesthetic experiences with respect to images, personalized image aesthetics assessment (PIAA), which can simulate the aesthetic experiences of individual users to estimate images, has received extensive attention from researchers in the computational intelligence and computer vision communities. Existing PIAA models are usually built on prior knowledge that directly learns the generic aesthetic results of images from most people or the personalized aesthetic results of images from a large number of individuals. However, the learned prior knowledge ignores the mutual influence of the multiple attributes of images and users in their personalized aesthetic experiences. To this end, this paper proposes a personalized image aesthetics assessment method via multi-attribute interactive reasoning. Different from existing PIAA models, the multi-attribute interaction constructed from both images and users is used as more effective prior knowledge. First, we designed a generic aesthetics extraction module from the perspective of images to obtain the aesthetic score distribution and multiple objective attributes of images rated by most users. Then, we propose a multi-attribute interactive reasoning network from the perspective of users. By interacting multiple subjective attributes of users with multiple objective attributes of images, we fused the obtained multi-attribute interactive features and aesthetic score distribution to predict personalized aesthetic scores. Experimental results on multiple PIAA datasets demonstrated our method outperformed state-of-the-art PIAA methods. [ABSTRACT FROM AUTHOR]

Sketch-to-photo face generation has recently gained remarkable attention in computer vision and signal processing communities, because the sketches that employ concise lines are easily available and can describe significant facial attributes conveniently. Most existing sketch-to-photo works fail to maintain geometric structures and improve local details simultaneously, which limits their performance. In this work, we propose a two-stage sketch-to-photo generative adversarial network for face generation. In the first stage, we propose a semantic loss to maintain semantic consistency. In the second stage, we define the similar connected component and propose a color refinement loss to generate fine-grained details. Moreover, we introduce a multi-scale discriminator and design a patch-level local discriminator. We also propose a texture loss to enhance the local fidelity of synthesized images. Experiments show that our proposed method can significantly generate better results while preserving facial attributes than the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Video Object Segmentation (VOS) methods have made many breakthroughs with the help of the continuous development and advancement of deep learning. However, the deep learning model is vulnerable to malicious adversarial attacks, which mislead the model to make wrong decisions by adding adversarial perturbation that humans cannot perceive to the input image. Threats to deep learning models remind us that video object segmentation methods are also vulnerable to attacks, thereby threatening their security. Therefore, we study adversarial attacks on the VOS task to better identify the vulnerabilities of the VOS method, which in turn provides an opportunity to improve its robustness. In this paper, we propose an attention-guided adversarial attack method, which uses spatial attention blocks to capture features with global dependencies to construct correlations between consecutive video frames, and performs multipath aggregation to effectively integrate spatial-temporal perturbation, thereby guiding the deconvolution network to generate adversarial examples with strong attack capability. Specifically, the class loss function is designed to enable the deconvolution network to better activate noise in other regions and suppress the activation related to the object class based on the enhanced feature map of the object class. At the same time, attentional feature loss is designed to enhance the transferability against attack. The experimental results on the DAVIS dataset show that the proposed attention-guided adversarial attack method can significantly reduce the segmentation accuracy of OSVOS, and the J&F mean on DAVIS 2016 can reach 73.6% drop rate. The generated adversarial examples are also highly transferable to other video object segmentation models. [ABSTRACT FROM AUTHOR]

Regression-based face alignment algorithms predict facial landmarks by iteratively updating an initial shape, and hence are always limited by the initialization. Usually, the initial shape is obtained from the average face or by randomly picking a face from the training set. In this study, we discuss how to improve initialization by studying a neighborhood representation prior, leveraging neighboring faces to obtain a high-quality initial shape. In order to further improve the estimation precision of each facial landmark, we propose a face-like landmark adjustment algorithm to refine the face shape. Extensive experiments demonstrate our algorithm achieves favorable results compared to the state-of-the-art algorithms. Moreover, our algorithm achieves a smaller normalized mean error than the human performance (5.54% vs. 5.6%) on the challenging dataset the Caltech Occluded Faces in the Wild (COFW). [ABSTRACT FROM AUTHOR]

The designed method aims to perform image classification tasks efficiently and accurately. Different from the traditional CNN-based image classification methods, which are greatly affected by the number of labels and the depth of the network. Although the deep network can improve the accuracy of the model, the training process is usually time-consuming and laborious. We explained how to use only a few of labels, design a more flexible network architecture and combine feature distillation method to improve model efficiency while ensuring high accuracy. Specifically, we integrate different network structures into independent individuals to make the use of network structures more flexible. Based on knowledge distillation, we extract the channel features and establish a feature distillation connection from the teacher network to the student network. By comparing the experimental results with other related popular methods on commonly used data sets, the effectiveness of the method is proved. The code can be found at https://github.com/ZhangXinba/Semi_FD. [ABSTRACT FROM AUTHOR]

In this work, Mg-3Y sheet was prepared by high temperature cross-rolling and subsequent short-term annealing. The effect of annealing on microstructure, texture, mechanical properties, and stretch formability of Mg-3Y sheet was primarily investigated. Micro-nano size coexistence of β-Mg24Y5 phases can be well deformed with matrix. The as-rolled Mg-3Y sheet exhibited a homogeneous deformation microstructure consisting of deformed grains with extensive kink bands and dispersed β-Mg24Y5 phases. A double peak texture character appeared in as-rolled Mg-3Y sheet with a split of the texture peaks of about ±20° tilted to rolling direction. After annealing, the as-annealed Mg-3Y sheet presented complete static recrystallized (SRXed) microstructure consisting of uniform equiaxed grains. The texture orientation distribution was more dispersed and a weakened multiple-peak texture orientation distribution appeared. In addition, the maximum intensity of basal plane decreased from 5.2 to 3.1. The change of texture character was attributed to static recrystallization (SRX) induced by kink bands and grain boundaries. The as-annealed Mg-3Y sheet with high Schmid factor (SF) for basal slip, prismatic slip, pyramidal slip, and pyramidal slip exhibited high ductility (~25.6%). Simultaneously, enhanced activity of basal slip and randomized grain orientation played a significant role in decreasing anisotropy for the as-annealed Mg-3Y sheet, which contributed to the formation of high stretch formability (~6.2 mm) at room temperature. [ABSTRACT FROM AUTHOR]

• We propose a novel geodesic guided convolution (GeoConv), which can be applied to fine-grained face analysis tasks such as AU recognition. • We propose an AU recognition network based on GeoConv. This is the first work of integrating 3D information into 2D convolution for AU recognition. • Experiments verify the effectiveness of our approach and show that our method soundly outperforms the SOTA methods on BP4D and DISFA benchmarks. Automatic facial action unit (AU) recognition has attracted great attention but still remains a challenging task, as subtle changes of local facial muscles are difficult to thoroughly capture. Most existing AU recognition approaches leverage geometry information in a straightforward 2D or 3D manner, which either ignore 3D manifold information or suffer from high computational costs. In this paper, we propose a novel geodesic guided convolution (GeoConv) for AU recognition by embedding 3D manifold information into 2D convolutions. Specifically, the kernel of GeoConv is weighted by our introduced geodesic weights, which are negatively correlated to geodesic distances on a coarsely reconstructed 3D morphable face model. Moreover, based on GeoConv, we further develop an end-to-end trainable framework named GeoCNN for AU recognition. Extensive experiments on BP4D and DISFA benchmarks show that our approach significantly outperforms the state-of-the-art AU recognition methods. [ABSTRACT FROM AUTHOR]

In this study the grain size effect on mechanical properties of a body-centered-cubic Ti–30Zr–5Mo alloy was investigated. Double yielding behavior in the stress-strain curves and four-stage behavior in the strain hardening rate curves can be seen in all Ti–30Zr–5Mo materials with different average grain sizes ranging from 6 to 475 μm, which is attributed to the occurrence of the stress-induced α′ transformation. The static Hall-Petch coefficient (k) for phase transformation was calculated to establish the relationship between grain size and trigger stress of the various materials. With the increase of strain, the hindrance of αʹ/β grain boundaries and αʹ/αʹ grain boundaries to dislocations gradually replaced β/β grain boundaries, thus the work hardening ability and k value changed. β grains were segmented by α′ martensite, resulting in a dynamic Hall‒Petch effect. Combined with a large stress field in the fine-grained materials with an average grain size of 6 μm, the highest work hardening rate with a value of 13 GPa was obtained. As the β grain size increased, the ultimate strength gradually decreased, while both trigger stress of the stress-induced αʹ transformation and elongation fluctuated. The trigger stress can be adjusted between 211 and 464 MPa by controlling the grain size. The grain size has little effect on the amount of the stress-induced αʹ phase. With a high trigger stress of 464 MPa in the material with the finest grains, the excellent ductility of 21% is obtained. The best comprehensive mechanical properties with a strength-ductility index value of 252 MPa is obtained in the material with an average grain size of 113 μm. • The grains in BCC allys can be refined via stress-induced phase transformation and its reverse transformation. • The grains are refined to 6 μm in metastable Ti-30Zr-5Mo alloy without severe plastic deformation. • The grain size has little effect on the amount of the stress-induced α′ phase. • The trigger stress can be adjusted between 211 and 464 MPa by controlling the grain size. • With a high trigger stress of 464 MPa in the material with the finest grains, the excellent ductility of 21% is obtained. [ABSTRACT FROM AUTHOR]

Adversarial attacks on visual object tracking have been gaining momentum in the quest to assess and enhance the robustness and security of object tracking models. Most adversarial attack strategies for object tracking rely on fully supervised attacks; that is, they necessitate the use of labels. However, it is not feasible to apply these labeled video object tracking techniques to all attacks. As such, this paper presents an unsupervised attack methodology against visual object tracking models. The approach employs the cycle consistency principle of object tracking models to maximize the inconsistency between forward and backward tracking, thereby providing effective countermeasures. Additionally, this paper introduces a contextual attack method, leveraging the information from the attack object's region and its surrounding contextual regions. This strategy attacks the object region and its surrounding context regions simultaneously, aiming to decrease its response score to the attack. The proposed attack method is assessed across various types of deep learning-based object trackers. Experimental results on multiple benchmarks reveal that the suggested method yields competitive attack outcomes. • We propose a cycle consistency attack method leveraging object tracking model's cycle consistency principle. • We introduce a contextual attack method targeting object and surrounding context. • Experiments across models and benchmarks show the efficacy of our tracking attack method. [ABSTRACT FROM AUTHOR]

Recently, inferring users' personality traits on social media has attracted extensive attention. Existing studies have shown that users' personality traits can be inferred from their preferences for images. However, since users' preferences on images are often affected by multiple factors, some liked images cannot effectively reflect their personality traits. To handle this issue, this paper proposes a personality modeling approach based on image aesthetic attribute-aware graph representation learning, which can leverage aesthetic attributes to refine the liked images that are consistent with users' personality traits. Specifically, we first utilize a Convolutional Neural Network (CNN) to train an aesthetic attribute prediction module. Then, attribute-aware graph representation learning is introduced to refine the images with similar aesthetic attributes from users' liked images. Finally, the aesthetic attributes of all refined images are combined to predict personality traits through a Multi-Layer Perceptron (MLP). Experimental results and visual analysis have shown that the proposed method is superior to state-of-the-art personality modeling methods. • We propose an attribute-aware graph representation learning for personality modeling. • The images with similar aesthetic attributes can be refined from users' liked images. • The combined aesthetic attributes of refined images are used to infer personality. [ABSTRACT FROM AUTHOR]

The effect of low-frequency electromagnetic field (LFEF) on temperature pattern and microstructure on both symmetry planes was investigated during the semi-continuous direct chilling (DC) casting process of magnesium slab. The distribution characteristics of magnetic flux density (B) and time average electromagnetic volume force density (F) were calculated. The weak value ofFwas found near the slab corner, which was clearly distinguished from the distribution ofB. According to the temperature measurement results, LFEF made the mushy region much flatter than that of conventional DC casting and it was much more obvious in the thickness direction. Compared with conventional DC casting, the low-frequency electromagnetic casting (LFEC) process could decrease the average temperature gradient between the centre and border in the mushy zone. The microstructures on both symmetry planes were also refined at different levels during the LFEC process. [ABSTRACT FROM PUBLISHER]