Search

Emotion recognition aims to discern the emotional state of subjects within an image, relying on subject-centric and contextual visual cues. Current approaches typically follow a two-stage pipeline: first localize subjects by off-the-shelf detectors, then perform emotion classification through the late fusion of subject and context features. However, the complicated paradigm suffers from disjoint training stages and limited interaction between fine-grained subject-context elements. To address the challenge, we present a single-stage emotion recognition approach, employing a Decoupled Subject-Context Transformer (DSCT), for simultaneous subject localization and emotion classification. Rather than compartmentalizing training stages, we jointly leverage box and emotion signals as supervision to enrich subject-centric feature learning. Furthermore, we introduce DSCT to facilitate interactions between fine-grained subject-context cues in a decouple-then-fuse manner. The decoupled query token--subject queries and context queries--gradually intertwine across layers within DSCT, during which spatial and semantic relations are exploited and aggregated. We evaluate our single-stage framework on two widely used context-aware emotion recognition datasets, CAER-S and EMOTIC. Our approach surpasses two-stage alternatives with fewer parameter numbers, achieving a 3.39% accuracy improvement and a 6.46% average precision gain on CAER-S and EMOTIC datasets, respectively.

In this paper, we define the upper (resp. lower) covariance under multiple probabilities via a corresponding max-min-max (resp. min-max-min) optimization problem and the related properties of covariances are obtained. In particular, we propose a fast algorithm of calculation for upper and lower covariances under the finite number of probabilities. As an application, our algorithm can be used to solve a class of quadratic programming problem exactly, and we obtain a probabilistic representation of such quadratic programming problem., Comment: 20 pages

Most existing inference methods for the uncertainty quantification of nonlinear inverse problems need repetitive runs of the forward model which is computationally expensive for high-dimensional problems, where the forward model is expensive and the inference need more iterations. These methods are generally based on the Bayes' rule and implicitly assume that the probability distribution is unique, which is not the case for scenarios with Knightian uncertainty. In the current study, we assume that the probability distribution is uncertain, and establish a new inference method based on the nonlinear expectation theory for 'direct' uncertainty quantification of nonlinear inverse problems. The uncertainty of random parameters is quantified using the sublinear expectation defined as the limits of an ensemble of linear expectations estimated on samples. Given noisy observed data, the posterior sublinear expectation is computed using posterior linear expectations with highest likelihoods. In contrary to iterative inference methods, the new nonlinear expectation inference method only needs forward model runs on the prior samples, while subsequent evaluations of linear and sublinear expectations requires no forward model runs, thus quantifying uncertainty directly which is more efficient than iterative inference methods. The new method is analysed and validated using 2D and 3D test cases of transient Darcy flows.

The notion of upper variance under multiple probabilities is defined by a corresponding minimax optimization problem. This paper proposes a simple algorithm to solve the related minimax optimization problem exactly. As an application, we provide the probabilistic representation for a class of quadratic programming problems., Comment: 8 pages

High-precision point cloud anomaly detection is the gold standard for identifying the defects of advancing machining and precision manufacturing. Despite some methodological advances in this area, the scarcity of datasets and the lack of a systematic benchmark hinder its development. We introduce Real3D-AD, a challenging high-precision point cloud anomaly detection dataset, addressing the limitations in the field. With 1,254 high-resolution 3D items from forty thousand to millions of points for each item, Real3D-AD is the largest dataset for high-precision 3D industrial anomaly detection to date. Real3D-AD surpasses existing 3D anomaly detection datasets available regarding point cloud resolution (0.0010mm-0.0015mm), 360 degree coverage and perfect prototype. Additionally, we present a comprehensive benchmark for Real3D-AD, revealing the absence of baseline methods for high-precision point cloud anomaly detection. To address this, we propose Reg3D-AD, a registration-based 3D anomaly detection method incorporating a novel feature memory bank that preserves local and global representations. Extensive experiments on the Real3D-AD dataset highlight the effectiveness of Reg3D-AD. For reproducibility and accessibility, we provide the Real3D-AD dataset, benchmark source code, and Reg3D-AD on our website:https://github.com/M-3LAB/Real3D-AD.

In this paper, we introduce a new approach for high-quality multi-exposure image fusion (MEF). We show that the fusion weights of an exposure can be encoded into a 1D lookup table (LUT), which takes pixel intensity value as input and produces fusion weight as output. We learn one 1D LUT for each exposure, then all the pixels from different exposures can query 1D LUT of that exposure independently for high-quality and efficient fusion. Specifically, to learn these 1D LUTs, we involve attention mechanism in various dimensions including frame, channel and spatial ones into the MEF task so as to bring us significant quality improvement over the state-of-the-art (SOTA). In addition, we collect a new MEF dataset consisting of 960 samples, 155 of which are manually tuned by professionals as ground-truth for evaluation. Our network is trained by this dataset in an unsupervised manner. Extensive experiments are conducted to demonstrate the effectiveness of all the newly proposed components, and results show that our approach outperforms the SOTA in our and another representative dataset SICE, both qualitatively and quantitatively. Moreover, our 1D LUT approach takes less than 4ms to run a 4K image on a PC GPU. Given its high quality, efficiency and robustness, our method has been shipped into millions of Android mobiles across multiple brands world-wide. Code is available at: https://github.com/Hedlen/MEFLUT.

Quantum circuit cutting has emerged as a promising method for simulating large quantum circuits using a collection of small quantum machines. Running low-qubit "circuit fragments" not only overcomes the size limitation of near-term hardware, but it also increases the fidelity of the simulation. However, reconstructing measurement statistics requires computational resources - both classical and quantum - that grow exponentially with the number of cuts. In this manuscript, we introduce the concept of a golden cutting point, which identifies unnecessary basis components during reconstruction and avoids related down-stream computation. We propose a hypothesis-testing scheme for identifying golden cutting points, and provide robustness results in the case of the test failing with low probability. Lastly, we demonstrate the applicability of our method on Qiskit's Aer simulator and observe a reduced wall time from identifying and avoiding obsolete measurements.

In this paper, we introduce a fundamental model for independent and identically distributed sequence with model uncertainty on the canonical space $(\mathbb{R}^\mathbb{N},\mathcal{B}(\mathbb{R}^\mathbb{N}))$ via probability kernels. Thanks to the well-defined upper and lower variances, we obtain a new functional central limit theorem with mean-uncertainty on the canonical space by the method based on the martingale central limit theorem and stability of stochastic integral in the classical probability theory. Then we extend it to the general sublinear expectation space through a new representation theorem. Our results generalize Peng's central limit theorem with zero-mean to the case of mean-uncertainty and provides a purely probabilistic proof instead of the existing nonlinear partial differential equation approach. As an application, we consider the two-armed bandit problem and generalize the corresponding central limit theorem from the case of mean-certainty to mean-uncertainty., Comment: 31 pages. arXiv admin note: substantial text overlap with arXiv:2203.00170

Image Quality Assessment (IQA) is a challenging task that requires training on massive datasets to achieve accurate predictions. However, due to the lack of IQA data, deep learning-based IQA methods typically rely on pre-trained networks trained on massive datasets as feature extractors to enhance their generalization ability, such as the ResNet network trained on ImageNet. In this paper, we utilize the encoder of Segment Anything, a recently proposed segmentation model trained on a massive dataset, for high-level semantic feature extraction. Most IQA methods are limited to extracting spatial-domain features, while frequency-domain features have been shown to better represent noise and blur. Therefore, we leverage both spatial-domain and frequency-domain features by applying Fourier and standard convolutions on the extracted features, respectively. Extensive experiments are conducted to demonstrate the effectiveness of all the proposed components, and results show that our approach outperforms the state-of-the-art (SOTA) in four representative datasets, both qualitatively and quantitatively. Our experiments confirm the powerful feature extraction capabilities of Segment Anything and highlight the value of combining spatial-domain and frequency-domain features in IQA tasks. Code: https://github.com/Hedlen/SAM-IQA

Efficient deep learning-based approaches have achieved remarkable performance in single image super-resolution. However, recent studies on efficient super-resolution have mainly focused on reducing the number of parameters and floating-point operations through various network designs. Although these methods can decrease the number of parameters and floating-point operations, they may not necessarily reduce actual running time. To address this issue, we propose a novel multi-stage lightweight network boosting method, which can enable lightweight networks to achieve outstanding performance. Specifically, we leverage enhanced high-resolution output as additional supervision to improve the learning ability of lightweight student networks. Upon convergence of the student network, we further simplify our network structure to a more lightweight level using reparameterization techniques and iterative network pruning. Meanwhile, we adopt an effective lightweight network training strategy that combines multi-anchor distillation and progressive learning, enabling the lightweight network to achieve outstanding performance. Ultimately, our proposed method achieves the fastest inference time among all participants in the NTIRE 2023 efficient super-resolution challenge while maintaining competitive super-resolution performance. Additionally, extensive experiments are conducted to demonstrate the effectiveness of the proposed components. The results show that our approach achieves comparable performance in representative dataset DIV2K, both qualitatively and quantitatively, with faster inference and fewer number of network parameters.

Rail detection, essential for railroad anomaly detection, aims to identify the railroad region in video frames. Although various studies on rail detection exist, neither an open benchmark nor a high-speed network is available in the community, making algorithm comparison and development difficult. Inspired by the growth of lane detection, we propose a rail database and a row-based rail detection method. In detail, we make several contributions: (i) We present a real-world railway dataset, Rail-DB, with 7432 pairs of images and annotations. The images are collected from different situations in lighting, road structures, and views. The rails are labeled with polylines, and the images are categorized into nine scenes. The Rail-DB is expected to facilitate the improvement of rail detection algorithms. (ii) We present an efficient row-based rail detection method, Rail-Net, containing a lightweight convolutional backbone and an anchor classifier. Specifically, we formulate the process of rail detection as a row-based selecting problem. This strategy reduces the computational cost compared to alternative segmentation methods. (iii) We evaluate the Rail-Net on Rail-DB with extensive experiments, including cross-scene settings and network backbones ranging from ResNet to Vision Transformers. Our method achieves promising performance in terms of both speed and accuracy. Notably, a lightweight version could achieve 92.77% accuracy and 312 frames per second. The Rail-Net outperforms the traditional method by 50.65% and the segmentation one by 5.86%. The database and code are available at: https://github.com/Sampson-Lee/Rail-Detection., Comment: Accepted by ACMMM 2022

Quantum circuit cutting has been proposed to help execute large quantum circuits using only small and noisy machines. Intuitively, cutting a qubit wire can be thought of as classically passing information of a quantum state along each element in a basis set. As the number of cuts increase, the number of quantum degrees of freedom needed to be passed through scales exponentially. We propose a simple reduction scheme to lower the classical and quantum resources required to perform a cut. Particularly, we recognize that for some cuts, certain basis element might pass "no information" through the qubit wire and can effectively be neglected. We empirically demonstrate our method on circuit simulators as well as IBM quantum hardware, and we observed up to 33 percent reduction in wall time without loss of accuracy., Comment: 7 pages, 5 figures, submitted to 37th IEEE International Parallel & Distributed Processing Symposium

Maximum likelihood estimation is a common method of estimating the parameters of the probability distribution from a given sample. This paper aims to introduce the maximum likelihood estimation in the framework of sublinear expectation. We find the maximum likelihood estimator for the parameters of the maximal distribution via the solution of the associated minimax problem, which coincides with the optimal unbiased estimation given by Jin and Peng \cite{JP21}. A general estimation method for samples with dependent structure is also provided. This result provides a theoretical foundation for the estimator of upper and lower variances, which is widely used in the G-VaR prediction model in finance., Comment: 11 pages

Studies have proven that domain bias and label bias exist in different Facial Expression Recognition (FER) datasets, making it hard to improve the performance of a specific dataset by adding other datasets. For the FER bias issue, recent researches mainly focus on the cross-domain issue with advanced domain adaption algorithms. This paper addresses another problem: how to boost FER performance by leveraging cross-domain datasets. Unlike the coarse and biased expression label, the facial Action Unit (AU) is fine-grained and objective suggested by psychological studies. Motivated by this, we resort to the AU information of different FER datasets for performance boosting and make contributions as follows. First, we experimentally show that the naive joint training of multiple FER datasets is harmful to the FER performance of individual datasets. We further introduce expression-specific mean images and AU cosine distances to measure FER dataset bias. This novel measurement shows consistent conclusions with experimental degradation of joint training. Second, we propose a simple yet conceptually-new framework, AU-aware Vision Transformer (AU-ViT). It improves the performance of individual datasets by jointly training auxiliary datasets with AU or pseudo-AU labels. We also find that the AU-ViT is robust to real-world occlusions. Moreover, for the first time, we prove that a carefully-initialized ViT achieves comparable performance to advanced deep convolutional networks. Our AU-ViT achieves state-of-the-art performance on three popular datasets, namely 91.10% on RAF-DB, 65.59% on AffectNet, and 90.15% on FERPlus. The code and models will be released soon.

The paper describes our proposed methodology for the six basic expression classification track of Affective Behavior Analysis in-the-wild (ABAW) Competition 2022. In Learing from Synthetic Data(LSD) task, facial expression recognition (FER) methods aim to learn the representation of expression from the artificially generated data and generalise to real data. Because of the ambiguous of the synthetic data and the objectivity of the facial Action Unit (AU), we resort to the AU information for performance boosting, and make contributions as follows. First, to adapt the model to synthetic scenarios, we use the knowledge from pre-trained large-scale face recognition data. Second, we propose a conceptually-new framework, termed as AU-Supervised Convolutional Vision Transformers (AU-CVT), which clearly improves the performance of FER by jointly training auxiliary datasets with AU or pseudo AU labels. Our AU-CVT achieved F1 score as $0.6863$, accuracy as $0.7433$ on the validation set. The source code of our work is publicly available online: https://github.com/msy1412/ABAW4

This paper reviews the challenge on constrained high dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2022. This manuscript focuses on the competition set-up, datasets, the proposed methods and their results. The challenge aims at estimating an HDR image from multiple respective low dynamic range (LDR) observations, which might suffer from under- or over-exposed regions and different sources of noise. The challenge is composed of two tracks with an emphasis on fidelity and complexity constraints: In Track 1, participants are asked to optimize objective fidelity scores while imposing a low-complexity constraint (i.e. solutions can not exceed a given number of operations). In Track 2, participants are asked to minimize the complexity of their solutions while imposing a constraint on fidelity scores (i.e. solutions are required to obtain a higher fidelity score than the prescribed baseline). Both tracks use the same data and metrics: Fidelity is measured by means of PSNR with respect to a ground-truth HDR image (computed both directly and with a canonical tonemapping operation), while complexity metrics include the number of Multiply-Accumulate (MAC) operations and runtime (in seconds)., Comment: CVPR Workshops 2022. 15 pages, 21 figures, 2 tables

We introduce a new basic model for independent and identical distributed sequence on the canonical space $(\mathbb{R}^\mathbb{N},\mathcal{B}(\mathbb{R}^\mathbb{N}))$ via probability kernels with model uncertainty. Thanks to the well-defined upper and lower variances, we obtain a new functional central limit theorem with mean-uncertainty by the means of martingale central limit theorem and stability of stochastic integral in the classical probability theory. Then we extend it from the canonical space to the general sublinear expectation space. The corresponding proofs are purely probabilistic and do not rely on the nonlinear partial differential equation., Comment: 26 pages

This paper focuses on the maximal distribution on sublinear expectation space and introduces a new type of random fields with the maximally distributed finite-dimensional distribution. The corresponding spatial maximally distributed white noise is constructed, which includes the temporal-spatial situation as a special case due to the symmetrical independence property of maximal distribution. In addition, the stochastic integrals with respect to the spatial or temporal-spatial maximally distributed white noises are established in a quite direct way without the usual assumption of adaptability for integrand.

In this paper, we first find a type of viscosity solution of $G$-heat equation under degenerate case, and then obtain the related $G$-capacity $c(\{B_{T}\in A\})$ for any Borel set $A$. Furthermore, we prove that $I_{A}(B_{T})$ has no quasi-continuous version when it is not a constant function., Comment: 10 pages

This short note provides a new and simple proof of the convergence rate for Peng's law of large numbers under sublinear expectations, which improves the corresponding results in Song [15] and Fang et al. [3].

In this paper, we firstly establish the weak laws of large numbers on the canonical space $(\br^\bn,\cb(\br^\bn))$ by traditional truncation method and Chebyshev's inequality as in the classical probability theory. Then we extend them from the canonical space to the general sublinear expectation space. The necessary and sufficient conditions for Peng's law of large numbers are obtained.

This paper introduces the notion of pseudo-independence on the sublinear expectation space $(\Omega,\mathcal{F},\mathcal{P})$ via the classical conditional expectation, and the relations between pseudo-independence and Peng's independence are detailed discussed. Law of large numbers and central limit theorem for pseudo-independent random variables are obtained, and a purely probabilistic proof of Peng's law of large numbers is also given. In the end, some relevant counterexamples are indicated that Peng's law of large numbers and central limit theorem are invalid only with the first and second moment conditions respectively.

In this paper, we present an attention-guided deformable convolutional network for hand-held multi-frame high dynamic range (HDR) imaging, namely ADNet. This problem comprises two intractable challenges of how to handle saturation and noise properly and how to tackle misalignments caused by object motion or camera jittering. To address the former, we adopt a spatial attention module to adaptively select the most appropriate regions of various exposure low dynamic range (LDR) images for fusion. For the latter one, we propose to align the gamma-corrected images in the feature-level with a Pyramid, Cascading and Deformable (PCD) alignment module. The proposed ADNet shows state-of-the-art performance compared with previous methods, achieving a PSNR-$l$ of 39.4471 and a PSNR-$\mu$ of 37.6359 in NTIRE 2021 Multi-Frame HDR Challenge., Comment: Accepted by CVPRW 2021

Recent works in domain adaptation always learn domain invariant features to mitigate the gap between the source and target domains by adversarial methods. The category information are not sufficiently used which causes the learned domain invariant features are not enough discriminative. We propose a new domain adaptation method based on prototype construction which likes capturing data cluster centers. Specifically, it consists of two parts: disentanglement and reconstruction. First, the domain specific features and domain invariant features are disentangled from the original features. At the same time, the domain prototypes and class prototypes of both domains are estimated. Then, a reconstructor is trained by reconstructing the original features from the disentangled domain invariant features and domain specific features. By this reconstructor, we can construct prototypes for the original features using class prototypes and domain prototypes correspondingly. In the end, the feature extraction network is forced to extract features close to these prototypes. Our contribution lies in the technical use of the reconstructor to obtain the original feature prototypes which helps to learn compact and discriminant features. As far as we know, this idea is proposed for the first time. Experiment results on several public datasets confirm the state-of-the-art performance of our method.

It is a recognized fact that the classification accuracy of unseen classes in the setting of Generalized Zero-Shot Learning (GZSL) is much lower than that of traditional Zero-Shot Leaning (ZSL). One of the reasons is that an instance is always misclassified to the wrong domain. Here we refer to the seen and unseen classes as two domains respectively. We propose a new approach to distinguish whether the instances come from the seen or unseen classes. First the visual feature of instance is projected into the semantic space. Then the absolute norm difference between the projected semantic vector and the class semantic embedding vector, and the minimum distance between the projected semantic vectors and the semantic embedding vectors of the seen classes are used as discrimination basis. This approach is termed as SD (Semantic Discriminator) because domain judgement of instance is performed in the semantic space. Our approach can be combined with any existing ZSL method and fully supervision classification model to form a new GZSL method. Furthermore, our approach is very simple and does not need any fixed parameters.