Search

With the rapid development of social media, tremendous videos with new classes are generated daily, which raise an urgent demand for video classification methods that can continuously update new classes while maintaining the knowledge of old videos with limited storage and computing resources. In this paper, we summarize this task as Class-Incremental Video Classification (CIVC) and propose a novel framework to address it. As a subarea of incremental learning tasks, the challenge of catastrophic forgetting is unavoidable in CIVC. To better alleviate it, we utilize some characteristics of videos. First, we decompose the spatio-temporal knowledge before distillation rather than treating it as a whole in the knowledge transfer process; trajectory is also used to refine the decomposition. Second, we propose a dual granularity exemplar selection method to select and store representative video instances of old classes and key-frames inside videos under a tight storage budget. We benchmark our method and previous SOTA class-incremental learning methods on Something-Something V2 and Kinetics datasets, and our method outperforms previous methods significantly.

We have studied polarized image of a Schwarzschild black hole with an equatorial thin accretion disk as photon couples to Weyl tensor. The birefringence of photon originating from the coupling affect the black hole shadow, the thin disk pattern and its luminosity distribution. We also analyze the observed polarized intensity in the sky plane. The observed polarized intensity in the bright region is stronger than that in the darker region. The stronger effect of the coupling on the observed polarized vector appears only in the bright region close to black hole. These features in the polarized image could help us to understand black hole shadow, the thin accretion disk and the coupling between photon and Weyl tensor., Comment: 15 pages, 11 figures, the accepted version for publication in EPJC

The success of machine learning applications often needs a large quantity of data. Recently, federated learning (FL) is attracting increasing attention due to the demand for data privacy and security, especially in the medical field. However, the performance of existing FL approaches often deteriorates when there exist domain shifts among clients, and few previous works focus on personalization in healthcare. In this article, we propose FedHealth 2, an extension of FedHealth \cite{chen2020fedhealth} to tackle domain shifts and get personalized models for local clients. FedHealth 2 obtains the client similarities via a pretrained model, and then it averages all weighted models with preserving local batch normalization. Wearable activity recognition and COVID-19 auxiliary diagnosis experiments have evaluated that FedHealth 2 can achieve better accuracy (10%+ improvement for activity recognition) and personalized healthcare without compromising privacy and security., Comment: Accepted by IJCAI-21 workshop on federated learning

The existing human pose estimation methods are confronted with inaccurate long-distance regression or high computational cost due to the complex learning objectives. This work proposes a novel deep learning framework for human pose estimation called composite localization to divide the complex learning objective into two simpler ones: a sparse heatmap to find the keypoint's approximate location and two short-distance offsetmaps to obtain its final precise coordinates. To realize the framework, we construct two types of composite localization networks: CLNet-ResNet and CLNet-Hourglass. We evaluate the networks on three benchmark datasets, including the Leeds Sports Pose dataset, the MPII Human Pose dataset, and the COCO keypoints detection dataset. The experimental results show that our CLNet-ResNet50 outperforms SimpleBaseline by 1.14% with about 1/2 GFLOPs. Our CLNet-Hourglass outperforms the original stacked-hourglass by 4.45% on COCO.

Temporal action localization is an important and challenging task that aims to locate temporal regions in real-world untrimmed videos where actions occur and recognize their classes. It is widely acknowledged that video context is a critical cue for video understanding, and exploiting the context has become an important strategy to boost localization performance. However, previous state-of-the-art methods focus more on exploring semantic context which captures the feature similarity among frames or proposals, and neglect positional context which is vital for temporal localization. In this paper, we propose a temporal-position-sensitive context modeling approach to incorporate both positional and semantic information for more precise action localization. Specifically, we first augment feature representations with directed temporal positional encoding, and then conduct attention-based information propagation, in both frame-level and proposal-level. Consequently, the generated feature representations are significantly empowered with the discriminative capability of encoding the position-aware context information, and thus benefit boundary detection and proposal evaluation. We achieve state-of-the-art performance on both two challenging datasets, THUMOS-14 and ActivityNet-1.3, demonstrating the effectiveness and generalization ability of our method.

It is expensive and time-consuming to collect sufficient labeled data for human activity recognition (HAR). Domain adaptation is a promising approach for cross-domain activity recognition. Existing methods mainly focus on adapting cross-domain representations via domain-level, class-level, or sample-level distribution matching. However, they might fail to capture the fine-grained locality information in activity data. The domain- and class-level matching are too coarse that may result in under-adaptation, while sample-level matching may be affected by the noise seriously and eventually cause over-adaptation. In this paper, we propose substructure-level matching for domain adaptation (SSDA) to better utilize the locality information of activity data for accurate and efficient knowledge transfer. Based on SSDA, we propose an optimal transport-based implementation, Substructural Optimal Transport (SOT), for cross-domain HAR. We obtain the substructures of activities via clustering methods and seeks the coupling of the weighted substructures between different domains. We conduct comprehensive experiments on four public activity recognition datasets (i.e. UCI-DSADS, UCI-HAR, USC-HAD, PAMAP2), which demonstrates that SOT significantly outperforms other state-of-the-art methods w.r.t classification accuracy (9%+ improvement). In addition, our mehtod is 5x faster than traditional OT-based DA methods with the same hyper-parameters., Comment: Accepted by Neurocomputing; 17 pages; Code: https://github.com/jindongwang/transferlearning/tree/master/code/traditional/sot

We have studied the shadow and its luminosity for a static and regular phantom compact object under the static spherical accretion and the infalling spherical accretion, respectively. Comparing with the usual Schwarzschild black hole, the presence of phantom hair yields the larger black hole shadow and the darker image. In both spherical accretion models, with the increase of phantom parameter, the maximum luminosity occurred at photon ring and the brightness of the central region in the shadow decrease, but in the region far from the shadow, the luminosity of image slightly increases. The image of a phantom wormhole and its luminosity have similar behaviors. Moreover, as the phantom charge parameter $\alpha$ increases up to the critical value at where the compact object changes from black hole to wormhole, there exists a jump for the specific intensity, which also appears in the slowly rotating case. This implies that the phantom hair is imprinted on both the shadow radius and the intensity of the electromagnetic flux radiation around compacted object., Comment: 19 pages, 13 figures. Accepted for publication in class. Quantum Grav

As an important and challenging problem, multi-domain learning (MDL) typically seeks for a set of effective lightweight domain-specific adapter modules plugged into a common domain-agnostic network. Usually, existing ways of adapter plugging and structure design are handcrafted and fixed for all domains before model learning, resulting in the learning inflexibility and computational intensiveness. With this motivation, we propose to learn a data-driven adapter plugging strategy with Neural Architecture Search (NAS), which automatically determines where to plug for those adapter modules. Furthermore, we propose a NAS-adapter module for adapter structure design in a NAS-driven learning scheme, which automatically discovers effective adapter module structures for different domains. Experimental results demonstrate the effectiveness of our MDL model against existing approaches under the conditions of comparable performance.

In this paper, we propose conformal inference based approach for statistical verification of CPS models. Cyber-physical systems (CPS) such as autonomous vehicles, avionic systems, and medical devices operate in highly uncertain environments. This uncertainty is typically modeled using a finite number of parameters or input signals. Given a system specification in Signal Temporal Logic (STL), we would like to verify that for all (infinite) values of the model parameters/input signals, the system satisfies its specification. Unfortunately, this problem is undecidable in general. {\em Statistical model checking} (SMC) offers a solution by providing guarantees on the correctness of CPS models by statistically reasoning on model simulations. We propose a new approach for statistical verification of CPS models for user-provided distribution on the model parameters. Our technique uses model simulations to learn {\em surrogate models}, and uses {\em conformal inference} to provide probabilistic guarantees on the satisfaction of a given STL property. Additionally, we can provide prediction intervals containing the quantitative satisfaction values of the given STL property for any user-specified confidence level. We also propose a refinement procedure based on Gaussian Process (GP)-based surrogate models for obtaining fine-grained probabilistic guarantees over sub-regions in the parameter space. This in turn enables the CPS designer to choose assured validity domains in the parameter space for safety-critical applications. Finally, we demonstrate the efficacy of our technique on several CPS models.

Ubiquitous systems with End-Edge-Cloud architecture are increasingly being used in healthcare applications. Federated Learning (FL) is highly useful for such applications, due to silo effect and privacy preserving. Existing FL approaches generally do not account for disparities in the quality of local data labels. However, the clients in ubiquitous systems tend to suffer from label noise due to varying skill-levels, biases or malicious tampering of the annotators. In this paper, we propose Federated Opportunistic Computing for Ubiquitous Systems (FOCUS) to address this challenge. It maintains a small set of benchmark samples on the FL server and quantifies the credibility of the client local data without directly observing them by computing the mutual cross-entropy between performance of the FL model on the local datasets and that of the client local FL model on the benchmark dataset. Then, a credit weighted orchestration is performed to adjust the weight assigned to clients in the FL model based on their credibility values. FOCUS has been experimentally evaluated on both synthetic data and real-world data. The results show that it effectively identifies clients with noisy labels and reduces their impact on the model performance, thereby significantly outperforming existing FL approaches., Comment: 7 pages

In combined heat and power systems, varying mass flow can better make use of the heating system inertia to increase the flexibility of electric power systems. This is challenging, however, due to integer variables and bilinear constraints in existing optimal dispatch models. In this paper, we incorporate an improved heat pipeline model to eliminate complexity from integer variables without compromise on accuracy. Subsequently, the resulting optimal dispatch model with bilinear constraints is solved by the proposed modified Generalized Benders Decomposition method, which decomposes the optimal dispatch model into a convex sub-problem with the fixed mass flow and a simple upper-level problem searching for the optimal mass flow. Comparisons with existing benchmarks show that the proposed method can achieve lower operation costs with outstanding computational efficiency., Comment: 8 pages

With more and more distributed energy resources (DERs) deployed in Integrated Energy Systems (IESs), frequency stability challenges the pursuit of reliability and efficiency. This paper proposes a fully-distributed frequency control method for load-side DERs, in which the optimality can be guaranteed in an IES where electricity and heat are coupled. Moreover, the global asymptotic stability of the closed-loop system is proved and the robustness with respect to inaccurate coefficients is shown. Case studies demonstrate the effectiveness of proposed method., Comment: 5 pages

Autonomous systems such as self-driving cars and general-purpose robots are safety-critical systems that operate in highly uncertain and dynamic environments. We propose an interactive multi-agent framework where the system-under-design is modeled as an ego agent and its environment is modeled by a number of adversarial (ado) agents. For example, a self-driving car is an ego agent whose behavior is influenced by ado agents such as pedestrians, bicyclists, traffic lights, road geometry etc. Given a logical specification of the correct behavior of the ego agent, and a set of constraints that encode reasonable adversarial behavior, our framework reduces the adversarial testing problem to the problem of synthesizing controllers for (constrained) ado agents that cause the ego agent to violate its specifications. Specifically, we explore the use of tabular and deep reinforcement learning approaches for synthesizing adversarial agents. We show that ado agents trained in this fashion are better than traditional falsification or testing techniques because they can generalize to ego agents and environments that differ from the original ego agent. We demonstrate the efficacy of our technique on two real-world case studies from the domain of self-driving cars.

With the rapid development of computing technology, wearable devices such as smart phones and wristbands make it easy to get access to people's health information including activities, sleep, sports, etc. Smart healthcare achieves great success by training machine learning models on a large quantity of user data. However, there are two critical challenges. Firstly, user data often exists in the form of isolated islands, making it difficult to perform aggregation without compromising privacy security. Secondly, the models trained on the cloud fail on personalization. In this paper, we propose FedHealth, the first federated transfer learning framework for wearable healthcare to tackle these challenges. FedHealth performs data aggregation through federated learning, and then builds personalized models by transfer learning. It is able to achieve accurate and personalized healthcare without compromising privacy and security. Experiments demonstrate that FedHealth produces higher accuracy (5.3% improvement) for wearable activity recognition when compared to traditional methods. FedHealth is general and extensible and has the potential to be used in many healthcare applications., Comment: IJCAI-19 Workshop on Federated Machine Learning for User Privacy and Data Confidentiality (IJCAI (FML)) 2019; fix typos; journal version: https://ieeexplore.ieee.org/document/9076082

Superconductivity in group IV semiconductors is desired for hybrid devices combining both semiconducting and superconducting properties. Following boron doped diamond and Si, superconductivity has been observed in gallium doped Ge, however the obtained specimen is in polycrystalline form [Herrmannsd\"orfer et al., Phys. Rev. Lett. 102, 217003 (2009)]. Here, we present superconducting single-crystalline Ge hyperdoped with gallium or aluminium by ion implantation and rear-side flash lamp annealing. The maximum concentration of Al and Ga incorporated into substitutional positions in Ge is eight times higher than the equilibrium solid solubility. This corresponds to a hole concentration above 10^21 cm-3. Using density functional theory in the local density approximation and pseudopotential plane-wave approach, we show that the superconductivity in p-type Ge is phonon-mediated. According to the ab initio calculations the critical superconducting temperature for Al- and Ga-doped Ge is in the range of 0.45 K for 6.25 at.% of dopant concentration being in a qualitative agreement with experimentally obtained values., Comment: including the suppl. information, 24 pages, accepted at Phys. Rev. Materials