Search

Out-of-distribution (OOD) detection is an important topic for real-world machine learning systems, but settings with limited in-distribution samples have been underexplored. Such few-shot OOD settings are challenging, as models have scarce opportunities to learn the data distribution before being tasked with identifying OOD samples. Indeed, we demonstrate that recent state-of-the-art OOD methods fail to outperform simple baselines in the few-shot setting. We thus propose a hypernetwork framework called HyperMix, using Mixup on the generated classifier parameters, as well as a natural out-of-episode outlier exposure technique that does not require an additional outlier dataset. We conduct experiments on CIFAR-FS and MiniImageNet, significantly outperforming other OOD methods in the few-shot regime.

Zero-shot learning (ZSL) is a promising approach to generalizing a model to categories unseen during training by leveraging class attributes, but challenges remain. Recently, methods using generative models to combat bias towards classes seen during training have pushed state of the art, but these generative models can be slow or computationally expensive to train. Also, these generative models assume that the attribute vector of each unseen class is available a priori at training, which is not always practical. Additionally, while many previous ZSL methods assume a one-time adaptation to unseen classes, in reality, the world is always changing, necessitating a constant adjustment of deployed models. Models unprepared to handle a sequential stream of data are likely to experience catastrophic forgetting. We propose a Meta-learned Attribute self-Interaction Network (MAIN) for continual ZSL. By pairing attribute self-interaction trained using meta-learning with inverse regularization of the attribute encoder, we are able to outperform state-of-the-art results without leveraging the unseen class attributes while also being able to train our models substantially faster (>100x) than expensive generative-based approaches. We demonstrate this with experiments on five standard ZSL datasets (CUB, aPY, AWA1, AWA2, and SUN) in the generalized zero-shot learning and continual (fixed/dynamic) zero-shot learning settings. Extensive ablations and analyses demonstrate the efficacy of various components proposed., Comment: Accepted in IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2024. arXiv admin note: substantial text overlap with arXiv:2102.11856

We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric video of skilled human activities (e.g., sports, music, dance, bike repair). 740 participants from 13 cities worldwide performed these activities in 123 different natural scene contexts, yielding long-form captures from 1 to 42 minutes each and 1,286 hours of video combined. The multimodal nature of the dataset is unprecedented: the video is accompanied by multichannel audio, eye gaze, 3D point clouds, camera poses, IMU, and multiple paired language descriptions -- including a novel "expert commentary" done by coaches and teachers and tailored to the skilled-activity domain. To push the frontier of first-person video understanding of skilled human activity, we also present a suite of benchmark tasks and their annotations, including fine-grained activity understanding, proficiency estimation, cross-view translation, and 3D hand/body pose. All resources are open sourced to fuel new research in the community. Project page: http://ego-exo4d-data.org/, Comment: updated baseline results and dataset statistics to match the released v2 data; added table to appendix comparing stats of Ego-Exo4D alongside other datasets

We introduce the notion of a risk-limiting financial auditing (RLFA): given $N$ transactions, the goal is to estimate the total misstated monetary fraction~($m^*$) to a given accuracy $\epsilon$, with confidence $1-\delta$. We do this by constructing new confidence sequences (CSs) for the weighted average of $N$ unknown values, based on samples drawn without replacement according to a (randomized) weighted sampling scheme. Using the idea of importance weighting to construct test martingales, we first develop a framework to construct CSs for arbitrary sampling strategies. Next, we develop methods to improve the quality of CSs by incorporating side information about the unknown values associated with each item. We show that when the side information is sufficiently predictive, it can directly drive the sampling. Addressing the case where the accuracy is unknown a priori, we introduce a method that incorporates side information via control variates. Crucially, our construction is adaptive: if the side information is highly predictive of the unknown misstated amounts, then the benefits of incorporating it are significant; but if the side information is uncorrelated, our methods learn to ignore it. Our methods recover state-of-the-art bounds for the special case when the weights are equal, which has already found applications in election auditing. The harder weighted case solves our more challenging problem of AI-assisted financial auditing., Comment: 23 pages, 8 figures, to appear in the Proceedings of Uncertainty in Artificial Intelligence (UAI) 2023

Many online action prediction models observe complete frames to locate and attend to informative subregions in the frames called glimpses and recognize an ongoing action based on global and local information. However, in applications with constrained resources, an agent may not be able to observe the complete frame, yet must still locate useful glimpses to predict an incomplete action based on local information only. In this paper, we develop Glimpse Transformers (GliTr), which observe only narrow glimpses at all times, thus predicting an ongoing action and the following most informative glimpse location based on the partial spatiotemporal information collected so far. In the absence of a ground truth for the optimal glimpse locations for action recognition, we train GliTr using a novel spatiotemporal consistency objective: We require GliTr to attend to the glimpses with features similar to the corresponding complete frames (i.e. spatial consistency) and the resultant class logits at time $t$ equivalent to the ones predicted using whole frames up to $t$ (i.e. temporal consistency). Inclusion of our proposed consistency objective yields ~10% higher accuracy on the Something-Something-v2 (SSv2) dataset than the baseline cross-entropy objective. Overall, despite observing only ~33% of the total area per frame, GliTr achieves 53.02% and 93.91% accuracy on the SSv2 and Jester datasets, respectively., Comment: Accepted to WACV 2023

Most existing OCR methods focus on alphanumeric characters due to the popularity of English and numbers, as well as their corresponding datasets. On extending the characters to more languages, recent methods have shown that training different scripts with different recognition heads can greatly improve the end-to-end recognition accuracy compared to combining characters from all languages in the same recognition head. However, we postulate that similarities between some languages could allow sharing of model parameters and benefit from joint training. Determining language groupings, however, is not immediately obvious. To this end, we propose an automatic method for multilingual text recognition with a task grouping and assignment module using Gumbel-Softmax, introducing a task grouping loss and weighted recognition loss to allow for simultaneous training of the models and grouping modules. Experiments on MLT19 lend evidence to our hypothesis that there is a middle ground between combining every task together and separating every task that achieves a better configuration of task grouping/separation., Comment: ECCV 2022: Text in Everything (TIE) Workshop (Oral)

An accurate and detailed account of patient medications, including medication changes within the patient timeline, is essential for healthcare providers to provide appropriate patient care. Healthcare providers or the patients themselves may initiate changes to patient medication. Medication changes take many forms, including prescribed medication and associated dosage modification. These changes provide information about the overall health of the patient and the rationale that led to the current care. Future care can then build on the resulting state of the patient. This work explores the automatic extraction of medication change information from free-text clinical notes. The Contextual Medication Event Dataset (CMED) is a corpus of clinical notes with annotations that characterize medication changes through multiple change-related attributes, including the type of change (start, stop, increase, etc.), initiator of the change, temporality, change likelihood, and negation. Using CMED, we identify medication mentions in clinical text and propose three novel high-performing BERT-based systems that resolve the annotated medication change characteristics. We demonstrate that our proposed systems improve medication change classification performance over the initial work exploring CMED.

The interaction between localized f and itinerant conduction electrons is crucial in the electronic properties of heavy fermion and valence fluctuating compounds. Using high-resolution angle-resolved photoemission spectroscopy, we systematically investigate the electronic structure of the archetypical valence fluctuating compound EuNi2P2 that hosts multiple f electrons. At low temperatures, we reveal the hybridization between Eu 4f and Ni 3d states, which contributes to the electron mass enhancement, consistent with the periodic Anderson model. With increasing temperature, interestingly, we observe opposite temperature evolution of electron spectral function above and below the Kondo coherence temperature near 110 K, which is in contrast to the monotonic valence change and beyond the expectation of the periodic Anderson model. We argue that both f-d hybridization and correlation are imperative in the electronic properties of EuNi2P2. Our results shed light on the understanding of novel properties, such as heavy fermion behaviors and valence fluctuation, of rare-earth transition-metal intermetallic compounds with multiple f electrons.

Real-world graphs can be difficult to interpret and visualize beyond a certain size. To address this issue, graph summarization aims to simplify and shrink a graph, while maintaining its high-level structure and characteristics. Most summarization methods are designed for homogeneous, undirected, simple graphs; however, many real-world graphs are ornate; with characteristics including node labels, directed edges, edge multiplicities, and self-loops. In this paper we propose LM-Gsum, a versatile yet rigorous graph summarization model that (to the best of our knowledge, for the first time) can handle graphs with all the aforementioned characteristics (and any combination thereof). Moreover, our proposed model captures basic sub-structures that are prevalent in real-world graphs, such as cliques, stars, etc. LM-Gsum compactly quantifies the information content of a complex graph using a novel encoding scheme, where it seeks to minimize the total number of bits required to encode (i) the summary graph, as well as (ii) the corrections required for reconstructing the input graph losslessly. To accelerate the summary construction, it creates super-nodes efficiently by merging nodes in groups. Experiments demonstrate that LM-Gsum facilitates the visualization of real-world complex graphs, revealing interpretable structures and high- level relationships. Furthermore, LM-Gsum achieves better trade-off between compression rate and running time, relative to existing methods (only) on comparable settings., Comment: 17 pages, 8 figures, 4 tables

Existing question answering (QA) datasets derived from electronic health records (EHR) are artificially generated and consequently fail to capture realistic physician information needs. We present Discharge Summary Clinical Questions (DiSCQ), a newly curated question dataset composed of 2,000+ questions paired with the snippets of text (triggers) that prompted each question. The questions are generated by medical experts from 100+ MIMIC-III discharge summaries. We analyze this dataset to characterize the types of information sought by medical experts. We also train baseline models for trigger detection and question generation (QG), paired with unsupervised answer retrieval over EHRs. Our baseline model is able to generate high quality questions in over 62% of cases when prompted with human selected triggers. We release this dataset (and all code to reproduce baseline model results) to facilitate further research into realistic clinical QA and QG: https://github.com/elehman16/discq.

Chiral materials have attracted significant research interests as they exhibit intriguing physical properties, such as chiral optical response, spin-momentum locking and chiral induced spin selectivity. Recently, layered transition metal dichalcogenide 1T-TaS2 has been found to host a chiral charge density wave (CDW) order. Nevertheless, the physical consequences of the chiral order, for example, in electronic structures and the optical properties, are yet to be explored. Here, we report the spectroscopic visualization of an emergent chiral electronic band structure in the CDW phase, characterized by windmill-shape Fermi surfaces. We uncover a remarkable chirality-dependent circularly polarized Raman response due to the salient chiral symmetry of CDW, although the ordinary circular dichroism vanishes. Chiral Fermi surfaces and anomalous Raman responses coincide with the CDW transition, proving their lattice origin. Our work paves a path to manipulate the chiral electronic and optical properties in two-dimensional materials and explore applications in polarization optics and spintronics.

Topological electronic structures are key to the topological classification of quantum materials and play an important role in their physical properties and applications. Recently, SnSe has attracted great research interests due to its superior thermoelectric performance. However, it's topological nature has long been ignored. In this work, by combining synchrotron-based angle-resolved photoemission spectroscopy and ab-initio calculations, we systematically investigated the topological electronic structure of orthorhombic SnSe. By identifying the continuous gap in the valence bands due to the band inversion and the topological surface states on its (001) surface, we establish SnSe as a strong topological insulator. Furthermore, we studied the evolution of the topological electronic structure and propose the topological phase diagram in SnSe1-xTex. Our work reveals the topological non-trivial nature of SnSe and provides new understandings of its intriguing transport properties., Comment: 17 pages, 4 figures, accepted for publication in Physical Review Materials

Few-shot learning (FSL) methods typically assume clean support sets with accurately labeled samples when training on novel classes. This assumption can often be unrealistic: support sets, no matter how small, can still include mislabeled samples. Robustness to label noise is therefore essential for FSL methods to be practical, but this problem surprisingly remains largely unexplored. To address mislabeled samples in FSL settings, we make several technical contributions. (1) We offer simple, yet effective, feature aggregation methods, improving the prototypes used by ProtoNet, a popular FSL technique. (2) We describe a novel Transformer model for Noisy Few-Shot Learning (TraNFS). TraNFS leverages a transformer's attention mechanism to weigh mislabeled versus correct samples. (3) Finally, we extensively test these methods on noisy versions of MiniImageNet and TieredImageNet. Our results show that TraNFS is on-par with leading FSL methods on clean support sets, yet outperforms them, by far, in the presence of label noise., Comment: Accepted to CVPR 2022

We study the challenging incremental few-shot object detection (iFSD) setting. Recently, hypernetwork-based approaches have been studied in the context of continuous and finetune-free iFSD with limited success. We take a closer look at important design choices of such methods, leading to several key improvements and resulting in a more accurate and flexible framework, which we call Sylph. In particular, we demonstrate the effectiveness of decoupling object classification from localization by leveraging a base detector that is pretrained for class-agnostic localization on a large-scale dataset. Contrary to what previous results have suggested, we show that with a carefully designed class-conditional hypernetwork, finetune-free iFSD can be highly effective, especially when a large number of base categories with abundant data are available for meta-training, almost approaching alternatives that undergo test-time-training. This result is even more significant considering its many practical advantages: (1) incrementally learning new classes in sequence without additional training, (2) detecting both novel and seen classes in a single pass, and (3) no forgetting of previously seen classes. We benchmark our model on both COCO and LVIS, reporting as high as 17% AP on the long-tail rare classes on LVIS, indicating the promise of hypernetwork-based iFSD., Comment: Accepted to CVPR 2022

In many applications, such as autonomous driving, hand manipulation, or robot navigation, object detection methods must be able to detect objects unseen in the training set. Open World Detection(OWD) seeks to tackle this problem by generalizing detection performance to seen and unseen class categories. Recent works have seen success in the generation of class-agnostic proposals, which we call Open-World Proposals(OWP), but this comes at the cost of a big drop on the classification task when both tasks are considered in the detection model. These works have investigated two-stage Region Proposal Networks (RPN) by taking advantage of objectness scoring cues; however, for its simplicity, run-time, and decoupling of localization and classification, we investigate OWP through the lens of fully convolutional one-stage detection network, such as FCOS. We show that our architectural and sampling optimizations on FCOS can increase OWP performance by as much as 6% in recall on novel classes, marking the first proposal-free one-stage detection network to achieve comparable performance to RPN-based two-stage networks. Furthermore, we show that the inherent, decoupled architecture of FCOS has benefits to retaining classification performance. While two-stage methods worsen by 6% in recall on novel classes, we show that FCOS only drops 2% when jointly optimizing for OWP and classification.

Non-Hermitian systems with specific forms of Hamiltonians can exhibit novel phenomena. However, it is difficult to study their quantum thermodynamical properties. In particular, the calculation of work statistics can be challenging in non-Hermitian systems due to the change of state norm. To tackle this problem, we modify the two-point measurement method in Hermitian systems. The modified method can be applied to non-Hermitian systems which are Hermitian before and after the evolution. In Hermitian systems, our method is equivalent to the two-point measurement method. When the system is non-Hermitian, our results represent a projection of the statistics in a larger Hermitian system. As an example, we calculate the work statistics in a non-Hermitian Su-Schrieffer-Heeger model. Our results reveal several differences between the work statistics in non-Hermitian systems and the one in Hermitian systems., Comment: 12 pages, 4 figures

Topological Weyl semimetals (TWSs) are exotic crystals possessing emergent relativistic Weyl fermions connected by unique surface Fermi-arcs (SFAs) in their electronic structures. To realize the TWS state, certain symmetry (such as the inversion or time reversal symmetry) must be broken, leading to a topological phase transition (TPT). Despite the great importance in understanding the formation of TWSs and their unusual properties, direct observation of such a TPT has been challenging. Here, using a recently discovered magnetic TWS Co3Sn2S2, we were able to systematically study its TPT with detailed temperature dependence of the electronic structures by angle-resolved photoemission spectroscopy. The TPT with drastic band structures evolution was clearly observed across the Curie temperature (TC = 177 K), including the disappearance of the characteristic SFAs and the recombination of the spin-split bands that leads to the annihilation of Weyl points with opposite chirality. These results not only reveal important insights on the interplay between the magnetism and band topology in TWSs, but also provide a new method to control their exotic physical properties., Comment: 15 pages, 4 figures

Recently, layered copper chalcogenides Cu2X family (X=S, Se, Te) has attracted tremendous research interests due to their high thermoelectric (TE) performance, which is partly due to the superionic behavior of mobile Cu ions, making these compounds phonon liquids. Here, we systematically investigate the electronic structure and its temperature evolution of the less studied single crystal Cu2-xTe by the combination of angle resolved photoemission spectroscopy (ARPES) and scanning tunneling microscope/spectroscopy (STM/STS) experiments. While the band structure of the Cu2-xTe shows agreement with the calculations, we clearly observe a 2 * 2 surface reconstruction from both our low temperature ARPES and STM/STS experiments which survives up to room temperature. Interestingly, our low temperature STM experiments further reveal multiple types of reconstruction patterns, which suggests the origin of the surface reconstruction being the distributed deficiency of liquid-like Cu ions. Our findings reveal the electronic structure and impurity level of Cu2Te, which provides knowledge about its thermoelectric properties from the electronic degree of freedom.

Topological materials host fascinating low dimensional gapless states at the boundary. As a prominent example, helical topological edge states (TESs) of two-dimensional topological insulators (2DTIs) and their stacked three-dimensional (3D) equivalent, weak topological insulators (WTIs), have sparked research enthusiasm due to their potential application in the next generation of electronics/spintronics with low dissipation. Here, we propose layered superconducting material CaSn as a WTI with nontrivial Z2 as well as nodal line semimetal protected by crystalline non-symmorphic symmetry. Our systematic angle-resolved photoemission spectroscopy (ARPES) investigation on the electronic structure exhibits excellent agreement with the calculation. Furthermore, scanning tunnelling microscopy/spectroscopy (STM/STS) at the surface step edge shows signatures of the expected TES. These integrated evidences from ARPES, STM/STS measurement and corresponding ab initio calculation strongly support the existence of TES in the non-symmorphic nodal line semimetal CaSn, which may become a versatile material platform to realize multiple exotic electronic states as well as topological superconductivity.

Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. Although it is recognized that statistical heterogeneity of the client local data yields slower global model convergence, it is less commonly recognized that it also yields a biased federated global model with a high variance of accuracy across clients. In this work, we aim to provide federated learning schemes with improved fairness. To tackle this challenge, we propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server). Empirical results on a suite of datasets demonstrate the effectiveness of our methods on simultaneously improving the test accuracy and fairness., Comment: Accepted by IEEE CVPR Workshop on Fair, Data Efficient And Trusted Computer Vision

Topological superconductors (TSCs) are unconventional superconductors with bulk superconducting gap and in-gap Majorana states on the boundary that may be used as topological qubits for quantum computation. Despite their importance in both fundamental research and applications, natural TSCs are very rare. Here, combining state of the art synchrotron and laser-based angle-resolved photoemission spectroscopy, we investigated a stoichiometric transition metal dichalcogenide (TMD), 2M-WS2 with a superconducting transition temperature of 8.8 K (the highest among all TMDs in the natural form up to date) and observed distinctive topological surface states (TSSs). Furthermore, in the superconducting state, we found that the TSSs acquired a nodeless superconducting gap with similar magnitude as that of the bulk states. These discoveries not only evidence 2M-WS2 as an intrinsic TSC without the need of sensitive composition tuning or sophisticated heterostructures fabrication, but also provide an ideal platform for device applications thanks to its van der Waals layered structure., Comment: Accepted by Nature Communications

Recent advances in OCR have shown that an end-to-end (E2E) training pipeline that includes both detection and recognition leads to the best results. However, many existing methods focus primarily on Latin-alphabet languages, often even only case-insensitive English characters. In this paper, we propose an E2E approach, Multiplexed Multilingual Mask TextSpotter, that performs script identification at the word level and handles different scripts with different recognition heads, all while maintaining a unified loss that simultaneously optimizes script identification and multiple recognition heads. Experiments show that our method outperforms the single-head model with similar number of parameters in end-to-end recognition tasks, and achieves state-of-the-art results on MLT17 and MLT19 joint text detection and script identification benchmarks. We believe that our work is a step towards the end-to-end trainable and scalable multilingual multi-purpose OCR system. Our code and model will be released.