Search

Employing Large Language Models (LLMs) to assess the quality of generated responses, such as prompting instruct-tuned models or fine-tuning judge models, has become a widely adopted evaluation method. It is also known that such evaluators are vulnerable to biases, such as favoring longer responses. While it is important to overcome this problem, the specifics of these biases remain under-explored. In this work, we qualitatively identify six types of biases inherent in various judge models. We propose EvalBiasBench as a meta-evaluation collection of hand-crafted test cases for each bias type. Additionally, we present de-biasing dataset construction methods and the associated preference dataset OffsetBias. Experimental results demonstrate that fine-tuning on our dataset significantly enhances the robustness of judge models against biases and improves performance across most evaluation scenarios. We release our datasets and the fine-tuned judge model to public., Comment: Work in Progress

Multimodal generative models have shown impressive advances with the help of powerful diffusion models. Despite the progress, generating sound solely from text poses challenges in ensuring comprehensive scene depiction and temporal alignment. Meanwhile, video-to-sound generation limits the flexibility to prioritize sound synthesis for specific objects within the scene. To tackle these challenges, we propose a novel video-and-text-to-sound generation method, called ReWaS, where video serves as a conditional control for a text-to-audio generation model. Our method estimates the structural information of audio (namely, energy) from the video while receiving key content cues from a user prompt. We employ a well-performing text-to-sound model to consolidate the video control, which is much more efficient for training multimodal diffusion models with massive triplet-paired (audio-video-text) data. In addition, by separating the generative components of audio, it becomes a more flexible system that allows users to freely adjust the energy, surrounding environment, and primary sound source according to their preferences. Experimental results demonstrate that our method shows superiority in terms of quality, controllability, and training efficiency. Our demo is available at https://naver-ai.github.io/rewas, Comment: Project page: https://naver-ai.github.io/rewas

Composed Image Retrieval (CIR) aims to retrieve a target image based on a reference image and conditioning text, enabling controllable searches. Due to the expensive dataset construction cost for CIR triplets, a zero-shot (ZS) CIR setting has been actively studied to eliminate the need for human-collected triplet datasets. The mainstream of ZS-CIR employs an efficient projection module that projects a CLIP image embedding to the CLIP text token embedding space, while fixing the CLIP encoders. Using the projected image embedding, these methods generate image-text composed features by using the pre-trained text encoder. However, their CLIP image and text encoders suffer from the task discrepancy between the pre-training task (text $\leftrightarrow$ image) and the target CIR task (image + text $\leftrightarrow$ image). Conceptually, we need expensive triplet samples to reduce the discrepancy, but we use cheap text triplets instead and update the text encoder. To that end, we introduce the Reducing Task Discrepancy of text encoders for Composed Image Retrieval (RTD), a plug-and-play training scheme for the text encoder that enhances its capability using a novel target-anchored text contrastive learning. We also propose two additional techniques to improve the proposed learning scheme: a hard negatives-based refined batch sampling strategy and a sophisticated concatenation scheme. Integrating RTD into the state-of-the-art projection-based ZS-CIR methods significantly improves performance across various datasets and backbones, demonstrating its efficiency and generalizability., Comment: 17 pages

NGC~613 is a nearby barred spiral galaxy with a nuclear ring. Exploiting high spatial resolution ($\approx20$ pc) Atacama Large Millimeter/sub-millimeter Array $^{12}$CO(1-0) observations, we study the giant molecular clouds (GMCs) in the nuclear ring and its vicinity, identifying $158$ spatially- and spectrally-resolved GMCs. The GMC sizes ($R_{\mathrm{c}}$) are comparable to those of the clouds in the Milky Way (MW) disc, but their gas masses, observed linewidths ($\sigma_{\mathrm{obs,los}}$) and gas mass surface densities are larger. The GMC size -- linewidth relation ($\sigma_{\mathrm{obs,los}}\propto R_{\mathrm{c}}^{0.77}$) is steeper than that of the clouds of the MW disc and centre, and the GMCs are on average only marginally gravitationally bound (with a mean virial parameter $\langle\alpha_{\mathrm{obs,vir}}\rangle\approx1.7$). We discuss the possible origins of the steep size -- linewidth relation and enhanced observed linewidths of the clouds and suggest that a combination of mechanisms such as stellar feedback, gas accretion and cloud-cloud collisions, as well as the gas inflows driven by the large-scale bar, may play a role., Comment: 15 pages, 8 figures, accepted for publication in MNRAS. arXiv admin note: text overlap with arXiv:2304.10471

Can web-based image processing and visualization tools easily integrate into existing websites without significant time and effort? Our Boostlet.js library addresses this challenge by providing an open-source, JavaScript-based web framework to enable additional image processing functionalities. Boostlet examples include kernel filtering, image captioning, data visualization, segmentation, and web-optimized machine-learning models. To achieve this, Boostlet.js uses a browser bookmark to inject a user-friendly plugin selection tool called PowerBoost into any host website. Boostlet also provides on-site access to a standard API independent of any visualization framework for pixel data and scene manipulation. Web-based Boostlets provide a modular architecture and client-side processing capabilities to apply advanced image-processing techniques using consumer-level hardware. The code is open-source and available., Comment: 5 pages, 5 figures

In an era where the volume of data drives the effectiveness of self-supervised learning, the specificity and clarity of data semantics play a crucial role in model training. Addressing this, we introduce HYPerbolic Entailment filtering (HYPE), a novel methodology designed to meticulously extract modality-wise meaningful and well-aligned data from extensive, noisy image-text pair datasets. Our approach leverages hyperbolic embeddings and the concept of entailment cones to evaluate and filter out samples with meaningless or underspecified semantics, focusing on enhancing the specificity of each data sample. HYPE not only demonstrates a significant improvement in filtering efficiency but also sets a new state-of-the-art in the DataComp benchmark when combined with existing filtering techniques. This breakthrough showcases the potential of HYPE to refine the data selection process, thereby contributing to the development of more accurate and efficient self-supervised learning models. Additionally, the image specificity $\epsilon_{i}$ can be independently applied to induce an image-only dataset from an image-text or image-only data pool for training image-only self-supervised models and showed superior performance when compared to the dataset induced by CLIP score., Comment: ECCV 2024; 33pages, 4.5MB

Recent Vision-Language Pre-training (VLP) models have demonstrated significant advancements. Nevertheless, these models heavily rely on image-text pairs that capture only coarse and global information of an image, leading to a limitation in their regional understanding ability. In this work, we introduce \textbf{RegionVLM}, equipped with explicit regional modeling capabilities, allowing them to understand user-indicated image regions. To achieve this, we design a simple yet innovative architecture, requiring no modifications to the model architecture or objective function. Additionally, we leverage a dataset that contains a novel source of information, namely Localized Narratives, which has been overlooked in previous VLP research. Our experiments demonstrate that our single generalist model not only achieves an interactive dialogue system but also exhibits superior performance on various zero-shot region understanding tasks, without compromising its ability for global image understanding., Comment: NAACL 2024 Main Conference

Melanoma is the most aggressive form of skin cancer, and early detection can significantly increase survival rates and prevent cancer spread. However, developing reliable automated detection techniques is difficult due to the lack of standardized datasets and evaluation methods. This study introduces a unified melanoma classification approach that supports 54 combinations of 11 datasets and 24 state-of-the-art deep learning architectures. It enables a fair comparison of 1,296 experiments and results in a lightweight model deployable to the web-based MeshNet architecture named Mela-D. This approach can run up to 33x faster by reducing parameters 24x to yield an analogous 88.8\% accuracy comparable with ResNet50 on previously unseen images. This allows efficient and accurate melanoma detection in real-world settings that can run on consumer-level hardware., Comment: 10 pages, 9 figures

Composed image retrieval (CIR) task takes a composed query of image and text, aiming to search relative images for both conditions. Conventional CIR approaches need a training dataset composed of triplets of query image, query text, and target image, which is very expensive to collect. Several recent works have worked on the zero-shot (ZS) CIR paradigm to tackle the issue without using pre-collected triplets. However, the existing ZS-CIR methods show limited backbone scalability and generalizability due to the lack of diversity of the input texts during training. We propose a novel CIR framework, only using language for its training. Our LinCIR (Language-only training for CIR) can be trained only with text datasets by a novel self-supervision named self-masking projection (SMP). We project the text latent embedding to the token embedding space and construct a new text by replacing the keyword tokens of the original text. Then, we let the new and original texts have the same latent embedding vector. With this simple strategy, LinCIR is surprisingly efficient and highly effective; LinCIR with CLIP ViT-G backbone is trained in 48 minutes and shows the best ZS-CIR performances on four different CIR benchmarks, CIRCO, GeneCIS, FashionIQ, and CIRR, even outperforming supervised method on FashionIQ. Code is available at https://github.com/navervision/lincir, Comment: CVPR 2024 camera-ready; First two authors contributed equally; 17 pages, 3.1MB

Masked image modeling (MIM) has become a leading self-supervised learning strategy. MIMs such as Masked Autoencoder (MAE) learn strong representations by randomly masking input tokens for the encoder to process, with the decoder reconstructing the masked tokens to the input. However, MIM pre-trained encoders often exhibit a limited attention span, attributed to MIM's sole focus on regressing masked tokens only, which may impede the encoder's broader context learning. To tackle the limitation, we improve MIM by explicitly incorporating unmasked tokens into the training process. Specifically, our method enables the encoder to learn from broader context supervision, allowing unmasked tokens to experience broader contexts while the decoder reconstructs masked tokens. Thus, the encoded unmasked tokens are equipped with extensive contextual information, empowering masked tokens to leverage the enhanced unmasked tokens for MIM. As a result, our simple remedy trains more discriminative representations revealed by achieving 84.2% top-1 accuracy with ViT-B on ImageNet-1K with 0.6%p gain. We attribute the success to the enhanced pre-training method, as evidenced by the singular value spectrum and attention analyses. Finally, our models achieve significant performance gains at the downstream semantic segmentation and fine-grained visual classification tasks; and on diverse robust evaluation metrics. Code is available at https://github.com/naver-ai/lut

We study the Bochner-Riesz problem for the twisted Laplacian $\mathcal L$ on $\mathbb R^2$. For $p\in [1, \infty]\setminus\{2\}$, it has been conjectured that the Bochner-Riesz means $S_\lambda^\delta(\mathcal L) f$ of order $\delta$ converges in $L^p$ for every $f\in L^p$ if and only if $\delta> \max(0,|(p-2)/p|-1/2)$. We prove the conjecture by obtaining uniform $L^p$ bounds on $S_\lambda^\delta(\mathcal L)$ up to the sharp summability indices., Comment: 15 pages

In this note we show that the strong spherical maximal function in $\mathbb R^d$ is bounded on $L^p$ if $p>2(d+1)/(d-1)$ for $d\ge 3$., Comment: 13 pages

Image-Text Matching (ITM) task, a fundamental vision-language (VL) task, suffers from the inherent ambiguity arising from multiplicity and imperfect annotations. Deterministic functions are not sufficiently powerful to capture ambiguity, prompting the exploration of probabilistic embeddings to tackle the challenge. However, the existing probabilistic ITM approach encounters two key shortcomings; the burden of heavy computations due to the Monte Carlo approximation, and the loss saturation issue in the face of abundant false negatives. To overcome the issues, this paper presents an improved Probabilistic Cross-Modal Embeddings (named PCME++) by introducing a new probabilistic distance with a closed-form solution. In addition, two optimization techniques are proposed to enhance PCME++ further: first, the incorporation of pseudo-positives to prevent the negative effect under massive false negatives; second, mixed sample data augmentation for probabilistic matching. Experimental results on MS-COCO Caption and two extended benchmarks, CxC and ECCV Caption, demonstrate the effectiveness of PCME++ compared to state-of-the-art ITM methods. The robustness of PCME++ is also evaluated under noisy image-text correspondences. In addition, the potential applicability of PCME++ in automatic prompt-filtering for zero-shot classification is shown. The code is available at https://github.com/naver-ai/pcmepp, Comment: ICLR 2024 camera-ready; Code: https://github.com/naver-ai/pcmepp. Project page: https://naver-ai.github.io/pcmepp/. 30 pages, 2.2 MB

We study the elliptic maximal functions defined by averages over ellipses and rotated ellipses which are multi-parametric variants of the circular maximal function. We prove that those maximal functions are bounded on $L^p$ for some $p\neq \infty$. For this purpose, we obtain some sharp multi-parameter local smoothing estimates.

Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), digital twin, Metaverse and other related digital technologies have attracted much attention in recent years. These new emerging technologies are changing the world significantly. This research introduces a fusion model, i.e. Fusion Universe (FU), where the virtual, physical, and cognitive worlds are merged together. Therefore, it is crucial to establish a set of principles for the fusion model that is compatible with our physical universe laws and principles. This paper investigates several aspects that could affect immersive and interactive experience; and proposes the fundamental principles for Fusion Universe that can integrate physical and virtual world seamlessly., Comment: 5 pages, 8 figures

Meta-learning performs adaptation through a limited amount of support set, which may cause a sample bias problem. To solve this problem, transductive meta-learning is getting more and more attention, going beyond the conventional inductive learning perspective. This paper proposes so-called task-adaptive pseudo labeling for transductive meta-learning. Specifically, pseudo labels for unlabeled query sets are generated from labeled support sets through label propagation. Pseudo labels enable to adopt the supervised setting as it is and also use the unlabeled query set in the adaptation process. As a result, the proposed method is able to deal with more examples in the adaptation process than inductive ones, which can result in better classification performance of the model. Note that the proposed method is the first approach of applying task adaptation to pseudo labeling. Experiments show that the proposed method outperforms the state-of-the-art (SOTA) technique in 5-way 1-shot few-shot classification.

In this paper, we propose a robustness benchmark for image-text matching models to assess their vulnerabilities. To this end, we insert adversarial texts and images into the search pool (i.e., gallery set) and evaluate models with the adversarial data. Specifically, we replace a word in the text to change the meaning of the text and mix images with different images to create perceptible changes in pixels. We assume that such explicit alterations would not deceive a robust model, as they should understand the holistic meaning of texts and images simultaneously. However, in our evaluations on the proposed benchmark, many state-of-the-art models show significant performance degradation, e.g., Recall@1: 81.9% $\rightarrow$ 64.5% in BLIP, 66.1% $\rightarrow$ 37.5% in VSE$\infty$, where the models favor adversarial texts/images over the original ones. This reveals the current vision-language models may not account for subtle changes or understand the overall context of texts and images. Our findings can provide insights for improving the robustness of the vision-language models and devising more diverse stress-test methods in cross-modal retrieval task. Source code and dataset will be available at https://github.com/pseulki/rococo.

Recovering 3D human mesh in the wild is greatly challenging as in-the-wild (ITW) datasets provide only 2D pose ground truths (GTs). Recently, 3D pseudo-GTs have been widely used to train 3D human mesh estimation networks as the 3D pseudo-GTs enable 3D mesh supervision when training the networks on ITW datasets. However, despite the great potential of the 3D pseudo-GTs, there has been no extensive analysis that investigates which factors are important to make more beneficial 3D pseudo-GTs. In this paper, we provide three recipes to obtain highly beneficial 3D pseudo-GTs of ITW datasets. The main challenge is that only 2D-based weak supervision is allowed when obtaining the 3D pseudo-GTs. Each of our three recipes addresses the challenge in each aspect: depth ambiguity, sub-optimality of weak supervision, and implausible articulation. Experimental results show that simply re-training state-of-the-art networks with our new 3D pseudo-GTs elevates their performance to the next level without bells and whistles. The 3D pseudo-GT is publicly available in https://github.com/mks0601/NeuralAnnot_RELEASE., Comment: Published at CVPRW 2023

Recently, the idea of using FP8 as a number format for neural network training has been floating around the deep learning world. Given that most training is currently conducted with entire networks in FP32, or sometimes FP16 with mixed-precision, the step to having some parts of a network run in FP8 with 8-bit weights is an appealing potential speed-up for the generally costly and time-intensive training procedures in deep learning. A natural question arises regarding what this development means for efficient inference on edge devices. In the efficient inference device world, workloads are frequently executed in INT8. Sometimes going even as low as INT4 when efficiency calls for it. In this whitepaper, we compare the performance for both the FP8 and INT formats for efficient on-device inference. We theoretically show the difference between the INT and FP formats for neural networks and present a plethora of post-training quantization and quantization-aware-training results to show how this theory translates to practice. We also provide a hardware analysis showing that the FP formats are somewhere between 50-180% less efficient in terms of compute in dedicated hardware than the INT format. Based on our research and a read of the research field, we conclude that although the proposed FP8 format could be good for training, the results for inference do not warrant a dedicated implementation of FP8 in favor of INT8 for efficient inference. We show that our results are mostly consistent with previous findings but that important comparisons between the formats have thus far been lacking. Finally, we discuss what happens when FP8-trained networks are converted to INT8 and conclude with a brief discussion on the most efficient way for on-device deployment and an extensive suite of INT8 results for many models.

This paper proposes a novel diffusion-based model, CompoDiff, for solving zero-shot Composed Image Retrieval (ZS-CIR) with latent diffusion. This paper also introduces a new synthetic dataset, named SynthTriplets18M, with 18.8 million reference images, conditions, and corresponding target image triplets to train CIR models. CompoDiff and SynthTriplets18M tackle the shortages of the previous CIR approaches, such as poor generalizability due to the small dataset scale and the limited types of conditions. CompoDiff not only achieves a new state-of-the-art on four ZS-CIR benchmarks, including FashionIQ, CIRR, CIRCO, and GeneCIS, but also enables a more versatile and controllable CIR by accepting various conditions, such as negative text, and image mask conditions. CompoDiff also shows the controllability of the condition strength between text and image queries and the trade-off between inference speed and performance, which are unavailable with existing CIR methods. The code and dataset are available at https://github.com/navervision/CompoDiff, Comment: TMLR camera-ready; First two authors contributed equally; TMLR Expert Certification; 30 pages, 5.9MB

We need billion-scale images to achieve more generalizable and ground-breaking vision models, as well as massive dataset storage to ship the images (e.g., the LAION-4B dataset needs 240TB storage space). However, it has become challenging to deal with unlimited dataset storage with limited storage infrastructure. A number of storage-efficient training methods have been proposed to tackle the problem, but they are rarely scalable or suffer from severe damage to performance. In this paper, we propose a storage-efficient training strategy for vision classifiers for large-scale datasets (e.g., ImageNet) that only uses 1024 tokens per instance without using the raw level pixels; our token storage only needs <1% of the original JPEG-compressed raw pixels. We also propose token augmentations and a Stem-adaptor module to make our approach able to use the same architecture as pixel-based approaches with only minimal modifications on the stem layer and the carefully tuned optimization settings. Our experimental results on ImageNet-1k show that our method significantly outperforms other storage-efficient training methods with a large gap. We further show the effectiveness of our method in other practical scenarios, storage-efficient pre-training, and continual learning. Code is available at https://github.com/naver-ai/seit, Comment: ICCV 2023; First two authors contributed equally; code url: https://github.com/naver-ai/seit; 17 pages, 1.2MB

Nuclear rings at the centers of barred galaxies are known to be strongly magnetized. To explore the effects of magnetic fields on star formation in these rings and nuclear gas flows, we run magnetohydrodynamic simulations in which there is a temporally-constant magnetized inflow to the ring, representing a bar-driven inflow. The mass inflow rate is $1\,M_\odot\,\mathrm{yr}^{-1}$, and we explore models with a range of field strength in the inflow. We adopt the TIGRESS framework developed by Kim & Ostriker to handle radiative heating and cooling, star formation, and resulting supernova (SN) feedback. We find that magnetic fields are efficiently amplified in the ring due to rotational shear and SN feedback. Within a few $100\,\mathrm{Myr}$, the turbulent component $B_\mathrm{trb}$ in the ring saturates at $\sim 35\,\mu\mathrm{G}$ (in rough equipartition with the turbulent kinetic energy density), while the regular component $B_\mathrm{reg}$ exceeds $50\,\mu\mathrm{G}$. Expanding superbubbles created by clustered SN explosions vertically drag predominantly-toroidal fields from near the midplane to produce poloidal fields in high-altitude regions. The growth of magnetic fields greatly suppresses star formation at late times. Simultaneously, strong magnetic tension in the ring drives radially inward accretion flows from the ring to form a circumnuclear disk in the central region; this feature is absent in the unmagnetized model., Comment: 29 pages, 17 figures, accepted for publication in ApJ

Let $\mathcal L$ denote the twisted Laplacian in $\mathbb C^d$. We study almost everywhere convergence of the Bochner--Riesz mean $S^\delta_{t}(\mathcal L) f$ of $f\in L^p(\mathbb C^d)$ as $t\to \infty$, which is an expansion of $f$ in the special Hermite functions. For $2\le p\le \infty$, we obtain the sharp range of the summability indices $\delta$ for which the convergence of $S^\delta_{t}(\mathcal L) f$ holds for all $f\in L^p(\mathbb C^d)$., Comment: 23 pages

Many existing group fairness-aware training methods aim to achieve the group fairness by either re-weighting underrepresented groups based on certain rules or using weakly approximated surrogates for the fairness metrics in the objective as regularization terms. Although each of the learning schemes has its own strength in terms of applicability or performance, respectively, it is difficult for any method in the either category to be considered as a gold standard since their successful performances are typically limited to specific cases. To that end, we propose a principled method, dubbed as \ours, which unifies the two learning schemes by incorporating a well-justified group fairness metric into the training objective using a class wise distributionally robust optimization (DRO) framework. We then develop an iterative optimization algorithm that minimizes the resulting objective by automatically producing the correct re-weights for each group. Our experiments show that FairDRO is scalable and easily adaptable to diverse applications, and consistently achieves the state-of-the-art performance on several benchmark datasets in terms of the accuracy-fairness trade-off, compared to recent strong baselines.

Machine-learned force fields have generated significant interest in recent years as a tool for molecular dynamics (MD) simulations, with the aim of developing accurate and efficient models that can replace classical interatomic potentials. However, before these models can be confidently applied to materials simulations, they must be thoroughly tested and validated. The existing tests on the radial distribution function and mean-squared displacements are insufficient in assessing the transferability of these models. Here we present a more comprehensive set of benchmarking tests for evaluating the transferability of machine-learned force fields. We use a graph neural network (GNN)-based force field coupled with the OpenMM package to carry out MD simulations for Argon as a test case. Our tests include computational X-ray photon correlation spectroscopy (XPCS) signals, which capture the density fluctuation at various length scales in the liquid phase, as well as phonon density-of-state in the solid phase and the liquid-solid phase transition behavior. Our results show that the model can accurately capture the behavior of the solid phase only when the configurations from the solid phase are included in the training dataset. This underscores the importance of appropriately selecting the training data set when developing machine-learned force fields. The tests presented in this work provide a necessary foundation for the development and application of machine-learned force fields for materials simulations., Comment: 27 pages, 14 figures, under review

We study $L^{p}\times L^{q}\rightarrow L^{r}$-boundedness of (sub)bilinear maximal functions associated with degenerate hypersurfaces. First, we obtain the maximal bound on the sharp range of exponents $p,q,r$ (except some border line cases) for the bilinear maximal functions given by the model surface $\big\{(y,z)\in\mathbb{R}^{n}\times \mathbb{R}^{n}:|y|^{l_{1}}+|z|^{l_{2}}=1\big\}$, $(l_{1},l_{2})\in [1,\infty)^2$, $n\ge 2$. Our result manifests that nonvanishing Gaussian curvature is not good enough, in contrast with $L^p$-boundedness of the (sub)linear maximal operator associated to hypersurfaces, to characterize the best possible maximal boundedness. Secondly, we consider the bilinear maximal function associated to the finite type curve in $\mathbb R^2$ and obtain a complete characterization of the maximal bound. We also prove multilinear generalizations of the aforementioned results., Comment: 21 pages, 3 figures

Vision Transformer (ViT) extracts the final representation from either class token or an average of all patch tokens, following the architecture of Transformer in Natural Language Processing (NLP) or Convolutional Neural Networks (CNNs) in computer vision. However, studies for the best way of aggregating the patch tokens are still limited to average pooling, while widely-used pooling strategies, such as max and GeM pooling, can be considered. Despite their effectiveness, the existing pooling strategies do not consider the architecture of ViT and the channel-wise difference in the activation maps, aggregating the crucial and trivial channels with the same importance. In this paper, we present Group Generalized Mean (GGeM) pooling as a simple yet powerful pooling strategy for ViT. GGeM divides the channels into groups and computes GeM pooling with a shared pooling parameter per group. As ViT groups the channels via a multi-head attention mechanism, grouping the channels by GGeM leads to lower head-wise dependence while amplifying important channels on the activation maps. Exploiting GGeM shows 0.1%p to 0.7%p performance boosts compared to the baselines and achieves state-of-the-art performance for ViT-Base and ViT-Large models in ImageNet-1K classification task. Moreover, GGeM outperforms the existing pooling strategies on image retrieval and multi-modal representation learning tasks, demonstrating the superiority of GGeM for a variety of tasks. GGeM is a simple algorithm in that only a few lines of code are necessary for implementation.

We investigate $L^p$ boundedness of the maximal function defined by the averaging operator $f\to \mathcal{A}_t^s f$ over the two-parameter family of tori $\mathbb{T}_t^{s}:=\{ ( (t+s\cos\theta)\cos\phi,\,(t+s\cos\theta)\sin\phi,\, s\sin\theta ): \theta, \phi \in [0,2\pi) \}$ with $c_0t>s>0$ for some $c_0\in (0,1)$. We prove that the associated (two-parameter) maximal function is bounded on $L^p$ if and only if $p>2$. We also obtain $L^p$--$L^q$ estimates for the local maximal operator on a sharp range of $p,q$. Furthermore, the sharp smoothing estimates are proved including the sharp local smoothing estimates for the operators $f\to \mathcal A_t^s f$ and $f\to \mathcal A_t^{c_0t} f$. For the purpose, we make use of Bourgain--Demeter's decoupling inequality for the cone and Guth--Wang--Zhang's local smoothing estimates for the $2$ dimensional wave operator., Comment: 30 pages, 2 figures

In recent years, many deep neural architectures have been developed for image classification. Whether they are similar or dissimilar and what factors contribute to their (dis)similarities remains curious. To address this question, we aim to design a quantitative and scalable similarity measure between neural architectures. We propose Similarity by Attack Transferability (SAT) from the observation that adversarial attack transferability contains information related to input gradients and decision boundaries widely used to understand model behaviors. We conduct a large-scale analysis on 69 state-of-the-art ImageNet classifiers using our proposed similarity function to answer the question. Moreover, we observe neural architecture-related phenomena using model similarity that model diversity can lead to better performance on model ensembles and knowledge distillation under specific conditions. Our results provide insights into why developing diverse neural architectures with distinct components is necessary., Comment: ECCV 2024; 35pages, 2.56MB

We study $L^p$-$L^q$ bounds on the spectral projection operator $\Pi_\lambda$ associated to the Hermite operator $H=|x|^2-\Delta$ in $\mathbb R^d$. We are mainly concerned with a localized operator $\chi_E\Pi_\lambda\chi_E$ for a subset $E\subset\mathbb R^d$ and undertake the task of characterizing the sharp $L^p$--$L^q$ bounds. We obtain sharp bounds in extended ranges of $p,q$. First, we provide a complete characterization of the sharp $L^p$--$L^q$ bounds when $E$ is away from $\sqrt{\lambda}\mathbb S^{d-1}$. Secondly, we obtain the sharp bounds as the set $E$ gets close to $\sqrt\lambda\mathbb S^{d-1}$. Thirdly, we extend the range of $p,q$ for which the operator $\Pi_\lambda$ is uniformly bounded from $L^p(\mathbb R^d)$ to $L^q(\mathbb R^d)$., Comment: The paper is a modified version of a part of the paper Hermite spectral projection operator (arXiv:2006.11762v3). The previous paper will remain unpublished