Search

Minority samples are underrepresented instances located in low-density regions of a data manifold, and are valuable in many generative AI applications, such as data augmentation, creative content generation, etc. Unfortunately, existing diffusion-based minority generators often rely on computationally expensive guidance dedicated for minority generation. To address this, here we present a simple yet powerful guidance-free approach called Boost-and-Skip for generating minority samples using diffusion models. The key advantage of our framework requires only two minimal changes to standard generative processes: (i) variance-boosted initialization and (ii) timestep skipping. We highlight that these seemingly-trivial modifications are supported by solid theoretical and empirical evidence, thereby effectively promoting emergence of underrepresented minority features. Our comprehensive experiments demonstrate that Boost-and-Skip greatly enhances the capability of generating minority samples, even rivaling guidance-based state-of-the-art approaches while requiring significantly fewer computations., Comment: 29 pages, 11 figures

$\textbf{Background}$: Cancer remains one of the leading causes of morbidity and mortality worldwide. Comprehensive datasets that combine histopathological images with genetic and survival data across various tumour sites are essential for advancing computational pathology and personalised medicine. $\textbf{Results}$: We present SurGen, a dataset comprising 1,020 H&E-stained whole slide images (WSIs) from 843 colorectal cancer cases. The dataset includes detailed annotations for key genetic mutations (KRAS, NRAS, BRAF) and mismatch repair status, as well as survival data for 426 cases. To demonstrate SurGen's practical utility, we conducted a proof-of-concept machine learning experiment predicting mismatch repair status from the WSIs, achieving a test AUROC of 0.8316. These preliminary results underscore the dataset's potential to facilitate research in biomarker discovery, prognostic modelling, and advanced machine learning applications in colorectal cancer. $\textbf{Conclusions}$: SurGen offers a valuable resource for the scientific community, enabling studies that require high-quality WSIs linked with comprehensive clinical and genetic information on colorectal cancer. Our initial findings affirm the dataset's capacity to advance diagnostic precision and foster the development of personalised treatment strategies in colorectal oncology. Data available online at https://doi.org/10.6019/S-BIAD1285., Comment: To download the dataset, see https://doi.org/10.6019/S-BIAD1285. See https://github.com/CraigMyles/SurGen-Dataset for GitHub repository and additional info

The goal of video moment retrieval and highlight detection is to identify specific segments and highlights based on a given text query. With the rapid growth of video content and the overlap between these tasks, recent works have addressed both simultaneously. However, they still struggle to fully capture the overall video context, making it challenging to determine which words are most relevant. In this paper, we present a novel Video Context-aware Keyword Attention module that overcomes this limitation by capturing keyword variation within the context of the entire video. To achieve this, we introduce a video context clustering module that provides concise representations of the overall video context, thereby enhancing the understanding of keyword dynamics. Furthermore, we propose a keyword weight detection module with keyword-aware contrastive learning that incorporates keyword information to enhance fine-grained alignment between visual and textual features. Extensive experiments on the QVHighlights, TVSum, and Charades-STA benchmarks demonstrate that our proposed method significantly improves performance in moment retrieval and highlight detection tasks compared to existing approaches. Our code is available at: https://github.com/VisualAIKHU/Keyword-DETR, Comment: Accepted at AAAI 2025

We present a novel methodology for modeling the influence of the unresolved scales of turbulence for sub-grid modeling. Our approach employs the differentiable physics paradigm in deep learning, allowing a neural network to interact with the differential equation evolution and performing an a posteriori optimization by incorporating the solver into the training iteration (an approach known as solver-in-the-loop), thus departing from the conventional a priori instantaneous training approach. Our method ensures that the model is exposed to equations-informed input distributions, accounting for prior corrections and often leading to more accurate and stable time evolution. We present results of our methodology applied to a shell model of turbulence, and we discuss further potential applications to Navier-Stokes equations.

We investigate the generation of minority samples using pretrained text-to-image (T2I) latent diffusion models. Minority instances, in the context of T2I generation, can be defined as ones living on low-density regions of text-conditional data distributions. They are valuable for various applications of modern T2I generators, such as data augmentation and creative AI. Unfortunately, existing pretrained T2I diffusion models primarily focus on high-density regions, largely due to the influence of guided samplers (like CFG) that are essential for producing high-quality generations. To address this, we present a novel framework to counter the high-density-focus of T2I diffusion models. Specifically, we first develop an online prompt optimization framework that can encourage the emergence of desired properties during inference while preserving semantic contents of user-provided prompts. We subsequently tailor this generic prompt optimizer into a specialized solver that promotes the generation of minority features by incorporating a carefully-crafted likelihood objective. Our comprehensive experiments, conducted across various types of T2I models, demonstrate that our approach significantly enhances the capability to produce high-quality minority instances compared to existing samplers., Comment: 20 pages, 9 figures

This paper introduces SpoofCeleb, a dataset designed for Speech Deepfake Detection (SDD) and Spoofing-robust Automatic Speaker Verification (SASV), utilizing source data from real-world conditions and spoofing attacks generated by Text-To-Speech (TTS) systems also trained on the same real-world data. Robust recognition systems require speech data recorded in varied acoustic environments with different levels of noise to be trained. However, existing datasets typically include clean, high-quality recordings (bona fide data) due to the requirements for TTS training; studio-quality or well-recorded read speech is typically necessary to train TTS models. Existing SDD datasets also have limited usefulness for training SASV models due to insufficient speaker diversity. We present SpoofCeleb, which leverages a fully automated pipeline that processes the VoxCeleb1 dataset, transforming it into a suitable form for TTS training. We subsequently train 23 contemporary TTS systems. The resulting SpoofCeleb dataset comprises over 2.5 million utterances from 1,251 unique speakers, collected under natural, real-world conditions. The dataset includes carefully partitioned training, validation, and evaluation sets with well-controlled experimental protocols. We provide baseline results for both SDD and SASV tasks. All data, protocols, and baselines are publicly available at https://jungjee.github.io/spoofceleb., Comment: 9 pages, 2 figures, 8 tables

Text-to-speech (TTS) systems are traditionally trained using modest databases of studio-quality, prompted or read speech collected in benign acoustic environments such as anechoic rooms. The recent literature nonetheless shows efforts to train TTS systems using data collected in the wild. While this approach allows for the use of massive quantities of natural speech, until now, there are no common datasets. We introduce the TTS In the Wild (TITW) dataset, the result of a fully automated pipeline, in this case, applied to the VoxCeleb1 dataset commonly used for speaker recognition. We further propose two training sets. TITW-Hard is derived from the transcription, segmentation, and selection of VoxCeleb1 source data. TITW-Easy is derived from the additional application of enhancement and additional data selection based on DNSMOS. We show that a number of recent TTS models can be trained successfully using TITW-Easy, but that it remains extremely challenging to produce similar results using TITW-Hard. Both the dataset and protocols are publicly available and support the benchmarking of TTS systems trained using TITW data., Comment: 5 pages, submitted to ICASSP 2025 as a conference paper

We present a novel approach for generating minority samples that live on low-density regions of a data manifold. Our framework is built upon diffusion models, leveraging the principle of guided sampling that incorporates an arbitrary energy-based guidance during inference time. The key defining feature of our sampler lies in its \emph{self-contained} nature, \ie, implementable solely with a pretrained model. This distinguishes our sampler from existing techniques that require expensive additional components (like external classifiers) for minority generation. Specifically, we first estimate the likelihood of features within an intermediate latent sample by evaluating a reconstruction loss w.r.t. its posterior mean. The generation then proceeds with the minimization of the estimated likelihood, thereby encouraging the emergence of minority features in the latent samples of subsequent timesteps. To further improve the performance of our sampler, we provide several time-scheduling techniques that properly manage the influence of guidance over inference steps. Experiments on benchmark real datasets demonstrate that our approach can greatly improve the capability of creating realistic low-likelihood minority instances over the existing techniques without the reliance on costly additional elements. Code is available at \url{https://github.com/soobin-um/sg-minority}., Comment: ECCV 2024

This paper addresses the challenge of representing complex human action (HA) in a nuclear power plant (NPP) digital twin (DT) and minimizing latency in partial computation offloading (PCO) in sixth-generation-enabled computing in the network (COIN) assisted multiaccess edge computing (MEC). Accurate HA representation in the DT-HA model is vital for modeling human interventions that are crucial for the safe and efficient operation of NPPs. In this context, DT-enabled COIN-assisted MEC harnesses DT (known as a cybertwin) capabilities to optimize resource allocation and reduce latency effectively. A two-stage approach is employed to address system complexity. First, a probabilistic graphical model (PGM) is introduced to capture HAs in the DT abstraction. In the PGM, HA and NPP asset-twin abstractions form coupled systems that evolve and interact through observable data and control input. Next, the underlying PCO problem is formulated as a multiuser game, where NPP assets can partially offload tasks to COIN and MEC. We propose a decentralized algorithm to optimize offloading decisions, offloading ratios, and resource allocation. The simulation results demonstrate the effectiveness of the proposed method in capturing complex HAs and optimal resource allocation in DT-enabled NPPs.

Researchers have focused on understanding how individual's behavior is influenced by the behaviors of their peers in observational studies of social networks. Identifying and estimating causal peer influence, however, is challenging due to confounding by homophily, where people tend to connect with those who share similar characteristics with them. Moreover, since all the attributes driving homophily are generally not always observed and act as unobserved confounders, identifying and estimating causal peer influence becomes infeasible using standard causal identification assumptions. In this paper, we address this challenge by leveraging latent locations inferred from the network itself to disentangle homophily from causal peer influence, and we extend this approach to multiple networks by adopting a Bayesian hierarchical modeling framework. To accommodate the nonlinear dependency of peer influence on individual behavior, we employ a Bayesian nonparametric method, specifically Bayesian Additive Regression Trees (BART), and we propose a Bayesian framework that accounts for the uncertainty in inferring latent locations. We assess the operating characteristics of the estimator via extensive simulation study. Finally, we apply our method to estimate causal peer influence in advice-seeking networks of teachers in secondary schools, in order to assess whether the teachers' belief about mathematics education is influenced by the beliefs of their peers from whom they receive advice. Our results suggest that, overlooking latent homophily can lead to either underestimation or overestimation of causal peer influence, accompanied by considerable estimation uncertainty., Comment: 39 pages, 15 figures, 2 tables

Understanding battery degradation in electric vehicles (EVs) under real-world conditions remains a critical yet under-explored area of research. Central to this investigation is the challenge of estimating the specific degradation modes in aged cells with no available information on usage history, bypassing the conventional yet invasive method of tear-down tests. Using an electrochemical model, this study pioneers a methodology to decouple and isolate the aging mechanisms in batteries sourced from EVs with varying mileages. A robust correlation is established between the model parameters and distinct degradation processes, enabling the diagnosis and estimation of each mechanism's impact on the battery's parameters. This paper sheds light on battery degradation in real-world scenarios and demonstrates the feasibility of their identification, isolation, and approximate quantification of their effects., Comment: 6 pages, 13 figures, Submitted to 2024 Modeling, Estimation, and Control Conference

This paper addresses the problem of minimizing latency with partial computation offloading within Industrial Internet-of-Things (IoT) systems in in-network computing (COIN)-assisted Multiaccess Edge Computing (C-MEC) via ultra-reliable and low latency communications (URLLC) links. We propose a digital twin (DT) scheme for a multiuser scenario, allowing collaborative partial task offloading from user equipment (UE) to COIN-aided nodes or MEC. Specifically, we formulate the problem as joint task offloading decision, ratio and resource allocation. We employ game theory to create a low-complexity distributed offloading scheme in which the task offloading decision problem is modelled as an exact potential game. Double Deep Q-Network (DDQN) is utilized within the game to proactively predict optimal offloading ratio and resource allocation. This approach optimizes resource allocation across the whole system and enhances the robustness of the computing framework, ensuring efficient execution of computation-intensive services. Additionally, it addresses centralized approaches and UE resource contention issues, thus ensuring faster and more reliable communication., Comment: 9 pages, 3 figures

The goal of the multi-sound source localization task is to localize sound sources from the mixture individually. While recent multi-sound source localization methods have shown improved performance, they face challenges due to their reliance on prior information about the number of objects to be separated. In this paper, to overcome this limitation, we present a novel multi-sound source localization method that can perform localization without prior knowledge of the number of sound sources. To achieve this goal, we propose an iterative object identification (IOI) module, which can recognize sound-making objects in an iterative manner. After finding the regions of sound-making objects, we devise object similarity-aware clustering (OSC) loss to guide the IOI module to effectively combine regions of the same object but also distinguish between different objects and backgrounds. It enables our method to perform accurate localization of sound-making objects without any prior knowledge. Extensive experimental results on the MUSIC and VGGSound benchmarks show the significant performance improvements of the proposed method over the existing methods for both single and multi-source. Our code is available at: https://github.com/VisualAIKHU/NoPrior_MultiSSL, Comment: Accepted at CVPR 2024

This study investigates the suitability of the annealed approximation in high-dimensional systems characterized by dense networks with quenched link disorder, employing models of coupled oscillators. We demonstrate that dynamic equations governing dense-network systems converge to those of the complete-graph version in the thermodynamic limit, where link disorder fluctuations vanish entirely. Consequently, the annealed-network systems, where fluctuations are attenuated, also exhibit the same dynamic behavior in the thermodynamic limit. However, a significant discrepancy arises in the incoherent (disordered) phase wherein the finite-size behavior becomes critical in determining the steady-state pattern. To explicitly elucidate this discrepancy, we focus on identical oscillators subject to competitive attractive and repulsive couplings. In the incoherent phase of dense networks, we observe the manifestation of random irregular states. In contrast, the annealed approximation yields a symmetric (regular) incoherent state where two oppositely coherent clusters of oscillators coexist, accompanied by the vanishing order parameter. Our findings imply that the annealed approximation should be employed with caution even in dense-network systems, particularly in the disordered phase.

This study addresses the challenge of real-time metaverse applications by proposing an in-network placement and task-offloading solution for delay-constrained computing tasks in next-generation networks. The metaverse, envisioned as a parallel virtual world, requires seamless real-time experiences across diverse applications. The study introduces a software-defined networking (SDN)-based architecture and employs graph neural network (GNN) techniques for intelligent and adaptive task allocation in in-network computing (INC). Considering time constraints and computing capabilities, the proposed model optimally decides whether to offload rendering tasks to INC nodes or edge server. Extensive experiments demonstrate the superior performance of the proposed GNN model, achieving 97% accuracy compared to 72% for multilayer perceptron (MLP) and 70% for decision trees (DTs). The study fills the research gap in in-network placement for real-time metaverse applications, offering insights into efficient rendering task handling.

The computing in the network (COIN) paradigm has emerged as a potential solution for computation-intensive applications like the metaverse by utilizing unused network resources. The blockchain (BC) guarantees task-offloading privacy, but cost reduction, queueing delays, and redundancy elimination remain open problems. This paper presents a redundancy-aware BC-based approach for the metaverse's partial computation offloading (PCO). Specifically, we formulate a joint BC redundancy factor (BRF) and PCO problem to minimize computation costs, maximize incentives, and meet delay and BC offloading constraints. We proved this problem is NP-hard and transformed it into two subproblems based on their temporal correlation: real-time PCO and Markov decision process-based BRF. We formulated the PCO problem as a multiuser game, proposed a decentralized algorithm for Nash equilibrium under any BC redundancy state, and designed a double deep Q-network-based algorithm for the optimal BRF policy. The BRF strategy is updated periodically based on user computation demand and network status to assist the PCO algorithm. The experimental results suggest that the proposed approach outperforms existing schemes, resulting in a remarkable 47% reduction in cost overhead, delivering approximately 64% higher rewards, and achieving convergence in just a few training episodes.