Search

Building an echocardiography view classifier that maintains performance in real-life cases requires diverse multi-site data, and frequent updates with newly available data to mitigate model drift. Simply fine-tuning on new datasets results in "catastrophic forgetting", and cannot adapt to variations of view labels between sites. Alternatively, collecting all data on a single server and re-training may not be feasible as data sharing agreements may restrict image transfer, or datasets may only become available at different times. Furthermore, time and cost associated with re-training grows with every new dataset. We propose a class-incremental learning method which learns an expert network for each dataset, and combines all expert networks with a score fusion model. The influence of ``unqualified experts'' is minimised by weighting each contribution with a learnt in-distribution score. These weights promote transparency as the contribution of each expert is known during inference. Instead of using the original images, we use learned features from each dataset, which are easier to share and raise fewer licensing and privacy concerns. We validate our work on six datasets from multiple sites, demonstrating significant reductions in training time while improving view classification performance., Comment: Accepted for Oral at MICCAI workshop ASMUS-2024

Prompt engineering is critical for effective interaction with large language models (LLMs) such as ChatGPT. However, efforts to teach this skill to students have been limited. This study designed and implemented a prompt engineering intervention, examining its influence on undergraduate students' AI self-efficacy, AI knowledge, and proficiency in creating effective prompts. The intervention involved 27 students who participated in a 100-minute workshop conducted during their history course at a university in Hong Kong. During the workshop, students were introduced to prompt engineering strategies, which they applied to plan the course's final essay task. Multiple data sources were collected, including students' responses to pre- and post-workshop questionnaires, pre- and post-workshop prompt libraries, and written reflections. The study's findings revealed that students demonstrated a higher level of AI self-efficacy, an enhanced understanding of AI concepts, and improved prompt engineering skills because of the intervention. These findings have implications for AI literacy education, as they highlight the importance of prompt engineering training for specific higher education use cases. This is a significant shift from students haphazardly and intuitively learning to engineer prompts. Through prompt engineering education, educators can faciitate students' effective navigation and leverage of LLMs to support their coursework., Comment: 34 pages, 6 figures

All human societies present unique narratives that shape their customs and beliefs. Despite cultural differences, some symbolic elements (e.g., heroes and tricksters) are common across many cultures. Here, we reconcile these seemingly contradictory aspects by analyzing mythological themes and traditions at various scales. Our analysis revealed that global mythologies exhibit both geographic and thematic nesting across different scales, manifesting in a layered structure. The largest geographic clusters correspond to the New and Old Worlds, which further divide into smaller bioregions. This hierarchical manifestation closely aligns with historical human migration patterns at a large scale, suggesting that narrative themes were carried through deep history. At smaller scales, the correspondence with bioregions indicates that these themes are locally adapted and diffused into variations across cultures over time. Our approach, which treats myths and traditions as random variables without considering factors like geography, history, or story lineage, suggests that the manifestation of mythology has been well-preserved over time and thus opens exciting research avenues to reconstruct historical patterns and provide insight into human cultural narratives.

While graphs and abstract data structures can be large and complex, practical instances are often regular or highly structured. If the instance has sufficient structure, we might hope to compress the object into a more succinct representation. An efficient algorithm (with respect to the compressed input size) could then lead to more efficient computations than algorithms taking the explicit, uncompressed object as input. This leads to a natural question: when does knowing the input instance has a more succinct representation make computation easier? We initiate the study of the computational complexity of problems on factored graphs: graphs that are given as a formula of products and unions on smaller graphs. For any graph problem, we define a parameterized version that takes factored graphs as input, parameterized by the number of (smaller) ordinary graphs used to construct the factored graph. In this setting, we characterize the parameterized complexity of several natural graph problems, exhibiting a variety of complexities. We show that a decision version of lexicographically first maximal independent set is $\mathbf{XP}$-complete, and therefore unconditionally not fixed-parameter tractable ($\mathbf{FPT}$). On the other hand, we show that clique counting is $\mathbf{FPT}$. Finally, we show that reachability is $\mathbf{XNL}$-complete. Moreover, $\mathbf{XNL}$ is contained in $\mathbf{FPT}$ if and only if $\mathbf{NL}$ is contained in some fixed polynomial time., Comment: To appear in ITCS 2025

Current image editing methods primarily utilize DDIM Inversion, employing a two-branch diffusion approach to preserve the attributes and layout of the original image. However, these methods encounter challenges with non-rigid edits, which involve altering the image's layout or structure. Our comprehensive analysis reveals that the high-frequency components of DDIM latent, crucial for retaining the original image's key features and layout, significantly contribute to these limitations. Addressing this, we introduce FlexiEdit, which enhances fidelity to input text prompts by refining DDIM latent, by reducing high-frequency components in targeted editing areas. FlexiEdit comprises two key components: (1) Latent Refinement, which modifies DDIM latent to better accommodate layout adjustments, and (2) Edit Fidelity Enhancement via Re-inversion, aimed at ensuring the edits more accurately reflect the input text prompts. Our approach represents notable progress in image editing, particularly in performing complex non-rigid edits, showcasing its enhanced capability through comparative experiments., Comment: ECCV 2024

VLT/UVES spectroscopic and TESS photometric observations for WASP 0346-21 allow the direct determination of its physical properties, along with the detection of a circumbinary object and oscillating signals. The high-resolution spectra yielded the radial velocities of all three stars and the atmospheric parameters of $T_{\rm eff,A}$ = 7225$\pm42$ K, [M/H] = 0.30$\pm$0.03 dex, and $v_{\rm A}$$\sin i$ = 78$\pm$5 km s$^{-1}$ of the primary component. The combined analysis of these observations resulted in the fundamental parameters of the eclipsing components and the third light of $l_3$ = 0.043$\pm$0.004, which is consistent with the light contribution of the tertiary star observed in the echelle spectra. WASP 0346-21 A resides within the overlapping main-sequence domain of $\delta$ Sct and $\gamma$ Dor variables, while the secondary component of $M_{\rm B}$ = 0.185$\pm$0.013 M$_\odot$, $R_{\rm B}$ = 0.308$\pm$0.023 R$_\odot$, $T_{\rm eff,B}$ = 10,655$\pm$146 K, and $L_{\rm B}$ = 1.09$\pm$0.17 L$_\odot$ matches well with the low-mass white dwarf (WD) model for $Z$ = 0.01, corresponding to the thick-disk population classified by the Galactic kinematics. Multifrequency analyses were performed on the residual TESS data after removing the binarity effects. The low frequencies around 26.348 day$^{-1}$ and 17.683 day$^{-1}$ are $\delta$ Sct pulsations originating from WASP 0346-21 A, and the high frequencies of 97.996 day$^{-1}$ and 90.460 day$^{-1}$ are considered to be extremely low-mass WD oscillations. These results demonstrate that WASP 0346-21 is a hierarchical triple system, consisting of an EL CVn binary with multiperiodic pulsations in each component and a distant outer tertiary., Comment: 29 pages, including 9 figures and 5 tables, accepted for publication in ApJ

Competing interactions in frustrated magnets can give rise to highly degenerate ground states from which correlated liquid-like states of matter often emerge. The scaling of this degeneracy influences the ultimate ground state, with extensive degeneracies potentially yielding quantum spin liquids, while sub-extensive or smaller degeneracies yield static orders. A longstanding problem is to understand how ordered states precipitate from this degenerate manifold and what echoes of the degeneracy survive ordering. Here, we use neutron scattering to experimentally demonstrate a new "nodal line" spin liquid, where spins collectively fluctuate within a sub-extensive manifold spanning one-dimensional lines in reciprocal space. Realized in the spin-orbit coupled, face-centered cubic iridate K$_2$IrCl$_6$, we show that the sub-extensive degeneracy is robust, but remains susceptible to fluctuations or longer range interactions which cooperate to select a magnetic order at low temperatures. Proximity to the nodal line spin liquid influences the ordered state, enhancing the effects of quantum fluctuations and stabilizing it through the opening of a large spin-wave gap. Our results demonstrate quantum fluctuations can act counter-intuitively in frustrated materials: instead of destabilizing ordering, at the brink of the nodal spin liquid they can act to stabilize it and dictate its low-energy physics., Comment: Main article: 9 pages, 3 figures. Supplemental material: 22 pages, 15 figures

The outstanding properties of transition metal dichalcogenide (TMD) monolayers and their van der Waals (vdW) heterostructures, arising from their structure and the modified electron-hole Coulomb interaction in two-dimension, make them promising candidates for potential electro-optical devices. However, the production of reproducible devices remains challenging, partly due to variability at the nanometer to atomic scales. Thus, access to chemical, structural, and optical characterization at these lengthscales is essential. While electron microscopy and spectroscopy can provide chemical and structural data, accessing the optical response at the nanoscale through electron spectroscopies has been hindered until recently. This review focuses on the application of two electron spectroscopies in scanning (transmission) electron microscopes, namely cathodoluminescence and electron energy-loss spectroscopy, to study the nano-optics of TMD atomic layers and their vdW heterostructures. How technological advancements that can improve these spectroscopies, many of which are already underway, will make them ideal for studying the physics of vdW heterostructures at the nanoscale will also be discussed.

We propose Cooperative Component Analysis (CoCA), a new method for unsupervised multi-view analysis: it identifies the component that simultaneously captures significant within-view variance and exhibits strong cross-view correlation. The challenge of integrating multi-view data is particularly important in biology and medicine, where various types of "-omic" data, ranging from genomics to proteomics, are measured on the same set of samples. The goal is to uncover important, shared signals that represent underlying biological mechanisms. CoCA combines an approximation error loss to preserve information within data views and an "agreement penalty" to encourage alignment across data views. By balancing the trade-off between these two key components in the objective, CoCA has the property of interpolating between the commonly-used principal component analysis (PCA) and canonical correlation analysis (CCA) as special cases at the two ends of the solution path. CoCA chooses the degree of agreement in a data-adaptive manner, using a validation set or cross-validation to estimate test error. Furthermore, we propose a sparse variant of CoCA that incorporates the Lasso penalty to yield feature sparsity, facilitating the identification of key features driving the observed patterns. We demonstrate the effectiveness of CoCA on simulated data and two real multiomics studies of COVID-19 and ductal carcinoma in situ of breast. In both real data applications, CoCA successfully integrates multiomics data, extracting components that are not only consistently present across different data views but also more informative and predictive of disease progression. CoCA offers a powerful framework for discovering important shared signals in multi-view data, with the potential to uncover novel insights in an increasingly multi-view data world.

The investigation of emerging non-toxic perovskite materials has been undertaken to advance the fabrication of environmentally sustainable lead-free perovskite solar cells. This study introduces a machine learning methodology aimed at predicting innovative halide perovskite materials that hold promise for use in photovoltaic applications. The seven newly predicted materials are as follows: CsMnCl$_4$, Rb$_3$Mn$_2$Cl$_9$, Rb$_4$MnCl$_6$, Rb$_3$MnCl$_5$, RbMn$_2$Cl$_7$, RbMn$_4$Cl$_9$, and CsIn$_2$Cl$_7$. The predicted compounds are first screened using a machine learning approach, and their validity is subsequently verified through density functional theory calculations. CsMnCl$_4$ is notable among them, displaying a bandgap of 1.37 eV, falling within the Shockley-Queisser limit, making it suitable for photovoltaic applications. Through the integration of machine learning and density functional theory, this study presents a methodology that is more effective and thorough for the discovery and design of materials.

Learning efficient representations of local features is a key challenge in feature volume-based 3D neural mapping, especially in large-scale environments. In this paper, we introduce Decomposition-based Neural Mapping (DNMap), a storage-efficient large-scale 3D mapping method that employs a discrete representation based on a decomposition strategy. This decomposition strategy aims to efficiently capture repetitive and representative patterns of shapes by decomposing each discrete embedding into component vectors that are shared across the embedding space. Our DNMap optimizes a set of component vectors, rather than entire discrete embeddings, and learns composition rather than indexing the discrete embeddings. Furthermore, to complement the mapping quality, we additionally learn low-resolution continuous embeddings that require tiny storage space. By combining these representations with a shallow neural network and an efficient octree-based feature volume, our DNMap successfully approximates signed distance functions and compresses the feature volume while preserving mapping quality. Our source code is available at https://github.com/minseong-p/dnmap., Comment: ECCV 2024

Vision language models (VLMs) perceive the world through a combination of a visual encoder and a large language model (LLM). The visual encoder, pre-trained on large-scale vision-text datasets, provides zero-shot generalization to visual data, and the LLM endows its high reasoning ability to VLMs. It leads VLMs to achieve high performance on wide benchmarks without fine-tuning, exhibiting zero or few-shot capability. However, recent studies show that VLMs are vulnerable to hallucination. This undesirable behavior degrades reliability and credibility, thereby making users unable to fully trust the output from VLMs. To enhance trustworthiness and better tackle the hallucination of VLMs, we curate a new evaluation dataset, called the BEfore-AFter hallucination dataset (BEAF), and introduce new metrics: True Understanding (TU), IGnorance (IG), StuBbornness (SB), and InDecision (ID). Unlike prior works that focus only on constructing questions and answers, the key idea of our benchmark is to manipulate visual scene information by image editing models and to design the metrics based on scene changes. This allows us to clearly assess whether VLMs correctly understand a given scene by observing the ability to perceive changes. We also visualize image-wise object relationship by virtue of our two-axis view: vision and text. Upon evaluating VLMs with our dataset, we observed that our metrics reveal different aspects of VLM hallucination that have not been reported before. Project page: \url{https://beafbench.github.io/}, Comment: Accepted at ECCV 2024. [Project Pages] https://beafbench.github.io/

In the pursuit of understanding the population of stellar remnants within the Milky Way, we analyze the sample of $\sim 950$ microlensing events observed by the Spitzer Space Telescope between 2014 and 2019. In this study we focus on a sub-sample of nine microlensing events, selected based on their long timescales, small microlensing parallaxes and joint observations by the Gaia mission, to increase the probability that the chosen lenses are massive and the mass is measurable. Among the selected events we identify lensing black holes and neutron star candidates, with potential confirmation through forthcoming release of the Gaia time-series astrometry in 2026. Utilizing Bayesian analysis and Galactic models, along with the Gaia Data Release 3 proper motion data, four good candidates for dark remnants were identified: OGLE-2016-BLG-0293, OGLE-2018-BLG-0483, OGLE-2018-BLG-0662, and OGLE-2015-BLG-0149, with lens masses of $2.98^{+1.75}_{-1.28}~M_{\odot}$, $4.65^{+3.12}_{-2.08}~M_{\odot}$, $3.15^{+0.66}_{-0.64}~M_{\odot}$ and $1.4^{+0.75}_{-0.55}~M_{\odot}$, respectively. Notably, the first two candidates are expected to exhibit astrometric microlensing signals detectable by Gaia, offering the prospect of validating the lens masses. The methodologies developed in this work will be applied to the full Spitzer microlensing sample, populating and analyzing the time-scale ($t_{\rm E}$) vs. parallax ($\pi_{\rm E}$) diagram to derive constraints on the population of lenses in general and massive remnants in particular., Comment: submitted to ApJ

Predicting future trajectories for other road agents is an essential task for autonomous vehicles. Established trajectory prediction methods primarily use agent tracks generated by a detection and tracking system and HD map as inputs. In this work, we propose a novel method that also incorporates visual input from surround-view cameras, allowing the model to utilize visual cues such as human gazes and gestures, road conditions, vehicle turn signals, etc, which are typically hidden from the model in prior methods. Furthermore, we use textual descriptions generated by a Vision-Language Model (VLM) and refined by a Large Language Model (LLM) as supervision during training to guide the model on what to learn from the input data. Despite using these extra inputs, our method achieves a latency of 53 ms, making it feasible for real-time processing, which is significantly faster than that of previous single-agent prediction methods with similar performance. Our experiments show that both the visual inputs and the textual descriptions contribute to improvements in trajectory prediction performance, and our qualitative analysis highlights how the model is able to exploit these additional inputs. Lastly, in this work we create and release the nuScenes-Text dataset, which augments the established nuScenes dataset with rich textual annotations for every scene, demonstrating the positive impact of utilizing VLM on trajectory prediction. Our project page is at https://moonseokha.github.io/VisionTrap/, Comment: Accepted at ECCV 2024

Deep learning-based segmentation techniques have shown remarkable performance in brain segmentation, yet their success hinges on the availability of extensive labeled training data. Acquiring such vast datasets, however, poses a significant challenge in many clinical applications. To address this issue, in this work, we propose a novel 3D brain segmentation approach using complementary 2D diffusion models. The core idea behind our approach is to first mine 2D features with semantic information extracted from the 2D diffusion models by taking orthogonal views as input, followed by fusing them into a 3D contextual feature representation. Then, we use these aggregated features to train multi-layer perceptrons to classify the segmentation labels. Our goal is to achieve reliable segmentation quality without requiring complete labels for each individual subject. Our experiments on training in brain subcortical structure segmentation with a dataset from only one subject demonstrate that our approach outperforms state-of-the-art self-supervised learning methods. Further experiments on the minimum requirement of annotation by sparse labeling yield promising results even with only nine slices and a labeled background region., Comment: Extended version of "3D Segmentation of Subcortical Brain Structure with Few Labeled Data using 2D Diffusion Models" (ISMRM 2024 oral)

The AMoRE collaboration searches for neutrinoless double beta decay of $^{100}$Mo using molybdate scintillating crystals via low temperature thermal calorimetric detection. The early phases of the experiment, AMoRE-pilot and AMoRE-I, have demonstrated competitive discovery potential. Presently, the AMoRE-II experiment, featuring a large detector array with about 90 kg of $^{100}$Mo isotope, is under construction.This paper discusses the baseline design and characterization of the lithium molybdate cryogenic calorimeters to be used in the AMoRE-II detector modules. The results from prototype setups that incorporate new housing structures and two different crystal masses (316 g and 517 - 521 g), operated at 10 mK temperature, show energy resolutions (FWHM) of 7.55 - 8.82 keV at the 2.615 MeV $^{208}$Tl $\gamma$ line, and effective light detection of 0.79 - 0.96 keV/MeV. The simultaneous heat and light detection enables clear separation of alpha particles with a discrimination power of 12.37 - 19.50 at the energy region around $^6$Li(n, $\alpha$)$^3$H with Q-value = 4.785 MeV. Promising detector performances were demonstrated at temperatures as high as 30 mK, which relaxes the temperature constraints for operating the large AMoRE-II array.

Traditional single-fibre spectroscopy provides a single galaxy spectrum, forming the basis for crucial parameter estimation. However, its accuracy can be compromised by various sources of contamination, such as the prominent \Ha~emission line originating from both Star-Forming (SF) regions and non-Star-Forming regions (NonSF), including Active Galactic Nuclei (AGN). The potential to dissect a spectrum into its SF and NonSF constituents holds the promise of significantly enhancing precision in parameter estimates. In contrast, Integral Field Unit (IFU) surveys present a solution to minimize contamination. These surveys examine spatially localized regions within galaxies, reducing the impact of mixed sources. Although an IFU survey's resulting spectrum covers a smaller region of a galaxy than single-fibre spectroscopy, it can still encompass a blend of heterogeneous sources. Our study introduces an innovative model informed by insights from the MaNGA IFU survey. This model enables the decomposition of galaxy spectra, including those from the Sloan Digital Sky Survey (SDSS), into SF and NonSF components. Applying our model to these survey datasets produces two distinct spectra, one for SF and another for NonSF components, while conserving flux across wavelength bins. When these decomposed spectra are visualized on a BPT diagram, interesting patterns emerge. There is a significant shift in the placement of the NonSF decomposed spectra, as well as the emergence of two distinct clusters in the LINER and Seyfert regions. This shift highlights the key role of SF `contamination' in influencing the positioning of NonSF spectra within the BPT diagram., Comment: 20 pages, 15 figures. Accepted for publication in APJ

Unsupervised video object segmentation (VOS), also known as video salient object detection, aims to detect the most prominent object in a video at the pixel level. Recently, two-stream approaches that leverage both RGB images and optical flow maps have gained significant attention. However, the limited amount of training data remains a substantial challenge. In this study, we propose a novel data generation method that simulates fake optical flows from single images, thereby creating large-scale training data for stable network learning. Inspired by the observation that optical flow maps are highly dependent on depth maps, we generate fake optical flows by refining and augmenting the estimated depth maps of each image. By incorporating our simulated image-flow pairs, we achieve new state-of-the-art performance on all public benchmark datasets without relying on complex modules. We believe that our data generation method represents a potential breakthrough for future VOS research.

We present multiwavelength polarization measurements of the luminous blazar Mrk~501 over a 14-month period. The 2--8 keV X-ray polarization was measured with the Imaging X-ray Polarimetry Explorer (IXPE) with six 100-ks observations spanning from 2022 March to 2023 April. Each IXPE observation was accompanied by simultaneous X-ray data from NuSTAR, Swift/XRT, and/or XMM-Newton. Complementary optical-infrared polarization measurements were also available in the B, V, R, I, and J bands, as were radio polarization measurements from 4.85 GHz to 225.5 GHz. Among the first five IXPE observations, we did not find significant variability in the X-ray polarization degree and angle with IXPE. However, the most recent sixth observation found an elevated polarization degree at $>3\sigma$ above the average of the other five observations. The optical and radio measurements show no apparent correlations with the X-ray polarization properties. Throughout the six IXPE observations, the X-ray polarization degree remained higher than, or similar to, the R-band optical polarization degree, which remained higher than the radio value. This is consistent with the energy-stratified shock scenario proposed to explain the first two IXPE observations, in which the polarized X-ray, optical, and radio emission arises from different regions., Comment: Accepted for publication in ApJ

In real-world scenarios, tabular data often suffer from distribution shifts that threaten the performance of machine learning models. Despite its prevalence and importance, handling distribution shifts in the tabular domain remains underexplored due to the inherent challenges within the tabular data itself. In this sense, test-time adaptation (TTA) offers a promising solution by adapting models to target data without accessing source data, crucial for privacy-sensitive tabular domains. However, existing TTA methods either 1) overlook the nature of tabular distribution shifts, often involving label distribution shifts, or 2) impose architectural constraints on the model, leading to a lack of applicability. To this end, we propose AdapTable, a novel TTA framework for tabular data. AdapTable operates in two stages: 1) calibrating model predictions using a shift-aware uncertainty calibrator, and 2) adjusting these predictions to match the target label distribution with a label distribution handler. We validate the effectiveness of AdapTable through theoretical analysis and extensive experiments on various distribution shift scenarios. Our results demonstrate AdapTable's ability to handle various real-world distribution shifts, achieving up to a 16% improvement on the HELOC dataset., Comment: Under Review at AAAI 2025

Deepfake detection is critical in mitigating the societal threats posed by manipulated videos. While various algorithms have been developed for this purpose, challenges arise when detectors operate externally, such as on smartphones, when users take a photo of deepfake images and upload on the Internet. One significant challenge in such scenarios is the presence of Moir\'e patterns, which degrade image quality and confound conventional classification algorithms, including deep neural networks (DNNs). The impact of Moir\'e patterns remains largely unexplored for deepfake detectors. In this study, we investigate how camera-captured deepfake videos from digital screens affect detector performance. We conducted experiments using two prominent datasets, CelebDF and FF++, comparing the performance of four state-of-the-art detectors on camera-captured deepfake videos with introduced Moir\'e patterns. Our findings reveal a significant decline in detector accuracy, with none achieving above 68% on average. This underscores the critical need to address Moir\'e pattern challenges in real-world deepfake detection scenarios., Comment: 7 page, 4 figures, 1 table, Accepted for publication in IEEE International Conference on Image Processing (ICIP 2024)

The fabrication of visual misinformation on the web and social media has increased exponentially with the advent of foundational text-to-image diffusion models. Namely, Stable Diffusion inpainters allow the synthesis of maliciously inpainted images of personal and private figures, and copyrighted contents, also known as deepfakes. To combat such generations, a disruption framework, namely Photoguard, has been proposed, where it adds adversarial noise to the context image to disrupt their inpainting synthesis. While their framework suggested a diffusion-friendly approach, the disruption is not sufficiently strong and it requires a significant amount of GPU and time to immunize the context image. In our work, we re-examine both the minimal and favorable conditions for a successful inpainting disruption, proposing DDD, a "Digression guided Diffusion Disruption" framework. First, we identify the most adversarially vulnerable diffusion timestep range with respect to the hidden space. Within this scope of noised manifold, we pose the problem as a semantic digression optimization. We maximize the distance between the inpainting instance's hidden states and a semantic-aware hidden state centroid, calibrated both by Monte Carlo sampling of hidden states and a discretely projected optimization in the token space. Effectively, our approach achieves stronger disruption and a higher success rate than Photoguard while lowering the GPU memory requirement, and speeding the optimization up to three times faster., Comment: 16 pages, 13 figures, IJCAI 2024

In this paper, we introduce a method to tackle Domain Generalized Semantic Segmentation (DGSS) by utilizing domain-invariant semantic knowledge from text embeddings of vision-language models. We employ the text embeddings as object queries within a transformer-based segmentation framework (textual object queries). These queries are regarded as a domain-invariant basis for pixel grouping in DGSS. To leverage the power of textual object queries, we introduce a novel framework named the textual query-driven mask transformer (tqdm). Our tqdm aims to (1) generate textual object queries that maximally encode domain-invariant semantics and (2) enhance the semantic clarity of dense visual features. Additionally, we suggest three regularization losses to improve the efficacy of tqdm by aligning between visual and textual features. By utilizing our method, the model can comprehend inherent semantic information for classes of interest, enabling it to generalize to extreme domains (e.g., sketch style). Our tqdm achieves 68.9 mIoU on GTA5$\rightarrow$Cityscapes, outperforming the prior state-of-the-art method by 2.5 mIoU. The project page is available at https://byeonghyunpak.github.io/tqdm., Comment: ECCV 2024

As Deep Neural Networks (DNNs) rapidly advance in various fields, including speech verification, they typically involve high computational costs and substantial memory consumption, which can be challenging to manage on mobile systems. Quantization of deep models offers a means to reduce both computational and memory expenses. Our research proposes an optimization framework for the quantization of the speaker verification model. By analyzing performance changes and model size reductions in each layer of a pre-trained speaker verification model, we have effectively minimized performance degradation while significantly reducing the model size. Our quantization algorithm is the first attempt to maintain the performance of the state-of-the-art pre-trained speaker verification model, ECAPATDNN, while significantly compressing its model size. Overall, our quantization approach resulted in reducing the model size by half, with an increase in EER limited to 0.07%., Comment: in Korean language, Accepted at Society of Electronic Engineers of Korea Conference 2024

Understanding the motion states of the surrounding environment is critical for safe autonomous driving. These motion states can be accurately derived from scene flow, which captures the three-dimensional motion field of points. Existing LiDAR scene flow methods extract spatial features from each point cloud and then fuse them channel-wise, resulting in the implicit extraction of spatio-temporal features. Furthermore, they utilize 2D Bird's Eye View and process only two frames, missing crucial spatial information along the Z-axis and the broader temporal context, leading to suboptimal performance. To address these limitations, we propose Flow4D, which temporally fuses multiple point clouds after the 3D intra-voxel feature encoder, enabling more explicit extraction of spatio-temporal features through a 4D voxel network. However, while using 4D convolution improves performance, it significantly increases the computational load. For further efficiency, we introduce the Spatio-Temporal Decomposition Block (STDB), which combines 3D and 1D convolutions instead of using heavy 4D convolution. In addition, Flow4D further improves performance by using five frames to take advantage of richer temporal information. As a result, the proposed method achieves a 45.9% higher performance compared to the state-of-the-art while running in real-time, and won 1st place in the 2024 Argoverse 2 Scene Flow Challenge. The code is available at https://github.com/dgist-cvlab/Flow4D., Comment: 8 pages, 4 figures

The gamma-ray emitting narrow-line Seyfert 1 galaxies are a unique class of objects that launch powerful jets from relatively lower-mass black hole systems compared to the Blazars. However, the black hole masses estimated from the total flux spectrum suffer from the projection effect, making the mass measurement highly uncertain. The polarized spectrum provides a unique view of the central engine through scattered light. We performed spectro-polarimetric observations of the gamma-ray emitting narrow-line Seyfert 1 galaxy 1H0323+342 using SPOL/MMT. The degree of polarization and polarization angle is 0.122 $\pm$ 0.040 % and 142 $\pm$ 9 degrees, while the H$\alpha$ line is polarized at 0.265 $\pm$ 0.280 %. We decomposed the total flux spectrum and estimated broad H$\alpha$ FWHM of 1015 km/s. The polarized flux spectrum shows a broadening similar to the total flux spectrum, with a broadening ratio of 1.22. The Monte Carlo radiative transfer code `STOKES' applied to the data provides the best fit for a small viewing angle of 9-24 degrees and a small optical depth ratio between the polar and the equatorial scatters. A thick BLR with significant scale height can explain a similar broadening of the polarized spectrum compared to the total flux spectrum with a small viewing angle., Comment: Accepted for publication in MNRAS (Jul. 08th, 2024). 12 pages, 6 figures

AMoRE searches for the signature of neutrinoless double beta decay of $^{100}$Mo with a 100 kg sample of enriched $^{100}$Mo. Scintillating molybdate crystals coupled with a metallic magnetic calorimeter operate at milli-Kelvin temperatures to measure the energy of electrons emitted in the decay. As a demonstration of the full-scale AMoRE, we conducted AMoRE-I, a pre-experiment with 18 molybdate crystals, at the Yangyang Underground Laboratory for over two years. The exposure was 8.02 kg$\cdot$year (or 3.89 kg$_{\mathrm{^{100}Mo}}\cdot$year) and the total background rate near the Q-value was 0.025 $\pm$ 0.002 counts/keV/kg/year. We observed no indication of $0\nu\beta\beta$ decay and report a new lower limit of the half-life of $^{100}$Mo $0\nu\beta\beta$ decay as $ T^{0\nu}_{1/2}>3.0\times10^{24}~\mathrm{years}$ at 90\% confidence level. The effective Majorana mass limit range is $m_{\beta\beta}<$(210--610) meV using nuclear matrix elements estimated in the framework of different models, including the recent shell model calculations., Comment: 8 pages, 5 figures

This paper introduces the Consumer Feedback Insight & Prediction Platform, a system leveraging machine learning to analyze the extensive Consumer Financial Protection Bureau (CFPB) Complaint Database, a publicly available resource exceeding 4.9 GB in size. This rich dataset offers valuable insights into consumer experiences with financial products and services. The platform itself utilizes machine learning models to predict two key aspects of complaint resolution: the timeliness of company responses and the nature of those responses (e.g., closed, closed with relief etc.). Furthermore, the platform employs Latent Dirichlet Allocation (LDA) to delve deeper, uncovering common themes within complaints and revealing underlying trends and consumer issues. This comprehensive approach empowers both consumers and regulators. Consumers gain valuable insights into potential response wait times, while regulators can utilize the platform's findings to identify areas where companies may require further scrutiny regarding their complaint resolution practices., Comment: 4 pages, 3 figures, 4 tables

While significant advancements have been made in music generation and differentiable sound synthesis within machine learning and computer audition, the simulation of instrument vibration guided by physical laws has been underexplored. To address this gap, we introduce a novel model for simulating the spatio-temporal motion of nonlinear strings, integrating modal synthesis and spectral modeling within a neural network framework. Our model leverages physical properties and fundamental frequencies as inputs, outputting string states across time and space that solve the partial differential equation characterizing the nonlinear string. Empirical evaluations demonstrate that the proposed architecture achieves superior accuracy in string motion simulation compared to existing baseline architectures. The code and demo are available online., Comment: 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

We investigate the properties of the recently observed "demon" mode, a 3D acoustic plasmon, in Sr$_2$RuO$_4$ with an emphasis on evaluating its role for the pairing interactions in this superconductor. The demon mode is a low-energy electronic excitation, and it has been suggested that it could contribute to a reduced Coulomb repulsion and even a possible attractive interaction between electrons. In this study, we explicitly calculate the dynamically screened Coulomb interaction for Sr$_2$RuO$_4$ by using a renormalized tight-binding band structure and the random phase approximation for the dielectric function. Although the focus here is on Sr$_2$RuO$_4$, this material is considered mainly as a prototype system, having an observed demon mode, and our results should be considered as a guide for application to other systems. Our calculations show that there are regions in ($\mathbf{q}$, $\omega$) space where the Coulomb interaction becomes attractive. We find that, although the demon mode is not capable of producing a total attractive electron pairing interaction in Sr$_2$RuO$_4$, it does contribute to a significant reduction in the Coulomb repulsion at the relevant pairing energy scale.

The coupled transport of the charge and orbital angular momentum of electrons is at the heart of orbitronics. Here, we discuss the reciprocal relation between the direct and inverse orbital Hall effects (OHEs) in thin films. We argue that the conventional orbital current is ill-defined as it does not satisfy the reciprocal relation owing to non-conservation of the orbital angular momentum. We resolve the problem by adopting the definition of the so-called \emph{proper} orbital current, which is directly related to orbital accumulation. We prove the reciprocal relation between the \emph{global} response of orbital and charge currents. However, we show that their \emph{local} distributions are generally different, especially due to gigantic contributions at surfaces, which may lead to unintuitive results when charge and orbital currents are locally measured. We demonstrate our predictions by first-principles calculations on W(110) and Pt(111) thin films. In W(110), the direct and inverse OHEs are severely non-reciprocal locally in each layer although the total responses are exactly reciprocal. Interestingly, the SHEs are almost reciprocal locally in each layer. On the other hand, in Pt(111), both OHEs and SHEs are locally non-reciprocal, which we attribute to the pronounced spin-orbit interaction. We propose that the locally distinct responses may be used to distinguish the spin and orbital currents in experiments.

In this paper, we propose a control algorithm based on reinforcement learning, employing independent rewards for each joint to control excavators in a 3D space. The aim of this research is to address the challenges associated with achieving precise control of excavators, which are extensively utilized in construction sites but prove challenging to control with precision due to their hydraulic structures. Traditional methods relied on operator expertise for precise excavator operation, occasionally resulting in safety accidents. Therefore, there have been endeavors to attain precise excavator control through equation-based control algorithms. However, these methods had the limitation of necessitating prior information related to physical values of the excavator, rendering them unsuitable for the diverse range of excavators used in the field. To overcome these limitations, we have explored reinforcement learning-based control methods that do not demand prior knowledge of specific equipment but instead utilize data to train models. Nevertheless, existing reinforcement learning-based methods overlooked cabin swing rotation and confined the bucket's workspace to a 2D plane. Control confined within such a limited area diminishes the applicability of the algorithm in construction sites. We address this issue by expanding the previous 2D plane workspace of the bucket operation into a 3D space, incorporating cabin swing rotation. By expanding the workspace into 3D, excavators can execute continuous operations without requiring human intervention. To accomplish this objective, distinct targets were established for each joint, facilitating the training of action values for each joint independently, regardless of the progress of other joint learning.

In the early universe, strong large scale random flows prevail, and when these flows collide it produces sheets and filaments for the conventional particle dark matter model. However, for wave dark matter, the sheets and filaments are parallelly modulated by fine-scale and large-amplitude interference fringes. These long fine fringes tend to be unstable initially and subsequently deform and collapse to form halos of various sizes, perhaps down to proto-globular cluster scales. This work aims to investigate the instability of such fringes. Two colliding streams of quantum waves result in a standing density wave. By contrast, two colliding beams in classical collisionless fluid systems often give rise to beam instability, characterized by unstable oscillations with a complex frequency. Will the quantum standing density wave be subject to a similar instability? In fact, such unstable oscillations have never been observed in conventional quantum mechanical systems. This study discovers that periodic fringes can indeed undergo unstable oscillations or beam instability. The presence of any local density null in the background density that fails the Madelung formulation is shown to be the necessary condition for unstable oscillations. We establish a phase diagram to distinguish between beam instability, Jeans instability, and stable oscillation through theoretical calculations and computer simulations. Roughly speaking, Jeans instability almost always occurs for long-wave density perturbations; beam instability on the other hand always occurs when the density wave is aligned with the beam direction and moreover the period of the density wave is twice the period of background density; lastly, both instabilities diminish for a very low background density. We have engaged in Blocks theorem extensively in the analysis., Comment: 34 pages, 4 figures, 1 table

Neural networks can learn spurious correlations that lead to the correct prediction in a validation set, but generalise poorly because the predictions are right for the wrong reason. This undesired learning of naive shortcuts (Clever Hans effect) can happen for example in echocardiogram view classification when background cues (e.g. metadata) are biased towards a class and the model learns to focus on those background features instead of on the image content. We propose a simple, yet effective random background augmentation method called BackMix, which samples random backgrounds from other examples in the training set. By enforcing the background to be uncorrelated with the outcome, the model learns to focus on the data within the ultrasound sector and becomes invariant to the regions outside this. We extend our method in a semi-supervised setting, finding that the positive effects of BackMix are maintained with as few as 5% of segmentation labels. A loss weighting mechanism, wBackMix, is also proposed to increase the contribution of the augmented examples. We validate our method on both in-distribution and out-of-distribution datasets, demonstrating significant improvements in classification accuracy, region focus and generalisability. Our source code is available at: https://github.com/kitbransby/BackMix, Comment: Accepted at MICCAI 2024 (Pre-print)

Real-time monitoring of dynamic biological processes in the body is critical to understanding disease progression and treatment response. This data, for instance, can help address the lower than 50% response rates to cancer immunotherapy. However, current clinical imaging modalities lack the molecular contrast, resolution, and chronic usability for rapid and accurate response assessments. Here, we present a fully wireless image sensor featuring a 2.5$\times$5 mm$^2$ CMOS integrated circuit for multicolor fluorescence imaging deep in tissue. The sensor operates wirelessly via ultrasound (US) at 5 cm depth in oil, harvesting energy with 221 mW/cm$^{2}$ incident US power density (31% of FDA limits) and backscattering data at 13 kbps with a bit error rate <$10^{-6}$. In-situ fluorescence excitation is provided by micro-laser diodes controlled with a programmable on-chip driver. An optical frontend combining a multi-bandpass interference filter and a fiber optic plate provides >6 OD excitation blocking and enables three-color imaging for detecting multiple cell types. A 36$\times$40-pixel array captures images with <125 $\mu$m resolution. We demonstrate wireless, dual-color fluorescence imaging of both effector and suppressor immune cells in ex vivo mouse tumor samples with and without immunotherapy. These results show promise for providing rapid insight into therapeutic response and resistance, guiding personalized medicine., Comment: *equally contributing authors

We present a novel speaker-independent acoustic-to-articulatory inversion (AAI) model, overcoming the limitations observed in conventional AAI models that rely on acoustic features derived from restricted datasets. To address these challenges, we leverage representations from a pre-trained self-supervised learning (SSL) model to more effectively estimate the global, local, and kinematic pattern information in Electromagnetic Articulography (EMA) signals during the AAI process. We train our model using an adversarial approach and introduce an attention-based Multi-duration phoneme discriminator (MDPD) designed to fully capture the intricate relationship among multi-channel articulatory signals. Our method achieves a Pearson correlation coefficient of 0.847, marking state-of-the-art performance in speaker-independent AAI models. The implementation details and code can be found online., Comment: Accepted to INTERSPEECH 2024

In this paper, we address the problem of compact model parameter extraction to simultaneously extract tens of parameters via derivative-free optimization. Traditionally, parameter extraction is performed manually by dividing the complete set of parameters into smaller subsets, each targeting different operational regions of the device, a process that can take several days or weeks. Our approach streamlines this process by employing derivative-free optimization to identify a good parameter set that best fits the compact model without performing an exhaustive number of simulations. We further enhance the optimization process to address three critical issues in device modeling by carefully choosing a loss function that focuses on relative errors rather than absolute errors to ensure consistent performance across different orders of magnitude, prioritizes accuracy in key operational regions above a specific threshold, and reduces sensitivity to outliers. Furthermore, we utilize the concept of train-test split to assess the model fit and avoid overfitting. We demonstrate the effectiveness of our approach by successfully modeling a diamond Schottky diode with the SPICE diode model and a GaN-on-SiC HEMT with the ASM-HEMT model. For the latter, which involves extracting 35 parameters for the ASM-HEMT DC model, we identified the best set of parameters in under 6,000 trials. Additional examples using both devices are provided to demonstrate robustness to outliers, showing that an excellent fit is achieved even with over 25% of the data purposely corrupted. These examples demonstrate the practicality of our approach, highlighting the benefits of derivative-free optimization in device modeling.

We introduce Cambrian-1, a family of multimodal LLMs (MLLMs) designed with a vision-centric approach. While stronger language models can enhance multimodal capabilities, the design choices for vision components are often insufficiently explored and disconnected from visual representation learning research. This gap hinders accurate sensory grounding in real-world scenarios. Our study uses LLMs and visual instruction tuning as an interface to evaluate various visual representations, offering new insights into different models and architectures -- self-supervised, strongly supervised, or combinations thereof -- based on experiments with over 20 vision encoders. We critically examine existing MLLM benchmarks, address the difficulties involved in consolidating and interpreting results from various tasks, and introduce a new vision-centric benchmark, CV-Bench. To further improve visual grounding, we propose the Spatial Vision Aggregator (SVA), a dynamic and spatially-aware connector that integrates high-resolution vision features with LLMs while reducing the number of tokens. Additionally, we discuss the curation of high-quality visual instruction-tuning data from publicly available sources, emphasizing the importance of data source balancing and distribution ratio. Collectively, Cambrian-1 not only achieves state-of-the-art performance but also serves as a comprehensive, open cookbook for instruction-tuned MLLMs. We provide model weights, code, supporting tools, datasets, and detailed instruction-tuning and evaluation recipes. We hope our release will inspire and accelerate advancements in multimodal systems and visual representation learning., Comment: NeurIPS 2024 (Oral). Website at https://cambrian-mllm.github.io

We explore the evolution of the ~107 degree hot gas in normal galaxies out to redshift = 0.5 (lookback time = 5 Gyr), using X-ray luminosity functions (XLF) built from a sample of 575 normal galaxies with z < 0.6 detected in five high galactic latitude Chandra wide-field surveys. After estimating the emission due to the hot gas component (reducing the sample to ~400 galaxies), we compared the XLF in three redshift bins (z = 0.1, 0.3, and 0.5), finding increases in the number of galaxies per unit co-moving volume from z = 0.1 to 0.3 and then from z = 0.3 to 0.5. These XLF changes suggest a significant (~5s) X-ray luminosity evolution of the hot gas, with LX,GAS decreasing by a factor of 6-10 in the last 5 Gyr (from z = 0.5 to 0.1). The relative abundance of LX,GAS~1041 erg s-1 galaxies at higher z, suggests that high z, moderate LX,GAS galaxies may be the optimal target to solve the missing baryon problem. In early-type galaxies, this observational trend is qualitatively consistent with (but larger than) the expected time-dependent mass-loss rate in cooling flow models without AGN feedback. In late-type galaxies, the observational trend is also qualitatively consistent with (but larger than) the effect of the z-dependent SFR., Comment: 29 pages, 14 figures, new Appendix added, accepted for publication in ApJ

In this paper, we study the homogeneous inelastic Boltzmann equation for hard spheres. We first prove that the solution $f(t,v)$ is bounded pointwise from above by \(C_{f_0}\langle t \rangle^3\) and establish that the cooling time is infinite (\( T_c = +\infty \)) under the condition \( f_0 \in L^1_2 \cap L^{\infty}_{s} \) for \( s > 2 \). Away from zero velocity, we further prove that $ f(t,v)\leq C_{f_0, |v|} \langle t \rangle $ for \(v \neq 0\) at any time \( t > 0 \). This time-dependent pointwise upper bound is natural in the cooling process, as we expect the density near \( v = 0 \) to grow rapidly. We also establish an upper bound that depends on the coefficient of normal restitution constant, $\alpha \in (0,1]$. This upper bound becomes constant when $\alpha = 1$, restoring the known upper bound for elastic collisions \cite{L1983}. Consequently, through these results, we obtain Maxwellian upper bounds on the solutions at each time., Comment: 33 pages, 4 figures, This version includes a new upper bound that also depends on the restitution coefficient $\alpha \in (0,1]$. It becomes constant when $\alpha=1$, restoring the known upper bound for elastic collisions

Multisequence Magnetic Resonance Imaging (MRI) provides a reliable diagnosis in clinical applications through complementary information within sequences. However, in practice, the absence of certain MR sequences is a common problem that can lead to inconsistent analysis results. In this work, we propose a novel unified framework for synthesizing multisequence MR images, called Hybrid Fusion GAN (HF-GAN). We introduce a hybrid fusion encoder designed to ensure the disentangled extraction of complementary and modality-specific information, along with a channel attention-based feature fusion module that integrates the features into a common latent space handling the complexity from combinations of accessible MR sequences. Common feature representations are transformed into a target latent space via the modality infuser to synthesize missing MR sequences. We have performed experiments on multisequence brain MRI datasets from healthy individuals and patients diagnosed with brain tumors. Experimental results show that our method outperforms state-of-the-art methods in both quantitative and qualitative comparisons. In addition, a detailed analysis of our framework demonstrates the superiority of our designed modules and their effectiveness for use in data imputation tasks., Comment: 11 pages, 7 figures

Recent advancements in Large Language Models (LLMs) have led to their adaptation in various domains as conversational agents. We wonder: can personality tests be applied to these agents to analyze their behavior, similar to humans? We introduce TRAIT, a new benchmark consisting of 8K multi-choice questions designed to assess the personality of LLMs. TRAIT is built on two psychometrically validated small human questionnaires, Big Five Inventory (BFI) and Short Dark Triad (SD-3), enhanced with the ATOMIC-10X knowledge graph to a variety of real-world scenarios. TRAIT also outperforms existing personality tests for LLMs in terms of reliability and validity, achieving the highest scores across four key metrics: Content Validity, Internal Validity, Refusal Rate, and Reliability. Using TRAIT, we reveal two notable insights into personalities of LLMs: 1) LLMs exhibit distinct and consistent personality, which is highly influenced by their training data (e.g., data used for alignment tuning), and 2) current prompting techniques have limited effectiveness in eliciting certain traits, such as high psychopathy or low conscientiousness, suggesting the need for further research in this direction., Comment: Preprint; Under review