Search

Rapid advancements in imaging techniques and analytical methods over the past decade have revolutionized our ability to comprehensively probe the biological world at multiple scales, pinpointing the type, quantity, location, and even temporal dynamics of biomolecules. The surge in data complexity and volume presents significant challenges in translating this wealth of information into knowledge. The recently emerged Multimodal Large Language Models (MLLMs) exhibit strong emergent capacities, such as understanding, analyzing, reasoning, and generalization. With these capabilities, MLLMs hold promise to extract intricate information from biological images and data obtained through various modalities, thereby expediting our biological understanding and aiding in the development of novel computational frameworks. Previously, such capabilities were mostly attributed to humans for interpreting and summarizing meaningful conclusions from comprehensive observations and analysis of biological images. However, the current development of MLLMs shows increasing promise in serving as intelligent assistants or agents for augmenting human researchers in biology research

The rapid pace of innovation in biological microscopy imaging has led to large images, putting pressure on data storage and impeding efficient sharing, management, and visualization. This necessitates the development of efficient compression solutions. Traditional CODEC methods struggle to adapt to the diverse bioimaging data and often suffer from sub-optimal compression. In this study, we propose an adaptive compression workflow based on Implicit Neural Representation (INR). This approach permits application-specific compression objectives, capable of compressing images of any shape and arbitrary pixel-wise decompression. We demonstrated on a wide range of microscopy images from real applications that our workflow not only achieved high, controllable compression ratios (e.g., 512x) but also preserved detailed information critical for downstream analysis.

Continual Test-Time Adaptation (CTTA), which aims to adapt the pre-trained model to ever-evolving target domains, emerges as an important task for vision models. As current vision models appear to be heavily biased towards texture, continuously adapting the model from one domain distribution to another can result in serious catastrophic forgetting. Drawing inspiration from the human visual system's adeptness at processing both shape and texture according to the famous Trichromatic Theory, we explore the integration of a Mixture-of-Activation-Sparsity-Experts (MoASE) as an adapter for the CTTA task. Given the distinct reaction of neurons with low/high activation to domain-specific/agnostic features, MoASE decomposes the neural activation into high-activation and low-activation components with a non-differentiable Spatial Differentiate Dropout (SDD). Based on the decomposition, we devise a multi-gate structure comprising a Domain-Aware Gate (DAG) that utilizes domain information to adaptive combine experts that process the post-SDD sparse activations of different strengths, and the Activation Sparsity Gate (ASG) that adaptively assigned feature selection threshold of the SDD for different experts for more precise feature decomposition. Finally, we introduce a Homeostatic-Proximal (HP) loss to bypass the error accumulation problem when continuously adapting the model. Extensive experiments on four prominent benchmarks substantiate that our methodology achieves state-of-the-art performance in both classification and segmentation CTTA tasks. Our code is now available at https://github.com/RoyZry98/MoASE-Pytorch.

One of the most critical factors in achieving sharp Novel View Synthesis (NVS) using neural field methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) is the quality of the training images. However, Conventional RGB cameras are susceptible to motion blur. In contrast, neuromorphic cameras like event and spike cameras inherently capture more comprehensive temporal information, which can provide a sharp representation of the scene as additional training data. Recent methods have explored the integration of event cameras to improve the quality of NVS. The event-RGB approaches have some limitations, such as high training costs and the inability to work effectively in the background. Instead, our study introduces a new method that uses the spike camera to overcome these limitations. By considering texture reconstruction from spike streams as ground truth, we design the Texture from Spike (TfS) loss. Since the spike camera relies on temporal integration instead of temporal differentiation used by event cameras, our proposed TfS loss maintains manageable training costs. It handles foreground objects with backgrounds simultaneously. We also provide a real-world dataset captured with our spike-RGB camera system to facilitate future research endeavors. We conduct extensive experiments using synthetic and real-world datasets to demonstrate that our design can enhance novel view synthesis across NeRF and 3DGS. The code and dataset will be made available for public access.

Self-supervised learning has emerged as a method for utilizing massive unlabeled data for pre-training models, providing an effective feature extractor for various mobile sensing applications. However, when deployed to end-users, these models encounter significant domain shifts attributed to user diversity. We investigate the performance degradation that occurs when self-supervised models are fine-tuned in heterogeneous domains. To address the issue, we propose ADAPT^2, a few-shot domain adaptation framework for personalizing self-supervised models. ADAPT2 proposes self-supervised meta-learning for initial model pre-training, followed by a user-side model adaptation by replaying the self-supervision with user-specific data. This allows models to adjust their pre-trained representations to the user with only a few samples. Evaluation with four benchmarks demonstrates that ADAPT^2 outperforms existing baselines by an average F1-score of 8.8%p. Our on-device computational overhead analysis on a commodity off-the-shelf (COTS) smartphone shows that ADAPT2 completes adaptation within an unobtrusive latency (in three minutes) with only a 9.54% memory consumption, demonstrating the computational efficiency of the proposed method.

Geometric phases in particle diffusion systems, an intriguing aspect enlightened from thermal systems, offer a different understanding beyond traditional Brownian motion and Fick's laws. This concept introduces a phase factor with significant implications for particle behavior, central to which is the non-Hermitian nature of the Hamiltonian in the diffusion system. A unique structure composed of two rings moving in opposite directions and a stationary intermediate layer plays multifunctional roles in controlling particle diffusion. For the real-world application, a bilayer particle-diffusion cloak is demonstrated, showcasing the extensive control and adaptability achievable through mastering geometric phases. This system has potential applications in industries like healthcare and environmental management, thus expanding the understanding of the geometric phase and offering insights for the design of particle-diffusion metamaterials., Comment: 14 pages, 4 figures. This review article explores the Non Hermitian topological theory in regulating particle diffusion, a concept prominently evidenced by the landmark thermal study Anti parity time Symmetry in Diffusive Systems published in Science, 2019. The review is slated for publication in the forthcoming book, Diffusionics, Diffusion Processes Governed by Diffusion Metamaterials

Thermal convection stands out as an exceptionally efficient thermal transport mechanism, distinctly separate from conduction and radiation. Yet, the inherently elusive nature of fluid motion poses challenges in accurately controlling convective heat flow. While recent innovations have harnessed thermal convection to achieve effective thermal conductivity, fusing thermal convection in liquids and thermal conduction in solids together to form hybrid thermal metamaterials is still challenging. In this review, we introduce the latest progress in convective heat transfer. Leveraging the right porous materials as a medium allows for a harmonious balance and synergy between convection and conduction, establishing stable heat and fluid flows. This paves the way for the innovative advancements in transformation thermotics. These findings demonstrate the remarkable tunability of convective heat transport in complex multicomponent thermal metamaterials.

Learned optimizers are a crucial component of meta-learning. Recent advancements in scalable learned optimizers have demonstrated their superior performance over hand-designed optimizers in various tasks. However, certain characteristics of these models, such as an unstable learning curve, limited ability to handle unseen tasks and network architectures, difficult-to-control behaviours, and poor performance in fine-tuning tasks impede their widespread adoption. To tackle the issue of generalization in scalable learned optimizers, we propose a hybrid-update-based (HUB) optimization strategy inspired by recent advancements in hard prompt tuning and result selection techniques used in large language and vision models. This approach can be easily applied to any task that involves hand-designed or learned optimizer. By incorporating hand-designed optimizers as the second component in our hybrid approach, we are able to retain the benefits of learned optimizers while stabilizing the training process and, more importantly, improving testing performance. We validate our design through a total of 17 tasks, consisting of thirteen training from scratch and four fine-tuning settings. These tasks vary in model sizes, architectures, or dataset sizes, and the competing optimizers are hyperparameter-tuned. We outperform all competitors in 94% of the tasks with better testing performance. Furthermore, we conduct a theoretical analysis to examine the potential impact of our hybrid strategy on the behaviours and inherited traits of learned optimizers., Comment: Some table information is not accurate, author information not correct inside the pdf

Transformation media provide a fundamental paradigm for field regulation, but their tricky anisotropy challenges fabrication. Though optical conformal mapping has been utilized to eliminate anisotropy, two key factors still hinder its development in thermotics, i.e., the distinct diffusion nature and inevitable interface mismatching. Here, we put forth the concept of diffusive pseudo-conformal mapping, overcoming the inherent difference between diffusion and waves and achieving perfect interface matching. The proposed mapping directly leads to heat guiding and expanding functions with anisotropy-free transformation thermal media, whose feasibility is confirmed by experiments or simulations. Besides diverse applications, we provide a unified perspective for two distinct types of prevailing bilayer cloaks by uncovering their profound ties with pseudo-conformal mapping. These results greatly simplify the preparation of transformation thermotics and have implications for regulating other diffusion and wave phenomena.

The method of transformation optics has been a powerful tool to manipulate physical fields if governing equations are formally invariant under coordinate transformations. However, regulation of hydrodynamics is still far from satisfactory due to the lack of rigorous arguments on the validation of transformation theory for various categories of fluids. In this paper, we systematically investigate the applicability of transformation optics to fluid mechanics. We find that the Stokes equation and the Navier-Stokes equations, respectively describing the Stokes flow and general flow, will alter their forms under curvilinear transformations. On the contrary, the Hele-Shaw flow characterized with shallow geometries rigidly retain the form of its governing equation under arbitrary transformations. Based on the derived transformation rules, we propose the design of multilayered structures with spatially varying cell depth, instead of engineering the rank-2 shear viscosity tensor, to realize the required anisotropy of transformation Hele-Shaw hydrodynamic metamaterials. The theoretical certify and fabrication method revealed in this work may pave an avenue for precisely controlling flow distribution with the concept of artificial structure design.

Nonlinear heat transfer can be exploited to reveal novel transport phenomena and thus enhance peo-ple's ability to manipulate heat flux at will. However, there hasn't been a mature discipline called nonlinear thermotics like its counterpart in optics or acoustics to make a systematic summary of rele-vant researches. In the current review, we focus on recent progress in an important part of nonlinear heat transfer, i.e., tailoring nonlinear thermal devices and metamaterials under the Fourier's law, especially with temperature-dependent thermal conductivities. We will present the basic designing techniques including solving the equation directly and the transformation theory. Tuning nonlinearity coming from multi-physical effects, and how to calculate effective properties of nonlinear conductive composites using the effective medium theory are also included. Based on these theories, researchers have successfully designed various functional materials and devices such as the thermal diodes, thermal transistors, thermal memory elements, energy-free thermostats, and intelligent thermal materials, and some of them have also been realized in experiments. Further, these phenomenological works can provide a feasible route for the development of nonlinear thermotics.

Heat can generally transfer via thermal conduction, thermal radiation, and thermal convection. All the existing theories of transformation thermotics and optics can treat thermal conduction and thermal radiation, respectively. Unfortunately, thermal convection has never been touched in transformation theories due to the lack of a suitable theory, thus limiting applications associated with heat transfer through fluids (liquid or gas). Here, we develop, for the first time, a general theory of transformation thermal convection by considering the convection-diffusion equation, the Navier-Stokes equation, and the Darcy law. By introducing porous media, we get a set of coupled equations keeping their forms under coordinate transformation. As model applications, the theory helps to show the effects of cloaking, concentrating, and camouflage. Our finite element simulations confirm the theoretical findings. This work offers a general transformation theory for thermal convection, thus revealing some novel behaviors of thermal convection; it not only provides new hints on how to control heat transfer by combining thermal conduction, thermal radiation, and thermal convection, but also benefits the study of mass diffusion and other related fields that contain a set of equations and need to transform velocities at the same time., Comment: 17 pages, 6 figures