Search

Multi-modal large language models (MLLMs) have achieved remarkable success in fine-grained visual understanding across a range of tasks. However, they often encounter significant challenges due to inadequate alignment for fine-grained knowledge, which restricts their ability to accurately capture local details and attain a comprehensive global perception. While recent advancements have focused on aligning object expressions with grounding information, they typically lack explicit integration of object images, which contain affluent information beyond mere texts or coordinates. To bridge this gap, we introduce a novel fine-grained visual knowledge alignment method that effectively aligns and integrates multi-scale knowledge of objects, including texts, coordinates, and images. This innovative method is underpinned by our multi-scale fine-grained enhancement data synthesis pipeline, which provides over 300K essential training data to enhance alignment and improve overall performance. Furthermore, we present TinyGroundingGPT, a series of compact models optimized for high-level alignments. With a scale of approximately 3B parameters, TinyGroundingGPT achieves outstanding results in grounding tasks while delivering performance comparable to larger MLLMs in complex visual scenarios.

We provide an observability inequality in terms of a measurable set for general Gevrey regular functions. As an application, we establish an observability estimate from a measurable set for sums of Laplace eigenfunctions in a compact and connected boundaryless Riemannian manifold that belongs to the Gevrey class. The estimate has an explicit dependence on the maximal eigenvalue., Comment: 10 pages

From serving a cup of coffee to carefully rearranging delicate items, stable object placement is a crucial skill for future robots. This skill is challenging due to the required accuracy, which is difficult to achieve under geometric uncertainty. We leverage differentiable contact dynamics to develop a principled method for stable object placement under geometric uncertainty. We estimate the geometric uncertainty by minimizing the discrepancy between the force-torque sensor readings and the model predictions through gradient descent. We further keep track of a belief over multiple possible geometric parameters to mitigate the gradient-based method's sensitivity to the initialization. We verify our approach in the real world on various geometric uncertainties, including the in-hand pose uncertainty of the grasped object, the object's shape uncertainty, and the environment's shape uncertainty.

We consider the size of the nodal set of the solution of the second order parabolic-type equation with Gevrey regular coefficients. We provide an upper bound as a function of time. The dependence agrees with a sharp upper bound when the coefficients are analytic.

This paper reports the achievement of 120W single-frequency narrow linewidth 1018nm laser based on wide-tunable DBR fiber seed source. The DBR structure seed source uses 8mm long doped optical fibers with a line width of 3.25k. The wavelength tuning range of this seed source exceeds 1.5 nm with the temperature range from 1{\deg}C to 95{\deg}C. The tuning wavelength and temperature show extremely high linearity, and there is no mode hopping during the tuning process. By adopting a multi-level fiber amplification structure, selecting appropriate doped fibers and optimizing their length, an output power exceeding 120W of 1018nm laser has been achieved. Measurement results indicate that the slope efficiency of the main amplification 77.3%, with an amplified spontaneous emission (ASE) suppression ratio greater than 60 dB. he output linewidth is 10.3 kHz, and the beam quality factor M2 is less than 1.3., Comment: 13 pages, 11 figures

Deformable object manipulation is a long-standing challenge in robotics. While existing approaches often focus narrowly on a specific type of object, we seek a general-purpose algorithm, capable of manipulating many different types of objects: beans, rope, cloth, liquid, . . . . One key difficulty is a suitable representation, rich enough to capture object shape, dynamics for manipulation and yet simple enough to be acquired effectively from sensor data. Specifically, we propose Differentiable Particles (DiPac), a new algorithm for deformable object manipulation. DiPac represents a deformable object as a set of particles and uses a differentiable particle dynamics simulator to reason about robot manipulation. To find the best manipulation action, DiPac combines learning, planning, and trajectory optimization through differentiable trajectory tree optimization. Differentiable dynamics provides significant benefits and enable DiPac to (i) estimate the dynamics parameters efficiently, thereby narrowing the sim-to-real gap, and (ii) choose the best action by backpropagating the gradient along sampled trajectories. Both simulation and real-robot experiments show promising results. DiPac handles a variety of object types. By combining planning and learning, DiPac outperforms both pure model-based planning methods and pure data-driven learning methods. In addition, DiPac is robust and adapts to changes in dynamics, thereby enabling the transfer of an expert policy from one object to another with different physical properties, e.g., from a rigid rod to a deformable rope.

We address a system of equations modeling an incompressible fluid interacting with an elastic body. We prove the local existence when the initial velocity belongs to the space $H^{1.5+\epsilon}$ and the initial structure velocity is in $H^{1+\epsilon}$, where $\epsilon \in (0, 1/20)$., Comment: 22 pages. arXiv admin note: substantial text overlap with arXiv:2104.13500

Existing salient object detection methods are capable of predicting binary maps that highlight visually salient regions. However, these methods are limited in their ability to differentiate the relative importance of multiple objects and the relationships among them, which can lead to errors and reduced accuracy in downstream tasks that depend on the relative importance of multiple objects. To conquer, this paper proposes a new paradigm for saliency ranking, which aims to completely focus on ranking salient objects by their "importance order". While previous works have shown promising performance, they still face ill-posed problems. First, the saliency ranking ground truth (GT) orders generation methods are unreasonable since determining the correct ranking order is not well-defined, resulting in false alarms. Second, training a ranking model remains challenging because most saliency ranking methods follow the multi-task paradigm, leading to conflicts and trade-offs among different tasks. Third, existing regression-based saliency ranking methods are complex for saliency ranking models due to their reliance on instance mask-based saliency ranking orders. These methods require a significant amount of data to perform accurately and can be challenging to implement effectively. To solve these problems, this paper conducts an in-depth analysis of the causes and proposes a whole-flow processing paradigm of saliency ranking task from the perspective of "GT data generation", "network structure design" and "training protocol". The proposed approach outperforms existing state-of-the-art methods on the widely-used SALICON set, as demonstrated by extensive experiments with fair and reasonable comparisons. The saliency ranking task is still in its infancy, and our proposed unified framework can serve as a fundamental strategy to guide future work., Comment: 16 pages, 14 figures, accepted by IEEE Transactions on Image Processing

Knowledge and skills can transfer from human teachers to human students. However, such direct transfer is often not scalable for physical tasks, as they require one-to-one interaction, and human teachers are not available in sufficient numbers. Machine learning enables robots to become experts and play the role of teachers to help in this situation. In this work, we formalize cooperative robot teaching as a Markov game, consisting of four key elements: the target task, the student model, the teacher model, and the interactive teaching-learning process. Under a moderate assumption, the Markov game reduces to a partially observable Markov decision process, with an efficient approximate solution. We illustrate our approach on two cooperative tasks, one in a simulated video game and one with a real robot., Comment: CoRL 2022

We derive a priori estimates for the compressible free boundary Euler equations in the case of a liquid without surface tension. We provide a new weighted functional framework which leads to the improved regularity of the flow map by using the Hardy inequality. One of main ideas is to decompose the initial density function. It is worth mentioning that in our analysis we do not need the higher order wave equation for the density., Comment: 40 pages

Deformable Object Manipulation (DOM) is of significant importance to both daily and industrial applications. Recent successes in differentiable physics simulators allow learning algorithms to train a policy with analytic gradients through environment dynamics, which significantly facilitates the development of DOM algorithms. However, existing DOM benchmarks are either single-object-based or non-differentiable. This leaves the questions of 1) how a task-specific algorithm performs on other tasks and 2) how a differentiable-physics-based algorithm compares with the non-differentiable ones in general. In this work, we present DaXBench, a differentiable DOM benchmark with a wide object and task coverage. DaXBench includes 9 challenging high-fidelity simulated tasks, covering rope, cloth, and liquid manipulation with various difficulty levels. To better understand the performance of general algorithms on different DOM tasks, we conduct comprehensive experiments over representative DOM methods, ranging from planning to imitation learning and reinforcement learning. In addition, we provide careful empirical studies of existing decision-making algorithms based on differentiable physics, and discuss their limitations, as well as potential future directions., Comment: ICLR 2023 (Oral)

We systematically study how the integrality of the conformal characters shapes the space of fermionic rational conformal field theories in two dimensions. The integrality suggests that conformal characters on torus with a given choice of spin structures should be invariant under a principal congruence subgroup of $\mathrm{PSL}(2,\mathbb{Z})$. The invariance strongly constrains the possible values of the central charge as well as the conformal weights in both Neveu-Schwarz and Ramond sectors, which improves the conventional holomorphic modular bootstrap method in a significant manner. This allows us to make much progress on the classification of fermionic rational conformal field theories with the number of independent characters less than five., Comment: 36 pages, 1 figure; minor changes, published version

Assuming only Sobolev regularity of the domain, we prove time-analyticity of Lagrangian trajectories for solutions of the Euler equations., Comment: The equation (3.19) may have some lower-order terms

We address a system of equations modeling a compressible fluid interacting with an elastic body in dimension three. We prove the local existence and uniqueness of a strong solution when the initial velocity belongs to the space $H^{2+\epsilon}$ and the initial structure velocity is in $H^{1.5+\epsilon}$ , where $\epsilon \in (0, 1/2)$., Comment: 30 pages

We consider the low Mach number limit problem of the Euler equations for isentropic fluids in the analytic spaces. We prove that, given general analytic initial data, the solution is uniformly bounded on a time interval independent of the small parameter and the incompressible limit holding in the analytic norm. The same results extend more generally to Gevrey initial data with convergence holds in a Gevrey norm. The results extend the isentropic fluids in arXiv:2102.11454 to more general pressure laws., Comment: 15 pages