Search

Hallucinations in multimodal large language models (MLLMs) hinder their practical applications. To address this, we propose a Magnifier Prompt (MagPrompt), a simple yet effective method to tackle hallucinations in MLLMs via extremely simple instructions. MagPrompt is based on the following two key principles, which guide the design of various effective prompts, demonstrating robustness: (1) MLLMs should focus more on the image. (2) When there are conflicts between the image and the model's inner knowledge, MLLMs should prioritize the image. MagPrompt is training-free and can be applied to open-source and closed-source models, such as GPT-4o and Gemini-pro. It performs well across many datasets and its effectiveness is comparable or even better than more complex methods like VCD. Furthermore, our prompt design principles and experimental analyses provide valuable insights into multimodal hallucination., Comment: 9 pages, 13 tables, 4 figures

Multimodal Large Language Models (MLLMs) hallucinate, resulting in an emerging topic of visual hallucination evaluation (VHE). We introduce in this paper PhD, a large-scale benchmark for VHE. The essence of VHE is to ask an MLLM the right questions concerning a specific image. Depending on what to ask (objects, attributes, sentiment, etc.) and how the questions are asked, we structure PhD along two dimensions, i.e. task and mode. Five visual recognition tasks, ranging from low-level (object / attribute recognition) to middle-level (sentiment / position recognition and counting), are considered. Besides a normal visual QA mode, which we term VHE-base, PhD also asks questions with inaccurate context (VHE-iac) or with incorrect context (VHE-icc), or with AI-generated counter common sense images (VHE-ccs). We construct PhD by a ChatGPT-assisted semi-automated pipeline, encompassing four pivotal modules: task-specific hallucinatory element (hitem) selection, hitem-embedded question generation, inaccurate / incorrect context generation, and CCS image generation. With over 102k VQA triplets in total, PhD reveals considerable variability in MLLMs' performance across various modes, offering valuable insights into the nature of hallucination issues. As such, PhD stands as a potent tool not only for VHE but may also play a significant role in the refinement of MLLMs.

We analyze three observations of GRS 1915+105 in 2017 by Insight-HXMT when the source was in a spectrally soft state. We find strong absorption lines from highly ionized iron, which are due to absorption by disk wind outflowing at a velocity of $\sim$ 1000 km s$^{-1}$ along our line of sight. Two of the three observations show large amplitude oscillation in their light curves and the variation pattern corresponds to state $\kappa$ of GRS 1915+105. From time-averaged and flux-resolved analysis, we find that the variation of the ionization state of the disk wind follows the X-ray continuum on timescales from hundreds seconds to months. The radial location of the disk wind is consistent with thermal driving. The mass-loss rate due to the outflowing wind is comparable to the mass accretion rate in the inner disk, which demonstrates the important role of the disk wind in the disk accretion system., Comment: 8 pages, 5 figures. v2: refereed version