Search

Tables contain factual and quantitative data accompanied by various structures and contents that pose challenges for machine comprehension. Previous methods generally design task-specific architectures and objectives for individual tasks, resulting in modal isolation and intricate workflows. In this paper, we present a novel large vision-language model, TabPedia, equipped with a concept synergy mechanism. In this mechanism, all the involved diverse visual table understanding (VTU) tasks and multi-source visual embeddings are abstracted as concepts. This unified framework allows TabPedia to seamlessly integrate VTU tasks, such as table detection, table structure recognition, table querying, and table question answering, by leveraging the capabilities of large language models (LLMs). Moreover, the concept synergy mechanism enables table perception-related and comprehension-related tasks to work in harmony, as they can effectively leverage the needed clues from the corresponding source perception embeddings. Furthermore, to better evaluate the VTU task in real-world scenarios, we establish a new and comprehensive table VQA benchmark, ComTQA, featuring approximately 9,000 QA pairs. Extensive quantitative and qualitative experiments on both table perception and comprehension tasks, conducted across various public benchmarks, validate the effectiveness of our TabPedia. The superior performance further confirms the feasibility of using LLMs for understanding visual tables when all concepts work in synergy. The benchmark ComTQA has been open-sourced at https://huggingface.co/datasets/ByteDance/ComTQA. The source code and model will be released later., Comment: 20 pages, 8 figures

Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of leading models like GPT4V and Gemini, partly due to a lack of extensive, high-quality instruction tuning data. To this end, we introduce a new approach for creating a massive, high-quality instruction-tuning dataset, Square-10M, which is generated using closed-source MLLMs. The data construction process, termed Square, consists of four steps: Self-Questioning, Answering, Reasoning, and Evaluation. Our experiments with Square-10M led to three key findings: 1) Our model, TextSquare, considerably surpasses open-source previous state-of-the-art Text-centric MLLMs and sets a new standard on OCRBench(62.2%). It even outperforms top-tier models like GPT4V and Gemini in 6 of 10 text-centric benchmarks. 2) Additionally, we demonstrate the critical role of VQA reasoning data in offering comprehensive contextual insights for specific questions. This not only improves accuracy but also significantly mitigates hallucinations. Specifically, TextSquare scores an average of 75.1% across four general VQA and hallucination evaluation datasets, outperforming previous state-of-the-art models. 3) Notably, the phenomenon observed in scaling text-centric VQA datasets reveals a vivid pattern: the exponential increase of instruction tuning data volume is directly proportional to the improvement in model performance, thereby validating the necessity of the dataset scale and the high quality of Square-10M.

The recent discovery of superconductivity and magnetism in trilayer rhombohedral graphene (RG) establishes an ideal, untwisted platform to study strong correlation electronic phenomena. However, the correlated effects in multilayer RG have received limited attention, and, particularly, the evolution of the correlations with increasing layer number remains an unresolved question. Here, we show the observation of layer-dependent electronic structures and correlations in RG multilayers from 3 to 9 layers by using scanning tunneling microscopy and spectroscopy. We explicitly determine layer-enhanced low-energy flat bands and interlayer coupling strength. The former directly demonstrates the further flattening of low-energy bands in thicker RG, and the later indicates the presence of varying interlayer interactions in RG multilayers. Moreover, we find significant splitting of the flat bands, ranging from ~50-80 meV, under liquid nitrogen temperature when they are partially filled, indicating the emergence of interaction-induced strongly correlated states. Particularly, the strength of the correlated states is notably enhanced in thicker RG and reaches its maximum in the six-layer, validating directly theoretical predictions and establishing abundant new candidates for strongly correlated systems. Our results provide valuable insights into the layer dependence of the electronic properties in RG, paving the way for investigating robust and highly accessible correlated phases in simpler systems., Comment: 21 pages, 4 figures

In recent years, raw video denoising has garnered increased attention due to the consistency with the imaging process and well-studied noise modeling in the raw domain. However, two problems still hinder the denoising performance. Firstly, there is no large dataset with realistic motions for supervised raw video denoising, as capturing noisy and clean frames for real dynamic scenes is difficult. To address this, we propose recapturing existing high-resolution videos displayed on a 4K screen with high-low ISO settings to construct noisy-clean paired frames. In this way, we construct a video denoising dataset (named as ReCRVD) with 120 groups of noisy-clean videos, whose ISO values ranging from 1600 to 25600. Secondly, while non-local temporal-spatial attention is beneficial for denoising, it often leads to heavy computation costs. We propose an efficient raw video denoising transformer network (RViDeformer) that explores both short and long-distance correlations. Specifically, we propose multi-branch spatial and temporal attention modules, which explore the patch correlations from local window, local low-resolution window, global downsampled window, and neighbor-involved window, and then they are fused together. We employ reparameterization to reduce computation costs. Our network is trained in both supervised and unsupervised manners, achieving the best performance compared with state-of-the-art methods. Additionally, the model trained with our proposed dataset (ReCRVD) outperforms the model trained with previous benchmark dataset (CRVD) when evaluated on the real-world outdoor noisy videos. Our code and dataset will be released after the acceptance of this work., Comment: 16 pages,15 figures

Epitaxial growth and controllable doping of wafer-scale single-crystal transition-metal dichalcogenides (TMDCs) are two central tasks for extending Moore's law beyond silicon. However, despite considerable efforts, addressing such crucial issues simultaneously under two-dimensional (2D) confinement is yet to be realized. Here we design an ingenious epitaxial strategy to synthesize record-breaking 4-inch single-crystal Fe-doped TMDCs monolayers on industry-compatible c-plane sapphire without miscut angle. In-depth characterizations and theoretical calculations reveal that the introduction of Fe significantly decreases the formation energy of parallel steps on sapphire surfaces and contributes to the edge-nucleation of unidirectional TMDCs domains (>99%). The ultrahigh electron mobility (~86 cm2 V -1 s-1) and remarkable on/off current ratio (~108) are discovered on 4-inch single-crystal Fe-MoS2 monolayers due to the ultralow contact resistance and perfect Ohmic contact with metal electrodes. This work represents a substantial leap in terms of bridging the synthesis and doping of wafer-scale single-crystal 2D semiconductors without the need for substrate miscut, which should promote the further device downscaling and extension of Moore's law., Comment: 17 pages, 5 figures

Magnetic insulators with strong perpendicular magnetic anisotropy (PMA) play a key role in exploring pure spin current phenomena and developing ultralow-dissipation spintronic devices, thereby it is highly desirable to develop new material platforms. Here we report epitaxial growth of La2/3Sr1/3MnO3 (LSMO)-SrIrO3 (SIO) composite oxide films (LSMIO) with different crystalline orientations fabricated by sequential two-target ablation process using pulsed laser deposition. The LSMIO films exhibit high crystalline quality with homogeneous mixture of LSMO and SIO at atomic level. Ferrimagnetic and insulating transport characteristics are observed, with the temperature-dependent electric resistivity well fitted by Mott variable-range-hopping model. Moreover, the LSMIO films show strong PMA. Through further constructing all perovskite oxide heterostructures of the ferrimagnetic insulator LSMIO and a strong spin-orbital coupled SIO layer, pronounced spin Hall magnetoresistance (SMR) and spin Hall-like anomalous Hall effect (SH-AHE) were observed. These results illustrate the potential application of the ferrimagnetic insulator LSMIO in developing all-oxide ultralow-dissipation spintronic devices.

Doping is of great importance to tailor the electrical properties of semiconductors. However, the present doping methodologies for organic semiconductors (OSCs) are either inefficient or can only apply to a small number of OSCs, seriously limiting their general application. Herein, we reveal a novel p-doping mechanism by investigating the interactions between the dopant trityl cation and poly(3-hexylthiophene) (P3HT). It is found that electrophilic attack of the trityl cations on thiophenes results in the formation of alkylated ions that induce electron transfer from neighboring P3HT chains, resulting in p-doping. This unique p-doping mechanism can be employed to dope various OSCs including those with high ionization energy (IE=5.8 eV). Moreover, this doping mechanism endows trityl cation with strong doping ability, leading to polaron yielding efficiency of 100 % and doping efficiency of over 80 % in P3HT. The discovery and elucidation of this novel doping mechanism not only points out that strong electrophiles are a class of efficient p-dopants for OSCs, but also provides new opportunities towards highly efficient doping of OSCs.

Unsupervised dialogue structure learning is an important and meaningful task in natural language processing. The extracted dialogue structure and process can help analyze human dialogue, and play a vital role in the design and evaluation of dialogue systems. The traditional dialogue system requires experts to manually design the dialogue structure, which is very costly. But through unsupervised dialogue structure learning, dialogue structure can be automatically obtained, reducing the cost of developers constructing dialogue process. The learned dialogue structure can be used to promote the dialogue generation of the downstream task system, and improve the logic and consistency of the dialogue robot's reply.In this paper, we propose a Bert-based unsupervised dialogue structure learning algorithm DSBERT (Dialogue Structure BERT). Different from the previous SOTA models VRNN and SVRNN, we combine BERT and AutoEncoder, which can effectively combine context information. In order to better prevent the model from falling into the local optimal solution and make the dialogue state distribution more uniform and reasonable, we also propose three balanced loss functions that can be used for dialogue structure learning. Experimental results show that DSBERT can generate a dialogue structure closer to the real structure, can distinguish sentences with different semantics and map them to different hidden states.

Topological polar vortices that are the electric analogues of magnetic objects, present great potential in applications of future nanoelectronics due to their nanometer size, anomalous dielectric response, and chirality. To enable the functionalities, it is prerequisite to manipulate the polar states and chirality by using external stimuli. Here, we probe the evolutions of polar state and chirality of polar vortices in PbTiO3/SrTiO3 superlattices under electric field by using atomically resolved in situ scanning transmission electron microscopy and phase-field simulations. We find that the adjacent clockwise and counterclockwise vortex usually have opposite chirality. The phase-field simulations suggest that the rotation reversal or axial polarization switching can lead to the chirality change. Guided by which, we experimentally validate that the vortex rotation direction can be changed by applying and subsequently removing of electric fields, offering a potential strategy to manipulate the vortex chirality. The revealed details of dynamic behavior for individual polar vortices at atomic scale and the proposed strategy for chirality manipulation provide fundamentals for future device applications.

In recent years, the supervised learning strategy for real noisy image denoising has been emerging and has achieved promising results. In contrast, realistic noise removal for raw noisy videos is rarely studied due to the lack of noisy-clean pairs for dynamic scenes. Clean video frames for dynamic scenes cannot be captured with a long-exposure shutter or averaging multi-shots as was done for static images. In this paper, we solve this problem by creating motions for controllable objects, such as toys, and capturing each static moment for multiple times to generate clean video frames. In this way, we construct a dataset with 55 groups of noisy-clean videos with ISO values ranging from 1600 to 25600. To our knowledge, this is the first dynamic video dataset with noisy-clean pairs. Correspondingly, we propose a raw video denoising network (RViDeNet) by exploring the temporal, spatial, and channel correlations of video frames. Since the raw video has Bayer patterns, we pack it into four sub-sequences, i.e RGBG sequences, which are denoised by the proposed RViDeNet separately and finally fused into a clean video. In addition, our network not only outputs a raw denoising result, but also the sRGB result by going through an image signal processing (ISP) module, which enables users to generate the sRGB result with their favourite ISPs. Experimental results demonstrate that our method outperforms state-of-the-art video and raw image denoising algorithms on both indoor and outdoor videos., Comment: CVPR2020 accepted paper