Search

While many BERT-based cross-modal pre-trained models produce excellent results on downstream understanding tasks like image-text retrieval and VQA, they cannot be applied to generation tasks directly. In this paper, we propose XGPT, a new method of Cross-modal Generative Pre-Training for Image Captioning that is designed to pre-train text-to-image caption generators through three novel generation tasks, including Image-conditioned Masked Language Modeling (IMLM), Image-conditioned Denoising Autoencoding (IDA), and Text-conditioned Image Feature Generation (TIFG). As a result, the pre-trained XGPT can be fine-tuned without any task-specific architecture modifications to create state-of-the-art models for image captioning. Experiments show that XGPT obtains new state-of-the-art results on the benchmark datasets, including COCO Captions and Flickr30k Captions. We also use XGPT to generate new image captions as data augmentation for the image retrieval task and achieve significant improvement on all recall metrics., Comment: 12 pages, 3 figures, 7 tables

The GABA deficit hypothesis remains one of the most compelling explanations for the information processing impairments in schizophrenia. However, much of the supportive evidence has been derived from post-mortem studies, whereas invivo studies have largely yielded inconsistent results. We undertook this single voxel proton magnetic resonance (MRS) GABA study to test in a sample of recent onset patients the replicability of our prior finding of reduced early visual cortex GABA in schizophrenia. We also examined the possibility that antipsychotics could represent a significant confound by studying a small subsample of antipsychotic naïve subjects. 23 adults with recent onset schizophrenia and a demographically matched sample of 31 healthy control subjects underwent MRS using a MEGA PRESS sequence on a 3T MR scanner to measure GABA concentration in early visual cortex. To control for in-scanner head movement confounding the results, we quantified the amount of head movement during GABA scans to identify and exclude from analysis scans with excessive movement. Patients demonstrated significantly reduced GABA levels compared to control subjects, p = 0.029. GABA levels did not differ significantly between patients who were antipsychotic naïve (n=7) and patients treated with antipsychotics. This replication in a recent onset sample suggest that diminished GABA in the visual cortex is a reliable finding, present in early phase of illness and not confounded by illness chronicity.