Search

Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model., Comment: Work in progress

We present a self-supervised approach to recolorization of images from design-oriented domains. Our approach can recolor images based on image exemplars or target color palettes provided by a user. In contrast with previous approaches, our method can reproduce color palettes with luminance distributions that differ significantly from input, and our method is the first palette-based approach to distinguish between recolorings that match reflectance and those that match illumination, making it particularly well-suited to visualizing different aesthetic decisions in design applications. The key to our approach is first to learn latent representations for texture and color in a setting where self-supervision is especially straightforward, and then to learn a mapping to our color representation from input color palettes and scene illumination, which offers a more intuitive space for controlling and exploring recolorization.

In this paper, we present the details of Women in Computer Vision Workshop - WiCV 2021, organized alongside the virtual CVPR 2021. It provides a voice to a minority (female) group in the computer vision community and focuses on increasing the visibility of these researchers, both in academia and industry. WiCV believes that such an event can play an important role in lowering the gender imbalance in the field of computer vision. WiCV is organized each year where it provides a)~opportunity for collaboration between researchers from minority groups, b)~mentorship to female junior researchers, c)~financial support to presenters to overcome monetary burden and d)~large and diverse choice of role models, who can serve as examples to younger researchers at the beginning of their careers. In this paper, we present a report on the workshop program, trends over the past years, a summary of statistics regarding presenters, attendees, and sponsorship for the WiCV 2021 workshop., Comment: Report on WiCV Workshop at CVPR 2021. arXiv admin note: substantial text overlap with arXiv:2101.03787

This article aims to improve the traditional lamps to improve the use of eyes in the usual study and work environment to prevent and control vision loss. The improvements include automatic adjustment of color temperature and brightness, as well as the use of the Internet of Things for eye protection and data recording and analysis.

This paper aims to improve the efficiency of hospitals and the experience of patients and doctors and proposes some improvements to the software design of most current hospital information systems, including the synchronization of online and offline consultation (online “one account pass” and offline “one card”), the implementation of electronic medical records (block chain), and the creation of a “paperless” hospital.

The current clinical diagnosis of depression in the medical community relies on self-rating scales and physician interviews, but this approach is limited by the expertise of clinicians and the uneven distribution of medical resources. This paper proposes the use of video multimodal techniques in clinical diagnosis, aiming to improve the efficiency and accuracy of depression detection in clinical settings.

Are extralinguistic signals such as image pixels crucial for inducing constituency grammars? While past work has shown substantial gains from multimodal cues, we investigate whether such gains persist in the presence of rich information from large language models (LLMs). We find that our approach, LLM-based C-PCFG (LC-PCFG), outperforms previous multi-modal methods on the task of unsupervised constituency parsing, achieving state-of-the-art performance on a variety of datasets. Moreover, LC-PCFG results in an over 50% reduction in parameter count, and speedups in training time of 1.7x for image-aided models and more than 5x for video-aided models, respectively. These results challenge the notion that extralinguistic signals such as image pixels are needed for unsupervised grammar induction, and point to the need for better text-only baselines in evaluating the need of multi-modality for the task.

Recent advances in data augmentation enable one to translate images by learning the mapping between a source domain and a target domain. Existing methods tend to learn the distributions by training a model on a variety of datasets, with results evaluated largely in a subjective manner. Relatively few works in this area, however, study the potential use of image synthesis methods for recognition tasks. In this paper, we propose and explore the problem of image translation for data augmentation. We first propose a lightweight yet efficient model for translating texture to augment images based on a single input of source texture, allowing for fast training and testing, referred to as Single Image Texture Translation for data Augmentation (SITTA). Then we explore the use of augmented data in long-tailed and few-shot image classification tasks. We find the proposed augmentation method and workflow is capable of translating the texture of input data into a target domain, leading to consistently improved image recognition performance. Finally, we examine how SITTA and related image translation methods can provide a basis for a data-efficient, "augmentation engineering" approach to model training. Codes are available at https://github.com/Boyiliee/SITTA., Comment: Learning from Limited and Imperfect Data (L2ID) Workshop, ECCV 2022

In this paper we present the details of Women in Computer Vision Workshop - WiCV 2020, organized in alongside virtual CVPR 2020. This event aims at encouraging the women researchers in the field of computer vision. It provides a voice to a minority (female) group in computer vision community and focuses on increasingly the visibility of these researchers, both in academia and industry. WiCV believes that such an event can play an important role in lowering the gender imbalance in the field of computer vision. WiCV is organized each year where it provides a.) opportunity for collaboration with between researchers b.) mentorship to female junior researchers c.) financial support to presenters to overcome monetary burden and d.) large and diverse choice of role models, who can serve as examples to younger researchers at the beginning of their careers. In this paper, we present a report on the workshop program, trends over the past years, a summary of statistics regarding presenters, attendees, and sponsorship for the current workshop.

In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet., A Technical Report

Although according to several benchmarks automatic machine reading comprehension (MRC) systems have recently reached super-human performance, less attention has been paid to their computational efficiency. However, efficiency is of crucial importance for training and deployment in real world applications. This paper introduces Integrated Triaging, a framework that prunes almost all context in early layers of a network, leaving the remaining (deep) layers to scan only a tiny fraction of the full corpus. This pruning drastically increases the efficiency of MRC models and further prevents the later layers from overfitting to prevalent short paragraphs in the training set. Our framework is extremely flexible and naturally applicable to a wide variety of models. Our experiment on doc-SQuAD and TriviaQA tasks demonstrates its effectiveness in consistently improving both speed and quality of several diverse MRC models., Comment: Technical report

We present a comprehensive study and evaluation of existing single image dehazing algorithms, using a new large-scale benchmark consisting of both synthetic and real-world hazy images, called REalistic Single Image DEhazing (RESIDE). RESIDE highlights diverse data sources and image contents, and is divided into five subsets, each serving different training or evaluation purposes. We further provide a rich variety of criteria for dehazing algorithm evaluation, ranging from full-reference metrics, to no-reference metrics, to subjective evaluation and the novel task-driven evaluation. Experiments on RESIDE shed light on the comparisons and limitations of state-of-the-art dehazing algorithms, and suggest promising future directions., Comment: IEEE Transactions on Image Processing(TIP 2019)

This paper proposes an image dehazing model built with a convolutional neural network (CNN), called All-in-One Dehazing Network (AOD-Net). It is designed based on a re-formulated atmospheric scattering model. Instead of estimating the transmission matrix and the atmospheric light separately as most previous models did, AOD-Net directly generates the clean image through a light-weight CNN. Such a novel end-to-end design makes it easy to embed AOD-Net into other deep models, e.g., Faster R-CNN, for improving high-level task performance on hazy images. Experimental results on both synthesized and natural hazy image datasets demonstrate our superior performance than the state-of-the-art in terms of PSNR, SSIM and the subjective visual quality. Furthermore, when concatenating AOD-Net with Faster R-CNN and training the joint pipeline from end to end, we witness a large improvement of the object detection performance on hazy images.

The recent development of CNN-based image dehazing has revealed the effectiveness of end-to-end modeling. However, extending the idea to end-to-end video dehazing has not been explored yet. In this paper, we propose an End-to-End Video Dehazing Network (EVD-Net), to exploit the temporal consistency between consecutive video frames. A thorough study has been conducted over a number of structure options, to identify the best temporal fusion strategy. Furthermore, we build an End-to-End United Video Dehazing and Detection Network (EVDD-Net), which concatenates and jointly trains EVD-Net with a video object detection model. The resulting augmented end-to-end pipeline has demonstrated much more stable and accurate detection results in hazy video.

This research examines the rationale of geographic co-location of entrepreneurs who do business on internet platforms. Prior research has shown that entrepreneurs gain valuable synergy benefits from being embedded in industrial networks. Nevertheless, the advantages of geographic clustering when business is conducted via the internet are still to be understood. This research aims to understand how internet-based economic activity interacts with local social relations and structures, thus seeking to explain the phenomenon of industrial clustering of internet-enabled entrepreneurial activity. Guided by theories of relational and institutional embeddedness, we examine the way social relations are formed online, trace the rationale of local social relations while business is conducted online, and study the role of major institutional actors that support the economic activities of the entrepreneurs. Empirically, this thesis examines two regional clusters of Chinese microentrepreneurs who conduct their business on an e-commerce platform and form dynamic interpersonal ties with business partners and customers both online and offline. The method of ethnographic case study is adopted to gain in depth understanding of the ways various internet networking tools have been appropriated in business practice in these two cases and the ways local microentrepreneurs build up collaborative networks in geographic place as well as cyberspace. The study of Chinese micro-entrepreneurs reveals and substantiates the formation of a hybrid sociality, whereby economic exchange via the internet and business conducted by electronic tools are complemented by local social relations and actively supported by local government and the IT service corporation. This research also contributes to development policy considerations; it shows that regions that are usually unattractive to capital and knowledge/talent flows can gain economic development momentum by entangling the conduct of business on web platforms with local social institutions.

Objective: The aim of this study was to determine whether the type of congenital heart disease (CHD) or heart function influence fetal cerebrovascular blood flow dynamics. Methods: Doppler flow velocimetry was performed in the umbilical artery (UA) and middle cerebral artery (MCA), and the ratio of the UA pulsatility index (PI) to the MCA PI (U/C PI) was determined in 45 fetuses with CHD at 20–40 weeks’ gestational age. The control group consisted of 275 healthy fetuses matched for gestational age. Individual PI measurements were converted into Z-scores for statistical analysis. Results: Fetuses with CHD (n = 45) had an increased UA PI (p = 0.001) and U/C PI (p < 0.001) when compared to controls. There was no significant difference in the MCA PI between fetuses with CHD (n = 45) and controls (n = 275), while fetuses with CHD complicated by congestive heart failure (CHF) (n = 10) had a decreased MCA PI (p < 0.001) compared to controls. Conclusions: The lower PIs observed in the MCA of fetuses with CHD complicated by CHF represents a marker of cerebral vasodilation that is due to cerebral hypoxemia and limiting perfusion. The heart function and type of CHD have an impact on fetal cerebrovascular blood flow distribution and this may contribute to the cause of abnormal neurologic development in these fetuses.

Objective To study the effectiveness of the pulsatility index for veins of ductus venosus (DV-PIV) and the Tei index in a prospective assessment of fetal hypoxic-ischemic brain damage in a near-term ovine fetus model with intermittent umbilical cord occlusion (UCO). Methods Twelve fetal sheep were studied with umbilical cord occlusion performed in the experimental group animals by complete inflation of an occluder cuff for 90 s, every 30 min for approximately 2.5 h. Fetal arterial blood was sampled at 5 min before the first umbilical cord occlusions, approximately 60 s of the first umbilical cord occlusions, and 3 min after each occlusion for blood gas, pH, neuron-specific enolase (NSE) and S100B. Doppler measurements and Doppler echocardiographic examinations were performed 5 min before the first umbilical cord occlusions and 3 min after each successive occlusion. Results In experimental group animals, UCO caused a large decline in arterial PaO2 (to ∼7.70 mmHg, p < 0.01), a modest decline in pH (to ∼7.24, p < 0.01), and a modest rise in PaCO2 (to ∼53.31 mmHg, p < 0.01), with a return more or less to baseline after occluder release. and there was significant change as compared with the control animals (all p < 0.01) with cumulative changes in responses to repetitive cord occlusions. The DV-PIV waveforms, right ventricle (RV) and LV Tei indices, the serum levels of NSE and S100B increased with cord occlusions (all p < 0.05), and were significantly higher than the control animals (all p < 0.05) with a cumulative changes in responses to repetitive cord occlusions. RV and LV Tei indices were significantly correlated with PaO2 (r = − 0.684, p < 0.01 and r = − 0.725, p < 0.01), PaCO2 (r = 0.682, p < 0.01 and r = 0.780, p < 0.01), pH (r = − 0.538, p < 0.01 and r = − 0.681, p < 0.01), NSE (r = 0.653, p < 0.01 and r = 0.687, p < 0.01), and S100B (r = 0.606, p < 0.01 and r = 0.640, p < 0.01). Significant but weaker correlations were also present between DV-PIV and the parameters considered. Conclusion Umbilical cord occlusion during the latter part of the pregnancy, enough to cause significant hypoxemia and acidosis, results in a significant increase of DV-PIV, RV and LV Tei indices, and the serum levels of NSE and S100B. There was a strong correlation between the RV and LV Tei indices and blood gas, pH, and NSE, S100B with hypoxia. Therefore, the Tei index might be an easy and useful quantitative parameter for assessing fetal hypoxic ischemia. Copyright © 2007 John Wiley & Sons, Ltd.