Search

As part of its broader Student Parent Success Initiative, Institute for Women's Policy Research (IWPR) conducted original research focusing specifically on Black single mother students. This report summarizes findings from 25 interviews IWPR conducted with Black single mother community college students, consisting of both students who were enrolled at the time of the interview and those who had been enrolled in the prior five years. These interviews provide insight into how college settings promote or inhibit the success of Black single mother community college students, how Black single mother students engage with institutional resources, which of these resources they find beneficial, and how institutional resources can better serve their needs.

3D head stylization transforms realistic facial features into artistic representations, enhancing user engagement across gaming and virtual reality applications. While 3D-aware generators have made significant advancements, many 3D stylization methods primarily provide near-frontal views and struggle to preserve the unique identities of original subjects, often resulting in outputs that lack diversity and individuality. This paper addresses these challenges by leveraging the PanoHead model, synthesizing images from a comprehensive 360-degree perspective. We propose a novel framework that employs negative log-likelihood distillation (LD) to enhance identity preservation and improve stylization quality. By integrating multi-view grid score and mirror gradients within the 3D GAN architecture and introducing a score rank weighing technique, our approach achieves substantial qualitative and quantitative improvements. Our findings not only advance the state of 3D head stylization but also provide valuable insights into effective distillation processes between diffusion models and GANs, focusing on the critical issue of identity preservation. Please visit the https://three-bee.github.io/head_stylization for more visuals., Comment: https://three-bee.github.io/head_stylization

3D GAN inversion aims to project a single image into the latent space of a 3D Generative Adversarial Network (GAN), thereby achieving 3D geometry reconstruction. While there exist encoders that achieve good results in 3D GAN inversion, they are predominantly built on EG3D, which specializes in synthesizing near-frontal views and is limiting in synthesizing comprehensive 3D scenes from diverse viewpoints. In contrast to existing approaches, we propose a novel framework built on PanoHead, which excels in synthesizing images from a 360-degree perspective. To achieve realistic 3D modeling of the input image, we introduce a dual encoder system tailored for high-fidelity reconstruction and realistic generation from different viewpoints. Accompanying this, we propose a stitching framework on the triplane domain to get the best predictions from both. To achieve seamless stitching, both encoders must output consistent results despite being specialized for different tasks. For this reason, we carefully train these encoders using specialized losses, including an adversarial loss based on our novel occlusion-aware triplane discriminator. Experiments reveal that our approach surpasses the existing encoder training methods qualitatively and quantitatively. Please visit the project page: https://berkegokmen1.github.io/dual-enc-3d-gan-inv., Comment: Joint first two authors. Accepted to NeurIPS 2024

Advanced image editing techniques, particularly inpainting, are essential for seamlessly removing unwanted elements while preserving visual integrity. Traditional GAN-based methods have achieved notable success, but recent advancements in diffusion models have produced superior results due to their training on large-scale datasets, enabling the generation of remarkably realistic inpainted images. Despite their strengths, diffusion models often struggle with object removal tasks without explicit guidance, leading to unintended hallucinations of the removed object. To address this issue, we introduce CLIPAway, a novel approach leveraging CLIP embeddings to focus on background regions while excluding foreground elements. CLIPAway enhances inpainting accuracy and quality by identifying embeddings that prioritize the background, thus achieving seamless object removal. Unlike other methods that rely on specialized training datasets or costly manual annotations, CLIPAway provides a flexible, plug-and-play solution compatible with various diffusion-based inpainting techniques., Comment: Project page: https://yigitekin.github.io/CLIPAway/

Earning a college degree has long been critical to unlocking many high-paying jobs -- and, as a result, to economic mobility and security. Increasingly, however, the traditional "norm" of a college student--one who enrolls straight out of high school, receives some support from their parents, lives on campus, and does not have substantial work or family responsibilities outside of school--does not fit the reality of much of the student population. Today, students who are themselves parents make up a significant percentage of those enrolled in college. While promoting student-parent success has mainly been left to institutions of higher education, several states have considered or adopted a range of policy initiatives to support student parents in postsecondary education. This paper provides state policy stakeholders, including governors, state agencies, and advocates, with a snapshot of the multitude of issues faced by student parents and with options to utilize state policy to address these issues.

Generative Adversarial Networks (GANs) have emerged as powerful tools for high-quality image generation and real image editing by manipulating their latent spaces. Recent advancements in GANs include 3D-aware models such as EG3D, which feature efficient triplane-based architectures capable of reconstructing 3D geometry from single images. However, limited attention has been given to providing an integrated framework for 3D-aware, high-quality, reference-based image editing. This study addresses this gap by exploring and demonstrating the effectiveness of the triplane space for advanced reference-based edits. Our novel approach integrates encoding, automatic localization, spatial disentanglement of triplane features, and fusion learning to achieve the desired edits. Additionally, our framework demonstrates versatility and robustness across various domains, extending its effectiveness to animal face edits, partially stylized edits like cartoon faces, full-body clothing edits, and 360-degree head edits. Our method shows state-of-the-art performance over relevant latent direction, text, and image-guided 2D and 3D-aware diffusion and GAN methods, both qualitatively and quantitatively., Comment: 20 pages, including supplementary material

Institute for Women's Policy Research (IWPR) conducted original research on an often-overlooked group of students--parents with children--as they struggle to make ends meet while pursuing academic degrees and certificates. Student parents often face enormous financial barriers to academic success. They report high financial insecurity including issues with food, housing and other basic needs. For students with children, borrowing for postsecondary education may mean accumulating debt without improved labor market outcomes or the financial ability to pay off loans. This is because the financial barriers and logistical challenges, such as inadequate access to reliable child care, student parents face may result in leaving college early without a credential, which has implications for future earnings.

StyleGAN models show editing capabilities via their semantically interpretable latent organizations which require successful GAN inversion methods to edit real images. Many works have been proposed for inverting images into StyleGAN's latent space. However, their results either suffer from low fidelity to the input image or poor editing qualities, especially for edits that require large transformations. That is because low-rate latent spaces lose many image details due to the information bottleneck even though it provides an editable space. On the other hand, higher-rate latent spaces can pass all the image details to StyleGAN for perfect reconstruction of images but suffer from low editing qualities. In this work, we present a novel image inversion architecture that extracts high-rate latent features and includes a flow estimation module to warp these features to adapt them to edits. The flows are estimated from StyleGAN features of edited and unedited latent codes. By estimating the high-rate features and warping them for edits, we achieve both high-fidelity to the input image and high-quality edits. We run extensive experiments and compare our method with state-of-the-art inversion methods. Qualitative metrics and visual comparisons show significant improvements.

Semantic image editing requires inpainting pixels following a semantic map. It is a challenging task since this inpainting requires both harmony with the context and strict compliance with the semantic maps. The majority of the previous methods proposed for this task try to encode the whole information from erased images. However, when an object is added to a scene such as a car, its style cannot be encoded from the context alone. On the other hand, the models that can output diverse generations struggle to output images that have seamless boundaries between the generated and unerased parts. Additionally, previous methods do not have a mechanism to encode the styles of visible and partially visible objects differently for better performance. In this work, we propose a framework that can encode visible and partially visible objects with a novel mechanism to achieve consistency in the style encoding and final generations. We extensively compare with previous conditional image generation and semantic image editing algorithms. Our extensive experiments show that our method significantly improves over the state-of-the-art. Our method not only achieves better quantitative results but also provides diverse results. Please refer to the project web page for the released code and demo: https://github.com/hakansivuk/DivSem.

Recent inversion methods have shown that real images can be inverted into StyleGAN's latent space and numerous edits can be achieved on those images thanks to the semantically rich feature representations of well-trained GAN models. However, extensive research has also shown that image inversion is challenging due to the trade-off between high-fidelity reconstruction and editability. In this paper, we tackle an even more difficult task, inverting erased images into GAN's latent space for realistic inpaintings and editings. Furthermore, by augmenting inverted latent codes with different latent samples, we achieve diverse inpaintings. Specifically, we propose to learn an encoder and mixing network to combine encoded features from erased images with StyleGAN's mapped features from random samples. To encourage the mixing network to utilize both inputs, we train the networks with generated data via a novel set-up. We also utilize higher-rate features to prevent color inconsistencies between the inpainted and unerased parts. We run extensive experiments and compare our method with state-of-the-art inversion and inpainting methods. Qualitative metrics and visual comparisons show significant improvements., Comment: ICCV 2023

The Postsecondary Data Partnership (PDP) is a service of the National Student Clearinghouse established in 2017 to empower institutions with more comprehensive data, easier analysis, centralized reporting functions, and interactive visualizations. Since its inception, over 500 colleges and universities have participated in the PDP. The 2022 PDP Insights report focuses on two primary metrics: students' first year credit completion ratio (CCR) and credit accumulation rate (CAR). The CCR is the ratio of credits earned to credits attempted. This measure of student success and credit attainment can provide insights into students' overall course completion outcomes and measure the efficiency of students' movement through coursework. The CAR directly measures students' timely accumulation of college credits by identifying the share of students who surpassed specific credit-hour thresholds within a given period. This early momentum measure can offer insights into students' progression toward degree completion and provide indicators of when and where gaps in ultimate degree attainment between groups may begin. Students in this analysis are first-time (including first-time in college and first-time, transfer-in), degree-seeking students entering a PDP-participating institution in the 2019-20 cohort.

This paper presents a method to reconstruct high-quality textured 3D models from single images. Current methods rely on datasets with expensive annotations; multi-view images and their camera parameters. Our method relies on GAN generated multi-view image datasets which have a negligible annotation cost. However, they are not strictly multi-view consistent and sometimes GANs output distorted images. This results in degraded reconstruction qualities. In this work, to overcome these limitations of generated datasets, we have two main contributions which lead us to achieve state-of-the-art results on challenging objects: 1) A robust multi-stage learning scheme that gradually relies more on the models own predictions when calculating losses, 2) A novel adversarial learning pipeline with online pseudo-ground truth generations to achieve fine details. Our work provides a bridge from 2D supervisions of GAN models to 3D reconstruction models and removes the expensive annotation efforts. We show significant improvements over previous methods whether they were trained on GAN generated multi-view images or on real images with expensive annotations. Please visit our web-page for 3D visuals: https://research.nvidia.com/labs/adlr/progressive-3d-learning, Comment: Web-page: https://research.nvidia.com/labs/adlr/progressive-3d-learning. arXiv admin note: text overlap with arXiv:2203.09362

Image inpainting task refers to erasing unwanted pixels from images and filling them in a semantically consistent and realistic way. Traditionally, the pixels that are wished to be erased are defined with binary masks. From the application point of view, a user needs to generate the masks for the objects they would like to remove which can be time-consuming and prone to errors. In this work, we are interested in an image inpainting algorithm that estimates which object to be removed based on natural language input and removes it, simultaneously. For this purpose, first, we construct a dataset named GQA-Inpaint for this task. Second, we present a novel inpainting framework, Inst-Inpaint, that can remove objects from images based on the instructions given as text prompts. We set various GAN and diffusion-based baselines and run experiments on synthetic and real image datasets. We compare methods with different evaluation metrics that measure the quality and accuracy of the models and show significant quantitative and qualitative improvements.

Estimating 3D human texture from a single image is essential in graphics and vision. It requires learning a mapping function from input images of humans with diverse poses into the parametric (UV) space and reasonably hallucinating invisible parts. To achieve a high-quality 3D human texture estimation, we propose a framework that adaptively samples the input by a deformable convolution where offsets are learned via a deep neural network. Additionally, we describe a novel cycle consistency loss that improves view generalization. We further propose to train our framework with an uncertainty-based pixel-level image reconstruction loss, which enhances color fidelity. We compare our method against the state-of-the-art approaches and show significant qualitative and quantitative improvements.

We present a novel image inversion framework and a training pipeline to achieve high-fidelity image inversion with high-quality attribute editing. Inverting real images into StyleGAN's latent space is an extensively studied problem, yet the trade-off between the image reconstruction fidelity and image editing quality remains an open challenge. The low-rate latent spaces are limited in their expressiveness power for high-fidelity reconstruction. On the other hand, high-rate latent spaces result in degradation in editing quality. In this work, to achieve high-fidelity inversion, we learn residual features in higher latent codes that lower latent codes were not able to encode. This enables preserving image details in reconstruction. To achieve high-quality editing, we learn how to transform the residual features for adapting to manipulations in latent codes. We train the framework to extract residual features and transform them via a novel architecture pipeline and cycle consistency losses. We run extensive experiments and compare our method with state-of-the-art inversion methods. Qualitative metrics and visual comparisons show significant improvements. Code: https://github.com/hamzapehlivan/StyleRes

The metal-organic (MO) compound titanium(IV)-isopropoxide (Ti(OiPr)4, TTIP) has tremendous technological relevance for thin film growth and coating technologies, offering a low-temperature deposition route for titania and titanium-oxide-based compounds. Thermal decomposition via the release of organic ligands, a key process in any TTIP-based synthesis approach, is commonly assumed to take place only via the beta-hydride elimination process. Here, we present reactive force field molecular dynamics (ReaxFF-MD) and metadynamics simulations that challenge this conventionally assumed scenario by revealing different, energetically preferred reaction pathways. The complete reaction scheme for the TTIP thermolysis, along with the statistics for the different ligand liberation steps and the associated reaction barriers for the bond dissociation events is presented. ReaxFF-MD simulations performed in the dilute limit realistically capture typical thin film deposition conditions, which in combination with metadynamics data, which produces free energies, constitutes a very powerful tool to quantitatively analyze the reaction dynamics of MO-based thin film growth processes and provide an atomic-scale understanding of how the remaining organic ligands detach from different titanium-containing MO fragments. The approach presented here allows for effective and straightforward identification of the undesirable temperature biasing effects in ReaxFF-MD and represents a predictive framework to identify chemical reaction pathways relevant to film growth processes at the atomic scale under realistic, experimentally relevant conditions. It enables computationally informed engineering of MO molecules with tailored decomposition and reaction pathways, and thus rapid and cost-effective advancements in MO molecule design for existing and future applications of thin film deposition and coating processes., Comment: 26 pages, 7 figures

Moment invariants are well-established and effective shape descriptors for image classification. In this report, we introduce a package for R-language, named IM, that implements the calculation of moments for images and allows the reconstruction of images from moments within an object-oriented framework. Several types of moments may be computed using the IM library, including discrete and continuous Chebyshev, Gegenbauer, Legendre, Krawtchouk, dual Hahn, generalized pseudo-Zernike, Fourier-Mellin, and radial harmonic Fourier moments. In addition, custom bivariate types of moments can be calculated using combinations of two different types of polynomials. A method of polar transformation of pixel coordinates is used to provide an approximate invariance to rotation for moments that are orthogonal over a rectangle. The different types of polynomials used to calculate moments are discussed in this report, as well as comparisons of reconstruction and running time. Examples of image classification using image moments are provided., Comment: Apr 2014, technical report

A thorough understanding of the neuroanatomy of peripheral nerves is required for a better insight into their function and the development of neuromodulation tools and strategies. In biophysical modeling, it is commonly assumed that the complex spatial arrangement of myelinated and unmyelinated axons in peripheral nerves is random, however, in reality the axonal organization is inhomogeneous and anisotropic. Present quantitative neuroanatomy methods analyze peripheral nerves in terms of the number of axons and the morphometric characteristics of the axons, such as area and diameter. In this study, we employed spatial statistics and point process models to describe the spatial arrangement of axons and Sinkhorn distances to compute the similarities between these arrangements (in terms of first- and second-order statistics) in various vagus and pelvic nerve cross-sections. We utilized high-resolution TEM images that have been segmented using a custom-built high-throughput deep learning system based on a highly modified U-Net architecture. Our findings show a novel and innovative approach to quantifying similarities between spatial point patterns using metrics derived from the solution to the optimal transport problem. We also present a generalizable pipeline for quantitative analysis of peripheral nerve architecture. Our data demonstrate differences between male- and female-originating samples and similarities between the pelvic and abdominal vagus nerves., Comment: 10 figures

We propose VecGAN, an image-to-image translation framework for facial attribute editing with interpretable latent directions. Facial attribute editing task faces the challenges of precise attribute editing with controllable strength and preservation of the other attributes of an image. For this goal, we design the attribute editing by latent space factorization and for each attribute, we learn a linear direction that is orthogonal to the others. The other component is the controllable strength of the change, a scalar value. In our framework, this scalar can be either sampled or encoded from a reference image by projection. Our work is inspired by the latent space factorization works of fixed pretrained GANs. However, while those models cannot be trained end-to-end and struggle to edit encoded images precisely, VecGAN is end-to-end trained for image translation task and successful at editing an attribute while preserving the others. Our extensive experiments show that VecGAN achieves significant improvements over state-of-the-arts for both local and global edits., Comment: ECCV 2022