Search

Your search keyword '"Shen, Jingjing"' showing total 611 results
Start Over Author "Shen, Jingjing"
611 results on '"Shen, Jingjing"'

1. Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion

Author

Wang, Tengfei, Zhang, Bo, Zhang, Ting, Gu, Shuyang, Bao, Jianmin, Baltrusaitis, Tadas, Shen, Jingjing, Chen, Dong, Wen, Fang, Chen, Qifeng, and Guo, Baining

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

This paper presents a 3D generative model that uses diffusion models to automatically generate 3D digital avatars represented as neural radiance fields. A significant challenge in generating such avatars is that the memory and processing costs in 3D are prohibitive for producing the rich details required for high-quality avatars. To tackle this problem we propose the roll-out diffusion network (Rodin), which represents a neural radiance field as multiple 2D feature maps and rolls out these maps into a single 2D feature plane within which we perform 3D-aware diffusion. The Rodin model brings the much-needed computational efficiency while preserving the integrity of diffusion in 3D by using 3D-aware convolution that attends to projected features in the 2D feature plane according to their original relationship in 3D. We also use latent conditioning to orchestrate the feature generation for global coherence, leading to high-fidelity avatars and enabling their semantic editing based on text prompts. Finally, we use hierarchical synthesis to further enhance details. The 3D avatars generated by our model compare favorably with those produced by existing generative techniques. We can generate highly detailed avatars with realistic hairstyles and facial hair like beards. We also demonstrate 3D avatar generation from image or text as well as text-guided editability., Comment: Project Webpage: https://3d-avatar-diffusion.microsoft.com/

Published
2022

2. DigiFace-1M: 1 Million Digital Face Images for Face Recognition

Author

Bae, Gwangbin, de La Gorce, Martin, Baltrusaitis, Tadas, Hewitt, Charlie, Chen, Dong, Valentin, Julien, Cipolla, Roberto, and Shen, Jingjing

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

State-of-the-art face recognition models show impressive accuracy, achieving over 99.8% on Labeled Faces in the Wild (LFW) dataset. Such models are trained on large-scale datasets that contain millions of real human face images collected from the internet. Web-crawled face images are severely biased (in terms of race, lighting, make-up, etc) and often contain label noise. More importantly, the face images are collected without explicit consent, raising ethical concerns. To avoid such problems, we introduce a large-scale synthetic dataset for face recognition, obtained by rendering digital faces using a computer graphics pipeline. We first demonstrate that aggressive data augmentation can significantly reduce the synthetic-to-real domain gap. Having full control over the rendering pipeline, we also study how each attribute (e.g., variation in facial pose, accessories and textures) affects the accuracy. Compared to SynFace, a recent method trained on GAN-generated synthetic faces, we reduce the error rate on LFW by 52.5% (accuracy from 91.93% to 96.17%). By fine-tuning the network on a smaller number of real face images that could reasonably be obtained with consent, we achieve accuracy that is comparable to the methods trained on millions of real face images., Comment: WACV 2023

Published
2022

3. VolTeMorph: Realtime, Controllable and Generalisable Animation of Volumetric Representations

Author

Garbin, Stephan J., Kowalski, Marek, Estellers, Virginia, Szymanowicz, Stanislaw, Rezaeifar, Shideh, Shen, Jingjing, Johnson, Matthew, and Valentin, Julien

Subjects
Computer Science - Graphics, Computer Science - Computer Vision and Pattern Recognition
Abstract

The recent increase in popularity of volumetric representations for scene reconstruction and novel view synthesis has put renewed focus on animating volumetric content at high visual quality and in real-time. While implicit deformation methods based on learned functions can produce impressive results, they are `black boxes' to artists and content creators, they require large amounts of training data to generalise meaningfully, and they do not produce realistic extrapolations outside the training data. In this work we solve these issues by introducing a volume deformation method which is real-time, easy to edit with off-the-shelf software and can extrapolate convincingly. To demonstrate the versatility of our method, we apply it in two scenarios: physics-based object deformation and telepresence where avatars are controlled using blendshapes. We also perform thorough experiments showing that our method compares favourably to both volumetric approaches combined with implicit deformation and methods based on mesh deformation., Comment: 18 pages, 21 figures

Published
2022

6. 3D face reconstruction with dense landmarks

Author

Wood, Erroll, Baltrusaitis, Tadas, Hewitt, Charlie, Johnson, Matthew, Shen, Jingjing, Milosavljevic, Nikola, Wilde, Daniel, Garbin, Stephan, Raman, Chirag, Shotton, Jamie, Sharp, Toby, Stojiljkovic, Ivan, Cashman, Tom, and Valentin, Julien

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Landmarks often play a key role in face analysis, but many aspects of identity or expression cannot be represented by sparse landmarks alone. Thus, in order to reconstruct faces more accurately, landmarks are often combined with additional signals like depth images or techniques like differentiable rendering. Can we keep things simple by just using more landmarks? In answer, we present the first method that accurately predicts 10x as many landmarks as usual, covering the whole head, including the eyes and teeth. This is accomplished using synthetic training data, which guarantees perfect landmark annotations. By fitting a morphable model to these dense landmarks, we achieve state-of-the-art results for monocular 3D face reconstruction in the wild. We show that dense landmarks are an ideal signal for integrating face shape information across frames by demonstrating accurate and expressive facial performance capture in both monocular and multi-view scenarios. This approach is also highly efficient: we can predict dense landmarks and fit our 3D face model at over 150FPS on a single CPU thread. Please see our website: https://microsoft.github.io/DenseLandmarks/., Comment: ECCV 2022

Published
2022

11. Learning to Fit Morphable Models

Author

Choutas, Vasileios, Bogo, Federica, Shen, Jingjing, and Valentin, Julien

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Fitting parametric models of human bodies, hands or faces to sparse input signals in an accurate, robust, and fast manner has the promise of significantly improving immersion in AR and VR scenarios. A common first step in systems that tackle these problems is to regress the parameters of the parametric model directly from the input data. This approach is fast, robust, and is a good starting point for an iterative minimization algorithm. The latter searches for the minimum of an energy function, typically composed of a data term and priors that encode our knowledge about the problem's structure. While this is undoubtedly a very successful recipe, priors are often hand defined heuristics and finding the right balance between the different terms to achieve high quality results is a non-trivial task. Furthermore, converting and optimizing these systems to run in a performant way requires custom implementations that demand significant time investments from both engineers and domain experts. In this work, we build upon recent advances in learned optimization and propose an update rule inspired by the classic Levenberg-Marquardt algorithm. We show the effectiveness of the proposed neural optimizer on three problems, 3D body estimation from a head-mounted device, 3D body estimation from sparse 2D keypoints and face surface estimation from dense 2D landmarks. Our method can easily be applied to new model fitting problems and offers a competitive alternative to well-tuned 'traditional' model fitting pipelines, both in terms of accuracy and speed., Comment: ECCV 2022

Published
2021

13. Derivative-free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problems

Author

Xu, Zi, Wang, Ziqi, Shen, Jingjing, and Dai, Yuhong

Subjects
Mathematics - Optimization and Control, Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

In this paper, we study zeroth-order algorithms for nonconvex-concave minimax problems, which have attracted widely attention in machine learning, signal processing and many other fields in recent years. We propose a zeroth-order alternating randomized gradient projection (ZO-AGP) algorithm for smooth nonconvex-concave minimax problems, and its iteration complexity to obtain an $\varepsilon$-stationary point is bounded by $\mathcal{O}(\varepsilon^{-4})$, and the number of function value estimation is bounded by $\mathcal{O}(d_{x}+d_{y})$ per iteration. Moreover, we propose a zeroth-order block alternating randomized proximal gradient algorithm (ZO-BAPG) for solving block-wise nonsmooth nonconvex-concave minimax optimization problems, and the iteration complexity to obtain an $\varepsilon$-stationary point is bounded by $\mathcal{O}(\varepsilon^{-4})$ and the number of function value estimation per iteration is bounded by $\mathcal{O}(K d_{x}+d_{y})$. To the best of our knowledge, this is the first time that zeroth-order algorithms with iteration complexity gurantee are developed for solving both general smooth and block-wise nonsmooth nonconvex-concave minimax problems. Numerical results on data poisoning attack problem and distributed nonconvex sparse principal component analysis problem validate the efficiency of the proposed algorithms.

Published
2021

18. Classification of Bus Rapid Transit-Rail Transit Stations Using Complex Network Analysis

Author

Chen Ye, Gao Yue'er, and Shen Jingjing

Subjects
urban transportation, station classification, space-l method, brt-metro transit network, smart card data, xiamen, Geography (General), G1-922
Abstract

Mass traffic stations are key nodes in integrated traffic networks. The scientific definition of station types is of great significance for the refined management of stations and network operations. Using the Xiamen BRT track double-mode network system as the research object, this study analyzed the characteristics and hierarchical relationships of the station network and used complex network analysis to classify stations within the network. By constructing a Space-L-directed weighted complex network model based on station passenger flows, this study proposes four indicators: node strength, weighted betweenness centrality, weighted closeness centrality, and weighted PageRank. A system-clustering method was used to classify the stations. The results are as follows: (1) In the complex network model, the network sites can be divided into six types: core-hub type, local connection, network transition, strong network conduction, weak network conduction, and network-end. The important sites of the network are gathered at the center of Xiamin island, and the secondary sites of the network are distributed on the periphery of the regular spatial network. The important sites of the network outside the island are distributed in mature urban construction areas, and the secondary sites are scattered in preliminary urban construction development areas. (2) All transfer station types have different characteristics. Single-mode transfer stations have the ability to adjust network passenger flow and accommodate the transfer capacity of passenger flow between various regions of the city, whereas double-mode transfer stations have the ability to attract network passenger flow, and the station and other stations on the network. It is closely connected and has high accessibility; single-mode and double-mode combined transfer stations have the ability to promote network passenger flow, and have high passenger flow connection strength and global passenger flow influence ability. (3) Among the single-mode stations in the BRT-metro transit network, the BRT stations perform better than the metro stations. This feature is more obvious in the island area owing to the unbalanced development of the east and west areas of the island. The station classification index and method proposed in this study can reveal the characteristics of station networks, which can improve the operational efficiency of urban BRT-rail network systems.

Published
2023
Full Text
View/download PDF

19. Functional Layout of Railway Station Areas and between Stations Based on Passenger Flow: A Case Study of Xiamen

Author

Wan Li, Gao Yue'er, and Shen Jingjing

Subjects
station classification, functional layout, passenger flow characteristics, central place theory, xiamen, Geography (General), G1-922
Abstract

The comprehensive development of rail stations combining the Transit Oriented Development (TOD) concept with rail transit has been a research and practical hotspot in recent years. The functional layout within the station catchment area and between rail stations is an important basis for implementing TOD to guide the optimization of urban spatial structures, and passenger flow is a direct reflection of its functional layout effectiveness. Studying the adaptability of rail station functions through passenger flow characteristics has important practical significance for the comprehensive development of rail transit as the main body and for promoting the optimization of urban spatial structures. However, in existing research, there has been no unified conclusion on the mixed and differentiated functional layout of the station catchment area, which has often been advocated for the differentiation of functionality between stations, however, it mostly remains at the guiding ideology level. Therefore, this study categorized and assigned weights to the POI in the station catchment area and divided the Xiamen rail stations into three categories: stations with single dominant functions, stations with dual dominant functions, and stations with mixed functions. The relationship between the functional layout of stations and their corresponding spatial passenger flow was analyzed based on the OD volume of passenger flow between different functional types of stations and the distribution of OD in different distance segments. Based on the current functional layout characteristics and corresponding passenger flow characteristics of rail stations, further optimization strategies are proposed. The results show that: 1) Stations with high passenger flow are mostly those with single and dual-dominant functions, and the passenger flow of stations with mixed functions is not prominent; high-passenger flow stations are mostly those with high-level central functions, mainly distributed in urban central areas; and low-passenger flow stations are spatially concentrated, mostly concentrated near the end of the line. 2) The functional complementarities between spatial stations directly affect passenger flow OD volume, and the OD volume between stations of the same type is small. Meanwhile, that between stations of different types is large. 3) The OD ratio of each distance segment of the station is affected by the functional grade of the center and functional complementarity of the surrounding stations. The proportion of short-distance OD at stations in the central urban area is relatively large, whereas the proportion of long-distance OD at stations near the end of the line is relatively large. Finally, the causes of the current rail station functions and corresponding passenger flow characteristics were analyzed based on the internal logic of the occurrence and attraction of passenger trips with urban functions, central place theory, and the market rules of urban functional layout. Based on the requirements of urban spatial structure, the principle of comprehensive development function layout for rail stations is proposed, which includes "mixed basic functions, concentrated dominant functions" in the station influence area and "complementary differences, connecting points and forming axes" between stations.

Published
2023
Full Text
View/download PDF

22. The Phong Surface: Efficient 3D Model Fitting using Lifted Optimization

Author

Shen, Jingjing, Cashman, Thomas J., Ye, Qi, Hutton, Tim, Sharp, Toby, Bogo, Federica, Fitzgibbon, Andrew William, and Shotton, Jamie

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Realtime perceptual and interaction capabilities in mixed reality require a range of 3D tracking problems to be solved at low latency on resource-constrained hardware such as head-mounted devices. Indeed, for devices such as HoloLens 2 where the CPU and GPU are left available for applications, multiple tracking subsystems are required to run on a continuous, real-time basis while sharing a single Digital Signal Processor. To solve model-fitting problems for HoloLens 2 hand tracking, where the computational budget is approximately 100 times smaller than an iPhone 7, we introduce a new surface model: the `Phong surface'. Using ideas from computer graphics, the Phong surface describes the same 3D shape as a triangulated mesh model, but with continuous surface normals which enable the use of lifting-based optimization, providing significant efficiency gains over ICP-based methods. We show that Phong surfaces retain the convergence benefits of smoother surface models, while triangle meshes do not.

Published
2020

37. Learning to Fit Morphable Models

Author

Choutas, Vasileios, Bogo, Federica, Shen, Jingjing, Valentin, Julien, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Avidan, Shai, editor, Brostow, Gabriel, editor, Cissé, Moustapha, editor, Farinella, Giovanni Maria, editor, and Hassner, Tal, editor

Published
2022
Full Text
View/download PDF

50. Prediction of immune therapy efficacy and prognosis for advanced non-small cell lung cancer using peripheral blood circulation tumor DNA.

Author

YUAN Shengfang, WANG Bu, XIANG Baoli, ZHAO Jianqing, SHEN Jingjing, and ZHANG Zhihua

Subjects
NON-small-cell lung carcinoma, PERIPHERAL circulation, BLOOD circulation, PROGRESSION-free survival, CELL-free DNA
Abstract

Objective To explore the value of peripheral blood circulating DNA in predicting the efficacy and prognosis of immunotherapy for advanced non-small cell lung cancer. Method A retrospective study was conducted on 78 NSCLC patients who were admitted to the Respiratory and Critical Care Medicine Department of the First Affiliated Hospital of Hebei North University and were treated with tirelizumab for advanced driver gene negativity from January 2021 to December 2021. After 2 cycles of immunotherapy, the efficacy was evaluated according to the Solid Tumor Efficacy Evaluation Criteria (RECIST 1.1), including complete remission, partial remission, disease stability, and disease progression. CR and PR patients were defined as the experimental group (n = 48) Other patients were defined as the control group (n = 30), and the ctDNA levels in peripheral blood were measured before and after treatment in both groups. ROC curves were used to analyze the predictive value of periph eral blood ctDNA levels for achieving objective remission after immunotherapy.All patients were followed up and their progression free survival were calcutated.Using univariate and multivariate regression analysis identified the factors affecting the prognosis of patients after immunotherapy. Using Spearman correlation coefficient analyzed the correlation between ctDNA levels and PFS. Kalplan Meier survival curve were used for survival analysis. Result The peripheral blood ctDNA levels before and after treatment in the experimental group were (4.47 ± 1.21)ng/μL and (2.65 ± 1.14) ng/μL, respectively (t = 7.559,P < 0.001), while those in the control group were (4.54 ± 1.15)ng/mL and (4.29 ± 1.57) ng/μL, respectively (t = 0.699,P = 0.487). There was no statistically significant difference in peripheral blood ctDNA levels between the two groups before treatment (t = -0.25,P = 0.801). The peripheral blood ctDNA levels in the experimental group decreased compared to the control group after treatment (t = -5.35,P < 0.001). The ROC curve analysis showed that the area under the curve for predicting objective remission after immunotherapy based on peripheral blood ctDNA levels was 0.819, with a sensitivity of 81.3% and specificity of 80%. Peripheral blood ctDNA levels were negatively correlated with progression free survival (r = -0.784,P = 0.000). Single factor COX regression was used to analyze the clinical and pathological characteristics and ctDNA levels of enrolled patients, and the results showed that the maximum tumor diameter was greater than 5 cm (HR = 0.501, 95% CI: 6.731 ~ 35.567) Tumor stage IV (HR = 0.392,95% CI: 0.227 ~ 0.677), treatment approach (HR = 15.473,95% CI: 6.731 ~ 35.567), and ctDNA levels (HR = 4.657,95% CI: 3.182 ~ 6.555) are all influencing factors for PFS in advanced NSCLC patients after immunotherapy. Multiple factor analysis was conducted on the appeal indicators with statistical differences, and the results showed that treatment approach (HR = 2.981,95% CI:1.019 ~ 8.722) and peripheral blood ctDNA levels (HR = 3.918,95% CI: 2.619 ~ 5.861) It is an independent influencing factor of PFS in advanced NSCLC patients. The Kalplan Meier survival curve was used for analysis, and the results showed that the median PFS of the treatment effective group was 8.4 months, while the median PFS of the control group was 5.4 months. (χ² = 49.277, P = 0.000). Conclusion Immunotherapy combined with chemotherapy can enhance the ability to kill tumor cells, and peripheral blood ctDNA levels can evaluate the efficacy and prognosis of immunotherapy, which can be used to guide immunotherapy in advanced NSCLC patients. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results