Search

Your search keyword '"Tang, Xinyu"' showing total 1,022 results
Start Over Author "Tang, Xinyu"
1,022 results on '"Tang, Xinyu"'

1. YuLan: An Open-source Large Language Model

Author

Zhu, Yutao, Zhou, Kun, Mao, Kelong, Chen, Wentong, Sun, Yiding, Chen, Zhipeng, Cao, Qian, Wu, Yihan, Chen, Yushuo, Wang, Feng, Zhang, Lei, Li, Junyi, Wang, Xiaolei, Wang, Lei, Zhang, Beichen, Dong, Zican, Cheng, Xiaoxue, Chen, Yuhan, Tang, Xinyu, Hou, Yupeng, Ren, Qiangqiang, Pang, Xincheng, Xie, Shufang, Zhao, Wayne Xin, Dou, Zhicheng, Mao, Jiaxin, Lin, Yankai, Song, Ruihua, Xu, Jun, Chen, Xu, Yan, Rui, Wei, Zhewei, Hu, Di, Huang, Wenbing, Gao, Ze-Feng, Chen, Yueguo, Lu, Weizheng, and Wen, Ji-Rong

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract

Large language models (LLMs) have become the foundation of many applications, leveraging their extensive capabilities in processing and understanding natural language. While many open-source LLMs have been released with technical reports, the lack of training details hinders further research and development. This paper presents the development of YuLan, a series of open-source LLMs with $12$ billion parameters. The base model of YuLan is pre-trained on approximately $1.7$T tokens derived from a diverse corpus, including massive English, Chinese, and multilingual texts. We design a three-stage pre-training method to enhance YuLan's overall capabilities. Subsequent phases of training incorporate instruction-tuning and human alignment, employing a substantial volume of high-quality synthesized data. To facilitate the learning of complex and long-tail knowledge, we devise a curriculum-learning framework throughout across these stages, which helps LLMs learn knowledge in an easy-to-hard manner. YuLan's training is finished on Jan, 2024 and has achieved performance on par with state-of-the-art LLMs across various English and Chinese benchmarks. This paper outlines a comprehensive technical roadmap for developing LLMs from scratch. Our model and codes are available at https://github.com/RUC-GSAI/YuLan-Chat.

Published
2024

2. Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning

Author

Wang, Xiaolei, Tang, Xinyu, Zhao, Wayne Xin, and Wen, Ji-Rong

Subjects
Computer Science - Machine Learning, Computer Science - Computation and Language
Abstract

The emergence of in-context learning (ICL) is potentially attributed to two major abilities: task recognition (TR) for recognizing the task from demonstrations and utilizing pre-trained priors, and task learning (TL) for learning from demonstrations. However, relationships between the two abilities and how such relationships affect the emergence of ICL is unclear. In this paper, we take the first step by examining the pre-training dynamics of the emergence of ICL. With carefully designed metrics, we find that these two abilities are, in fact, competitive during pre-training. Moreover, we observe a strong negative correlation between the competition and ICL performance. Further analysis of common pre-training factors (i.e., model size, dataset size, and data curriculum) demonstrates possible ways to manage the competition. Based on these insights, we propose a simple yet effective method to better integrate these two abilities for ICL at inference time. Through adaptive ensemble learning, the performance of ICL can be significantly boosted, enabling two small models to outperform a larger one with more than twice the parameters. The code is available at https://github.com/RUCAIBox/Competitive-ICL., Comment: work in progress

Published
2024

3. Unleashing the Potential of Large Language Models as Prompt Optimizers: An Analogical Analysis with Gradient-based Model Optimizers

Author

Tang, Xinyu, Wang, Xiaolei, Zhao, Wayne Xin, Lu, Siyuan, Li, Yaliang, and Wen, Ji-Rong

Subjects
Computer Science - Computation and Language
Abstract

Automatic prompt optimization is an important approach to improving the performance of large language models (LLMs). Recent research demonstrates the potential of using LLMs as prompt optimizers, which can generate improved task prompts via iterative refinement. In this paper, we propose a novel perspective to investigate the design of LLM-based prompt optimizers, by drawing an analogy with gradient-based model optimizers. To connect these two approaches, we identify two pivotal factors in model parameter learning: update direction and update method. Focused on the two aspects, we borrow the theoretical framework and learning methods from gradient-based optimization to design improved strategies for LLM-based prompt optimizers. By systematically analyzing a rich set of improvement strategies, we further develop a capable Gradient-inspired LLM-based Prompt Optimizer called GPO. At each step, it first retrieves relevant prompts from the optimization trajectory as the update direction. Then, it utilizes the generation-based refinement strategy to perform the update, while controlling the edit distance through a cosine-based decay strategy. Extensive experiments demonstrate the effectiveness and efficiency of GPO. In particular, GPO brings an additional improvement of up to 56.8% on Big-Bench Hard and 55.3% on MMLU compared to baseline methods.

Published
2024

4. Private Fine-tuning of Large Language Models with Zeroth-order Optimization

Author

Tang, Xinyu, Panda, Ashwinee, Nasr, Milad, Mahloujifar, Saeed, and Mittal, Prateek

Subjects
Computer Science - Machine Learning, Computer Science - Computation and Language, Computer Science - Cryptography and Security
Abstract

Fine-tuning large pretrained models on private datasets may run the risk of violating privacy. Differential privacy is a framework for mitigating privacy risks by enforcing algorithmic stability. DP-SGD enables training models with private data in a privacy-preserving manner, but raises new obstacles in the form of performance loss and significant engineering challenges. We introduce DP-ZO, a new method for fine-tuning large language models that preserves the privacy of training data by privatizing zeroth-order optimization. A key insight into the design of our method is that the direction of the gradient in SPSA, the zeroth-order algorithm we use, is always random and the only information that depends on private data is the step size, i.e., a scalar. Therefore, we only need to privatize the scalar step size, which is memory-efficient. DP-ZO, which can be instantiated with either Laplace or Gaussian noise, provides a strong privacy-utility trade-off across different tasks, and model sizes, under conservative privacy budgets. One noteworthy result is that DP-ZO exhibits just $1.86\%$ performance degradation due to privacy at $(1,10^{-5})$-DP when fine-tuning OPT-66B on 1000 training samples from SQuAD.

Published
2024

5. Lutein and Zeaxanthin Enhance, Whereas Oxidation, Fructosylation, and Low pH Damage High-Density Lipoprotein Biological Functionality

Author

Zheng, Jingyuan, Hong, Brian V, Agus, Joanne K, Tang, Xinyu, Klebaner, Nola R, Chen, Siyu, Guo, Fei, Harvey, Danielle J, Lebrilla, Carlito B, and Zivkovic, Angela M

Subjects
Biochemistry and Cell Biology, Biological Sciences, Atherosclerosis, Prevention, Aetiology, 2.1 Biological and endogenous factors, HDL, carotenoids, chronic inflammation, oxidative stress, Biochemistry and cell biology, Medical biochemistry and metabolomics, Pharmacology and pharmaceutical sciences
Abstract

High-density lipoproteins (HDLs) are key regulators of cellular cholesterol homeostasis but are functionally altered in many chronic diseases. The factors that cause HDL functional loss in chronic disease are not fully understood. It is also unknown what roles antioxidant carotenoids play in protecting HDL against functional loss. The aim of this study was to measure how various disease-associated chemical factors including exposure to (1) Cu2+ ions, (2) hypochlorous acid (HOCL), (3) hydrogen peroxide (H2O2), (4) sialidase, (5) glycosidase, (6) high glucose, (7) high fructose, and (8) acidic pH, and the carotenoid antioxidants (9) lutein and (10) zeaxanthin affect HDL functionality. We hypothesized that some of the modifications would have stronger impacts on HDL particle structure and function than others and that lutein and zeaxanthin would improve HDL function. HDL samples were isolated from generally healthy human plasma and incubated with the corresponding treatments listed above. Cholesterol efflux capacity (CEC), lecithin-cholesterol acyl transferase (LCAT) activity, and paraoxonase-1 (PON1) activity were measured in order to determine changes in HDL functionality. Median HDL particle diameter was increased by acidic pH treatment and reduced by HOCl, high glucose, high fructose, N-glycosidase, and lutein treatments. Acidic pH, oxidation, and fructosylation all reduced HDL CEC, whereas lutein, zeaxanthin, and sialidase treatment improved HDL CEC. LCAT activity was reduced by acidic pH, oxidation, high fructose treatments, and lutein. PON1 activity was reduced by sialidase, glycosidase, H2O2, and fructose and improved by zeaxanthin and lutein treatment. These results show that exposure to oxidizing agents, high fructose, and low pH directly impairs HDL functionality related to cholesterol efflux and particle maturation, whereas deglycosylation impairs HDL antioxidant capacity. On the other hand, the antioxidants lutein and zeaxanthin improve or preserve both HDL cholesterol efflux and antioxidant activity but have no effect on particle maturation.

Published
2024

6. Price of Stability in Quality-Aware Federated Learning

Author

Yan, Yizhou, Tang, Xinyu, Huang, Chao, and Tang, Ming

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence
Abstract

Federated Learning (FL) is a distributed machine learning scheme that enables clients to train a shared global model without exchanging local data. The presence of label noise can severely degrade the FL performance, and some existing studies have focused on algorithm design for label denoising. However, they ignored the important issue that clients may not apply costly label denoising strategies due to them being self-interested and having heterogeneous valuations on the FL performance. To fill this gap, we model the clients' interactions as a novel label denoising game and characterize its equilibrium. We also analyze the price of stability, which quantifies the difference in the system performance (e.g., global model accuracy, social welfare) between the equilibrium outcome and the socially optimal solution. We prove that the equilibrium outcome always leads to a lower global model accuracy than the socially optimal solution does. We further design an efficient algorithm to compute the socially optimal solution. Numerical experiments on MNIST dataset show that the price of stability increases as the clients' data become noisier, calling for an effective incentive mechanism., Comment: Accepted to IEEE GLOBECOM 2023

Published
2023

7. Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation

Author

Tang, Xinyu, Shin, Richard, Inan, Huseyin A., Manoel, Andre, Mireshghallah, Fatemehsadat, Lin, Zinan, Gopi, Sivakanth, Kulkarni, Janardhan, and Sim, Robert

Subjects
Computer Science - Machine Learning, Computer Science - Cryptography and Security
Abstract

We study the problem of in-context learning (ICL) with large language models (LLMs) on private datasets. This scenario poses privacy risks, as LLMs may leak or regurgitate the private examples demonstrated in the prompt. We propose a novel algorithm that generates synthetic few-shot demonstrations from the private dataset with formal differential privacy (DP) guarantees, and show empirically that it can achieve effective ICL. We conduct extensive experiments on standard benchmarks and compare our algorithm with non-private ICL and zero-shot solutions. Our results demonstrate that our algorithm can achieve competitive performance with strong privacy levels. These results open up new possibilities for ICL with privacy protection for a broad range of applications.

Published
2023

8. Which is a better programming assistant? A comparative study between chatgpt and stack overflow

Author

Liu, Jinrun, Tang, Xinyu, Li, Linlin, Chen, Panpan, and Liu, Yepang

Subjects
Computer Science - Software Engineering
Abstract

Programmers often seek help from Q\&A websites to resolve issues they encounter during programming. Stack Overflow has been a widely used platform for this purpose for over a decade. Recently, revolutionary AI-powered platforms like ChatGPT have quickly gained popularity among programmers for their efficient and personalized programming assistance via natural language interactions. Both platforms can offer valuable assistance to programmers, but it's unclear which is more effective at enhancing programmer productivity. In our paper, we conducted an exploratory user study to compare the performance of Stack Overflow and ChatGPT in enhancing programmer productivity. Two groups of students with similar programming abilities were instructed to use the two platforms to solve three different types of programming tasks: algorithmic challenges, library usage, and debugging. During the experiments, we measured and compared the quality of code produced and the time taken to complete tasks for the two groups. The results show that, concerning code quality, ChatGPT outperforms Stack Overflow significantly in helping complete algorithmic and library-related tasks, while Stack Overflow is better for debugging tasks. Regarding task completion speed, the ChatGPT group is obviously faster than the Stack Overflow group in the algorithmic challenge, but the two groups have a similar performance in the other two tasks. Additionally, we conducted a post-experiment survey with the participants to understand how the platforms have helped them complete the programming tasks. We analyzed the questionnaires to summarize ChatGPT and Stack Overflow's strengths and weaknesses pointed out by the participants. By comparing these, we identified the reasons behind the two platforms' divergent performances in programming assistance.

Published
2023

9. Seasonal Factors Are Associated with Activities of Enzymes Involved in High-Density Lipoprotein Metabolism among Pregnant Females in Ghana

Author

Hong, Brian V, Zheng, Jack Jingyuan, Romo, Eduardo Z, Agus, Joanne K, Tang, Xinyu, Arnold, Charles D, Adu-Afarwuah, Seth, Lartey, Anna, Okronipa, Harriet, Dewey, Kathryn G, and Zivkovic, Angela M

Subjects
Reproductive Medicine, Biomedical and Clinical Sciences, Nutrition, Clinical Trials and Supportive Activities, Pediatric, Cardiovascular, Clinical Research, Atherosclerosis, Reproductive health and childbirth, Good Health and Well Being, cholesterol efflux capacity, CETP activity, Ghana, HDL, LCAT activity, PLTP activity, pregnancy, SQ-LNS, Animal production, Food sciences, Nutrition and dietetics
Abstract

BackgroundSmall-quantity lipid-based nutrient supplements (SQ-LNS) during pregnancy and postnatally were previously shown to improve high-density lipoprotein (HDL) cholesterol efflux capacity (CEC) and length in the children of supplemented mothers at 18 mo of age in the International Lipid-Based Nutrient Supplements (iLiNS) DYAD trial in Ghana. However, the effects of SQ-LNS on maternal HDL functionality during pregnancy are unknown.ObjectiveThe goal of this cross-sectional, secondary outcome analysis was to compare HDL function in mothers supplemented with SQ-LNS vs. iron and folic acid (IFA) during gestation.MethodsHDL CEC and the activities of 3 HDL-associated enzymes were analyzed in archived plasma samples (N = 197) from a subsample of females at 36 weeks of gestation enrolled in the iLiNS-DYAD trial in Ghana. Correlations between HDL function and birth outcomes, inflammatory markers C-reactive protein (CRP) and alpha-1-acid glycoprotein (AGP), and the effects of season were explored to determine the influence of these factors on HDL function in this cohort of pregnant females.ResultsThere were no statistically significant differences in HDL CEC, plasma lecithin-cholesterol acyltransferase (LCAT) activity, cholesteryl ester transfer protein (CETP) activity, or phospholipid transfer protein (PLTP) activity between mothers supplemented with SQ-LNS compared with IFA control, and no statistically significant relationships between maternal HDL function and childbirth outcomes. LCAT activity was negatively correlated with plasma AGP (R = -0.19, P = 0.007) and CRP (R = -0.28, P < 0.001), CETP and LCAT activity were higher during the dry season compared to the wet season, and PLTP activity was higher in the wet season compared to the dry season.ConclusionsMothers in Ghana supplemented with SQ-LNS compared with IFA during gestation did not have measurable differences in HDL functionality, and maternal HDL function was not associated with childbirth outcomes. However, seasonal factors and markers of inflammation were associated with HDL function, indicating that these factors had a stronger influence on HDL functionality than SQ-LNS supplementation during pregnancy.Clinical trial registry numberThe study was registered as NCT00970866. https://clinicaltrials.gov/study/NCT00970866.

Published
2023

11. Differentially Private Image Classification by Learning Priors from Random Processes

Author

Tang, Xinyu, Panda, Ashwinee, Sehwag, Vikash, and Mittal, Prateek

Subjects
Computer Science - Computer Vision and Pattern Recognition, Computer Science - Cryptography and Security, Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data. In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, a three-phase approach. We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets $\varepsilon \in [1, 8]$. In particular, we improve the previous best reported accuracy on CIFAR10 from $60.6 \%$ to $72.3 \%$ for $\varepsilon=1$.

Published
2023

12. Improving Conversational Recommendation Systems via Counterfactual Data Simulation

Author

Wang, Xiaolei, Zhou, Kun, Tang, Xinyu, Zhao, Wayne Xin, Pan, Fan, Cao, Zhao, and Wen, Ji-Rong

Subjects
Computer Science - Computation and Language, Computer Science - Information Retrieval
Abstract

Conversational recommender systems (CRSs) aim to provide recommendation services via natural language conversations. Although a number of approaches have been proposed for developing capable CRSs, they typically rely on sufficient training data for training. Since it is difficult to annotate recommendation-oriented dialogue datasets, existing CRS approaches often suffer from the issue of insufficient training due to the scarcity of training data. To address this issue, in this paper, we propose a CounterFactual data simulation approach for CRS, named CFCRS, to alleviate the issue of data scarcity in CRSs. Our approach is developed based on the framework of counterfactual data augmentation, which gradually incorporates the rewriting to the user preference from a real dialogue without interfering with the entire conversation flow. To develop our approach, we characterize user preference and organize the conversation flow by the entities involved in the dialogue, and design a multi-stage recommendation dialogue simulator based on a conversation flow language model. Under the guidance of the learned user preference and dialogue schema, the flow language model can produce reasonable, coherent conversation flows, which can be further realized into complete dialogues. Based on the simulator, we perform the intervention at the representations of the interacted entities of target users, and design an adversarial training method with a curriculum schedule that can gradually optimize the data augmentation strategy. Extensive experiments show that our approach can consistently boost the performance of several competitive CRSs, and outperform other data augmentation methods, especially when the training data is limited. Our code is publicly available at https://github.com/RUCAIBox/CFCRS., Comment: Accepted by KDD 2023. Code: https://github.com/RUCAIBox/CFCRS

Published
2023
Full Text
View/download PDF

13. Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models

Author

Wang, Xiaolei, Tang, Xinyu, Zhao, Wayne Xin, Wang, Jingyuan, and Wen, Ji-Rong

Subjects
Computer Science - Computation and Language, Computer Science - Information Retrieval
Abstract

The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs), which rely on natural language conversations to satisfy user needs. In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. It might over-emphasize the matching with the ground-truth items or utterances generated by human annotators, while neglecting the interactive nature of being a capable CRS. To overcome the limitation, we further propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators. Our evaluation approach can simulate various interaction scenarios between users and systems. Through the experiments on two publicly available CRS datasets, we demonstrate notable improvements compared to the prevailing evaluation protocol. Furthermore, we emphasize the evaluation of explainability, and ChatGPT showcases persuasive explanation generation for its recommendations. Our study contributes to a deeper comprehension of the untapped potential of LLMs for CRSs and provides a more flexible and easy-to-use evaluation framework for future research endeavors. The codes and data are publicly available at https://github.com/RUCAIBox/iEvaLM-CRS., Comment: Accepted by EMNLP 2023

Published
2023
Full Text
View/download PDF

14. Precision Nutrition and Cardiovascular Disease Risk Reduction: the Promise of High-Density Lipoproteins

Author

Hong, Brian V, Agus, Joanne K, Tang, Xinyu, Zheng, Jack Jingyuan, Romo, Eduardo Z, Lei, Susan, and Zivkovic, Angela M

Subjects
Biomedical and Clinical Sciences, Nutrition and Dietetics, Heart Disease, Genetics, Cardiovascular, Clinical Research, Complementary and Integrative Health, Prevention, Nutrition, Atherosclerosis, Aetiology, 2.2 Factors relating to the physical environment, Good Health and Well Being, Humans, Lipoproteins, HDL, Cardiovascular Diseases, Biomarkers, Nutritional Status, Risk Reduction Behavior, Precision nutrition, Cardiometabolic health, Cardiovascular disease, Personalized nutrition, Cardiorespiratory Medicine and Haematology, Cardiovascular System & Hematology, Cardiovascular medicine and haematology
Abstract

Purpose of reviewEmerging evidence supports the promise of precision nutritional approaches for cardiovascular disease (CVD) prevention. Here, we discuss current findings from precision nutrition trials and studies reporting substantial inter-individual variability in responses to diets and dietary components relevant to CVD outcomes. We highlight examples where early precision nutrition research already points to actionable intervention targets tailored to an individual's biology and lifestyle. Finally, we make the case for high-density lipoproteins (HDL) as a compelling next generation target for precision nutrition aimed at CVD prevention. HDL possesses complex structural features including diverse protein components, lipids, size distribution, extensive glycosylation, and interacts with the gut microbiome, all of which influence HDL's anti-inflammatory, antioxidant, and cholesterol efflux properties. Elucidating the nuances of HDL structure and function at an individual level may unlock personalized dietary and lifestyle strategies to optimize HDL-mediated atheroprotection and reduce CVD risk.Recent findingsRecent human studies have demonstrated that HDL particles are key players in the reduction of CVD risk. Our review highlights the role of HDL and the importance of personalized therapeutic approaches to improve their potential for reducing CVD risk. Factors such as diet, genetics, glycosylation, and gut microbiome interactions can modulate HDL structure and function at the individual level. We emphasize that fractionating HDL into size-based subclasses and measuring particle concentration are necessary to understand HDL biology and for developing the next generation of diagnostics and biomarkers. These discoveries underscore the need to move beyond a one-size-fits-all approach to HDL management. Precision nutrition strategies that account for personalized metabolic, genetic, and lifestyle data hold promise for optimizing HDL therapies and function to mitigate CVD risk more potently. While human studies show HDL play a key role in reducing CVD risk, recent findings indicate that factors such as diet, genetics, glycosylation, and gut microbes modulate HDL function at the individual level, underscoring the need for precision nutrition strategies that account for personalized variability to optimize HDL's potential for mitigating CVD risk.

Published
2023

15. A Survey of Large Language Models

Author

Zhao, Wayne Xin, Zhou, Kun, Li, Junyi, Tang, Tianyi, Wang, Xiaolei, Hou, Yupeng, Min, Yingqian, Zhang, Beichen, Zhang, Junjie, Dong, Zican, Du, Yifan, Yang, Chen, Chen, Yushuo, Chen, Zhipeng, Jiang, Jinhao, Ren, Ruiyang, Li, Yifan, Tang, Xinyu, Liu, Zikang, Liu, Peiyu, Nie, Jian-Yun, and Wen, Ji-Rong

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract

Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable AI algorithms for comprehending and grasping a language. As a major approach, language modeling has been widely studied for language understanding and generation in the past two decades, evolving from statistical language models to neural language models. Recently, pre-trained language models (PLMs) have been proposed by pre-training Transformer models over large-scale corpora, showing strong capabilities in solving various NLP tasks. Since researchers have found that model scaling can lead to performance improvement, they further study the scaling effect by increasing the model size to an even larger size. Interestingly, when the parameter scale exceeds a certain level, these enlarged language models not only achieve a significant performance improvement but also show some special abilities that are not present in small-scale language models. To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size. Recently, the research on LLMs has been largely advanced by both academia and industry, and a remarkable progress is the launch of ChatGPT, which has attracted widespread attention from society. The technical evolution of LLMs has been making an important impact on the entire AI community, which would revolutionize the way how we develop and use AI algorithms. In this survey, we review the recent advances of LLMs by introducing the background, key findings, and mainstream techniques. In particular, we focus on four major aspects of LLMs, namely pre-training, adaptation tuning, utilization, and capacity evaluation. Besides, we also summarize the available resources for developing LLMs and discuss the remaining issues for future directions., Comment: ongoing work; 124 pages, 946 citations

Published
2023

16. The role of FUT8‐catalyzed core fucosylation in Alzheimer's amyloid‐β oligomer‐induced activation of human microglia

Author

Jin, Lee‐Way, di Lucente, Jacopo, Mendiola, Ulises Ruiz, Tang, Xinyu, Zivkovic, Angela M, Lebrilla, Carlito B, and Maezawa, Izumi

Subjects
Biomedical and Clinical Sciences, Neurosciences, Brain Disorders, Dementia, Neurodegenerative, Alzheimer's Disease, Alzheimer's Disease including Alzheimer's Disease Related Dementias (AD/ADRD), Acquired Cognitive Impairment, Aging, 2.1 Biological and endogenous factors, Aetiology, Humans, Mice, Animals, Fucosyltransferases, Microglia, Amyloid beta-Peptides, Alzheimer Disease, Tumor Suppressor Protein p53, Induced Pluripotent Stem Cells, Cytokines, Catalysis, Alzheimer's, amyloid, fucosylation, fucosyltransferase, glycosylation, microglia, p53, Neurology & Neurosurgery
Abstract

Fucosylation, especially core fucosylation of N-glycans catalyzed by α1-6 fucosyltransferase (fucosyltransferase 8 or FUT8), plays an important role in regulating the peripheral immune system and inflammation. However, its role in microglial activation is poorly understood. Here we used human induced pluripotent stem cells-derived microglia (hiMG) as a model to study the role of FUT8-catalyzed core fucosylation in amyloid-β oligomer (AβO)-induced microglial activation, in view of its significant relevance to the pathogenesis of Alzheimer's disease (AD). HiMG responded to AβO and lipopolysaccharides (LPS) with a pattern of pro-inflammatory activation as well as enhanced core fucosylation and FUT8 expression within 24 h. Furthermore, we found increased FUT8 expression in both human AD brains and microglia isolated from 5xFAD mice, a model of AD-like cerebral amyloidosis. Inhibition of fucosylation in AβO-stimulated hiMG reduced the induction of pro-inflammatory cytokines, suppressed the activation of p38MAPK, and rectified phagocytic deficits. Specific inhibition of FUT8 by siRNA-mediated knockdown also reduced AβO-induced pro-inflammatory cytokines. We further showed that p53 binds to the two consensus binding sites in the Fut8 promoter, and that p53 knockdown abolished FUT8 overexpression in AβO-activated hiMG. Taken together, our evidence supports that FUT8-catalyzed core fucosylation is a signaling pathway required for AβO-induced microglia activation and that FUT8 is a component of the p53 signaling cascade regulating microglial behavior. Because microglia are a key driver of AD pathogenesis, our results suggest that microglial FUT8 could be an anti-inflammatory therapeutic target for AD.

Published
2023

17. A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization

Author

Panda, Ashwinee, Tang, Xinyu, Mahloujifar, Saeed, Sehwag, Vikash, and Mittal, Prateek

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Cryptography and Security
Abstract

An open problem in differentially private deep learning is hyperparameter optimization (HPO). DP-SGD introduces new hyperparameters and complicates existing ones, forcing researchers to painstakingly tune hyperparameters with hundreds of trials, which in turn makes it impossible to account for the privacy cost of HPO without destroying the utility. We propose an adaptive HPO method that uses cheap trials (in terms of privacy cost and runtime) to estimate optimal hyperparameters and scales them up. We obtain state-of-the-art performance on 22 benchmark tasks, across computer vision and natural language processing, across pretraining and finetuning, across architectures and a wide range of $\varepsilon \in [0.01,8.0]$, all while accounting for the privacy cost of HPO., Comment: ICML 2024

Published
2022

18. Human fasting modulates macrophage function and upregulates multiple bioactive metabolites that extend lifespan in Caenorhabditis elegans: a pilot clinical study

Author

Rhodes, Christopher H, Zhu, Chenghao, Agus, Joanne, Tang, Xinyu, Li, Qianyan, Engebrecht, JoAnne, and Zivkovic, Angela M

Subjects
Medical Biochemistry and Metabolomics, Biomedical and Clinical Sciences, Prevention, Aging, Clinical Research, Nutrition, Inflammatory and immune system, Good Health and Well Being, Animals, Humans, Caenorhabditis elegans, Longevity, Caenorhabditis elegans Proteins, Pilot Projects, Fasting, Macrophages, prolonged fasting, fasting, intermittent fasting, innate immune system, anti-inflammatory, immunology, lifespan, longevity, fasting mimetic, macrophage, Engineering, Medical and Health Sciences, Nutrition & Dietetics, Clinical sciences, Nutrition and dietetics
Abstract

BackgroundPeriodic prolonged fasting (PF) extends lifespan in model organisms and ameliorates multiple disease states both clinically and experimentally owing, in part, to its ability to modulate the immune system. However, the relationship between metabolic factors, immunity, and longevity during PF remains poorly characterized especially in humans.ObjectiveThis study aimed to observe the effects of PF in human subjects on the clinical and experimental markers of metabolic and immune health and uncover underlying plasma-borne factors that may be responsible for these effects.MethodsIn this rigorously controlled pilot study (ClinicalTrial.gov identifier, NCT03487679), 20 young males and females participated in a 3-d study protocol including assessments of 4 distinct metabolic states: 1) overnight fasted baseline state, 2) 2-h postprandial fed state, 3) 36-h fasted state, and 4) final 2-h postprandial re-fed state 12 h after the 36-h fasting period. Clinical and experimental markers of immune and metabolic health were assessed for each state along with comprehensive metabolomic profiling of participant plasma. Bioactive metabolites identified to be upregulated in circulation after 36 h of fasting were then assessed for their ability to mimic the effects of fasting in isolated human macrophage as well as the ability to extend lifespan in Caenorhabditis elegans.ResultsWe showed that PF robustly altered the plasma metabolome and conferred beneficial immunomodulatory effects on human macrophages. We also identified 4 bioactive metabolites that were upregulated during PF (spermidine, 1-methylnicotinamide, palmitoylethanolamide, and oleoylethanolamide) that could replicate these immunomodulatory effects. Furthermore, we found that these metabolites and their combination significantly extended the median lifespan of C. elegans by as much as 96%.ConclusionsThe results of this study reveal multiple functionalities and immunological pathways affected by PF in humans, identify candidates for the development of fasting mimetic compounds, and uncover targets for investigation in longevity research.

Published
2023

19. A single 36-h water-only fast vastly remodels the plasma lipidome

Author

Hong, Brian V, Rhodes, Christopher H, Agus, Joanne K, Tang, Xinyu, Zhu, Chenghao, Zheng, Jack Jingyuan, and Zivkovic, Angela M

Subjects
Medical Biochemistry and Metabolomics, Biomedical and Clinical Sciences, Nutrition, Aging, lipidomic, fasting, cardiometabolic health, lipids, free fatty acids, Cardiovascular medicine and haematology
Abstract

BackgroundProlonged fasting, characterized by restricting caloric intake for 24 h or more, has garnered attention as a nutritional approach to improve lifespan and support healthy aging. Previous research from our group showed that a single bout of 36-h water-only fasting in humans resulted in a distinct metabolomic signature in plasma and increased levels of bioactive metabolites, which improved macrophage function and lifespan in C. elegans.ObjectiveThis secondary outcome analysis aimed to investigate changes in the plasma lipidome associated with prolonged fasting and explore any potential links with markers of cardiometabolic health and aging.MethodWe conducted a controlled pilot study with 20 male and female participants (mean age, 27.5 ± 4.4 years; mean BMI, 24.3 ± 3.1 kg/m2) in four metabolic states: (1) overnight fasted (baseline), (2) 2-h postprandial fed state (fed), (3) 36-h fasted state (fasted), and (4) 2-h postprandial refed state 12 h after the 36-h fast (refed). Plasma lipidomic profiles were analyzed using liquid chromatography and electrospray ionization mass spectrometry.ResultsSeveral lipid classes, including lysophosphatidylcholine (LPC), lysophosphatidylethanolamine (LPE), phosphatidylethanolamine, and triacylglycerol were significantly reduced in the 36-h fasted state, while free fatty acids, ceramides, and sphingomyelin were significantly increased compared to overnight fast and fed states (P

Published
2023

20. Transcriptomic and glycomic analyses highlight pathway-specific glycosylation alterations unique to Alzheimer’s disease

Author

Tang, Xinyu, Tena, Jennyfer, Di Lucente, Jacopo, Maezawa, Izumi, Harvey, Danielle J, Jin, Lee-Way, Lebrilla, Carlito B, and Zivkovic, Angela M

Subjects
Biochemistry and Cell Biology, Bioinformatics and Computational Biology, Genetics, Biological Sciences, Neurodegenerative, Dementia, Alzheimer's Disease, Neurosciences, Acquired Cognitive Impairment, Aging, Alzheimer's Disease including Alzheimer's Disease Related Dementias (AD/ADRD), Brain Disorders, Aetiology, 2.1 Biological and endogenous factors, Neurological, Humans, Alzheimer Disease, Glycosylation, Transcriptome, Glycomics, MicroRNAs, Glycosyltransferases, Polysaccharides, Mannosyltransferases
Abstract

Glycosylation has been found to be altered in the brains of individuals with Alzheimer's disease (AD). However, it is unknown which specific glycosylation-related pathways are altered in AD dementia. Using publicly available RNA-seq datasets covering seven brain regions and including 1724 samples, we identified glycosylation-related genes ubiquitously changed in individuals with AD. Several differentially expressed glycosyltransferases found by RNA-seq were confirmed by qPCR in a different set of human medial temporal cortex (MTC) samples (n = 20 AD vs. 20 controls). N-glycan-related changes predicted by expression changes in these glycosyltransferases were confirmed by mass spectrometry (MS)-based N-glycan analysis in the MTC (n = 9 AD vs. 6 controls). About 80% of glycosylation-related genes were differentially expressed in at least one brain region of AD participants (adjusted p-values

Published
2023

23. Quantitative glycoproteomics of high-density lipoproteins

Author

Tang, Xinyu, Wong, Maurice, Tena, Jennyfer, Zhu, Chenghao, Rhodes, Christopher, Zhou, Qingwen, Vinjamuri, Anita, Oloumi, Armin, Boddu, Sucharita, Luxardi, Guillaume, Maverakis, Emanual, Lebrilla, Carlito B, and Zivkovic, Angela M

Subjects
Analytical Chemistry, Chemical Sciences, Prevention, Clinical Research, Atherosclerosis, Nutrition, Chemical sciences
Abstract

In this work, we developed a targeted glycoproteomic method to monitor the site-specific glycoprofiles and quantities of the most abundant HDL-associated proteins using Orbitrap LC-MS for (glyco)peptide target discovery and QqQ LC-MS for quantitative analysis. We conducted a pilot study using the workflow to determine whether HDL protein glycoprofiles are altered in healthy human participants in response to dietary glycan supplementation.

Published
2022

30. Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture

Author

Tang, Xinyu, Mahloujifar, Saeed, Song, Liwei, Shejwalkar, Virat, Nasr, Milad, Houmansadr, Amir, and Mittal, Prateek

Subjects
Computer Science - Cryptography and Security, Computer Science - Machine Learning
Abstract

Membership inference attacks are a key measure to evaluate privacy leakage in machine learning (ML) models. These attacks aim to distinguish training members from non-members by exploiting differential behavior of the models on member and non-member inputs. The goal of this work is to train ML models that have high membership privacy while largely preserving their utility; we therefore aim for an empirical membership privacy guarantee as opposed to the provable privacy guarantees provided by techniques like differential privacy, as such techniques are shown to deteriorate model utility. Specifically, we propose a new framework to train privacy-preserving models that induces similar behavior on member and non-member inputs to mitigate membership inference attacks. Our framework, called SELENA, has two major components. The first component and the core of our defense is a novel ensemble architecture for training. This architecture, which we call Split-AI, splits the training data into random subsets, and trains a model on each subset of the data. We use an adaptive inference strategy at test time: our ensemble architecture aggregates the outputs of only those models that did not contain the input sample in their training data. We prove that our Split-AI architecture defends against a large family of membership inference attacks, however, it is susceptible to new adaptive attacks. Therefore, we use a second component in our framework called Self-Distillation to protect against such stronger attacks. The Self-Distillation component (self-)distills the training dataset through our Split-AI ensemble, without using any external public datasets. Through extensive experiments on major benchmark datasets we show that SELENA presents a superior trade-off between membership privacy and utility compared to the state of the art.

Published
2021

35. Multi-Omic Analyses Reveal Bifidogenic Effect and Metabolomic Shifts in Healthy Human Cohort Supplemented With a Prebiotic Dietary Fiber Blend

Author

Kang, Jea Woo, Tang, Xinyu, Walton, Charles J, Brown, Mark J, Brewer, Rachel A, Maddela, Rolando L, Zheng, Jack Jingyuan, Agus, Joanne K, and Zivkovic, Angela M

Subjects
Medical Biochemistry and Metabolomics, Biomedical and Clinical Sciences, Nutrition and Dietetics, Cancer, Prevention, Clinical Trials and Supportive Activities, Clinical Research, Nutrition, Complementary and Integrative Health, 3.3 Nutrition and chemoprevention, 6.1 Pharmaceuticals, Evaluation of treatments and therapeutic interventions, Prevention of disease and conditions, and promotion of well-being, prebiotic, gut microbiome, bifidobacteria, indolepropionate, cholines, Agricultural Biotechnology, Nutrition and dietetics
Abstract

Dietary fiber, a nutrient derived mainly from whole grains, vegetables, fruits, and legumes, is known to confer a number of health benefits, yet most Americans consume less than half of the daily recommended amount. Convenience and affordability are key factors determining the ability of individuals to incorporate fiber-rich foods into their diet, and many Americans struggle to access, afford, and prepare foods rich in fiber. The objective of this clinical study was to test the changes in microbial community composition, human metabolomics, and general health markers of a convenient, easy to use prebiotic supplement in generally healthy young participants consuming a diet low in fiber. Twenty healthy adults participated in this randomized, placebo-controlled, double-blind, crossover study which was registered at clinicaltrials.gov as NCT03785860. During the study participants consumed 12 g of a prebiotic fiber supplement and 12 g of placebo daily as a powder mixed with water as part of their habitual diet in randomized order for 4 weeks, with a 4-week washout between treatment arms. Fecal microbial DNA was extracted and sequenced by shallow shotgun sequencing on an Illumina NovaSeq. Plasma metabolites were detected using liquid chromatography-mass spectrometry with untargeted analysis. The phylum Actinobacteria, genus Bifidobacterium, and several Bifidobacterium species (B. bifidum, B. adolescentis, B. breve, B. catenulatum, and B. longum) significantly increased after prebiotic supplementation when compared to the placebo. The abundance of genes associated with the utilization of the prebiotic fiber ingredients (sacA, xfp, xpk) and the production of acetate (poxB, ackA) significantly changed with prebiotic supplementation. Additionally, the abundance of genes associated with the prebiotic utilization (xfp, xpk), acetate production (ackA), and choline to betaine oxidation (gbsB) were significantly correlated with changes in the abundance of the genus Bifidobacterium in the prebiotic group. Plasma concentrations of the bacterially produced metabolite indolepropionate significantly increased. The results of this study demonstrate that an easy to consume, low dose (12 g) of a prebiotic powder taken daily increases the abundance of beneficial bifidobacteria and the production of health-promoting bacteria-derived metabolites in healthy individuals with a habitual low-fiber diet.Clinical trial registrationwww.clinicaltrials.gov/, identifier: NCT03785860.

Published
2022

36. Glycosylation alterations in serum of Alzheimer's disease patients show widespread changes in N‐glycosylation of proteins related to immune function, inflammation, and lipoprotein metabolism

Author

Tena, Jennyfer, Tang, Xinyu, Zhou, Qingwen, Harvey, Danielle, Barajas‐Mendoza, Maria, Jin, Lee‐Way, Maezawa, Izumi, Zivkovic, Angela M, and Lebrilla, Carlito B

Subjects
Biological Psychology, Biomedical and Clinical Sciences, Neurosciences, Psychology, Aging, Dementia, Alzheimer's Disease including Alzheimer's Disease Related Dementias (AD/ADRD), Prevention, Acquired Cognitive Impairment, Alzheimer's Disease, Neurodegenerative, Clinical Research, Brain Disorders, 4.1 Discovery and preclinical testing of markers and technologies, Detection, screening and diagnosis, Alzheimer's disease, blood-based biomarker, mass spectrometry, site-specific glycosylation, blood‐based biomarker, site‐specific glycosylation, Genetics, Biological psychology
Abstract

IntroductionThere is an increased need for the development of novel blood-based biomarkers for early detection, prevention, or intervention in Alzheimer's disease (AD). This study sought to determine whether serum glycopeptide analysis holds potential for identifying novel diagnostics and prognostics of AD.MethodsThe study involved 195 participants, including 96 patients with an AD diagnosis and 99 controls with no cognitive deficit. Utilizing a validated analytical mass spectrometry method, we monitored the site-specific glycosylation of 52 serum glycoproteins.ResultsPartial least-squares discriminant analysis revealed that changes in overall sialylation and fucosylation of serum glycoproteins may be indicators of an AD disease state. Loss of fucosylation of immunoglobulin G1 (IgG1) and IgG2 was indicative of AD diagnosis. Individual glycopeptide analysis found separation between the AD patients and controls on complement proteins and apolipoprotein B.DiscussionThe results of this study suggest that serum glycoprofiling may be a promising approach for biomarker discovery.

Published
2022

37. High-Density Lipoprotein Changes in Alzheimer’s Disease Are APOE Genotype-Specific

Author

Hong, Brian V, Zheng, Jingyuan, Agus, Joanne K, Tang, Xinyu, Lebrilla, Carlito B, Jin, Lee-Way, Maezawa, Izumi, Erickson, Kelsey, Harvey, Danielle J, DeCarli, Charles S, Mungas, Dan M, Olichney, John M, Farias, Sarah T, and Zivkovic, Angela M

Subjects
Biochemistry and Cell Biology, Biological Sciences, Alzheimer's Disease including Alzheimer's Disease Related Dementias (AD/ADRD), Clinical Research, Alzheimer's Disease, Brain Disorders, Aging, Cardiovascular, Atherosclerosis, Dementia, Neurodegenerative, Acquired Cognitive Impairment, Neurosciences, 2.1 Biological and endogenous factors, Aetiology, Neurological, Alzheimer's disease, APOE, cholesterol efflux capacity, HDL, LCAT, Alzheimer’s disease, Biochemistry and cell biology, Pharmacology and pharmaceutical sciences, Medicinal and biomolecular chemistry
Abstract

High-density lipoproteins (HDL) play a critical role in cholesterol homeostasis. Apolipoprotein E (APOE), particularly the E4 allele, is a significant risk factor for Alzheimer's disease but is also a key HDL-associated protein involved in lipid transport in both the periphery and central nervous systems. The objective was to determine the influence of the APOE genotype on HDL function and size in the context of Alzheimer's disease. HDL from 194 participants (non-demented controls, mild cognitive impairment, and Alzheimer's disease dementia) were isolated from the plasma. The HDL cholesterol efflux capacity (CEC), lecithin-cholesterol acyltransferase (LCAT) activity, and particle diameter were measured. Neuropsychological test scores, clinical dementia rating, and magnetic resonance imaging scores were used to determine if cognition is associated with HDL function and size. HDL CEC and LCAT activity were reduced in APOE3E4 carriers compared to APOE3E3 carriers, regardless of diagnosis. In APOE3E3 carriers, CEC and LCAT activity were lower in patients. In APOE3E4 patients, the average particle size was lower. HDL LCAT activity and particle size were positively correlated with the neuropsychological scores and negatively correlated with the clinical dementia rating. We provide evidence for the first time of APOE genotype-specific alterations in HDL particles in Alzheimer's disease and an association between HDL function, size, and cognitive function.

Published
2022

38. Stockyard Storage Space Allocation in Dry Bulk Terminals Considering Mist Cannons and Energy Expenditure

Author

Tang, Xinyu, Schulte, Frederik, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Daduna, Joachim R., editor, Liedtke, Gernot, editor, Shi, Xiaoning, editor, and Voß, Stefan, editor

Published
2023
Full Text
View/download PDF

41. Isolation of HDL by sequential flotation ultracentrifugation followed by size exclusion chromatography reveals size-based enrichment of HDL-associated proteins.

Author

Zheng, Jack Jingyuan, Agus, Joanne K, Hong, Brian V, Tang, Xinyu, Rhodes, Christopher H, Houts, Hannah E, Zhu, Chenghao, Kang, Jea Woo, Wong, Maurice, Xie, Yixuan, Lebrilla, Carlito B, Mallick, Emily, Witwer, Kenneth W, and Zivkovic, Angela M

Subjects
Atherosclerosis, Cardiovascular
Abstract

High-density lipoprotein (HDL) particles have multiple beneficial and cardioprotective roles, yet our understanding of their full structural and functional repertoire is limited due to challenges in separating HDL particles from contaminating plasma proteins and other lipid-carrying particles that overlap HDL in size and/or density. Here we describe a method for isolating HDL particles using a combination of sequential flotation density ultracentrifugation and fast protein liquid chromatography with a size exclusion column. Purity was visualized by polyacrylamide gel electrophoresis and verified by proteomics, while size and structural integrity were confirmed by transmission electron microscopy. This HDL isolation method can be used to isolate a high yield of purified HDL from a low starting plasma volume for functional analyses. This method also enables investigators to select their specific HDL fraction of interest: from the least inclusive but highest purity HDL fraction eluting in the middle of the HDL peak, to pooling all of the fractions to capture the breadth of HDL particles in the original plasma sample. We show that certain proteins such as lecithin cholesterol acyltransferase (LCAT), phospholipid transfer protein (PLTP), and clusterin (CLUS) are enriched in large HDL particles whereas proteins such as alpha-2HS-glycoprotein (A2HSG), alpha-1 antitrypsin (A1AT), and vitamin D binding protein (VDBP) are enriched or found exclusively in small HDL particles.

Published
2021

43. Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning

Author

Fan, Lifeng, Wang, Wenguan, Huang, Siyuan, Tang, Xinyu, and Zhu, Song-Chun

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

This paper addresses a new problem of understanding human gaze communication in social videos from both atomic-level and event-level, which is significant for studying human social interactions. To tackle this novel and challenging problem, we contribute a large-scale video dataset, VACATION, which covers diverse daily social scenes and gaze communication behaviors with complete annotations of objects and human faces, human attention, and communication structures and labels in both atomic-level and event-level. Together with VACATION, we propose a spatio-temporal graph neural network to explicitly represent the diverse gaze interactions in the social scenes and to infer atomic-level gaze communication by message passing. We further propose an event network with encoder-decoder structure to predict the event-level gaze communication. Our experiments demonstrate that the proposed model improves various baselines significantly in predicting the atomic-level and event-level gaze, Comment: Accepted by ICCV 2019

Published
2019

Catalog

Books, media, physical & digital resources
See catalog results