Search

Your search keyword '"Guo, Shangqi"' showing total 28 results
Start Over Author "Guo, Shangqi"
28 results on '"Guo, Shangqi"'

1. Biologically Plausible Variational Policy Gradient with Spiking Recurrent Winner-Take-All Networks

Author

Yang, Zhile, Guo, Shangqi, Fang, Ying, and Liu, Jian K.

Subjects
Computer Science - Neural and Evolutionary Computing, Computer Science - Machine Learning, Quantitative Biology - Neurons and Cognition
Abstract

One stream of reinforcement learning research is exploring biologically plausible models and algorithms to simulate biological intelligence and fit neuromorphic hardware. Among them, reward-modulated spike-timing-dependent plasticity (R-STDP) is a recent branch with good potential in energy efficiency. However, current R-STDP methods rely on heuristic designs of local learning rules, thus requiring task-specific expert knowledge. In this paper, we consider a spiking recurrent winner-take-all network, and propose a new R-STDP method, spiking variational policy gradient (SVPG), whose local learning rules are derived from the global policy gradient and thus eliminate the need for heuristic designs. In experiments of MNIST classification and Gym InvertedPendulum, our SVPG achieves good training performance, and also presents better robustness to various kinds of noises than conventional methods., Comment: Accepted to BMVC 2022

Published
2022

3. Adjacency constraint for efficient hierarchical reinforcement learning

Author

Zhang, Tianren, Guo, Shangqi, Tan, Tian, Hu, Xiaolin, and Chen, Feng

Subjects
Computer Science - Machine Learning
Abstract

Goal-conditioned Hierarchical Reinforcement Learning (HRL) is a promising approach for scaling up reinforcement learning (RL) techniques. However, it often suffers from training inefficiency as the action space of the high-level, i.e., the goal space, is large. Searching in a large goal space poses difficulty for both high-level subgoal generation and low-level policy learning. In this paper, we show that this problem can be effectively alleviated by restricting the high-level action space from the whole goal space to a $k$-step adjacent region of the current state using an adjacency constraint. We theoretically prove that in a deterministic Markov Decision Process (MDP), the proposed adjacency constraint preserves the optimal hierarchical policy, while in a stochastic MDP the adjacency constraint induces a bounded state-value suboptimality determined by the MDP's transition structure. We further show that this constraint can be practically implemented by training an adjacency network that can discriminate between adjacent and non-adjacent subgoals. Experimental results on discrete and continuous control tasks including challenging simulated robot locomotion and manipulation tasks show that incorporating the adjacency constraint significantly boosts the performance of state-of-the-art goal-conditioned HRL approaches., Comment: Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence. DOI: 10.1109/TPAMI.2022.3192418. arXiv admin note: substantial text overlap with arXiv:2006.11485

Published
2021

4. Subjective Learning for Open-Ended Data

Author

Zhang, Tianren, Jiang, Yizhou, Su, Xin, Guo, Shangqi, and Chen, Feng

Subjects
Computer Science - Machine Learning
Abstract

Conventional supervised learning typically assumes that the learning task can be solved by learning a single function since the data is sampled from a fixed distribution. However, this assumption is invalid in open-ended environments where no task-level data partitioning is available. In this paper, we present a novel supervised learning framework of learning from open-ended data, which is modeled as data implicitly sampled from multiple domains with the data in each domain obeying a domain-specific target function. Since different domains may possess distinct target functions, open-ended data inherently requires multiple functions to capture all its input-output relations, rendering training a single global model problematic. To address this issue, we devise an Open-ended Supervised Learning (OSL) framework, of which the key component is a subjective function that allocates the data among multiple candidate models to resolve the "conflict" between the data from different domains, exhibiting a natural hierarchy. We theoretically analyze the learnability and the generalization error of OSL, and empirically validate its efficacy in both open-ended regression and classification tasks.

Published
2021

6. CRIL: Continual Robot Imitation Learning via Generative and Prediction Model

Author

Gao, Chongkai, Gao, Haichuan, Guo, Shangqi, Zhang, Tianren, and Chen, Feng

Subjects
Computer Science - Robotics, Computer Science - Artificial Intelligence
Abstract

Imitation learning (IL) algorithms have shown promising results for robots to learn skills from expert demonstrations. However, they need multi-task demonstrations to be provided at once for acquiring diverse skills, which is difficult in real world. In this work we study how to realize continual imitation learning ability that empowers robots to continually learn new tasks one by one, thus reducing the burden of multi-task IL and accelerating the process of new task learning at the same time. We propose a novel trajectory generation model that employs both a generative adversarial network and a dynamics-aware prediction model to generate pseudo trajectories from all learned tasks in the new task learning process. Our experiments on both simulation and real-world manipulation tasks demonstrate the effectiveness of our method.

Published
2021

7. Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learning

Author

Zhang, Tianren, Guo, Shangqi, Tan, Tian, Hu, Xiaolin, and Chen, Feng

Subjects
Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

Goal-conditioned hierarchical reinforcement learning (HRL) is a promising approach for scaling up reinforcement learning (RL) techniques. However, it often suffers from training inefficiency as the action space of the high-level, i.e., the goal space, is often large. Searching in a large goal space poses difficulties for both high-level subgoal generation and low-level policy learning. In this paper, we show that this problem can be effectively alleviated by restricting the high-level action space from the whole goal space to a $k$-step adjacent region of the current state using an adjacency constraint. We theoretically prove that the proposed adjacency constraint preserves the optimal hierarchical policy in deterministic MDPs, and show that this constraint can be practically implemented by training an adjacency network that can discriminate between adjacent and non-adjacent subgoals. Experimental results on discrete and continuous control tasks show that incorporating the adjacency constraint improves the performance of state-of-the-art HRL approaches in both deterministic and stochastic environments., Comment: Accepted by NeurIPS 2020

Published
2020

8. Subjectivity Learning Theory towards Artificial General Intelligence

Author

Su, Xin, Guo, Shangqi, and Chen, Feng

Subjects
Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

The construction of artificial general intelligence (AGI) was a long-term goal of AI research aiming to deal with the complex data in the real world and make reasonable judgments in various cases like a human. However, the current AI creations, referred to as "Narrow AI", are limited to a specific problem. The constraints come from two basic assumptions of data, which are independent and identical distributed samples and single-valued mapping between inputs and outputs. We completely break these constraints and develop the subjectivity learning theory for general intelligence. We assign the mathematical meaning for the philosophical concept of subjectivity and build the data representation of general intelligence. Under the subjectivity representation, then the global risk is constructed as the new learning goal. We prove that subjectivity learning holds a lower risk bound than traditional machine learning. Moreover, we propose the principle of empirical global risk minimization (EGRM) as the subjectivity learning process in practice, establish the condition of consistency, and present triple variables for controlling the total risk bound. The subjectivity learning is a novel learning theory for unconstrained real data and provides a path to develop AGI.

Published
2019

10. Numerical Simulation and Experimental Study of Carbon Fiber-Reinforced Polymer Single-Bar Extrusion Anchorage Structure.

Author

Zhu, Wanxu, Xiong, Chengyang, Cheng, Boxuan, Shen, Quanxi, Cheng, Hongbin, and Guo, Shangqi

Subjects
CARBON fiber-reinforced plastics, BEARING capacity (Bridges), RADIAL stresses, DEAD loads (Mechanics), FINITE element method
Abstract

The reliable anchorage of carbon fiber-reinforced polymer (CFRP) tendons is a critical issue influencing the stable bearing capacity of bridge cables. This study introduces a novel CFRP single-strand extrusion anchoring structure, where the strand is compressed at its end. By integrating this with internal cone filler wrapping, we create a CFRP multi-strand cable composite anchoring system. This innovative design not only minimizes the overall dimensions of the anchoring system but also significantly improves its anchoring efficiency coefficient. An axisymmetric model was developed using ANSYS finite element software. The radial stress distribution and anchorage efficiency coefficient in the anchorage zone of Φ7 CFRP bar and Φ13.6 extrusion die were analyzed with varying parameters, such as chamfering, outer diameter, and length of the extrusion sleeve, and were validated through static load anchorage tests. The results indicate that the highest anchoring efficiency is achieved when four extrusion sleeves with a chamfer angle of 5°, an outer diameter of Φ14.4, and a length of 15 mm are connected in series, reaching a coefficient of 61.04%. Furthermore, this study proposes an anchorage structure where multiple extrusion sleeves are connected in series and sequentially compressed to overcome the limitations of increasing anchorage length for enhancing the anchorage coefficient. The test results demonstrate that with equal total anchorage length, connecting four 15 mm extrusion sleeves in series enhances the anchorage efficiency coefficient by 24.98% compared to a single 60 mm extrusion sleeve structure. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

12. Suitability evaluation on construction of planned urban areas in Guilin based on AHP method and GIS platform.

Author

WANG Xinwei, ZHANG Lili, GUO Shangqi, YE Zongda, and JIANG Fan

Subjects
CITIES & towns, GEOGRAPHIC information systems, BEARING capacity of soils, BUILDING sites, ANALYTIC hierarchy process, SPELEOTHEMS, IMPACT craters
Abstract

The engineering construction of karst city is often affected by karst geological hazards. Guilin City is a typical representative of karst city; therefore, we should consider the influence of karst hazards when analyzing the suitability for engineering construction in planned urban areas of Guilin. In this study, we established a method for evaluating the construction suitability based on the analytic hierarchy process (AHP) on the geographic information system (GIS) platform. Firstly, the landform of planned urban areas in Guilin is mainly composed of peak-cluster depression, peak-forest valley, peak-forest plain, isolated peak plain, karst-ridge valley, karst-hill plain, low mountain with steep hillslope, hill, hill with gentle slope, and undulating hills. The areas intensely developed with karst are mainly located along the banks of the Lijiang River and in the Lingui district. The areas with karst development at a medium level are located in the middle and southwestern parts of Guilin City and on the periphery of the Lijiang River banks. The areas with weak karst development are mainly located at the Yanshan district, Dingjiang town, and southwestern Lingui district. The groundwater types of the study area are mainly composed of carbonate karst water, impure carbonate cave-fissure water, and clastic bedrock fissure water. For the suitability evaluation of engineering construction in planned urban areas of Guilin, the topographic slope, geomorphic unit, bearing capacity of soil foundation, bearing capacity of rock foundation, special soil distribution, soil thickness, soil structure, groundwater depth, surface water system, karst influence, and fault factors are selected as evaluation indexes, all of which have passed the consistency test of judgment matrix. The weight of these eleven evaluation indexes was calculated by AHP. Then, the engineering construction suitability index (P) was computed from multi hierarchy evaluation with AHP. The P values can be classified as P≥5.00, 2.50≤P<5.00, 1.00≤P<2.50, and P<1.00, which may respectively fall into the category of suitable area, relatively suitable area, poorly suitable area, and unsuitable area for engineering construction. The zoning of planned unban areas in Guilin was visualized based on the suitability for engineering construction on the GIS spatial platform. Finally, in terms of the zoning based on suitability for engineering construction, six sites were chosen and were verified that the zoning by AHP method is consistent with that by field engineering drilling surveys. The consistency may indicate the reliability of APH method with GIS platform. Main conclusions can be drawn as follows,(1) The impact indexes of suitability for engineering construction of the study area include topography (slope degree and geomorphic unit), engineering geological conditions (bearing capacity of soil foundation, bearing capacity of rock foundation, special soil distribution, soil thickness and soil structure), hydrological conditions (groundwater depth and surface water system), and unfavorable geological condition (karst influence and fault factors). (2) Planned urban areas in Guilin are generally suitable for engineering construction, because suitable and relatively suitable areas account for 3.1% and 48.3% of the total area respectively, which can meet the land use needs of recent urban development planning and major infrastructure projects. However, the poorly suitable areas account for 11.4%, which are unsuitable for large-scale engineering construction and require necessary engineering treatment in practical construction. Unsuitable areas account for 37.2%, mostly located at medium-sized mountain, peak-cluster, and isolated peak. The site selection of major construction should be avoided in the unsuitable area. (3) If the major construction site is selected in a poorly suitable area, measures should be taken to prevent and control the existing and induced geological disasters and uneven settlement of foundations. The unsuitable area is proposed to be planned as the ecological land, which can serve as the ecological barrier for Guilin. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

13. Partial Consistency for Stabilizing Undiscounted Reinforcement Learning

Author

Gao, Haichuan, Yang, Zhile, Tan, Tian, Zhang, Tianren, Ren, Jinsheng, Sun, Pengfei, Guo, Shangqi, and Chen, Feng

Abstract

Undiscounted return is an important setup in reinforcement learning (RL) and characterizes many real-world problems. However, optimizing an undiscounted return often causes training instability. The causes of this instability problem have not been analyzed in-depth by existing studies. In this article, this problem is analyzed from the perspective of value estimation. The analysis result indicates that the instability originates from transient traps that are caused by inconsistently selected actions. However, selecting one consistent action in the same state limits exploration. For balancing exploration effectiveness and training stability, a novel sampling method called last-visit sampling (LVS) is proposed to ensure that a part of actions is selected consistently in the same state. The LVS method decomposes the state-action value into two parts, i.e., the last-visit (LV) value and the revisit value. The decomposition ensures that the LV value is determined by consistently selected actions. We prove that the LVS method can eliminate transient traps while preserving optimality. Also, we empirically show that the method can stabilize the training processes of five typical tasks, including vision-based navigation and manipulation tasks.

Published
2023
Full Text
View/download PDF

21. Adjacency Constraint for Efficient Hierarchical Reinforcement Learning

Author

Zhang, Tianren, Guo, Shangqi, Tan, Tian, Hu, Xiaolin, and Chen, Feng

Abstract

Goal-conditioned Hierarchical Reinforcement Learning (HRL) is a promising approach for scaling up reinforcement learning (RL) techniques. However, it often suffers from training inefficiency as the action space of the high-level, i.e., the goal space, is large. Searching in a large goal space poses difficulty for both high-level subgoal generation and low-level policy learning. In this article, we show that this problem can be effectively alleviated by restricting the high-level action space from the whole goal space to a $k$k-step adjacent region of the current state using an adjacency constraint. We theoretically prove that in a deterministic Markov Decision Process (MDP), the proposed adjacency constraint preserves the optimal hierarchical policy, while in a stochastic MDP the adjacency constraint induces a bounded state-value suboptimality determined by the MDP's transition structure. We further show that this constraint can be practically implemented by training an adjacency network that can discriminate between adjacent and non-adjacent subgoals. Experimental results on discrete and continuous control tasks including challenging simulated robot locomotion and manipulation tasks show that incorporating the adjacency constraint significantly boosts the performance of state-of-the-art goal-conditioned HRL approaches.

Published
2023
Full Text
View/download PDF

22. LINC01615 activates ZEB2 through competitively binding with miR-3653-3p to promote the carcinogenesis of colon cancer cells.

Author

Hu, Zhen, Yang, Chong, Guo, Shangqi, Li, Yiqun, and Li, Yaoping

Subjects
COLON cancer, CANCER cells, ZINC-finger proteins, CARCINOGENESIS, POLYMERASE chain reaction
Abstract

As a newly discovered cancer-related molecule, we explored the unreported mechanism of LINC01615 intervention in colon cancer. LINC01615 expression in clinical samples and cells were detected. Effects of LINC01615 silencing/overexpression on the malignant development of colon cancer cells were analyzed through cell function experiments. Changes at the level of molecular biology were detected by quantitative real-time polymerase chain reaction and Western blot. Bioinformatics analysis and dual luciferase reporter assay were involved in the display and verification of targeted binding sequences. The rescue tests and correlation analysis examined the relationship among LINC01615, miR-3653-3p and zinc finger E-box binding homeobox 2 (ZEB2) in colon cancer cells. The xenograft experiment and immunohistochemistry were performed to verify these results. TCGA suggested that LINC01615 was high-expressed in colon cancer, as verified in clinical and cell samples, and patients with LINC01615 overexpression suffered from a poor prognosis. Silent LINC01615 blocked the malignant development of colon cancer cells through regulating related genes expressions, while overexpressed LINC01615 had the opposite effect. LINC01615, which was targeted by miR-3653-3p, partially offset the inhibitory effect of miR-3653-3p on colon cancer cells. The downstream target gene ZEB2 of miR-3653-3p was high-expressed in colon cancer. MiR-3653-3p was negatively correlated with LINC01615 or ZEB2, while LINC01615 was positively correlated with ZEB2. Therefore, LINC01615 induced ZEB2 up-regulation, while miR-3653-3p reduced ZEB2 level. The results of in vivo studies were consistent with cell experiments. LINC01615 competitively binds with miR-3653-3p to regulate ZEB2 and promote canceration of colon cancer cells. [ABSTRACT FROM AUTHOR]

Published
2022
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results