Search

Previous deep multi-agent reinforcement learning (MARL) algorithms have achieved impressive results, typically in homogeneous scenarios. However, heterogeneous scenarios are also very common and usually harder to solve. In this paper, we mainly discuss cooperative heterogeneous MARL problems in Starcraft Multi-Agent Challenges (SMAC) environment. We firstly define and describe the heterogeneous problems in SMAC. In order to comprehensively reveal and study the problem, we make new maps added to the original SMAC maps. We find that baseline algorithms fail to perform well in those heterogeneous maps. To address this issue, we propose the Grouped Individual-Global-Max Consistency (GIGM) and a novel MARL algorithm, Grouped Hybrid Q Learning (GHQ). GHQ separates agents into several groups and keeps individual parameters for each group, along with a novel hybrid structure for factorization. To enhance coordination between groups, we maximize the Inter-group Mutual Information (IGMI) between groups' trajectories. Experiments on original and new heterogeneous maps show the fabulous performance of GHQ compared to other state-of-the-art algorithms.

Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but demand massive GPU resources for training. Lowering the threshold for LLMs training would encourage greater participation from researchers, benefiting both academia and society. While existing approaches have focused on parameter-efficient fine-tuning, which tunes or adds a small number of parameters, few have addressed the challenge of tuning the full parameters of LLMs with limited resources. In this work, we propose a new optimizer, LOw-Memory Optimization (LOMO), which fuses the gradient computation and the parameter update in one step to reduce memory usage. By integrating LOMO with existing memory saving techniques, we reduce memory usage to 10.8% compared to the standard approach (DeepSpeed solution). Consequently, our approach enables the full parameter fine-tuning of a 65B model on a single machine with 8 RTX 3090, each with 24GB memory.

In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction. It has been shown highly dependent on the provided demonstrations and thus promotes the research of demonstration retrieval: given a test input, relevant examples are retrieved from the training set to serve as informative demonstrations for in-context learning. While previous works focus on training task-specific retrievers for several tasks separately, these methods are often hard to transfer and scale on various tasks, and separately trained retrievers incur a lot of parameter storage and deployment cost. In this paper, we propose Unified Demonstration Retriever (\textbf{UDR}), a single model to retrieve demonstrations for a wide range of tasks. To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback. Then we propose a multi-task list-wise ranking training framework, with an iterative mining strategy to find high-quality candidates, which can help UDR fully incorporate various tasks' signals. Experiments on 30+ tasks across 13 task families and multiple data domains show that UDR significantly outperforms baselines. Further analyses show the effectiveness of each proposed component and UDR's strong ability in various scenarios including different LMs (1.3B - 175B), unseen datasets, varying demonstration quantities, etc., Comment: ACL 2023 camera ready version

Recently, contrastive learning attracts increasing interests in neural text generation as a new solution to alleviate the exposure bias problem. It introduces a sequence-level training signal which is crucial to generation tasks that always rely on auto-regressive decoding. However, previous methods using contrastive learning in neural text generation usually lead to inferior performance. In this paper, we analyse the underlying reasons and propose a new Contrastive Neural Text generation framework, CoNT. CoNT addresses bottlenecks that prevent contrastive learning from being widely adopted in generation tasks from three aspects -- the construction of contrastive examples, the choice of the contrastive loss, and the strategy in decoding. We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation. Experimental results show that CoNT clearly outperforms the conventional training framework on all the ten benchmarks with a convincing margin. Especially, CoNT surpasses previous the most competitive contrastive learning method for text generation, by 1.50 BLEU on machine translation and 1.77 ROUGE-1 on summarization, respectively. It achieves new state-of-the-art on summarization, code comment generation (without external data) and data-to-text generation., Accepted by NeurIPS 2022

Safety exploration can be regarded as a constrained Markov decision problem where the expected long-term cost is constrained. Previous off-policy algorithms convert the constrained optimization problem into the corresponding unconstrained dual problem by introducing the Lagrangian relaxation technique. However, the cost function of the above algorithms provides inaccurate estimations and causes the instability of the Lagrange multiplier learning. In this paper, we present a novel off-policy reinforcement learning algorithm called Conservative Distributional Maximum a Posteriori Policy Optimization (CDMPO). At first, to accurately judge whether the current situation satisfies the constraints, CDMPO adapts distributional reinforcement learning method to estimate the Q-function and C-function. Then, CDMPO uses a conservative value function loss to reduce the number of violations of constraints during the exploration process. In addition, we utilize Weighted Average Proportional Integral Derivative (WAPID) to update the Lagrange multiplier stably. Empirical results show that the proposed method has fewer violations of constraints in the early exploration process. The final test results also illustrate that our method has better risk control., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Establishing the correspondence between two images is an important research direction of computer vision. When estimating the relationship between two images, it is often disturbed by outliers. In this paper, we propose a convolutional neural network that can filter the noise of outliers. It can output the probability that the pair of feature points is an inlier and regress the essential matrix representing the relative pose of the camera. The outliers are mainly caused by the noise introduced by the previous processing. The outliers rejection can be treated as a problem of noise elimination, and the soft threshold function has a very good effect on noise reduction. Therefore, we designed an adaptive denoising module based on soft threshold function to remove noise components in the outliers, to reduce the probability that the outlier is predicted to be an inlier. Experimental results on the YFCC100M dataset show that our method exceeds the state-of-the-art in relative pose estimation., Comment: 6 pages

To investigate the molecular mechanism in stable cell strains expressing Mini-hF9 gene with nonsense mutation.Mini-hF9 gene and its nonsense mutants were transfected into HeLa cells independently, and stable cell strains were obtained after G418 resistance screening and monoclonal transformation. The altered splicing and protein expression of mRNA in Mini-hF9 gene in stable cell strains were detected by using RT-PCR and Western blot.The wild type and nonsense mutated human coagulation factor IX stable cell strains were constructed successfully, which were named HeLa-F9-WT, HeLa-F9-M1 and HeLa-F9-M2. Only normal splicing Norm was detected in the wild-type cell strain HeLa-F9-WT; Norm and Alt-S1 splicing were detected in HeLa-F9-M1; while Norm, Alt-S1 and Alt-S2 splicing were detected in HeLa-F9-M2.The nonsense associated altered splicing (NAS) pathway, which generated alternately spliced transcripts, might be triggered in coagulation factor IX gene with nonsense mutation.无义突变的人凝血因子IX小基因稳定细胞株选择性剪接的研究.探讨无义突变的人凝血因子IX小基因稳定细胞株发生选择性剪接的分子机制.将人凝血因子IX小基因（Mini-hF9）及其无义突变体分别转染哺乳动物细胞HeLa，经G418抗性筛选、单克隆化得到稳定细胞株；通过RT-PCR及Western blot分别检测稳定细胞株中Mini-hF9基因mRNA的选择性剪接及蛋白表达情况.成功构建了野生型和无义突变的人凝血因子IX小基因稳定细胞株HeLa-F9-WT、HeLa-F9-M1和HeLa-F9-M2。野生型稳定细胞株HeLa-F9-WT中扩增到正常剪接产物Norm；无义突变稳定细胞株HeLa-F9-M1中扩增到Norm和Alt-S1这两种剪接产物，HeLa-F9-M2中扩增到Norm、Alt-S1和Alt-S2这三种剪接产物.无义突变的凝血因子IX基因可能触发无义相关的选择性剪接（NAS）途径，导致选择性剪接的产生.

Object visual navigation aims to steer an agent towards a target object based on visual observations of the agent. It is highly desirable to reasonably perceive the environment and accurately control the agent. In the navigation task, we introduce an Agent-Centric Relation Graph (ACRG) for learning the visual representation based on the relationships in the environment. ACRG is a highly effective and reasonable structure that consists of two relationships, i.e., the relationship among objects and the relationship between the agent and the target. On the one hand, we design the Object Horizontal Relationship Graph (OHRG) that stores the relative horizontal location among objects. Note that the vertical relationship is not involved in OHRG, and we argue that OHRG is suitable for the control strategy. On the other hand, we propose the Agent-Target Depth Relationship Graph (ATDRG) that enables the agent to perceive the distance to the target. To achieve ATDRG, we utilize image depth to represent the distance. Given the above relationships, the agent can perceive the environment and output navigation actions. Given the visual representations constructed by ACRG and position-encoded global features, the agent can capture the target position to perform navigation actions. Experimental results in the artificial environment AI2-Thor demonstrate that ACRG significantly outperforms other state-of-the-art methods in unseen testing environments.

To improve the dewatering of coal slurry, Zheng Gangfeng and his colleagues at Anhui University of Science and Technology in China have developed and built a new design of bowl dehydrator with a spiral screen. Here, they discuss its operating principles and salient features, and present test results that demonstrate its potential for industrial-scale application.

an area efficient clamp is presented and validated in 65nm low leakage CMOS process. With this novel design, only a very short time constant RC timer is required for triggering and keeping the clamp turning on for shunting the ESD current. And the leakage is greatly reduced in normal operation because of the small capacitor. Robust ESD protection capability and no risk of power on mis-triggering problems are also studied in the paper.

Fil: LV, Kai. Universidad de Buenos Aires. Facultad de Ciencias Económicas

Objective To reveal the characteristic and distribution of length of hospital stay (LOS) and direct hospitalisation costs of paediatric scald. Methods A prospective case series observation was performed from January 2005 to December 2006 at the Burn Center, Changhai Hospital, Shanghai, China. The information, such as demographics, clinical diagnosis and treatments since admission, of the paediatric scald patients included in the series was recorded. The direct cost of a treatment event was recorded into the price system when it was incurred. All cost data were summarised on completion of the study. The distribution of LOS and the hospitalisation costs were recorded by gender, age, total burn area, depth of burn, blood transfusion and patterns of treatment. Mann–Whitney signed-rank test was used to assess the differences between continuous, non-normally distributed variables, and multiple linear regression was used to model LOS and direct hospitalisation costs. Statistical analyses were undertaken with SPSS 15.0 statistical software. Results Patients aged 3 years or less accounted for more than half of the total LOS and hospitalisation costs, patients with burn area less than 10%TBSA (total burn surface area) accounted for more than 70% of the total LOS and more than half of the hospitalisation costs and patients with second-degree burn accounted for more than 78% of the total LOS and hospitalisation costs. Depth of burn, area of burn, patterns of treatment and blood transfusion were independent predictors of LOS; whereas LOS, area of burn and blood transfusion were independent predictors of hospitalisation costs. Conclusion Paediatric scalds have particular characteristics in terms of distribution of LOS and direct hospitalisation costs and the factors influencing them. The data presented in this study should assist burn care practitioners and hospital epidemiologists estimate and compare the economic burden of paediatric burns at other institutions; it may also be useful in resource allocation and cost-effectiveness analysis of treatment versus prevention strategies.

OBJECTIVE. This review was an effort to systematically examine the nationwide data available on pediatric burns requiring hospitalization to reveal burn epidemiology and guide future education and prevention. METHODS. The China Biomedical Disk Database, Chongqing VIP Database, and China Journal Full-Text Database were searched for articles reporting data on children and their burns from January 2000 through December 2005. Studies were included that systematically investigated the epidemiology of pediatric burns requiring hospitalization in China. Twenty-eight articles met the inclusion criteria, all of which were retrospective analyses. For each study included, 2 investigators independently abstracted the data related to the population description by using a standard form and included the percentage of patients with burn injury who were RESULTS. Of the pediatric patients studied, the proportion of children with burn injury ranged from 22.50% to 54.66%, and the male/female ratio ranged from 1.25:1 to 4.42:1. The ratio of children aged ≤3 years to those >3 years was 0.19:1 to 4.18:1. The rural/urban ratio was 1.60:1 to 12.94:1. The ratio of those who were burned indoors versus outdoors was 1.62 to 17.00, and there were no effective hints on the distribution of seasons and anatomical sites of burn that could be found. The peak hours of pediatric burn were between 17:00 and 20:00. Most articles reported the sequence of reasons as hot liquid > flame > electricity > chemical, and scalding was, by far, the most predominant reason for burn. The majority of the studies reported the highest proportion involved in moderate burn, and the lowest proportion was for critical burn. The mortality rate ranged from 0.49% to 9.08%, and infection, shock, and multiple organ dysfunction syndrome were the most common causes of death. CONCLUSIONS. Considering the national proportion of children, a high proportion of hospitalized patients with burn injury were children; those who were male, aged ≤3 years, and lived most of the time indoors were especially susceptible. Great attention should be paid when hot water is used or during suppertime. This compilation and analysis of hospitalization-based information has proved useful in establishing the rational priorities for prevention; a family-school-factory-government mode of preventive strategy has come into being and was performed effectively.