Search

Recent advancements in deep reinforcement learning (RL) have demonstrated notable progress in sample efficiency, spanning both model-based and model-free paradigms. Despite the identification and mitigation of specific bottlenecks in prior works, the agent's exploration ability remains under-emphasized in the realm of sample-efficient RL. This paper investigates how to achieve sample-efficient exploration in continuous control tasks. We introduce an RL algorithm that incorporates a predictive model and off-policy learning elements, where an online planner enhanced by a novelty-aware terminal value function is employed for sample collection. Leveraging the forward predictive error within a latent state space, we derive an intrinsic reward without incurring parameters overhead. This reward establishes a solid connection to model uncertainty, allowing the agent to effectively overcome the asymptotic performance gap. Through extensive experiments, our method shows competitive or even superior performance compared to prior works, especially the sparse reward cases., Comment: Preprint

In this paper, we introduce a novel method with graph-to-segment mapping for question answering over knowledge graphs, which helps understanding question utterances. This method centers on semantic parsing, a key approach for interpreting these utterances. The challenges lie in comprehending implicit entities, relationships, and complex constraints like time, ordinality, and aggregation within questions, contextualized by the knowledge graph. Our framework employs a combination of rule-based and neural-based techniques to parse and construct highly accurate and comprehensive semantic segment sequences. These sequences form semantic query graphs, effectively representing question utterances. We approach question semantic parsing as a sequence generation task, utilizing an encoder-decoder neural network to transform natural language questions into semantic segments. Moreover, to enhance the parsing of implicit entities and relations, we incorporate a graph neural network that leverages the context of the knowledge graph to better understand question representations. Our experimental evaluations on two datasets demonstrate the effectiveness and superior performance of our model in semantic parsing for question answering., Comment: arXiv admin note: text overlap with arXiv:2401.02968

Hengshuai Zhang, Xingyou Dong, Zhenxing Yang, Jiang Zhao, Qudong Lu, Jingzhen Zhu, Longkun Li, Shanhong Yi, Jie Xu Department of Urology, Second Affiliated Hospital, Army Medical University, Chongqing, 400037, People’s Republic of ChinaCorrespondence: Jie Xu; Shanhong YiDepartment of Urology, Second Affiliated Hospital, Army Medical University, Chongqing, 400037, People’s Republic of ChinaEmail xujie1981@tmmu.edu.cn; 2644057844@qq.comBackground: Cross-sensitization of pelvic organs is one theory for why symptoms of gut sickness and interstitial cystitis/bladder pain syndrome overlap. Experimental colitis has been shown to trigger bladder hyperactivity and hyperalgesia in rats. The chemokine receptor CXCR4 plays a key role in bladder function and central sensitization. We aim to study the role of CXCR4 and its inhibitor AMD3100 in colon-bladder cross-organ sensitization.Methods: The colitis model was established by rectal infusion of trinitrobenzene sulfonic acid. Western blot and immunofluorescence were used to assess the expression and distribution of CXCR4. Intrathecal injection of AMD3100 (a CXCR4 inhibitor) and PD98059 (an ERK inhibitor) were used to inhibit CXCR4 and downstream extracellular signal-regulated kinase (ERK) in the spinal cord and dorsal root ganglion (DRG). Intravesical perfusion of resiniferatoxin was performed to measure the pain behavior counts of rats, and continuous cystometry was performed to evaluate bladder voiding function.Results: Compared to the control group, CXCR4 was expressed more in bladder mucosa and colon mucosa, L6-S1 dorsal root ganglion (DRG), and the corresponding segment of the spinal dorsal horn (SDH) in rats with colitis. Moreover, intrathecal injection of the AMD3100 suppressed bladder overactivity, bladder hyperalgesia, and mastocytosis symptoms caused by colitis. Furthermore, AMD3100 effectively inhibited ERK activation in the spinal cord induced by experimental colitis. Finally, treatment with PD98059 alleviated bl

Theory identifies factors that can undermine the evolutionary stability of mutualisms. However, theory's relevance to mutualism stability in nature is controversial. Detailed comparative studies of parasitic species that are embedded within otherwise mutualistic taxa (e.g., fig pollinator wasps) can identify factors that potentially promote or undermine mutualism stability. We describe results from behavioral, morphological, phylogenetic, and experimental studies of two functionally distinct, but closely related, Eupristina wasp species associated with the monoecious host fig, Ficus microcarpa, in Yunnan Province, China. One (Eupristina verticillata) is a competent pollinator exhibiting morphologies and behaviors consistent with observed seed production. The other (Eupristina sp.) lacks these traits, and dramatically reduces both female and male reproductive success of its host. Furthermore, observations and experiments indicate that individuals of this parasitic species exhibit greater relative fitness than the pollinators, in both indirect competition (individual wasps in separate fig inflorescences) and direct competition (wasps of both species within the same fig). Moreover, phylogenetic analyses suggest that these two Eupristina species are sister taxa. By the strictest definition, the nonpollinating species represents a "cheater" that has descended from a beneficial pollinating mutualist. In sharp contrast to all 15 existing studies of actively pollinated figs and their wasps, the local F. microcarpa exhibit no evidence for host sanctions that effectively reduce the relative fitness of wasps that do not pollinate. We suggest that the lack of sanctions in the local hosts promotes the loss of specialized morphologies and behaviors crucial for pollination and, thereby, the evolution of cheating.

Qudong Lu,1 Yang Yang,1 Hengshuai Zhang,1 Cheng Chen,2 Jiang Zhao,1 Zhenxing Yang,1 Yi Fan,1 Longkun Li,1 Huan Feng,1 Jingzhen Zhu,1 Shanhong Yi1 1Department of Urology, Second Affiliated Hospital, Army Medical University, Chongqing, 400037, People’s Republic of China; 2Department of Burns, First Affiliated Hospital, Army Medical University, Chongqing, 400037, People’s Republic of ChinaCorrespondence: Jingzhen Zhu; Shanhong YiDepartment of Urology, Second Affiliated Hospital, Army Medical University, Chongqing, People’s Republic of ChinaTel +86 23-68774453Email zhujz1114@163.com; 2644057844@qq.comPurpose: Hyperalgesia and bladder overactivity are two main symptoms of interstitial cystitis/bladder pain syndrome (IC/BPS). Cannabinoid receptors participate in the modulation of pain and bladder function. GPR18, a member of the cannabinoid receptor family, also participates in the regulation of pain and bladder function, but its underlying mechanisms are unknown. In this work, we sought to study the role of GPR18 in IC/BPS.Methods: A rat model of IC/BPS was established with cyclophosphamide (CYP). Paw withdrawal threshold (PWT) measurement and cystometry were used to evaluate pain and bladder function, respectively. RT-PCR, Western blotting and immunofluorescence were used to assess the expression and distribution of GPR18. The role of GPR18 in pain and bladder function was studied by intrathecal injection of resolvin D2 (RvD2, a GPR18 agonist) and O-1918 (a GPR18 antagonist). Calcium imaging was used to study the relationship between GPR18 and TRPV1.Results: A rat model of IC/BPS, which exhibited a decreased PWT and micturition interval, was successfully established with CYP. The mRNA and protein expression of GPR18 was reduced in the bladder and dorsal root ganglia (DRG) in rats with CYP-induced cystitis. Intrathecal injection of RvD2 increased the PWT and micturition interval. However, O-1918 blocked the therapeutic effect of RvD2. GPR18 was present in bladder a

BackgroundEsophageal Squamous Cell Carcinoma (ESCC) is a major subtype of esophageal cancer causing significant morbility and mortality in Asia. Mechanism of initiation and progression of this disease is unclear. Tumor initiating cells (TICs) are the subpopulation of cells which have the ability to self-renew, as well as, to drive initiation and progression of cancer. Increasing evidence has shown that TICs exist in a variety of tumors. However, the identification and characterization of TICs in esophageal carcinoma has remained elusive.Methodology/principal findingsto identify TICs in ESCC, ESCC cell lines including two primary cells were used for screening suitable surface marker. Then colony formation assay, drug resistant assay and tumorigenicity assay in immune deficient mice were used to characterize TICs in ESCC. We found that just the CD44 expression correlated with tumorigenicity in ESCC cell lines. And then induced differentiation of ESCC cells by all-trans retinoic acid treatment led to decreased expression of CD44. The FACS isolated cell subpopulations with high CD44 expression showed increased colony formation and drug resistance in vitro, as well as significantly enhanced tumorigenicity in NOD/SICD mice, as compared to the low expressing CD44 ESCC cells.Conclusions/significanceour study has discovered a novel TIC surface marker, CD44, which can be utilized to enrich efficiently the TICs in ESCC. These findings will be useful for further studies of these cells and exploring therapeutic approaches.

At the onset of a period of intense synaptic refinement initiated by synchronized eye opening (EO), rapid changes in postsynaptic NMDA receptor and AMPA receptor currents (NMDARcs and AMPARcs) occur within the superficial visual layers of the rodent superior colliculus (sSC; Lu and Constantine-Paton [2004]: Neuron 43:237–249). Subsequently, evoked non-NMDARc amplitudes increase, but by 2 weeks after EO (AEO) they decrease significantly. Here, using whole-cell patch-clamp recording, we demonstrate that small, slowly desensitizing excitatory kainate receptor currents (KARcs) are responsible for the rise and subsequent fall in non-NMDARcs. The increase in KAR transmission parallels inhibitory GABAA responses that plateau at 7 days AEO. By 2 weeks AEO, KARcs are gone. AMPARcs remain unchanged during the appearance and disappearance of the KARcs, despite increases in sSC neuropil activity and continued refinement of inputs to individual sSC neurons. We suggest that in the interval of heightened activity, before SC inhibition matures, many AMPARcs desensitize and are relatively ineffective at relieving the Mg2+ block on NMDARs. This transient appearance of slowly desensitizing, long-duration KARcs may provide increased membrane depolarization necessary for NMDAR function and continuation of synaptic refinement. © 2010 Wiley Periodicals, Inc. Develop Neurobiol 70: 737–750, 2010, National Institutes of Health (U.S.) (Grant EY01407), Amyotrophic Lateral Sclerosis Association (Pierre L.de Bourgknecht ALS Research Foundation)

The time effect of ultrasonication was investigated for dispersing carbon nanofibers (CNFs) into a polycarbonate (PC) matrix on the mechanical properties of thus-produced composites. The effects of CNF surface modification by plasma treatment and the CNF concentration in composites on their mechanical properties were also explored. The plasma coating was characterized by HRTEM and FT-IR. Furthermore, the plasma polymerization (10 w) treatment on the CNF enhanced the CNF dispersion in the polymer matrix. The mechanical properties of the CNF???PC composites varied with the dispersion time, at first increasing to a maximum value and then dropping down. After a long ultrasonic treatment (24 h), the properties increased again. At a high concentration, the CNF-PC suspension became difficult to disperse. Additionally, the possible mechanisms for these behaviors are simply proposed. ?? 2006 Wiley Periodicals, Inc. J Appl Polym Sci 103: 3792???3797, 2007

Three energy mechanisms invoking large-scale magnetic fields are incorporated in a model to interpret jet production in black hole (BH) systems, i.e., the Blandford-Znajek (BZ), the magnetic coupling (MC) and Blandford-Payne (BP) processes. These energy mechanisms can coexist in BH accretion disc based on the magnetic field configurations constrained by the screw instability, provided that the BH spin and the power-law index indicating the variation of the magnetic field at an accretion disc are greater than some critical values. In this model the jets are driven by the BZ process in the Poynting flux regime and by the BP process in the hydromagnetic regime, being consistent with the spine/sheath jet structure observed in BH sources of stellar and supermassive size., Comment: 9 pages, 6 figures, accepted by MNRAS

Rupture of the preocular tear film leads to formation of a dry spot on the cornea with ocular irritation and symptoms of dry eye. One of the factors determining the stability of the tear film is its surface activity. The purpose of this study was to examine the surface activity of tear fluid from patients with Sjögren's syndrome. Tear fluid was sampled from the eyes of 16 patients with primary Sjögren's syndrome. The surface activity of the sample was measured on a Wilhelmy balance. Maximum and minimum surface tension was 72·2 ± 1·7 and 52·9 ± 7·4 mN m-1, respectively. Corresponding values in a previously studied group of normal subjects were 71·5 ± 1·3 and 46·6 ± 3·8 mN m-1, respectively. The difference in minimal surface tension was statistically significant (P<0·001). Reduced surface activity may be caused by dysfunction of the Meibomian glands and suggests a mechanism for causing the symptoms of dry eyes.