Search

Offline reinforcement learning (RL) algorithms promise to learn policies directly from offline data sets without environmental interaction. This arrangement enables successful RL applications in the real world, particularly in robots and autonomous driving, where sampling is costly and dangerous. However, the existing offline RL algorithms suffer from insufficient performance attributed to extrapolation error caused by out-of-distribution (OOD) actions. In this work, we propose an offline RL algorithm with an uncertain action constraint (UAC). The design principle of UAC is to minimize the extrapolation error via eliminating unknown and uncertain actions. Concretely, we first theoretically analyze the effects of different types of actions on the extrapolation error. Based on this, we propose an action-constrained strategy that exploits the uncertainty of the environmental dynamics model to eliminate unknown and uncertain actions in the $Q$ -value evaluation process. Furthermore, the convex combination of trajectory information and Gaussian noise is novelly leveraged to enhance the generation probability of the optimal actions. Finally, we carry out the comparison and ablation experiments on the standard D4RL data set. Experimental results indicate that UAC achieves competitive performance, especially in the field of robotic manipulation.

Metabolic dysfunction is associated with nonalcoholic steatohepatitis (NASH) development. However, omics studies investigating metabolic changes in NASH patients are limited. In this study, metabolomics and lipidomics in plasma, as well as proteomics in the liver, were performed to characterize the metabolic profiles of NASH patients. Moreover, the accumulation of bile acids (BAs) in NASH patients prompted us to investigate the protective effect of cholestyramine on NASH. The liver expression of essential proteins involved in FA transport and lipid droplets was significantly elevated in patients with NASH. Furthermore, we observed a distinct lipidomic remodeling in patients with NASH. We also report a novel finding suggesting an increase in the expression of critical proteins responsible for glycolysis and the level of glycolytic output (pyruvic acid) in patients with NASH. Furthermore, the accumulation of branched chain amino acids, aromatic amino acids, purines, and BAs was observed in NASH patients. Similarly, a dramatic metabolic disorder was also observed in a NASH mouse model. Cholestyramine not only significantly alleviated liver steatosis and fibrosis but also reversed NASH-induced accumulation of BAs and steroid hormones. In conclusion, NASH patients were characterized by perturbations in FA uptake, lipid droplet formation, glycolysis, and accumulation of BAs and other metabolites.

Metabolic reprogramming is important in primary biliary cholangitis (PBC) development. However, studies investigating the metabolic signature within the liver of PBC patients are limited. In this study, liver biopsies from 31 PBC patients and 15 healthy controls were collected, and comprehensive metabolomics, lipidomics, and proteomics analysis were conducted to characterize the metabolic landscape in PBC. We observed distinct lipidome remodeling in PBC with increased polyunsaturated fatty acid levels and augmented fatty acid β-oxidation (FAO), evidenced by the increased acylcarnitine levels and upregulated expression of proteins involved in FAO. Notably, PBC patients exhibited an increase in glucose-6-phosphate (G6P) and purines, alongside a reduction in pyruvate, suggesting impaired glycolysis and increased purines biosynthesis in PBC. Additionally, the accumulation of bile acids as well as a decrease in branched chain amino acids and aromatic amino acids were observed in PBC liver. We also observed an aberrant upregulation of proteins associated with ductular reaction, apoptosis, and autophagy. In conclusion, our study highlighted substantial metabolic reprogramming in glycolysis, fatty acid metabolism, and purine biosynthesis, coupled with aberrant upregulation of proteins associated with apoptosis and autophagy in PBC patients. Targeting the specific metabolic reprogramming may offer potential targets for the therapeutic intervention of PBC.

In this work, a novel coumarin derivative-modified cellulose acetate (DCB-CA) was synthesized as a fluorescent probe for highly selective and sensitive determination of CN−in food samples. The DCB-CAwas synthesized by using CAas a skeleton, and the coumarin derivative as the fluorophore. The DCB-CAobtained was characterized by different methods including FTIR, SEM, 1H-NMR, TGA and UV-vis spectroscopy. The DCB-CAexhibited a significant “turn-off” fluorescence response to CN−, accompanied by a distinct fluorescence color change from bright yellow to colorless. The detection limit of CN−using DCB-CAwas calculated to be 5.8 × 10−7M, which was much lower than the threshold limit of CN−recommended by the World Health Organization (1.9 × 10−6M). Because of the favorable solubility and processability of the CA, the DCB-CAwas easily processed into different fluorescent materials including fluorescent films and coatings. The fluorescent film obtained was also applied to the selective detection of CN−. Furthermore, the DCB-CAwas successfully applied to determine CN−in food samples.

A new colorimetric and fluorescent turn-on probe named 2,4-bis(camphor-3-methylene)phenylacrylate (BCP–Cys) was designed for highly sensitive and specific monitoring of cysteine (Cys). The probe BCP–Cyswas strategically constructed by employing a new bis-camphor-derived scaffold (BCP-OH) as the fluorophore and an acrylate group as the recognition site and fluorescence quencher. The acrylate group of BCP–Cyscould be exclusively cleaved by Cys and release the fluorophore BCP-OH, thereby causing a significantly enhanced red fluorescence and a naked-eye colorimetric change from colorless to yellow. The probe BCP–Cysexhibited promising sensing performances for Cys including large Stokes shift (184 nm), fast response time (<1 min), wide linear range (0–100 nM), and low detection limit (0.0728 μM). Moreover, the probe BCP–Cyscould be utilized as a powerful tool for real-time determination of Cys levels within different food samples, such as onion, cabbage, broccoli, garlic, cauliflower, and bamboo sprout. In addition, this probe was also capable of imaging endogenous and exogenous Cys in living cells.

Studying the trace organic matter (OM) in shale is of great significance to develop exploration plans for shale oil and gas exploration. Due to the low OM content, currently there are significant limitations in determining shale trace OM characteristics. This paper reports the analysis of the pyrolyzation products of shale using THz time-domain spectroscopy (THz-TDS) technology combined with pyrolyzation technology, based on the sensitivity of THz to organic and mineral differences. The THz parameters, including the time delay and amplitude of the THz peaks, were extracted to describe the pyrolyzation of shale. In the first-stage of pyrolyzation, the THz response does not change significantly in the range of < 300 °C. In the second stage of 300–700 °C, the THz parameters in the second stage decrease or increase sharply because the OM gradually decomposes. In the third stage, in the high temperature from 700 to 1000 °C, minerals such as calcite decompose. In addition, a turning point between organic matter decomposition and mineral reactions occurs at  700 °C, which helps to identify organic matter and minerals More importantly, the THz parameter can reflect a rate of change of up to 41%, which is far higher than the 4.1% rate of mass change during the pyrolyzation stage at 300–700 °C. Based on verification through thermal gravimetric analysis (TGA), mass spectroscopy (MS), X-ray diffraction (XRD), and scanning electron microscopy (SEM), we believe that the use of the THz combined with pyrolyzation technology method is practical for characterizing the organic matter in shale.

A novel turn-on fluorescent chemosensor 4was designed to distinguish highly toxic hydrazine (N2H4) that relied on a new reaction mechanism. The probe 4was synthesized from nopinone, which is one of the natural product derivatives. As expected, the developed probe 4has high selectivity towards hydrazine over various metal ions, anions, amino acids, and other amines. The fluorescence intensity has a positive correlation with the N2H4concentration from 0 to 900 μM and the limitation of detection was calculated to be 1.03 μM. The possible mechanism of probe 4detecting hydrazine was confirmed by high-performance liquid chromatography (HPLC), high-resolution mass spectrometry (HRMS), and density functional theory (DFT) analysis. Moreover, the cytotoxicity and cell imaging experiments proved that probe 4has the ability to distinguish hydrazine in living cells.

In this work, a novel fluorescent probe CDA, which was synthesized viaa facile derivatization from natural and renewable camphor, exhibited aggregation-induced emission (AIE) characteristics in mixed aqueous media. The fluorescent aggregates showed an obviously enhanced fluorescence response toward N2H4. The response time of CDAaggregates for detecting N2H4was within 20 s, and the detection limit was 2.26 ppb, which was far lower than the U.S. Environmental Protection Agency (EPA) standard (10 ppb). Moreover, probe CDAwas also applied to detect gaseous and aqueous N2H4. Additionally, the cell imaging experiments demonstrated that probe CDAcould be employed to visualize N2H4in living organisms.

The automation level of autonomous marine vehicle is limited which is always semi-autonomy and reliant on operator interactions. In order to improve it, an autonomous collision avoidance method is proposed based on the visual technique as human’s visual system. A deep convolutional neural network (Alexnet), with strong visual processing capability, is adopted for encounter pattern recognition. European Ship Simulator is used to generate some encounter scenes and record the corresponding maneuver operation conforming to the COLREGs (International Regulations for Preventing Collisions at Sea) rules as samples. After the training phase, of Alexnet, it can successfully predict the collision avoidance operation according to the input scene image like crewman; moreover, this can provide operation guidance for the automatic navigation, guidance and control system. Some different encounter situations are simulated, and used to testify the validity of the proposed approach.