Search

Reinforcement Learning (RL) algorithms aim to learn an optimal policy by iteratively sampling actions to learn how to maximize the total expected return, $R(x)$. GFlowNets are a special class of algorithms designed to generate diverse candidates, $x$, from a discrete set, by learning a policy that approximates the proportional sampling of $R(x)$. GFlowNets exhibit improved mode discovery compared to conventional RL algorithms, which is very useful for applications such as drug discovery and combinatorial search. However, since GFlowNets are a relatively recent class of algorithms, many techniques which are useful in RL have not yet been associated with them. In this paper, we study the utilization of a replay buffer for GFlowNets. We explore empirically various replay buffer sampling techniques and assess the impact on the speed of mode discovery and the quality of the modes discovered. Our experimental results in the Hypergrid toy domain and a molecule synthesis environment demonstrate significant improvements in mode discovery when training with a replay buffer, compared to training only with trajectories generated on-policy., Comment: Accepted to ICML 2023 workshop on Structured Probabilistic Inference & Generative Modeling

Building Agent Assistants that can help improve customer service support requires inputs from industry users and their customers, as well as knowledge about state-of-the-art Natural Language Processing (NLP) technology. We combine expertise from academia and industry to bridge the gap and build task/domain-specific Neural Agent Assistants (NAA) with three high-level components for: (1) Intent Identification, (2) Context Retrieval, and (3) Response Generation. In this paper, we outline the pipeline of the NAA's core system and also present three case studies in which three industry partners successfully adapt the framework to find solutions to their unique challenges. Our findings suggest that a collaborative process is instrumental in spurring the development of emerging NLP models for Conversational AI tasks in industry. The full reference implementation code and results are available at \url{https://github.com/VectorInstitute/NAA}, Comment: Camera Ready Version of Paper Published in EMNLP 2022 Industry Track

Mechanical ventilation is a key form of life support for patients with pulmonary impairment. Healthcare workers are required to continuously adjust ventilator settings for each patient, a challenging and time consuming task. Hence, it would be beneficial to develop an automated decision support tool to optimize ventilation treatment. We present DeepVent, a Conservative Q-Learning (CQL) based offline Deep Reinforcement Learning (DRL) agent that learns to predict the optimal ventilator parameters for a patient to promote 90 day survival. We design a clinically relevant intermediate reward that encourages continuous improvement of the patient vitals as well as addresses the challenge of sparse reward in RL. We find that DeepVent recommends ventilation parameters within safe ranges, as outlined in recent clinical trials. The CQL algorithm offers additional safety by mitigating the overestimation of the value estimates of out-of-distribution states/actions. We evaluate our agent using Fitted Q Evaluation (FQE) and demonstrate that it outperforms physicians from the MIMIC-III dataset., Comment: to be published in IAAI (Innovative Applications of Artificial Intelligence) 2023

OBJECTIVE To evaluate the influence exerted by different dental specialty backgrounds as well as the validity and reproducibility of the Pink Esthetic Score/White Esthetic Score (PES/WES) and the modified Implant Crown Aesthetic Index (mod-ICAI) on the assessment of esthetic aspects of maxillary implants supported single-tooth prosthesis. MATERIAL AND METHODS A total of fourteen examiners (Two orthodontists, two prosthodontists, two oral surgeons, two periodontists, two dental technicians, two dental assistants, and two postgraduate students in Implant Dentistry evaluated 20 photographs of single-implant-supported crowns and five photographs of unrestored teeth of esthetic zone in a two part study. The examiners assessed the photographs with each index (Pink Esthetic Score/White Esthetic Score and modified Implant Crown Aesthetic Index), twice with a week's interval. Orders of photographs were rearranged in the second assessment. RESULTS Kruskal-Wallis test results showed significant differences among all the six specialties (P ≤ 0.001). DAs and periodontists had significantly better ratings than other specialties with both indices. Prosthodontists had the lowest mean rank scores regardless of the index. Interobserver agreement was also lowest between the two prosthodontists (4-28%), rest of the groups had low-to-moderate agreement (20-80%) when limited allowance was accepted. With mod-ICAI, more interobserver agreement was noted within the specialty group than with PES/WES. CONCLUSIONS The PES/WES and the modified ICAI can be reliable estimates of esthetic outcomes. The assessor degree of specialization affected the esthetic evaluation with both the PES/WES and the modified ICAI. DAs and periodontists were identified to provide more favorable ratings than other specialties while prosthodontists were most critical in this study. With modified ICAI, more interobserver agreement within specialty resulted. The interexaminer agreement may be increased if more tolerance of 1-2 points is considered.