Search

Identifying the trade-offs between model-based and model-free methods is a central question in reinforcement learning. Value-based methods offer substantial computational advantages and are sometimes just as statistically efficient as model-based methods. However, focusing on the core problem of policy evaluation, we show information about the transition dynamics may be impossible to represent in the space of value functions. We explore this through a series of case studies focused on structures that arises in many important problems. In several, there is no information loss and value-based methods are as statistically efficient as model based ones. In other closely-related examples, information loss is severe and value-based methods are severely outperformed. A deeper investigation points to the limitations of the representational power as the driver of the inefficiency, as opposed to failure in algorithm design.

Given a dataset on actions and resulting long-term rewards, a direct estimation approach fits value functions that minimize prediction error on the training data. Temporal difference learning (TD) methods instead fit value functions by minimizing the degree of temporal inconsistency between estimates made at successive time-steps. Focusing on finite state Markov chains, we provide a crisp asymptotic theory of the statistical advantages of this approach. First, we show that an intuitive inverse trajectory pooling coefficient completely characterizes the percent reduction in mean-squared error of value estimates. Depending on problem structure, the reduction could be enormous or nonexistent. Next, we prove that there can be dramatic improvements in estimates of the difference in value-to-go for two states: TD's errors are bounded in terms of a novel measure - the problem's trajectory crossing time - which can be much smaller than the problem's time horizon., Comment: 26 pages, 7 figures, submitted to ICML 2023

In this note, we show that one can use average embeddings, introduced recently in [Naor'20, arXiv:1905.01280], to obtain efficient algorithms for approximate nearest neighbor search. In particular, a metric $X$ embeds into $\ell_2$ on average, with distortion $D$, if, for any distribution $\mu$ on $X$, the embedding is $D$ Lipschitz and the (square of) distance does not decrease on average (wrt $\mu$). In particular existence of such an embedding (assuming it is efficient) implies a $O(D^3)$ approximate nearest neighbor search under $X$. This can be seen as a strengthening of the classic (bi-Lipschitz) embedding approach to nearest neighbor search, and is another application of data-dependent hashing paradigm.

We present a new class of stochastic, geometrically-driven optimization algorithms on the orthogonal group $O(d)$ and naturally reductive homogeneous manifolds obtained from the action of the rotation group $SO(d)$. We theoretically and experimentally demonstrate that our methods can be applied in various fields of machine learning including deep, convolutional and recurrent neural networks, reinforcement learning, normalizing flows and metric learning. We show an intriguing connection between efficient stochastic optimization on the orthogonal group and graph theory (e.g. matching problem, partition functions over graphs, graph-coloring). We leverage the theory of Lie groups and provide theoretical results for the designed class of algorithms. We demonstrate broad applicability of our methods by showing strong performance on the seemingly unrelated tasks of learning world models to obtain stable policies for the most difficult $\mathrm{Humanoid}$ agent from $\mathrm{OpenAI}$ $\mathrm{Gym}$ and improving convolutional neural networks.

Background: Software product maintainability prediction (SPMP) is an important task to control software maintenance activity, and many SPMP techniques for improving software maintainability have been proposed. In this study, we performed a systematic mapping and review on SPMP studies to analyze and summarize the empirical evidence on the prediction accuracy of SPMP techniques in current research. Objective: The objective of this study is twofold: (1) to classify SPMP studies reported in the literature using the following criteria: publication year, publication source, research type, empirical approach, software application type, datasets, independent variables used as predictors, dependent variables (e.g. how maintainability is expressed in terms of the variable to be predicted), tools used to gather the predictors, the successful predictors and SPMP techniques, (2) to analyze these studies from three perspectives: prediction accuracy, techniques reported to be superior in comparative studies and accuracy comparison of these techniques. Methodology: We performed a systematic mapping and review of the SPMP empirical studies published from 2000 up to 2018 based on an automated search of nine electronic databases. Results: We identified 82 primary studies and classified them according to the above criteria. The mapping study revealed that most studies were solution proposals using a history-based empirical evaluation approach, the datasets most used were historical using object-oriented software applications, maintainability in terms of the independent variable to be predicted was most frequently expressed in terms of the number of changes made to the source code, maintainability predictors most used were those provided by Chidamber and Kemerer (C&K), Li and Henry (L&H) and source code size measures, while the most used techniques were ML techniques, in particular artificial neural networks. Detailed analysis revealed that fuzzy & neuro fuzzy (FNF), artificial neural

Chronic environmental exposure to arsenic in drinking water is a major public health concern affecting the health of more than 130 million people worldwide. In addition to causing cancer and non-cancer diseases, arsenic causes muscle weakness and dysfunction. We found that arsenic targets muscle stem cell mitochondrial functionality to impair muscle maintenance and regeneration. Stem cell behavior was determined by their epigenetic memory. Although mitochondrial remodeling is inescapable for successful differentiation and pluripotency reprogramming and mitochondria-derived outputs shape cellular epigenetic landscape, their role in the regulation of cellular memory is poorly understood. Arsenic is a mitochondrial stressor and epigenetic modifier that alters stem-cell fate determination making it an excellent tool to determine the mitochondrial contribution to regulation of cellular memory and fate decisions. Using a model of myogenic differentiation, we imprinted a memory of arsenite-induced stress into myogenic reserve cells (RC) and monitored the fate of their progeny in arsenic-free medium. Phenotyping of primed RC revealed inherited aberrant mitochondrial dynamics that limited self-renewal capacity and enhanced their proliferation; trapping them in a poised state. Non-Gaussian statistics demonstrated that nuclear protein profiles and morphometrics were distributed as power-laws, indicating the presence of critical self-organization at the cell population level. Using graph theory, we modeled the mitochondria and showed an increased connectivity of the primed RC progeny mitochondrial networks. We identified crucial interplay of H2A.Z acetylation with RNA polymerase II at bivalent chromatin domains that define chromatin state and transcriptional plasticity of the primed RC. Importantly, the intervention by mitochondria-targeted XJB-5-131 fully restored mitochondrial functionality and dynamics, RC cellular phenotypic identity, nuclear morphometrics, and epigenetic r

Chronic environmental exposure to arsenic in drinking water is a major public health concern affecting the health of more than 130 million people worldwide. In addition to causing cancer and non-cancer diseases, arsenic causes muscle weakness and dysfunction. We found that arsenic targets muscle stem cell mitochondrial functionality to impair muscle maintenance and regeneration. Stem cell behavior was determined by their epigenetic memory. Although mitochondrial remodeling is inescapable for successful differentiation and pluripotency reprogramming and mitochondria-derived outputs shape cellular epigenetic landscape, their role in the regulation of cellular memory is poorly understood. Arsenic is a mitochondrial stressor and epigenetic modifier that alters stem-cell fate determination making it an excellent tool to determine the mitochondrial contribution to regulation of cellular memory and fate decisions. Using a model of myogenic differentiation, we imprinted a memory of arsenite-induced stress into myogenic reserve cells (RC) and monitored the fate of their progeny in arsenic-free medium. Phenotyping of primed RC revealed inherited aberrant mitochondrial dynamics that limited self-renewal capacity and enhanced their proliferation; trapping them in a poised state. Non-Gaussian statistics demonstrated that nuclear protein profiles and morphometrics were distributed as power-laws, indicating the presence of critical self-organization at the cell population level. Using graph theory, we modeled the mitochondria and showed an increased connectivity of the primed RC progeny mitochondrial networks. We identified crucial interplay of H2A.Z acetylation with RNA polymerase II at bivalent chromatin domains that define chromatin state and transcriptional plasticity of the primed RC. Importantly, the intervention by mitochondria-targeted XJB-5-131 fully restored mitochondrial functionality and dynamics, RC cellular phenotypic identity, nuclear morphometrics, and epigenetic r

There exist two ongoing large repositories of software projects in the software engineering community: the repository of the International Software Benchmarking Standards Group (ISBSG) and the Repository referred to as PROMISE (PRedictOr Models In Software Engineering). Researchers interested in using the datasets have to conduct their own analysis of the datasets within these repositories that figure out what contents are suitable for their purposes. Repositories designed without users (researchers and Industrial) needs in mind greatly are more challenging to use. This paper present an analysis of both repositories and provide users with additional information on these datasets by identifying the topics addressed, highlighting their descriptiveness, and their availability, and by indicating whether or not further details are available in order to enhance their reusability in further empirical studies. Recommendations to both PROMISE managers and datasets owners are also suggested to improve the usefulness of the data provided.

Avec l'évolution du domaine du genie logiciel, la gestion de la qualite du logiciel a evolue : elle s'est orientee non seulement vers la production de logiciels qui s'executent, mais aussi vers la satisfaction des objectifs pour lesquels ces logiciels sont con(;us. Contrairement au genie industriel, la gestion de la qualite du logiciel ne porte pas sur le traitement des produits physiques (hard) mais des produits logiques (soft), incluant le developpement du produit logiciel. En genie logiciel la gestion de la qualite requiert I'elaborafion d'un ensemble de criteres de qualite que le logiciel doit satisfaire et la proposition de mesures pour evaluer cette qualite, tel que presente dans ISO 9126. La serie ISO 9126 comprend une norme intemationale sur la qualite du produit logiciel et trois rapports techniques. La norme ISO 9126-1 definit deux modeles de qualite du produit logiciel. Le premier modele de « qualite inteme » et de « qualite exteme » comprend un ensemble de six caracteristiques, subdivisees en un ensemble de 27 souscaracteristiques pour lesquelles des mesures sont proposees dans les rapports techniques ISO TR 9126-2 et 3. Le deuxieme modele de « qualite en utilisation » comprend un ensemble de quatre caracteristiques et des mesures sont proposees dans le rapport technique ISO TR 9126-4 pour evaluer ces caracteristiques. Cette serie ISO (9126 parties 1 a 4) propose des liens entre ces trois modeles de qualite. Cependant, ces liens, bien que definis par un consensus international d'experts ISO, n'ont pas necessairement ete demontres objectivement et empiriquement: ces liens ne devraient done etre consideres que comme un ensemble theorique interessant, et pour lequel des experimentations rigoureuses sont requises afin d'en demontrer indubitablement la validite. L'experimentation necessite une collecte de donnees pour realiser les experiences et faire ressortir les resultats des experiences. Cependant, en absence d'opportunites d'experimentations en Industrie