Search

We consider a nonzero-sum N-player Markov game on an abstract measurable state space with compact metric action spaces. The payoff functions are bounded Carathéodory functions and the transitions of the system are assumed to have a density function satisfying some continuity conditions. The optimality criterion of the players is given by a total expected payoff on an infinite discrete-time horizon. Under the condition that the game model is absorbing, we establish the existence of Markov strategies that are a noncooperative equilibrium in the family of all history-dependent strategies of the players for both the constrained and the unconstrained problems. We obtain, as a particular case of our results, the existence of Nash equilibria for discounted constrained and unconstrained game models. [ABSTRACT FROM AUTHOR]

Immune interventions consisting in repeated injections are broadly used as they are thought to improve the quantity and the quality of the immune response. However, they also raise several questions that remain unanswered, in particular the number of injections to make or the delay to respect between different injections to achieve this goal. Practical and financial considerations add constraints to these questions, especially in the framework of human studies. We specifically focus here on the use of interleukin-7 (IL-7) injections in HIV-infected patients under antiretroviral treatment, but still unable to restore normal levels of CD4+ T lymphocytes. Clinical trials have already shown that repeated cycles of injections of IL-7 could help maintaining CD4+ T lymphocytes levels over the limit of 500 cells/μL, by affecting proliferation and survival of CD4+ T cells. We then aim at answering the question: how to maintain a patients level of CD4+ T lymphocytes by using a minimum number of injections (i.e., optimizing the strategy of injections)? Based on mechanistic models that were previously developed for the dynamics of CD4+ T lymphocytes in this context, we model the process by a piecewise deterministic Markov model. We then address the question by using some recently established theory on impulse control problem in order to develop a numerical tool determining the optimal strategy. Results are obtained on a reduced model, as a proof of concept: the method allows to define an optimal strategy for a given patient. This method could be applied to optimize injections schedules in clinical trials. [ABSTRACT FROM AUTHOR]

This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of ϵ -optimal strategies. The construction of such strategies is explicit and only necessitates the previous knowledge of the cost of the no-impulse strategy. In particular, it does not require the resolution of auxiliary optimal stopping problem or the computation of the value function at each point of the state space. This approach is based on the iteration of a single-jump-or-intervention operator associated to the piecewise deterministic Markov process. [ABSTRACT FROM AUTHOR]

In this paper, we propose an approach for approximating the value function and an ∈ -optimal policy of continuous-time Markov decision processes with Borel state and action spaces, with possibly unbounded cost and transition rates, under the total expected discounted cost optimality criterion. Under adequate assumptions, which in particular include that the transition rate has a density function with respect to a reference measure, together with piecewise Lipschitz continuity of the elements of the control model, we approximate the original controlled process by a model with finite state and action spaces. The approximation error is related to the 1-Wasserstein distance between suitably defined probability measures and approximating measures with finite support. We also study the case when the reference measure is approximated with empirical distributions and we show that convergence of the approximations takes place at an exponential rate in probability. [ABSTRACT FROM AUTHOR]

We consider a discrete-time Markov decision process with Borel state and action spaces, and possibly unbounded cost function. We assume that the Markov transition kernel is absolutely continuous with respect to some probability measure. By replacing this probability measure with its empirical distributionfor a sample of sizen, we obtain a finite state space control problem, which is used to provide an approximation of the optimal value and an optimal policy of the original control model. We impose Lipschitz continuity properties on the control model and its associated density functions. We measure the accuracy of the approximation of the optimal value and an optimal policy by means of a non-asymptotic concentration inequality based on the 1-Wasserstein distance betweenand. Obtaining numerically the solution of the approximating control model is discussed and an application to an inventory management problem is presented. [ABSTRACT FROM PUBLISHER]

ABSTRACT This paper presents a non-parametric method for estimating the conditional density associated to the jump rate of a piecewise-deterministic Markov process. In our framework, the estimation needs only one observation of the process within a long time interval. Our method relies on a generalization of Aalen's multiplicative intensity model. We prove the uniform consistency of our estimator, under some reasonable assumptions related to the primitive characteristics of the process. A simulation study illustrates the behaviour of our estimator. [ABSTRACT FROM AUTHOR]

Abstract: We consider a discrete-time constrained Markov decision process under the discounted cost optimality criterion. The state and action spaces are assumed to be Borel spaces, while the cost and constraint functions might be unbounded. We are interested in approximating numerically the optimal discounted constrained cost. To this end, we suppose that the transition kernel of the Markov decision process is absolutely continuous with respect to some probability measure μ. Then, by solving the linear programming formulation of a constrained control problem related to the empirical probability measure of μ, we obtain the corresponding approximation of the optimal constrained cost. We derive a concentration inequality which gives bounds on the probability that the estimation error is larger than some given constant. This bound is shown to decrease exponentially in n. Our theoretical results are illustrated with a numerical application based on a stochastic version of the Beverton–Holt population model. [Copyright &y& Elsevier]

This paper is devoted to the nonparametnc estimation of the jump rate and the cumulative rate for a general class of non-homogeneous marked renewal processes, defined on a separable metric space. In our framework, the estimation needs only one observation of the process within a long time. Our approach is based on a generalization of the multiplicative intensity model, introduced by Aalen in the seventies. We provide consistent estimators of these two functions, under some assumptions related to the ergodicity of an embedded chain and the characteristics of the process. The paper is illustrated by a numerical example. [ABSTRACT FROM AUTHOR]

We report on X-ray properties of the gamma-ray binary 1FGL J1018.6–5856 using observations obtained with the Swift X-ray telescope. Using 54 observations made between MJD 55575 and 55984, we find that the X-ray flux is modulated at a period of 16.57 ± 0.11 days, which is consistent with previous reports based on gamma-ray data. We find that the X-ray maximum at phase 0 previously reported may not be a persistent feature of the source: the dramatic increases at phase 0 were only detected for ∼100 days and were not detected thereafter. Rather, the persistent sinusoidal maximum seems to be at phase 0.3-0.4, and is misaligned with the gamma-ray (GeV) peak. We also find evidence that the source's X-ray flux is correlated with the spectral hardness in the 0.5-10 keV band. Such a correlation has also been reported in the gamma-ray binaries LS 5039 and LS I +61°303 and can help us to understand the X-ray emission mechanisms of the sources. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a numerical method to calculate the value function for a general discounted impulse control problem for piecewise deterministic Markov processes. Our approach is based on a quantization technique for the underlying Markov chain defined by the post jump location and inter-arrival time. Convergence results are obtained and more importantly we are able to give a convergence rate of the algorithm. The paper is illustrated by a numerical example. [Copyright &y& Elsevier]

Type 1 diabetes is one of the most prevalent autoimmune conditions that develops during childhood and has an increasing incidence worldwide. The disease results from the destruction of pancreatic β cells mediated by autoreactive T-lymphocytes. In order to develop preventive therapies, the cellular mechanisms responsible for the generation and activation of β cell-specific T-lymphocytes need to be characterized. Recent studies in the NOD mouse model of autoimmune diabetes suggest that the MHC Class II presentation of β cell-derived antigens by B-lymphocytes could support the development and activity of autoreactive CD4+ T-lymphocytes in this disease. Interestingly, B-lymphocytes are also the most frequent antigen presenting cells expressing the MHC Class I like molecule, CD1d, the restriction molecule responsible for presentation of lipid and glycolipid antigens to Type 1 NKT cells. Splenic marginal zone B-lymphocytes, which express CD1d at particularly high levels, seem poised to signal to Type 1 NKT cells. In contrast to the disease-promoting role of conventional CD4+ T-lymphocytes, several lines of evidence have shown that Type 1 NKT cells are involved in the prevention of Type 1 Diabetes. This review will analyze current knowledge on the roles of B-lymphocytes and Type 1 NKT cells in the onset of Type 1 Diabetes and explore possible outcomes of their interactions in relation to disease. [ABSTRACT FROM AUTHOR]

In this paper we study the problem of the existence of a solution for the Poisson equation (PE) associated to a piecewise-deterministic Markov process (PDP). It is well known that the long run average cost of a stochastic process can be obtained through a solution of the PE associated with the process. Our first result will show that the existence of a solution for the PE of a PDP is equivalent to the existence of a solution for the PE of an embedded discrete- time Markov chain associated with the PDP. It is important to point out that, due to the possibility of jumps from the boundary, the differential formula for the PDPs has a special form, so that general results for the PE of continuous- time stochastic processes cannot be directly applied. Usually discrete-time conditions for the existence of a solution of a PE of a Markov chain are easier to apply than the continuous-time counterpart. We follow this approach to derive our second result, which establishes sufficient conditions for the existence of a PE to the embedded Markov chain, and consequently for the PE of the PDP. The condition is illustrated with an application to the capacity expansion model. [ABSTRACT FROM AUTHOR]

In this paper, we study a form of stability for a general family of nondiffusion Markov processes known in the literature as piecewise-deterministic Markov process (PDMP). By stability here we mean the existence of an invariant probability measure for the PDMP. It is shown that the existence of such an invariant probability measure is equivalent to the existence of a a-finite invariant measure for a Markov kernel G linked to the resolvent operator U of the PDMP, satisfying a boundedness condition or, equivalently, a Radon-Nikod&yactue;m derivative. Here we generalize existing results of the literature [O. Costa, J. Appl. Prob., 27, (1990), pp. 60-73; M. Davis, Markov Models and Optimization, Chapman and Hall, 19931 since we do not require any additional assumptions to establish this equivalence. Moreover, we give sufficient conditions to ensure the existence of such a &sgr;-finite measure satisfying the boundedness condition. They are mainly based on a modified Foster-Lyapunov criterion for the case in which the Markov chain generated by G is either recurrent or weak Feller. To emphasize the relevance of our results, we study three examples and in particular, we are able to generalize the results obtained by Costa and Davis on the capacity expansion model. [ABSTRACT FROM AUTHOR]

Abstract: This paper is keeping with the topic of two papers which treated dynamic reliability problems and were presented in previous conferences. Its aim is to confirm the potentialities of a method which combines the high modeling ability of the piecewise deterministic processes and the great computing power inherent to the Monte Carlo simulation. This method is now applied to a simplified but realistic offshore oil production system which is a hybrid system combining continuous-time and discrete-time dynamics. The results thus obtained have been compared with those given by an ad hoc Petri net model for comparison and validation purposes. [Copyright &y& Elsevier]

Abstract: This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of the partially observed optimal stopping problem. Then, we propose a numerical method, based on the quantization of the discrete-time filter process and the inter-jump times, to approximate the value function and to compute an -optimal stopping time. We prove the convergence of the algorithms and bound the rates of convergence. [Copyright &y& Elsevier]

We present the X-ray timing and spectral evolution of the Galactic Center magnetar SGR J1745–2900 for the first ∼4 months post-discovery using data obtained with the Nuclear Spectroscopic Telescope Array and Swift observatories. Our timing analysis reveals a large increase in the magnetar spin-down rate by a factor of 2.60 ± 0.07 over our data span. We further show that the change in spin evolution was likely coincident with a bright X-ray burst observed in 2013 June by Swift, and if so, there was no accompanying discontinuity in the frequency. We find that the source 3-10 keV flux has declined monotonically by a factor of ∼2 over an 80 day period post-outburst accompanied by a ∼20% decrease in the source's blackbody temperature, although there is evidence for both flux and kT having leveled off. We argue that the torque variations are likely to be magnetospheric in nature and will dominate over any dynamical signatures of orbital motion around Sgr A*. [ABSTRACT FROM AUTHOR]

We report new spectral and temporal observations of the magnetar 1E 1841–045 in the Kes 73 supernova remnant obtained with the Nuclear Spectroscopic Telescope Array. Combined with new Swift and archival XMM-Newton and Chandra observations, the phase-averaged spectrum is well characterized by a blackbody plus double power law, in agreement with previous multimission X-ray results. However, we are unable to reproduce the spectral results reported based on Suzaku observations. The pulsed fraction of the source is found to increase with photon energy. The measured rms pulsed fractions are ∼12% and ∼17% at ∼20 and ∼50 keV, respectively. We detect a new feature in the 24-35 keV band pulse profile that is uniquely double peaked. This feature may be associated with a possible absorption or emission feature in the phase-resolved spectrum. We fit the X-ray data using the recently developed electron-positron outflow model by Beloborodov for the hard X-ray emission from magnetars. This produces a satisfactory fit, allowing a constraint on the angle between the rotation and magnetic axes of the neutron star of ∼20° and on the angle between the rotation axis and line of sight of ∼50°. In this model, the soft X-ray component is inconsistent with a single blackbody; adding a second blackbody or a power-law component fits the data. The two-blackbody interpretation suggests a hot spot of temperature kT ≈ 0.9 keV occupying ∼1% of the stellar surface. [ABSTRACT FROM AUTHOR]

The article describes the development of an experimental device which collects particles in real-scale powder avalanches for the quantification of particle size distributions. The device was placed on the concrete bunker of the Swiss Vallée de la Sionne avalanche dynamics site. Findings revealed that the size of the particles changes during the avalanche and that fragmentation or fusion may have resulted from local shear due to turbulence, collisions and energy dissipation. It concludes that the device can slow down the particles to avoid commutations by collisions.