Branching process models for mutant genes in nonstationary populations.

A deleterious gene achieves a population balance between the opposing forces of selection and mutation. In this paper we explore the nature of this stochastic balance when the surrounding normal population is not at equilibrium. Assuming that new mutations occur according to a Poisson process and thereafter evolve by the rules of a continuous time branching process, we derive explicit formulas and recurrence relations determining the probability distribution of the current number of mutant individuals. In fact, we compute expectations for a variety of interesting random variables for genetic models involving autosomal dominant and X-linked diseases. We can also handle haplotype information on linked markers. This feature will be especially helpful in understanding the linkage disequilibrium strategy of positional cloning in population isolates. In the presence of exponential growth of the normal population, our formulas reduce to the evaluation of certain Laplace transforms.