Search

Four related procedures provided a basis for comparing the linear-optimality principle with a principle based on the sums of reciprocals of distances to reinforcement, and to explore the generality of the sums-of-reciprocals principle as a description of choice patterns in situations of diminishing returns. The procedures all arranged choices between fixed-ratio schedules and progressive-ratio schedules, which escalated with each consecutive choice. In contrast to previous work that involved constant ratio increments, two sets of procedures in this study involved relatively small increments that are similar to the early values when a progressive schedule is increasing proportionally. The remaining two sets of procedures examined progressive schedules with proportional increments. In addition, the initial value of the progressive alternative was manipulated to determine its effects on patterns of choice with both linear and proportional types of escalation. With the exception of one phase, regardless of the initial/reset value and the patterns of escalation, patterns of choice with pigeons were well characterized by the sums-of-reciprocals principle. This supports previous research with pigeons using fixed-increment progressive schedules, as well as situations in which the progressive schedule increased by constant proportions instead of by constant increments. The findings are attributed to the feature of this averaging technique whereby it differentially values reinforcers based on their relative proximity to a particular choice point.

College undergraduates were given repeated opportunities to choose between a fixed-ratio and a progressive-ratio schedule of reinforcement. Completions of a progressive-ratio schedule produced points (exchangeable for money) and incremented that response requirement by 20 responses with each consecutive choice. In the reset condition, completion of a fixed ratio produced the same number of points and also reset the progressive ratio back to its initial value. In the no-reset condition, the progressive ratio continued to increase by increments of 20 throughout the session with each successive selection of this schedule, irrespective of fixed-ratio choices. Subjects' schedule choices were sensitive to parametric manipulations of the size of the fixed-ratio schedule and were consistent with predictions made on the basis of minimizing the number of responses emitted per point earned, which is a principle of most optimality theories. Also, the present results suggest that if data from human performances are to be compared with results for other species, humans should be exposed to schedules of reinforcement for long periods of time, as is commonly done with nonhuman subjects.

Pigeons chose between two schedules of food presentation, a fixed-interval schedule and a progressive-interval schedule that began at 0 s and increased by 20 s with each food delivery provided by that schedule. Choosing one schedule disabled the alternate schedule and stimuli until the requirements of the chosen schedule were satisfied, at which point both schedules were again made available. Fixed-interval duration remained constant within individual sessions but varied across conditions. Under reset conditions, completing the fixed-interval schedule not only produced food but also reset the progressive interval to its minimum. Blocks of sessions under the reset procedure were interspersed with sessions under a no-reset procedure, in which the progressive schedule value increased independent of fixed-interval choices. Median points of switching from the progressive to the fixed schedule varied systematically with fixed-interval value, and were consistently lower during reset than during no-reset conditions. Under the latter, each subject's choices of the progressive-interval schedule persisted beyond the point at which its requirements equaled those of the fixed-interval schedule at all but the highest fixed-interval value. Under the reset procedure, switching occurred at or prior to that equality point. These results qualitatively confirm molar analyses of schedule preference and some versions of optimality theory, but they are more adequately characterized by a model of schedule preference based on the cumulated values of multiple reinforcers, weighted in inverse proportion to the delay between the choice and each successive reinforcer.

As in studies of self-control, a tit-for-tat contingency in an iterated prisoner's dilemma game creates a conflict between maximization of local and global reinforcement. The present experiments examine this conflict in a multiplayer prisoner's dilemma game. Versus tit for tat, cooperation corresponds to self-control; defection, always immediately reinforced, corresponds to impulsiveness. Three experiments examined sensitivity of behavior to the global reinforcement contingency imposed by tit for tat. Undergraduates played a five-player prisoner's dilemma game against four dummy players programmed to play tit for tat or randomly. With tit for tat, a player's cooperation (or defection) increased dummy players' cooperation (or defection) on subsequent trials-reinforcing cooperation and punishing defection in the long run. Participants cooperated at a higher rate when the dummy players played tit for tat than when the dummy players played randomly. These results are consistent with findings in corresponding studies of self-control. Some participants, caught in a trap of mutual defection with the tit-for-tat playing dummy players, came to cooperate when the tit-for-tat contingency was reset ("forgiving" participants' previous defections) during a pause in the game. This increase was a result of the combined effects of a pause and reset; neither pausing nor resetting independently resulted in an increase in cooperation.

Four adult humans made repeated choices between two time-based schedules of points exchangeable for money: a fixed-interval schedule and a progressive-interval schedule that began at 0 s and increased in fixed increments following each point delivered by that schedule. Under reset conditions, selection of the fixed schedule not only produced a point but also reset the progressive interval to 0 s. Reset conditions alternated with no-reset conditions, in which the progressive-interval duration was independent of fixed-interval choices. Fixed-interval duration and progressive-interval step size were varied independently across conditions. Subjects were exposed to all step sizes in ascending order at a given fixed-interval value before the value was changed. Switching from the progressive-interval schedule to the fixed-interval schedule was systematically related to fixed-interval duration, particularly under no-reset conditions. Switching occurred more frequently and earlier in the progressive-schedule sequence under reset conditions than under no-reset conditions. Overall, the switching patterns conformed closely to predictions of an optimization account based upon maximization of overall reinforcement density, and did not appear to depend on schedule-controlled response patterns or on verbal descriptions of the contingencies.

Three experiments examined adult humans' choices in situations with contrasting short-term and long-term consequences. Subjects were given repeated choices between two time-based schedules of points exchangeable for money: a fixed schedule and a progressive schedule that began at 0 s and increased by 5 s with each point delivered by that schedule. Under "reset" conditions, choosing the fixed schedule not only produced a point but it also reset the requirements of the progressive schedule to 0 s. In the first two experiments, reset conditions alternated with "no-reset" conditions, in which progressive-schedule requirements were independent of fixed-schedule choices. Experiment 1 entailed choices between a progressive-interval schedule and a fixed-interval schedule, the duration of which varied across conditions. Switching from the progressive- to the fixed-interval schedule was systematically related to fixed-interval size in 4 of 8 subjects, and in all subjects occurred consistently sooner in the progressive-schedule sequence under reset than under no-reset procedures. The latter result was replicated in a second experiment, in which choices between progressive- and fixed-interval schedules were compared with choices between progressive- and fixed-time schedules. In Experiment 3, switching patterns under reset conditions were unrelated to variations in intertrial interval. In none of the experiments did orderly choice patterns depend on verbal descriptions of the contingencies or on schedule-controlled response patterns in the presence of the chosen schedules. The overall pattern of results indicates control of choices by temporarily remote consequences, and is consistent with versions of optimality theory that address performance in situations of diminishing returns.

Three behavioral options were available to food-deprived pigeons: (1) pecking one key resulted in food reinforcement according to a 50-response progressive-ratio schedule, (2) pecking a second key reset the progressive-ratio schedule to the initial progressive-ratio step, and (3) pecking a third key produced a 3-min timeout period. Pecks on the reset key were shocked. Under low and intermediate shock intensities, timeouts were not produced; under high shock levels, timeouts were produced regularly. Timeouts occurred during the initial period of a progressive-ratio step and were more frequent during the longer steps of the progressive-ratio schedule. Response-produced timeouts under these experimental conditions could be interpreted either as an escape from aversive behavioral options or as a low-probability behavior emerging when the food reinforcement schedule exerted weaker control.

In two different discrete-trial procedures, pigeons were faced with choices between fixed-ratio and progressive-ratio schedules. The latter schedules entail diminishing returns, a feature analogous to foraging situations in the wild. In the first condition (no reset), subjects chose between a progressive-ratio schedule that increased in increments of 20 throughout a session and a fixed-ratio schedule that was constant across blocks of sessions. The size of the fixed ratio was varied parametrically through an ascending and then a descending series. In the reset condition, the same fixed-ratio values were used, but each selection (and completion) of the fixed ratio reset the progressive-ratio schedule back to its minimal value. In the no-reset procedure, the pigeons tended to cease selecting the progressive ratio when it equaled or slightly exceeded the fixed-ratio value, whereas in reset, they chose the fixed ratio well in advance of that equality point. These results indicate sensitivity to molar as well as to molecular reinforcement rates, and those molar relationships are similar to predictions based on the marginal value theorem of optimal foraging theory (e.g., Charnov, 1976). However, although previous results with monkeys (Hineline & Sodetz, 1987) appeared to minimize responses per reinforcement, the present results corresponded more closely to predictions based on sums-of-reciprocals of distance from point of choice to each of the next four reinforcers. Results obtained by Hodos and Trumbule (1967) with chimpanzees in a similar procedure were intermediate between these two relationships. Variability of choices, as well as median choice points, differed between the reset and no-reset conditions.(ABSTRACT TRUNCATED AT 250 WORDS)

A progressive-ratio reinforcement schedule, in which successive reinforcements required an additional 50 responses, was programmed on one key. A response on a second key reset the progressive-ratio schedule to the first step. Before punishment, all pigeons consistently reset the schedule after reinforcement on the first step, thereby minimizing the number of responses required for reinforcement. Punishment was a brief electric shock contingent upon each response on the reset key. The first effect of punishment was to change the frequency of extra responses on the reset key. Under higher intensities of punishment, the pigeons completed the advanced steps of the progressive-ratio schedule before resetting to the first step. Completions of advanced steps were accompanied by decreases in the overall rate of responding and the rate of reinforcement. When the punishment contingency was removed, the major features of pre-punishment performance were recovered.

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.

In a discrete-trials procedure, pigeons chose between a fixed-ratio 81 schedule and a progressive-ratio schedule by making a single peck at the key correlated with one or the other of these schedules. The response requirement on the progressive-ratio schedule began at 1 and increased by 10 each time the progressive-ratio schedule was chosen. Each time the fixed-ratio schedule was chosen, the requirement on the progressive-ratio schedule was reset to 1 response. In conditions where there was no intertrial interval, subjects chose the progressive-ratio schedule for an average of about five consecutive trials (during which the response requirement increased to 41), and then chose the fixed-ratio schedule. This ratio was larger than that predicted by an optimality analysis that assumes that subjects respond in a pattern that minimizes the response-reinforcer ratio or one that assumes that subjects respond in a pattern that maximizes the overall rate of reinforcement. In conditions with a 25-s or 50-s intertrial interval, subjects chose the progressive-ratio schedule for an average of about eight consecutive trials before choosing the fixed-ratio schedule. This change in performance with the addition of an intertrial interval was also not predicted by an optimality analysis. On the other hand, the results were consistent with the theory that choice is determined by the delays to the reinforcers delivered on the present trial and on subsequent trials.