Thorndike EL. Equation 2 can account for the PREE because as rs increases, the disruptor term in the numerator increases more rapidly than the strength- or mass-like term in the denominator, so that at high values, log(Bx/Bo) becomes increasingly negative. RET assumes that during extinction, the decision to respond depends on the ratio of time from the last reinforcer to the average time between reinforcers during training; when this ratio falls below a threshold criterion value, responding diminishes. To make sure that the PREE could be obtained in discrete trials with pigeon subjects, with CRF and PRF signaled and alternating irregularly within sessions as in multiple schedules, Nevin and Grace (2005, Experiment 3)1 arranged that a single peck at a white light on the left key of a standard pigeon chamber was always reinforced, and that a single peck at a red light on the right key was reinforced with probability .25. The positive relation between reinforcer rate during training and omitted reinforcers to an extinction criterion is not limited to operant paradigms. (1980; see Gallistel, this issue). The general finding that resistance to change is directly related to the signaled rate of reinforcement has one well-known exception: When a response has been reinforced every time it occurs (continuous reinforcement or CRF), it usually extinguishes more rapidly than if it has been reinforced intermittently (partial reinforcement or PRF), a result known as the partial reinforcement extinction effect (PREE). Thus, responding to a key that signaled CRF during training was less resistant to extinction than to a key that signaled PRF, whereas responding to a key color that signaled 30 rft/hr was more resistant to extinction than to a color that signaled 10 rft/hr. The cost of added terms and parameters in the application of BMT to extinction may be justified by the advantage of its unified framework for the study of resistance to change. This result was replicated by Shull and Grimes (2006) over a wider range of schedules that varied in successive conditions from 3.33/hr to 164/hr, with a single 2-hr session of extinction. However, reinforcer duration varied inversely with reinforcer rate for most schedules in his study, and the extent to which this confound affects the reported extinction criteria is unknown. Nevin and Grace (2000) and Nevin, McLean, and Grace (2001) formalized Catanias suggestion by replacing x in Equation 1 with two terms reflecting the effects noted by Catania: where Bo is baseline response rate, Bt is response rate at time t in extinction, c represents the effect of suspending the contingency between responses and reinforcers, and d represents the magnitude of generalization decrement resulting from the change from rs to 0. In particular, the transition from CRF or a high reinforcer rate to 0 must be a more potent disruptor than the transition from PRF or a low reinforcer rate to 0. Rescorla RA, Skucy JC. Bethesda, MD 20894, Web Policies The new PMC design is here! Nevin JA.
Catania (1973) suggested that the discontinuation of reinforcement has two effects: A dependency between responses and reinforcers ends, and reinforcers are no longer delivered (p. 49). Thorndike (1911) proposed the well-known Law of Effect, whereby reinforcement strengthened the connection between an antecedent stimulus and a response, thus increasing the probability of the response when the stimulus was next presented. The smooth line is the prediction of Equation 2; see text for explanation and parameter values. I will suggest that these ordinal differences can be reconciled by analyzing the disruptive effects of extinction. Rescorla (1999) replicated the usual PREE in three within-subject autoshaping experiments, and reported that when the extinction data were expressed in relation to omitted reinforcers, extinction was more rapid after partial reinforcement the reverse of the usual PREE. And just as the change in motion of a physical body depends directly on the magnitude of the external force and inversely on the bodys mass, the change in the rate of responding depends directly on the magnitude of the external variable and inversely on the behavioral equivalent of inertial mass. Moreover, as noted in connection with Equation 2, BMT can explain the PREE only if the exponent on reinforcer rate, in the denominator, is less than 1.0 so that as reinforcer rate increases, the disruptor term drs in the numerator increases faster than the reinforcer term in the denominator. Resistance to extinction in a component with a constant reinforcer rate also depends inversely on the reinforcer rate in an alternated component. Nevertheless, Equation 2 provides at least a rough description of these data, especially after the first session. Therefore, in Equation 1, x must be in units of (reinforcers/time)0.5. (2003) depicted in Figure 6. Cohen SL, Riley DS, Weigle PA. Tests of behavioral momentum in simple and multiple schedules with rats and pigeons. An integrative model for the study of behavioral momentum. Resistance to extinction, expressed as log proportion of baseline, after training in irregularly alternating trials signaling CRF (probability of reinforcement = 1.0) or PRF (probability of reinforcement = 0.25), exemplifying the within-subject PREE (from Nevin & Grace, 2005, Experiment 1). Equation 3 cannot, however, be used for single schedules because the discriminative stimulus and the context are undefined (see Cohen, 1998 for a thoughtful discussion of these matters). Note that in two-component multiple schedules, ra must be the same for both components, rendering the denominator dimensionless.3 When extinction is arranged in both components, the predictions of Equation 3 are identical to those of Equation 2, albeit with different values of c and d. The value of ra affects predicted resistance to extinction for a component with a given reinforcer rate only when the overall average reinforcer rate differs between conditions as in the studies of Nevin et al. However, Equation 2 will be used here because it is a linear decreasing function of time in extinction, and deviations from linearity are readily detected. The effects of alternated-component reinforcer rate during training on subsequent resistance to extinction can be modeled by incorporating the context of reinforcement into the denominator of Equation 2: where ra is the overall average reinforcer rate in the experimental session. A review of empirical support for differential reinforcement of alternative behavior. In addition, the rs/ra ratio is the inverse of Gallistels (this issue) expression for the informativeness of a CS the extent to which it reduces uncertainty about when or whether the next US will occur. The number of reinforcers omitted to a 50% extinction criterion is an increasing function of reinforcer rate in all data reviewed here. Resistance to extinction was evaluated over 7 consecutive sessions; the average data are shown in Figure 3. To reconcile the effects of extinction with those of other disruptors, one must assume that the disruptors differ between CRF and PRF. Resistance to extinction, expressed as log proportion of baseline, in a single 5.5-hr session after training on alternating multiple-schedule components with VI schedules arranging 30 or 10 reinforcers/hr, exemplifying the positive relation between component reinforcer rate on resistance to disruption in multiple schedules (from Nevin, 1974, Experiment 2). This manuscript is based on a presentation at the meetings of the Society for the Quantitative Analysis of Behavior, May 2011, in Denver, CO. 1These data were originally reported by J. The difference in parameter values may reflect differences between pigeons and rats, between single and multiple schedules, or both. government site. The fit is at least tolerable, but note that the values of d differs by 2 orders of magnitude from those used to describe the multiple-schedule data in Figure 5 (c = 1.1, d = 0.0025). Extinction is probably the most commonly employed method for assessing the effects of a history of reinforcement on the persistence of responding, and resistance to extinction was explicitly identified as a measure of habit strength by Hull (1943). to equate total extinction time. The reinforcer rates for the discrete-trial CRF/PRF data in Figure 1 were estimated by assuming 1-s latencies to both sorts of trials. (1983), Nevin (1992a), and Grace et al. Multiple schedules allow the comparison of asymptotic response rates and their resistance to change within subjects and sessions. Behavioral momentum theory can explain extinction in multiple schedules, including the PREE in discrete trials. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. Clark FC. Therefore, c must be in units of 1/time, and d (and da in Equation 4) must be in units of 1/(time*reinforcers). Thus, by analogy, when the same disruptor is applied to two asymptotic discriminated operants, the one that is more resistant to change is construed as having the greater behavioral mass, or in traditional terms, greater strength (see Nevin & Grace, 2000, for elaboration of the metaphor and its linkage to preference). Mace FC, Lalli JS, Shea MC, Lalli EP, West BJ, Roberts M, Nevin JA. The https:// ensures that you are connecting to the Table 2 displays the average numbers of reinforcers omitted to a criterion of 50% of baseline, estimated by linear interpolation, for the data presented in Figures 1, ,2,2, ,3,3, and and6.6. extinction The discriminated operant is defined by an antecedent stimulus, a response that occurs in its presence, and the reinforcing consequences of that response. One possible answer is that their history of frequent reinforcement in the rich component had strengthened their tendency to peck, so that the pigeons require far more stringent evidence of nonreinforcement to overcome that tendency. In: Zeiler MD, Harzem P, editors. (1983) with 10 rft/hr arranged by VI 6-min schedules. Careers, The publisher's final edited version of this article is available at. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.
Although Equation 2 has been used most often to characterize data for a single extinction test, or for a few extinction tests separated by extensive baseline training, it has also been applied to extinction in the steady state. The inverse relation between resistance to extinction and reinforcer rate is quite similar across studies, and is exactly opposite to the relation shown in Figure 5 for pigeons trained on multiple schedules arranging the same range of reinforcer rates. The corresponding alternated-component data are presented in the right column. In a within-subject study, Cohen (1998) compared extinction after single-schedule training on VI 30 s (120/hr) and VI 120 s (30/hr) in successive conditions with extinction after training with the same VI schedules in multiple-schedule components. As a service to our customers we are providing this early version of the manuscript. Despite the differences in parameter values, I suggest that the basic ideas underlying Equation 2 the enhancement of response persistence by reinforcement and the countervailing effects of disruption by reinforcer omission can accommodate the data on resistance to extinction in single as well as multiple schedules. Partial reinforcement in autoshaping with pigeons. This general result has been obtained with goldfish (Igaki & Sakagami, 2004), rats (e.g., Blackman, 1968; Shahan & Burke, 2004), pigeons (e.g., Nevin, 1974; Nevin, Tota, Torquato, & Shull, 1990), normal children (Tota-Faucette, 1991), children with developmental disabilities (Ahearn et al., 2003; Mace et al., 2010), college students (Cohen, 1996), and adults with mental retardation (Mace et al., 1990). Intuitively at least, it should be quite difficult to detect a change from baseline until several sessions of extinction had elapsed. Resistance to change is often studied in multiple schedules, where two or more stimuli signaling different rates or amounts of reinforcement are presented successively in effect, defining two discriminated operants in the schedule components. sharing sensitive information, make sure youre on a federal Federal government websites often end in .gov or .mil. Catania AC. Shahan TA, Burke KA. Behavioral momentum: The effects of the temporal separation of rates of reinforcement. (2003). The extinction data for discrete-trial CRF-PRF and for free-operant multiple and single schedules, described above, show that if extinction is rescaled in relation to omitted reinforcers, resistance to extinction depends directly on reinforcer rate during training. Igaki T, Sakagami T. Resistance to change in goldfish. The momentum of human behavior in a natural setting. For example, there were fewer total responses in 12 sessions of extinction for CRF (101) than for PRF (153), but more total responses in 5.5 hr for VI 30/hr (6420) than for VI 10/hr (2036). In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. Cohen SL. Mean log proportions of baseline predicted by Equation 4 plotted against mean log proportions calculated from the data presented in Figures 3 and and6;6; parameter values for each study are given in Table 1. Why did they continue to peck in the rich component for several more sessions? Baum, this issue). The overwhelming majority of studies of the PREE have employed discrete trials and compared independent groups of subjects, most often rats. For example, in multiple schedules with 129 and 10 reinforcers/hr in 1-min components, the average pigeon detected the omission of 6 reinforcers in the lean component, as shown by the decrease in response rate to 50% of baseline. The behavioral equivalent is B = x/m, where x designates the value of a variable that disrupts or decreases the rate responding and m is behavioral mass. Accordingly, I will use the average log proportion of baseline response rate for each function over the full course of extinction depicted in Figures 1, ,2,2, and and33 to summarize the data. For all 3 pigeons, extinction proceeded more rapidly in the component that had arranged less frequent (10 rft/hr) reinforcement, ordinally opposite to the PREE. National Library of Medicine By contrast, the extinction data of Nevin (1974, Experiment 2) accord with the predictions of Equation 1. Equation 3 suggests that the determiners of the strength or behavioral mass of a discriminated operant in BMT can also be expressed as the ratio of the signaled reinforcer rate to the context in which that signal occurs a potentially unifying link in otherwise diverse perspectives. Grace RC, McLean AP, Nevin JA. A number of parametric studies with pigeons have found that m is approximately equal to the square root of the rate of reinforcement (Nevin, 2002). Nevin JA. Although only the CRF/PRF data of Nevin and Grace (2005) confirm the predicted downturn at the right, those data accord at least ordinally with the findings of many studies of the PREE. The same ordinal relation holds for the multiple-schedule data reviewed here. official website and that any information you provide is encrypted They varied the rate of reinforcement for rats on single VI schedules from 30/hr to 120/hr over successive conditions, and found that resistance to extinction in 3 1-hr sessions was inversely related to reinforcer rate. I thank the reviewers of my original submission for helpful comments, and the editors for organizing the meetings and this special issue. (1980) may be limited to between-group studies. Moreover, extinction is commonly used to reduce or eliminate problem behavior in clinical settings (for review see Petscher, Rey, & Bailey, 2009), so understanding the determiners of resistance to extinction is important for application as well as for behavior theory. Therefore, c must be in units of (reinforcers/time)0.5/time, and d must be in units of 1/[time*(reinforcers/time)0.5]. 2. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. In: Locurto CM, Terrace HS, Gibbon J, editors. When the contingency was terminated, response rates decreased over successive sessions, and the decrease was smaller in the richer component, consistent with the effects of other disruptors as described above. For all 4 pigeons, extinction proceeded more rapidly on the white (CRF) key than the red (PRF) key, contrary to the predictions of Equation 1. Parameter values and variance accounted for by fits of Equation 3 to the data of Nevin et al. 8600 Rockville Pike The points for the first extinction session sometimes lie above 0 (no change from baseline), and there is evidence of curvature in some of the functions. Cohen SL.
I suggest that the rate of reinforcement obtained in the presence of a distinctive stimulus determines the strength of discriminated operant behavior in accordance with Equation 1, and that resistance to extinction reflects the strengthening effects of reinforcement during training when the disruptive effects of discontinuing reinforcement are taken into account by the terms in the numerator of Equation 2. Behavioral contrast and behavioral momentum. Equation 2 is mathematically equivalent to an exponential decay function, which Clark (1959) used to describe empirical extinction functions for rats trained on VI schedules. A. Nevin at the meetings of the Eastern Psychological Association, March 1989. The metaphor is based on Newtonian mechanics.
Within-subject partial reinforcement extinction effect in autoshaping. Gallistel and Gibbon (2000) discussed the detection of nonreinforcement in the context of their Rate Estimation Theory (RET; Gallistel, this issue, formalizes the detection of changes in reinforcer rate more generally in terms of information theory). Ethanol-maintained responding of rats is more resistant to change in a context with added non-drug reinforcement. Thus, if the time between reinforcers during training is doubled, the time required to reach a given threshold criterion must also be doubled. To extend Rescorlas findings to multiple schedules, Nevin et al. The metaphor may be expressed quantitatively by analogy to Newtons Second Law, v = f/m. Nevin JA. Trials were separated by 25 s, alternated irregularly, and were terminated if no peck occurred within 5 s. Twelve consecutive 40-trial sessions of extinction were conducted after 55 training sessions. The smooth curve is the prediction of Equation 2 (see text for explanation and parameter values). To evaluate d, Nevin et al. PMC legacy view Mean log proportions of baseline during 2 hr of extinction after training on single VI schedules as a function of reinforcer rates on a logarithmic axis. The analysis of behavioral momentum. This result may be modeled by adding terms to the numerator of Equation 3, but the data are too limited to warrant the addition of further parameters. Equation 3 was fitted to the data for all 7 sessions of extinction for all components and conditions from these studies depicted in Figures 3 and and6.6. All fits were performed by Microsoft Excel Solver; parameter values and variance accounted for are given in Table 1. However, some free-operant multiple-schedule studies have yielded more orderly extinction functions. Note that the number of omitted reinforcers to the 50% criterion is greater for CRF than for PRF (Figure 1), and the same ordering holds for richer vs. leaner components in all of the multiple schedules described above. (1980) found the usual PREE: The lower the probabilty of reinforcement, the greater the persistence of extinction responding over trials. will also be available for a limited time. I will review some studies of resistance to extinction and interpret their data in relation to BMT. The averages are greater than 0 in the richer component and less than 0 in the leaner component of each pair, strikingly so for the 129/hr, 10/hr pair for which discriminability of extinction should differ most between components (but in the opposite direction). (1983), Nevin (1992a), and Grace et al. The psychology of learning. Mace FC, McComas JJ, Mauro BC, Progar PR, Ervin R, Zangrillo AN.
Nevin JA, Grace RC. Despite these limitations in the application of BMT to extinction, the simplest version of BMT, Equation 1, captures the relation between resistance to disruption and reinforcer rate in multiple schedules in a wide range of studies when reinforcement remains in effect. The top row is from Nevin (1992), conditions with a 2-s ICI; the middle row is from conditions with a 2-min ICI; and the bottom row is from Grace et al. Indeed, when prefeeding or intercomponent food were superimposed on reinforced CRF and PRF trials in the study reported by Nevin and Grace (2005, Experiment 3), resistance to disruption was consistently greater on CRF than on PRF trials opposite to the effects of extinction shown in Figure 1. He also presents a set of data for repeated extinction following training with VI schedules ranging from VI 1200 s (3/hr) to CRF, and shows that they are reasonably consistent with constancy of reinforcer omissions to several different extinction criteria. Before Time, rate, and conditioning. Gallistel CR, Gibbon J. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Gibbon showed that the ratio of the overall time between reinforcers to the time between key-light onsets and reinforcers accounted quite accurately for acquisition of autoshaped pecking across conditions with varied reinforcer probabilities and with different intertrial and trial durations.
Catania (1973) suggested that the discontinuation of reinforcement has two effects: A dependency between responses and reinforcers ends, and reinforcers are no longer delivered (p. 49). Thorndike (1911) proposed the well-known Law of Effect, whereby reinforcement strengthened the connection between an antecedent stimulus and a response, thus increasing the probability of the response when the stimulus was next presented. The smooth line is the prediction of Equation 2; see text for explanation and parameter values. I will suggest that these ordinal differences can be reconciled by analyzing the disruptive effects of extinction. Rescorla (1999) replicated the usual PREE in three within-subject autoshaping experiments, and reported that when the extinction data were expressed in relation to omitted reinforcers, extinction was more rapid after partial reinforcement the reverse of the usual PREE. And just as the change in motion of a physical body depends directly on the magnitude of the external force and inversely on the bodys mass, the change in the rate of responding depends directly on the magnitude of the external variable and inversely on the behavioral equivalent of inertial mass. Moreover, as noted in connection with Equation 2, BMT can explain the PREE only if the exponent on reinforcer rate, in the denominator, is less than 1.0 so that as reinforcer rate increases, the disruptor term drs in the numerator increases faster than the reinforcer term in the denominator. Resistance to extinction in a component with a constant reinforcer rate also depends inversely on the reinforcer rate in an alternated component. Nevertheless, Equation 2 provides at least a rough description of these data, especially after the first session. Therefore, in Equation 1, x must be in units of (reinforcers/time)0.5. (2003) depicted in Figure 6. Cohen SL, Riley DS, Weigle PA. Tests of behavioral momentum in simple and multiple schedules with rats and pigeons. An integrative model for the study of behavioral momentum. Resistance to extinction, expressed as log proportion of baseline, after training in irregularly alternating trials signaling CRF (probability of reinforcement = 1.0) or PRF (probability of reinforcement = 0.25), exemplifying the within-subject PREE (from Nevin & Grace, 2005, Experiment 1). Equation 3 cannot, however, be used for single schedules because the discriminative stimulus and the context are undefined (see Cohen, 1998 for a thoughtful discussion of these matters). Note that in two-component multiple schedules, ra must be the same for both components, rendering the denominator dimensionless.3 When extinction is arranged in both components, the predictions of Equation 3 are identical to those of Equation 2, albeit with different values of c and d. The value of ra affects predicted resistance to extinction for a component with a given reinforcer rate only when the overall average reinforcer rate differs between conditions as in the studies of Nevin et al. However, Equation 2 will be used here because it is a linear decreasing function of time in extinction, and deviations from linearity are readily detected. The effects of alternated-component reinforcer rate during training on subsequent resistance to extinction can be modeled by incorporating the context of reinforcement into the denominator of Equation 2: where ra is the overall average reinforcer rate in the experimental session. A review of empirical support for differential reinforcement of alternative behavior. In addition, the rs/ra ratio is the inverse of Gallistels (this issue) expression for the informativeness of a CS the extent to which it reduces uncertainty about when or whether the next US will occur. The number of reinforcers omitted to a 50% extinction criterion is an increasing function of reinforcer rate in all data reviewed here. Resistance to extinction was evaluated over 7 consecutive sessions; the average data are shown in Figure 3. To reconcile the effects of extinction with those of other disruptors, one must assume that the disruptors differ between CRF and PRF. Resistance to extinction, expressed as log proportion of baseline, in a single 5.5-hr session after training on alternating multiple-schedule components with VI schedules arranging 30 or 10 reinforcers/hr, exemplifying the positive relation between component reinforcer rate on resistance to disruption in multiple schedules (from Nevin, 1974, Experiment 2). This manuscript is based on a presentation at the meetings of the Society for the Quantitative Analysis of Behavior, May 2011, in Denver, CO. 1These data were originally reported by J. The difference in parameter values may reflect differences between pigeons and rats, between single and multiple schedules, or both. government site. The fit is at least tolerable, but note that the values of d differs by 2 orders of magnitude from those used to describe the multiple-schedule data in Figure 5 (c = 1.1, d = 0.0025). Extinction is probably the most commonly employed method for assessing the effects of a history of reinforcement on the persistence of responding, and resistance to extinction was explicitly identified as a measure of habit strength by Hull (1943). to equate total extinction time. The reinforcer rates for the discrete-trial CRF/PRF data in Figure 1 were estimated by assuming 1-s latencies to both sorts of trials. (1983), Nevin (1992a), and Grace et al. Multiple schedules allow the comparison of asymptotic response rates and their resistance to change within subjects and sessions. Behavioral momentum theory can explain extinction in multiple schedules, including the PREE in discrete trials. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. Clark FC. Therefore, c must be in units of 1/time, and d (and da in Equation 4) must be in units of 1/(time*reinforcers). Thus, by analogy, when the same disruptor is applied to two asymptotic discriminated operants, the one that is more resistant to change is construed as having the greater behavioral mass, or in traditional terms, greater strength (see Nevin & Grace, 2000, for elaboration of the metaphor and its linkage to preference). Mace FC, Lalli JS, Shea MC, Lalli EP, West BJ, Roberts M, Nevin JA. The https:// ensures that you are connecting to the Table 2 displays the average numbers of reinforcers omitted to a criterion of 50% of baseline, estimated by linear interpolation, for the data presented in Figures 1, ,2,2, ,3,3, and and6.6. extinction The discriminated operant is defined by an antecedent stimulus, a response that occurs in its presence, and the reinforcing consequences of that response. One possible answer is that their history of frequent reinforcement in the rich component had strengthened their tendency to peck, so that the pigeons require far more stringent evidence of nonreinforcement to overcome that tendency. In: Zeiler MD, Harzem P, editors. (1983) with 10 rft/hr arranged by VI 6-min schedules. Careers, The publisher's final edited version of this article is available at. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.
Although Equation 2 has been used most often to characterize data for a single extinction test, or for a few extinction tests separated by extensive baseline training, it has also been applied to extinction in the steady state. The inverse relation between resistance to extinction and reinforcer rate is quite similar across studies, and is exactly opposite to the relation shown in Figure 5 for pigeons trained on multiple schedules arranging the same range of reinforcer rates. The corresponding alternated-component data are presented in the right column. In a within-subject study, Cohen (1998) compared extinction after single-schedule training on VI 30 s (120/hr) and VI 120 s (30/hr) in successive conditions with extinction after training with the same VI schedules in multiple-schedule components. As a service to our customers we are providing this early version of the manuscript. Despite the differences in parameter values, I suggest that the basic ideas underlying Equation 2 the enhancement of response persistence by reinforcement and the countervailing effects of disruption by reinforcer omission can accommodate the data on resistance to extinction in single as well as multiple schedules. Partial reinforcement in autoshaping with pigeons. This general result has been obtained with goldfish (Igaki & Sakagami, 2004), rats (e.g., Blackman, 1968; Shahan & Burke, 2004), pigeons (e.g., Nevin, 1974; Nevin, Tota, Torquato, & Shull, 1990), normal children (Tota-Faucette, 1991), children with developmental disabilities (Ahearn et al., 2003; Mace et al., 2010), college students (Cohen, 1996), and adults with mental retardation (Mace et al., 1990). Intuitively at least, it should be quite difficult to detect a change from baseline until several sessions of extinction had elapsed. Resistance to change is often studied in multiple schedules, where two or more stimuli signaling different rates or amounts of reinforcement are presented successively in effect, defining two discriminated operants in the schedule components. sharing sensitive information, make sure youre on a federal Federal government websites often end in .gov or .mil. Catania AC. Shahan TA, Burke KA. Behavioral momentum: The effects of the temporal separation of rates of reinforcement. (2003). The extinction data for discrete-trial CRF-PRF and for free-operant multiple and single schedules, described above, show that if extinction is rescaled in relation to omitted reinforcers, resistance to extinction depends directly on reinforcer rate during training. Igaki T, Sakagami T. Resistance to change in goldfish. The momentum of human behavior in a natural setting. For example, there were fewer total responses in 12 sessions of extinction for CRF (101) than for PRF (153), but more total responses in 5.5 hr for VI 30/hr (6420) than for VI 10/hr (2036). In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. Cohen SL. Mean log proportions of baseline predicted by Equation 4 plotted against mean log proportions calculated from the data presented in Figures 3 and and6;6; parameter values for each study are given in Table 1. Why did they continue to peck in the rich component for several more sessions? Baum, this issue). The overwhelming majority of studies of the PREE have employed discrete trials and compared independent groups of subjects, most often rats. For example, in multiple schedules with 129 and 10 reinforcers/hr in 1-min components, the average pigeon detected the omission of 6 reinforcers in the lean component, as shown by the decrease in response rate to 50% of baseline. The behavioral equivalent is B = x/m, where x designates the value of a variable that disrupts or decreases the rate responding and m is behavioral mass. Accordingly, I will use the average log proportion of baseline response rate for each function over the full course of extinction depicted in Figures 1, ,2,2, and and33 to summarize the data. For all 3 pigeons, extinction proceeded more rapidly in the component that had arranged less frequent (10 rft/hr) reinforcement, ordinally opposite to the PREE. National Library of Medicine By contrast, the extinction data of Nevin (1974, Experiment 2) accord with the predictions of Equation 1. Equation 3 suggests that the determiners of the strength or behavioral mass of a discriminated operant in BMT can also be expressed as the ratio of the signaled reinforcer rate to the context in which that signal occurs a potentially unifying link in otherwise diverse perspectives. Grace RC, McLean AP, Nevin JA. A number of parametric studies with pigeons have found that m is approximately equal to the square root of the rate of reinforcement (Nevin, 2002). Nevin JA. Although only the CRF/PRF data of Nevin and Grace (2005) confirm the predicted downturn at the right, those data accord at least ordinally with the findings of many studies of the PREE. The same ordinal relation holds for the multiple-schedule data reviewed here. official website and that any information you provide is encrypted They varied the rate of reinforcement for rats on single VI schedules from 30/hr to 120/hr over successive conditions, and found that resistance to extinction in 3 1-hr sessions was inversely related to reinforcer rate. I thank the reviewers of my original submission for helpful comments, and the editors for organizing the meetings and this special issue. (1980) may be limited to between-group studies. Moreover, extinction is commonly used to reduce or eliminate problem behavior in clinical settings (for review see Petscher, Rey, & Bailey, 2009), so understanding the determiners of resistance to extinction is important for application as well as for behavior theory. Therefore, c must be in units of (reinforcers/time)0.5/time, and d must be in units of 1/[time*(reinforcers/time)0.5]. 2. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. In: Locurto CM, Terrace HS, Gibbon J, editors. When the contingency was terminated, response rates decreased over successive sessions, and the decrease was smaller in the richer component, consistent with the effects of other disruptors as described above. For all 4 pigeons, extinction proceeded more rapidly on the white (CRF) key than the red (PRF) key, contrary to the predictions of Equation 1. Parameter values and variance accounted for by fits of Equation 3 to the data of Nevin et al. 8600 Rockville Pike The points for the first extinction session sometimes lie above 0 (no change from baseline), and there is evidence of curvature in some of the functions. Cohen SL.
I suggest that the rate of reinforcement obtained in the presence of a distinctive stimulus determines the strength of discriminated operant behavior in accordance with Equation 1, and that resistance to extinction reflects the strengthening effects of reinforcement during training when the disruptive effects of discontinuing reinforcement are taken into account by the terms in the numerator of Equation 2. Behavioral contrast and behavioral momentum. Equation 2 is mathematically equivalent to an exponential decay function, which Clark (1959) used to describe empirical extinction functions for rats trained on VI schedules. A. Nevin at the meetings of the Eastern Psychological Association, March 1989. The metaphor is based on Newtonian mechanics. Within-subject partial reinforcement extinction effect in autoshaping. Gallistel and Gibbon (2000) discussed the detection of nonreinforcement in the context of their Rate Estimation Theory (RET; Gallistel, this issue, formalizes the detection of changes in reinforcer rate more generally in terms of information theory). Ethanol-maintained responding of rats is more resistant to change in a context with added non-drug reinforcement. Thus, if the time between reinforcers during training is doubled, the time required to reach a given threshold criterion must also be doubled. To extend Rescorlas findings to multiple schedules, Nevin et al. The metaphor may be expressed quantitatively by analogy to Newtons Second Law, v = f/m. Nevin JA. Trials were separated by 25 s, alternated irregularly, and were terminated if no peck occurred within 5 s. Twelve consecutive 40-trial sessions of extinction were conducted after 55 training sessions. The smooth curve is the prediction of Equation 2 (see text for explanation and parameter values). To evaluate d, Nevin et al. PMC legacy view Mean log proportions of baseline during 2 hr of extinction after training on single VI schedules as a function of reinforcer rates on a logarithmic axis. The analysis of behavioral momentum. This result may be modeled by adding terms to the numerator of Equation 3, but the data are too limited to warrant the addition of further parameters. Equation 3 was fitted to the data for all 7 sessions of extinction for all components and conditions from these studies depicted in Figures 3 and and6.6. All fits were performed by Microsoft Excel Solver; parameter values and variance accounted for are given in Table 1. However, some free-operant multiple-schedule studies have yielded more orderly extinction functions. Note that the number of omitted reinforcers to the 50% criterion is greater for CRF than for PRF (Figure 1), and the same ordering holds for richer vs. leaner components in all of the multiple schedules described above. (1980) found the usual PREE: The lower the probabilty of reinforcement, the greater the persistence of extinction responding over trials. will also be available for a limited time. I will review some studies of resistance to extinction and interpret their data in relation to BMT. The averages are greater than 0 in the richer component and less than 0 in the leaner component of each pair, strikingly so for the 129/hr, 10/hr pair for which discriminability of extinction should differ most between components (but in the opposite direction). (1983), Nevin (1992a), and Grace et al. The psychology of learning. Mace FC, McComas JJ, Mauro BC, Progar PR, Ervin R, Zangrillo AN.
Nevin JA, Grace RC. Despite these limitations in the application of BMT to extinction, the simplest version of BMT, Equation 1, captures the relation between resistance to disruption and reinforcer rate in multiple schedules in a wide range of studies when reinforcement remains in effect. The top row is from Nevin (1992), conditions with a 2-s ICI; the middle row is from conditions with a 2-min ICI; and the bottom row is from Grace et al. Indeed, when prefeeding or intercomponent food were superimposed on reinforced CRF and PRF trials in the study reported by Nevin and Grace (2005, Experiment 3), resistance to disruption was consistently greater on CRF than on PRF trials opposite to the effects of extinction shown in Figure 1. He also presents a set of data for repeated extinction following training with VI schedules ranging from VI 1200 s (3/hr) to CRF, and shows that they are reasonably consistent with constancy of reinforcer omissions to several different extinction criteria. Before Time, rate, and conditioning. Gallistel CR, Gibbon J. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Gibbon showed that the ratio of the overall time between reinforcers to the time between key-light onsets and reinforcers accounted quite accurately for acquisition of autoshaped pecking across conditions with varied reinforcer probabilities and with different intertrial and trial durations.