Behavioural Brain Research 137 (2002) 3 /25 www.elsevier.com/locate/bbr

Research report

Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine John D. Salamone *, Merce` Correa 1 Department of Psychology, University of Connecticut, Storrs, CT 06269-1020, USA Received 5 August 2001; accepted 27 June 2002

Abstract Although the Skinnerian ‘Empirical Law of Effect’ does not directly consider the fundamental properties of stimuli that enable them to act as reinforcers, such considerations are critical for determining if nucleus accumbens dopamine systems mediate reinforcement processes. Researchers who have attempted to identify the critical characteristics of reinforcing stimuli or activities have generally arrived at an emphasis upon motivational factors. A thorough review of the behavioral literature indicates that, across several different investigators offering a multitude of theoretical approaches, motivation is seen by many as being fundamental to the process of reinforcement. The reinforcer has been described as a goal, a commodity, an incentive, or a stimulus that is being approached, self-administered, attained or preserved. Reinforcers also have been described as activities that are preferred, deprived or in some way being regulated. It is evident that this ‘motivational’ or ‘regulatory’ view of reinforcement has had enormous influence over the hypothesis that DA directly mediates ‘reward’ or ‘reinforcement’ processes. Indeed, proponents of the DA/reward hypothesis regularly cite motivational theorists and employ their language. Nevertheless, considerable evidence indicates that low/moderate doses of DA antagonists, and depletions of DA in nucleus accumbens, can suppress instrumental responding for food while, at the same time, these conditions leave fundamental aspects of reinforcement (i.e. primary or unconditioned reinforcement; primary motivation or primary incentive properties of natural reinforcers) intact. Several complex features of the literature on dopaminergic involvement in reinforcement are examined below, and it is argued that the assertions that DA mediates ‘reward’ or ‘reinforcement’ are inaccurate and grossly oversimplified. Thus, it appears as though it is no longer tenable to assert that drugs of abuse are simply turning on the brain’s natural ‘reward system’. In relation to the hypothesis that DA systems are involved in ‘wanting’, but not ‘liking’, it is suggested in the present review that ‘wanting’ has both directional aspects (e.g. appetite to consume food) and activational aspects (e.g. activation for initiating and sustaining instrumental actions; tendency to work for food). The present paper reviews findings in support of the hypothesis that low doses of DA antagonists and accumbens DA depletions do not impair appetite to consume food, but do impair activational aspects of motivation. This suggestion is consistent with the studies showing that low doses of DA antagonists and accumbens DA depletions alter the relative allocation of instrumental responses, making the animals less likely to engage in instrumental responses that have a high degree of work-related response costs. In addition, this observation is consistent with studies demonstrating that accumbens DA depletions make rats highly sensitive to ratio requirements on operant schedules. Although accumbens DA is not seen as directly mediating appetite to consume food, principles of behavioral economics indicate that accumbens DA could be involved in the elasticity of demand for food in terms of the tendency to pay work-related response costs. Future research must focus upon how specific aspects of task requirements (i.e. ratio requirements, intermittence of reinforcement, temporal features of response requirements, dependence upon conditioned stimuli) interact with the effects of accumbens DA depletions, and which particular factors determine sensitivity to the effects of DA antagonism or depletion. # 2002 Elsevier Science B.V. All rights reserved. Keywords: Reward; Activation; Ergonomics; Fatigue; Incentive salience; Striatum; Motivation

* Corresponding author. Tel.:
/1-860-486-4302; fax:
/1-860-486-2760 E-mail address: [email protected] (J.D. Salamone). 1 ` rea de Psicobiologia, Campus de Riu Sec, Universitat Jaume I, 12079 Castello´, Spain. Present address: A 0166-4328/02/$ - see front matter # 2002 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 6 - 4 3 2 8 ( 0 2 ) 0 0 2 8 2 - 6

4

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

1. Introduction

2. DA Hypothesis of reinforcement

One of the most intense and dynamic areas of research in behavioral neuroscience is the study of the functions of brain dopamine (DA). DA has been implicated in several disorders, including schizophrenia, depression and Parkinson’s disease. Moreover, the dominant paradigm in drug abuse research has been, for the past several years, the hypothesis that DA is the critical neurotransmitter for the mediation of reinforcement phenomena. The DA hypothesis of ‘reward’ or reinforcement has become one of the most ubiquitous and popular hypotheses in the history of neuroscience. Virtually every textbook in neuroscience, psychopharmacology or physiological psychology makes reference to the DA hypothesis of reinforcement, and one can easily find statements in support of this hypothesis on the world wide web, or in the popular press. For example, in an article in Newsweek magazine (12 February 2001), it was stated that ‘‘the pleasure circuit communicates in the language of dopamine’’ (p. 40). In a popular university-level textbook [29], it declares authoritatively that ‘‘Reinforcement occurs when neural circuits detect a reinforcing stimulus and cause the activation of dopaminergic neurons in the ventral tegmental area’’ (p. 363). In a recent review article, nucleus accumbens was referred to as the ‘Universal Addiction Site’ [48]. Indeed, the DA hypothesis of reward is no longer merely a testable scientific hypothesis, and instead has become a widely promoted dogma. In the last few years, several papers have provided critical evaluations of the DA hypothesis of reinforcement [18,93,114,128,154,155,157,160,165,193]. The present review is intended to have a different focus than most previous papers. The hypothesis that DA is involved in reinforcement presupposes that scientists understand or agree upon what the term ‘reinforcement’ actually means. In the present paper, we will argue that the particular model of reinforcement that one employs is of critical relevance for interpreting the literature on dopaminergic involvement in reinforcement. In summarizing various views of reinforcement, it will be suggested that learning and motivational processes are two critical aspects of reinforcement, and particular emphasis will be placed upon the fundamental importance of motivational factors for reinforcement processes involving natural stimuli such as food. Finally, this article will review the literature on aspects of motivation that are relatively preserved after interference with DA systems, and will discuss why such preserved functions render the DA hypothesis of reinforcement inadequate as a global theoretical framework.

Several researchers are associated with the DA hypothesis of reinforcement, but the person most widely credited for the formal inception of this hypothesis is Wise (e.g. [215,216,218,219,223,224] but see also [217,222] for a more recent re-appraisal by this author). According to this hypothesis, DA systems mediate the reinforcing effects of several different classes of stimuli. The DA hypothesis of reinforcement was offered to explain the neural mechanisms underlying intracranial self-stimulation [215], and was eventually extended to include natural reinforcers such as food, water and sex, as well as drugs of abuse [218,219,223,224]. Of course, no serious scientist would maintain that brain DA systems evolved so that mammals could abuse drugs and respond to electrodes; thus, a fundamental aspect of the DA hypothesis of reinforcement is that DA neurons constitute a critical link in the brain’s natural reinforcement system, which evolved to mediate reinforcing effects of natural stimuli such as food. In turn, it is maintained that reinforcing brain stimulation and rewarding drugs of abuse activate the natural reinforcement system in the brain by interacting with dopaminergic neurons and synapses. The hypothesis that DA systems mediate the reinforcing value of natural stimuli, electrical stimulation and drugs of abuse has been referred to as the General Anhedonia Model [165]. It should be pointed out that the preponderance of the work in this area has focused upon the reinforcing effects of drugs of abuse. Nevertheless, the present review will focus on natural reinforcers, especially food. This is important in view of the conceptual structure of the General Anhedonia Model. Because it is argued that drugs of abuse turn on the brain’s natural ‘reward system’, studies of the role of DA in reinforcement processes related to food and other natural stimuli are essentially the linchpin of the General Anhedonia Model. Often, it is simply assumed that DA mediates food reinforcement, and there are sparse and selective citations of research in this area; this notion is rarely challenged or examined critically in the drug selfadministration literature. Nevertheless, several recent papers have highlighted some of the difficulties with the hypothesis that accumbens DA mediates reinforcement for natural stimuli such as food [18,93,96,155,157,160,165,193]. Parkinson et al. [143] noted that dissociations appear to exist between the role of accumbens DA in behavior controlled by drug and natural reinforcers, and stated that ‘‘differences between the effects of drugs and natural reinforcers on the brain might hold important keys to the etiology of addiction’’ (p. 384). Thus, the present review will focus on various views of reinforcement phenomena that have emerged from the literature dealing with natural reinforcers. The hypothesis that DA systems directly

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

mediate food ‘reinforcement’ will be critically evaluated in light of this literature.

3. What is reinforcement? The empirical law of effect described The modern study of instrumental conditioning is built upon the seminal work of Thorndike and Skinner. It was Thorndike (e.g. [197]), who invented the term ‘The Law of Effect’ to describe the processes involved in instrumental conditioning. According to Thorndike, if a response occurred in the presence of a stimulus, and this led to a ‘satisfier’, then that response was likely to occur again in the presence of the same stimulus. Thorndike [197] maintained that responses ‘which are accompanied or closely followed by satisfaction to the animal will, other things being equal, be more firmly connected with the situation’. A generation later, Skinner (e.g. [181,182]) composed a more empirical form of the Law of Effect, which simply and elegantly focussed on the relation between response output and the stimulus events that followed these responses. According to Skinner, positive reinforcement occurs when a response is followed by a stimulus, and response probability increases. The stimulus that leads to such an outcome is known as a positive reinforcer. In contemporary psychology and neuroscience, it is Skinner’s language and description that generally predominates. One rarely sees reference to the Thorndikian terminology of ‘satisfiers’ and ‘annoyers’, except in reference to historical developments in the field. In contrast, the terms ‘positive reinforcement’, ‘negative reinforcement’, and ‘reinforcer’ are ubiquitous in the scientific and clinical literatures, as well as in the classroom and the workplace. Skinner attempted to devise a technology of behavior modification using instrumental conditioning techniques, and also attempted to enunciate an empirically-based language of behavior. In striving for these goals, Skinner was enormously productive and successful. Nevertheless, one of the issues that Skinner did not discuss in detail was the characteristics of stimuli that allow them to act as reinforcers. If one were to evaluate the claim that a neurotransmitter such as DA mediates the effects of reinforcing stimuli, then it would seem advisable to review the literature on the critical characteristics of reinforcing stimuli.

4. What are the critical characteristics of reinforcing stimuli? Motivational and regulatory views of reinforcement Typically, when a student first learns about reinforcement, it is in the context of a course in ‘learning’. When

5

one wants to read about instrumental conditioning procedures, or schedules of reinforcement, one often looks in a book on ‘learning’. Moreover, anytime a scholar wishes to learn more about Thorndike, Skinner, Tolman, Hull, Spence, or various other researchers who have studied instrumental behavior, this information is typically discussed in terms of ‘Learning Theory’. Of course, learning is a critical aspect of the acquisition of new instrumental behaviors. Nevertheless, it is something of a misapprehension to place a selective emphasis on learning, as opposed to other processes, when studying instrumental behavior. Instrumental behavior is a complex and multifaceted process, which involves learning mechanisms, but also involves perceptual, motor and motivational functions output [20,36,50,61]. It will be argued in this paper that research and scholarship on the fundamental characteristics of reinforcing stimuli have inevitably led to an emphasis on motivational factors, and that current behavioral theory emphasizes a combination of associative and motivational processes to explain reinforcement phenomena. Before discussing various views of reinforcement, it would be useful to define the term ‘motivation’. As with any psychological construct, motivation has been defined in several different ways. For the present paper, we will employ a definition of motivation from some of our earlier papers (e.g. [154,155]). According to this earlier work, motivation is the set of processes through which organisms regulate the probability, proximity and availability of stimuli. This definition is intended to involve the regulation of both internal and external stimuli. As discussed below, motivated behavior typically takes place in phases, including a terminal or consummatory phase in which the organism directly interacts with the motivational stimulus, and also an instrumental phase in which the organism regulates the proximity or delivery of stimuli [154,155]. In addition, motivation has both directional and activational aspects [35,55,154,155,159,165]. Directional aspects of motivation refer to the fact that behavior is directed towards or away from particular stimuli; activation refers to the energetic aspect of motivated behavior, i.e. the vigor or persistence. As noted by Cofer [34] ‘Motivational concepts, then, have had at least two major functions with respect to behavior. One is to energize responses, either in general or specifically, and to control their vigor and efficiency. The other is to guide behavior to specific ends, i.e. to give direction to behavior’. Although there are a number of distinct views of the critical characteristics of reinforcing stimuli, there is a common pattern that runs through much of the psychological literature. According to several different investigators, reinforcing stimuli act as motivational stimuli; reinforcers are seen as goal stimuli towards which behavior is directed, or as stimuli or activities that are being regulated by the organism. For most of these

6

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

researchers, this was not some incidental or epiphenomenal characteristic of reinforcers; rather, this property is viewed as a fundamental feature of reinforcement. As noted above, Thorndike was the first researcher to study the principles of instrumental conditioning in detail. In discussing the characteristics of stimuli that can act as satisfiers, Thorndike [197] stated that a satisfier is a stimulus that ‘‘the animal does nothing to avoid, often doing such things as to attain and preserve it’’. Thus, although Thorndike often is cited as the founder of the S-R learning school, and a precursor of the ‘responsereinforcement’ view of instrumental behavior (e.g. see [21]), a close examination of his original work demonstrates that a satisfier was viewed by Thorndike as having important motivational characteristics. Tolman [200] emphasized that the demand for the ‘goal-object’ (i.e. the reinforcer) was a critical factor that promoted maze learning. The concept of ‘need reduction’ was important for Hull [88] as a fundamental condition for reinforcement to occur. As indicated by his discussion of Cannon’s work on homeostasis, it is evident that Hull thought of instrumental behavior as a learned extension of the general principle of homeostatic regulation [7], which is clearly a motivational concept. Hull [88], and later Spence [186] also emphasized the importance of the ‘incentive’ properties of reinforcers. Spence [186] summarized his view of the relation between reinforcement and motivation by writing that ‘‘The combination of a motivating state and the environmental situation impels the subject to respond and to continue responding to various aspects of the situation until a reinforcer is obtained or until removed from the situation’’ (p. 29). In a later work, Cofer [34] stated that ‘motivation provides the conditions for reinforcing behavior or weakening it’. As noted by Domjan and Burkhardt several years later [54] ‘‘At present, it is convenient to think of all sources of motivation as contributing to reinforcement effects’’. Although need reduction and drive reduction were later discarded as the main mechanism through which reinforcers were thought to act, these concepts were replaced by other ideas that also were related to aspects of motivation. Sheffield and Roby [180] observed that a non-nutritive sweetener such as saccharine could act as a reinforcer. According to these researchers, the critical characteristic of a reinforcer, therefore, could not be drive or need reduction. Rather, they suggested that ‘elicitation of the consummatory response’ is the critical factor in determining the reinforcing characteristics of a stimulus. This idea is closely related to the view of Glickman and Schiff [71], who maintained that the most important characteristic of a reinforcing stimulus is the ability to instigate species-specific consummatory responses that help the organism adapt to its environment. The person most widely credited with integrating many of these diverse motivationally-based views of

reinforcement is Bindra. Bindra [21] built upon several earlier concepts, including incentive, the importance of goal-directedness, and the induction of consummatory responses, to construct a view of reinforcement phenomena known as ‘incentive-motivation’. According to Bindra [21], instrumental learning does not so much reflect response-reinforcer associations as it does the acquisition of stimulus-stimulus associations and the modification of goal-directed responses. He suggested that central motivational states influence the responseeliciting properties of stimuli, and that behavior initially directed towards primary incentive stimuli (e.g. food) can, through a perceptual learning process, be flexibly redirected towards conditioned stimuli (e.g. the goal-box or lever). Some views of reinforcement characterize the reinforcer as an activity that is being regulated, as opposed to a stimulus. In attempting to identify the principles that allow activities to be used as reinforcers, Premack [147] observed that reinforcers are activities that occur with a relatively high probability, or are relatively preferred. According to the principle that now bears Premack’s name, a high probability activity (such as eating) can be used to reinforce a low probability activity (such as lever pressing). Another view of reinforcement based upon behavioral regulation is referred to as response deprivation theory [6,198,199]. Reinforcing activities are described as being relatively deprived, and this deprivation disturbs the normal response equilibrium; engaging in the reinforced response is seen as restoring the response equilibrium. Timberlake and colleagues (e.g. [145,199]) have built upon the notion of response deprivation theory, and developed the ‘behavior systems’ approach. According to this view, food reinforcement is clearly seen in the general context of the behavioral systems regulating food-searching and food-consuming behaviors. In recent years, economic models have been used to describe instrumental conditioning phenomena. Reinforcers can be said to be ‘goods’ or ‘commodities’ that are being procured. As stated by Lea [117] ‘‘reinforcers and commodities are both classes of things that a subject will do something to get’’ (p. 443). Consistent with this emphasis on economic principles, instrumental responding is seen as the ‘cost’ that is paid to obtain access to these commodities. Staddon and Ettinger [190] discussed economic models in their comprehensive overview of instrumental behavior, and noted that the ‘preference structure’ of the environment is an important determinant of the reinforcing characteristics of stimuli. At this point, it should be emphasized that each of these behavioral researchers offers a distinct view of instrumental conditioning, and the purpose of this brief review is not to provide details about any one theory, or to contrast them seriously with each other. Moreover, this review is not intended to obscure the distinction

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

between each of these different theoretical systems. Rather, the overview provided above was meant to show that, despite the differences between various behavioral theories, a common principle emerges when one considers the issue of the fundamental characteristics of reinforcing stimuli. The vast majority of researchers who have pondered this issue have arrived at the conclusion that appetitive motivation is critical for the process of positive reinforcement. According to Tapp [195] ‘‘At the simplest level, reinforcers have the capacity to direct an organism’s behavior. Those stimuli that are approached are regarded as positively reinforcing. . .’’ For some researchers (i.e. Bindra), positive reinforcement phenomena are largely explained by a combination of appetitive motivation and classical conditioning. In contemporary learning theory, motivational and response-reinforcer associative mechanisms are seen as jointly determining the acquisition of instrumental behavior. For example, Timberlake [199] emphasizes that response deprivation determines which activities have reinforcing characteristics, but the organism must still learn ‘what leads to what’. Dickinson and Balleine [50] emphasized the importance of motivational factors in regulating goal-directed instrumental actions, and also stressed the importance of learning about the contingency between the instrumental action and the goal or outcome. Thus, it can safely be said that, for a wide variety of researchers who represent a broad spectrum of opinions, appetitive motivation is a fundamental aspect of positive reinforcement. The reinforcer can be described as a goal, a commodity, an incentive, or a stimulus that is being approached, self-administered, attained or preserved. Reinforcers have been described as activities that are preferred, deprived or in some way being regulated. Nevertheless, despite the rich variety of descriptive phrases being used, these are different ways of emphasizing the same, overriding, motivational principle. This fundamental motivational characteristic of reinforcing stimuli is sometimes referred to as the primary (or unconditioned) reinforcing, or primary incentive property of those stimuli [136,191].

5. The empirical law of effect revisited: motivational corollary of the empirical law of effect As noted above, Skinner did not invest a great deal of effort in discussing the critical characteristics of stimuli that allow them to act as reinforcers. Indeed, if one’s goal is to develop a system of behavioral control using operant conditioning, the fact that a stimulus is reinforcing may be more important than why it is reinforcing. Nevertheless, it is useful to consider the relation between the motivational or regulatory views of reinforcement described above and Skinner’s Empirical Law of Effect.

7

First of all, it is important to recognize that Skinner did, on occasion, consider the role of motivational variables such as food deprivation in the process of reinforcement. For example, Skinner [180] stated ‘‘Reinforcement thus brings behavior under the control of an appropriate deprivation. After we have conditioned a pigeon to stretch its neck by reinforcing with food, the variable, which controls the neck-stretching, is food deprivation’’. Designed as it was to allow for control of the behavior of organisms, the Empirical Law of Effect was oriented from the perspective of the one doing the controlling, i.e. the experimenter. In contrast, the motivational or regulatory view is oriented from the perspective of the organism. According to the Empirical Law of Effect, the experimenter presents the reinforcer after a response occurs and observes an increase in response rate. However, if there is a contingent relation between the response and the reinforcer, then it must also be true that there is an increase in reinforcer delivery that accompanies the increase in response probability. In summarizing Thorndike’s experiments, Rachlin [148] stated that, after the organism was reinforced, ‘‘This, in turn, increased the likelihood of the behavior recurring and producing more rewards’’ (p. 75, my italics). Thus, consistent with what has previously been described as a motivational corollary of the Empirical Law of Effect [155,165], a reinforcer can be described as a stimulus that is increased in probability by the organism. This observation was an attempt to bridge the apparent gap between the Law of Effect and the motivational or regulatory views of reinforcement. In taking both the Law of Effect and the regulatory view of reinforcement into account, it can be said that instrumental behavior is characterized by a complex organism /environment system, in which the environment shapes the behavior of the organism, but the behavior of the organism, in turn, regulates environmental events. To take the example from Skinner, noted above, it may be true that food deprivation controls the reinforced operant of neck-stretching, but it also is true that neck-stretching controls the delivery of food, and ultimately, regulates the deprivation itself. Indeed, the main adaptive value of instrumental behavior processes to the organism is that these processes allow the organism to regulate its environment, making some stimuli proximal, or some activities possible, while avoiding, delaying or escaping others. These behavioral processes have adaptive value for the organism in the sense that patterns of behavior are selected in relation to the consequences of the instrumental actions emitted by the organism, which would compliment and extend the epigenetic mechanisms of adaptive behavior naturally selected by evolutionary processes [188].

8

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

6. The importance of motivational concepts for the dopamine hypothesis of reinforcement In the prefatory remarks that began this article it was suggested that, for evaluating the DA hypothesis of reinforcement, it is important to identify the particular model or definition of reinforcement being employed. A careful examination of the literature shows that the motivational view of reinforcement has had a powerful influence over the DA/reinforcement hypothesis. Although some researchers have emphasized the effect of DA antagonists or depletions on response-reinforcement associative processes, in general, proponents of the DA/reinforcement hypothesis have used some form of the motivational view of reinforcement. For several decades, Wise emphasized a motivational interpretation of the effects of DA antagonists. For example, Wise et al. [223] stated that pimozide suppressed operant lever pressing for food because this DA antagonist took the ‘goodness’ out of the food. In explaining the effects of DA antagonists on schedules supported by intermittent reinforcement, Gray and Wise [76] cited Bindra’s work on incentive motivation to provide background for their position. Emphasis on motivational effects of DA antagonists and DA depletions was evident throughout a review by Wise published in 1982 [219]. In a subsequent review Wise [218] explicitly identified his view of instrumental behavior as being one very similar to Bindra. Wise ([218], p. 183) stated that he did not believe the ‘response-reinforcement’ view of instrumental behavior, but instead emphasized Bindra’s incentivemotivation view. Thus, in discussing the effects of DA antagonists on instrumental behavior, Wise ([218], p. 183) stated that ‘‘What is important here is the nature of the unconditioned responses to a rewarding stimulus, for I worded my statements of the anhedonia hypothesis to imply that neuroleptics block these responses. . .’’ (my emphasis added). Consistent with this conceptualization of reward as based upon primary motivation, it should be emphasized that several researchers who support the DA hypothesis have employed measures of consummatory behavior to study the putative effects of DA antagonists on ‘reward’. These measures include food intake [220,221] and sucrose consumption ([135,174,226]; see review in [184]). Berridge and Robinson [18] maintain that DA systems are necessary for aspects of incentive motivation related to appetite, and also emphasize the importance of ‘consumption tests’ as a measure of reward (p. 313). In discussing the hypothesized motivational functions of DA, some researchers have focussed upon emotional, or hedonic aspects of motivation. Thus, the term ‘anhedonia’ was meant to signify that DA antagonists suppressed reinforced behavior because they blunted the hedonic impact of pleasurable stimuli [219]. This notion also has been popular in the drug abuse literature. For

example, Gardner [68] stated that ‘‘the mesotelencephalic DA system almost certainly serves as an important substrate for the encoding in the brain of hedonic value imparted by abusable substances’’ (p. 88). Yet, the notion that interference with DA systems causes ‘anhedonia’ is highly controversial. In the drug abuse literature, the work of Gunne [77] often is cited, because this study suggested that the euphoric effects of amphetamine could by blocked by DA antagonism. Yet, subsequent research has challenged this notion. For example, Gawin [70] reported on four cocaine users who were prescribed DA antagonists because of their paranoid reactions to the drug. According to Gawin [70] ‘‘The patients reported continued euphoria from cocaine readministrations and lengthened cocaine binges because the dysphoric paranoia that often forced an end to their drug use did not occur’’. Brauer and De Wit [24] also noted that pimozide failed to blunt amphetaminestimulated euphoria. More recently, the novel D1 antagonist ecopipam failed to blunt the self-administration and the subjective euphoria reported to be induced by cocaine [79,137]. Regardless of whether or not future research demonstrates clear dopaminergic involvement in the pleasure produced by some drugs of abuse, it remains true that studies of reactivity to food are certainly problematic for the General Anhedonia Model. Considerable research with the taste reactivity paradigm has demonstrated that interference with DA by systemic administration of DA antagonists or by local depletions of DA in nucleus accumbens or neostriatum failed to alter appetitive taste reactivity for sucrose [16,18,204]. This has led Berridge, Robinson and colleagues to conclude that brain DA does not mediate ‘liking’ (i.e. the hedonic reaction to food). Nevertheless, these authors have suggested that DA may be involved in ‘wanting’ food (i.e. incentive salience of food, or the ‘appetite’ for food; see discussion below). Although the motivational or regulatory view of reinforcement seems to be the predominant view employed by advocates of the DA/reinforcement hypothesis, this is not universally true. Ettenberg and colleagues (e.g. [31,83,131]) have taken the perspective that interference with DA systems does not impair primary motivation for reinforcers, but does impair reinforcement. In these studies, the task used to identify the ‘reinforcement’ function being disturbed is the reinstatement of responding after extinction. Thus, for Ettenberg and his colleagues, reinforcement can be defined simply by the effect of a reinforcer on subsequent behavior. Essentially, this argument is based upon a definition of reinforcement that solely involves response-reinforcement associations, and not motivational processes. Such a view is rather unique, because most advocates of the DA hypothesis of reinforcement rely heavily upon the motivational view of reinforcement processes. Nevertheless, in the context of the present

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

paper it also should be pointed out that, based upon results from their own paradigm, Chausmer and Ettenberg [32] concluded that accumbens DA did not mediate reinforcement as measured by the response-reinstatement paradigm. In some sense, there appears to be a considerable range of opinions as to what constitutes the ‘reinforcement’ or ‘reward’ process that is supposedly being blunted by DA antagonists or depletions. Indeed, this variability can make the scientific refutability of the hypothesis quite difficult. Some investigators maintain that DA antagonism does not blunt primary motivation, but nevertheless impairs reinforcement processes, defined as the effect of a reinforcer on subsequent behavior [131]. Several papers have offered a hedonically based incentive-motivational view, and maintain that DA antagonists blunt pleasure (e.g. [219,223,224]). Other references seem to downplay the role of pleasure but emphasize motivational aspects of reinforcing stimuli in claiming that DA systems interfere with the incentive motivation process (e.g. [217,218]). Additional work argues forcefully against the position that DA systems mediate pleasure, but argues in favor of dopaminergic involvement in aspects of incentive motivation related to appetite and reinforcer consumption (i.e. ‘wanting’; [18]). Yet despite the wide variety of theoretical perspectives being offered, it should be recognized that a majority of papers offering support for the DA hypothesis of reward or reinforcement seem to provide a view of instrumental behavior processes that is consistent in some way with the motivational view of reinforcement [17,18,184,187,214,219,223,224].

7. Dissociable aspects of reinforcement and motivation: on the role of accumbens dopamine In the pages above, evidence was reviewed indicating that motivation is seen by many investigators as a critical, even defining feature of the effects of reinforcing stimuli. Moreover, it is evident that most proponents of the DA hypothesis of reinforcement not only adhere to this view, but also actively employ it as an explanation of the impairments induced by DA antagonists. Thus, in evaluating the DA hypothesis of reinforcement, it is crucial to consider the effects of dopaminergic manipulations on various aspects of food motivation. In order to provide a detailed focus on particularly relevant areas of this vast literature, one question will be addressed initially. Are the suppressive effects of low doses of DA antagonists or accumbens DA depletions on steady-state food-reinforced operant behavior due to reductions in directional aspects of primary food motivation?. Essentially, this was the claim that was offered by Wise and colleagues in the early, formative years of the DA/ reward hypothesis [219,223,224]. These early papers are

9

still cited in the contemporary literature as providing critical evidence in support of the General Anhedonia Model. Smith [184], who claimed to offer ‘proof’ of the DA hypothesis of reward, maintains that suppression of sucrose intake by DA antagonists is a key marker of these reward functions. Why are low doses of neuroleptics being emphasized? It is evident to most proponents of the DA/reward hypothesis that higher doses of DA antagonists cause motor problems that severely impair a number of different types of behavior (e.g. [218]). Nucleus accumbens is receiving particular attention because most researchers recognize the important motor functions played by the neostriatum, and also because this structure has often been cited as the critical anatomical locus for the putative reward functions of DA (e.g. [51,82,184]). Accumbens DA depletions severely alter self-administration of stimulants (e.g. [26]), and this evidence is frequently cited as providing essential support for the General Anhedonia Model; this makes an assessment of the effects of accumbens DA depletions on food motivation particularly crucial. Finally, it must be stressed that, as noted above, motivation is seen as having both directional and activational components. When referring to the hypothesized motivational functions of DA, proponents of the hypothesis that DA mediates ‘reward’ are clearly emphasizing directional aspects of motivation. This is not to say that activational aspects have been completely ignored; Wise has written about ‘motivational arousal’ and ‘energizing’ effects of rewards (e.g. [217,222]). Nevertheless, the statements that DA antagonists take the ‘goodness’ out of food, or blunt the ‘hedonic’ or ‘reward’ value of sucrose, or suppress the appetite for food, or impair the unconditioned responses to food, are certainly not emphasizing the involvement of DA in activational aspects of food motivation, but rather are stressing that DA is thought to be involved in the process of directing behavior towards the acquisition and consumption of food. After addressing the initial question on the behavioral effects of low doses of DA antagonists and accumbens DA depletions, the discussion will be broadened to include other brain areas and additional behavioral functions. 7.1. Preserved aspects of motivational functions after injections of low doses of DA antagonists and accumbens DA depletions If low doses of DA antagonists suppress lever pressing for food because they produce a broad or general reduction in food motivation, or appetite, or the primary reinforcing or primary incentive properties of food, then reductions of food intake, perceived reward magnitude, or other behavioral markers of diminished motivation should be evident in the same dose range as the suppression of lever pressing. Yet, the literature

10

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

Table 1 Evidence against the DA/reward hypothesis: aspects of natural motivation, reward or reinforcement that are preserved after accumbens DA depletion or antagonism Little or no effect on fundamental aspects of motivation Free food intake [10,11,111,168,207] Detailed parameters of time spend feeding, [168] rate of feeding, forepaw usage during feeding, food intake Approach to a estrous females, maze choice [89] of estrous females, and execution of copulation Lever pressing for food in low rate operant [1,26,41,57,127,150,166] schedules: CRF, VI30 s Discrimination of Reinforcement Magnitude [125,162] Taste reactivity for sucrose [16,18,19] Lack of similarity of accumbens da depletions to effects of motivational manipulations Extinction [127,130,160,166,167] Pre-feeding [1,160,166,169]

shows clearly that this does not occur (see Table 1 for summary). It has been demonstrated repeatedly that DA antagonists substantially impair lever pressing for food at doses lower than those that suppress food intake or simple approach responses for food [63,151,152,156]. Similar effects have been reported for water reinforcement as well [84,120/122]. These results are consistent with many findings from diverse laboratories, which have involved several different behavioral procedures, instrumental responses and reinforcers. There are several previous reports indicating that doses of DA antagonists that impair response rate measures of behavior did not alter response choice measures [23,42,59,156,162,201]. Instrumental responses with very low response requirements are extremely resistant to moderate/high doses of DA antagonists [30,58,126,156], which demonstrates that the capacity to reinforce some types of instrumental behavior is left intact despite severe impairments in lever pressing that can occur at these same doses. Martin-Iverson et al. [125] used an operant psychophysical procedure to study the effects of haloperidol, and reported that this DA antagonist did not alter behavior in a way that is consistent with a reduction in perceived reinforcement magnitude (see also [108]). Agmo et al. [5] studied the effects of DA antagonism on place preference, and noted that although low doses of DA antagonists may have affected associative processes, these low doses did not affect sucrose consumption and thus ‘the reward value of food or water was not reduced’. The dissociative effects of low doses of DA antagonists on lever pressing versus food intake can clearly be demonstrated in procedures that offer both activities concurrently (see Table 2 for summary). Several studies have been published employing concurrent lever pressing/chow feeding procedures, in which rats can lever

Table 2 Effects of DA manipulations on measures of instrumental response choice Decrease in lever pressing, increase in chow consumption Increasing the ratio requirements [165] Systemic DA antagonism Cis-flupentixol (non selective) Haloperidol, Raclopride (D2) SCH 23390, SKF 83566 (D1)

[47,161,164,169] [47,161,164,169] [47,161]

Intra-accumbens DA antagonism Haloperidol, Raclopride (D2) SCH 23390 (D1)

[141,169] [141]

Accumbens DA depletions 6-OHDA

[44,45,169,185]

Decrease in lever pressing */decrease in chow consumption Conditions that reduce appetite Pre-feeding Amphetamine Fenfluramine

[169] [41] [161]

Conditions that produce severe motor impairments Ventrolateral Striatal DA depletions [45]

press on a continuous or FR5 schedule to receive a more preferred food (Bioserve pellets), or can approach and consume a less preferred food (lab chow) that is concurrently available [42,44,45,169]. Typically, rats under control conditions get the vast majority of their food by lever pressing, and consume only small amounts of chow. Behavioral research shows that the lever pressing in this task is under control of motivational factors; increasing the ratio requirement decreased food acquisition through lever pressing and increased chow consumption [165], while pre-feeding reduced both lever pressing and chow intake [169]. Administration of appetite suppressants such as amphetamine and fenfluramine also suppressed both lever pressing and chow consumption [47,161]. Nevertheless, moderate to low doses of DA antagonists produce a very different pattern of effects. The non-selective DA antagonist cisflupenthixol, the D2 antagonists haloperidol, sulpiride and raclopride, and the D1 antagonists SCH 23390 and SKF 83566 all decrease lever pressing for food but substantially increase chow consumption [47,110,161,164,169]. DA antagonism did not alter consumption of or preference for these different foods in free-feeding choice tests [169]. Thus, low doses of neuroleptics that reduce lever pressing for food nevertheless leave the animal directed towards the acquisition and consumption of food. Based upon all of these studies, it seems untenable to argue that low doses of DA antagonists suppress lever pressing for food reinforcement because they suppress appetite for food, or broadly reduce directional aspects of food motivation. Nucleus accumbens DA depletions induced by local injections of 6-hydroxydopamine (6-OHDA) have gen-

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

erally been shown not to suppress 24 h food intake [111,168,207]. Salamone et al. [168] studied the effects of regional DA depletions on detailed parameters of chow consumption. Nucleus accumbens DA depletions did not suppress total chow consumed, total time spent feeding, or the rate of feeding, nor did they alter forepaw usage during feeding [168]. Bakshi and Kelley [10] reported that intra-accumbens injections of haloperidol did not suppress chow intake. Ikemoto and Panksepp [92] demonstrated that intra-accumbens injections of the DA antagonist flupenthixol reduced run speed for sucrose reinforcement in a runway task, but did not impair sucrose consumption. The concurrent lever pressing/feeding task described above also has been used to assess the role of DA in accumbens. Injections of DA antagonists directly into the nucleus accumbens, or DA depletions produced by local 6-OHDA injections, have been shown to decrease lever pressing for food and increase chow consumption on this task [44,45,110,141,169]. The nucleus accumbens often is divided into core and shell subregions based upon its anatomical organization [228]. The shift from lever pressing to chow intake on the concurrent lever pressing/chow intake task occurred whether D1 or D2 antagonists, or 6-OHDA, were injected into the core or the shell region [141,185], although with 6-OHDA injections stronger effects occurred after core infusions [185]. Taken together, these data demonstrate that depletions of DA in nucleus accumbens do not generally impair directional aspects of food motivation, and indicate that the suppression of lever pressing resulting from those depletions does not result from a loss of food motivation. The shift in behavior away from lever pressing and towards chow consumption that is seen in the concurrent lever pressing/chow feeding task after interference with DA does not appear to be occurring simply because of a shift from an ‘instrumental’ behavior to a ‘consummatory’ behavior. A T-maze choice task was developed to assess this possibility [162]. In this task, one arm of the maze contained a high density of food on each trial (four pellets), while another arm contained a low density (two pellets); rats were tested on 30 trials per day. If both arms were unobstructed, the rats consistently chose the high density arm on the majority of trials, and this choice behavior was unaffected by a low dose (i.e. 0.1 mg/kg) of haloperidol, or by accumbens DA depletions. If rats were trained and tested with a 44 cm barrier obstructing the high density arm, rats under control conditions still climbed the barrier on most trials to obtain the high density of food. However, with the barrier present, the low dose of haloperidol or depletion of accumbens DA caused a dramatic shift in behavior, such that choosing to climb the barrier in the high density arm was substantially reduced, but rats nevertheless left the startbox, entered the low density arm,

11

and consumed the food [42,162]. Together with the operant concurrent choice studies, the T-maze data indicate that low doses of neuroleptics, or depletions of accumbens DA, act on instrumental behavior in a way that interacts strongly with task requirements. Rats with impairments in dopaminergic function remain directed towards the acquisition and consumption of food, and their behavior is still under the control of differences in reinforcement magnitude or quality. Nevertheless, DA antagonists or depletions cause a shift in terms of response allocation (i.e. which path to the food is selected); rats shift away from lever pressing or barrier climbing towards simple locomotion to and consumption of the food. The continuous reinforcement schedule (CRF or FR1) is the most basic example of a simple schedule supported by primary reinforcement. In view of the hypothesized importance of nucleus accumbens DA for reinforcement or ‘reward’, this schedule should be highly sensitive to the effects of accumbens DA depletions. In fact, the CRF schedule is relatively insensitive to the effects of depletions of DA in nucleus accumbens [1,127,166]. There are some mild suppressive effects of accumbens DA depletions on response rate, which occur within the first 15 min of the session during the first few days of testing after DA depletion [127,166]. In fact, the temporal characteristics of these relatively minor effects raises some interesting questions about the nature of the behavioral impairments induced by accumbens DA depletions. One of the most common claims made by advocates of the DA/reward hypothesis is that interference with DA induces ‘extinction’, i.e. a withinsession decline in responding that resembles the effects of withdrawal of reinforcement (e.g. [223]). According to this view, an extinction-like effect is produced because neuroleptic-treated animals experience a reduction in reward value when they encounter the food, and they cease responding as the session progresses when the reward deficit has an impact on their motivation to continue responding. At this point, it must be stated emphatically that, although systemic neuroleptics produce within-session declines in responding, accumbens DA depletions do not produce a pattern of effects that resembles extinction with natural reinforcers such as food. Injections of the DA antagonist flupenthixol into nucleus accumbens also were shown to produce effects that did not resemble extinction [15]. In extinction, response rate starts out very high, and then gradually declines over time. In stark contrast, rats with accumbens DA depletions tend to start out slow, and over the course of the session they either maintain a slow, steady rate of responding or get faster as time goes on [127,166,167]. On the CRF schedule, rats with accumbens DA depletions responded at rates at or slightly higher than normal, and higher than extinction, by the end of the session [127,166]. It is not clear why an animal

12

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

with blunted primary reinforcement would respond more than normal at the end of the session, or hold to a steady rate of responding, after multiple encounters with a reinforcer of a supposedly diminished quality or magnitude. The CRF schedule should be particularly prone to diminished experience of primary reward, because this schedule offers the most opportunities for encountering the reinforcer. The CRF schedule is sensitive to the effects of extinction [166], and to the effects of pre-feeding [1]. Yet, the CRF schedule is relatively insensitive to accumbens DA depletions. Rats with accumbens DA depletions continue to respond throughout the session on the CRF schedule, which indicates that primary food reinforcement is not diminished. These observations are consistent with the broader literature showing that, across a broad range of schedules and conditions, the effects of DA antagonists or DA depletions do not closely resemble the effects of extinction [56,72,73,160,165,201,203]. The insensitivity of food-reinforced CRF responding to the effects of accumbens DA depletions brings up another important point. As a part of the logic of the General Anhedonia Model, the involvement of accumbens DA in stimulant self-administration often is attributed to the notion that accumbens DA is a critical part of the natural ‘reward system’, which mediates food reinforcement. Thus, it is suggested that drugs of abuse act by turning on the ‘natural reward system’. Yet, it should be emphasized that, in spite of the generally severe alterations in stimulant self-administration that are seen after accumbens DA depletions, these same depletions have surprisingly little effect on food-reinforced responding on several schedules [26,57,150]. There are several factors that determine why some food-reinforced tasks are affected by accumbens DA depletions, while others are not. The response allocation literature shows that accumbens DA depletions affect the selection of vigorous instrumental responses if there are other, less effortful alternatives available [165]. Shifts in behavior away from FR5 responding or barrier climbing, and towards alternative paths to reinforcement, occurred even though the accumbens DA depletions did not severely suppress FR5 lever pressing or barrier climbing when no alternative sources of food were available [42,44]. Another possible determinant of the effects of accumbens DA depletions is the baseline rate of lever pressing [160]. Thus, it is possible that performance on some schedules (e.g. CRF, VI 30 s) is relatively unimpaired because these schedules generate relatively low baseline rates. An additional factor influencing the effects of accumbens DA depletions is the ratio requirement of the schedule. In one study, rats were tested on FR1, 4, 16 and 64 schedules [1]. Accumbens DA depletions had no effect on FR1 performance and had only transient effects on the FR4 schedule. More substantial impairments were shown

with the FR16 schedule, while FR64 responding was severely impaired. Severe impairments in responding also were shown by animals responding on very large ratios, such as FR200 and FR300 [170]. These results indicate that accumbens DA depletions enhance ‘ratio strain’ (see Section 7.5 below for more discussion on the sensitivity of ratio schedules to the effects of accumbens DA depletions). If interference with DA systems blunted primary food motivation, then the effects of DA antagonists or DA depletions should closely mimic the effects of prefeeding to reduce food motivation, or should resemble the effects of appetite suppressants. In fact, several studies have demonstrated that the effects of DA antagonists or DA depletions differ substantially from the effects of pre-feeding. In terms of food intake, the effects of haloperidol or forebrain DA depletions on the patterns of eating (e.g. rate of feeding, time spent feeding) differ substantially from the effects of prefeeding [171]. On the concurrent lever pressing/chow feeding task, pre-feeding suppressed both lever pressing and chow consumption, while DA antagonists and accumbens DA depletions decreased lever pressing but increased chow consumption [169]. The effects of DA antagonists and accumbens DA depletions on concurrent lever pressing/feeding tasks do not closely resemble the effects of the appetite suppressant, fenfluramine. Fenfluramine, like pre-feeding, does not cause a shift from lever pressing to chow feeding, but instead suppresses food consumption from both sources [161]. More recently, the effects of accumbens DA depletions and pre-feeding were compared with a variety of different FR schedules (FR1, 4, 16 and 64) [1]. As described above, the effects of accumbens DA depletions on lever pressing were highly dependent upon the ratio requirement, with FR1 responding being unaffected, and greater impairments being observed as ratio requirements of FR16 and FR64 were used. Yet, although FR1 performance was unimpaired by DA depletions, pre-feeding substantially reduced lever pressing on this schedule. Regression analyses demonstrated that accumbens DA depletions a pattern of suppressive effects across schedules that was quite different from the effects of pre-feeding [1]. In addition, several studies involving analysis of interresponse time (IRT) distributions have shown that accumbens DA depletions produce a general slowing of the IRT distribution [160,166,167], while extinction and pre-feeding produce very different patterns [160,166]. In summary, interference with DA systems by administration of low doses of DA antagonists or by local depletions of accumbens DA does not produce effects that closely resemble the effects of extinction, prefeeding or appetite suppression produced by a variety of pharmacological agents. Instead, interference with DA systems leaves fundamental aspects of food motiva-

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

tion and primary food reinforcement intact. Consistent with the motivational or regulatory view of food reinforcement reviewed above, these data demonstrate that food-reinforced lever pressing on some schedules can be suppressed under conditions that leave the primary, or unconditioned reinforcing properties (or primary incentive properties) of food unimpaired. In view of these findings, it seems inaccurate or overly simplistic to summarize the effects of DA antagonists or accumbens DA depletions by claiming that they block primary ‘reward’ or ‘reinforcement’. 7.2. Feeding deficits produced by substantial challenges to brain DA systems In the passages above, it was emphasized that low doses of DA antagonists, or accumbens DA depletions, did not fundamentally block directional aspects of food motivation. Of course, dopaminergic impairments can disrupt food intake. Nevertheless, the preponderance of evidence indicates that these effects occur with higher doses DA antagonists, or with depletions in motor related areas in neostriatum, and also that these feeding deficits are accompanied by clear signs of motor or sensorimotor disruption. For example, haloperidol can suppress lever pressing substantially in doses lower than 0.1 mg/kg; in a recent article the ED50 for suppression of FR5 lever pressing was reported to be 0.08 mg/kg [206]. Studies of chow feeding indicate that higher doses (e.g. 0.2 /0.4 mg/kg) are necessary for suppressing feeding [171]. These higher doses of neuroleptics that suppress feeding also suppress locomotor activity and induce catalepsy [94,146]. Detailed examination of the patterns of feeding indicate that the predominant effect of pre-feeding is a reduction in time spent feeding [39,171], while haloperidol has much greater effects on feeding rate (i.e. feeding efficiency or grams of food eaten per min spent eating; [171]). Several other DA antagonists also suppress rate of feeding [22,33,40]. Rate of feeding also is the primary marker of the impairments in feeding shown by rats recovering from the effects of forebrain DA depletions. In one study of recovery of function, rats initially made aphagic by near-total forebrain DA depletions were akinetic, and spent very little time engaging in any activity, including feeding. Yet, by the 2 week after surgery, despite the fact that their total food intake remained dramatically impaired, most DA-depleted rats spent more time feeding than normal rats [171]. The impairment in feeding was due to a substantially reduced rate of feeding and severe alterations in forepaw and oral motor control [171]. Large forebrain DA depletions produce severe feeding deficits [124], and on occasion this effect has been attributed to the so-called ‘reward’ system of the nucleus accumbens. For example, in figure 5 of [17] it is stated that 6-OHDA lesions of the ‘mesolimbic DA projec-

13

tions’ produce aphagia. Nevertheless, the literature shows that the anatomical locus at which DA depletions impair eating is not the nucleus accumbens [111,168,207]. Rather, it is the lateral neostriatum, or more specifically, the ventrolateral neostriatum (VLS) that is the site at which DA depletions or DA antagonists severely disrupt food intake ([10,56,95,168] see also [193]). Rats with VLS DA depletions have severe deficits in feeding rate and forepaw usage during feeding, which result in a substantial deficit in total food intake that is not seen after DA depletions in other areas, including the nucleus accumbens, the anteroventromedial striatum, or the dorsomedial striatum [45,95,168]. Considerable evidence indicates that VLS is involved in motor function, and that VLS DA depletions produce a constellation of motor deficits involving the head, orofacial and forepaw regions. For example, lateral striatal DA depletions impair reaching and grasping with the contralateral forepaw [153], and induce tremulous movements of the jaw that have the temporal characteristics of Parkinsonian tremor [64,95]. Anatomical evidence indicates that the lateral striatum in general and the VLS in particular receive inputs from sensory and motor areas of neocortex [132]. Interestingly, rats with VLS DA depletions, despite the severe difficulties with skilled movements, remain directed towards food intake. In spite of the profound impairments in feeding rate that were seen, time spent eating was not reduced by VLS DA depletions [168]. Although DA depleted rats are less successful than untreated rats at reaching and grasping, rats with depletions of DA in striatal areas that include VLS actually made more attempts to grasp the food pellets than normal animals [153]. Most of the rats with VLS DA depletions can maintain their body weight by eating wet mash, and often these rats are seen to devour the mash in a vigorous, if somewhat uncoordinated manner if it is put in front of them; only the most severely impaired rats have to be tube fed [43,168]. Thus, there is no evidence that VLS DA depletions impair feeding through a deficit that is primarily motivational, while overwhelming evidence indicates that these rats have impairments in head, orofacial and forepaw motor control. The precise nature of the impairments induced by VLS DA depletions are still being characterized. Motor activity counts based upon gross locomotion tend to be normal, and it is not clear if their fundamental problem is with the execution of skilled motor acts, the parallel execution of simultaneous actions, coordination, sequencing or sensorimotor integration [43,45,46,168]. Nevertheless, the literature on the effects of VLS DA depletions does not provide support for the hypothesis that primary food reward is mediated by DA systems. Sucrose drinking is another task that has been used to measure the putative reward functions of DA [17,18,141,174,184]. Smith [184] constructed an elabo-

14

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

rate ‘proof’ of the DA hypothesis of reward based largely upon studies of sucrose drinking. In this article Smith [184] discussed studies in which doses of 12 mg of the D1 antagonist SCH 23390 were injected directly into the nucleus accumbens, and claimed that the resulting suppression of sucrose consumption reflected a blunting of reward. In considering the relevance of these data for operant conditioning effects, it should be emphasized that doses of 75 mg/kg SCH 23390 injected IP suppress lever pressing substantially [2]. In a 300 g rat, this dose would be equivalent to an IP injection of 22.5 mg into the gut. Thus, an intracranial dose that is relatively high by comparison (e.g. 12.0 mg) hardly argues for anatomical specificity of the D1 antagonist effect. In our laboratory, the entire intra-accumbens dose-response curve for the suppression of lever pressing by SCH 23390 is evident at doses under 2.0 mg per side [205], and the entire doseresponse curve for the shift from lever pressing to chow intake produced by this D1 antagonist is evident at doses less than 1.0 mg per side [141]. Relatively high doses of the D2 antagonist raclopride (e.g. 40.0 mg) also have been injected into nucleus accumbens in some of these studies [174,184], and these doses are much higher than the 1.0 mg per side dose of raclopride that suppresses lever pressing [141]. Smith [184] also makes the argument that, because the local rate of licking is not affected by DA antagonists, then this is prima facie evidence that no motor effects of any type are occurring. This spurious argument ignores certain basic features of the literature. First of all, there is no evidence that the oscillatory pattern of tongue movements involved in drinking has a local frequency set by basal ganglia mechanisms. In fact, evidence indicates that this frequency is set by brainstem pattern generators [213]. Cataleptic doses of DA antagonists have little or no effect upon the local frequency of licking [65]. Nevertheless, a number of other parameters of oral motor function are altered by systemic or intra-accumbens injections of DA antagonists. Jones and Mogenson [97] observed that intra-accumbens injections of low doses of spiroperidol impaired lap volume and tongue extension in a water licking task. Several additional motor parameters related to licking are impaired by DA antagonists, including lick force [65,66], lick duration [65,66,74], and lick efficiency [87,173]. Striatal DA depletions that severely affected lick force had only slight effects on lick rhythm [183]. Hsiao and Chen [86] demonstrated that the response requirement (i.e. height of the spout) was an important determinant of the effect of DA antagonists on drinking. In short, the literature on dopaminergic involvement in sucrose drinking, like the literature on chow consumption, demonstrates that conditions that suppress sucrose drinking are accompanied by signs of motor dysfunction. These motoric effects of high doses of DA antagonists, therefore, place additional emphasis on studies that employed low to

moderate doses. Injections of up to 25 mg of flupenthixol directly into nucleus accumbens slowed instrumental run speed in an alleyway, but did not suppress sucrose consumption [92]. More recently, it has been demonstrated that DA antagonists injected into nucleus accumbens in low doses that impaired learning or suppressed locomotion did not suppress sucrose consumption [11]. In the presence of such evidence, there is little support for the notion that interference with DA in nucleus accumbens results in deficits in the primary reinforcing or primary incentive characteristics of sucrose, or that these supposed deficits underlie the effects of DA antagonism on instrumental behavior. In the literature dealing with suspected reward functions of DA systems there are two related conceptual problems in terms of how motor functions of DA are discussed. The first problem relates to what has been called the ‘absolute dichotomy’ between reward function and motor function [165]. Before discussing what this refers to, it is first useful to say what this does not refer to. Obviously, proponents of the General Anhedonia Model recognize that DA is involved in motor functions, and according to the DA reinforcement hypothesis DA antagonists should produce both types of effects (e.g. [218]). The term ‘absolute dichotomy’ between reward functions and motor functions does not refer at all to the notion that both types of effects can occur in the same animal at the same time. Instead, this term refers to the notion that the specific reward/reinforcement/motivational functions supposedly impaired by DA antagonists are completely distinct from all aspects of motor function. If one conceives of DA antagonists as blunting ‘hedonia’ or ‘appetite’, then it is clear that these terms are meant to be conceptually distinct from aspects of motor function. Yet a problem with this distinction is that a broader consideration of motivational functions leads one to recognize the overlap between aspects of motor function and aspects of motivation. For example, in describing the incentivemotivation concept, Bindra [21] discusses the production of instrumental behavior in terms of the ‘instigation of action’. Yet, neurologists maintain that DA is important for the ‘initiation of movement’. It is not clear how different these concepts are, though one is meant to have a meaning in a motivational context and the other is clearly motoric, and although one is meant to be ‘psychological’ and the other more ‘neurological’. As noted above, motivational stimuli are conceived of as having activational as well as directional aspects. Obviously, the behavioral activation produced by conditioned or unconditioned motivational stimuli, or by novelty, is related to both motor and motivational functions [154,155,158,159,165]. In this context, it is interesting to note that the words ‘emotion’ and ‘motivation’ are both derived from the Latin word movere (to move).

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

A related problem is that researchers who emphasize the supposed ‘reward’ functions of DA fail to confront directly the wide variety of functions that can be described as motor, and the diverse nature of the motor impairments that can emerge with disruption of the nervous system. In arguing against the notion that DA antagonists produce motor ‘incapacitation’, the fact that motor function is not a unitary phenomenon often is ignored, and the subtle aspects of some motor dysfunctions often are overlooked. Various types of motor dysfunctions can occur, including lower motor neuron paralysis, upper motor neuron paralysis, cerebellar ataxia, parkinsonian akinesia, and apraxia, all of which are quite distinct from each other. It would be convenient for advocates of the DA hypothesis of reward if interference with the basal ganglia simply produced paralysis or simple incapacitation; yet, the basal ganglia just have not cooperated. The DA synapses in accumbens and striatum are several junctions removed from the alpha motor neurons themselves, and it is generally agreed that the basal ganglia are diffuse regulators of motor output [25,138,155,165]. Different aspects of motor function are anatomically dissociable from each other, so it is possible to observe that VLS DA depletions impair skilled arm movements and orofacial motor function but leave gross locomotion basically spared [43,45,95]. The literature on the symptoms of human parkinsonism, as well as studies with animal models, are full of examples of subtle and paradoxical manifestations of the motor sequelae of DA dysfunction. For example, although the within-session ‘extinction’ of responding has become an icon for proponents of the DA hypothesis of reward, similar effects are seen in the motor symptoms of parkinsonism. Haase and Janssen [78] reported that the micrographia shown by patients with neuroleptic-induced parkinsonism is characterized by a progressive worsening within a writing session. They note that ‘‘An increasing degree of narrowing of the writing from stanza to stanza is particularly characteristic, and in typical cases the area covered by the writing assumes the shape of an inverted pyramid’’ (p. 43) [78]. These authors also report that the intensity of finger tapping decreases within a session in patients with neuroleptic-induced parkinsonism (p. 234). Similarly, repeated compression of the hands in parkinsonian patients reveals progressively diminishing motor output [175]. In rats, DA antagonists cause withinsession increments in response duration [118], and within session decrements in lick force [49]. It should be emphasized that parkinsonian patients are not paralyzed, despite an array of motor dysfunctions. Akinetic parkinsonian patients can respond to intense stimulation by showing ‘paradoxical’ kinesia [176], a phenomenon also demonstrable in rats with DA depletions (e.g. [99]). Researchers often use the term ‘sensorimotor’, as well as motor, to describe the functions of

15

DA in basal ganglia [119,196,208]. The notion that interference with DA can blunt behavioral responsiveness to stimuli has been a very powerful one, and has been employed to explain deficits in responsiveness to tactile stimuli [124] eating [212], gating of the startle response [192], and operant reinforcement ‘threshold’ measurements [165]. In discussing the effects of DA antagonists on sucrose consumption, Muscat and Willner [135] noted that ‘The ‘dopamine hypothesis of (sweet) reward’ may thus be a misrepresentation of data derived from under limited conditions of reinforcement. The blunting of reward by DA antagonists may be a specific case of a more general attenuation of the influence of sensory stimuli over behavior’. In considering the cogent nature of this statement, the validity of the use of the term ‘reward’ to describe the deficits produced by DA antagonism or depletion can be called into serious question. In summary, the characteristics of the feeding deficits produced by high doses of DA antagonists and neostriatal DA depletions do not closely resemble the appetite suppression produced by pre-feeding. In contrast, there is clear evidence of dopaminergic involvement in several distinct aspects of motor/sensorimotor function that are important for feeding. Although rats with impaired DA function are not paralyzed, and tend to show normal lick frequencies, they do have problems with lick force, lick duration, lap volume, tongue extension, reaching, grasping, and food handling. Considerable evidence indicates that rats with compromised DA systems generally also show reduced behavioral activity and diminished responsiveness to stimuli, and are highly sensitive to the kinetic requirements of the task being performed. Thus, in the face of such deficits, and in the absence of compelling evidence that the major problem is motivational in nature, attributing the behavioral effects of high doses of DA antagonists or striatal DA depletions to a ‘reward’ deficit seems to be inappropriate. 7.3. Brain DA and learning A recent trend in the literature has been to emphasize that DA systems are involved in aspects of learning. In view of the emphasis of this review on motivational processes, a thorough review of the learning literature would be impossible in the present context. Nevertheless, it is useful to discuss briefly a few points that need to be considered in attempting to integrate this literature with the rest of the paper. First of all, there have been a number of studies of the effects of DA antagonism and DA depletions in various learning tasks, and the results appear to vary greatly from task to task. Short term memory and instrumental discrimination processes were not found to be impaired in some studies [8,12,162,163,202]. Some aspects of S /S associa-

16

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

tive processes may be impaired by DA antagonism, while others appear to be intact [18,210]. Spatial working memory performance in the radial arm maze was not disrupted by high doses of haloperidol injected directly into nucleus accumbens [107], while memory consolidation in a water maze was disrupted by post-trial infusions of sulpiride into nucleus accumbens [179]. The establishment of place conditioning is affected consistently by DA antagonists [4,5,53]. DA systems are involved in the establishment of conditioned reinforcement [14,60,100,225]. Striatal and accumbens DA systems also are involved in aspects of procedural learning [178]. Nevertheless, some important points need to be emphasized in considering this literature in the context of the present review. Studies indicating that DA systems are involved in aspects of learning do not provide support for the notion that DA mediates motivational aspects of primary reward or positive reinforcement. Although it is true that reinforcement involves associative mechanisms, it certainly is not true that all associative mechanisms can be described as positive reinforcement. For example, accumbens DA is known to be activated by stress (see reviews by Gray et al. [75], and Salamone, [157]), and accumbens DA depletions severely disrupt lever pressing avoidance responses [130]. Place aversion, which is a form of punishment, is impaired by DA antagonists [4,53]. Passive avoidance learning, which again involves punishment, and not reinforcement, also is impaired by accumbens DA depletions [177]. It should be emphasized that implicating a brain system in some aspect of learning does not provide evidence that it mediates motivational aspects of ‘reward’. For example, several researchers have suggested that accumbens DA may be involved in aspects of associative conditioning to appetitive, aversive, and even neutral stimuli [75,142,144,227]. Thus, it is clearly inaccurate to refer to these learning studies as providing support for the DA hypothesis of reinforcement or ‘reward’. Most importantly for the scope of the present review, it cannot be said that the learning literature provides any support for the notion that DA systems mediate primary appetitive motivation for natural stimuli such as food. Although DA agonists and antagonists can affect the process of conditioned reinforcement, according to Wolterink et al. [225] ‘‘alterations in primary motivation do not underlie the changes in response to conditioned reinforcement’’. In reviewing the literature on the involvement of nucleus accumbens circuitry in learning, Kelley [102] stated that ‘‘dopamine depletion or antagonism in the accumbens does not affect primary motivation for food’’. Another important consideration is that manipulations can alter the behavioral outcomes in learning experiments by affecting processes other than S /S or S / R associations. The learning literature is full of examples

of attentional, arousal, or behavioral organization problems resulting in impaired performance on memory tasks, either in acquisition or retrieval. Studies of classical conditioning demonstrated that DA antagonists did not appear to block the formation of S /S associations directly, but did affect performance by blunting the conditioned and unconditioned excitatory properties of stimuli [80]. Considerable evidence indicates that accumbens DA is involved in the sensorimotor gating of the startle response [192]. In short, it is possible that DA in accumbens is involved in arousal or attentional processes that affect learning and other cognitive processes [69,75,85,109,165], but that DA itself is not necessarily a direct mediator of the formation of S /S or S /R associations. DA could be acting as a modulator of input/output relations in the neural circuitry in accumbens, and different channels of information could pass through distinct local circuits, so that some specific pathways amplify behavioral reactivity to stimuli, while other pathways amplify the impact of stimuli on cognitive functions. 7.4. Wanting versus liking One of the most interesting and influential ideas to emerge from the DA literature in the last few years has been the notion of ‘incentive-salience’, put forth by Berridge, Robinson and their colleagues. In a series of articles [16 /18], they have argued adamantly and cogently against the notion that DA mediates hedonic aspects of the response to motivational stimuli. These authors point to the substantial literature demonstrating that appetitive taste reactivity to sweet solutions is unimpaired by DA antagonists, whole forebrain DA depletions, or by DA depletions specific to nucleus accumbens [16,18,19,204]. Nevertheless, these authors also seem to agree with the position offered by Smith [184] and others that appetite for food is impaired by interfering with accumbens DA. In order to reconcile what is seen as a potential discrepancy in the literature, it is argued by Berridge and Robinson that incentive motivation is not a unitary phenomenon. Thus, an important distinction is made between ‘liking’ versus ‘wanting’. According to Berridge and Robinson [18] ‘‘dopamine systems are necessary for ‘wanting’ incentives, but not for ‘liking’ them or for learning new ‘likes’ or ‘dislikes’’’. Behavioral tests that are said to measure wanting include ‘consumption tests, choice tests, place preference, instrumental performance’ [18]. In discussing their views, Berridge and Robinson [18] also provided an thorough review of the literature involving studies of the effects of DA depletions on the concurrent lever pressing/chow feeding task and the T-maze task [42,45,162,163,169], which are discussed above. In the Berridge and Robinson [18] article, this discussion was in a section that was intended to review

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

areas of research that apparently do not fit with their incentive salience model. In fact, there only appears to be one major source of disagreement between the position offered by Berridge and Robinson [18] and that offered in the present work. Berridge and Robinson [18] appear to be convinced that the fundamental nature of the impairments produced by DA antagonists and accumbens DA depletions is somewhere in the realm of directional aspects of primary incentive motivation, i.e. the tendency to engage directly in food intake, or the appetite or ‘attraction’ for food. These authors reviewed the results of the concurrent lever pressing/feeding studies (i.e. decreased lever pressing and increased chow consumption after accumbens DA depletions, see references [42,45,162,163,169]), but nevertheless argued that interfering with DA does not necessarily preserve the original degree of incentive motivation for food [18]. Yet, a detailed examination of the literature does not provide evidence for a deficit in ‘attraction’ to food or consumption of food by the conditions that induce the shift away from lever pressing or barrier climbing towards the approach and consumption of food. For example, 0.1 mg/kg haloperidol caused shifts in both the operant and T-maze versions of the response allocation tasks [42,162,163,169]. Yet, this dose of haloperidol did not alter consumption of pellets or chow, or preference between pellets or chow, in freefeeding choice tests, nor did it alter preference between a high versus a low density of food reinforcement when no barrier was present [162,163,169]. Nucleus accumbens DA depletions also caused shifts in behavior in operant and T-maze versions of the response allocation tasks [43 /45,162,163,169], yet detailed analysis of feeding behavior showed that rats with accumbens DA depletions did not show impairments in total food intake, rate of feeding or total time spent feeding [168]. The time spent feeding is an important measure in view of the operant literature showing that time allocation is an important behavioral marker of reinforcement value (e.g. [13]). Reducing appetite for food by pre-feeding or by injection of the stimulant and appetite suppressant amphetamine did not decrease lever pressing and increase chow consumption, but instead, suppressed both types of food gathering behavior [42,169]. In addition, the pattern of effects produced by DA antagonists and accumbens DA depletions in rats performing on the concurrent lever pressing/chow feeding task was different from the pattern of effects produced by the serotonergic appetite suppressant, fenfluramine [161]. Thus, there is no evidence to indicate that the specific conditions that alter behavior on the operant and T-maze response allocation tasks, (e.g. low doses of DA antagonists, accumbens DA depletions) are suppressing lever pressing or barrier climbing because of reduced appetite, or a diminished attraction to food, as measured by consumption itself. Rather, the evidence

17

from this literature indicates that the instrumental response requirement is the major factor determining the shifts in behavior that are observed after DA antagonism or depletion. A potential resolution to this apparent difference of opinion could perhaps be achieved by applying the same logic to ‘wanting’ that Berridge and Robinson apply to the general concept of incentive motivation. It has been argued that reward is not a unitary phenomenon, and this argument provided the conceptual basis for dissociating this complex process into liking versus wanting [16,18,19]. Yet, by a similar logic, it can also be argued that ‘wanting’ is not a unitary phenomenon, and that this aspect of incentive motivation can be further divided into components. In other words, while one could define ‘wanting’ in terms of how much food an organism consumes if it is directly in front of them, other aspects of ‘wanting’ include the tendency to instigate and sustain instrumental actions, and to work to obtain the food (i.e. effort in the reinforcementseeking behavior). It can be argued that, just as ‘wanting’ can be distinguished from ‘liking’, then wanting as ‘appetite to consume’ can be distinguished from wanting as ‘working to obtain’ (i.e. tendency to work for motivational stimulus, and overcome response constraints, activation for engaging in vigorous instrumental actions). Indeed, this distinction is apparent in some of the language used by Berridge and his colleagues in attempting to describe ‘wanting’ [17,18] (see Table 3). Moreover, much of the literature on the effects of accumbens DA depletions and DA antagonism seems to offer evidence in favor of the validity of such a distinction [11,43 /45,162,163,168,169]. Thus, accumbens DA depletions appear to dissociate between different components of ‘wanting’, impairing some aspects, while leaving others intact (Fig. 1). 7.5. Behavioral activation, behavioral economics, and the ergonomics of work requirements in instrumental behavior It is important for organisms to be able to detect and consume food if it is immediately available, yet it also is Table 3 Language that represents distinct components of ‘wanting’ in the incentive salience literature Appetite to consume food ‘. . .an object of attraction’ (p. 313; ref. [18]) ‘. . .whether an animal will choose it, consume it’ (p. 316, ref. [18]) ‘. . .measured by voluntary intake, preference tests’ (p. 317, ref. [18]) Activation to work for food ‘animals will work to acquire. . .’ ‘promote motivation to work for a food reward’ ‘willingness to obtain a reward’

(p. 313, ref. [18]) (p. 182, ref. [17]) (p. 182, ref. [17])

All of these phrases occur in the cited articles, in the context of discussing ‘wanting’ for food and other motivational stimuli.

18

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

Fig. 1. According to the incentive salience model, the process of incentive motivation can be broken down into two components, which are known as ‘liking’ and ‘wanting’. However, the present review of the literature indicates that accumbens DA depletions can blunt components of wanting, while leaving others intact. Thus, in the present review it is suggested that ‘wanting’ can itself be dissociated into distinct components, and that accumbens DA depletions have a greater effect upon the expenditure of effort in reinforcement-seeking behavior, while leaving appetite components basically intact.

critical for organisms to employ instrumental behavior processes (i.e. food seeking behavior) to procure food when it is not immediately accessible, even if it requires considerable time or effort. Instrumental behavior is a very complex and multi-faceted phenomenon, with several factors influencing response output (e.g. Pavlovian and instrumental associative factors, motivational processes, sensorimotor processes, executive function; [1,20,36,50,61,62,154,155,165]). As noted above, instrumental actions increase the proximity and availability of motivational stimuli, but in addition they often are characterized by a high degree of activation, vigor, effort or persistence in work output. Behavioral activation has been emphasized as a fundamental aspect of motivation for several decades [34,35,55,81,106,172]. In some cases, behavioral activation is manifested as broad or general increases in a variety of behaviors such as adjunctive behaviors or schedule-induced motor activities [103,104,129,211]. In addition, the behavioral processes that activate or invigorate animals also influence instrumental behavior [106,154,156,159], and this invigoration conveys adaptive advantages because organisms often are separated from significant stimuli such as food by environmental constraints or obstacles

(i.e. response or procurement ‘costs’). A wide variety of investigators who come from different fields such as psychology and ethology, some of whom study foraging, while others employ ‘economic’ models of operant conditioning, have come to the conclusion that workrelated response procurement ‘costs’ exert a profound effect over instrumental behavior [3,37,38,67,90,91, 98,115 /117,149,188,189]. Behavioral activation can be seen, in part, as an aspect of motivation that facilitates the ability of organisms to overcome instrumental response costs. As well as being an important part of the behavioral literature on motivation, the concept of behavioral activation has had substantial influence over studies of DA function. For example, scheduled presentation of food to food deprived animals can result in a high degree of motor activity [106,129], and this scheduleinduced activity is accompanied by substantial increases in accumbens DA release [129,211]. Schedule-induced activity is blunted by systemic DA antagonists [156,159] and by accumbens DA depletions [129]. Several years ago, it was suggested that nucleus accumbens acts as an interface between the limbic system and the motor system, which facilitates the influence of emotional and motivational systems on motor output [134]. Several researchers have emphasized that accumbens DA is involved in various aspects of behavioral activation (also known as ‘invigoration’ or ‘specific activation’) [52,62,93,105,111/113,129,130,133,154 /156,158, 159,165]. It has been suggested that accumbens DA may be particularly important for mediating the activating effects of conditioned stimuli that elicit and sustain instrumental behavior in the absence of primary reinforcement [154], and several recent studies have elegantly demonstrated how nucleus accumbens and related circuitry is involved in Pavlovian processes related to behavioral activation and the facilitation of instrumental behavior [61,142,144]. Several studies have demonstrated that DA antagonists and accumbens DA depletions reduce response speed [9,41,92,160, 165,168,185]. Moreover, as reviewed above, there is an enormous body of evidence demonstrating that DA systems, particularly in nucleus accumbens, are important for enabling animals to overcome the response procurement costs that separate them from significant stimuli such as food [42,44,45,93,101,110,123,139, 140,154,155,157,159/163,165,169,170,194]. It is evident that the effects of accumbens DA depletions interact very powerfully with the task requirements of the instrumental response. Work-related response procurement costs strongly influence which schedules or tasks are sensitive to the effects of accumbens DA depletions. Nevertheless, one should not assume that caloric expenditure or force requirements, in a literal sense, are the sole factors that make work more difficult in animals that have compromised

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

DA function. Various forms of work, including schedules of reinforcement, not only have force requirements, but they also have temporal components as well. Although we can state, with reasonable support, that the effects of accumbens DA depletions differ depending upon the work requirements of the task, it is nevertheless true that we don’t fully understand which particular aspects of effort render some tasks relatively sensitive to dopaminergic manipulations. Across a broad range of schedules, the effects of accumbens DA depletions seem to depend upon baseline response rate, with higher rate schedules being more greatly sensitive to the effects of DA depletion [160]. Yet, under other conditions, the baseline response rate seems to be less important. For example, if an animal is responding on a FR300 schedule for 6 reinforcement pellets per ratio, accumbens DA depletions have a much greater effect than if a rat is responding on a FR50 schedule for one pellet [170]. This difference is evident even though the two schedules produce basically the same rates of responding under control conditions, and both schedules have the same molar relation between responding and reinforcement. Clearly, animals with DA depletions are not merely sensitive to the instrumental response requirement, but also are sensitive to how the response requirement is organized [170]. In addition, these data suggest that temporal factors, such as the delayed or intermittent nature of reinforcement, can also be important. Rats with accumbens DA depletions appear to be more dependent upon primary reinforcement, and are unable to sustain the output of large ratios over long periods of time in the absence of primary reinforcement [170]. This observation is consistent with data showing that dopaminergic drugs or cell body lesions of nucleus accumbens alter choice based upon different delays of reinforcement [27,28,209]. Yet, the intermittent nature of reinforcement alone also cannot explain why performance on some schedules is so greatly impaired by accumbens DA depletions. For example, the high sensitivity of the FR64 schedule to DA depletion [1] cannot be explained simply because of the intermittence of reinforcement, because interval schedules with similar molar reinforcement densities are much less affected by DA depletion (e.g. [185]). In a recent study [41], the effects of accumbens DA depletions were evaluated using two different interval schedules. One schedule was a VI-30 s, and the other was a VI-30 s with an additional ratio requirement (FR5) attached. Accumbens DA depletion had no effect on VI-30 performance, but did substantially impair responding on the schedule that had the ratio requirement added [41]. Thus, despite the fact that the programmed intermittence of reinforcement was the same in both schedules, and obtained rates of reinforcement were very similar, only the schedule with the higher ratio requirement was sensitive to accumbens DA depletions. It is possible that several

19

factors, including ratio requirements, baseline response rate, dependence upon conditioned stimuli, organizational constraints and temporal factors jointly determine the sensitivity of some schedules to the effects of accumbens DA depletions [41].

8. Conclusions Researchers who have attempted to identify the critical characteristics of reinforcing stimuli or reinforcing activities have generally arrived at an emphasis upon motivational factors. A thorough review of the behavioral literature indicates that, across several different investigators offering a multitude of theoretical approaches, motivation is seen by many as being fundamental to the process of reinforcement. The reinforcer has been described as a goal, a commodity, an incentive, or a stimulus that is being approached, self-administered, attained or preserved. Reinforcers also have been described as activities that are preferred, deprived or in some way being regulated. It is evident that this ‘motivational’ or ‘regulatory’ view of reinforcement has had a profound influence over the hypothesis that DA directly mediates ‘reward’ or ‘reinforcement’ processes. Indeed, proponents of the DA/reward hypothesis regularly cite motivational theorists and employ their language. Nevertheless, considerable evidence indicates that low/moderate doses of DA antagonists, and depletions of DA in nucleus accumbens, can suppress instrumental responding for food while, at the same time, these conditions leave fundamental aspects of reinforcement (i.e. primary or unconditioned reinforcement; primary motivation; primary incentive processes) intact. Several complex features of the literature on dopaminergic involvement in reinforcement were examined above, and it was argued that the simple assertions that DA mediates ‘reward’ or ‘reinforcement’ for natural stimuli such as food are inaccurate and grossly oversimplified. Based upon this review of the literature, it appears as though it is no longer tenable to assert that drugs of abuse are simply turning on the brain’s natural ‘reward system’. In relation to the hypothesis that DA systems are involved in ‘wanting’, but not ‘liking’, it is suggested in the present review that ‘wanting’ has directional aspects (e.g. appetite to consume food) as well as activational aspects (e.g. tendency to instigate instrumental behavior; working to obtain food). Thus, low doses of DA antagonists and accumbens DA depletions do not impair the appetite to consume food, but are seen as disrupting the tendency to work for food. According to behavioral economic principles, DA could be involved in the elasticity of demand for food in terms of the tendency to pay workrelated response costs. This view is consistent with the literature showing that low doses of DA antagonists and

20

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

accumbens DA depletions alter the relative allocation of instrumental responses, making the animals less likely to engage in instrumental responses that have a high degree of work-related response costs ([1,160,161,170], see review in [165]). The effects of accumbens DA depletions on instrumental lever pressing are highly scheduledependent, and ratio schedules in particular are very sensitive to the effects of accumbens DA depletions [1,41,170]. Overall, it appears as though accumbens DA is important for maintaining effort in instrumental responding over time in the absence of primary reinforcement [41,170]. Future research must focus upon how specific aspects of task requirements (i.e. ratio requirements, task difficulty, intermittence of reinforcement, temporal features of response requirements, dependence upon conditioned stimuli) interact with the effects of accumbens DA depletions, and which particular factors determine the sensitivity of some tasks to the effects of DA antagonism or depletion.

[9] [10]

[11]

[12]

[13] [14]

[15]

[16] [17]

Acknowledgements Much of the research by John D. Salamone cited in this manuscript was supported by grants from the National Science Foundation. Many thanks to Keri Nowend, Brian Carlson, Jennifer Trevitt, Susana Mingote and Suzanne Weber for their helpful comments.

[18]

[19]

[20]

References [1] Aberman JE, Salamone JD. Nucleus accumbens dopamine depletions make animals more sensitive to high ratio requirements but do not impair primary food reinforcement. Neuroscience 1999;92:545 /52. [2] Aberman JE, Ward SJ, Salamone JD. Effects of dopamine antagonists and accumbens dopamine depletions on time-constrained progressive ratio performance. Pharmacol Biochem Behav 1998;61:341 /8. [3] Ackroff K, Sclafani A. Palatability and foraging cost interact to control caloric intake. J Exp Psychol Anim Behav Proc 1999;25:28 /36. [4] Acquas E, Carboni E, Leone P, Di Chiarra G. SCH 23390 blocks drug-conditioned place-preference and place aversion: anhedonia (lack of reward) or apathy (lack of motivation) after dopamine receptor blockade. Psychopharmacology 1989;99:151 /5. [5] Agmo A, Galvan A, Talamantes B. Reward and reinforcement produced by drinking sucrose: two processes that may depend on different neurotransmitters. Pharmacol Biochem Behav 1995;52:403 /14. [6] Allison J. Response deprivation, reinforcement and economics. J Exp Anal Behav 1993;60:129 /40. [7] Amsel A, Rashotte ME. Mechanisms of adaptive behavior: Clark L. Hull’s theoretical papers, with commentary. New York: Columbia University Press, 1984. [8] Anisman H, Corradini A, Tombaugh TN, Zacharko RM. Avoidance performance, cue and response /choice discrimina-

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

tion after neuroleptic treatment. Pharmacol Biochem Behav 1982;17:1245 /9. Aparicio C. Assessing haloperidol in rats with the barrier choice paradigm. Suma Psicologica 1998;5:1 /20. Bakshi VP, Kelley AE. Dopaminergic regulation of feeding behavior: I. Differential effects of haloperidol microinjection in three striatal subregions. Psychobiology 1991;19:223 /32. Baldo BA, Sadeghian K, Basso AM, Kelley AE. Effects of selective dopamine D1 or D2 receptor blockade within nucleus accumbens subregions on injestive behavior and associated motor activity, Behav Brain Res, this issue. Bartus RT. Short-term memory in the rhesus monkey: effects of dopamine blockade via acute haloperidol administration. Pharmacol Biochem Behav 1978;9:353 /7. Baum WM, Rachlin HC. Choice as time allocation. J Exp Anal Behav 1969;12:861 /74. Beninger RJ, Miller R. Dopamine D1-like receptors and rewardrelated incentive learning. Neurosci Biobehav Rev 1998;22:335 / 45. Beninger RJ, Ranaldi R. Microinjections of flupenthixol into the caudate-putamen but not the nucleus accumbens, amygdala or frontal cortex of rats produce intra-session declines in foodrewarded operant responding. Behav Brain Res 1993;55:203 /12. Berridge KC. Brain substrates of liking and wanting. Neurosci Biobehav Rev 1995;20:1 /25. Berridge KC. Measuring hedonic impact in animals and infants: microstructure of affective taste reactivity patterns. Neurosci Biobehav Rev 2000;24:173 /98. Berridge KC, Robinson TE. What is the role of dopamine in reward: hedonic impact, reward learning or incentive salience. Brain Res Rev 1998;28:309 /69. Berridge KC, Venier IL, Robinson TE. Taste reactivity analysis of 6-hydroxydopamine-induced aphagia: implications for arousal and anhedonia hypotheses of dopamine function. Behav Neurosci 1989;103:36 /45. Bickel WK, Marsch LA, Carroll ME. Deconstructing relative reinforcement efficacy and situating the measures of pharmacological reinforcement with behavioral economics: a theoretical proposal. Psychopharmacology 2000;153:44 /56. Bindra D. How adaptive behavior is produced: a perceptualmotivational alternative to response-reinforcement. Behav Brain Sci 1978;1:41 /91. Blundell JE. Structure, process and mechanism: case studies in the psychopharmacology of feeding. In: Iversen LL, Iversen SD, Snyder SH, editors. Handbook of psychopharmacology, vol. 19. New York: Plenum Press, 1987:123 /82. Bowers W, Hamilton M, Zacharcho RM, Anisman H. Differential effects of pimozide on response-rate and choice accuracy in a self-stimulation paradigm in mice. Pharmacol Biochem Behav 1985;22:521 /6. Brauer LH, De Wit H. High dose pimozide does not block amphetamine-induced euphoria in normal volunteers. Pharmacol Biochem Behav 1997;56:265 /72. Buchwald NA, Hull CD, Levine MS, Villablanca J. The basal ganglia and the regulation of response and cognitive sets. In: Brazier MA, editor. Growth and development of the brain. New York: Raven Press, 1975:171 /89. Caine SB, Koob GF. Effects of mesolimbic dopamine depletion on responding maintained by cocaine and food. J Exp Anal Behav 1994;61:213 /21. Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex, Neurosci Biobehav Rev, 2002;26:321 /52. Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ. Impulsive choice induced in rats by lesions of the nucleus accumbens core, Science, 2001;292:2499 /501.

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25 [29] Carlson NR. Foundations of physiological psychology. Boston: Allyn and Bacon, 1999. [30] Caul WF, Brindle NA. Schedule-dependent effects of haloperidol and amphetamine: multiple-schedule task shows withinsubject effects. Pharmacol Biochem Behav 2001;68:53 /63. [31] Chausmer AL, Ettenberg A. A role for D2, but not D1, dopamine receptors in the response-reinstating effects of food reinforcement. Pharmacol Biochem Behav 1997;57:681 /5. [32] Chausmer AL, Ettenberg A. Intraacumbens raclopride attenuates amphetamine-induced locomotion, but fails to prevent the response-reinstating properties of food reinforcement. Pharmacol Biochem Behav 1999;62:299 /305. [33] Clifton PG, Rusk IN, Cooper SJ. Effects of dopamine D1 and dopamine D2 antagonists on the free feeding and drinking patterns of rats. Behav Neurosci 1991;105:272 /81. [34] Cofer CN. Motivation and emotion. Glenview, IL: Scott, Foresman, 1972. [35] Cofer CN, Appley MH. Motivation: theory and research. New York: Wiley, 1964. [36] Colwill RM, Rescorla RA. Associative structures in instrumental learning. In: Bower GH, editor. The psychology of learning and motivation. New York: Academic Press, 1986:55 /104. [37] Collier GH, Jennings W. Work as a determinant of instrumental performance. J Comp Physiol Psychol 1969;68:659 /62. [38] Collier GH, Johnson DF, Hill WL, Kaufman LW. The economics of the law of effect. J Exp Anal Behav 1986;46:113 /36. [39] Cooper SJ, Francis J. A microstructural analysis of the effects of presatiation on feeding behavior in the rat. Physiol Behav 1993;53:413 /9. [40] Cooper SJ, Sweeny KF. Effects of spiperone alone and in combination with anorectic agents on feeding parameters in the rat. Neuropharmacology 1980;19:997 /1003. [41] Correa M, Carlson BB, Wisniecki A, Salamone JD. Nucleus accumbens dopamine and work requirements on interval schedules, Behav Brain Res, this issue. [42] Cousins MS, Atherton A, Turner L, Salamone JD. Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behav Brain Res 1996;74:189 /97. [43] Cousins MS, Salamone JD. Involvement of ventrolateral striatal dopamine in movement initiation and execution: a microdialysis and behavioral investigation. Neuroscience 1996;70:849 /59. [44] Cousins MS, Salamone JD. Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/benefit paradigm. Pharmacol Biochem Behav 1994;49:85 / 91. [45] Cousins MS, Sokolowski JD, Salamone JD. Different effects of nucleus accumbens and ventrolateral striatal dopamine depletions on instrumental response selection in the rat. Pharmacol Biochem Behav 1993;46:943 /51. [46] Cousins MS, Trevitt J, Atherton A, Salamone JD. Different behavioral functions of dopamine in nucleus accumbens and ventrolateral striatum: a microdialysis and behavioral investigation. Neuroscience 1999;91:925 /34. [47] Cousins MS, Wei W, Salamone JD. Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology 1994;116:529 /37. [48] Dackis CA, O’Brien CP. Cocaine dependence: a disease of the brain’s reward centers. J Subst Abuse Treat 2001;21:111 /7. [49] Das S, Fowler SC. An update of Fowler and Das: anticholinergic reversal of haloperidol-induced, within-session decrements in rats’ lapping behavior. Pharmacol Biochem Behav 1996;53:853 / 5.

21

[50] Dickinson A, Balleine B. Motivational control of goal-directed action. Anim Learn Behav 1994;22:1 /18. [51] Di Chiara G, Imperato A. Preferential stimulation of dopamine release in the nucleus accumbens by opiates, alcohol, and barbiturates: studies with transcerebral dialysis in freely moving rats. Annal New York Acad Sci 1986;473:367 /81. [52] DiChiara G. Nucleus accumbens shell dopamine and Pavlovian incentive learning: role in motivation and addiction, Behav Brain Res, in press. [53] DiScala G, Sandner G. Conditioned place aversion produced by FG 7142 is attenuated by haloperidol. Psychopharmacology 1989;99:176 /80. [54] Domjan M, Burkhard B. The principles of learning and behavior. Monterey, CA: Brooks/Cole Publishing, 1986. [55] Duffy E. Activation and behavior. New York: Wiley, 1963. [56] Dunnett SB, Iversen SD. Regulatory impairments following selective 6-OHDA lesions of the neostriatum. Behav Brain Res 1982;4:195 /202. [57] Dworkin SI, Guerin GF, Co C, Goeders NE, Smith JE. Lack of an effect of 6-hydroxydopamine lesions of the nucleus accumbens on intravenous morphine self-administration. Pharmacol Biochem Behav 1988;30:1051 /7. [58] Ettenberg A, Koob GF, Bloom F. Response artifact in the measurement of neuroleptic-induced anhedonia. Science 1981;209:357 /9. [59] Evenden JL, Robbins TW. Dissociable effects of D-amphetamine, chlordiazepoxide and alpha-flupenthixol on choice and rate measures of reinforcement in the rat. Psychopharmacology 1983;79:180 /6. [60] Everitt BJ, Cador M, Robbins TW. Interactions between the amygdala and ventral striatum in stimulus-reward association: studies assessing a second-order schedule of sexual reinforcement. Neuroscience 1989;30:63 /75. [61] Everitt BJ, Dickinson A, Robbins TW. The neuropsychological basis of addictive behavior. Brain Res Rev 2001;36:129 /38. [62] Everitt BJ, Parkinson JA, Olmstead MC, Arroyo M, Robledo P, Robbins TW. Associative processes in addiction and reward. The role of amygdala-ventral striatal subsystems. Annal New York Acad Sci 1999;877:412 /38. [63] Fibiger HC, Carter DA, Phillips AG. Decreased intracranial self-stimulation after neuroleptics or 6-hydroxydopamine: evidence for mediation by reward deficits rather than by reduced reward. Psychopharmacology 1976;47:21 /7. [64] Finn M, Jassen A, Baskin P, Salamone JD. Tremulous characteristics of the jaw movements induced by pilocarpine and ventrolateral striatal dopamine depletions. Pharmacol Biochem Behav 1997;57:243 /9. [65] Fowler SC, Das S. Haloperidol-induced decrements in force and duration of rats tongue movements during licking are attenuated by concomitant anticholinergic treatment. Pharmacol Biochem Behav 1994;49:813 /7. [66] Fowler SC, Mortell C. Low doses of haloperidol interfere with rat tongue extensions during licking: a quantitative analysis. Behav Neurosci 1992;106:1 /9. [67] Gannon KN, Smith HV, Tierney KJ. Effects of procurement cost on food consumption in rats. Physiol Behav 1983;31:331 /7. [68] Gardner EL. Brain reward mechanisms. In: Lowinson JH, Ruiz P, Millman RB, editors. Substance abuse. New York: Williams & Wilkins, 1992:70 /99. [69] Garris PA, Rebec GV. Modeling fast dopamine neurotransmission in the nucleus accumbens during behavior, Behav Brain Res, this issue. [70] Gawin FH. Neuroleptic reduction of cocaine-induced paranoia but not euphoria. Psychopharmacology 1986;90:142 /3. [71] Glickman SE, Schiff BB. A biological theory of reinforcement. Psychol Rev 1967;74:81 /109.

22

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

[72] Gramling SE, Fowler SC, Collins KR. Some effects of pimozide on nondeprived rats licking sucrose solutions in an anhedonia paradigm. Pharmacol Biochem Behav 1984;21:617 /24. [73] Gramling SE, Fowler SC, Tizzano JP. Some effects of pimozide on nondeprived rats lever pressing maintained by a sucrose reward in an anhedonia paradigm. Pharmacol Biochem Behav 1987;27:67 /72. [74] Gramling SE, Fowler SC. Effects of neuroleptics on rate and duration of operant versus reflexive licking in rats. Pharmacol Biochem Behav 1985;22:541 /5. [75] Gray JA, Kumari V, Lawrence N, Young AM. Functions of the dopaminergic innervation of the nucleus accumbens. Psychobiology 1999;27:225 /35. [76] Gray T, Wise RA. Effects of pimozide on lever-pressing behavior maintained on an intermittent reinforcement schedule. Pharmacol Biochem Behav 1980;12:931 /5. [77] Gunne LM, Anggard E, Jonsson LE. Clinical trials with amphetamine-blocking drugs. Psychiatr Neurol Neurosurg 1972;75:225 /6. [78] Haase HJ, Janssen PAJ. The action of neuroleptic drugs. Amsterdam: Elsevier, 1985. [79] Haney M, Ward AS, Foltin RW, Fischman MW. Effects of ecopipam, a selective dopamine D1 antagonist, on smoked cocaine self-administration by humans. Psychopharmacology 2001;155:330 /7. [80] Harvey JA, Gormezano I. Effects of haloperidol and pimozide on classical conditioning of the rabbit nictitating membrane response. J Pharmacol Exp Ther 1981;218:712 /9. [81] Hinde RA. Animal behavior: a synthesis of ethology and comparative psychology. New York: McGraw Hill, 1970. [82] Hoebel BG. Neuroscience and motivation: pathways and peptides that define motivational systems. In: Atkinson RC, Herrnstein RJ, Lindzey G, Luce RD, editors. Stevens handbook of experimental psychology, vol. 1. New York: Wiley, 1988:527 / 625. [83] Horvitz JC, Ettenberg A. Haloperidol blocks the responsereinstating effects of food reward: a methodology for separating neuroleptic effects on reinforcement and motor processes. Pharmacol Biochem Behav 1989;31:861 /5. [84] Horvitz JC, Richardson WB, Ettenberg A. Dopamine receptor blockade and reductions in thirst produce different effects on drinking behavior. Pharmacol Biochem Behav 1993;45:725 /8. [85] Horvitz JC. Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum, Behav Brain Res, this issue. [86] Hsiao S, Chen BH. Complex response competition and dopamine blocking: choosing of high cost sucrose solution vs. low cost water solution in rats. Chin J Physiol 1995;38:99 /109. [87] Hsiao S, Smith GP. Raclopride reduces sucrose preference in rats. Pharmacol Biochem Behav 1995;50:121 /5. [88] Hull CL. Principles of behavior. New York: Appleton-CenturyCrofts, 1943. [89] Hull EM, Weber MS, Eaton RC, Dua R, Markowski VP, Lumley L, Moses J. Dopamine receptors in the ventral tegmental area affect motor, but not motivational or reflexive, components of copulation in male rats. Brain Res 1991;554:72 /6. [90] Hursh SR. Behavioral economics. J Exp Anal Behav 1990;42:435 /52. [91] Hursh SR, Raslear TG, Shurtleff D, Bauman R, Simmons L. A cost-benefit analysis of demand for food. J Exp Anal Behav 1988;30:419 /40. [92] Ikemoto S, Panksepp J. Dissociations between appetitive and consummatory responses by phamacological manipulations of reward-relevant brain regions. Behav Neurosci 1996;110:331 /45. [93] Ikemoto S, Panksepp J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with

[94]

[95]

[96]

[97]

[98] [99]

[100]

[101]

[102]

[103]

[104] [105]

[106] [107]

[108]

[109]

[110]

[111]

[112]

special reference to reward-seeking. Brain Res Rev 1999;31:6 / 41. Janssen PAJ, Niemegeers CJE, Schellekens KHL. Is it possible to predict the clinical effects of neuroleptic drugs (major tranquillisers) from animal data. Arzneimittel Forschung 1965;15:104 /17. Jicha GA, Salamone JD. Vacuous jaw movements and feeding deficits in rats with ventrolateral striatal dopamine depletions: possible relation to parkinsonian symptoms. J Neurosci 1991;11:3822 /9. Johnson PI, Parente MA, Stellar JR. NMDA-induced lesions of the nucleus accumbens or the ventral pallidum increase the rewarding efficacy of food deprived rats. Brain Res 1996;722:109 /17. Jones DL, Mogenson GL. Oral motor performance following central dopamine receptor blockade. Eur J Pharmacol 1979;59:11 /21. Kaufman LW. Foraging costs and meal patterns in ferrets. Physiol Behav 1980;25:139 /41. Keefe KA, Salamone JD, Zigmond MJ, Stricker EM. Paradoxical kinesia in Parkinsonism is not caused by dopamine release: studies in an animal model. Arch Neurol 1989;46:1070 /5. Kelley AE, Delfs JM. Dopamine and conditioned reinforcement II. Contrasting effects of amphetamine microinjection into the nucleus accumbens with peptide microinjection into the ventral tegmental area. Psychopharmacology 1991;103:197 /203. Kelley AE, Stinus L. Disappearance of hoarding behavior after 6-hydroxydopamine lesions of the mesolimbic dopamine neurons and its reinstatement with L-DOPA. Behav Neurosci 1985;99:531 /5. Kelley AE. Neural integrative activities of nucleus accumbens subregions in relation to learning and motivation. Psychobiology 1999;27:198 /213. Killeen P. Incentive theory. In: Bernstein D, editor. Response structure and organization. Lincoln, NE: University of Nebraska Press, 1981. Killeen P. On the temporal control of behavior. Psychol Rev 1975;82:89 /115. Killeen PR. Economics, ecologics, and mechanics: the dynamics of responding under conditions of varying motivation. J Exp Anal Behav 1995;64:405 /31. Killeen P, Hanson S, Osbourne S. Arousal: its genesis and manifestation as response rate. Psychol Rev 1978;85:571 /81. Kim JS, Levin ED. Nicotinic, muscarinic and dopaminergic actions in the ventral hippocampus and the nucleus accumbens: effects on spatial working memory in rats. Brain Res 1996;725:231 /40. Kirkpatrick MA, Fowler SC. Force-proportional reinforcement: pimozide does not reduce rats’ emmission of higher forces for sweeter rewards. Pharmacol Biochem Behav 1989;32:499 /504. Kiyatkin EA. Dopamine in the nucleus accumbens: cellular actions, drug- and behavior-associated fluctuations, and a possible role in the organism’s adaptive activity, Behav Brain Res, this issue. Koch M, Schmid A, Scnhnitzler HU. Role of nucleus accumbens dopamine D1 and D2 receptors in instrumental and pavlovian paradigms of conditioned reward. Psychopharmacology 2000;152:67 /73. Koob GF, Riley SJ, Smith SC, Robbins TW. Effects of 6hydroxydopamine lesions of the nucleus accumbens septi and olfactory tubercle on feeding, locomotor activity, and amphetamine anorexia in the rat. J Comp Physiol Psychol 1978;92:917 / 27. Koob GF, Stinus L, Le Moal M. Hyperactivity and hypoactivity produced by lesions to the mesolimbic dopamine system. Behav Brain Res 1981;3:341 /59.

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25 [113] Koob GF, Swerdlow NR. The functional output of the mesolimbic dopamine system. Ann New York Acad Sci 1988;537:216 /27. [114] Kosobud AE, Harris GC, Chapin JK. Behavioral associations of neuronal activity in the ventral tegmental area of the rat. J Neurosci 1994;14:7117 /29. [115] Krebs JR. Optimal foraging: decision rules for predators. In: Krebs JR, Davies WB, editors. Behavioral ecology. Sunderland, MA: Sinauer Associates, 1978. [116] Krebs JR. Optimal foraging: theory and experiment. Nature 1977;268:583 /4. [117] Lea SEG. The psychology and economics of demand. Psychol Bull 1978;85:441 /66. [118] Liao RM, Fowler SC. Haloperidol produces within-session increments in operant response duration in rats. Pharmacol Biochem Behav 1990;36:199 /201. [119] Lidsky TI, Buchwald NA, Manetto C, Schneider JS. A consideration of sensory factors involved in the motor functions of the basal ganglia. Brain Res Rev 1985;9:133 /46. [120] Ljungberg T. Blockade by neuroleptics of water intake and operant responding in the rat: anhedonia, motor deficit or both. Pharmacol Biochem Behav 1987;27:341 /50. [121] Ljungberg T. Differential attenuation of water intake and waterrewarded operant responding by repeated administration of haloperidol and SCH 23390 in the rat. Pharmacol Biochem Behav 1990;35:111 /5. [122] Ljungberg T. Scopolamine reverses haloperidol-attenuated lever pressing for water but not haloperidol-attenuated water intake in the rat. Pharmacol Biochem Behav 1988;29:205 /11. [123] Marinelli M, Barrot M, Simon H, Oberlander C, Dekeyne A, Le Moal M, Piazza PV. Pharmacological stimuli decreasing nucleus accumbens dopamine can act as positive reinforcers but have a low addictive potential. Eur J Neurosci 1998;10:3269 /75. [124] Marshall JF, Richardson JS, Teitelbaum P. Nigrostriatal bundle damage and the lateral hypothalamic syndrome. J Comp Physiol Psychol 1974;87:808 /30. [125] Martin-Iverson MT, Wilke D, Fibiger HC. Effect of haloperidol and D-amphetamine on perceived quantity of food and tones. Psychopharmacology 1987;93:374 /81. [126] Mason ST, Beninger RJ, Fibiger HC, Phillips AG. Pimozideinduced suppression of responding: evidence against a block of food reward. Pharmacol Biochem Behav 1980;12:917 /23. [127] McCullough LD, Cousins MS, Salamone JD. The role of nucleus accumbens dopamine in responding on a continuous reinforcement operant schedule: a neurochemical and behavioral study. Pharmacol Biochem Behav 1993;46:581 /6. [128] McCullough LD, Salamone JD. Anxiogenic drugs beta-CCE and FG 7142 increase extracellular dopamine levels in the nucleus accumbens. Psychopharmacology 1992;109:379 /82. [129] McCullough LD, Salamone JD. Involvement of nucleus accumbens dopamine in the motor activity induced by periodic food presentation: a microdialysis and behavioral study. Brain Res 1992;592:29 /36. [130] McCullough LD, Sokolowsi JD, Salamone JD. A neurochemical and behavioral investigation of the involvement of nucleus accumbens dopamine in instrumental avoidance. Neuroscience 1993;52:919 /25. [131] McFarland K, Ettenberg A. Haloperidol does not affect motivational processes in an operant runway model of foodseeking behavior. Behav Neurosci 1998;112:630 /5. [132] McGeorge AJ, Faull RLM. The organization of the projection from the cerebral cortex to the striatum in the rat, Neuroscience 29:503 /37. [133] Mobini S, Chiang TJ, Ho MY, Bradshaw CM, Szabadi E. Comparison of the effects of clozapine, haloperidol, chlorpromazine and D-amphetamine on performance on a time-con-

[134]

[135]

[136]

[137]

[138]

[139]

[140]

[141]

[142]

[143]

[144]

[145]

[146]

[147] [148] [149]

[150]

[151]

[152]

23

strained progressive ratio schedule and on locomotor behavior in the rat. Psychopharmacology 2000;152:47 /54. Mogenson G, Jones D, Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 1980;14:69 /97. Muscat R, Willner P. Effects of dopamine receptor antagonists on sucrose consumption and preference. Psychopharmacology 1989;99:98 /102. Nader K, Bechara A, van der Kooy D. Neurobiological constraints on behavioral models of motivation. Annu Rev Psychol 1997;48:85 /114. Nann-Vernotica E, Donny EC, Bigelow GE, Walsh SL. Repeated administration of the D1/5 antagonist ecopipam fails to attenuate subjective effects of cocaine. Psychopharmacology 2001;155:338 /47. Neafsy EJ, Hull CD, Buchwald NA. Preparation for movement in the cat. II. Unit activity in the basal ganglia and thalamus. Electroencephalogr Clin Neurophysiol 1978;44:714 /23. Neill DB, Justice JB. An hypothesis for a behavioral function of dopaminergic transmission in nucleus accumbens. In: Chronister RB, Defrance JF, editors. The neurobiology of the nucleus accumbens. Brunswick, Canada: Huer Institute, 1981. Neill DB, Fenton H, Justice JB. Increase in accumbal dopaminergic transmission correlates with response cost not reward of rewarding hypothalamic stimulation, Behav Brain Res, this issue. Nowend KL, Arizzi MN, Carlson BB, Salamone JD. D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption. Pharmacol Biochem Behav 2001;69:373 /82. Parkinson JA, Dalley JW, Cardinal RN, Bamford A, Fehnert B, Lachenal G, Rudarakanchana GN, Halkerston KM, Robbins TW, Everitt BJ. Nucleus accumbens dopamine depletion impairs both acquisistion and performance of appetitive Pavlovian approach behavior: implications for mesoaccumbens dopamine function, Behav Brain Res, this issue. Parkinson JA, Fudge J, Hurd Y, Pennartz C, Peoples L. Finding motivation at Seabrook Island: the ventral striatum, learning, and plasticity. Trends Neurosci 2000;23:383 /4. Parkinson JA, Robbins TW, Everitt BJ. Selective excitotoxic lesions of the nucleus accumbens core and shell differentially affect aversive Pavlovian conditioning to discrete and contextual cues. Psychobiology 1999;27:256 /66. Pecoraro NC, Timberlake WD, Tinsley M. Incentive downshifts evoke search repertoires in rats. J Exp Psychol Anim Behav Proc 1999;25:153 /67. Pitts SM, Horvitz JC. Similar effects of D1/D2 receptor blockade on feeding and locomotor activity. Pharmacol Biochem Behav 2000;65:433 /8. Premack D. Toward empirical behavior laws. I: Positive reinforcement. Psychol Rev 1959;66:219 /33. Rachlin H. Introduction to modern behaviorism. San Francisco: Freeman, 1970. Rashotte ME, Henderson D. Coping with rising food costs in a closed economy: feeding behavior and nocturnal hypothermia in pigeons. J Exp Anal Behav 1988;50:441 /56. Roberts DCS, Corcoran ME, Fibiger HC. On the role of ascending catecholaminergic systems in intravenous self-administration of cocaine. Pharmacol Biochem Behav 1977;6:615 /20. Rolls ET, Rolls BJ, Kelly PH, Shaw SG, Wood RJ, Dale R. The relative attenuation of selstimulation, eating and drinking produced by dopamine receptor blockade. Psychopharmacology 1974;38:219 /30. Rusk IN, Cooper SJ. Parametric studies of selective D1 and D2 antagonists: effects on appetitive and feeding behavior. Behav Pharmacol 1994;5:615 /22.

24

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25

[153] Sabol KE, Neill DB, Wages SA, Church WH, Justice JB. Dopamine depletion in a striatal subregion disrupts performance of a skilled motor task in the rat. Brain Res 1985;335:33 /43. [154] Salamone JD. Behavioral pharmacology of dopamine systems: a new synthesis. In: Willner P, Scheel-Kruger J, editors. The mesolimbic dopamine system: from motivation to action. Cambridge, UK: Cambridge University Press, 1991:598 /613. [155] Salamone JD. Complex motor and sensorimotor functions of accumbens and striatal dopamine: involvement in instrumental behavior processes. Psychopharmacology 1992;107:160 /74. [156] Salamone JD. Different effects of haloperidol and extinction on instrumental behaviors. Psychopharmacology 1986;88:18 /23. [157] Salamone JD. Involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res 1994;61:117 /33. [158] Salamone JD. The actions of neuroleptic drugs on appetitive instrumental behaviors. In: Iversen LL, Iversen SD, Snyder SH, editors. Handbook of psychopharmacology. New York: Plenum Press, 1987:575 /608. [159] Salamone JD. Dopaminergic involvement in activational aspects of motivation: effects of haloperidol on schedule-induced activity, feeding and foraging in rats. Psychobiology 1988;16:96 /206. [160] Salamone JD, Aberman JE, Sokolowski JD, Cousins MS. Nucleus accumbens dopamine and rate of responding: neurochemical and behavioral studies. Psychobiology 1999;27:236 /47. [161] Salamone JD, Arizzi MN, Sandoval MD, Cervone KM, Aberman JE. Dopamine antagonists alter response allocation but do not suppress appetite for food: contrasts between the effects of SKF 83566, raclopride and fenfluramine on a concurrent lever pressing/chow consumption task. Psychopharmacology 2002;160:371 /80. [162] Salamone JD, Cousins MS, Bucher S. Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure, Behav Brain Res, 1994;221 /29. [163] Salamone JD, Cousins MS, McCullough LD, Carriero DL, Berkowitz RJ. Nucleus accumbens dopamine release increases during instrumental lever pressing for food but not food consumption. Pharmacol Biochem Behav 1994;49:25 /31. [164] Salamone JD, Cousins MS, Maio C, Champion M, Turski T, Kovach J. Different behavioral effects of haloperidol, clozapine and thioridazine in a concurrent lever pressing and feeding procedure. Psychopharmacology 1996;125:105 /12. [165] Salamone JD, Cousins MS, Snyder BJ. Behavioral functions of nucleus accumbens dopamine: empirical and conceptual problems with the anhedonia hypothesis. Neurosci Biobehav Rev 1997;21:341 /59. [166] Salamone JD, Kurth P, McCullough LD, Sokolowski JD. The effects of nucleus accumbens dopamine depletions on continuously reinforced operant responding: contrasts with the effects of extinction. Pharmacol Biochem Behav 1995;50:437 /43. [167] Salamone JD, Kurth PA, McCullough LD, Sokolowski JD, Cousins MS. The role of brain dopamine in response initiation: effects of haloperidol and regionally-specific dopamine depletions on the local rate of instrumental responding. Brain Res 1993;628:218 /26. [168] Salamone JD, Mahan K, Rogers S. Ventrolateral striatal dopamine depletions impair feeding and food handling in rats. Pharmacol Biochem Behav 1993;44:605 /10. [169] Salamone JD, Steinpreis RE, McCullough LD, Smith P, Grebel D, Mahan K. Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food-choice procedure. Psychopharmacology 1991;104:515 /21. [170] Salamone JD, Wisniecki A, Carlson BB, Correa M. Nucleus accumbens dopamine depletions make animals highly sensitive

[171]

[172] [173]

[174]

[175] [176]

[177]

[178]

[179]

[180] [181] [182] [183]

[184] [185]

[186] [187]

[188] [189] [190]

[191]

[192]

[193]

to high fixed ratio requirements but do not impair primary food reinforcement. Neuroscience 2001;105:863 /70. Salamone JD, Zigmond MJ, Stricker EM. Characterization of the impaired feeding behavior in rats given haloperidol or dopamine-depleting brain lesions. Neuroscience 1990;39:17 /24. Schlosberg H. Three dimensions in emotion. Psychol Rev 1954;61:81 /8. Schneider LH, Davis JD, Watson CA, Smith GP. Similar effects of raclopride and reduced sucrose concentration on the microstructure of sucrose sham feeding. Eur J Pharmacol 1990;186:61 /70. Schneider LH, Sikorsky CA, Rauhofer EA, Davis JD, Smith GP. Raclopride microinjected into nucleus accumbens inhibits sucrose sham feeding. Appetite 1992;19:215. Schwab RS. Akinesia paradoxica. Electroencephalogr Clin Neurophysiol 1972;31:87 /92. Schwab RS, Zieper I. Effects of mood, motivation, stress and altertness on the performance in Parkinson’s disease. Psychiatry Neurobiol 1965;150:345 /57. Schwarting R, Carey RJ. Deficits in inhibitory avoidance after neurotoxic lesions of the ventral striatum are neurochemically and behaviorally selective. Behav Brain Res 1985;18:279 /83. Setlow B, McGaugh JL. Differential effects of immediate posttraining sulpiride microinfusions into the nucleus accumbens shell and core on Morris water maze retention. Psychobiology 1999;27:248 /55. Setlow B, McGaugh JL. Sulpiride infused into the nucleus accumbens posttraining impairs memory of spatial water maze training. Behav Neurosci 1998;112:603 /10. Sheffield FD, Roby TR. Reward value of a non-nutritive sweet taste. J Comp Physiol Psychol 1950;43:471 /81. Skinner BF. Science and human behavior. New York: Macmillan, 1953. Skinner BF. The behavior of organisms: an experimental analysis. New York: Appleton-Century, 1938. Skitek EB, Fowler SC, Tessel RE. Effects of unilateral striatal dopamine depletion on tongue force and rhythm during licking in rats. Behav Neurosci 1999;113:567 /73. Smith GP. Dopamine and food reward. Prog Psychobiol Physiol Psychol 1995;16:83 /144. Sokolowski JD, Salamone JD. The role of nucleus accumbens dopamine in lever pressing and response allocation: effects of 6OHDA injected into core and dorsomedial shell. Pharmacol Biochem Behav 1998;59:557 /66. Spence KW. Behavior theory and conditioning. New Haven: Yale University Press, 1956. Spivak KJ, Amit Z. Effects of pimozide on appetitive behavior and locomotor activity: dissimilarity of effects when compared to extinction. Physiol Behav 1986;36:457 /63. Staddon JER. Adaptive behavior and learning. Cambridge, UK: Cambridge University Press, 1983. Staddon JER. Operant behavior as adaptation to constraint. J Exp Psychol Gen 1979;108:48 /67. Staddon JER, Ettenger RH. Learning: an Introduction to the principles of adaptive behavior. New York: Harcourt Brace and Jovanovitch, 1989. Stefurak TL, van der Kooy D. Tegmental pedunculopontine lesions in rats decrease saccharin’s rewarding effects but no its memory-improving effects. Behav Neurosci 1994;108:972 /80. Swerdlow NR, Braff DL, Taaid N, Geyer MA. Assessing the validity of am animal model of deficient sensorimotor gating in scnizophrenic patients. Arch Gen Psychiatry 1994;51:139 /54. Szczpka MS, Kwok K, Brot MD, Marck BT, Matsumoto AM, Donahue BA, Palmiter RD. Dopamine production in the caudate putamen restores feeding in dopamine-deficient mice. Neuron 2001;30:819 /28.

J.D. Salamone, M. Correa / Behavioural Brain Research 137 (2002) 3 /25 [194] Szechtman H, Talangbayan H, Ganaran G, Dai H, Eilam D. Dynamics of behavioral sensitization induced by the dopamine agonist quinpirole and a proposed central energy control mechanism. Psychopharmacology 1994;115:95 /104. [195] Tapp JT. Activity, reactivity, and the behavior-directing properties of stimuli. In: Tapp JT, editor. Reinforcement and behavior. New York: Academic Press, 1969:387 /416. [196] Teuber HL, Proctor F. Some effects of basal ganglia lesions in subhuman primates and man. Neuropsychologia 1964;2:85 /93. [197] Thorndike EL. Animal intelligence. New York: Macmillan, 1911. [198] Timberlake W, Allison J. Response deprivation: an empirical approach to instrumental performance. Psychol Rev 1974;81:146 /64. [199] Timberlake W. Behavior systems and reinforcement: an integrative approach. J Exp Anal Behav 1993;60:105 /28. [200] Tolman EC. Purposive behavior in animals and men. New York: Appleton, Century Crofts, 1967. [201] Tombaugh TN, Anisman H, Tombaugh J. Extinction and dopamine receptor blockade after intermittent reinforcement training: failure to observe functional equivalence. Psychopharmacology 1980;70:19 /28. [202] Tombaugh TN, Szostak C, Mills P. Failure of pimozide to disrupt acquisition of light-dark and spatial discrimination problems. Psychopharmacology 1983;79:161 /8. [203] Tombaugh TN, Szostak C, Voorneveld P, Tombaugh JW. Failure to obtain functional equivalence between dopamine receptor blockade and extinction: evidence supporting an sensory-motor conditioning hypothesis. Pharmacol Biochem Behav 1982;16:67 /72. [204] Treit D, Berridge KC. A comparison of benzodiazepine, serotonin, and dopamine agents in taste-reactivity paradigm. Pharmacol Biochem Behav 1990;37:451 /6. [205] Trevitt JT, Carlson BB, Nowend KL, Cervone K, Salamone JD. Substantia nigra pars reticulata is a highly potent site of action for the behavioral effects of the D1 antagonist SCH 23390. Psychopharmacology 2001;156:32 /41. [206] Trevitt JT, Lyons M, Aberman J, Carriero D, Finn M, Salamone JD. Effects of clozapine, thioridazine, risperidone and haloperidol on behavioral tests related to extrapyramidal motor function. Psychopharmacology 1997;132:74 /81. [207] Ungerstedt U. Aphagia and adipsia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. Acta Physiol Scand 1971;82:95 /122. [208] Ungerstedt U, Ljungberg T. Central dopamine neurons and sensory processing. J Psychiatr Res 1974;55:149 /50. [209] Wade TR, de Wit H, Richards JB. Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacology 2000;150:90 /101. [210] Weiner I, Feldon J. Facilitation of latent inhibition by haloperidol in rats. Psychopharmacology 1987;91:248 /53.

25

[211] Weissenborn R, Blaha CD, Winn P, Phillips AG. Scheduleinduced polydipsia and the nucleus accumbens: electrochemical measurements of dopamine efflux and effects of excitotoxic lesions in the core. Behav Brain Res 1996;75:147 /58. [212] White NM. Control of sensorimotor functions by dopaminergic nigrostriatal neurons: influence on eating and drinking. Neurosci Biobehav Rev 1986;10:15 /36. [213] Wiesenfeld Z, Halpern BP, Tapper DN. Licking behavior: evidence of a hypoglossal oscillator. Science 1977;196:1122 /4. [214] Willner P, Chawala K, Sampson D, Sophokleous S, Muscat R. Tests of functional equivalence between pimozide pretreatment, extinction and free feeding. Psychopharmacology 1988;95:423 / 6. [215] Wise RA. Catecholamine theories of reward: a critical review. Brain Res 1978;152:215 /47. [216] Wise RA. Cognitive factors in addiction and nucleus accumbens function: some hints from rodent models. Psychobiology 1986;27:300 /10. [217] Wise RA. Psychomotor stimulant properties of addictive drugs. Annal New York Acad Sci 1988;537:228 /34. [218] Wise RA. The anhedonia hypothesis: Mark III. Behav Brain Sci 1985;8:178 /86. [219] Wise RA. Neuroleptics and operant behavior: the anhedonia hypothesis. Behav Brain Sci 1982;5:39 /87. [220] Wise RA, Colle LM. Pimozide attenuates free feeding: best scores analysis reveals a motivational deficit. Psychopharmacology 1984;84:446 /51. [221] Wise RA, Raptis L. Effects of pre-feeding on food-approach latency and food consumption speed in food deprived rats. Physiol Behav 1985;35:961 /3. [222] Wise RA, Rompre PP. Brain dopamine and reward. Annu Rev Psychol 1989;40:191 /225. [223] Wise RA, Spindler J, De Witt H, Gerber GJ. Neurolepticinduced ‘anhedonia’ in rats: pimozide blocks reward quality of food. Science 1978;201:262 /4. [224] Wise RA, Spindler J, Legult L. Major attenuation of food reward with performance-sparing doses of pimozide in the rat. Can J Psychol 1978;32:77 /85. [225] Wolterink G, Phillips G, Cador M, Donselaar-Wolterink I, Robbins TW, Everitt BJ. Relative roles of ventral striatal D1 and D2 dopamine receptors in responding with conditioned reinforcement. Psychopharmacology 1993;110:355 /64. [226] Xenakis S, Sclafani A. The dopaminergic mediation of a sweet reward in normal and VMH hyperphagic rats. Pharmacol Biochem Behav 1982;16:293 /302. [227] Young AMJ, Ahier RG, Upton RL, Joseph MH, Gray JA. Increased extracellular dopamine in the nucleus accumbens of the rat during associative learning of neutral stimuli. Neuroscience 1998;83:1175 /83. [228] Zahm DS, Brog JS. On the significance of subterritories in the ‘accumbens’ part of the rat ventral striatum. Neuroscience 1992;50:751 /67.