Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

Contents lists available at ScienceDirect

Neuroscience and Biobehavioral Reviews journal homepage: www.elsevier.com/locate/neubiorev

Review

Dopamine modulates neural networks involved in effort-based decision-making Seyed M. Assadi a,b,*, Murat Yu¨cel a,c, Christos Pantelis a a

Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne and Melbourne Health, Australia Psychiatry and Psychology Research Centre, Department of Psychiatry, Tehran University of Medical Sciences, Tehran, Iran c ORYGEN Research Centre, Department of Psychiatry, The University of Melbourne, Australia b

A R T I C L E I N F O

A B S T R A C T

Article history: Received 21 March 2008 Received in revised form 25 October 2008 Accepted 27 October 2008

Recent animal and human studies suggest that the dorsal anterior cingulate cortex (dACC) and its related subcortical structures including nucleus accumbens (NAc) are in the center of a brain network that determines and pursues the best option from available alternatives. Specifically, the involvement of the dACC network in decision-making can be categorized under two broad processes of evaluation and execution. The former aims to determine the most cost-effective option while the latter aims to attain the preferred option. The present article reviews neural and molecular findings to show that the dopamine system might modulate this dACC network at multiple levels to optimize both processes. Several lines of evidence suggest that the dopamine system has a bimodal effect, allows the network to compare different representations in the evaluation phase, and focuses the network on the preferred representation in the execution phase. This is apparently achieved by modulating other neurotransmission systems and by transmitting different signals via D1 vs. D2 receptor subtypes and phasic vs. tonic firing. ß 2008 Elsevier Ltd. All rights reserved.

Keywords: Decision-making Dopamine D1 receptor D2 receptor Dorsal anterior cingulate cortex Basal ganglia

Contents 1. 2.

3.

4.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Neural network of decision-making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1. Rats and T-maze studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2. Primate and human studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Molecular basis of decision-making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1. Dopamine in cortico-basal ganglia models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2. Dopamine’s role in decision-making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions and implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1. Introduction All animals continuously face decision-making circumstances in which they must choose between two or more options. Making apt decisions is vital as it has a significant impact on key evolutionary outcomes such as survival and reproduction (Gibson and Langen, 1996; Krebs, 1978; Mulder, 1990). Decision-making is

* Corresponding author at: Melbourne Neuropsychiatry Centre, Sunshine Hospital, 176 Furlong Road, St. Albans, Vic. 3021, Australia. Tel.: +61 3 83451303; fax: +61 3 83450599. E-mail address: [email protected] (S.M. Assadi). 0149-7634/$ – see front matter ß 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.neubiorev.2008.10.010

383 384 384 386 388 388 388 389 390 390

a complex process and involves different variables. Recent studies have started to shed light on the neurobiology of decision-making. However, such studies vary widely and include investigations conducted across different species from rats to humans, with investigative methods ranging from the behavioral through to the neural and molecular levels. The aim of the present article is to review the previous findings, using a common framework and terminology. We focus specifically on the decisions that are made to solve a current problem and not on those that are aimed at longterm outcomes. The review also focuses on the decisions that are made to exploit the available options to greatest benefit, rather than those that are adventurous and explore novel alternatives. We provide evidence that decision-making is related to a brain

384

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

network centered on the dorsal anterior cingulate cortex (dACC) and its related subcortical structures. We also consider how the dopamine system modulates this network at multiple levels to facilitate an optimal decision. 2. Neural network of decision-making There are considerable connections between the anterior cingulate cortex (ACC) and the striatum including the nucleus accumbens (NAc) across mammalian species. Reciprocal connections between these brain regions were described in humans more than two decades ago (Alexander et al., 1986); the ACC projects to the NAc, which projects back to the ACC via the mediodorsal (Ongur and Price, 2000; Ray and Price, 1993; Yeterian and Pandya, 1988) and anteroventral nuclei of the thalamus (Xiaob and Barbas, 2004; Zikopoulos and Barbas, 2007). Both regions are connected with other limbic structures, namely the amygdala and hippocampus, and with the dopamine-enriched area in the ventral tegmentum (VTA) (Baleydier and Mauguiere, 1980; Floresco and Ghods-Sharifi, 2007; Morecraft et al., 2007; Morecraft and Van Hoesen, 1998; O’Donnell and Grace, 1995). Although the original model of the segregated parallel loops proposed that the ACC was mainly linked with the NAc (Alexander et al., 1986), recent studies have shown that the ACC is also connected with the sensorimotor striatum (Ferry et al., 2000; Voorn et al., 2004). In addition, it has been shown that the lower and upper tiers of the ACC have their own specific basal ganglia connections. The lower tier is linked to the ventromedial striatum (i.e. the NAc and ventral putamen) while the upper tier to the dorsolateral striatum (i.e. the dorsal caudate and the putamen) (Ferry et al., 2000). We review both animal and human studies to provide evidence that this brain network is important in decision-making, with the lower and upper tiers involved in different aspects of decisionmaking. We start with rat studies to provide an overall view and proceed to primate and human studies to examine specific aspects of decision-making. 2.1. Rats and T-maze studies Rat studies have suggested that damage to or dopamine depletion in the ACC-NAc circuit leads to an abnormality in decision-making. In a paradigm developed by Salamone et al. (1994), rats were placed in a T-maze to choose between two arms with different amounts of food (benefits) and task difficulties (costs). In this task, rats had a choice between climbing a barrier to obtain a large reward in one arm vs. running into the other arm without any barrier in order to obtain a small reward (Fig. 1A). Normally, rats preferred the high reward arm even though they had to exert greater effort. However, manipulations affecting the ACC, the NAc, and their dopamine system dramatically altered choice behavior so that rats tended to put less effort and to content themselves with the small reward in the alternative arm (Fig. 1B). The manipulations included lesioning the ACC (Schweimer and Hauber, 2005; Walton et al., 2003) and NAc (Hauber and Sommer, 2007), disrupting ACC-NAc connection (Hauber and Sommer, 2007), and dopamine depletion or blockade in the ACC (Schweimer and Hauber, 2006; Schweimer et al., 2005; but see also Walton et al., 2005) or the NAc (Cousins et al., 1996; Salamone et al., 2003, 1994). A recent study also showed that a disconnection between the ACC and basolateral amygdala led to a similar abnormality in Tmaze performance (Floresco and Ghods-Sharifi, 2007). Interestingly, rats with ACC lesions or dopamine depletion seem to preserve their ability to appreciate costs and benefits in the Tmaze task because reducing the cost or increasing the reward in

Fig. 1. (A) Schematic diagram of the T-maze task. Normal rats tend to choose the left arm (high-cost–high-benefit option) instead of the right arm (low-cost–low-benefit option). (B) Effect of lesion or dopamine depletion on reallocation (R) of the behavioral response from the left arm to the right arm.

the high-cost arm caused these rats to return to the high-cost, high-reward option (Salamone et al., 2007; Schweimer et al., 2005; Walton et al., 2003). Moreover, NAc dopamine depletion does not generally change hedonic and aversive reactions to rewarding and aversive stimuli, respectively, which suggests that dopaminedepleted rats retain their ability to recognize the benefits of rewarding stimuli (Berridge and Robinson, 1998). It has also been shown that genetically modified mice that lack dopamine demonstrate normal preference for sucrose over water, suggesting that perception of the beneficial aspects of stimuli remains intact in the absence of dopamine in the brain (Cannon and Bseikri, 2004). Moreover, motor deficit does not appear to be the cause of abnormal T-maze performance as it has been found following both NAc dopamine depletion, which makes animals slow and hypoactive (Berridge and Robinson, 1998; Cousins et al., 1994), and ACC dopamine depletion, where animals do not show any noticeable hypoactivity (Rudebeck et al., 2006b; Schweimer and Hauber, 2006; Schweimer et al., 2005). Overall, it seems that the ACC, NAc, and associated dopamine system are involved in the decision-making process that occurs between initial sensory perception and final motor performance. However, rat studies have not determined which specific aspects of decision-making are processed by these brain structures (Salamone et al., 2007). Decision-making consists of a set of complex multivariate events such as analyzing different costs and benefits (Stevens et al., 2005; Maynard Smith, 1982) and their probabilities (Green and Myerson, 2004; Kacelnik and Bateson, 1997), taking account of previous outcomes (Kennerley et al., 2006; Nishida, 1997), and processing different motor variables for pursuing the preferred option (Barnes et al., 2005; Graybiel, 2005; Berridge and Robinson, 1998). Taken as a whole, however, decision-making can

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

Fig. 2. Decision-making cycle, which consists of two general processes. The first process is evaluation, which is composed of analysis of cost and benefit and appraisal of pervious outcomes. Evaluation ends in an overall preference for one option over other competing options. The second process is execution of the preference and consists of motivation and action. Motivation refers to mobilization of resources and action to planning a sequence of movements to attain the chosen option. Execution ends in attaining the preferred option.

be considered as a cycle of two general processes, namely, evaluation and execution (Fig. 2). Evaluation refers to all events that lead to a new decision. The process starts with exposure to a new set of options and ends with an overall preference. This includes analysis of costs and benefits of different options as well as appraisal of previous outcomes. Execution refers to all events that attempt to actualize the preference. Execution starts with the overall preference and ends with the attainment of the preferred option. This requires the mobilization of resources and the planning of a sequence of movements to attain the preferred option. Mobilizing resources can be thought of as the physiological aspect of motivation that is aimed at providing the required resources to pursue the preferred option, while action sequencing is aimed at using the mobilized resources in the most effective and efficient manner to achieve the goal. Both evaluation and execution have been suggested to be processed by the ACC-NAc circuit and its dopamine system. For example, a number of studies have proposed that behavioral reallocation following dopamine abnormality in this circuit is due to abnormal cost–benefit analysis (Floresco et al., 2008; Phillips et al., 2007; Rudebeck et al., 2006b). Cost–benefit analysis is a major component of evaluation and involves assessing and weighing up the perceived costs and benefits (Maynard Smith, 1982). Phillips et al. (2007) have recently developed a mathematical model for cost–benefit analysis in the ACC-NAc circuit. In this model, the maximum response cost that the animal would afford for different reward magnitudes follows a hyperbolic curve (Fig. 3). Inhibition of dopamine transmission shifts the curve downwards and reduces the maximum response cost that the animal would allocate to obtain a reward. Enhancement of dopamine transmission, on the other hand, shifts this curve upwards and increases the affordable response cost. In other words, dopamine blockade biases the cost–benefit analysis towards avoiding excessive expenditure whereas dopamine enhancement biases it towards achieving maximum reward. While this model can satisfactorily explain T-maze findings with dopamine depletion, the upward shift following dopamine enhancement is rather hypothetical and does not accord with the bidirectional and biphasic effects observed with dopamine

385

Fig. 3. The proposed effect of dopamine on the maximum affordable cost (modified from Phillips et al., 2007). The maximum response cost as a function of reward magnitude follows a hyperbolic curve. Enhancement or attenuation of the dopamine system shifts the curve upwards or downwards, respectively. The enhancement portion, however, is rather theoretical and is depicted as dopamine dysregulation. See text for further discussion.

enhancement, in which low and high doses of dopamine agonists have opposite and/or dissimilar effects (Seamans and Yang, 2004). This biphasic effect was recently reported in a decision-making task study (Floresco et al., 2008). Therefore, it is probable that moderate dopamine enhancement shifts the cost–benefit curve upwards while extreme augmentation shifts it in the opposite direction. In addition, the hypothesis does not address the complexity of cost–benefit analysis. Costs and benefits are not unitary parameters but, rather, consist of several variables. For example, benefits are characterized by the quality, quantity, and probability of gaining a reward (Green and Myerson, 2004; Kacelnik and Bateson, 1997). Animals prefer certain types of reward over others, large rewards over small ones, and highly probable rewards over those that are poorly predictable. The ‘‘net benefit’’ can be defined as the overall advantage of approached rewards over those of the ignored rewards in a given set of decisions. In contrast, cost variables include the hazard of approaching the reward, the energy and time required for obtaining the given reward (Green and Myerson, 2004; Stevens et al., 2005), and the probability of cost occurrence (Kacelnik and Bateson, 1997). Animals prefer costs that inflict minor or no hazards compared with those with major hazards. They also consider the energy and/or the time that they have to spend in order to obtain a reward (Stevens et al., 2005). The ‘‘net cost’’ can be considered as the sum of the purchased costs minus the sum of the avoided costs in a given set of decisions. However, previous studies including the above model have not clarified which variables in the cost–benefit analysis are changed by dopamine blockade and future studies are needed to explore this. Other studies have suggested that the ACC-NAc circuit is involved in the execution phase of a decision. It has been proposed that lesioning or dopamine depletion in the ACC-NAc circuit dampens motivation and impairs the ability to mobilize the resources needed to pursue a preferred reward. This is usually inferred from animals’ instrumental responses such as bar pressing

386

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

(Nowend et al., 2001) or running in a maze to reach food (Salamone et al., 2003). The inference is based on reasoning that rats’ behavioral responses indicate that the energy resource has been mobilized beforehand. Other studies have suggested that rats’ cortico-basal ganglia circuit is involved in action sequencing (Barnes et al., 2005; Graybiel, 2005), with the dopamine system playing a key role in sequence generation (Daberkow et al., 2005). Involvement of the ACC-NAc circuit and its dopamine system in motivation and action sequencing has also received empirical support from recent primate and human studies (reviewed below). In summary, rat studies have provided some evidence that the ACC-NAc circuit contributes to evaluation and execution and to the analysis, motivation, and action elements of the decision-making cycle. However, inconsistencies exist and further studies are needed to verify the above conclusion; for example, ACC dopamine depletion did not cause abnormal T-maze choice in one study (Walton et al., 2005), the effect of ACC lesion was transient in another study (Rudebeck et al., 2006b), and the lesion did not affect all the effort-based tasks in yet another study (Schweimer and Hauber, 2005), suggesting that the ACC may not be involved in all types of effort-related decisions. Further, it has remained unclear how exactly dopamine depletion or blockade changes behavioral responses in the T-maze task. In particular, further studies are required to clearly untangle variables involved in the cost–benefit analysis. For example, recent rat studies have suggested that the ACC-NAc circuit is not involved in all aspects of cost–benefit analysis. Analysis of time-related costs (i.e. waiting for reward and the phenomenon of delay discounting) is not impaired by NAc dopamine depletion (Wakabayashi et al., 2004; Winstanley et al., 2005) or by an ACC lesion (Rudebeck et al., 2006b; Walton et al., 2007). Instead, delay discounting has been attributed to the orbitofrontal cortex (Kheramin et al., 2003; Mobini et al., 2002; Roesch et al., 2007; Walton et al., 2007), the NAc core (Bezzina et al., 2007; Cardinal et al., 2001), and their dopamine innervations (Cheung et al., 2007; Kheramin et al., 2004; Mobini et al., 2000; Wade et al., 2000). Further, recent studies have proposed that the orbitofrontal cortex is also involved in processing reward probability (Kheramin et al., 2003; Mobini et al., 2002). 2.2. Primate and human studies A paradigm similar to the T-maze task, in which subjects choose between two options with different benefits and costs, has not been tested in primates or humans as yet. However, several studies have examined different aspects of decision-making in humans and non-human primates. These studies have generally indicated that decision-making is dependent on the brain circuit involving the ventral striatum and the medial wall of the prefrontal cortex, especially the dACC. A recent functional imaging study used a probabilistic decision-making task in which normal human subjects chose between two options with different probabilities of winning or loosing money (Hampton et al., 2006; Hampton and O’Doherty, 2007). The study concluded that the combined signals from three brain regions, i.e. the ACC, the medial prefrontal cortex, and the ventral striatum provided the information sufficient to decode the decisions made by subjects. Of these three regions, the dACC stood out as contributing the most in this regard (Hampton and O’Doherty, 2007). Another research group has shown that the activity of the dACC increased as a result of an increase in the number of reward options (Marsh et al., 2007), which can implicate the general involvement of this region in decision-making. Furthermore, studies suggested that the key step of decisionmaking, i.e. choosing one option rather than another, is dependent on the dACC and the ventral striatum (Ernst et al., 2004; Eshel et al., 2007). Finally, a recent study has shown that inhibition of the dACC

using repetitive transcranial magnetic stimulation (rTMS) disrupts subjects’ response-switching (Rushworth et al., 2002). These findings have led to a recent hypothesis suggesting that the primate dACC network is involved in decision-making similar to that of the rat ACC-NAc circuit (Rushworth et al., 2004). There is accumulating evidence that dACC and its related subcortical structures are important in the evaluation process of decision-making. Studies have suggested that this network is involved in cost–benefit analysis. A recent study showed that dACC activity increased when the distance between the reward value of two options diminished (Blair et al., 2006), suggesting a role of the dACC in analyzing the benefits of one option over another. There is also evidence for the involvement of the dACC in analyzing the costs of options. For example, it has been found that tasks that contrast a demanding option with a less demanding one generally activate the dACC (Paus et al., 1998; Raichle et al., 2001). The role of the dACC in the other elements of evaluation has also been studied. For example, the involvement of the dACC in appraisal has been suggested by studies assessing error detection and conflict monitoring, where this region is activated by the occurrence of error or conflict from one trial to another. Recently, it has been proposed that the involvement of the dACC in the monitoring of response conflict could be considered as one instance of a more general outcome monitoring function (Botvinick et al., 2004). In line with this suggestion, a recent study has suggested that the dACC is involved in assessing the consequences of choices (Walton et al., 2004). Two other studies found that the reward-related activity of dACC neurons in a given trial was often modulated by the rewards that were delivered in previous trials (Kennerley et al., 2006; Seo and Lee, 2007). This suggests that neurons in the dACC might be involved in the evaluation of outcomes of a decision (Seo and Lee, 2007) and adjust future decisions on the basis of the history of previous decisions and their outcomes (Kennerley et al., 2006). This may be partly accomplished by predicting error likelihood (Brown and Braver, 2005; Yu¨cel et al., 2007b). Other lines of evidence indicate that the dACC is also important in the execution process and the motivation and action elements of decision-making. Some evidence comes from recent attempts to unravel the functional anatomy of the ACC. There are currently two hypotheses about the functional subdivisions of the ACC. Bush et al. (2000) have proposed that the dACC is mainly involved in the cognitive aspects of behavior while the ventral part of the anterior cingulate cortex (vACC) is involved in affective aspects such as emotion and motivation. This conclusion follows evidence that the vACC is anatomically connected with brain structures such as the NAc and the hypothalamus (Devinsky et al., 1995) and that this region, rather than the dACC, is activated during emotion-related tasks such as the Emotional Counting Stroop (Whalen et al., 1998). This conclusion, however, is in sharp contrast with recent findings that the dACC activation is associated with autonomic responses (Critchley et al., 2003, 2005). Based on the anatomical and physiological data, several researchers have proposed that the supracallosal portion of the ACC can be divided into the lower and upper tiers (Koski and Paus, 2000; Picard and Strick, 1996; Rahm et al., 2006; Yu¨cel et al., 2003). The dACC comprises areas 24a0 –c0 in macaque monkey and 24a0 –c0 and 320 in humans (see Fig. 4). The lower tier consists of 24a0 and 24b0 and the upper tier of 24c0 and 320 . Although the specific roles of each tier remains to be clarified, it seems that the two tiers differ in terms of their association with affective and motor aspects, with the lower tier related to affective aspects and the upper tier to motor aspects of decision-making. Below we provide evidence regarding this functional partitioning. We believe that the evidence supports the notion that the dACC is involved not only in the cognitive, evaluative aspects of decision-

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

Fig. 4. Relative density of connections between different subregions of the ACC and anterior/posterior hypothalamus in macaque monkeys. The density is scaled to the value of Brodmann area 25 (subcallosal cingulate). Similar pattern was observed with the NAc shell. The pattern of connectivity suggests that the dACC (Brodmann area 24) can be divided into upper and lower tiers. Drawn based on Ferry et al. (2000), Ongur et al. (1998).

making (i.e. analysis and appraisal) but also in its affective (i.e. motivation) and motor (i.e. action) aspects. Anatomically, the lower tier of the dACC is connected to the subcortical regions involved in affective processing while the upper tier is not. As shown in Fig. 4, the lower tier, rather than the upper tier, has anatomical connections with the NAc and hypothalamus. Tracer studies in macaque monkeys have shown that the anterior and posterior hypothalamus (Ongur et al., 1998 and personal communication with J.L. Price) as well as the shell region of the NAc (Ferry et al., 2000; Kunishio and Haber, 1994) receive fibers from areas 24a0 –b0 but scarcely from area 24c0 . The latter, as expected of its executive role, sends fibers to the dorsal sensorimotor striatum. Recent findings indicate that the dACC is involved in the execution process including its motivation and action elements. PET studies have shown that the regional blood flow to the dACC increases along with blood pressure during physical or mental effort (e.g. isometric exercise and mental arithmetic tasks). This effect intensifies in patients with pure autonomic failure—a disorder that results in failure of autonomic response to physical or mental effort. The compensatory hyperactivity of the dACC in this disorder strongly suggests that the dACC is important in provoking autonomic responses during goal-directed behaviors (see for review Critchley, 2005). There is also ample evidence that the dACC is involved in action sequencing and in initiating and maintaining goal-directed behaviors (Dosenbach et al., 2006). Several primate studies have identified dACC activation at different stages of goal-directed behaviors including action selection (Matsumoto et al., 2003; Shima and Tanji, 1998) and action progression, with the lower bank of 24c0 likely involved in evaluating the progression by monitoring the reward expectancy in the course of a given trial (Shidara and Richmond, 2002) and the upper bank of 24c0 involved in directing the progression via a general role in keeping track of the progress of a given behavior (Hoshi et al., 2005). In concert with primate findings, a recent human study using rapid event-related 3-T fMRI suggested that the upper tier of the dACC was involved in

387

integrating the target and arm information to conduct a goaldirected behavior (Beurze et al., 2007). Together, it could be concluded that the dACC participates in motivation via mobilization of resources and in action via initiation and maintenance of goal-directed behaviors. The dACC might do the latter not only through its extensive connections with the motor cortex but also by activating the midbrain gray matter. The dACC sends projections to different columns of midbrain periaqueductal gray matter (An et al., 1998). This region is wellknown for generating various behavioral patterns such as foraging (Comoli et al., 2003), active coping strategies (e.g. fight and flight), passive coping strategies (i.e. immobility and withdrawal) (Keay and Bandler, 2001), and sexual and nursing behaviors (Lonstein and Stern, 1998) and is necessary for switching between different behavioral patterns (Sukikara et al., 2006). In summary, recent primate and human studies support the idea that the dACC and its related subcortical structures are important both in the evaluation and execution processes of decision-making. In particular, the dACC seems to process a diverse array of cognitive, affective, and motor data to help in making an apt decision. This view is supported by the monkey electrophysiological studies that have shown that the dACC neurons are functionally heterogeneous (Ito et al., 2003; Matsumoto et al., 2003; Shidara et al., 2005; Shidara and Richmond, 2002; Shima and Tanji, 1998) and have suggested that this region processes various elements of goal-directed behavior (Bush et al., 2002; Richmond et al., 2003; Satterthwaite et al., 2007). This conclusion has the advantage of integrating the diverse roles attributed to this brain region (see Bush et al., 2000 for a review). We suggest that the previous findings about the dACC can be better understood within this framework. The attributed roles might merely reflect the different paradigms used to evaluate dACC function. Roles related to analysis include anticipation of reward and punishment (Knutson et al., 2000), attention for action (Posner et al., 1988), response competition monitoring (Carter et al., 1998), and motor response selection (Paus et al., 1998). Roles related to appraisal include error detection (Kiehl et al., 2000), conflict monitoring (Botvinick et al., 1999), and outcome monitoring (Ito et al., 2003). Roles related to motivation could be motivational valence assignment (Mesulam, 1990) and autonomic control (Critchley et al., 2003). Finally, roles related to action could include reward expectancy encoding (Shidara and Richmond, 2002) and performance adjustment (Ridderinkhof et al., 2004). Decision-making, however, is not limited to this circuit. The circuit acts in concert with other brain regions, especially the dorsolateral, orbitofrontal, and ventromedial prefrontal cortices, the amygdala, the hippocampus, and the dorsal/ventral striatum. It has been suggested that the integrity of the whole prefrontalstriatal circuit is necessary for error detection, which could emphasize that an intact cortical–subcortical network is crucial in decision-making (Hogan et al., 2006; Ullsperger and von Cramon, 2006). Moreover, some aspects of cost–benefit analysis are probably processed in other brain areas. Studies suggested that reward probability may be processed in the mesial prefrontal cortex (Amiez et al., 2005) and the anterior insula (Rolls et al., 2008). The cost related to time (delay discounting) is probably processed in the posterior insula (Wittmann et al., 2007), the lateral prefrontal cortex, and the posterior parietal cortex (McClure et al., 2004). Finally, evidence suggests that social cues for making decisions are processed in the vACC (Somerville et al., 2006) and the orbitofrontal cortex (Rudebeck et al., 2006a). It is also noteworthy to mention that the dACC is probably involved in here-and-now, exploiting decision-making and not in decisions concerning long-term outcomes and exploring novel options. Short-term, here-and-know decisions are concerned with

388

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

the immediate outcomes of ongoing events. Long-term decisions, on the other hand, aim at the ultimate outcome of a series of decisions, irrespective of immediate gains or losses. While the dACC is important for short-term decisions, long-term decisions probably involve the orbitofrontal cortex (Wallis, 2007). Studies of patients with ventromedial prefrontal lesions have shown that short-term decision-making is preserved in these patients but they fail to consider long-term outcomes (Bechara et al., 1999, 2000). Risk-taking, exploratory decisions aim to identify novel options, compared to the conservative, exploiting decisions that aim to exploit the current options as well as possible. It has been suggested that exploratory decision-making is related to the frontopolar cortex and the intraparietal sulcus in humans (Daw et al., 2006). Further studies are needed to assess whether the dACC has any role in exploratory decisions. Finally, it is required to assess whether this circuit is related to conscious aspects of decision-making, to unconscious aspects, or both. A recent study suggested that dACC activity is related to conscious aspects of decision-making (Dehaene et al., 2003), though further studies are needed to examine this finding. 3. Molecular basis of decision-making Assuming that the dACC-striatum-thalamus circuit and its dopamine system are involved in decision-making, the next question is how dopamine modulates this circuit to facilitate an optimal decision. Below, we first review different cortico-basal ganglia models to illustrate an overall view of how the dopamine system modulates the network. Then, we review dopamine studies to provide clues on how this system can facilitate progression of different aspects of decision-making at the molecular level. 3.1. Dopamine in cortico-basal ganglia models Dopamine has a prominent, decision-making-related role in all the models that have been developed in the past two decades to explain the function of the cortico-basal ganglia circuits (see BarGad et al., 2003 for review of these models). In single pathway models, information flows from the cortex, through the striatum to the globus pallidus internus and back to the cortex through the thalamus. Information from multiple sources can either converge (leading to information sharing; Percheron et al., 1984) or flow in parallel (leading to segregated parallel processing; Alexander et al., 1986). Studies in monkeys and humans have suggested that the dopamine system keeps brain circuits segregated both in the basal ganglia (Bergman et al., 1998; Pessiglione et al., 2005) and the cortex (Constantinidis et al., 2002; Goldman-Rakic et al., 2000; Yang et al., 1999), allowing parallel processing of cognitive, affective, and motor components. This could be crucial in decision-making where these different aspects need to happen almost simultaneously. In multiple pathway models, each cortico-basal ganglia loop consists of two competing pathways: the direct pathway mediates a positive feedback while the indirect pathway a negative feedback. It has been proposed that the relative activity of the pathways is controlled by dopaminergic input, which enhances direct pathway activity through D1 receptors while attenuates indirect pathway activity through D2 receptors (Alexander et al., 1986; Gerfen et al., 1990). Dopamine activity, therefore, results in a net positive feedback and leads to the initiation and continuance of loop activity, which is considered essential for goal-directed behavior (DeLong, 1990; Wichmann and DeLong, 1996). In action selection models, the role of the dopamine system is also decisive. These models assume that the basal ganglia choose an action out of the numerous such actions presented by the cortex

(Mink, 1996; Plenz, 2003; Wickens, 1997). For example, in the focused selection model, the basal ganglia potentiates a given activation pattern (promoting a certain response) while inhibiting other competing activation patterns (preventing alternative responses). It has been proposed that the dopamine system triggers a particular activation pattern by strengthening the efficacy of some cortico-striatal synapses while weakening others (Calabresi et al., 2007; Reynolds and Wickens, 2002). Sequence generation models propose that the cortico-basal ganglia circuits are involved in selecting and/or generating action sequences (Berns and Sejnowski, 1998). This is crucial to goaldirected behavior and the action element of decision-making. Studies have suggested that the dopamine system in the basal ganglia is essential for sequence learning (Daberkow et al., 2005; Suri and Schultz, 1998). Finally, the dimensionality reduction model considers the basal ganglia as a multi-layered neural network with a dynamic pattern of activity. The network receives data from various sources, highlights important data, and neglects unimportant ones. This model suggests the dopamine activity via cortico-striatal-dopaminergic synapse triads acts as a reinforcement signal, indicating which data are important (Bar-Gad et al., 2003). Therefore, the dopamine system would be crucial in this model for evaluating the significance of different options/responses during decision-making. Overall, the models can be broadly divided into two groups; those supporting the role of dopamine in evaluation and those in execution. The first group assumes that the dopamine system is involved in the evaluation phase of decision-making. They propose that dopamine helps to differentiate important signals from unimportant ones (e.g. dimensionality reduction model) and to choose a favorable response over alternative responses (e.g. action selection models). The second group considers the dopamine system as a facilitator of the execution phase. These models assume that dopamine manipulates the cortico-basal ganglia macrocircuit to initiate and maintain an action (e.g. multiple pathway models) and to generate the motor sequence that is necessary for that action (e.g. sequence generation models). 3.2. Dopamine’s role in decision-making Human and animal studies have assessed the role of the dopamine system in different aspects of decision-making. These studies, in line with cortico-basal ganglia models, suggest that the dopamine system has a pervasive effect on different elements of decision-making (Schultz, 2007a), including analysis (Cohen et al., 2005; Knutson et al., 2004; Salamone et al., 2007; VrshekSchallhorn et al., 2006), appraisal (de Bruijn et al., 2006; Morris et al., 2006; Schultz and Dickinson, 2000), motivation (Salamone et al., 2007; Both et al., 2005; McLean et al., 2004; Satoh et al., 2003), and action (Cromwell et al., 1998; Daberkow et al., 2005). This pervasive role of the dopamine system in decision-making is not surprising, considering that this system acts in a variety of brain regions, ranging from the midbrain gray matter through to the prefrontal cortex (Hurd et al., 2001). Furthermore, evidence suggests that the dopamine system extensively modulates other neurotransmitter systems (Greengard, 2001), transmits different signals via phasic vs. tonic firing (Grace et al., 2007; Schultz, 2007b), and modulates neural networks differently through low vs. high dopamine concentrations (Trantham-Davidson et al., 2004), pre- vs. post-synaptic receptors (Adell and Artigas, 2004) and D1 vs. D2 receptor subtypes (Seamans et al., 2001; see for review Seamans and Yang, 2004). It has recently been proposed that the dopamine system modulates the pattern of activity in prefrontal networks (Lapish

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

389

Fig. 5. Effect of D1 vs. D2 receptor activation on the prefrontal network. Reproduced with permission from Seamans et al. (2001).

et al., 2007; Seamans et al., 2001; Trantham-Davidson et al., 2004) (Fig. 5). The model has taken account of the complex interplay between dopamine and a variety of ionic and synaptic currents in the prefrontal cortex, especially GABA and NMDA currents. This model has suggested that a predominantly D2 receptor activation (i.e. D2 state) leads to a net reduction of network inhibition. As a result, multiple inputs can gain access to the prefrontal network. This allows multiple representations to be processed simultaneously in prefrontal networks. On the other hand, the model has proposed that a predominantly D1 receptor activation (i.e. D1 state) leads to a net increase in network inhibition so that only strong inputs can persist in the prefrontal network. It has been suggested that phasic, high concentrations of dopamine (>1 mM) induce a transient D2 state while tonic, low concentrations (>500 nM) induce a long-lasting D1 state (Seamans et al., 2001; Trantham-Davidson et al., 2004). These two states fit well with the evaluation and execution phases of decision-making. First, the transient D2 state allows the simultaneous existence of multiple representations, which is required for cost–benefit analysis and outcome appraisal. Subsequently, the longer lasting D1 state stabilizes the selected representation and shuts off the influence of other representations; this keeps the animal focused on the selected goal. A recent study has suggested that D2 receptor reduction is associated with impaired learning from negative outcomes and with decreased dACC activity during performance monitoring (Klein et al., 2007); this supports the above hypothesis that the D2 state of the dACC network is important in outcome appraisal. The above-mentioned changes, however, occur in minutes and have been used to explain the role of the dopamine system in working memory and the dorsolateral prefrontal cortex (Seamans and Yang, 2004). Therefore, it remains to be determined whether similar changes occur in the medial prefrontal cortex within milliseconds to explain the rapid sequences relevant to decision-making. Studies of the basal ganglia have also indicated a similar biphasic reward-related response in the dopamine system. There is some evidence that fast, phasic dopamine activity (lasting 100 nM dopamine concentration) could be considered as an evaluating signal that encodes a prediction error, signaling whether the event is better or worse than expected (Schultz et al., 1997). In contrast, slow, tonic dopamine activity (lasting for seconds with dopamine concentrations of 10–30 nM) seems to be an arousal signal while the animal pursues a motivationally relevant goal. Tonic activity appears to encode reward uncertainty in the period between occurrences of conditioned stimuli to the expected time of reward (Fiorillo et al., 2003). Therefore, the phasic, evaluating signal might be

related to the evaluation phase, where the significance of stimuli is analyzed and the tonic, arousal signal might be related to the execution phase, when the level of arousal could be adjusted according to the level of reward uncertainty (Salamone, 1996). At the basal ganglia level, phasic and tonic firings have been suggested to act via different dopamine receptor subtypes (Bilder et al., 2004; Floresco et al., 2003; Grace, 1991; Schultz, 2007b). In addition, studies have indicated that NAc dopamine might act at the macrocircuit level, modulating inputs from the extrastriatal regions involved in decision-making. These studies have suggested that the NAc dopamine system plays a critical role in response selection and perseverance (see Grace et al., 2007 for review). Studies have found that dopamine signals, through D1 receptors, potentiate the hippocampal inputs while they attenuate the inputs from the basolateral amygdala (Floresco et al., 2001). Dopamine signals also inhibit the medial prefrontal inputs to the NAc through D2 receptors. The net result would be the acquisition and maintenance of a new response strategy because the D1mediated potentiation of the hippocampal inputs leads to the acquisition of a response strategy while the D2-mediated attenuation of the medial prefrontal inputs prevents responseswitching and results in response perseverance (Goto and Grace, 2005). Overall, recent studies provide evidence that the dopamine system probably has a bimodal effect on the dACC-NAc network. In the evaluation phase, dopamine modulates this network so that different variables can be considered, processed, and compared. This enables the network to appraise previous outcomes and to analyze existing costs and benefits. In the execution phase, on the other hand, the dopamine system makes the network focused on the chosen stimulus and the variables related to its attainment; therefore, the network concentrates on the planning and execution of a series of mental and motor actions to attain the chosen option without interference by irrelevant stimuli. 4. Conclusions and implications Decision-making requires processing of a variety of different variables. These variables can be categorized under two broad processes of evaluation and execution. The former consists of outcome appraisal and cost–benefit analysis and aims to determine the best possible option while the latter consists of motivation and action sequencing and aims to attain the preferred option. Recent animal and human studies suggest that the dACC and its related subcortical structures are at the center of a brain network responsible for evaluation and execution of decisions. In addition, accumulating evidence suggests that the dopamine

390

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

system modulates this network at multiple levels to optimize both processes. The dopamine system probably causes this bimodal effect via D1 vs. D2 receptor subtypes and tonic vs. phasic firing and by modulating different synaptic inputs and neurotransmission systems. However, further human studies are needed to verify these hypotheses and to determine the specific variables that are processed by the dACC and its related subcortical regions. Future studies are also needed to tailor the general finding about the dopamine system to the brain network of decision-making. One important step for future studies would be to develop a human decision-making test that specifically assesses the type of decision-making that the dACC network is involved in, i.e. hereand-now, exploiting, effort-related decision-making. The test should incorporate both evaluation and execution phases and should be versatile enough to be conducted with neuroimaging techniques. The test can consist of two options with different amounts of reward that require different levels of effort but should control for the time that subjects spend on each option to rule out time-related costs. These new insights into the possible function of the dACC and its dopamine innervations can have important implications for a number of psychiatric illnesses, especially schizophrenia and impulsive–compulsive spectrum disorders. There is ample evidence that these disorders are associated with significant abnormalities in the dACC (Yu¨cel et al., 2007a,c,d, 2002) and in the dopamine system (Toda and Abi-Dargham, 2007; Volkow et al., 2007). A common feature of these disorders might be that patients often experience the feeling of being forced to pursue deviant ideas and impulses over socially acceptable ones. A dysfunctional dACC network, therefore, may underlie some of the clinical presentations of these disorders. For example, dopamine dysregulation in this network could impair the evaluation phase and the individual’s ability to accurately assess the costs of pursuing deviant ideas and impulses and to learn from previous negative outcomes of such decisions. In addition, an inefficient network that cannot choose and focus on appropriate options may suffer simultaneous presence of different mental representations and frequent intrusion of irrelevant representations, which can lead to formal thought disorders in extreme conditions. Finally, complete failure of the network may lead to an inability to choose, initiate, and maintain goal-directed behavior, resulting in symptoms such as indecisiveness, avolition, and impersistence. These hypotheses need to be examined by future studies.

Acknowledgements Dr Yu¨cel is supported by a NH&MRC Clinical Career Development Award (ID: 509345). Prof Pantelis’s research is supported by NHMRC Program Grant (ID: 350241). The authors thank Dr. Alex Fornito for his helpful comments on the manuscript. References Adell, A., Artigas, F., 2004. The somatodendritic release of dopamine in the ventral tegmental area and its regulation by afferent transmitter systems. Neuroscience and Biobehavioral Reviews 28, 415–431. Alexander, G.E., DeLong, M.R., Strick, P.L., 1986. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience 9, 357–381. Amiez, C., Joseph, J.P., Procyk, E., 2005. Anterior cingulate error-related activity is modulated by predicted reward. European Journal of Neuroscience 21, 3447– 3452. An, X., Bandler, R., Ongur, D., Price, J.L., 1998. Prefrontal cortical projections to longitudinal columns in the midbrain periaqueductal gray in macaque monkeys. Journal of Comparative Neurology 401, 455–479. Baleydier, C., Mauguiere, F., 1980. The duality of the cingulate gyrus in monkey. Neuroanatomical study and functional hypothesis. Brain 103, 525–554.

Bar-Gad, I., Morris, G., Bergman, H., 2003. Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Progress in Neurobiology 71, 439–473. Barnes, T.D., Kubota, Y., Hu, D., Jin, D.Z., Graybiel, A.M., 2005. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161. Bechara, A., Damasio, H., Damasio, A.R., Lee, G.P., 1999. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. Journal of Neuroscience 19, 5473–5481. Bechara, A., Tranel, D., Damasio, H., 2000. Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. Brain 123, 2189– 2202. Bergman, H., Feingold, A., Nini, A., Raz, A., Slovin, H., Abeles, M., Vaadia, E., 1998. Physiological aspects of information processing in the basal ganglia of normal and Parkinsonian primates. Trends in Neurosciences 21, 32–38. Berns, G.S., Sejnowski, T.J., 1998. A computational model of how the basal ganglia produce sequences. Journal of Cognitive Neuroscience 10, 108–121. Berridge, K.C., Robinson, T.E., 1998. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Research Brain Research Reviews 28, 309–369. Beurze, S.M., de Lange, F.P., Toni, I., Medendorp, W.P., 2007. Integration of target and effector information in the human brain during reach planning. Journal of Neurophysiology 97, 188–199. Bezzina, G., Cheung, T.H., Asgari, K., Hampson, C.L., Body, S., Bradshaw, C.M., Szabadi, E., Deakin, J.F., Anderson, I.M., 2007. Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis. Psychopharmacology 193, 423–436. Blair, K., Marsh, A.A., Morton, J., Vythilingam, M., Jones, M., Mondillo, K., Pine, D.C., Drevets, W.C., Blair, J.R., 2006. Choosing the lesser of two evils, the better of two goods: specifying the roles of ventromedial prefrontal cortex and dorsal anterior cingulate in object choice. Journal of Neuroscience 26, 11379–11386. Bilder, R.M., Volavka, J., Lachman, H.M., Grace, A.A., 2004. The catechol-O-methyltransferase polymorphism: relations to the tonic-phasic dopamine hypothesis and neuropsychiatric phenotypes. Neuropsychopharmacology 29, 1943–1961. Both, S., Everaerd, W., Laan, E., Gooren, L., 2005. Effect of a single dose of levodopa on sexual response in men and women. Neuropsychopharmacology 30, 173–183. Botvinick, M., Nystrom, L.E., Fissell, K., Carter, C.S., Cohen, J.D., 1999. Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature 402, 179–181. Botvinick, M.M., Cohen, J.D., Carter, C.S., 2004. Conflict monitoring and anterior cingulate cortex: an update. Trends in Cognitive Sciences 8, 539–546. Brown, J.W., Braver, T.S., 2005. Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118–1121. Bush, G., Luu, P., Posner, M.I., 2000. Cognitive and emotional influences in anterior cingulate cortex. Trends in Cognitive Sciences 4, 215–222. Bush, G., Vogt, B.A., Holmes, J., Dale, A.M., Greve, D., Jenike, M.A., Rosen, B.R., 2002. Dorsal anterior cingulate cortex: a role in reward-based decision making. Proceedings of the National Academy of Sciences of the United States of America 99, 523–528. Calabresi, P., Picconi, B., Tozzi, A., Di Filippo, M., 2007. Dopamine-mediated regulation of corticostriatal synaptic plasticity. Trends in Neurosciences 30, 211–219. Cannon, C.M., Bseikri, M.R., 2004. Is dopamine required for natural reward? Physiology and Behavior 81, 741–748. Cardinal, R.N., Pennicott, D.R., Sugathapala, C.L., Robbins, T.W., Everitt, B.J., 2001. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292, 2499–2501. Carter, C.S., Braver, T.S., Barch, D.M., Botvinick, M.M., Noll, D., Cohen, J.D., 1998. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749. Cheung, T.H., Bezzina, G., Hampson, C.L., Body, S., Fone, K.C., Bradshaw, C.M., Szabadi, E., 2007. Evidence for the sensitivity of operant timing behaviour to stimulation of D(1) dopamine receptors. Psychopharmacology 195, 213–222. Cohen, M.X., Young, J., Baek, J.M., Kessler, C., Ranganath, C., 2005. Individual differences in extraversion and dopamine genetics predict neural reward responses. Brain Research Cognitive Brain Research 25, 851–861. Comoli, E., Ribeiro-Barbosa, E.R., Canteras, N.S., 2003. Predatory hunting and exposure to a live predator induce opposite patterns of Fos immunoreactivity in the PAG. Behavioural Brain Research 138, 17–28. Constantinidis, C., Williams, G.V., Goldman-Rakic, P.S., 2002. A role for inhibition in shaping the temporal flow of information in prefrontal cortex. Nature Neuroscience 5, 175–180. Cousins, M.S., Atherton, A., Turner, L., Salamone, J.D., 1996. Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behavioural Brain Research 74, 189–197. Cousins, M.S., Wei, W., Salamone, J.D., 1994. Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology 116, 529–537. Critchley, H.D., 2005. Neural mechanisms of autonomic, affective, and cognitive integration. Journal of Comparative Neurology 493, 154–166. Critchley, H.D., Mathias, C.J., Josephs, O., O’Doherty, J., Zanini, S., Dewar, B.K., Cipolotti, L., Shallice, T., Dolan, R.J., 2003. Human cingulate cortex and autonomic control: converging neuroimaging and clinical evidence. Brain 126, 2139–2152. Critchley, H.D., Tang, J., Glaser, D., Butterworth, B., Dolan, R.J., 2005. Anterior cingulate activity during error and autonomic response. Neuroimage 27, 885–895.

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393 Cromwell, H.C., Berridge, K.C., Drago, J., Levine, M.S., 1998. Action sequencing is impaired in D1 A-deficient mutant mice. European Journal of Neuroscience 10, 2426–2432. Daberkow, D.P., Kesner, R.P., Keefe, K.A., 2005. Relation between methamphetamine-induced monoamine depletions in the striatum and sequential motor learning. Pharmacology, Biochemistry and Behavior 81, 198–204. Daw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., Dolan, R.J., 2006. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879. de Bruijn, E.R., Sabbe, B.G., Hulstijn, W., Ruigt, G.S., Verkes, R.J., 2006. Effects of antipsychotic and antidepressant drugs on action monitoring in healthy volunteers. Brain Research 1105, 122–129. Dehaene, S., Artiges, E., Naccache, L., Martelli, C., Viard, A., Schurhoff, F., Recasens, C., Martinot, M.L., Leboyer, M., Martinot, J.L., 2003. Conscious and subliminal conflicts in normal subjects and patients with schizophrenia: the role of the anterior cingulate. Proceedings of the National Academy of Sciences of the United States of America 100, 13722–13727. DeLong, M.R., 1990. Primate models of movement disorders of basal ganglia origin. Trends in Neurosciences 13, 281–285. Devinsky, O., Morrell, M.J., Vogt, B.A., 1995. Contributions of anterior cingulate cortex to behaviour. Brain 118, 279–306. Dosenbach, N.U., Visscher, K.M., Palmer, E.D., Miezin, F.M., Wenger, K.K., Kang, H.C., Burgund, E.D., Grimes, A.L., Schlaggar, B.L., Petersen, S.E., 2006. A core system for the implementation of task sets. Neuron 50, 799–812. Ernst, M., Nelson, E.E., McClure, E.B., Monk, C.S., Munson, S., Eshel, N., Zarahn, E., Leibenluft, E., Zametkin, A., Towbin, K., Blair, J., Charney, D., Pine, D.S., 2004. Choice selection and reward anticipation: an fMRI study. Neuropsychologia 42, 1585–1597. Eshel, N., Nelson, E.E., Blair, R.J., Pine, D.S., Ernst, M., 2007. Neural substrates of choice selection in adults and adolescents: development of the ventrolateral prefrontal and anterior cingulate cortices. Neuropsychologia 45, 1270–1279. Ferry, A.T., Ongur, D., An, X., Price, J.L., 2000. Prefrontal cortical projections to the striatum in macaque monkeys: evidence for an organization related to prefrontal networks. Journal of Comparative Neurology 425, 447–470. Fiorillo, C.D., Tobler, P.N., Schultz, W., 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902. Floresco, S.B., Blaha, C.D., Yang, C.R., Phillips, A.G., 2001. Modulation of hippocampal and amygdalar-evoked activity of nucleus accumbens neurons by dopamine: cellular mechanisms of input selection. Journal of Neuroscience 21, 2851–2860. Floresco, S.B., Ghods-Sharifi, S., 2007. Amygdala-prefrontal cortical circuitry regulates effort-based decision making. Cerebral Cortex 17, 251–260. Floresco, S.B., Tse, M.T., Ghods-Sharifi, S., 2008. Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology 33, 1966–1979. Floresco, S.B., West, A.R., Ash, B., Moore, H., Grace, A.A., 2003. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nature Neuroscience 6, 968–973. Gerfen, C.R., Engber, T.M., Mahan, L.C., Susel, Z., Chase, T.N., Monsma Jr., F.J., Sibley, D.R., 1990. D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250, 1429–1432. Gibson, R.M., Langen, T.A., 1996. How do animals choose their mates? Trends in Ecology & Evolution 11, 468–470. Goldman-Rakic, P.S., Muly 3rd, E.C., Williams, G.V., 2000. D(1) receptors in prefrontal cells and circuits. Brain Research. Brain Research Reviews 31, 295–301. Goto, Y., Grace, A.A., 2005. Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nature Neuroscience 8, 805–812. Grace, A.A., 1991. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41, 1–24. Grace, A.A., Floresco, S.B., Goto, Y., Lodge, D.J., 2007. Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends in Neurosciences 30, 220–227. Graybiel, A.M., 2005. The basal ganglia: learning new tricks and loving it. Current Opinion in Neurobiology 15, 638–644. Green, L., Myerson, J., 2004. A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin 130, 769–792. Greengard, P., 2001. The neurobiology of dopamine signaling. Bioscience Reports 21, 247–269. Hampton, A.N., Bossaerts, P., O’Doherty, J.P., 2006. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. Journal of Neuroscience 26, 8360–8367. Hampton, A.N., O’Doherty, J.P., 2007. Decoding the neural substrates of rewardrelated decision making with functional MRI. Proceedings of the National Academy of Sciences of the United States of America 104, 1377–1382. Hauber, W., Sommer, S., 2007. Direct interactions between anterior cingulate cortex and nucleus accumbens mediate effort-based decision making in rats. Society of Neuroscience Abstract 741.27. Hogan, A.M., Vargha-Khadem, F., Saunders, D.E., Kirkham, F.J., Baldeweg, T., 2006. Impact of frontal white matter lesions on performance monitoring: ERP evidence for cortical disconnection. Brain 129, 2177–2188. Hoshi, E., Sawamura, H., Tanji, J., 2005. Neurons in the rostral cingulate motor area monitor multiple phases of visuomotor behavior with modest parametric selectivity. Journal of Neurophysiology 94, 640–656. Hurd, Y.L., Suzuki, M., Sedvall, G.C., 2001. D1 and D2 dopamine receptor mRNA expression in whole hemisphere sections of the human brain. Journal of Chemical Neuroanatomy 22, 127–137.

391

Ito, S., Stuphorn, V., Brown, J.W., Schall, J.D., 2003. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122. Kacelnik, A., Bateson, M., 1997. Risk-sensitivity: crossroads for theories of decision making. Trends in Cognitive Sciences 1, 304–309. Keay, K.A., Bandler, R., 2001. Parallel circuits mediating distinct emotional coping reactions to different types of stress. Neuroscience and Biobehavioral Reviews 25, 669–678. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J., Rushworth, M.F., 2006. Optimal decision making and the anterior cingulate cortex. Nature Neuroscience 9, 940–947. Kheramin, S., Body, S., Ho, M., Velazquez-Martinez, D.N., Bradshaw, C.M., Szabadi, E., Deakin, J.F., Anderson, I.M., 2003. Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis. Behavioural Processes 64, 239–250. Kheramin, S., Body, S., Ho, M.Y., Velazquez-Martinez, D.N., Bradshaw, C.M., Szabadi, E., Deakin, J.F., Anderson, I.M., 2004. Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 175, 206–214. Kiehl, K.A., Liddle, P.F., Hopfinger, J.B., 2000. Error processing and the rostral anterior cingulate: an event-related fMRI study. Psychophysiology 37, 216–223. Klein, T.A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D.Y., Ullsperger, M., 2007. Genetically determined differences in learning from errors. Science 318, 1642–1645. Knutson, B., Bjork, J.M., Fong, G.W., Hommer, D., Mattay, V.S., Weinberger, D.R., 2004. Amphetamine modulates human incentive processing. Neuron 43, 261– 269. Knutson, B., Westdorp, A., Kaiser, E., Hommer, D., 2000. FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 12, 20–27. Koski, L., Paus, T., 2000. Functional connectivity of the anterior cingulate cortex within the human frontal lobe: a brain-mapping meta-analysis. Experimental Brain Research 133, 55–65. Krebs, R.E., 1978. Optimal foraging: decision rules for predators. In: Krebs, J.R., Davies, N.B. (Eds.), Behavioral Ecology: An Evolutionary Approach. Blackwell Scientific Publications, London, pp. 23–63. Kunishio, K., Haber, S.N., 1994. Primate cingulostriatal projection: limbic striatal versus sensorimotor striatal input. Journal of Comparative Neurology 350, 337– 356. Lapish, C.C., Kroener, S., Durstewitz, D., Lavin, A., Seamans, J.K., 2007. The ability of the mesocortical dopamine system to operate in distinct temporal modes. Psychopharmacology 191, 609–625. Lonstein, J.S., Stern, J.M., 1998. Site and behavioral specificity of periaqueductal gray lesions on postpartum sexual, maternal, and aggressive behaviors in rats. Brain Research 804, 21–35. Marsh, A.A., Blair, K.S., Vythilingam, M., Busis, S., Blair, R.J., 2007. Response options and expectations of reward in decision-making: the differential roles of dorsal and rostral anterior cingulate cortex. Neuroimage 35, 979–988. Matsumoto, K., Suzuki, W., Tanaka, K., 2003. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301, 229–232. Maynard Smith, J., 1982. Evolution and the Theory of Games. Cambridge University Press, London. McClure, S.M., Laibson, D.I., Loewenstein, G., Cohen, J.D., 2004. Separate neural systems value immediate and delayed monetary rewards. Science 306, 503–507. McLean, A., Rubinsztein, J.S., Robbins, T.W., Sahakian, B.J., 2004. The effects of tyrosine depletion in normal healthy volunteers: implications for unipolar depression. Psychopharmacology 171, 286–297. Mesulam, M.M., 1990. Large-scale neurocognitive networks and distributed processing for attention, language, and memory. Annals of Neurology 28, 597–613. Mink, J.W., 1996. The basal ganglia: focused selection and inhibition of competing motor programs. Progress in Neurobiology 50, 381–425. Mobini, S., Body, S., Ho, M.Y., Bradshaw, C.M., Szabadi, E., Deakin, J.F., Anderson, I.M., 2002. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology 160, 290–298. Mobini, S., Chiang, T.J., Ho, M.Y., Bradshaw, C.M., Szabadi, E., 2000. Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology 152, 390–397. Morecraft, R.J., McNeal, D.W., Stilwell-Morecraft, K.S., Gedney, M., Ge, J., Schroeder, C.M., van Hoesen, G.W., 2007. Amygdala interconnections with the cingulate motor cortex in the rhesus monkey. Journal of Comparative Neurology 500, 134–165. Morecraft, R.J., Van Hoesen, G.W., 1998. Convergence of limbic input to the cingulate motor cortex in the rhesus monkey. Brain Research Bulletin 45, 209–232. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., Bergman, H., 2006. Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience 9, 1057–1063. Mulder, M.B., 1990. Kipsigis women’s preferences for wealthy men: evidence for female choice in mammals? Behavioral Ecology and Sociobiology 27, 255–264. Nishida, T., 1997. Sexual behavior of adult male chimpanzees of the Mahale Mountains National Park, Tanzania. Primates 38, 379–398. Nowend, K.L., Arizzi, M., Carlson, B.B., Salamone, J.D., 2001. D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption. Pharmacology, Biochemistry and Behavior 69, 373–382. O’Donnell, P., Grace, A.A., 1995. Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. Journal of Neuroscience 15, 3622–3639.

392

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393

Ongur, D., An, X., Price, J.L., 1998. Prefrontal cortical projections to the hypothalamus in macaque monkeys. Journal of Comparative Neurology 401, 480–505. Ongur, D., Price, J.L., 2000. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cerebral Cortex 10, 206– 219. Paus, T., Koski, L., Caramanos, Z., Westbury, C., 1998. Regional differences in the effects of task difficulty and motor output on blood flow response in the human anterior cingulate cortex: a review of 107 PET activation studies. Neuroreport 9, R37–47. Percheron, G., Yelnik, J., Francois, C., 1984. A golgi analysis of the primate globus pallidus. III. Spatial organization of the striato-pallidal complex. Journal of Comparative Neurology 227, 214–227. Pessiglione, M., Czernecki, V., Pillon, B., Dubois, B., Schupbach, M., Agid, Y., Tremblay, L., 2005. An effect of dopamine depletion on decision-making: the temporal coupling of deliberation and execution. Journal of Cognitive Neuroscience 17, 1886–1896. Phillips, P.E., Walton, M.E., Jhou, T.C., 2007. Calculating utility: preclinical evidence for cost–benefit analysis by mesolimbic dopamine. Psychopharmacology (Berlin) 191, 483–495. Picard, N., Strick, P.L., 1996. Motor areas of the medial wall: a review of their location and functional activation. Cerebral Cortex 6, 342–353. Plenz, D., 2003. When inhibition goes incognito: feedback interaction between spiny projection neurons in striatal function. Trends in Neurosciences 26, 436–443. Posner, M.I., Petersen, S.E., Fox, P.T., Raichle, M.E., 1988. Localization of cognitive operations in the human brain. Science 240, 1627–1631. Rahm, B., Opwis, K., Kaller, C.P., Spreer, J., Schwarzwald, R., Seifritz, E., Halsband, U., Unterrainer, J.M., 2006. Tracking the subprocesses of decision-based action in the human frontal lobes. Neuroimage 30, 656–667. Raichle, M.E., MacLeod, A.M., Snyder, A.Z., Powers, W.J., Gusnard, D.A., Shulman, G.L., 2001. A default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America 98, 676–682. Ray, J.P., Price, J.L., 1993. The organization of projections from the mediodorsal nucleus of the thalamus to orbital and medial prefrontal cortex in macaque monkeys. Journal of Comparative Neurology 337, 1–31. Reynolds, J.N., Wickens, J.R., 2002. Dopamine-dependent plasticity of corticostriatal synapses. Neural Networks 15, 507–521. Richmond, B.J., Liu, Z., Shidara, M., 2003. Neuroscience. Predicting future rewards. Science 301, 179–180. Ridderinkhof, K.R., Ullsperger, M., Crone, E.A., Nieuwenhuis, S., 2004. The role of the medial frontal cortex in cognitive control. Science 306, 443–447. Roesch, M.R., Calu, D.J., Burke, K.A., Schoenbaum, G., 2007. Should I stay or should I go? Transformation of time-discounted rewards in orbitofrontal cortex and associated brain circuits. Annals of the New York Academy of Sciences 1104, 21–34. Rolls, E.T., McCabe, C., Redoute, J., 2008. Expected value, reward outcome, and temporal difference error representations in a probabilistic decision task. Cerebral Cortex 18, 652–663. Rudebeck, P.H., Buckley, M.J., Walton, M.E., Rushworth, M.F., 2006a. A role for the macaque anterior cingulate gyrus in social valuation. Science 313, 1310–1312. Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman, D.M., Rushworth, M.F., 2006b. Separate neural pathways process different decision costs. Nature Neuroscience 9, 1161–1168. Rushworth, M.F., Hadland, K.A., Paus, T., Sipila, P.K., 2002. Role of the human medial frontal cortex in task switching: a combined fMRI and TMS study. Journal of Neurophysiology 87, 2577–2592. Rushworth, M.F., Walton, M.E., Kennerley, S.W., Bannerman, D.M., 2004. Action sets and decisions in the medial frontal cortex. Trends in Cognitive Sciences 8, 410– 417. Salamone, J.D., 1996. The behavioral neurochemistry of motivation: methodological and conceptual issues in studies of the dynamic activity of nucleus accumbens dopamine. Journal of Neuroscience Methods 64, 137–149. Salamone, J.D., Correa, M., Farrar, A., Mingote, S.M., 2007. Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191, 461–482. Salamone, J.D., Correa, M., Mingote, S., Weber, S.M., 2003. Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse. Journal of Pharmacology and Experimental Therapeutics 305, 1–8. Salamone, J.D., Cousins, M.S., Bucher, S., 1994. Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behavioural Brain Research 65, 221–229. Satoh, T., Nakai, S., Sato, T., Kimura, M., 2003. Correlated coding of motivation and outcome of decision by dopamine neurons. Journal of Neuroscience 23, 9913– 9923. Satterthwaite, T.D., Green, L., Myerson, J., Parker, J., Ramaratnam, M., Buckner, R.L., 2007. Dissociable but inter-related systems of cognitive control and reward during decision making: evidence from pupillometry and event-related fMRI. Neuroimage 37, 1017–1031. Schultz, W., 2007a. Behavioral dopamine signals. Trends in Neurosciences 30, 203– 210. Schultz, W., 2007b. Multiple dopamine functions at different time courses. Annual Review of Neuroscience 30, 259–288. Schultz, W., Dayan, P., Montague, P.R., 1997. A neural substrate of prediction and reward. Science 275, 1593–1599.

Schultz, W., Dickinson, A., 2000. Neuronal coding of prediction errors. Annual Review of Neuroscience 23, 473–500. Schweimer, J., Hauber, W., 2005. Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy. Learning and Memory 12, 334–342. Schweimer, J., Hauber, W., 2006. Dopamine D1 receptors in the anterior cingulate cortex regulate effort-based decision making. Learning and Memory 13, 777– 782. Schweimer, J., Saft, S., Hauber, W., 2005. Involvement of catecholamine neurotransmission in the rat anterior cingulate in effort-related decision making. Behavioral Neuroscience 119, 1687–1692. Seamans, J.K., Gorelova, N., Durstewitz, D., Yang, C.R., 2001. Bidirectional dopamine modulation of GABAergic inhibition in prefrontal cortical pyramidal neurons. Journal of Neuroscience 21, 3628–3638. Seamans, J.K., Yang, C.R., 2004. The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Progress in Neurobiology 74, 1–58. Seo, H., Lee, D., 2007. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. Journal of Neuroscience 27, 8366–8377. Shidara, M., Mizuhiki, T., Richmond, B.J., 2005. Neuronal firing in anterior cingulate neurons changes modes across trials in single states of multitrial reward schedules. Experimental Brain Research 163, 242–245. Shidara, M., Richmond, B.J., 2002. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711. Shima, K., Tanji, J., 1998. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338. Somerville, L.H., Heatherton, T.F., Kelley, W.M., 2006. Anterior cingulate cortex responds differentially to expectancy violation and social rejection. Nature Neuroscience 9, 1007–1008. Stevens, J.R., Rosati, A.G., Ross, K.R., Hauser, M.D., 2005. Will travel for food: spatial discounting in two new world monkeys. Current Biology 15, 1855–1860. Sukikara, M.H., Mota-Ortiz, S.R., Baldo, M.V., Felicio, L.F., Canteras, N.S., 2006. A role for the periaqueductal gray in switching adaptive behavioral responses. Journal of Neuroscience 26, 2583–2589. Suri, R.E., Schultz, W., 1998. Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Experimental Brain Research 121, 350–354. Toda, M., Abi-Dargham, A., 2007. Dopamine hypothesis of schizophrenia: making sense of it all. Current Psychiatry Reports 9, 329–336. Trantham-Davidson, H., Neely, L.C., Lavin, A., Seamans, J.K., 2004. Mechanisms underlying differential D1 versus D2 dopamine receptor regulation of inhibition in prefrontal cortex. Journal of Neuroscience 24, 10652–10659. Ullsperger, M., von Cramon, D.Y., 2006. The role of intact frontostriatal circuits in error processing. Journal of Cognitive Neuroscience 18, 651–664. Volkow, N.D., Fowler, J.S., Wang, G.J., Swanson, J.M., Telang, F., 2007. Dopamine in drug abuse and addiction: results of imaging studies and treatment implications. Archives of Neurology 64, 1575–1579. Voorn, P., Vanderschuren, L.J., Groenewegen, H.J., Robbins, T.W., Pennartz, C.M., 2004. Putting a spin on the dorsal-ventral divide of the striatum. Trends in Neurosciences 27, 468–474. Vrshek-Schallhorn, S., Wahlstrom, D., Benolkin, K., White, T., Luciana, M., 2006. Affective bias and response modulation following tyrosine depletion in healthy adults. Neuropsychopharmacology 31, 2523–2536. Wade, T.R., de Wit, H., Richards, J.B., 2000. Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacology 150, 90–101. Wakabayashi, K.T., Fields, H.L., Nicola, S.M., 2004. Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward. Behavioural Brain Research 154, 19–30. Wallis, J.D., 2007. Orbitofrontal cortex and its contribution to decision-making. Annual Review of Neuroscience 30, 31–56. Walton, M.E., Bannerman, D.M., Alterescu, K., Rushworth, M.F., 2003. Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. Journal of Neuroscience 23, 6475–6479. Walton, M.E., Croxson, P.L., Rushworth, M.F., Bannerman, D.M., 2005. The mesocortical dopamine projection to anterior cingulate cortex plays no role in guiding effort-related decisions. Behavioral Neuroscience 119, 323–328. Walton, M.E., Devlin, J.T., Rushworth, M.F., 2004. Interactions between decision making and performance monitoring within prefrontal cortex. Nature Neuroscience 7, 1259–1265. Walton, M.E., Rudebeck, P.H., Bannerman, D.M., Rushworth, M.F., 2007. Calculating the cost of acting in frontal cortex. Annals of the New York Academy of Sciences 1104, 340–356. Whalen, P.J., Bush, G., McNally, R.J., Wilhelm, S., McInerney, S.C., Jenike, M.A., Rauch, S.L., 1998. The emotional counting Stroop paradigm: a functional magnetic resonance imaging probe of the anterior cingulate affective division. Biological Psychiatry 44, 1219–1228. Wichmann, T., DeLong, M.R., 1996. Functional and pathophysiological models of the basal ganglia. Current Opinion in Neurobiology 6, 751–758. Wickens, J., 1997. Basal ganglia: structure and computations. Network: Computation in Neural Systems 8, R77–R109. Winstanley, C.A., Theobald, D.E., Dalley, J.W., Robbins, T.W., 2005. Interactions between serotonin and dopamine in the control of impulsive choice in rats: therapeutic implications for impulse control disorders. Neuropsychopharmacology 30, 669–682.

S.M. Assadi et al. / Neuroscience and Biobehavioral Reviews 33 (2009) 383–393 Wittmann, M., Leland, D.S., Paulus, M.P., 2007. Time and decision making: differential contribution of the posterior insular cortex and the striatum during a delay discounting task. Experimental Brain Research 179, 643–653. Xiaob, D., Barbas, H., 2004. Circuits through prefrontal cortex, basal ganglia, and ventral anterior nucleus map pathways beyond motor control. Thalamus & Related Systems 2, 325–343. Yang, C.R., Seamans, J.K., Gorelova, N., 1999. Developing a neuronal model for the pathophysiology of schizophrenia based on the nature of electrophysiological actions of dopamine in the prefrontal cortex. Neuropsychopharmacology 21, 161–194. Yeterian, E.H., Pandya, D.N., 1988. Corticothalamic connections of paralimbic regions in the rhesus monkey. Journal of Comparative Neurology 269, 130–146. Yu¨cel, M., Brewer, W.J., Harrison, B.J., Fornito, A., O’Keefe, G.J., Olver, J., Scott, A.M., Egan, G.F., Velakoulis, D., McGorry, P.D., Pantelis, C., 2007a. Anterior cingulate activation in antipsychotic-naive first-episode schizophrenia. Acta Psychiatrica Scandinavica 115, 155–158. Yu¨cel, M., Harrison, B.J., Wood, S.J., Fornito, A., Clarke, K., Wellard, R.M., Cotton, S., Pantelis, C., 2007b. State, trait and biochemical influences on human anterior cingulate function. Neuroimage 34, 1766–1773.

393

Yu¨cel, M., Harrison, B.J., Wood, S.J., Fornito, A., Wellard, R.M., Pujol, J., Clarke, K., Phillips, M.L., Kyrios, M., Velakoulis, D., Pantelis, C., 2007c. Functional and biochemical alterations of the medial frontal cortex in obsessive–compulsive disorder. Archives of General Psychiatry 64, 946–955. Yu¨cel, M., Lubman, D.I., Harrison, B.J., Fornito, A., Allen, N.B., Wellard, R.M., Roffel, K., Clarke, K., Wood, S.J., Forman, S.D., Pantelis, C., 2007d. A combined spectroscopic and functional MRI investigation of the dorsal anterior cingulate region in opiate addiction. Molecular Psychiatry 12 (611), 691– 702. Yu¨cel, M., Pantelis, C., Stuart, G.W., Wood, S.J., Maruff, P., Velakoulis, D., Pipingas, A., Crowe, S.F., Tochon-Danguy, H.J., Egan, G.F., 2002. Anterior cingulate activation during Stroop task performance: a PET to MRI coregistration study of individual patients with schizophrenia. American Journal of Psychiatry 159, 251–254. Yu¨cel, M., Wood, S.J., Fornito, A., Riffkin, J., Velakoulis, D., Pantelis, C., 2003. Anterior cingulate dysfunction: implications for psychiatric disorders? Journal of Psychiatry and Neuroscience 28, 350–354. Zikopoulos, B., Barbas, H., 2007. Parallel driving and modulatory pathways link the prefrontal cortex and thalamus. PLoS ONE 2, e848.