L-DOPA modulates activity in the vmPFC, Nucleus Accumbens and VTA during threat extinction learning in humans

Learning to be safe is central for adaptive behaviour when threats are no longer present. Detecting the absence of an expected threat is key for threat extinction learning and an essential process for the behavioural treatment of anxiety related disorders. One possible mechanism underlying extinction learning is a dopaminergic mismatch signal that encodes the absence of an expected threat. Here we show that such a dopamine-related pathway underlies extinction learning in humans. Dopaminergic enhancement via administration of L-DOPA (vs. Placebo) was associated with reduced retention of differential psychophysiological threat responses at later test, which was mediated by activity in the ventromedial prefrontal cortex that was specific to extinction learning. L-DOPA administration enhanced signals at the time-point of an expected, but omitted threat in extinction learning within the nucleus accumbens, which were functionally coupled with the ventral tegmental area and the amygdala. Computational modelling of threat expectancies further revealed prediction error encoding in nucleus accumbens that was reduced when L-DOPA was administered. Our results thereby provide evidence that extinction learning is influenced by L-DOPA and provide a mechanistic perspective to augment extinction learning by dopaminergic enhancement in humans. Summary COI statement No conflict of interest for any of the authors. The funder had no role in the design of the study, the collection and analysis of data, or the preparation of the manuscript.


Introduction
In order to thrive in dangerous environments, it is important to know when threats are disappearing and situations become safe. As such, safety learning is central for adaptive behaviour and deficits characterize symptoms in a wide range of anxiety related disorders [1][2][3][4]. Yet, the neuropharmacological mechanism to augment safety learning by encoding the absence of potential threats or aversive outcomes in humans are not completely understood.
Safety learning is often investigated in laboratory protocols of extinction training. Here, a learned predictor (conditioned stimulus, CS) for an aversive outcome (unconditioned stimulus, US) is turning into a safety signal when the expected aversive outcome is omitted. This omission of the expected US after CS presentation is thought to drive extinction or safety learning. Extinction learning is held to involve learning of a new CS-no US association that inhibits the acquired CS-US association [5,6]. However, it is only incompletely understood which neural system in humans detects the omission of the expected aversive outcome and, hence, initiates a shift from threat to safety. Studies in drosophila [7] and rodents [8][9][10][11][12][13][14][15] revealed that the omission of an expected aversive outcomes depends on signals in the dopaminergic system. In rodents, this involved dopaminergic neurons in the ventral tegmental area (VTA), the nucleus accumbens and the medial prefrontal cortex, as well as projections between the VTA and nucleus accumbens [8,11,16,17]. Importantly, these neural regions were also found to underpin the processing of rewarding outcomes. When signalling rewards, this system does not simply detect a rewarding outcome, but codes a difference between the expected reward and the actual outcome in form of an expectancy violation or prediction error [18]. In other words, reward-related response in the VTA, nucleus accumbens and vmPFC reflect outcomes that are better than expected.
Similarly, the omission of an expected aversive US, which could be framed as "better than expected," might well be under the influence of a dopaminergic signal: At the time-point of US omission a dopaminergic system might encode an expectancy violation that signals the difference between the expected aversive US and the omitted aversive outcome (for review see [19][20][21][22]).
Even though this idea has not been formally tested, it is supported by two functional neuroimaging study in humans. These studies provided initial evidence that computational modelling of an prediction error for the omitted aversive outcome during extinction training involves activity in the nucleus accumbens [23,24] and that this activity was modulated by a genetic variance of the dopamine transporter gene [24]. Additionally, there is cross-species evidence for enhanced extinction memory consolidation by augmented dopaminergic transmission after extinction training (by administration of L-DOPA [25][26][27]). These latter studies suggest that dopaminergic enhancement of extinction memory retrieval is mediated by augmenting activity in the ventral part of the medial prefrontal cortex (vmPFC), a structure that is central for extinction learning and memory retrieval [28][29][30][31]. It is, however, not clear if enhancing of dopaminergic neurotransmission would strengthen extinction learning by modulating vmPFC activity.
In this study, we tested if extinction learning is associated with activity changes in the vmPFC and if such activity is modulated by administration of the dopaminergic precursor L-DOPA. We further tested if the unexpected omission of the US during extinction learning is coded in midbrain pathways that connect the VTA and the nucleus accumbens and if activity within this pathway is modulated by L-DOPA. Based on previous studies [25,27] we hypothesized that L-DOPA administration before extinction training would decrease threat responses at retention tests.

Figure 1 | Behavioural and psychophysiological outcome measures on day 1,2 and 3 a) US expectancy, b) SCR and c) Fear ratings reflect successful acquisition of CS-US
contingencies during acquisition and decreasing responses during extinction training. Retention of CS-US memory was evident during retention test on day 3, as well as initial enhancement of responses after reinstatement within three trials after presentation of the reinstatement USs. Differential SCRs (CS+ -CS-) in three trials after reinstatement were lower in the L-DOPA, when compared to the Placebo group (see figure 2). SCR=skin conductance responses, CS=conditioned stimulus.

Acquisition of CS-US contingencies on day 1
Participants in both groups learned CS-US contingencies during acquisition training, which was indicated by a CS-type main effect that consisted of enhanced responses to the CS+ as compared to the CS-in all dependent measurements, namely binary (yes/no), trial-wise US Hence, we found no statistical support for reduced differential fear appraisal of the CSs in the L-DOPA group, as compared to the Placebo group. We further found no statistical evidence that would support a difference between groups in US expectancy ratings or SCR (see

Supplementary File 1d).
Next, we examined how decreasing US expectancy, which indicates extinction learning is driven by expectancy violation from the omission of the US, by fitting US expectancy ratings with a Rescorla-Wagner-Pearce-Hall-Hybrid model [32,33]. The fitted prediction error (as a measure of expectancy violation), associability (as a measure of prediction error-guided surprise) and learning rate did not differ between groups (two-sided independent sample t-test

Memory retrieval on day 3
Retrieval was tested on day 3 within an generalization context that consisted of a mixture of the acquisition and extinction context, i.e. one context that 50% of the furniture from the acquisition context A and 50% from the extinction context B [34], which also involves contextual renewal of conditioned threat responses [35] Importantly, the SCR analyses of the three trials before and after reinstatement revealed a difference between groups in differential CS responses (CS-type by group interaction . While our analysis revealed a difference between groups in differential SCRs, there was no strong support for difference between groups in CS+ or CS-responses. While our a priory hypothesis was an effect of L-DOPA on the psychophysiological measurements at retrieval-test on day 3, our analyses suggest that L-DOPA administration during extinction training reduced differential threat responses after reinstatement. a) VmPFC responses that reflected CS+ trials during extinction training when subjects no longer expected an US that are contrasted with trials in which an US was expected (i.e., extinction learning).

Administration of L-DOPA enhances vmPFC responses reflecting decreasing US expectancy during extinction learning
First, our analyses of neural responses focused on the effect of L-DOPA on extinction learning, where we expected an involvement of the vmPFC that is modulated by L-DOPA. To this end we examined brain regions that increased their activity to a decrease of US expectation. In order to examine extinction learning by decreasing US expectancy, we contrasted responses during extinction training to CS+ trials when participants expected no US against CS+ trials in which participants expected an US (i.e., expectation of no US > expectation of a US). We found that decreasing US expectancy was accompanied by more pronounced signalling in the right vmPFC in the L-DOPA group as compared to the placebo group (see Figure 3 a). Thus, administration of L-DOPA augmented vmPFC activity during extinction learning, i.e., when participants decreased their US-expectancy.
Next, we tested if this difference in the right vmPFC activity was related to individual differences in the retrieval of conditioned threat responses. A previous study indicated that vmPFC activation during extinction learning was associated with retention of extinction memory (measured as differential SCR) 24 hours later [25]. Indeed, we found that higher vmPFC activation is associated with reduced differential SCR, which could indicate better individual extinction memory retention, 24 hours later (two-sided Pearson correlation: t(38) = -2.18, pvalue = 0.035, r=-0.3273302 (95 percent CI: -0.58 -0.018, see Figure 3 b). Hence, vmPFC responses during extinction learning were elevated after L-DOPA administration and such enhanced vmPFC activity is associated with reduced retrieval of differential threat responses (measured as SCR) 24 hours later. Importantly, there was no difference between groups detectable in SCRs during retrieval test, which might have biased this correlation [36]. However, it might be possible that L-DOPA treatment has an indirect effect on SCR during retrieval test, which was mediated by vmPFC activity during extinction learning. Indeed, we found support for a treatment effect of L-DOPA on SCRs during retrieval test, which was indirectly mediated by vmPFC activity during extinction learning (average causal mediation effect: b=0.0563, 95% confidence intervals = 0.007-0.12, p=0.038, quasi-Bayesian estimation of confidence intervals with 1000 iterations, N=40, see figure 3 for detailed statistics).
As such, L-DOPA strengthens vmPFC activation that accompanies decreasing expectation of the US (i.e., extinction learning) and this enhancement of vmPFC activity mediates reduced differential SCR 24 hours later (i.e., better extinction memory retrieval). Our results thereby reveal an effect of L-DOPA on memory retrieval that is meditated by augmentation of vmPFC activity, which was specific for the individual time-course of decreasing expectancy of the aversive outcome. We found no group differences in functional connectivity during CS+ presentations in extinction training or acquisition training, as well as no differences during US presentations during acquisition training (see table Supplementary File 1o). a.u.=arbitrary units, NAcc= nucleus accumbens, US=unconditioned stimulus.

Omission of an expected aversive outcome is coded in the nucleus accumbens and modulated by L-DOPA
In the next step, we examined if decreasing US expectancy during extinction learning is driven by the omission of the US in form of an expectancy violation (i.e., prediction error) and if this process is modulated by dopamine. To this end, we used the modelled US expectancy ratings from the Rescorla-Wagner-Pearce-Hall hybrid model that has previously been used to describe computational processes in associative threat learning [32,33,37]. In order to test for signals that reflect expectancy violation, we examined responses at the time-point of US omission that correlated with the modelled prediction error term (parametric modulator). We found that activation in the right nucleus accumbens in the Placebo group reflected the time-course of the Enhancing dopaminergic transmission (i.e., in the L-DOPA group), in contrast, seems to sustain responses to the omitted outcomes in the nucleus accumbens, irrespectively of expectancy of the US or value caching. Hence, our results imply a dopaminergic modulation of expectancy violation in the nucleus accumbens when expected aversive outcomes are omitted during extinction training. Note that within exploratory analyses that were suggested by the reviewers we found no activity in the nucleus accumbens to CS+ presentations during extinction (see Supplementary File 1n), which might suggest that the nucleus accumbens might rather be involved in processing of expectancy violation coding when expected USs are omitted. During acquisition training, however, we found activity in the nucleus accumbens to presentations of the CS+ (as well as in the contrast CS+>CS-) and the US (see Supplementary File 1n), which resembles salience encoding, as reported in a recent study in rodents [38].
In addition to neural signaling that aligned with the prediction error term, we further investigated potential differences between groups in neural signals that follow the associability term, which provides a measure of prediction error-guided attention shift. Such attention-shifts denoted by associability involve several additional processes like arousal or awareness of the participants.
We found that administration of L-DOPA enhanced associability related neural signals in the amygdala at the time-point of US omission, when compared to Placebo (see Figure 4-figure supplement 1). Our results suggest that dopaminergic enhancement might enhance shifting of attention or surprise that is initiated by unexpected omission of the US during extinction training. . This contrast is specific to the enhancement of connectivity by L-DOPA, while making no assumptions about the Placebo group. The connectivity was condition-specific to the time-point of US omission (psycho-physiological interaction, PPI in SPM). We found functional connectivity during CS+ presentations in extinction training or acquisition training, as well as no differences during US presentations during acquisition training between these regions, as well as no differences between groups during CS+ presentations in extinction training (see Supplementary File 1o). T-maps are displayed at threshold punc<.005; colour bar represents t-values. a.u.=arbitrary units, NAcc= nucleus accumbens, SN/VTA=substantia nigra/ventral tegmental area complex.

L-DOPA modulates functional connectivity between responses in the nucleus accumbens and the VTA when the US is omitted
Results in animals suggested that processes at the time-point of US omission involve not only the nucleus accumbens, but dopaminergic neurons in the VTA [13,38] and projections from the VTA to the nucleus accumbens [11], as well as projections from the basolateral complex of the amygdala to the nucleus accumbens [9].
To test if the reported results in the nucleus accumbens at the time-point of the omitted US are functionally connected with other regions in the brain, we employed a condition-specific connectivity analysis with an anatomical nucleus accumbens (bilateral) mask as a seed region.
In line with animal data, we found stronger connectivity between the nucleus accumbens and

Discussion
Our results provide evidence that dopaminergic processes are involved in threat extinction learning. Dopaminergic enhancement during extinction learning augmented extinction memory at a later test, which was mediated by extinction learning specific vmPFC responses (i.e., reflecting decreasing US expectancy). Decreased US expectancy in extinction learning was further driven by dopaminergic activity within the nucleus accumbens that signalled the omission of expected aversive outcomes. This activity in the nucleus accumbens, when the US was omitted, was functionally coupled with the midbrain SN/VTA complex, as well as the amygdala. Additionally, we found reduced differential SCRs (CS+> CS-, without strong support for a reduction in CS+ responses, only) in the L-DOPA group when compared to Placebo after the reinstatement one day after extinction learning. Hence, our results suggest that the activity of the nucleus accumbens, but not the vmPFC, encoded absent threats (prediction error) during extinction learning. Decreasing expectancy of threats in extinction learning was reflected by vmPFC activity, which was further enhanced by L-DOPA. These results corroborate findings from reinforcement learning in which outcome and prediction error signals were encoded in the striatum, whereas value was reflected by prefrontal regions [39].
The main finding is that L-DOPA reduced retrieval of conditioned threat responses (SCR) by a mediation through enhanced vmPFC activity. In detail, we found that enhancement of dopaminergic transmission by the administration of L-DOPA (as compared to Placebo) during extinction learning enhanced individual neural signalling in the vmPFC that reflected the reduction of US expectation (i.e., extinction learning).
These enhanced vmPFC responses were found to mediate the effect of L-DOPA on extinction memory retention, measured as reduced differential SCRs 24 hours later.
Besides the implication of the vmPFC in safety signal processing [40] and threat extinction in humans [28,29,31,41,42], our results align specifically with a previous finding of vmPFC activity pattern during extinction memory consolidation that mediates the effect of L-DOPA on extinction memory retention [25]. Our results extend this finding on memory consolidation by providing a link between dopaminergic effects on vmPFC activity that is specific for individual extinction learning (i.e., decreasing US expectancy) and augmentation of extinction memory. L-DOPA might have the potential to improve (otherwise low, see parameter estimates in the Placebo group, figure 3 B) vmPFC activity during extinction learning. Hence, rather than enhancing extinction learning per se, L-DOPA administration seems to augment vmPFC responses that accompany decreased US expectancy. This suggests a benefit of L-DOPA for extinction learning processes. Our findings would fit to previous results that link the benefit of neuropharmacological intervention in extinction learning to decreasing threat expectancies [22,[43][44][45]. Thereby, L-DOPA might have the potential for a psychopharmacological treatment that augments threat extinction learning instead of dampening overall threat responses, like classic anxiolytics. Additionally, we found that L-DOPA administration during extinction training reduced differential SCRs after reinstatement, which aligns with a finding of decreased SCRs after reinstatement by L-DOPA administration that followed extinction training in women diagnosed with posttraumatic stress disorder [46].
A second set of our results implicate that decrement of US expectancy in extinction training involves a dopaminergic coding of expectancy violation in form of a prediction error at the time-point of US omission. In detail, administration of L-DOPA enhanced general (categorical) activity during omitted US, which was paralleled by reduced prediction error signals that we found in the placebo group. The enhanced general activity in the nucleus accumbens to omitted outcomes might results from enhanced activity in dopaminergic projections from midbrain, which would be in line with L-DOPA induced dopamine release in the midbrain in humans at rest [47]. L-DOPA has further been found to activate otherwise silent dopaminergic neurons in animals [48], which thereby might enhance activity in the nucleus accumbens in general. In parallel, L-DOPA could have induced inhibitory signals (via dopamine D2-receptors) that limit phasic burst-firing and thereby reduce the pattern of prediction error encoding [49].
Such a shift in striatal activity by L-DOPA would fit to the observation of an opposite effect by blockade of dopamine D2 receptors (using haloperidol [50]): reduced categorical activity during (rewarding) outcomes and enhanced prediction error signals.
Our findings furthermore align with a negative association between striatal dopamine synthesis capacity (measured by radiolabelled L-DOPA during positron emission tomography) and BOLD responses related to reward prediction errors [51]. Hence, higher dopamine synthesis capacity (indicated by uptake of L-DOPA in the striatum) was associated with lower prediction error related BOLD activity.
Our study was intentionally not designed to disentangle details of expectancy violation coding of omitted USs, but rather to provide a scenario of safety learning. While we cannot disentangle encoding of salience and expectancy violation in the nucleus accumbens within our experiment, we conducted additional analyses in order to differentiate activity to the omitted US from other salient events like the CS+ or the US.
During extinction training, we found no involvement of the nucleus accumbens during presentations of the CS+, which is more in line with coding of expectancy violation rather than salience (see Supplementary File 1n). During acquisition training, however, the nucleus accumbens was associated with salience encoding, since we found activation to presentations of the CS+ (and in the contrast CS+>CS-), as well as the US (see Supplementary File 1n). Additionally, we found that connectivity between the nucleus accumbens and the VTA during CS+ presentations, albeit less statistical support (p(FWE)=0.064, see Supplementary File 1o). A recent study in rodents could disentangle encoding of salience and expectancy violation during extinction learning in lateral and medial part of the VTA, respectively [38]. While we cannot examine such fine-grained processes in the VTA with fMRI in humans, we nevertheless provide evidence for a role of the nucleus accumbens in processing of US omissions, which is in line with a function of the nucleus accumbens in rodents [8,16,52] in particular during extinction learning [9,10,17,53]. Our results furthermore dovetail with two previous neuroimaging study reporting an association between prediction error signals in the nucleus accumbens during extinction training [23,24]. Our results point moreover to a dopaminergic modulation of surprise in the amygdala that is evoked by US omission, which fits well to previous reports of associability coding in the amygdala during threat learning [32,37].
We further show that signals in nucleus accumbens at the time-point of US omission were functionally coupled with activation in the amygdala and the SN/VTA, which were enhanced by administration of the dopaminergic precursor L-DOPA. This finding mirrors findings in animals implying neurons in the VTA, as well as projections from the VTA to the nucleus accumbens, in the encoding of the omission of an expected US [11,13]. Furthermore, our results would align with studies in animals that provided evidence for amygdala to nucleus accumbens projections that underlie extinction of threat responses [9]. Such interaction between dopaminergic pathways and amygdala activity during threat extinction is moreover in line with a study in rodents that revealed changes in inhibitory interneuron activity in the amygdala by dopaminergic projections from the VTA that enable suppression of freezing during early extinction [15].

Limitations of this study
This pharmacological fMRI study in human volunteers is only suited to draw inferences on blood-oxygen-level-dependent signals as a function of L-DOPA administration.
Hence, the changes reported here are only indirect markers of changes in dopaminergic neurotransmission, which might suggest together with fine-grained studies in animals that dopaminergic neurotransmission is involved in extinction learning across species [7,11]. Dopamine specific markers in humans (e.g. using PET) might be suitable to unambiguously link changes in vmPFC and nucleus accumbens activity during extinction learning to dopaminergic neurotransmission.
The behavioural effects of pre-extinction administration of L-DOPA in our study were weaker when compared to studies that employed post-extinction administration [25][26][27]43]. We found no support in univariate analyses that L-DOPA administration decreased conditioned responding across all outcome measures that reflect different threat processing, such as US expectancy, psychophysiological arousal (SCR) and affective (fear) ratings during extinction learning and retrieval test. Future studies might employ more fine-grained explicit ratings of threat expectancy or use other psychophysiological such as pupil-size or startle responses [54,55].
Nevertheless, we found that L-DOPA to decreased differential SCRs during retrieval test by mediation of vmPFC activity during extinction learning, which converges with a previous finding [25]. Hence, the current state of research might suggest that L-DOPA enhances vmPFC activity during or after extinction training, rather than blunting responses, per se.
Our current sample size was based on effect-sizes that result from previous reports of post-extinction administration [25,27] and hence our sample size might have been too small to detect univariate differences between the L-DOPA and placebo group with sufficient power.
Our results were furthermore derived from a population of healthy male volunteers and a role for L-DOPA in extinction learning within a representative sample, as well as populations with anxiety disorders have just been recently pioneered [46]. Future studies that further investigate L-DOPA as a novel augmentation strategy for the therapy of anxiety related disorders (in which extinction mechanisms are only one part of the process) are warranted.
In sum, our results thereby provide a neuropharmacological mechanism that augments the neural substrates underling extinction learning in humans [43][44][45]45], which could provide a promising novel strategy to augment behavioural treatments of anxiety related disorders [46]. 50 healthy male subjects without self-reported psychiatric and neurological diseases, without current medication (including no "over the counter drugs" without prescription within the last two weeks) or current use of illicit drugs (urine toxicology) were recruited in this study. Illegal drug-screening test was carried out prior to testing at day 1 (M-10/3-DT; Diagnostik Nord). The The study (including sample size approximation) was approved by local ethics committee in Hamburg (Ärztekammer Hamburg). Full participation of this study was remunerated with 120,-EURO.

Conditioned stimuli
Contexts surrounding the CSs were employed as computerized environments of virtual offices (Source Engine, Valve Corporation, Bellevue, USA, used in [34]. Each office image was depicted from two different vantage points (on the wall opposite the door vs. on the wall to the right of the door). Three different contexts were used, context A, context B and a mixture of both in order to induce a contextual generalization [34]. The context that included a mixture of context A and B contained 50% of the furniture from context A and 50% from context A, equally distributed in the room. Virtual offices consisted of the same floor plan but differed regarding the furniture. A blue or a yellow color filter illuminating the whole room (duration of 6 sec) served as CSs, indicating either CS+ or CS-. Colors of the CSs and contextual backgrounds were counterbalanced across participants. Presentation of the context served as the inter-trial intervals (ITIs, duration range 7-11 sec, mean 7.8). The visual stimulus material was presented in pseudo-randomized order on a computer screen using Presentation® software (NeuroBehavioral Systems, Albany California, USA; RRID:SCR_002521).

Unconditioned stimulus
An electrotactile stimulus consisting of a train of 3 square-wave pulses of 2 ms duration each (interval 50 ms) served as the US that the CS+ onset after 5 sec. The US was delivered through a surface electrode with platinum pin (Specialty Developments, Bexley, UK) on the right dorsal hand using a DS7A electrical stimulator (Digitimer, Welwyn Garden City, UK). US intensity was individually adjusted prior to acquisition training (day 1) to a level of maximal tolerable pain (mean 8.1 ± 0.5 mA, range 2.5-21.0 mA) and participants were asked to rate the aversiveness of the US between 0 ("I feel nothing") and 10 ("maximally unpleasant"; rating: mean 7.1 ± 0.1,

Study medication
Study medication included an oral administration of 150mg L-DOPA (including 37.5 mg Benserazide) in a double-blind and placebo-controlled protocol 60min before extinction training.
Participants were allocated into the placebo or L-DOPA group before day 1 in a restricted randomization procedure that allocated 5 subjects to the L-DOPA and 5 subjects to the placebo group for each group of 10 participants. The dose of 150mg has been found effective in previous studies to enhance the consolidation of extinction memories in humans [25][26][27].

Experimental Procedure
Using a three-day paradigm, acquisition training (day 1) and extinction training (day 2, approx. 24 h after acquisition) were conducted in the fMRI scanner, while retrieval test (day 3, approx. 24 h after extinction), including reinstatement, were employed within the psychophysiological laboratory. Acquisition training took place in context A, whereas extinction training was employed in context B. Retrieval test (including reinstatement procedure) was conducted in a 50/50-mixture of context A and B in order to examine contextual generalization [34], which also involves contextual renewal of conditioned threat responses [35]. Twenty-four hours after acquisition training participants received L-DOPA (see study medication) before the extinction learning session (the CS+ was no longer followed by an aversive outcome). L-DOPA administration thereby affected extinction training, while acquisition training, as well as retention and reinstatement-test were conducted drug-free. Data collection on day1 included sampling of plasma concentration of endocannabinoids as part of a different project.
Acquisition training (day 1) A short habituation phase preceded acquisition training (6 trials: 3 CS+, 3 CS-) without any presentation of the US. Subsequent acquisition training consisted of 24 trials for each CS (in context A). The CS+ was followed by a US in 75% of the trials, whereas the CS-was never followed by a US. Participants were not informed about the conditioning contingencies or the learning element beforehand.

Extinction training (day 2)
Approximately 24 hours after conditioning, participants returned to the fMRI laboratory. US and SCR electrodes were attached exactly as the day before, without US intensity adjustment.
During extinction training, 24 trials (context B) were presented for each CS and no US was administered. Participants were not informed beforehand about any change in CS-US contingencies.

Retrieval test and reinstatement (day 3)
Participants returned to the psychophysiological laboratory and US and SCR electrodes were again attached without further US adjustment. A retrieval test (contextual generalization in a 50/50-mixture of context A and B, i.e. one context that 50% of the furniture from context A and 50% from context B) consisted of 8 unreinforced trials of each CS and was followed by 4 unsignaled reinstatement-USs (interval duration range 10-15 sec). Here, the same individual electrical stimulation intensity was used as during acquisition training. 6-10 sec after the last reinstatement-US, a second retrieval test (reinstatement-test) was employed, including 16 trials (with no US) of each CS. The order of CS+ and CS-after the reinstatement US was counterbalanced across subjects. At the end of the experiment, CS-US contingency awareness was assessed using a semi-structured interview [56] and based on these results 37 participants were classified as aware and 5 were classified as unaware (no differences between groups, χ 2 -Test, p = .634).

US expectancy
On each CS trial presentation, participants had to rate their US expectancy as a binary choice (key press for yes/no) without any scale presented to avoid any distraction. Participants were excluded from the analyses (day-wise) if less than one third of all data points were missing [excluded participants: N(day 1) = 0, N(day 2) = 3, N(day 3) = 3].

Fear ratings
At the beginning as well as at the end of each experimental day, participants were asked to rate the fear/stress/tension level that was elicited by each CS. On day 1 the first rating was

Statistical analysis
Outcome measures were analyzed (using JASP 0.11.1, JASP Team (2020) [Computer software], RRID:SCR_015823) employing repeated-measures ANOVAs. For acquisition and extinction training, these ANOVAS included CS-type (2) and the effect of time (fear ratings: 2 ratings, SCR and US expectancy: 3 blocks, each average across 8 trials). Pharmacological group was entered as a between subject factor. For day 3, we analyzed the first block separately as the retrieval test and the reinstatement analyses included two comparisons of trials before and after the reinstatement USs [57]. First, we compared responses averaged across the whole block (8 trials) before and after reinstatement. Since reinstatement effect are transient and only detectable over a few trials, we added a second, more detailed analysis, which compared responses averaged across the 3 trials before and after reinstatement, based on previous findings indicating that transient reinstatement effects can be found up to 3 trials after the US presentation [58]. In all analyses, an α-level of p<0.05 was adopted and sphericity correction (Greenhouse-Geisser) was applied. Follow-up post-hoc test on measurement on day 2 and 3 were performed as one-sided independent t-test to examine the hypothesis of L-DOPA responses < Placebo responses. During data acquisition, preprocessing and initial analyses, the experimenter were masked to the drug conditions.

Hybrid model
To examine how decreasing US expectancy is driven by expectancy violation from the omission of the US, we fitted trial-wise US expectancy ratings with a Rescorla-Wagner-Pearce-Hall-Hybrid model, which is the same model employed in previous neurocomputational studies of aversive learning in humans [32,33]).
In order to examine associative threat learning processes, which can be described by classical formal learning theory such as the Rescorla-Wagner (R-W) [59] and Pearce-Hall (P-H) model [60], we analyzed extinction learning underlying mechanisms based on trial-by-trial US expectancy ratings. Therefore, a Rescorla-Wagner-Pearce-Hall-Hybrid model (HM) [61] was used, which algebraically describes error-driven learning based on prediction errors (PE, i.e., mismatches) between the predicted (aversive) outcomes (denoted as expected "values," v) and the received outcomes (RO), which in this case corresponded to the omissions of the US.
Extending the RW-model, the HM explicitly accounts for dynamically changing learning rates α (i.e., surprising absence of the US) that is updated depending on the associability η (i.e., the reliability of prior predictions). That means the associability η increases in proportion to the absolute prediction error (PE) on the last interaction with a stimulus, allowing the agent to adapt to changing environments, which leads to larger prediction errors (PE), and thereby higher associability η. The hybrid model (HM) is formalized by the following equation: "#$ = " + α * " * PE The predicted "values" (v) on the next trial t+1 are based on the "value" at the current trial t and on prediction errors (PE) scaled by the learning rate α and the current associability ηt. Prediction errors (PE) are calculated as the difference between the current predicted values (vt) and the received outcomes (RO).

= − "
The current associability ηt is updated according to the absolute prediction error (PE) and the associability of the preceding trial ηt-1 with the free scaling parameter ω.
The model employs a softmax function with a free "inverse temperature" parameter β to generate trial-by-trial probabilities (p) for the binary US expectancy ratings.
The model thus contains three free parameters: (i) the learning rate α, (ii) the scaling parameter ω for the associability η, and (iii) the inverse temperature parameter β. These three free parameters were initialized in the fitting procedure as 0.5, 0.5, and 4, respectively. The starting point for the initial "value" v0 was set to 0.75, i.e., the probability for a US following a CS+ in the acquisition phase. The starting point for the initial associability η0 was set to 1, which assumes that the associability is initially fully dependent on the prediction error (PE).
We fitted model parameters using maximum likelihood estimation (MLE). Specifically, we used the non-linear Nelder-Mead simplex search algorithm (implemented in the MATLAB (RRID:SCR_001622) function fminsearch) to minimize negative log-likelihood summed over all trials for each participant.

fMRI acquisition and analysis
MRI data were obtained on a 3T Magnetom-PRISMA System (Siemens, Erlangen, Germany) using a 64-channel head coil. fMRI measurements were performed using single-shot echoplanarimaging with parallel imaging (GRAPPA, in-plane acceleration factor 2) [ introductions, ratings, and button presses) were defined as separate regressors modeling the predicted time-courses of experimentally induced brain activation changes as a stick function.
Furthermore, CS+ regressors included a parametric modulation of individual US expectancy ratings in order to examine dopamine-dependent differences in neural representation in decreasing US expectancy during extinction learning. Additionally, parametrical modulation of the omitted US was applied to examine neural responses that are related to changes in expectancy-violation over trials. Therefore, the modelled prediction error-term (as a measure of expectancy violation, averaged across the whole sample) and the orthogonalized associability-term (as a measure of prediction error-guided surprise, averaged across the whole sample) were entered trial-wise.
In a next step, subject-and regressor-specific parameter estimate images of interest were normalized to a sample-customized DARTEL template [66] smoothed with an isotropic fullwidth at half-maximum Gaussian kernel of 4 mm. These estimates were then included into separate random-effects group analysis using SPM's "full factorial" model, which permits correction for possible non-sphericity of the error term (here, dependence of conditions). Model factors for the respective analysis were CS+*US expectancy ratings (extinction learning), omitted US*mean prediction error and omitted US*associability (expectancy violation), always including the factor group (Placebo, L-DOPA). Analyses main objective was to reveal dopamine-specific effects (L-DOPA vs Placebo) during extinction learning depending on time-point related changes in US expectation. Significance of effects was tested by using voxelwise one-tailed t tests. According to our hypotheses, we expected enhanced signaling in the L-DOPA group as compared to the Placebo group in CS+*US expectancy ratings (extinction learning) in the vmPFC (L-DOPA > Placebo). We further expected a modulation of omitted US signals by L-DOPA administration and examined all contrast for the time-point of US omission (omitted US*mean prediction error, omitted US*associability, categorical omitted US) in both directions (L-DOPA> Placebo and Placebo> L-DOPA).
Regions of interest (ROI) were the defined as 1) dopaminergic key structures, such as the nucleus accumbens and the VTA/SN and 2) key structures in extinction learning, such as the amygdala and the vmPFC. These structures were defined by Havard-Oxford probability maps for the nucleus accumbens and the amygdala [67]. For the SN/VTA and vmPFC is no anatomical mask available, therefore we defined both ROIs as in a previous study that revealed an effect of L-DOPA treatment in both, the SN/VTA and vmPFC ROI [68].

Connectivity analysis
Psycho-physiological interaction (PPI, as implemented in SPM12) was used to examine functional connectivity differences of responses in the nucleus accumbens towards the omitted US between groups. Extracted eigenvariates of nucleus accumbens (bilateral ROI mask) were used as the seed region, deconvolved and multiplied with the condition specific onsets of the omitted US. The product (PPI) was entered as a regressor into an individual GLM for each participant, controlling for the time-course of the nucleus accumbens, the onset regressor and movement as nuisance regressors. Parameter estimates of the omitted US-PPI were then contrasted between groups.

Mediation analysis
To test if the effect of L-DOPA vs Placebo on differential SCRs at retrieval test on day 3 was mediated by the activity in the vmPFC that aligned with decreasing US expectancy, we employed a mediation analysis (R Studio, Version 1.2.1335, package "mediation"; R Project for statistical computing RRID:SCR_1905). This analysis was based on a prior analysis that revealed that effects of L-DOPA on extinction memory retention (differential SCRs during retention test) were mediated by vmPFC activity during consolidation [25].