Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
190 Cards in this Set
- Front
- Back
|
What is a reinforcer? |
strengthens the connection between S-R |
|
|
Controlling associations are revealed by sensitivity to what 2 things? |
outcome devaluation contingency degradation |
|
|
What is an RI30? |
random interval 30--> get reward approx every 30 secs |
|
|
What are habits? |
learned sequences of behaviour that are acquired by repetition |
|
|
Specific satiety |
won't eat one food that already eaten when full but will eat another different food unless formed habits then will eat both regardless of full or not |
|
|
Examples of mental disorders involving habits |
OCD, Tourette syndrome, schizophrenia, Huntington's disease |
|
|
What do habits reflect a shift from/to? |
from evaluation-based circuitry to performance-based transition from exploration to exploitation |
|
|
What is meant by exploration vs. exploitation? |
exploring the many possible choices in behaviour vs exploiting the certainty of a single action |
|
|
Transition from A-O→ S-R involves what areas of the brain?
|
the PFC, amygdala & striatal systems
|
|
|
What are these circuits called? |
cortico-striato-pallido-thalamo-cortical loops |
|
|
Describe the circuits and the shift from A-O to S-R. |
The striatum receives input from the entire cortex (except primary visual and auditory cotices); the striatum integrates information from a number of (related) cortical areas; these areas represent convergent information in the striatum outputs to the globus pallidus and sustantia nigra; these structures send outputs via the thalamus back to cortical areas that include the original cortical input to the loop and regions of the motor cortex |
|
|
What are basal-ganglia loops? |
hypothetical segregated circuits connecting the basal ganglia, thalamus and cerebral cortex the 5 circuits are named according to the primary cortical target of the output from the basal ganglia |
|
|
Limbic regions |
processing of reward, whats good/bad, evaluation of reward, emotional bits |
|
|
What is the transition in the brain? |
transition from areas in the brain that are emotional and care about rewards to reas that just care about motor over time, shift in which neural circuits are underpinning response logical pathway in brain cortical regions--> subcortical regions |
|
|
Large loops summarised |
excitation coming into cortex--> subcortical regions where processed and inhibited--> single outputs back through thalamus to cortex integrates info from subcortical regions puts all info together to produce a response which produces the best outcome spit back out to cortex to update what think about world and produce appropriate response |
|
|
What are the basal ganglia and cerebellum responsible for? |
ensuring movement is carried out in a smooth and fluid way |
|
|
What is a instrumental/operant response? |
a situation where the responses made by an organism bring about changes in the environment the critical thing that defines and instrumental response is that there is a contingency between an outcome and an outcome |
|
|
What 2 processes does instrumental learning depend on? |
goal-directed actions (A-O) stimulus-response habits (S-R) |
|
|
What is initial learning and what does it become? |
Initial learning is A-O, gradually dominated by S-R |
|
|
After more extensive training... |
behaviour becomes habit based |
|
|
Overtrained rats will press a lever when full for that food and for a different food. What does this show? |
Outcome devalued, are just pressing lever, don't know why |
|
|
What is loss of cognitive control in terms of habits? |
loss of ability to displace habits |
|
|
What is critical in disorders such as OCD and addiction? |
circuit controlling change from goal-directed to automatic |
|
|
What systems does the transition from A-O to S-R areas of the brain involve? |
PFC, amygdala, striatal systems |
|
|
As habits and procedures are learned there is a dynamic shift in activity.... |
in cortical and striatal regions |
|
|
Where is the anterior cingulate cortex? |
bit down middle at front |
|
|
What are associative areas? |
transition between 2 different forms of processing e.g., reward processing--> motor processing; visual--> recognition |
|
|
What does the thalamus do? |
The function of the thalamus is to regulate the body's voluntary motor control, consciousness and its sleep/wake cycle. ... The thalamus decides which signals from the ears, eyes, mouth and skin to relay to its area in the cerebral cortex.
|
|
|
What do these large loops do? |
Integrates info from subcortical regions excitation coming into cortex--> subcortical regions where processed and inhibited--> single outputs back through thalamus to cortex Puts all info together to produce a response which produces the best outcome cortex--> subcortical regions--> thalamus--> cortical regions acts as a filter; lots of exitation and inhibition going through; critical in controlling GD, SR actions |
|
|
How does Parkinson's affect the brain? |
degenerative disorder affecting nerve cells in deep parts of brain in basal ganglia and substantia nigra |
|
|
What are nerve cells in the substantia nigra responsible for? |
produce dopamine and are responsible for relaying messages that plan and control body movement |
|
|
What is the caudate and ventral striatum? |
striatum= caudate putanem ventral striatum= nucleus accumbens |
|
|
Every bit of the cortex has a projection that goes through... |
the striatum |
|
|
What does the globus pallidus do? |
The globus pallidus is a structure in the brain involved in the regulation of voluntary movement. It is part of the basal ganglia, which, among many other things, regulate movements that occur on the subconscious level.
|
|
|
Basal ganglia loops are spatially segregated within the brain. What commonalities is there between them? |
all go through striatum and thalamus and all go to cortex but all connect to slightly different bits of these structures (small differences, but v distinct loops) |
|
|
What is at each end of these loops? |
one end= reward process, cares about value other end is an S-R process cares about SR habits value-based system to non-value base system |
|
|
What is critical in the transition between habits and GD responding? |
prelimbic and infralimbic regions of medial PRF |
|
|
What do lesions of the prelimbic cortex do? |
abolish GD lever pressing similar effects of dorsomedial striatal lesions |
|
|
Killcross & Coutureau (2003): give animals small amount of training, respond on lever for food. Test with food valuable vs not valuable (satiety). What will normal animals show with a small amount of training? What about if you take out the prelimbic medial PFC? What happens if you damage the infralimbic medial PFC which is right next to the prelimbic? |
Normal animals with a small amount of training will show a devaluation effect If you take out the prelimbic medial PFC, then this is blocked so animals looked as if they were overtrained They no longer were able to learn or represent R-O association; so just left with underpinning S-R associations Damage IL- has no impact |
|
|
After damaging the PL, what happens when you look at the cortico-striato-pallido-thalama cortical loop? |
If go next step down (currently in cortex), and look at striatum, then see same sort of effect on anterior dorsal medial striatum and posteriorDMS; show again, absence of devaluation effect- hints at circuitry Circuit for supporting R-O associations |
|
|
What do lesions of the infralimbic cortex do? |
abolish habit-based lever pressing |
|
|
Animals are overtrained; normal animals (sham lesion) vs damage to prelimbic cortex vs damage to the infralimbic cortex. What did these animals show after being overtrained? |
Normal, sham lesion, animals show no devaluation effect damage PL show no effect (didn't anyway in undertraining scenario) damage IL and they still show G-D responding even though even though you've got rid of GD behaviour by overtraining in the normal animal |
|
|
IL lesion in terms of A-O and S-R responding |
can show AO association but not SR |
|
|
How would you describe the IL and PL in relation to each other? |
double dissociation between them can be tracked to striatum too |
|
|
What happens if you lesion the dorsal lateral striatum? |
persistance of devaluation even when normal animals show SR |
|
|
Reversible lesions of the IL cortex |
produce G-D performance despite overtraining |
|
|
What do these lesion studies highlight? |
A-O and S-R transitions are not all or nothing, but remain fluid- we can break a habit and the underlying value-based system is not lost |
|
|
Overtrain all animals with lever press--> food. Temporarily activate IL region or sham infusion of vehicle Inject muscimol into IL region, turning off temporarily. |
Temporarily inactivating IL region--> impairs S-R responding; converts animals from S-R habits to showing GD responding Sham infusion: show habit based performance |
|
|
What dos muscimol do? |
GABA agonist, stimulates GABAergic receptors, inhibition causes cells to shut down |
|
|
Similar effects are also seen following manipulation of the basolateral nucleus of the amygdala. Describe these |
Train animals, devaluation; normal animals show devaluation effect Take out BLA- abolish this effect |
|
|
How do you know if an animal is dominated by RO associations? |
see if values outcome (responds because knows what the outcome is and values it) has to know there is a relationship between the response and outcome |
|
|
What do orbitofrontal lesions do to devaluation? |
no effect on instrumental devaluation |
|
|
Neural systems underpinning habits depend on the performance-related areas of what circuitry? |
evaluation<-> selection <-> chunking <-> habit |
|
|
Circuits mediating evaluation of actions gradually lead to what? |
selection of particular behaviours that, through the chunking process, become habits |
|
|
What 3 systems do S-R habits depend on? |
Depend on PFC, striatal and thalamic systems |
|
|
What signal to habits depend on? |
habits depend on a reinforcement signal to 'stamp in' the SR association it seems likely that dopamine serves as this reinforcement signal |
|
|
Why are these systems important? |
the controlling aspect of habits in everyday life the interaction of controlled and automatic processes in cognition the disruption of systems in mental illness and drug addiction |
|
|
Describe how do these systems lead to addiction |
over-evaluation--> over-selection--> chunking-> extreme habits, addiction |
|
|
How does amphetamine sensitisation before training affect lever pressing? |
enhances acquisition of lever press habits overactive dopaminergic system is sensitised, producing more than normal similar patterns are seen with cocaine and other psychostimulant drugs of abuse show loads of pressing whether devalued or not system more potent, transition directly into habits |
|
|
Contingency degradation: free outcomes that are the same anyway as if do response. Lesion BLA? |
can perform instrumental response, not because know going to get particular outcome |
|
|
Contingency degradation: free outcome that are same anyway. NAc core vs shell. What does devaluation look like? |
Devaluation is normal in lesion of shell, but not in lesion of core. Lesion core and act as if both contingencies are devalued Disrupting ability to know which response is associated with which outcome--> generalising devalued outcome to everything Dissociation between core and shell when you look at contingency degradation |
|
|
What do amygdala lesions do? |
produce effects on contingency degradation and devaluation |
|
|
What does taking out the amygdala core do? |
affects devaluation but doesn't affect contingency devaluation |
|
|
Describe how the substantia nigra is involved with dopamine in forming habits. |
sn releases dopamine in the dorsolateral striatum--> dopamine stamps in that 'this is a good circuit to use' comes to dominate RO circuit, dominates performance rather than GD |
|
|
How are reinforcement and reward the same? |
big distinction between reinforcement and reward reward: outcome in a GD action reinforcement: the thing that makes that association stronger, reinforcing association |
|
|
What regions are important in a whole range of different mental illnesses and drug addiction? |
regions of frontal cortex and striatum |
|
|
Is there any evidence that number of rewards, or degree of exposure to reward vs consistency of reward is important? use a variable interval schedule; varies from vi5 to vi90, about a mean of vi30; shifts every 2 mins across training, 9 sessions, 40 rewards per session; compare to consistent vi30 group that gets, on average , the same rewards and makes the same number of responses Change vi5-->vi90 every 2 mins: very rich--> very long periods between reinforcement; vi90 might not get anything in 2 mins (1.5 mins=90); animal experiences as very rich reward and low levels. Both have the same training average vi30, just experienced differently. What is found? |
static vi30--> normal transition to habits; doesn't decrease responding when devalued varying--> still goal-directed |
|
|
What can be concluded develops habits? |
It's not the absolute amount of training, it is the consistency that develops habits No effect of number of rewards, exposure to reward, or number of responses |
|
|
What is a variable interval schedule? |
Variable-interval schedule is a schedule of reinforcement where a response is rewarded after an unpredictable amount of time has passed. This schedule produces a slow, steady rate of response
(VI-30) schedule. This means that the pigeon will receive reinforcement an average of every 30 seconds. It is important to note that this is an average, however. Sometimes the pigeon might be reinforced after 10 seconds; sometimes it might have to wait 45 seconds. |
|
|
Transition from GD--> habitual responding may occur when... |
there is a consistent (predictable?) pattern of reward expectancy |
|
|
So, can habits be reversed when this expectancy is violated? Rats given 9 sessions on a RI60, earning 40 rewards in each (360 in total) Then receive 1 session of RI9, earning 40 rewards or a further RI60 session, followed by devaluation treatment and extinction test. What happens? |
animals think know whats going on/reward every minute, but suddenly get reward every 9 secs--> flip back to being GD reward no longer consistent and predictable, become GD again, show devaluation effect |
|
|
When do we let the S-R system dominate? When are we able to reestablish cognitive control? |
when the world is predictable but this can change are able to reestablish cognitive control when need to change behaviour e.g., change side of the road when driving in different countries |
|
|
When are we more likely to lapse back into habits? |
stress Cognitive load is depleted; can't exert top-down control and habits re-emerge |
|
|
What stamps in the S-R? |
dopamine reinforcement says its something nice, stamps in SR |
|
|
How do GD and habitual exist? |
exist within parallel circuitries |
|
|
Animals will alter responding in order to avoid aversive consequences. When is this most obvious? |
in the case of negative reinforcement (avoidance responding) e.g., press lever can offset shock |
|
|
This also occurs in the case of conditioned punishment. What is this? |
where an aversive cue punishes an instrumental response appetitive responses e.g., to get food, response occasionally produces food, sometimes a shock cue paired with shock occasionally rat can look at trade-off |
|
|
What responses do you see in involuntary, reflexive Pavlovian condition of tone (CS)--> footshock (US) |
skeleto-muscular: freezing, fear potentiated startle, conditioned suppression; autonomic: BP, HR, galvanic skin responses; endocrine: stress hormones via alterations in HPA axis |
|
|
What do aversive outcomes emphasise the importance of? |
the amygdala important distinction between lateral (basolateral) and central nuclei |
|
|
What is fear largely tested through the assessment of? |
freezing or fear-potentiated startle |
|
|
Describe the circuit of the CS & US with the amygdala |
CS (tone)-->auditory thalamus--> auditory cortex--> lateral amygdala US (shock)--> somatosensory thalamus--> sensorymotor cortex--> lateral amygdala -->--> lateral amygdala--> central nucleus-->freezing/BP/hormones |
|
|
Compare sham lesioned animals with BLA lesion and CE lesion in terms of looking for food when shocked |
Sham lesioned animals- normal suppression, stop looking for food when shocked/lever pressing when see cue for shock Lesion to Bla should in theory mean animal is unable to form fear responses; have stopped CS & US coming together and projecting down to central nucleus of amygdala but shows normal suppression Lesion central nucleus of amygdala or lesion BLA & Ce, show reduces suppression |
|
|
There are multiple outputs for these 2 regions of the amygdala. Where does the BLA project to? |
The BLA projects to NAc and PFC and these regions project to the amygdala |
|
|
What are the shell and core of the NAc important in? What about the PL, IL and anterior cingulate of the PFC? |
reward value important in instrumental |
|
|
The info in the BLA does 2 different things |
either projection into the central nucleus is telling you to do reflexive things or other stuff (nucleus/PFC) |
|
|
What's wrong with measuring freezing as fear? |
tells us the neural basis of conditioned freezing not the neural basis of fear--> not an option to show fear in another way, can only freeze in this procedure need a system to measure multiple outputs at once to see if the animal is frightened |
|
|
What is conditioned suppression and conditioned punishment? |
conditioned suppression is more reflexive, animal just stops responding when an aversive outcome comes on conditioned punishment looks at choice behaviour of animal--> 2 levers, both produce food, but one produces aversive cue (shock) so animal has choice--> another way to tell if the animal is scared; when cue comes on can stop responding or can make a choice |
|
|
Lever 1= pellets and occasionally 10 sec and 1/2 sec shock Lever 2= pellets and 10sec train of clicks with no further consequences What was found? |
a lot more pressing on lever 2 than 1, even though same amount of reward animal is making active choice to choose 2; changed representation of value of lever 1 vs 2 animal stops pressing 1 when hears the tone but also stops pressing on lever 2--> conditioned suppression, just stops what doing measures: relative responding on 1 & 2 and also amount of presses when aversive vs neutral cue is on |
|
|
What does conditioned punishment depend on in the amygdala? Where else? |
the BLA not the CE also depends on the infralimbic medial PFC and the shell subregion of the NAc |
|
|
The value of an outcome may be encoded in (at least) two ways.. How might the interaction of these 2 systems be observed? |
reflecting the expected consequences of an action and hence the likelihood of that action reflecting the expected consequences of an event in the world (and hence independent of response) via PIT, which examines the influence of the cue-outcome associations and A-O relationships |
|
|
What 2 effects can be observed in Pavlovian-instrumental interactions? |
motivational: changes in arousal or activation influence the vigor of responding (outcome general) outcome specific: can prime specific RO associations and hence bias response choice |
|
|
How are there Pavlovian-instrumental interactions in drug addiction? |
if in environment where would normally take drugs |
|
|
What happens if you expect something bad? |
damps down appetitive system activity |
|
|
2 cues: s1 & s1--> o1 & o2 (outcomes) One Pav session (S1=O1; S2=O2) One instrumental session (R1=O1; R2=O2) Play S1 & S2--> choice between R1 & R2, record responding, compared with with no stimulus. Results? |
transfer effect of s1 and s2 on responding more likely to press lever that relates to cue (specific) more likely to press lever if there is a cue than if not (general) |
|
|
What affect does BLA have on PIT? (lesioning) |
general pit- none no BLA--> don't show outcome specific PIT BLA lesion, increase general processing but not more to same lever rather than different Know something good is going to happen, don't know if O1 or O2 Lesion BLA get rid of same-different transfer, but general transfer stays |
|
|
What about the central nucleus? |
impair general but same-different is intact |
|
|
How do PL/ OFC/ BLA lesions affect value/PIT? |
PL lesions influence instrumental devaluation, but do not influence PIT or Pavlovian devaluation OFC lesions influence PIT & Pavlovian devaluation but not instrumental devaluation bLA lesions influence Pavlovian and instrumental devaluation and PIT |
|
|
What is dissociable in value? |
Dissociation of value from action and value from cue True value and info about the value seem to be dissociable |
|
|
What is Pavlovian devaluation? |
devalue food value that is simply cued--> pure info, pellets arrive regardless of actions |
|
|
What is true value? |
value including cost |
|
|
What about value in an instrumental system? |
Don't know the value of an action/outcome until learns |
|
|
What is surprising about instrumental learning and value? |
the instrumental system does not seem to have obligatory access to value |
|
|
Value is learned through a process known as... |
incentive learning |
|
|
Animals are trained to lever press in a low deprivation stat and tested in a high deprivation state. What happens? |
lever pressing is not sensitive to this shift unless animals have had a chance to learn about the value of the reward in a high state If given the opportunity to learn that food pellets taste good while hungry and that they are more valuable when in a hungry state then rats will lift their responding rate |
|
|
Particular instrumental outcomes are more valuable in particular motivational states. If rats learn that when they're not hungry the pellets don't taste so good... |
they won't then lever press later on even if hungry |
|
|
There are multiple forms of learning with multiple neural systems such as... |
Pavlovian, goal-directed, habitual |
|
|
Different representations of the motivational impact of events through what different values |
value of action outcomes (goals) value of info about outcomes reinforcing impact of motivationally important events |
|
|
Appetitive and aversive systems are mutually... |
inhibitory A strong appetitive cue can overcome an aversive cue Appetitive and aversive systems are in conflict |
|
|
Give an example of conflict and cognitive control |
Stroop task |
|
|
What two processes are there in conflict and cognitive control? |
one well-practiced, automatic (read word) competition between well-trained almost habitual response and one that requires really strong cognitive control |
|
|
What is CPT-AX? |
continuous performance task based around A & X Need to respond to an X but only if it occurs after A, if almost all trials are A then X, becomes almost automatic and then throw in an X--> do badly Can manipulate responding Hardly any AX then are very slow e.g., bank can prime as to whether money/pond with cues beforehand contextual cues telling you what something is most likely to mean |
|
|
CPT-AX and Stroop task share much in common with several other executive tasks.. |
require that participants make use of info about a real cue held in mind to disambiguate target cues with multiple other meanings or outcomes CPT-AX: hold in mind presence of cue A in order to disambiguate responding to cue X Stroop: hold in mind rule cue 'respond to colour' to disambiguate colour naming from word reading |
|
|
When does response conflict occur? |
in situations where there is an explicit conflict due to bivalent cues typified by the Stroop task, similar characteristics may be made for many tasks, including aspects of lexical ambiguity and the continuous performance task |
|
|
Ambiguous cues activate a couple of input channels... |
PFC remembers the context you are in: top-down cognitive control to decide which S-R decides behaviour anterior cingulate doesn't care whether colour or reading Need top-down control for colour naming--> DL PFC ramps up activity The PFC is not heavily connected with primary sensory or motor cortices but instead connected with higher level 'association' and premotor cortices Excitatory signals from the PFC feed back to other brain systems to allow task-relevant neural pathways |
|
|
See face, need to look right to get reward; other pic need to look left for other cue--> simple rule--> object task; pick what saw, need to integrate what stimulus is with rule in the first task. Where is there a lot of firing? |
Lot more firing in the DL PFC when need to hold rule and manipulate it |
|
|
The neurons in the PFC are responsible for... |
producing response at conflict level |
|
|
Rats receive training with conditional cues, which are then combined at test to create congruent and incongruent compound cues. Correct responding may be determined by the environment in which the compound occurs. Training: 2 sessions a day, 1 in each context Test: audiovisual compounds of training stimuli in extinction 1 test session a day, 1 in each context If you're in context A, should respond to the noise, not the light. What is acquisition to each bioconditional task like? What is performance like? |
Acquisition to both bioconditional tasks is equivalent. Test performance reveal contextual control of responding to incongruent cues Performance better in congruent than incongruent trials Slightly better at solving visual than auditory discrimination Incongruent, not as good as congruent but can still use contextual cue to help solve problem |
|
|
What happens when you lesion the mPFC? Anterior cingulate region of mPFC? |
lesion to mPFC--> incongruent drops to chance although can still do congruent or neutral (unable to solve response conflict) Anterior cingulate region of mPFC: first 10 seconds, no problem with congruent or neutral but struggle with conflict in incongruent. As time goes on, able to solve conglict in the last 10 seconds of 1 minute (adapts to damage?) |
|
|
What effect does lesioning the IL and PL have? |
Lesion IL & PL regions: congruent: can solve, inactivating region doesn't make a difference incongruent: not as good in IL but if PL is lesioned then selectively can't solve the incongruent trials |
|
|
Reversible inactivation of the __, but not the __ ____, abolishes incongruent trial performance |
PL IL mPFC |
|
|
How does dopamine interact with response conflict tasks? |
An optimal level of DA activity in the mPFC is needed for good task performance Hence, test performance should depend on both DAergic manipulation and baseline task performance Animals that arent good a the task have too much or too little DAergic activity |
|
|
Microinfusions of D1 agonist into the PL mPFC What effect did this have on Incongruent/congruent/neutral performance? |
Boosting dopamine made no real difference to congruent trials; helped a bit with neutral; biggest impact on incongruent performance--> biggest boost on individual level, can see that dopamine boosts the performance of low performing animals |
|
|
Incongruent task performance is dependent on ... |
prefrontal dopaminergic D1 activity An optimal level of DA activity is necessary for good task performance this effect is localised to the prelimbic region of the mPFC |
|
|
What do typical antipsychotics do? |
block dopamine; block D2 receptors However, damps down dopaminergic activity in the PFC--> not good |
|
|
Meta-analyses of patients with schizophrenia indicate the importance of cognitive deficits in schizophrenia. What are these characterised as? |
deficits in tasks tapping executive function such as the CPT-AX and Stroop tasks, as well as memory and verbal fluency tasks Schizophrenics have very strong executive function deficits |
|
|
What is ketamine? |
an NMDA antagonist--> produces a dissociative experience |
|
|
What happens if you give ketamine to normal people/schizophrenics? |
give to normal people and they develop elements of psychosis give to schizophrenics and psychosis gets much worse |
|
|
Acute ketamine administration affects response conflict tasks by... |
selectively disrupts the use of contextual cues to disambiguate conflicting response info provided by incongruent stimulus compounds but left congruent and hence, biconditional performance intact |
|
|
How does acute PCP (phenylclidine) affect response conflict tasks? |
disrupted the use of contextual cues to disambiguate conflicting response info provided by incongruent stimulus compounds and reduced correct responding to congruent stimulus compounds |
|
|
What about chronic PCP administration (twice daily for 7 days)? |
selectively disrupted the use of contextual cues to disambiguate conflicting response info provided by incongruent stimulus compounds, but left congruent, and hence biconditional performance intact |
|
|
What is amphetamine? |
Another psychotic--> indirect DAergic agonist, boosts effect of dopamine |
|
|
What is +ve psychosis? |
delusions, hallucinations |
|
|
What well manages +ve psychosis? What doesnt it hep in schizophrenia? |
fluphenazine doesn't help cognitive function; most og cognitive function knocked off can't solve response-conflict tasks (critical to everything we do) |
|
|
Problems requiring task-setting cues are sensitive to ___ function These tasks are highly relevant to cognitive deficits in __ In rats these tasks are dependent on ___ function in the ___ |
prefrontal schiophrenia DAergic medial PFC dependent on D1 receptor subclass |
|
|
What does an animal need in order to make a choice? |
some sort of measure of how valuable an outcome is |
|
|
Measure of value comprises a range of different things... |
changes dependent on motivational state, how animals learn about these things; dependent on choice presented with |
|
|
Where does input about value come from? |
secondary, visual, auditory, somatosensory, gustatory and olfactory cortices |
|
|
Where do value input connections go to? |
few connections to motor output areas |
|
|
What areas connect to motor and sensory areas |
DLPFC and VLPFC
|
|
|
Where does the mPFC connect to? |
motor and reward areas |
|
|
Orbital and medial cortex receive polymodal info from... |
info from the sensory cortex, info from the thalamus (as does most of the PFC) and info from the medial temporal lobe visual and emotional info from temporal lobe |
|
|
Orbital and medial areas... |
have no arrows coming out of them other than going to other areas of the PFC |
|
|
Areas like the dorsolateral PFC project out directly to... |
motor structures and frontal eyefields |
|
|
Orbital and medial areas project primarily to... |
other areas of PFC some projections down to basal ganglia that share with the rest of the PFC |
|
|
Primates with damage to the OFC... CS1- reward; CS2- no rewards CS1- no reward; CS2- reward |
problems with reversal learning may underpin poor-decision making in patients with OFC damage can't revalue initial cues and modify behaviour accordingly perfectly normal with learning stimulus-reward associations switch cue and there are very severe deficits with OFC damage |
|
|
What happens to patients with OFC damage? |
gamble too much, make poor life decisions; poor impulse control; can’t revalue initial cues and couldn’t modify behaviour accordingly; value of things can change over time, isn’t always fixed- but these individuals can’t change the value in their head. Need to be able to behave flexibly in the world- OFC modifies understanding of value
|
|
|
But, effects not found following autonomic denervation... |
possible role for somatosensory cortex, but not proven no deficits if no initial bias in control subjects on a modified version of the IGT effects may be nothing more than more evidence of reversal learning |
|
|
Where do orbitofrontal regions receive a lot of info from? |
orbitofrontal region receives a lot of info from somatic regions (receives autonomic info)
|
|
|
What is somatic-marker hyp? |
the emotional response you get when you are presented with a cue which predicts a particular outcome |
|
|
What is the way you place value on stuff? |
isn't just by a cognitive process saying one thing is better than another; its a much more emotional process whereby you assign value based on the perception of the orbitofrontal cortex of your own internal state--> listen to emotional side to generate representation of value |
|
|
How do patients with OFC damage perform on the IGT? |
huge deficits as soon as they get the high value pack just keep picking that one completely ignoring the huge losses immediate gratification is good, doesn't matter adverse consequences may have down the line, always choose immediate gratification |
|
|
What are issues with the somatic marker hyp? |
have someone had autonomic denervation (autonomic nervous system not working) they don't show problems so cant really claim that all autonomic info is the critical feature |
|
|
Modified IGT: stop initial bias to high value pack; give exposure to graded set of cards then... |
can get people to perform more normally |
|
|
What is the IGT evidence for? |
nothing more than a reversal deficit? initially go for high value card and have to reverse choice--> all have to do |
|
|
What does reward reflect? |
hedonic or incentive value |
|
|
What is the distinction between these 2? |
sweetness doesnt actually carry info about value--> depends whether like sweet or not distinction between sensory properties which produce a pleasant feeling but not something which necessarily indicates value |
|
|
What does value reflect? |
Whether you want something rather than whether you necessarily like it |
|
|
What does the OFC distinguish between? |
distinguishes between appetitive and aversive events so not just salience alone |
|
|
what does ofc encode? |
reward alone not reward and motor functions |
|
|
Recordings from OFC of primates; give exposure to particular solutions/ get to do certain tasks. What is seen? |
OFC distinguishes between appetitive and aversive things. Not responding just to salience (both appetitive and aversive events can be very salient); OFC doesn’t have projections to motor regions so encodes information about reward by itself; a lot of regions on the PFC encode both
|
|
|
Info about a reward after onset of a reward-predictive cue goes where first? |
Info about a reward after onset of reward-predictive cue gets to OFC before other regions of the PFC
info about reward into OFC before gets to other regions |
|
|
When reward-predictive cue comes on, animal has to look to left or right then gets reward delivered. Different number of drops of juice and different directions to look in. What are the DLPFC and OFC encoding/caring about? |
Cells in DLPFC fire when anything rewarding is going to come along Cares which response you make--> fires most to 8 drops of juice when make right saccade (encoding something about reward as well as something about response have to make to get that reward) The OFC cares acutely whether going to get 2/4/8 drops of juice but not whether looking left/right to get that--> encodes reward in sense of how much juice going to get but doesn't care about motor response DLPFC isn't encoding much about reward itself; cares about the combination of reward and motor activity OFC not caring about A-O, just what the outcome is |
|
|
How does the OFC encode/respond to reward? |
Carries info about trade-off, representing value in abstract way. Can learn cue-reward associations; responds differently for 2/4/8 drops of juice; different levels of reward. Can choose between rewarded and unrewarded cues. If had 0 reward, get no firing/ no change in firing in OFC cells. Seems to carry info about relative value of different rewards If it was purely magnitude then more means bigger dip in firing rate and less reward means less of a dip in firing rate. But if you look more in depth at the cells in the OFC, seem to be encoding something about value of things relative to other things going on |
|
|
Choice between apple juice and water: juice is preferred, but what if you are thirsty and there is more water available? |
If water more available and the monkey is thirsty then water becomes more valuable than juice; more water and satisfies thirst in a way that juice can't |
|
|
The OFC isn't just encoding a simple magnitude, what is it encoding? |
is encoding something that is much more relative value is always relative to multiple things |
|
|
Value is relative to.. |
options available and motivational state |
|
|
Some cells in the OFC will increase firing to a pellet when an individual is hungry but will reduce firing to a weak salt solution. The extent to which cells fire varies with... |
motivational state Change evaluation of reward based on motivational state and based on what other rewards are around and about Something a lot more complicated than just 8 drops is twice as good as 4 drops |
|
|
Why represent value in this way as opposed to some more veridical representation of the outcome or simply as a product of choice? |
abstract value representations constrain the info needed as choices multiply; e.g., A vs B: value A, B or AB A vs B vs C vs D: value A, B, C, D or choices AB, AC, AD etc... avoids a combinatorial explosion allows abstract volume representations e.g., money very rapidly, if you're trying to value things by assigning a number then comparing--> huge combinatorial explosion assign value to each is much easier than remembering which prefer--> remember less |
|
|
Moving away from hedonic/incentive value; not about how sweet it is in your mouth (probably remains fairly constant) also about... |
how valuable it is in the context you find yourself; given your motivational stat; given that you might have grape and apple available, given that you might be getting thirstier V complex abstract calculation |
|
|
Abstract value representations allow... |
a trade-off calculation to compare how valuable is sucrose solution to a hungry rat compared to a thirsty rat |
|
|
What is incentive learning? |
animals have to learn the value of things by experiencing them in the motivational state they're in also gives rise to the irrelevant incentive effect |
|
|
What is the irrelevant incentive effect? |
if rats learn that sucrose is a good thing when hungry, they don't learn anything about value when thirsty and needs to learn this separately |
|
|
What does the OFC represent? |
the product of this incentive learning learning how to value things relative to emotional state and relative to what other things were around |
|
|
What other things can influence value? |
Delay, effort (or cost) and probability (or risk) |
|
|
How does delay effect value? |
cue that predicts a pellet in 10 secs is not as valuable/rewarding as a cue that tells you you're going to get a pellet in 5 secs follows fairly standard hyperbolic decay function |
|
|
What is temporal discounting? |
things that are more distal are valued less; predictable in extent to which they're valued less |
|
|
How does effort/cost effect value? |
more effort have to put into getting something, trade-off and becomes less valuable |
|
|
How does probability/risk affect value? |
low prob of outcome, outcome less valuable less likely to respond to that outcome vs one that gives you a higher probability even of a smaller outcome |
|
|
What areas are activated by immediate over delayed reward? |
postrior cingulate cortex, mPFC, VStr, mOFC |
|
|
What areas are activated by reward in a delay independent manner? |
dlPFC, lOFC, PMA, SMA, RPar; VCtx |
|
|
There are different representations of reward. The one most closely associated with value is seen in the ___ |
OFC |
|
|
Effort seems to rely on what area? |
anterior cingulate cortex of the mPFC |
|
|
Rat runs down a T maze: can get 4 pellets on one side and 2 pellets on the other; has to climb up a barrier to get the 4 pellets. Does animal care in terms of choice? How can you affect behaviour? |
About 80% of time, rats climb over barrier Immediately after you damage the anterior cingulate cortex, rats no longer show ANY willingness to put in any effort to get high reward Automatically choose 2 pellet direction and avoid barrier If put barrier in both directions then lesioned animals will begin to go back to where other animals are; if has to climb over in both directions then may as well go for 4 pellets over 2 pellets; just unwilling to put effort in for higher reward; no longer capable of doing appropriate trade-off not assessing value vs effort very well although can assess value seems as if any effort at all means it has no value |
|
|
Probability-based trade-offs are multiply represented in the brain, in areas including the... |
OFC, mFPC, parietal cortex & even in the firing of midbrain dopamine neurons
|
|
|
What else do probability-based trade-offs depend on? |
the nature of the risk--> is this a certain or uncertain risk |
|
|
What is neuroeconomics? |
the neuroscience of decision-making and value the way we get around the world trying to factor in all these things such as delay, risk, probability etc |
|
|
Rats are very rational, very rapidly work out what is worth their while doing. What do they factor in? |
delay, which lever gives more pellets, magnitude of reward--> rational. Also good at assessing probability |
|
|
People/rats/primates are pretty rational, why? |
really good systems for representing these abstract notions of value and weighting one thing up against another people are worst at temporal discounting |
|
|
Describe the systems in reward processing in producing a behavioural response |
OFC integrates sensory (gustatory/temporal cortex- e.g., how tastes/feels), affective (amygdala- e.g., how pleasant) and motivational (hypothalamus- e.g., how hungry/thirsty) info to derive the value of potential reward outcomes. OFC calculates value relative to other trade-offs, relative to need and on an arbitrary scale that allows abstract comparisons- abstract representation of all these things of value OFC--> DLPFC and MPFC DLPFC<->MPFC DLPFC- construction of plan to obtain reward outcome MPFC- evaluation of effort involved in plan DLPFC & MPFC--> behavioural response |
|
|
Why can't the OFC directly inform the value of an action |
Remember that the OFC is not connected to motor regions, so can't directly inform the value of an action
|
