Psych 371 Chapter 10

Front 1

Provide and recognize original examples of learning in which no new behavior is acquired. (pp. 347-348)

Back 1

Changes in rate of behaviour (ie. Pigeon that once spun clockwise once per minute now spins 10 times per minute). Any change in rate, pattern, quality, etc.

Front 2

Comment: Forms of learning

Back 2

Comment: Learning can take many different forms, only some which involve the acquisition of new behaviors. Sometimes learning consists of discovering when to apply a previous learned response or how many times to apply it. The author begins the chapter by providing many examples in which learning occurs without the acquisition of new behaviors.

Front 3

Define a schedule of reinforcement

Back 3

The rule describing the delivery of reinforcement

Front 4

Define schedule effects. (p. 348)

Back 4

the pattern and rate of perfrmance produced by a schedule of reinforcement.

Front 5

Define continuous reinforcement schedules.

Back 5

Behaviour reinforced every time it occurs. Opposite is extinction (a schedule of continuous non-reinforcement)

Front 6

Provide and recognize original examples of continuous reinforcement.

Back 6

Rat reinforced every time presses lever, pigeon pecking disk, child hanging up coat

Front 7

Specify the abbreviation used for continuous reinforcement.

Back 7

CRF

Front 8

Define an intermittent reinforcement schedule. (p. 349)

Back 8

When reinforcement occurs on some occassions but not on others.

Front 9

Define a fixed-ratio schedule of reinforcement.

Back 9

Behavior reinforced when it has occurred a fixed number of times.

Front 10

Provide and recognize original examples of fixed-ratio reinforcement schedules.

Back 10

Rat trained to press lever, then every third lever press is reinforced. Many games. Piecework pay (pay per number of items completed)

Front 11

Specify the abbreviation used for fixed-ratio schedules of reinforcement.

Back 11

FR (FR 3 means 1 reinforcement per 3 behaviours. FR 1 is continuous reinforcement)

Front 12

Identify the response pattern that a fixed-ratio schedule typically generates.

Back 12

High rate of performance, often punctuated by shrt pauses after reinforcement(called "post-reinforcement pauses. More work required, the longer the post reinforcement pause)

Front 13

What is meant by the run rate of a fixed-ratio schedule? (pp. 349-351)

Back 13

The rate at which the organism performs once it has resumed work after reinforcement.

Front 14

Define a variable-ratio schedule of reinforcement.

Back 14

The number of behaviours required for reinforcment varies around an average.

Front 15

Provide and recognize original examples of variable-ratio reinforcement schedules.

Back 15

Rat presses lever, reinforced every 2, 8, 6, 4, lever press. Pervert called 30,000 women, claiming to have family hostage and told her he would kill them if she did not stand naked outside of her house. Woman who called her ex-lover 1,000 times a day, found new phone numbers, continued calls, and added 500 faxes/day. Cheetah chasing prey. Salesman on commission. Gambling.

Front 16

Specify the abbreviation used for variable-ratio schedules of reinforcement.

Back 16

VR

Front 17

What response pattern does a variable-ratio schedule typically generate? (pp. 351-353)

Back 17

Steady performance. Run rates similar to comparable FR schedules. Post-reinforcement pauses are shorter and less frequent than with FR. Pauses influenced by average ratio and lowest ratio.

Front 18

Comment on gambling verses drugs

Back 18

Comment: As discussed in this chapter, most gambling games are based on variable reinforcement schedules in which the pay-offs occur after a variable and unpredictable number of responses. Such schedules of reinforcement are such powerful behavior generators that gambling is labelled as addictive, something it shares with addictive drugs. Curiously though, our society condones and even profits from variable ratio gambling schedules of reinforcement, but prohibits or limits the use of addictive drugs. Addictive gambling is, however, a behavior-environment relation, whereas addiction to drugs is a physiological effect caused by the biopsychological actions of the drug. Why does our society allow one type of addiction but prohibits the other?

Front 19

Define a fixed-interval schedule of reinforcement.

Back 19

Behavior is reinforced the first time it occurs after a constant interval.

Front 20

Provide and recognize original examples of fixed-interval reinforcement schedules.

Back 20

Pigeon pecking disk: reinforced after 5-second interval. Not many such schedules in nature. Mating is more under control of hormones than schedule. Baking. Waiting for the bus. Studying as exams approach.

Front 21

Specify the abbreviation used for fixed-interval schedules of reinforcement.

Back 21

FI 5" (read FI 5-second)

Front 22

Identify the response pattern that a fixed-interval schedule typically generates. (pp. 353-354)

Back 22

Produces post-reinforcement pauses, then steady increase in behaviour until interval has elapsed. Scalloped-shaped cumulative record. This never changes, no matter how experienced the animal is.

Front 23

Define a variable-interval schedule of reinforcement.

Back 23

Length of interval during which performing is not reinforced varies around some averabe.

Front 24

Provide and recognize original examples of variable-interval reinforcement schedules.

Back 24

Leopards, snakes, or spiders, human hunters or photographers who are waiting for prey. Air traffic controller watching radar screen.

Front 25

Specify the abbreviation used for variable-interval schedules of reinforcement.

Back 25

VI 5" (read VI 5-seond)

Front 26

Identify the response pattern that a variable-interval schedule typically generates. (pp. 354-355)

Back 26

Slight increase in likelihood of behaviour as reinforcement time elapses, but not so scallop shaped as FI

Front 27

Define a fixed-duration schedule of reinforcement.

Back 27

Reinforcement is contingent on the continuous performance of a behavior for some period of time.

Front 28

Provide and recognize original examples of fixed-duration reinforcement schedules.

Back 28

Child practicing the piano for half an hour. Reinforced if they practiced for the whole time.

Front 29

Specify the abbreviation used for fixed-duration schedules of reinforcement. (p. 356)

Back 29

FD

Front 30

Define a variable-duration schedule of reinforcement.

Back 30

Required period of performance varies around some average.

Front 31

Provide and recognize original examples of variable-duration reinforcement schedules.

Back 31

Child practicing for 10 minutes, 50 minutes, 30 minutes, 20 minutes, but no telling when reinforcers will appear.

Front 32

Specify the abbreviation used for variable-duration schedules of reinforcement. (p. 356)

Back 32

VD

Front 33

Define a differential-reinforcement-of-low-rate schedule of reinforcement.

Back 33

Behavior reinforced only if a specified period of time has elapsed since last behavior.

Front 34

Provide and recognize original examples of differential-reinforcement-of-low-rate reinforcement schedules.

Back 34

Rat receives food for pressing a lever only if 5 seconds have elapsed since last lever press.

Front 35

Specify the abbreviation used for differential-reinforcement-of- low-rate schedules of reinforcement.

Back 35

DRL 5"

Front 36

Identify the response pattern that a differential-reinforcement-of-low-rate schedule typically generates. (pp. 356-357)

Back 36

Produces extremely low rates of behavior. Some times result in performance of behaviours irrelevant to reinforcement (ie. Superstitious behavior).

Front 37

Define a differential-reinforcement-of-high-rate schedule of reinforcement.

Back 37

Require that a behavior be performed a minimum number of times in a given period.

Front 38

Provide and recognize original examples of differential-reinforcement-of-high-rate reinforcement schedules.

Back 38

Pigeon must peck a disk 5 times in 10 second period to be reinforced. Can be used to encourage student participation.

Front 39

Specify the abbreviation used for differential-reinforcement-of-high-rate schedules of reinforcement. (p. 357)

Back 39

DRH

Front 40

Define a fixed-time schedule of reinforcement.

Back 40

Reinforcer delibered after given period without regard to behavior. Similar to FI schedules, but require no behavior.

Front 41

Provide and recognize original examples of fixed-time reinforcement schedules.

Back 41

Not common out of lab. Unemployment and welfare payments don't fit the bill, but close.

Front 42

Specify the abbreviation used for fixed-time schedules of reinforcement. (p. 358)

Back 42

FT

Front 43

Define a variable-time schedule of reinforcement.

Back 43

Reinforcement delivered periodically at irregular intervals of an average length, without regard for behavior.

Front 44

Provide and recognize original examples of variable-time reinforcement schedules.

Back 44

Establishing superstitious behaviors. Sport fishing (lucky catch may be attributed to the lure)

Front 45

Specify the abbreviation used for variable-time schedules of reinforcement.

Back 45

VT

Front 46

What are variable-time schedules of reinforcement used for?

Back 46

Establishing superstitious behaviors

Front 47

Comment on FT and VT schedules

Back 47

Comment: Fixed and variable time schedules involve reinforcement after a fixed amount of time or a variable amount of time. Fixed and variable time schedules are uncommon outside the laboratory because reinforcers most often depend on behavior involved in collecting them. Even in the examples of FT schedules cited in the text, pigeons still have to step up to the grain hopper (i.e. dispenser) and place their heads in an eating position once reinforcement becomes available.

Front 48

What is meant by thinning a reinforcement schedule?

Back 48

Shaping a behavior so that it will occur in spite of infrequent reinforcement.Same as stretching the ratio.

Front 49

What is meant by stretching the ratio?

Back 49

Shaping a behavior so that it will occur in spite of infrequent reinforcement. Same as thinning a schedule

Front 50

What is meant by ratio strain?

Back 50

When stretching the ratio occurs to far or too rapidly and breaks downs.

Front 51

Provide and recognize original examples of the thinning of reinforcement schedules,

Back 51

Card sharks that let customers win a lot at beginning and then slowly decrease reinforcement. Praising studying frequently at first, then less and less. Starting with lots of praise to encourage reticent person to speak, then slowly decreasing that.

Front 52

Provide and recognize original examples of stretching a ratio

Back 52

Card sharks that let customers win a lot at beginning and then slowly decrease reinforcement. Praising studying frequently at first, then less and less. Starting with lots of praise to encourage reticent person to speak, then slowly decreasing that.

Front 53

Provide and recognize original examples of ratio strain. (pp. 359-360)

Back 53

Workers who grumble about being overworked and underpaid and shirk responsibilities.

Front 54

The density (or frequency) of a reinforcement schedule is a continuum. What extremes are on either end? (pp. 360-361)

Back 54

Continuous reinforcement (FR 1 schedule) and extinction.

Front 55

Define and describe the partial-reinforcement effect or PRE.

Back 55

Tendency of behavior maintained on an intermitten schedule to be more resistant to extinction than behavior on continuoous reinforcement.

Front 56

Be able to cite and recognize original examples of this (partial-reinforcement/PRE) effect.

Back 56

Mowrer and Jones: 4 groups of rats: CRF, FR 2, FR 3, FR 4. Daily training until lever pressing reinforced 20 times. 7 days (140 reinforcements). Then extinction. Thinner reinforcement schedule before extinction, the greater the number of lever presses during extinction. Lane and Shinkman: College student behavior on extinction after VI 100" schedule. Students worked for 11 hours and repeteated behavior 8,000 times without reinforcement. Practical: therapist establishes behavior, then stretches the ratio, and behavior becomes more resistant to extinction.

Front 57

Why does the author state that the partial-reinforcement effect is paradoxical? (pp. 361-362)

Back 57

Law of effect implies that unreinforced lever presses during intermittent schedule should weaken the behavior tendencies, not make it stronger.

Front 58

Identify and describe the four major hypotheses that seek to explain the partial reinforcement effect. (pp. 362-367)

Back 58

1. Discrimination hypothesis 2. Frustration hypothesis 3. Sequential hypothesis 4. Response Unit hypothesis

Front 59

Discrimination hypothesis

Back 59

Extinction takes longer after intermittent reinforcement because it is harder to discriminate between extinction and intermittent schedule than between extinction and continuous reinforcement. E.g. slot machine that always pays off, then never pays off compared to slot machine that sometimes pays off, then never. Doesn't predict behavior well.

Front 60

Frustration hypothesis

Back 60

Abram Amsel: Nonreinforcement of previously reinforced bhavior is frustrating (an aversive emotional state). Anything that reduces frustration will be reinforcing. Continuous reinforcement: no frustration. Extinction: lots of frustration, so avoiding behavior is negatively reinforced. Intermittent reinforcement: periods of nonreinforcement and frustration. Lever pressing while frustrated is reinforced. Frustration becomes S+ for lever pressing. Extinction causes frustration, which is S+ for lever pressing, which causes more frustration.

Front 61

Sequential Hypothesis

Back 61

Capaldi: attributes PRE to differences in sequence of cues during training. Each repeat of bheavior is followed by one of two events: reinforcement or nonreinforcement. Continuous reinforcement: every repeat is reinforced. Reinforcement is S+ for lever pressing. Presence of reinforcement is a cue for lever pressing, and absent during extinction. Intermittent reinforcement: sequence of reinforcement and non reinforcement is the S+ for lever pressing. Even nonreinfoced lever presses are an S+ cue for lever pressing. Therefore, thinner the schedule, the more resistant to extinction.

Front 62

Response Unit Hypothesis

Back 62

Mowrer and Jones: Consider behaviour in terms of what is being reinforced: CRF schedule, 1 lever press is reinforced. FR 2 schedule, 2 lever presses is reinforced, etc. VR schedules more complicated, because the number of behaviours being reinforced varies, but the concept still holds. Consider previous Mowrer and Jones experiemtn: units of behaviour until extinction was CRF: 128, FR 2: 94 responses. FR 3 71.8 responses, etc. When responses are defined as the units required for reinforcement, total number of responses during extinction declines as reinforcement schedule gets thinner.

Front 63

Define a multiple schedule of reinforcement.

Back 63

Behavior under influence of two or more simple schedules, each associated with particular stimulus.

Front 64

Provide and recognize original examples of multiple reinforcement schedules.

Back 64

Pigeon pecking reinforced on FI 10" schedule when red light on, but on VR 10 when yellow light is on. Schedules alternate.

Front 65

Specify the abbreviation used for multiple schedules of reinforcement. (p. 367)

Back 65

MULT FI 10" VR 10 schedule

Front 66

Define a mixed schedule of reinforcement.

Back 66

Same as multiple schedule, except no stimuli associated with change in reinforcement contingencies. schedule.

Front 67

Provide and recognize original examples of mixed reinforcement schedules.

Back 67

So, pigeon pecking reinforced on FI 10" schedule for 30 seconds, then on VR 10 for 60 seconds, no indication of the switch.

Front 68

Specify the abbreviation used for mixed schedules of reinforcement. (p. 367)

Back 68

MIX FI 10" VR 10

Front 69

Define a co-operative schedule of reinforcement.

Back 69

Reinforcement depends on behavior of two or more individuals.

Front 70

Provide and recognize original examples of co-operative reinforcement schedules. (p. 369)

Back 70

Pair of pigeons receive food when the two of them peck a disk 20 times. Can simply require 20, or require each to peck at least 10 times.

Front 71

Define a concurrent schedule of reinforcement.

Back 71

2 or more schedules available at once.

Front 72

Provide and recognize original examples of concurrent reinforcement schedules. (p. 369)

Back 72

Pigeon has option of pecking red disk on VR 50 or yellow disk on VR 20.

Front 73

Why are concurrent schedules of reinforcement used for studying behavior that involves a choice? (p. 370)

Back 73

The subject must choose one schedule over another: can't do both simultaneously. Moreso: predicting behavior becomes difficult when both alternatives are reinforced and the only variable is the frequency of reinforcement.

Front 74

Note on agency: ponder this (my addition)

Back 74

"cogitating is part of the behavior of choosing, and behavior cannot explain itself." Task: "to be able to predict, from the reinforcement schedules in force, how the person or animal will behave in a choice stuation."

Front 75

What is the matching law? (p. 371)

Back 75

B1/B2=r1/r2

Front 76

Why is it a poor strategy to switch back and forth between two different ratio schedules of reinforcement? (Or, why is it a good strategy to stay with one ratio schedule?)

Back 76

One ratio schedule will always produce more than another, and switching to the lower schedule occasionally does not provide an advantage.

Front 77

Why is it a good strategy to switch between two different interval schedules of reinforcement? (pp. 371-372)

Back 77

Can receive more total reinforcement by devoting some time to a lower interval schedule. E.g. VI 10" and VI 30" schedule: some time pressing the lever on VI 10" is not reinforced, but could be if devoted to VI 30" schedule.

Front 78

What is Herrnstein's formula that predicts choice in a two-choice situation?

Back 78

BA/BA+BB = rA/rA+rB or, for more than two behaviors: BA/BA+BO=rA/rA+rO (this has less predictive value)

Front 79

Describe each of the terms in the equation (Herrnstein’s formula), and explain the overall meaning of the equation. (pp. 372-373)

Back 79

Behaviors A and B, reinforcement rates A and B. In a 2 choice situation, the ratio of reinforcement rates AT THEIR MAXIMUM LEVELS predicts the ratio of behavior rates.

Front 80

Cite and recognize original examples of the matching law describing human behavior. (p. 374)

Back 80

Hedging one's bet: using most of your land to plant a crop that does well, turns nice profit under good weather conditions, and planing smaller area with less profitable crop that is hardier. College student: more time goes to 5 credit course than 1 credit course. More time at high paying job than low paying one. More time with someone who's company we enjoy than one we find tiresome.

Front 81

The choice to gamble:

Back 81

Text says anxiety likely a result of gambling rather than a cause. True, but that still makes the gambler nothing more than a dumb animal "acted upon" by his environment. No treatment is made of the choice BEFORE gambling produced any reinforcement schedule: the first time they walk into the Casino. And Satan surely knows about these schedules of reinforcement, and he will produce the strongest, most addictive possible reinforcers to capture the person who makes that first choice to gamble, or drink, or click on the shady link.

Front 82

What type of reinforcement schedule maintains gambling behavior?

Back 82

Variable ratio schedule: produces high rates of behaior that are highly resistant to change.

Front 83

Explain how and why “early wins” and “near misses” can lead to compulsive gambling.

Back 83

Kassinove and Schare: college students and computer slot machines. First 50 trials: win rate of 50%, and 90% return. Near misses where 3/4 numbers were right (e.g. 3337). 1/2 of students had big win ($10) on trial 8. After 50 trials, gambling put on extinction. Near misses were significant. Near messes on 30% of first 50 gambled much more during extinction phase than those who had only 15% near misses. Big wins had little impact (perhaps because of wealthy backgrounds)

Front 84

Gambling pigeons

Back 84

Alan Christopher (1988): trained pigeons to work for food. 50 pecks gives them three seconds at food tray. Half hour of work allows them to maintain healthy body weight. Illuminated second disk that worked like slot machine: unpredictable returns. Novice gamblers got "lucky." First 3 days, pigeons could earn up to 15 seconds at food tray. After 3rd day, better to work than gamble. They were hooked. Pounded relentlessly at the less rewarding gambling disk, and began to lose weight. Christopher prevented gambling to prevent starvation. Birds worked, and gained weight. Gambling disk presented again: back to the same problem.

Front 85

If you were the owner of a gambling enterprise, how could you use this knowledge (of early wins and near misses) to make more money? (pp. 376-377)

Back 85

Find a way to provide early wins and near misses to new gamblers. Have an intro section to the Casino. "Beginner" gambling with smaller stakes, but better success rates. Have big events where new gamblers are likely to show up, and tip the scales in their favor (but just for that night).

Front 86

Describe the ways in which reinforcement schedules have been studied in experimental (or behavioral) economics. (pp. 377-378)

Back 86

Price in luxory items rises, consumption declines. Price of essential items rises, consumption stays the same. Rats: when they have to work for psychoactive drugs (luxory) increase in number of lever presses required decreases consumption. Large increases in price of food does not affect consumption much. Battalio and Kagel studied female patients in psych ward. Tokens given for some tasks, could be traded for cigarettes, candy, etc. Choice was between TV and co and tokens, with their various reinforcers. Results: uneven distribution of wealth. Top 20% held 41% of tokens. Bottom 20% held 7% of tokens. Very similar to wealth distribution in US at the time.

Front 87

Define malingering

Back 87

Pretending to be in pain in order to avoid unpleasant duties.

Front 88

Explain how Goldbert and Cheney (1984) set up an experimental analogue for malingering.

Back 88

Four phases: 1. Trained rats to press lever for food. 2. Switched to a cooperative schedule with 2 rats, total of 50 presses for food, minimum of five presses each. 3. One rat exposed to milk continuous electric shock (like chronic pain). Workload decreased, partner picked up slack. 4. Shock eliminated, rat only very slowly reassumed its share of the workload, though this meant it ate less.

Front 89

What implications does this animal study (Goldbert and Cheney on malingering) have for human behavior? (pp. 378-379)

Back 89

People may malinger if others are willing to pick up the slack for someone they believe is hurting. Malingering may occur even though everyone, including the malingerer, loses by it.

Front 90

What criticisms have been made of research in reinforcement schedules?

Back 90

Artificial constructions not found in the real world. True: real world is seldom so simple. Lab is designed for simplicity so we can identify rules of how environment affects behavior. 2 Trivial findings with little application.

Front 91

Why is examining the effects of schedules preferable to explaining behavior in terms of personality characteristics and states of mind?

Back 91

It is not trivial to discover that personality is a function of history of reinforcement. Traditional explanations of "stick-to-it-iveness" are circular.Scientific way of accounting for differences in behavior: giving up after failure may be due to reinforcement history not included reinforcement for persisting in face of failure. Steady diet of easy success (or continuous reinforcement) makes one vulnerable to easy extinction.

Front 92

What other advantages accrue from research into schedules of reinforcement? (pp. 380-383)

Back 92

Answers tough questions: e.g. lots of reward or frequent rewards? Concurrent schedules says frequent rewards. Concurrent schedules also undermines the idea that extrinsic rewards undermine motivation.

Front 93

How can reinforcement schedules be used as a baseline to study the effects of different independent variables on behavior?

Back 93

Lever pressing reinforced on FI schedule showed familiar scallop pattern (baseline). Cocain administration basically eliminated the ability of the subject to learn the behavior (test). Can also be applied to comparing alcohol and cocain's effect on human performance, as well as effects of toxins, diet, sleep deprivation, exercise, brain stimulation, and many other variables.

Front 94

Give an original example of the use of reinforcement schedules as a baseline. (pp. 382)

Back 94

Test an FI schedule (or other) on learning in one group after a calming moving (or boring movie), then test the same reinforcement schedule with another group after they have watched a terrifying movie, to measure the effect of adrenaline on the reinforcement schedule.

Front 95

Reinforcement schedules, consistent patterns, and drug testing.

Back 95

Comment: As discussed at various points in this chapter, reinforcement schedules generate consistent and stable patterns of performance. Examples include a variable interval schedule that generates a moderate, even rate of response, a fixed-ratio schedule that generates a high rate of response followed by a post-reinforcement pause, and a fixed-interval schedule that generates a scalloped response pattern. After stable patterns of responding (called steady states) are achieved, an independent variable, such as a drug or a change in the environment, can be introduced to see how the variable affects the steady state.

Front 96

John Malone (my addition)

Back 96

Those who feel that an unfathomable gulf exists between human and beast are well advised to watch a pigeon pecking for food delivered according to a VR schedule and then go to a video arcade."