Psyc 387 Chapter 5

Front 1

1. Describe Thorndike's puzzle box apparatus and his use of it to study the behavior of cats (and other animals). Why did he study animal behavior? What were his findings? What conclusions did he draw from those findings?

Back 1

Thorndike placed a hungry cat in a “puzzle box” and put food in plain view but out of reach. The box had a door that could be opened by some simple task (pulling a wire loop). That cat began by performing a number of ineffective acts but would eventually figure it out.
He studied animal behavior because he wanted to study animal intelligence scientifically. He recognized the impossibility of estimating animal abilities from anecdotal evidence.
He found that with each succeeding trial the cat would make fewer mistakes.
Thorndike concluded that behaviour has two types of consequences – a) satisfying state of affairs – cat pulls wire loop and finds food – satisfying consequence b) annoying state of affairs – cat squeezes through bars of cage but remains confined and hungry – an annoying consequence

Front 2

2. Define Thorndike's law of effect. What did Thorndike speculate about reinforcement's neural affect? What is this view called?

Back 2

The law of effect says that the strength of a behavior depends on the consequences the behavior has had in the past – behavior is a function of its consequences.
Thorndike speculated that reinforcement strengthened bonds or connections between neurons, a view that became known as connectionism.

Front 3

3. How does reinforcement of a response give that response momentum? Explain Nevin's use of the metaphor of momentum to describe the effects of reinforcement.
Comment: Everyday observations may help you confirm the usefulness of the metaphor of momentum in describing behavior. For example, children have a hard time tearing themselves away from video games or television to do something else.

Back 3

Behavior that has been reinforced many times is more likely to persist when “obstructed” in some way. Just as a heavy ball rolling down a hill is less likely than a light ball to be stopped by an obstruction in its path.

Front 4

4. In what way did Thorndike's work depart from previous conceptions of the learning process? How did Page and Neuringer show that randomness is a reinforceable property of behavior? (p.137)

Back 4

Thorndike was the first person to show that behavior is systematically strengthened or weakened by its consequences. He shifted our attention from inside the organism to the external environment. Page and Neuringer provided reinforcers to pigeons for a series of 8 keypecks, but only when the series of keypecks was different from the previous 50 sequences. Under these circumstances, the keypeck patterns became almost truly random.

Front 5

5. Describe the essential components of a Skinner box. How did the Skinner box get its name? (p. 138)

Back 5

Essential components of a Skiner box: consists of a lever that when pressed releases food into a tray.
Clark Hull, a psychologist at Yale University, dubbed the chamber the “skinner box” and the name quickly caught on and is still used today.

Front 6

6. What is operant learning or operant conditioning? What other name refers to operant conditioning? How does operant conditioning differ from Pavlovian conditioning?
(pp. 138-139)

Back 6

Operant learning: when a behavior is strengthened or weakened by its consequences because the behavior is said to operate on the environment.
Also sometimes called instrumental learning.
In operant learning the principal behavior is not S-R learning; the principal behavior involved is not reflexive and is often complex. In operant learning the organism acts on the environment and changes it, and the change produced strengthens or weakens the behavior. The organism is necessarily active.

Front 7

7. Define reinforcement. Name the three essential features of reinforcement. Be able to explain and reproduce the contingency square (Figure 5-5). (pp. 141-142)

Back 7

Reinforcement is the procedure of providing consequences for a behavior that increase or maintain the strength of that behavior.
First: a behavior must have consequences
Second: the behavior must increase in strength (eg occur more often)
Third: the increase in strength must be the result of the consequence

Front 8

8. What are the two basic types of reinforcement? Be able to provide and recognize original examples of each type. (pp. 142-143)

Back 8

In positive reinforcement (reward training) a behavior is followed by the appearance of, or an increase in the intensity of, a stimulus. The stimulus is called a positive reinforcer (something the organism seeks out) Ex: Saying “please” gets you a drink of water.
In positive reinforcement the behavior is more likely to occur due to the stimulus (pos. reinforcer)

In negative reinforcement, a behavior is strengthened by the removal of, or a decrease in the intensity of a stimulus – called a negative reinforcer – ordinarily something the organism tries to escape or avoid. Ex: putting headphones in on the airplane to avoid listening to a screaming child. Putting the headphones in blocks out the noise and is therefore reinforced.

Front 9

9. What is escape training?

Back 9

Comment: The text does not make a clear distinction between escape and avoidance. In escape, an organism's response terminates an aversive stimulus. For example, on a sunny day, putting on a pair of sunglasses (a response) terminates the glare of the sun (an aversive stimulus) that you (the organism) are experiencing. In avoidance, the response prevents or postpones a consequence. For example, suppose you put on your sunglasses while still inside your house. Putting on the sunglasses does not terminate the glaring sun consequence, but it does prevent it from occurring (avoidance).
In escape, the response terminates some aversive stimulus. In avoidance, the response produces no immediate consequence. Instead, in avoidance, failing to respond produces a consequence. In our sunglasses example, when you put on the sunglasses while still inside, it has no consequence: it does not produce or remove the sun's glare. However, failure to but the sunglasses (while still inside) will lead to the

Front 10

10. Define discrete-trial and free operant procedures. Be able to provide and recognize original examples of each procedure. (pp. 144-147)

Back 10

The defining feature of a discrete-trial procedure is that the behavior of the participant ends the trial. The dependent variable is often the time taken to perform some behavior under study. Ex: Each time Thorndike’s cats escaped from the box, that marked the end of the trial. For example, a trainer teaching colors to a child might begin by teaching red. She would ask the child to point to red and then reward the behavior

Skinner used a free operant procedure. In this approach the behavior may be repeated any number of times. For example, placing a rat in a Skinner box – pressing the lever might cause a bit of food to fall into the tray, but the rat was free to return to the lever and press it again and again.
Usually the dependent variable in free operant experiments is the number of times a particular behavior, such as pressing a lever or pecking a disk, occurs per minute. Example: if trying to teach the color red – the therapist may play a game with the child and every time the ch

Front 11

11. Explain why scientists often simplify problems to study them. What are the advantages and disadvantages of this approach?

Back 11

Laboratory researchers simplify problems so they can identify functional relationships between independent and dependent variables. Advantages: If the relations so identified are valid, they will enable the researcher to predict and control the phenomenon in future experiments. They will also lead to hypotheses about how real world problems may be solved. Disadvantages: Most people have difficulty in seeing the relevance of laboratory experiments to their everyday lives.

Front 12

12. Compare and contrast operant and Pavlovian conditioning. Describe the parallel Skinner drew between natural selection and reinforcement.

Back 12

Differences: in Pavlovian conditioning one stimulus (US) is contingent on another stimulus (CS), whereas in operant learning, a stimulus is contingent on behavior – useful behaviors strive and others die out. Pavlovian and operant procedures also usually involve different kinds of behavior. Pavlovian involves involuntary (reflexes) behavior (blink of an eye).
Compare: They both often occur together and can be so intertwined that it is hard to say when one begins and the other ends.

Front 13

13. What are primary (unconditioned) and secondary (conditioned) reinforcers?

Back 13

Comment: It is common to refer to primary reinforcers as unconditioned reinforcers and to secondary reinforcers as conditioned reinforcers. Chance neglects to mention that primary reinforcers are also called unconditioned reinforcers.
Primary (unconditioned) reinforcers: those that are not dependent on their association with other reinforcers – often natural – food, water, sexual stimulation, weak electrical stimulation of certain brain tissues, relief from heat/cold, certain drugs – powerful but few in #s – play a limited role in human learning in advanced societies
Secondary (conditioned) reinforcers: are those that are dependent on their association with other reinforcers. Examples: praise, recognition, smiles and positive feedback

Front 14

14. What four advantages do conditioned (secondary) reinforcers have over unconditioned (or primary) reinforcers? What key disadvantage do conditioned reinforcers have?

Back 14

• Primary reinforcers lose much of their reinforcing power quickly. Conditioned reinforcers sometimes become less effective with repeated use, but this occurs much more slowly
• It is often much easier to reinforce behavior immediately with conditioned reinforcers than primary reinforcers
• Conditioned reinforcers are often a lot less disruptive than primary reinforcers
• Conditioned reinforcers can be used in many different situations
An important disadvantage: Their effectiveness depends on their association with primary reinforcers. Primary reinforcers are much more resilient – a hungry person will work for food even if the food cannot be exchanged for anything else

Front 15

15. What are generalized reinforcers?

Back 15

Generalized reinforcers have been paired with many different kinds of reinforcers and can be used in a variety of situations

Front 16

16. What is shaping? Provide and recognize original examples of shaping. (pp. 151-156)

Back 16

Shaping is the reinforcement of successive approximations of a desired behavior. Ex: Teaching a child to say a word – in the beginning you give them praise for anything that even slightly resembles the word until eventually you only give praise if they say the word correctly

Front 17

17. What five factors are responsible for the effective use of shaping?

Back 17

• Reinforcing small steps – don’t require too much at once
• Provide immediate reinforcement – cannot hesitate
• Provide small reinforcers
• Reinforce the best approximation available – use the plan as a rough guide
• Back up when necessary – Learning doesn’t always progress smoothly

Front 18

18. Explain how adults often unwittingly shape undesirable behavior in children

Back 18

Tantrums are typically the result of parents shaping a child to cry louder and louder until they get their way. Parents often give in to “shut the child up”

Front 19

19. What is a behavior chain? Provide and recognize original examples of behavior chains

Back 19

A behavior chain is a connected sequence of behavior. Ex: making a telephone call: pick up receiver, listen for tone, dial numbers, hold receiver to ear

Front 20

20. What is a chaining procedure? What is the first step?

Back 20

Training an animal or person to perform a behavior chain is called chaining. The first step in chaining is to break the task down into its component elements, a procedure called task analysis.

Front 21

21. Name two types of chaining procedures. Be able to provide and recognize original examples of forward and backward chaining. What reinforces each link of a behavior chain?

Back 21

Forward chaining: the trainer begins by reinforcing performance of the first link in the chain. This is repeated until the task is performed without hesitation. Next the trainer requires performance of the first two links, etc. Example: baking a cake

If any of the links does not readily occur, the trainer uses shaping to build it.
Backward chaining: The trainer begins by reinforcing performance of the last link in the chain. This is repeated until the task is performed without hesitation. Next the trainer requires performance of the last two links. The parts of the chain are performed in the proper sequence (not backwards).

Even a well-established chain eventually breaks down if the final link in the chain does not produce a reinforcer (ex: getting to eat the cake)

Front 22

22. Describe the concept of contingency in reinforcement. How and why does it influence reinforcement?

Back 22

Contingency refers to the degree of correlation between a behavior and its consequence. Lynn Hammond found that if rats were as likely to get food pressing the lever as they were if they didn’t press the lever, they did not continue to press the lever. A contingency between lever pressing and the reinforcer was essential. Numerous small reinforcers, when contingent on a behavior, are generally more effective than a few large ones.

Front 23

23. Describe the concept of contiguity in reinforcement. How and why does it influence reinforcement?

Back 23

Comment: The author often discusses contiguity in terms of delay of reinforcement. The general principle of contiguity is that immediate reinforcement is more effective than delayed reinforcement.
The gap between a behavior and its reinforcing consequence has a powerful effect on the rate of operant learning. In general, the shorter this interval is, the faster learning occurs.
Ex: if a 10 sec delay between when the bird pecks the disk and food arrives – operant learning will not occur
A delay often allows for other behaviors to occur.

Front 24

24. How do the following conditions affect the effectiveness of a reinforcer?: (a) size of the reinforcer,

Back 24

(a) size of the reinforcer – small reinforcers given frequently usually produce faster learning than large reinforcers given infrequently. But the size of the reinforcer does matter. Other things being equal, a large reinforcer is more effective than a small one. Ex: $100 on the street vs $1.00 on the street. The relation between reinforcer size and learning is not linear. In general the more you increase the reinforcer size, the less benefit you get from the increase. Qualitative differences in reinforcers is also important (ex: sunflower seeds vs bread/milk)

Front 25

24. How do the following conditions affect the effectiveness of a reinforcer?:(b) task characteristics,
(pp. 163-167)

Back 25

(b) task characteristics – Certain qualities of the behavior being reinforced affect the ease with which it can be strengthened. Ex: learning to walk a balance beam vs a tightrope.

Front 26

24. How do the following conditions affect the effectiveness of a reinforcer?: (c) deprivation level,

Back 26

(c) deprivation level – the effectiveness of food, water, and warmth as reinforcers varies with the extent to which an organism has been deprived of food, water, and warmth. Rats deprived of food will show much better progress through a maze reinforced with food than those who aren’t.
Deprivation is less important where secondary reinforcers are concerned. Money is not always less reinforcing for those who are rich than for those who are broke.

Front 27

24. How do the following conditions affect the effectiveness of a reinforcer?: (d) previous learning history,

Back 27

(d) previous learning history – much of the difference between fast- and slow-learning children disappears when both have similar learning histories

Front 28

24. How do the following conditions affect the effectiveness of a reinforcer?: and (e) competing contingencies.

Back 28

(e) competing contingencies – the effects of reinforcing a behavior will be very different if the behavior also produces punishing consequences or if reinforcers are simultaneously available for other kinds of behavior.

Front 29

25. What is an operant extinction procedure? What is the immediate effect of an extinction procedure? What is an extinction burst?

Back 29

Operant extinction means withholding the consequences that reinforce a behavior. The immediate effect is often an abrupt increase in the behavior on extinction (the rat will press the lever many times). This behavior is called an extinction burst. The extinction burst is typically followed by a steady and fairly rapid decline in the behavior.

Front 30

26. What effects does operant extinction have on behavioral variability? On aggression?

Back 30

The organism will often “try something else” – often a variation of the previously reinforced behavior – useful during shaping.
Extinction increases aggression. Rats have been known to bite the lever if it no longer provides food. Ex: pounding a vending machine that doesn’t work

Front 31

27. How long does it normally take to extinguish a behavior?

Back 31

The rate of the previously reinforced behavior declines and finally stabilizes.

Front 32

28. Describe spontaneous recovery with respect to operant extinction. What conditions facilitate spontaneous recovery?

Back 32

If the person/animal are put back into the training situation, the extinguished behavior occurs again, almost as though it had not been extinct. The longer the interval between the two extinction sessions, the greater the recovery.

Front 33

29. What is resurgence? Be able to provide and recognize original examples of resurgence. How can resurgence help to explain some instances of regression?

Back 33

Resurgence is the reappearance during extinction of a previously reinforced behavior. Ex: disk pecking is extinct, bird flapping is reinforced and then made extinct – the bird may go back to disk pecking. This helps to explain regression. Ex: a man who is unable to get his wife to behave as he would like by asking her nicely may resort to having a tantrum – asking nicely is on extinction, and the man reverts to a form of behavior that had been reinforced

Front 34

30. What conditions are responsible for the rate of extinction?

Back 34

The number of times the behavior was reinforced before extinction, the effort the behavior requires, the size of the reinforcer used during training.

Front 35

31. The author claims that reinforcement and extinction are parallel procedures, but that they do not have equal effects. Explain.

Back 35

One nonreinforcement does not “cancel out” one reinforcement. A single reinforcement might be followed by dozens of unreinforced lever presses. Behavior is usually acquired rapidly and extinguished slowly.

Front 36

32. Describe Thorndike's work in which he tried to separate the effects of reinforcement from those of practice.

Back 36

Comment: Thorndike showed that simply telling people to draw lines of a specified length, and having them repeatedly practice line drawing did not induce people to draw lines of accurate lengths. The people also needed feedback, that is, information about whether their attempts were correct or incorrect. It is unclear whether Thorndike actually separated the effects of practice and reinforcement. What he appeared to have done is to separate the value of practice without specific feedback from the value of practice with specific feedback. Thorndike's subjects would likely have been more successful at the task if they had been shown a line of the correct length before they began practicing and were instructed to copy it repeatedly. This wouldn't have been extrinsic reinforcement, but it would have been practice with line-length information. To show that the line-length information was a reinforcer, one would have to show the importance of it occurring only as a consequence for responding

Front 37

34. Describe Premack's relative value theory. What are the advantages and disadvantages of this approach?

Back 37

Relative value theory: Theory of reinforcement that considers reinforcers to be behaviors rather than stimuli and that attributes a reinforcer’s effectiveness to its probability relative to other behaviors.
Advantage: Premack’s theory of relative value is strictly empirical; no hypothetical concepts, such as drive, are required. An event is reinforcing simply because it provides the opportunity to engage in preferred behavior
Disadvantage: Does not explain why the word yes (for example) is often reinforcing. Another problem with the theory is that low probability behavior will reinforce high probability behavior if the participant has been prevented from performing the low probability behavior for some time.

Front 38

35. Describe Timberlake and Allison's response deprivation theory of reinforcement. What are the advantages and disadvantages of this theory?

Back 38

Response deprivation theory: behavior becomes reinforcing when the organism is prevented from engaging in it at its normal frequency. What is vital is the extent to which each behavior occurs below its baseline rate.
Advantage: The relative value of one reinforcer to the other is not vital. This theory works well for many reinforcers.
Disadvantage: Like the theories of Hull and Premack, it has trouble explaining the reinforcing power of yes.

Front 39

36. The author claims that negative reinforcement often starts out as escape responding and ends up as avoidance responding. Provide an original example illustrating this transition.

Back 39

The last 5 times Sally walked to the store she ran into a vicious dog, so she backed up and took an alternate route. Now, when Sally walks to the store she always chooses the alternate route.

Front 40

37. Explain why avoidance responding has been considered a puzzling phenomenon.

Back 40

Successful avoidance meant that something – the shock – did not happen, but how could something that did not happen be a reinforcer?

Front 41

38. Describe the two-process theory of avoidance and the evidence for and against this theory.

Back 41

Two-process theory of avoidance says that two kinds of learning experiences are involved in avoidance learning: Pavlovian and operant. Two-process theory fits all the essential facts. In addition the theory leads to logical predictions that can be tested. Ex: Miller: rats would do anything to avoid the white compartment (had become a CS for fear) – escaping from it was reinforcing.
Not all tests of two-process theory have produced supportive evidence. The idea that the avoidance behavior is reinforced by escape from an aversive CS leads logically to the prediction that if the CS were to lose its aversiveness, the avoidance behavior would cease to occur. There is evidence that the signal for shock does lose its aversiveness, yet the avoidance response persists. If the CS becomes less frightening as avoidance training continues, what reinforces the avoidance behavior?
Another problem has to do with the failure of avoidance behaviors to extinguish.

Front 42

What is a Sidman avoidance procedure? (pp. 179-182)

Back 42

Sidman avoidance procedure: An escape-avoidance training procedure in which no stimulus regularly precedes the aversive stimulus

Front 43

39. Describe the one-process theory of avoidance and the evidence that supports this approach (pp. 182-183)

Back 43

One-process theory proposes that avoidance involves only one process: operant learning. Both escape and avoidance behaviors are reinforced by a reduction in aversive stimulation. Two-process theorists said that the absence of shock could not reinforce behavior – How can something that does no happen be a reinforcer? But one-process theory says that something does happen: There is a reduction in exposure to shock, and this is reinforcing. In the case of extinction – the best way to get an animal/person to stop performing an unnecessary avoidance behavior is to prevent both the behavior and its aversive consequences from occurring (stop the animal from jumping the hurdle and stop the shock)