Statistics 2

Front 1

What are 3 Causes of Experimental Variability?

Back 1

* Experimental variability
* Extraneous variability
* Error variability

Front 2

What is the optimal Research Design?

Back 2

* MAXimize experimental variability.
* CONtrol extraneous variability.
* MINimize error variability.
---(MAX-MIN-CON)

Front 3

What are 4 Characteristics of Good Research Design?

Back 3

1. Freedom from Bias
2. Freedom from Confounding
3. Control of Extraneous Variables
4. Statistical Precision for Testing Hypotheses

Front 4

What is the Concept of Controlling Variance?

Back 4

An extraneous variable, also know as a confounding variable, will confuse the results of a study. An extraneous variable will affect the DV so that the researcher will not know if the changes in the DV are the result of changes in the IV or due to the influence of the extraneous variable.

Front 5

Freedom from confounding

Back 5

Bias can enter the data through the confounding of variables. Suppose I wanted to know which of 3 RM&S textbooks would contribute to the highest NCE score in the research section. Bias could enter this data if I have 3 different professors each using one of the textbooks. I really couldn’t be sure in the scores were the result of the textbooks or the quality of the instruction from the professor.

Front 6

Control extraneous variability

Back 6

This means that you want to structure the experiment so that there won’t be any extraneous variables affecting the DV.

Front 7

Holding factors constant

Back 7

EX: If I was attempting to bake some excellent bread and was experimenting with different ingredients, it would certainly be in my best interests to use the same type of pan (i.e., my glass pan) so I can keep everything else constant except the ingredient that I’m changing.

Front 8

Maximize experimental variability

Back 8

This means to structure the experiment in such a way that any changes in the IV will show up as changes in the DV.

Front 9

Minimize error variability

Back 9

This means that when you structure the experiment, you do so in a way that will try and minimize any fluctuations in the subjects, conditions, and measurement.

Front 10

Single variable rule

Back 10

Changing only one variable at a time.

Front 11

What is Randomization?

Back 11

Random assignment of subjects to treatment groups – Random assignment of subjects to the control group and treatment groups is the most powerful method of experimental control available to a researcher. It is random assignment that distinguishes between true experimental research and Quasi-experimental research.
o A major feature of Between-Group Designs is the random assignment of participants.
o Random assignment is the hallmark of true experimental designs
o Random assignment reduces threats to internal validity. A researcher will have good internal validity if the changes seen in the DV are due to changes in the IV and not due to some extraneous variable.

Front 12

What are 4 ways by which variance is controlled?

Back 12

1. Randomization
2. Building Conditions or factors into the design as independent variables
3. Holding conditions or factors constant
4. Statistical adjustments

Front 13

What is the Statistical Precision for Testing Hypotheses?

Back 13

Statistical precision is increased when random or error variance is decreased.

Front 14

What are 2 Purposes of Research Design?

Back 14

a. To provide answers to research questions
b. To control variance or, in other words, to control for differences

Front 15

What does "building in factors as Independent Variables" mean?

Back 15

This means making the extraneous variable an independent variable (IV). This is also called “Blocking.”

Front 16

What does Freedom From Bias mean?

Back 16

Neither the data nor the statistics used in the computation have any bias (systematic differences) in them. Bias would exist, for instance, if I were comparing cognitive-behavior therapy and hypnosis and put all the high-functioning individuals in the cognitive-behavior group.
* Bias can be eliminated by random assignment

Front 17

What is Extraneous variability?

Back 17

Variability due to extraneous variables

Front 18

What is Error variability?

Back 18

Variability due to random fluctuations in the subjects, or the experimental conditions, or even the methods of measurement.

Front 19

What is Experimental variability?

Back 19

Variability in the DV due to changes in the IV caused by the researcher

Front 20

What are 2 Criteria's for a Well-Designed Experiment?

Back 20

a. Experimental Validity
b. Internal Validity

Front 21

What are the threats to experimental validity?

Back 21

1. Non-equivalent control group
2. History effect
3. Maturations effect
4. Testing
5. Regression toward the mean.
6. Instrumentation Effect
7. Mortality or Attrition
8. Diffusion of Treatment
9. Experimenter&Participant Effects
10. Floor & Ceiling Effects

Front 22

What is a non-equivalent control group and how does a researcher reduce its effect on the study?

Back 22

* Description – problems in participant selection and/or assignment may lead to important differences between groups.
* Means of Controlling/Minimizing – random sampling and random assignment

Front 23

What is the history effect and how does a researcher reduce its effect on the study?

Back 23

* Description – refers to an event that transpires during the time when the treatment is given and may affect the observations.
* Means of Controlling/Minimizing – the use of equivalent control group

Front 24

What is the maturation effect and how does a researcher reduce its effect on the study?

Back 24

* Description – refers to normal developmental changes in participants between the pre-test and the posttest that might affect the results.
* Means of Controlling/Minimizing – use of equivalent control group

Front 25

What is the testing effect and how does a researcher reduce its effect on the study?

Back 25

* Description – refers to changes in scores on a test due to taking the test more than once. Participants’ scores often improve due to familiarization with the test.
* Means of Controlling/Minimizing – use of equivalent control group

Front 26

What is the regression toward the mean and how does a researcher reduce its effect on the study?

Back 26

* Description – refers to changes in scores due to an initial score that is really high or really low. For example, if someone did extremely well on a test, there is a greater likelihood that the score for the same person the next time will be closer to the mean. Consider the problem that arises when a group is selected because of their extreme scores (say, a group of low scoring students on the ACT), then given a treatment (workshop to boost scores) followed by a re-test. How much of the increase in scores is due to the workshop effect and how much is due to the phenomenon of regression?
* Means of Controlling/Minimizing – the use of a control group with extreme scores

Front 27

What is the instrumental effect and how does a researcher reduce its effect on the study?

Back 27

* Description – refers to changes in the measuring device or procedure over the course of a study. Examples of these types of devices include: observations, interviews, electronic or mechanical devices, the coders, etc.
* Means of Controlling/Minimizing – the use of equivalent control group

Front 28

What is the morality effect and how does a researcher reduce its effect on the study?

Back 28

* Description – refers to the effect of participants dropping out of a study. The occasional and haphazard dropping out of a few subjects due to personal reasons such as sickness, relocation, etc., does not present a serious bias. The more serious problem is the systematic loss of subjects who drop out because of a common reaction to the treatment. Consider, for example, a 100 high school students who were participating in a workshop designed to improve their ACT scores. Furthermore, keep in mind that research was being conducted on this group to see if participation in this workshop actually increased scores. If 20 of these students dropped out of the workshops after scoring poorly on the ACT practice exam, while the top 80% remained, the average score at the time of the posttest would be much higher than if the lowest 20% had remained.
* Means of Controlling/Minimizing – monitoring for differential loss.

Front 29

What is the diffusion of Tx effect and how does a researcher reduce its effect on the study?

Back 29

* Description – sometimes, the treatment delivered to one group is unwittingly allowed to spread to other groups. This frequently occurs when the treatment is primarily informational and of much interest.
* Means of Controlling/Minimizing – attempt to minimize by running participants as close together as possible

Front 30

What is Experimenter and Participant effects and how can a researcher control them?

Back 30

* Description – either experimenters or participants consciously or unconsciously affect the results of the study
* Means of Controlling/Minimizing – the use of double-blind or single blind procedures

Front 31

What is Floor and Ceiling effects and how can a researcher control them?

Back 31

* Description – occurs when using an instrument that is not sensitive enough to detect differences
* Means of Controlling/Minimizing – ensuring that instrument is reliable and valid before the study

Front 32

What is External Validity?

Back 32

* An experiment is externally valid to the extent that the results may be generalized to people and situations beyond the study.
* Problems with external validity also occur when opinion surveys are solicited, not randomly selected.

Front 33

What are 3 threats to External Validity?

Back 33

1. Selection of Subjects
2. Ecology Validity
3. Subject Reactions (Reactive Arrangement)

Front 34

How is Selection of Subjects a threat to external validity?

Back 34

If subjects are not randomly selected, the results may only apply to the subjects in the study. In other words, if a sample is not representative of the population, the results can not be generalized to the population.

Front 35

How is Ecology Validity a threat to external validity?

Back 35

The research has ecological validity if the results can be generalized from one setting or circumstance to another. Sometimes the circumstances or conditions of the research are so unique that the results cannot be generalized beyond that study.

Front 36

What is the Hawthorne Effect?

Back 36

Refers to the influence in performance, which occurs when subjects receive attention or know they are participating in research. The Hawthorne Effectwas “discovered” during a series of experiments at the Hawthorne Western Electric Plant during 1924-1932.

Front 37

What is the John Henry Effect?

Back 37

Occurs when subjects feel threatened and therefore outperform their previous performances.

Front 38

What are Demand Characteristics?

Back 38

These are all the cues, information, knowledge, even rumors the subject has heard about the experiment, which are likely to influence his or her performance. So, anything the participant has heard that may influence his or her performance.

Front 39

What is Experimenter Bias?

Back 39

These are the changes in the subject’s behavior brought about by the researcher’s expectations. It may also be called the Rosenthal Effect. He conducted research into this phenomenon and called it the Pygmalion Effect.

Front 40

What is a placebo?

Back 40

Broadly, this refers to any control treatment. This should be identical to the experimental treatment except for the critical item being studied (new drug, parenting skills training, reaction to violence in films, etc). Even so, control subjects may be influenced by this and react in unintended ways.

Front 41

What are Novelty and Disruption Effects?

Back 41

This occurs when subjects know they are getting special attention through participating in an experiment and thus increase their interest or motivation. If this occurs, the research may find that there are no long-term effects of the treatment.

Front 42

What are Between-Group Designs?

Back 42

The essential feature of of this design is the comparison of variables across two or more groups under tightly controlled experimental conditions. A common comparison group is some type of control group, a group that did not receive one of the active treatments in the study. To be able to make adequate comparisons between the groups the groups must be as similar as possible. Therefore, individual difference variables, i.e., demographic and situational variable, must be minimized to reduce threats to internal validity. Because of the emphasis on comparison and equivalent groups, assignment of participants to groups is a critical consideration. Random assignment is often the most strongly favored design.

Front 43

Name 3 experimental designs.

Back 43

1. Posttest-Only Control Group Design
2. Pretest-Posttest Control Group Design
3. Solomon Four-Group Design

Front 44

What is the make-up of a Posttest-Only Control Group Design?

Back 44

R G1 X O1

R G2 O2

The posttest-only, nonequivalent control group design contains as many groups as there are experimental treatments, plus a control group. Intact groups are used, and subjects are measured only once, after the experimental treatments have been applied.
This design is generally weak in validity

Front 45

What are the strengths of a posttest-only control group design?

Back 45

It controls for most of the threats to internal validity: maturation, instrumentation, testing effects, and regression

Front 46

What is the weakness of a posttest-only control group design?

Back 46

Statements about actual change cannot be made because there is no evidence to show that the treatment group improved because of the treatment. This is because there is no comparison to a level of functioning before the treatment.

Front 47

What is the make-up of a Pretest-Posttest Control Group Design?

Back 47

R G1 O1 X O2

R G2 O3 O4

Front 48

What is the strength of a pretest-posttest control group design?

Back 48

It controls for most threats to internal validity

Front 49

What is the weakness of a pretest-posttest control group design?

Back 49

Repeated testing can sensitize participants to a particular treatment.

Front 50

What is the make-up of a Solomon Four-Group Design?

Back 50

R G1 O1 X O2

R G2 O3 O4

R G3 X O5

R G4 O6

Front 51

What is the strength of a Solomon Four-Group Design?

Back 51

Controls for most threats to internal validity. Especially good because it can determine if there are any effects due to pretesting.

Front 52

What is the weakness of a Solomon Four-Group Design?

Back 52

Cost a great deal of time, energy, and resources.

Front 53

When are Factorial Designs used?

Back 53

When two or more independent variables are used.

Front 54

Factorial designs are similar to experimental designs except____.

Back 54

Factorial designs have more than one independent variable (IV).

Front 55

What is Statistical Conclusion Validity?

Back 55

Refers to the degree to which the researcher has come to the correct conclusion about the relationship between variables.

Front 56

What is Construct Validity?

Back 56

Is concerned with how well the variable chosen to represent a hypothetical construct actually capture the essence of the hypothetical construct.

Front 57

Time-series Designs

Back 57

Quite commonly, these are used with one or more intact groups, and with this condition they become quasi-experimental designs. These designs involve repeated measurement of one or more intact groups, with an experimental treatment inserted between two of the measurements of at least one group.

Front 58

What is the following type of design?

G O1 O2 O3 X O4 O5

Back 58

Single-Group Time Series Design

Front 59

What is the following type of design?
G1 O1 O2 X O3 O4 O5 O6 O7
G2 O8 O9 O10 O11 O12 O13 O14

Back 59

Multiple-Group Time Series Design

Front 60

What is Single-subject research?

Back 60

Is also called intrasubject research. In intrasubject research, the variables of interest are assumed to vary within the subject over time. Therefore, intrasubject research involved a longitudinal perspective achieved by repeated measurement of the variable(s).

Front 61

What are the 3 dimensions of Single-subject research?

Back 61

The first is the type of data, the second involved the operation of the independent variable, and the third involved the focus of the research. Quantitative data can be nominal (categorical), ordinal, interval, or ratio. The independent variable can either be manipulated or simply observed. The research can be focused on generating or testing a hypothesis. Hilliard (1993) used these three dimensions to define three categories of single-subject (intrasubject) designs: a) single-case experiments, b) single-case quantitative analyses, and c) case-studies

Front 62

Describe A-B designs.

Back 62

Is basically a two-phase experiment; the A phase is a baseline period, and the B phase in a treatment phase.
A problem with this design is the researcher cannot rule out threats to internal validity such as history or maturation.
The AB design is the weakness of the single-subject designs with respect to internal validity.

Front 63

Describe the A-B-A design.

Back 63

Is just like the AB design except that it has one more period where baseline measurements are taken.

This added period of baseline measurements improve the design’s internal validity.

Front 64

What is an ABAB design?

Back 64

This design is an extension of the ABA design.
Because there are two baselines and two treatments, the design has better internal validity than the two previous ones.

Front 65

What is a multiple-baseline design?

Back 65

These are like the AB designs except they involve two or more behaviors, situations, or some combination of these.

Front 66

What is a multiple-baseline design across behaviors?

Back 66

Here a single subject is observed on two or more behaviors in the same situation.
Once the baseline is stable, a treatment (X) is applied to one of the behaviors for a period of time; then it is applied to the second behavior for the same period of time.
The researcher looks to see if any changes occurred in the behaviors.
* It’s not good if the treatment caused a change in the first behavior as well as the second behavior.