Research Design Mid-Term

Front 1

Name John Stuart Mill's 3 characteristics of a CAUSAL REALTIONSHIP?

Back 1

1) the cause precedes the effect
2) the cause is related to the efect
3) there is no other PLAUSIBLE alternative explanation for the effect, other than the cause

Front 2

Briefly explain how an EFFECT can be defined using the concept of a "counterfactual model."

Back 2

A counterfactual model defines the effect as the difference between what did happen (in the factual) and what WOULD HAVE happened (in the counterfactual) if the cause did not happen.

Front 3

Mediator VS. Moderator

Back 3

A mediator is a 3rd variable that comes b/t the cause and effect and that transmits the causal influence from the cause to effect. So X causes Y but goes through M (mediator), X causes M and M causes Y.
*X-->M-->Y

A Moderator is a variable that influences the effects of treatment.
*A moderator variable does not change the outcome b/t variables, BUT rather, changes the relationship b/t them

Ex. the light bulbs and the power plant are related, but the ligth SWITCH is the moderator...the switch doesn't cause the energy itself (the plant does), it just moderates between them.

X--->Y
I
MOD

Front 4

INUS Condition

Back 4

Insufficient but Non-redundant part of an Unnessessary but Sufficient condition

Front 5

Factual VS Counterfactual

Back 5

the factual is a fact..it is simply something that happened (a simple statement about what happened), while a counterfactual is what WOULD have happened, it is not real b/c it never happened

Front 6

Randomized VS Quasi-Experiment

Back 6

Both require manipulation but in a Randomized Exp. you control for other variables (other possible causes), while Quasi-Exp. you do NOT control for other variables.
*Randomized Exp.'s also randomly assign participants to conditions so they are beter, but we usually do quasi-exp b/c they are easier to do.

Front 7

Correlational Study VS Quasi-Experimental Study

Back 7

-A Correlational Study is an empirical study with NO MANIPULATION and NO RANDOM ASSIGNMENT
(this study simply observes the realtionship among variables)
*Correlational study is not an experiment!

-Quasi-Experimental Study DOES entail manipulation (but also no random assignment)

Front 8

UTOS

Back 8

Units (of observation)
Treatment (manipulation)
Observations (which variables do we want to look at?)
Settings (including time)

Each experiment consists of UNITS that receive the experiences being contrasted, of the TREATMENTS themselves, of OBSERVATIONS made on the units, and of the SETTINGS in which the study is conducted

*"UTOS are "instances in which data are collected"

Front 9

Modus Operandi

Back 9

"Methods of Operation" A method for inferring the cause of an observed effect by matching the pattern of observed effects to the pattern usually left by known causes

*It is a diagnostic technique that goes in the opposite direction of the cause-effect relationship

Ex. Doctors use it b/c they match the symptoms you have (the effect) to causes when you are in pain.
Ex. Robbery commited and suspect wore baseball cap and smoke menthols...so look for individuals who fit this pattern (wear baseball cap and smoke menthols)

Front 10

Confounded Variables

Back 10

-When the relationship may not be causal at all, but rather may be due to a 3rd variable (the counded variable)

*An extraneous variable that covaries with the variable of interest.

*Confounded variables are correlated AND have an influence on a 3rd variable.

EX. V1 and V2 together or BOTH cause V3

V1---->V3
I I
I I
V2-------I

Front 11

Causal Relationship

Back 11

(Any changes in X will result in changes in Y)

Front 12

Latent (construct) Variables VS Manifest Variables

Back 12

-Latent Variable: a variable that you CANNOT observe or manipulate (so a variable that is not directly observed but is inferred or estimated from observed variables)
-EX. Gravity, intelligence, etc.
-It doesn't really physically exist

-Manifest Variable: a variable that you can directly observe or manipulate (can put your hands on them)

*So we often have to substitute manifest variables for latent variables for the sake of research, so that we have something that we can physically measure
-EX sub an IQ test for general mental ability

Front 13

Validity

Back 13

-The evidence you provide to support your claim and counter any other threats

*The truth, correctness, and degree of support for an inference

-To what degree do we find an inference to be true?

-Since it's a claim, we have to verify it

-EX: "X causes Y" this is a knowledge claim...validity would be the evidence that is provided to show that the knowledge claim is PLAUSIBLE (not possible)

*Overall, validity is your ability to counter all the "buts"

Front 14

4 Types of Validity

Back 14

1) Statistical Conclusion Validity: the validity of inferences about the correlation b/t treatment and outcome (basically says show me the evidence that X is correlated with Y)

2) Internal Validity: the validity of inferences about whether observed covariation b/t A (treatment) and B (outcome) reflects a causal relationship from A to B as those variables were manipulated or measured
(asks: where is the evidence that X causes Y? and says that there are no other plausible alternatives)

3) Construct Validity: the validity of inferences about the higher order constructs that represent sampling particulars (asks where is your evidence that this test really measures what it says it's going to measure?)

4) External Validity: the validity of inferences about whether the cause-effect relationship holds over variation in persons, settings, treatment variables, and measurement fvariables (asks how do you know your population really comes from where you say it comes from? To what degree can you generalize your results to other types of instances?)

Front 15

Grounded Theory of Causal Generalizations

Back 15

-Suggests that scientists make causal generalizations in their work by using 5 closely realted principles:

1.Surface Similarity
2.Ruling out Irrelevances
3. Making Discriminations
4. Interpolation & Extrapolation
5. Causal Explanation

Front 16

THREATS to Statistical Conclusion Validity

Back 16

1) Statistical Power
2) Violated Assumptions of Statistical Tests
3) Fishing & the Error Rate Problem
4) Unrealiability of Measures
5) Restriction of Range

Front 17

Restriction of Range problem (one of the threats to statistical validity)

Back 17

-When you cut off the extremes, the correlation goes down

Front 18

NHST

Back 18

-Null Hypothesis Significance Testing

*used to address whether cause and effect co-vary

Front 19

THREATS to Internal Validity

Back 19

-(the validity of your claim that a cause-effect relationship exists b/t A and B)

1) Ambiguous Temporal Precedence (cause precedes effect)
2) Selection
3) History
4) Maturation (change INSIDE the participants)
5) Regression (to the mean)
6) Attrition

Front 20

Attrition

Back 20

(one of the threats to Internal Validity)

-loss of respondants to treatment/measurement (ie drop out rate)

-Can cause threat to statistical conclusion validity as well b/c it REDUCES POWER, also if one group has more or less people than another group, this is DIFFERENTIAL attrition, which is even worse b/c we wonder WHY there are more people in one group over the other (this threatens internal validity b/c the treatment itself may be what encouraged/discouraged people

Front 21

Regression (to the mean)

Back 21

-(one of the threats to internal validity)

-People in extreme groups will tend to shift to opposite end

-Occurs when respondents were chosen b/c they had scores that were higher or lower than the average

-Regression to the mean occurs b/c measures are not perfectly correlated with each other

*Can only happen if you have MORE THAN ONE test

Front 22

On what 2 levels is construct validity threatened??

Back 22

1) on the definition of what it is itself

2) on the measurement of it

EX-Pluto

Front 23

THREATS to Construct Validity

Back 23

1) Inadequate Explication of Constructs
2) Construct Confounding (EX ESL student doing math problem)
3) Experimental Expectations (aka Rosenthal bias)
4) Novelty & Disruption Effects (hawthorne effects)
5) Treatment Diffusion (treatment leaks from the treatment group)

Front 24

Experimental Expectancies

Back 24

(Threat to Construct Validity)

-(Rosenthal Studies)

-The experimenter can influence participants' responses

Front 25

Treatment Diffusion

Back 25

(Threat to Construct Validity)

-You have a treatment condition and a control condition, but the control condition ends up doing the treatment anyway

-(treatment somehow leaks into control group, so the control group now becomes a light version of the treatment group

Front 26

THREATS to External Validity

Back 26

**you have problems with it when you have problems with motivator variables

-Interaction of causal relationship with:
1) Units
2) Treatments
3) Observations
4) Settings

Front 27

Multiple Regression

Back 27

Finding the combination of weights that maximizes the outcome

Front 28

Random Selection VS Random Assignment

Back 28

-random selection is a representation of a population as a whole (participants are randomly selected to represent that particular population) *(tries to make samples similar to the population)

-Random Assignment is when you randomly assign which group each participant will be in (i.e. group A, group B, control group, etc) **tries to make samples similar to EACH OTHER

Front 29

Type 1 and Type 2 Errors

Back 29

Type 1 Error: you reject the null when you shouldn't

Type 2 Error: You accept the null when you shouldn't

Front 30

Statistical Power

Back 30

The probablity of accepting the null when you SHOULD

Front 31

The power of a test is effected by what 3 things?

Back 31

1) effect size
2) sample size
3) alpha level

Front 32

DESIGN: Pre-test/Post-test control group design

Back 32

R X O
R O

Front 33

DESIGN: Multiple Treatments and controls with pre-test

Back 33

R O Xa O
R O Xb O
R O O
I I
pretest post tst

Condition A = red and condition B = blue

Front 34

DESIGN: Longitudinal Design

Back 34

R O O O O ...O X OO..OO
R O O O O ...O OO..OO

(looks at the long-term effects of the treatment..gives multiple pre and post tests)

Front 35

DESIGN: Cross-Over Design

Back 35

R O Xa O Xb O
R O Xb O Xa O

(you have 2 groups that are randomly assigned and you give each group both treatments at different times)

Front 36

Quasi-Experimental Design w/o control groups:

DESIGN: One-group Post test only Design

Back 36

X O

Front 37

double blind experiment

Back 37

both the exprimenter and the patient do not know whether they are in an experiemtn group or in a control group

Front 38

effect size

Back 38

the difference in the outcome variable, between the factual and the counterfatual

Front 39

disclosure

Back 39

you need to disclose relevant factgs and important information and get informed consent from participants

Front 40

longitudinal design

Back 40

one or more groups are looked at over time

Front 41

time series

Back 41

*most popular anima design* you have a number of observations before and after treatment (it requires one treatment group)

Front 42

anima process

Back 42

auto-regressive (AR), intigrated (I), moving average (MA). Allows us to model the fluctuations (where do the fluctuations come from)

Front 43

auto-correltaion

Back 43

correlation in a particular pattern in time series design

Front 44

purposive sampling

Back 44

a method by which units are selected to be in a sample by a deliberate method that is not random

Front 45

attrition

Back 45

participants leave the study prematurely (can result in false positive or false negative results) 2 types

Front 46

treatment attrition & measurement attrition

Back 46

treatment attrition: ppl just die out over time
measurement attrition: ppl just are not available for your measure

Front 47

meta-analysis

Back 47

procedure for systematically cumulating single studies and deriving an overall, quantitative index of those studies (gives you an idea of what conditions best suit your situation/study)
*-Strengths: possible solution for some practical problems in single studies, increased power

Front 48

Judgement

Back 48

-Is the number 1 criteria in determining whether ther eis really significance

Front 49

effectiveness vs efficacy

Back 49

effectiveness: how well an intervention works when it is implemented under conditions of actual application.

efficacy: how well an internvetion works when it is implemented under ideal conditions.

Front 50

treatment attrition vs measurement attrition

Back 50

treatment attrition: refers to research participants who do not cont. in treatment, whether or not they continue taking the meaurement protocol *(failure of units to receive treatment)

measuremetn attrition: refers to failure to complete outcome measurement, whether or not treatment is completed *(failure to obtain measures on units (whether or not they are treated))