Research Design Flashcards

Front 1

Cohort Effects

Back 1

Differences between subjects of cross-sectional research that have to do with experience rather than with age. Cohort effects threaten the conclusion that observed differences between subjects are related to age.

Front 2

Case Study

Back 2

The in-depth study of a single unit of investigation (e.g., individual, community, institution). From an experimental viewpoint, case studies are primarily useful to identify variables for future research.

Front 3

Control Group

Back 3

In a research study, the comparison group that is not exposed to the "active" level of the independent variable; i.e., a "no-treatment" or placebo group.

Front 4

Cross-Sectional Research

Back 4

Research design in which a sample of individuals representative of several dimensions of the population is assessed at the same time. For instance, in studying the effects of age, the design might include young, middle-aged, and older subjects.

Front 5

Cause-and-Effect Sequence

Back 5

the framework for explanation in scientific research
Policy sciences are a special case because you are evaluating a policy or program intervention to see if it work and if it makes a difference

Front 6

Proof

Back 6

not the outcome of social science research
the most we can do is gradually strengthen our confidence in the validity of a causal sequence by eliminating possible alternative explanations

Front 7

Repeated testing

Back 7

Through repeated testing we build confidence in a causal sequence

Front 8

Controls

Back 8

In testing controls must be introduced for alternative explanations that are suggested by the theory, intuition, and observation

Front 9

Science is empirical

Back 9

Measurement in the defining feature of science
Precision is achieved though quantitative measurement
“When you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind…”
But quantification isn’t enough (“When you can measure it, when you can express it in numbers, your knowledge is still of a meager and unsatisfactory kind…”)

Front 10

“When you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind…”

Back 10

science is empirical but quantification isn't enough (“When you can measure it, when you can express it in numbers, your knowledge is still of a meager and unsatisfactory kind…”)

Front 11

Unit of Analysis

Back 11

The major entity that you are analyzing in your study
U/A determined by the actual analysis done in the study
All of the study factors are things that describe the u/a

Front 12

Data sets

Back 12

Collections of measurements for specified units of analysis

Front 13

“clean” data

Back 13

Not collected but developed
Developed by filtering out unwanted influences by
Statistical analysis (most common)
Research design
Better data = more reliable study = great validity to relationships

Front 14

Research Design

Back 14

the study of techniques for organizing a research process to achieve variance control prior to and during data development
compare to after-the-fact cleaning and controlling achieved through statistics and econometrics

Front 15

Bias

Back 15

Errors that seep into a structured inquiry at one or more junctures

Front 16

Five Types of Bias

Back 16

Sampling
Selection
Measurement
Model Specification
Statistical

Front 17

Sampling Bias

Back 17

From not selecting a "representative” group to analyze
“Is the sample representative of the larger population?”

Front 18

Model Specification Bias

Back 18

In Model Specificatin you define basic concepts in a model

Model Conceptualization and Omitted Variable Bias:
-Omitted variable bias (forgotten “z” has an effect)
-Functional form
-“Is the model complete?”
-“Are all of the major influences present?”

Front 19

Measurement Bias

Back 19

In measurement you assign values to the u/a atrributes

Measurement Bias:
-Response and/or non-response, coding errors, etc.
-Do we have sloppy measurement?
-“Do variables vary sufficiently?” (If variables don’t vary than they can’t explain)
-“Is indicator measurement reliable (accurate) and valid?”

Front 20

Statistical Bias

Back 20

Select appropriate data analysis model

Statistical Bias
-Violation of the assumptions associated with data analytical techniques (biased estimators)
-“Are the assumptions underlying the analysis model met?”

Front 21

Selection Bias

Back 21

Estimation and inference

Selection bias
-When background characteristics other than the main IV can explain an outcome
-Means a better model needs built
-Draws attention to the u/a
-“How adequate are the controls for eliminating alternative explanations?”

Front 22

4 Elements of Experimental Design

Back 22

IV & DV
Randomization & Ceteris Paribus
Comparison Group
Quasi-Experiments

Front 23

Quasi-experiments

Back 23

introduced in 1950’s by Campbell
Do not use randomization to create comparison groups
Confidence in causal process can be developed across a sequence of studies linked to relatively weak non-experimental designs

Depends on design structure
Uses either time or comparison group or both to diminish/eliminate specific threat to internal validity

Front 24

Randomization

Back 24

Assigning randomly U/A’s to differing levels of IVs

Front 25

Control

Back 25

Rule out threats to causal inference

Front 26

5 Schools of Thought

Back 26

Positivism (Hume)
Essentialism (probabilistic versus deterministic)
Concomitant Variation (Mill)
Falsification (Popper)
Activity (“cause” as manipulation)

Front 27

Main philosopher for Positivism

Back 27

Hume

Front 28

Positivism (Hume)

Back 28

Only recognized observable phenomena
Causal relationship based only on highly correlated variables

Front 29

Hume's 3 Conditions for Inferring Cause

Back 29

Contiguity
Temporal precedence
Constant conjunction – connection must be apparent constantly, when one variable changes so does the other

Front 30

Philosopher who Challenged Positivism

Back 30

B. Russel

Front 31

Russel's arguments against positivism

Back 31

challenged positivism & the need for causal relationships (not needed in hard science)
causal chains or intervening variables establish unacceptability of positivism

Front 32

Essentialism

Back 32

Associated with reductionism and determinism
“cause” suggests variables both necessary and sufficient for effect to occur
Factor must work all of the time to be “cause”, but causal relationships more likely to be probabilistic than deterministic

Front 33

Philosopher for Concomitant Variation

Back 33

Mill

Front 34

Concomitant Variation (Mill)

Back 34

Key to design is establishing “counterfactual”
Implied comparison, sets up elementary comparison with and without treatment/IV
Great Contribution: realize comparison of situations where particular threat to valid inference was/wasn’t operating provided key for assessing whether the threat may occur for any observed relationship

Front 35

Mill's Criterian for Cause-Effect Relationships

Back 35

1. Variable must co-relate
2. Time Order – cause precedes effect
3. Eliminate alternative explanations – the reason why “control” is critical

Front 36

Philosopher for Falsification

Back 36

Popper

Front 37

Falsification (Popper)

Back 37

Base of knowledge is ruling out alternative explanations
Can never prove theory to be true, but failures to prove true can prove false
The more causal variables ruled out, the more confident we become with the relationship between IV and DV

Front 38

Three Types of Relationships

Back 38

1. Symmetrical
2. Reciprocal
3. Asymmetrical

Front 39

Symmetrical Relationship

Back 39

2 variables co-vary, but neither responds to the other
• Alternative indicators
• Effects of a common cause
• Functional interdependence
• Parts of a system w/o interdependence
• chance

Front 40

Reciprocal Relationships

Back 40

Two variables co-vary, but each may influence the other
• Difficult to trace
• Non-recursive systems: allows for influence feedback, analytic techniques include OLS w/ lagged variables or simultaneous equations

Front 41

Asymmetrical Relationships

Back 41

one variable influences the other, influence flows in one direction only
• “Heart” of Causal logic
• Rational for independent and dependent variables
• Recursive systems: ideal for analytic techniques e.g. OLS

Front 42

Recursive Systems

Back 42

influence arrows only run in one directions i.e. asymmetrical relationships

Front 43

Variation

Back 43

Greater variation in IV is good when trying to examine variation in DV

Front 44

“sine qua non”

Back 44

without which there is nothing

The essential element in causal inference is variation across U/As

Front 45

Types of Data Structures

Back 45

Panel
Cohort
Cross Sectional

Front 46

Panel

Back 46

• Internal focus
• Use U/As as their own control
• Controls for unwanted variance associated with individual differences

Front 47

Cohort

Back 47

• External focus
• Controls for sources of unwanted influence associated with period influences
• Controls for temporally defined influences (birth, graduation)

Front 48

Cross-Sectional Data Structure

Back 48

•Controls specified variables
•Not located in time and space

Front 49

Types of Design Structures

Back 49

Experiments
Quasi-Experiments
Cross Sectional

Front 50

Experimental Design Structure

Back 50

• BEST
• Controls for all extraneous influences known & unkown
• Effectiveness may diminish over time

Front 51

Cross-Sectional Design Structure

Back 51

• Depends on theory guided model specification
• Weakest and least effective control strategy