Quantitative Methods

Front 1

Basic rules for survey questions (3)

Back 1

1. Quality

- Questions should be easy to comprehend. The shorter, the better.

- No specialized jargon, double-barreled questions or double negatives.

2. Bias

- Avoid leading questions, loaded words, and mentioning well-known individuals or institutions.

3. Order of questions

- Start with an introduction, questions that relate to the same topic should be grouped together, place the demographic questions at the end.

- Less sensitive questions should precede more sensitive questions.

- General questions should precede more specific questions.

- The order in which the questions are presented can produce biased
responses.

Front 2

Pretest

Back 2

Initial testing of the instrument among a small group of respondents who are similar to the larger sample that the researcher hopes to target with the survey.

Front 3

Purposes of pretest (3)

Back 3

1. Identify questions that are unclear to respondents
2. Identify any problems with the organizational structure of the survey instrument
3. Avoid wasting time and money

Front 4

Sampling goals (3 steps)

Back 4

1. To identify a population

2. To survey a small selection of that population and

3. To draw accurate conclusions about the entire population based on the small selection

Front 5

Population

Back 5

An entire group of people, a whole collection of objects.

Example: in the media world, all newspaper stories on a subject

Front 6

Sample

Back 6

A small portion of the population that is used to represent the population

Front 7

Random selection

Back 7

Each member of the population has an equal chance of being selected for the sample

Front 8

Elements

Back 8

The individual members of the population

Front 9

Sampling frame

Back 9

A list of all elements in the population

Front 10

Types of sampling (2)

Back 10

1. Probability ( = random)

2. Non-probability ( = non-random)

Front 11

4 main points in deciding whether to use probability or non-probability sampling

Back 11

- purpose

- cost

- time

- acceptable error

Front 12

Simple random sampling

Back 12

Selecting subjects based on the premise that each subject in the population has an equal chance of being selected.

There is no pattern.

Front 13

Systematic random sampling

Back 13

Variant of simple random sampling that requires a list of the population.

Pattern: numbering of every subject and then using a mathematical process to select participants

Front 14

Stratified random sampling

Back 14

The population is divided into homogenous subgroups, from which several simple random samples are conducted to determine participants for the study.

Pattern: follow equal representation of subgroups

Front 15

Cluster random sampling

Back 15

Dividing the population into distinct clusters, usually geographic locations.

Pattern: randomly selecting one cluster and surveying everyone in the cluster

Front 16

Multi-stage cluster sampling

Back 16

Similar to cluster sampling, but the researcher chooses a sample at various stages.

Front 17

5 types of probability sampling

Back 17

1. Simple random sampling

2. Stratified random sampling

3. Systematic random sampling

4. Cluster random sampling

5. Multi-stage cluster sampling

Front 18

Causes of sampling errors (3)

Back 18

- Chance

- Poor sampling techniques

- Non-sampling error, caused by the measuring instrument or the participants themselves

Front 19

3 types of non-probability sampling

Back 19

1. Available / convenience sampling

2. Snowball sampling

3. Quota sampling

- Not generalizable to population as a whole.

Front 20

Available / Convenience sampling

Back 20

Participants are chosen for inclusion in a study because they are a captive audience and/or are willing to participate

Front 21

Snowball sampling

Back 21

Asking respondents to forward the survey to friends and family.

Front 22

Quota sampling

Back 22

Similar to a stratified random sample.

Difference: the participants in a stratified sample are randomly selected, but those in a quota sample are not.

Front 23

Survey types (5)

Back 23

1. Mail

2. Telephone

3. Personal

4. Group administered

5. Online

Front 24

Survey

Back 24

Any procedure to ask questions of respondents'

behavior, opinions, attitudes, tastes.

Front 25

Comparability (3)

Back 25

- To compare social groups (within society and between societies)

- To assess trends

- To apply statistical analyses

Front 26

Advantages mail survey (3)

Back 26

• Less costly than telephone or personal interview
• Respondents have more time to contemplate questions

• Respondents might feel less inhibited since interviewer is not present

Front 27

Disadvantages mail survey (4)

Back 27

• Low response rate
• Low motivation to take survey
• Response bias (e.g., more educated might be more likely to take survey)
• Delays in survey returns

Front 28

Advantages telephone survey (4)

Back 28

• Good response rate
• Data are collected in time- efficient manner
• Interviewer can probe respondents
• Interviewer can urge respondents to participate

Front 29

Disadvantages telephone survey (3)

Back 29

• Technology-related difficulties in reaching respondents
• Length of questions must be short

• Does not allow use of visuals

Front 30

Advantages personal interview (3)

Back 30

• Good response rate
• Interviewer can build connection with respondent
• Allows use of visuals and audio

Front 31

Disadvantages personal interview (3)

Back 31

• High costs
• Consumes time
• Response bias (e.g., respondents might provide socially desirable responses)

Front 32

Advantages group-administered survey (2)

Back 32

• Good response rate
• Researcher available to answer questions from respondents

Front 33

Disadvantages group-administered survey (2)

Back 33

• Potential interaction between respondents

• Potential high costs

Front 34

Advantages online survey (4)

Back 34

• Can access large number of respondents at one time
• Low costs in data collection

• Data can be automatically entered into program for analysis

• Allows use of visuals and audio

Front 35

Disadvantages online survey (3)

Back 35

• Not all potential respondents have Internet access or use the Internet
• Response bias (e.g., people who are not computer savvy might not respond)
• Survey length must be short

Front 36

Advantages open-ended questions (2)

Back 36

• Respondents can freely respond in unique manner

• Respondents might reveal unexpected insights

Front 37

Disadvantages open-ended questions (2)

Back 37

• Takes longer to answer questions

• Responses might not be legible

Front 38

Advantages closed-ended questions (2)

Back 38

• Respondents can answer questions quickly

• Responses can be tabulated quickly

Front 39

Disadvantages closed-ended questions (2)

Back 39

• Respondents are limited in their responses

• Respondents might not agree on the meaning of response categories

Front 40

Hypothesis testing

Back 40

The statistical procedure designed to test a claim.

- Involve at least 2 variables and the relationship between them.

Front 41

2 types of hypotheses

Back 41

1. Null hypothesis (H0)

- No difference or change

- Never stated, always implied

2. Alternative hypothesis (H1)

- Statement of prediction

- Actual research hypothesis

The goal of hypothesis testing is to reject one hypothesis and accept the other.

Front 42

Which values are important when deciding to reject or accept the hypothesis? (4x)

Back 42

1. P-value (significance level)

2. The effect size value
3. The test statistic
4. Degrees of freedom

Front 43

Significance level (P-value)

Back 43

The probability that you are wrong when rejecting the null hypothesis.

- This is the idea of determining if results observed may be due to chance alone.

- With alpha = 0.5 you are 95% sure the outcome is not due to chance.

Front 44

Errors of analysis: Type I error

Back 44

Type I error refers to being wrong when rejecting the null hypothesis when the researcher really should have accepted it.

- Alpha error

- False positive

- Falsely accepting H1

The two types of error have an inverse relationship.

Front 45

Errors of analysis: Type II error

Back 45

Type II error refers to being wrong when accepting the null hypothesis when the researcher really should have rejected it.

- Beta error

- False negative

- Falsely rejecting H1

The two types of error have an inverse relationship.

Front 46

Effect size

Back 46

The degree to which variables are interdependent.

- Reflects the proportion of variance in the dependent variable that is associated with levels of an independent variable.

Front 47

Arrays

Back 47

A listing of the variable for each case in the data set.

Front 48

Frequency distributions

Back 48

Each unique attribute of a variable is listed, along with a count of the frequency with which it occurs.

Front 49

Relative frequencies

Back 49

With large numbers statisticians convert frequencies to relative frequencies (i.e. percentages).

Front 50

Rate

Back 50

A proportion converted to a whole number (i.e. birth rate per 100 individuals).

Front 51

Ratio

Back 51

Compares the size of two numbers by placing them in a fraction.

Front 52

Percentage change

Back 52

Generates a percentage to reveal the amount of change over time.

Front 53

Central tendency

Back 53

A construct that refers to the score or attribute of a variable that is most typical or representative of the variable distribution.

• Mode
• Median
• Arithmetic Mean
• Standard Deviation

• Skewness
• Range

Front 54

Mode

Back 54

Defines what is most typical or representative as the attribute or score in a variable distribution that occurs most frequently.

Front 55

Median

Back 55

The score or attribute of the case that is in the middle.

Front 56

Arithmetic mean

Back 56

Defines the most typical or representative score as a distribution’s arithmetic balance point ( = average).

Front 57

Standard deviation

Back 57

Measures the amount of variation or dispersion from the average.

Front 58

Range

Back 58

The difference between the highest and lowest scores of a distribution.

• Inter-decile range, inter-quintile range, and inter-quartile range

Front 59

Choosing the best measure

Back 59

1. Use the mode or percentage for a nominal variable.

2. Report the arithmetic mean and the standard deviation for distributions for quantitative (interval and ratio) variables in which the median and arithmetic mean are relatively close together.

3. Report the median for distributions in which the arithmetic mean and the median are not very close together.

4. Continue to use the commonly accepted measure of central tendency for a particular variable.

Front 60

Contingency table

Back 60

Cross tabulation or crosstabs table.

Occurrences of attributes on one variable are tabulated across = contingent on the attributes of a second variable.

Front 61

Theory

Back 61

A group of abstract constructs including statements about the relationship between them.

Front 62

Construct

Back 62

Abstract term. Not directly observable, has to be made observable > operationalization into variables.

Example: agression, intelligence

Front 63

Variable

Back 63

A characteristic or attribute that varies among that which is being studied.

- Temporal order and probable causation

Front 64

Independent Variable (IV)

Back 64

Those that probably influence or affect outcomes.

Examples: treatment, manipulated, antecedent, predictor

Front 65

Dependent Variable (DV)

Back 65

Those that are the presumed result of the influence of the IV.

Examples: criterion, outcome, effect

Front 66

Important criteria for a good operationalization (3)

Back 66

– You have valid measures

You measure what you wanted to measure

– You have a reliable measure

The measure always measures the same

– You have an objective measure

The measure measures the same when you conduct the study or when your neighbor does it

Front 67

Rule of thumb

Back 67

Use well-established scales, control the environment, use pc-driven measures and assign participants randomly to the conditions.

Front 68

Real experiment

Back 68

Participants have to be randomly assigned

IV: has to be manipulated

DV: has to be measured

Front 69

Quasi experiment

Back 69

Experimental conditions depend on attributes of participants.

• Some IVs cannot be manipulated

Examples: gender, culture, age

(Can never have within design)

Front 70

Field experiment

Back 70

- Intervention in the "real" world

- No laboratory

Advantages:
– Real situation

– Good external validity

Disadvantages:

– Influence of further factors?

– Control conditions?

Front 71

Repeated measurements

Back 71

DV is measured more than once.

- Example: communication skills after 1, 2, 3, or 4 glasses of alcohol.

Front 72

Within design

Back 72

Same sample is used for each condition.

– At least two repeated measurements

– Participants are the same or...
– “Twins”

- Example: interview same person after 1, 2, 3, and 4 glasses of wine.

- Problems: 1) boredom and 2) training effect (less errors)

Front 73

Between design

Back 73

Different sample is used for each condition.

– Each condition has different participants

– No repeated measurements

- Example: different sample/person for 1, 2, 3, and 4 glasses of wine.

- Problems: 1) different drinking capabilities, 2) communication skills, and 3) sample too small.

Front 74

Which measure of the central tendency do you use for which level of measurement?

Back 74

Nominal - mode

Ordinal - median

Interval - mean, standard deviation

Ratio - mean, standard deviation

Front 75

What is a measurement?

Back 75

x = the measurement

m = the mean

aj = condition effect

E = error in measurement

– Condition

– Participant attributes (e.g., intelligence, drinking capabilities)

Front 76

Problems with repeating (3)

Back 76

• Some measurements are not repeatable

• Some cover stories only work once

• Data collection

– Training effects?

– Comparable?
– Boring?

Front 77

Mixed designs

Back 77

Some IVs are repeated, some are not.

Example:

Within: pre- and post measurements of aggressive behavior.

Between: violent vs. non-violent digital game.

IV 1: T1 and T2 of measurement

IV 2: violent versus non-violent game

Front 78

Explicit measurements

Back 78

Participants might not know that there is something measured, but they can actively influence it.

Examples:

• Recall
• Recognition
• Explicit attitudes

• Behavior

Front 79

Implicit measurements

Back 79

Participants might know that there is something measured but they cannot actively influence it.

Examples:

• Response latencies
• Eye-Tracking
• Psychophysiological arousal

• fMRT

Front 80

Advantages and disadvantages explicit measurements

Back 80

Advantages:
- (often) easy to assess

- (often) easy to interpret
- (often) no laboratory necessary

Disadvantages:

- Self-fullfilling prophecy?

- Participants can actively influence the measurement

- No underlying processes

Front 81

Advantages and disadvantages implicit measurements

Back 81

Advantages:
- Participants cannot influence the results

- Underlying processes
- Fine-grained analyses possible
- (often) online-measurement possible

Disadvantages:

- Interpretation?

- How to analyze?

- (Often) laboratory necessary

- Some measurements need expensive equipment and a very good technician

- Data confusion?

- Fragile: Fine measurements can be easily destroyed (e.g., noise, concentration)

Front 82

Information reduction

Back 82

Filter information on what's important and what's not.

- Start by finding correlations (r).

• r can be between -1 and +1
• 0 = means no correlation at all

Front 83

Index (vs. factor)

Back 83

Type of composite measure that summarizes and rank-orders several specific observations and represents some more general dimension

– Several items which all count the same

Example: final exam > only 1 grade

Front 84

Factor (vs. index)

Back 84

Type of composite measure composed of several items that have a logical or empirical structure among them.

– Finding cohesion/pattern

- (Factor = scale)

Example: personality test > big 5 (extraversion, agreeableness, etc.)

Front 85

Factor loading

Back 85

The contribution of items to the factor

Front 86

Factor-analytic (inductive) approach

Back 86

- Use theory to create items.

- Analyze how these items cluster together.

The correlations among a number of items is the basis.

Front 87

Factor analysis + Reliability analysis

Back 87

Factor loading
– The higher the better; usually >.50

Next step: Reliability analysis
– Testing how ‘reliable’ the scale is
– Testing how ‘reliable’ each item for its scale is

Reliability is done by Cronbach’s alpha
– Between 0 and 1; the higher the better; >.70 is ok

– Run a reliability analysis for each factor

Front 88

Data reduction

Back 88

Find the pattern/structure among many items.

Confirmative (theoretically based)

• Example: Big Five
Explorative (don’t know yet)

• Media induced emotions

Front 89

Factor analysis assumptions

Back 89

Assumptions
– Measurement level: interval/ratio, ordinal if clear metric

• Dichotomous variables also possible
– Linear relations
– Variables have normal distribution
– Sample size large enough (>150)
– Strength of relationships / Multicollinearity

Front 90

Rotation (2 types)

Back 90

To create a better distribution of variables.

Varimax

- Factors are independent from each other

- Factors do not correlate with other factors

Oblique (Oblimin and Promax)
- Factors are allowed to correlate

Based on theoretical assumptions.

Front 91

Item

Back 91

The questions asked (to create the factor).

Front 92

Which test in SPSS for...

...two levels of IV + repeated measurements?

Back 92

Paired sample t-test

- Within design

Front 93

Which test in SPSS for...

...two levels of IV + unrepeated measurements?

Back 93

Independent sample t-test

- Between design

Front 94

Which test in SPSS for...

... more than two levels of IV + repeated measurements?

Back 94

ANOVAr (= ANOVA with repeated measurements)

- Within design

Front 95

Which test in SPSS for...

... more than two levels of IV + unrepeated measurements?

Back 95

One-factorial ANOVA

- Between design

Front 96

Which test in SPSS for...

... more than two IVs + only unrepeated measurements?

Back 96

Factorial ANOVA

- Between design

Front 97

Degrees of Freedom (df)

Back 97

The number of values in a statistical analyses that are free to vary.

- They can tell you something about sample size, number of IV levels, number of factors etc.

- N-1, N = sample size

- related to critical value, alpha level, and sample size