Ch. 9 Mass Media Research

Front 1

Types of Experimental Research

Back 1

Controlled laboratory experiment

Quasi-experimental Designs

Field Experiments

Front 2

Experimental Research

Back 2

Older approach in mass media research

Research activities in which the researcher manipulates one or more variable to see the effect of independent variables on dependent variables (casual affect)

In an experiment, a researcher:

-Manipulates an independent variable (casual effect)
-Selects and measures a dependent variable
-Select and assign subjects
-Controls extraneous variation

Front 3

Advantages of Experimental Research

Back 3

Evidence of Causality

Researchers controls the time order of the presentation of 2 variables and ensures the cause precedes the effect
Control

Allows researcher to control the environment (testing situation) from competing influences of normal activity

Allows researcher to control the numbers and types of independent and dependent variables selected and the way they are manipulated

Allows researchers to control the subjects

Includes control over the selection process, assignment to the control or experimental group, and exposure to experimental treatment.

Cost is lower due to fewer subjects

Easy for others to Replicate

Front 4

Disadvantages of Experimental Research

Back 4

Artificiality

-Ability to control the environment lends itself to an air of artificiality for the subject.

-Little direct application to real world settings (field experiments can help avert this problem)

-Behavior of interest to researchers is altered when studied out of its natural environment

Experimenter Bias

-Whether experimenter was told what findings to expect (use of double-blind technique: neither subjects nor researchers know which variables are controlled)

Limited Scope

-Most communication research topics concerned with the collective behavior

-Only can do small-scale experiments (small group of subjects)

Front 5

Eight Steps

Back 5

Select setting

Select experimental design

Operationalize variables

Decide how to manipulate the independent variable

Select and assign subjects to experimental conditions

Conduct a pilot study

Administer the experiment

Analyze and Interpret results

Front 6

Select setting

Back 6

Laboratory (control)

Natural setting (no control, field observation)

Front 7

Select experimental design

Back 7

Depends upon:

-Hypothesis or research question
-Types of variables to be manipulated or measured
-Availability of subjects
-Amount of resources available

Front 8

Operationalize Variables

Back 8

Independent variables

Dependent variables

Front 9

Independent Variables

Back 9

in terms of manipulation done to create them

Front 10

Dependent Variables

Back 10

by constructing scales or rues for categorizing observations of behavior

Front 11

Decide how to manipulate the variable

Back 11

Choose a manipulation that is as strong as possible to maximize potential differences between groups

To manipulate the variables (or variables) a set of specific instructions, events, or stimuli are developed for presentation to the experimental subjects

Straightforward manipulation:

Staged manipulation:

Can be complicated or simple

Usually involves the use of a “confederate” - causes controversy

Front 12

Select and assign subjects to experimental conditions

Back 12

Random selection process to ensure external validity

Front 13

Conduct a Pilot Study

Back 13

Manipulation check to see whether the manipulation has the intended effect

Isolate bugs

Front 14

Administer the Experiment

Back 14

Main phase

Dependent variable is measured

Subjects are debriefed

Explain purpose and implications of study, and any deception

Front 15

Analyze and Interpret results

Back 15

Subjects’ scores are tabulated and subjected to statistical analysis

Determine what results mean

Front 16

Rules to Infer Cause

Back 16

Temporal precedence
-Causes must precede effects in time

Concomitant Variation
-When causes take on higher values, effects also must take on higher values

Elimination of alternative explanations

Front 17

Manipulation

Back 17

The creation of differences between cases to see the causal effect of independent variables and dependent variables

Separating units of observation into groups and treating them differently

In the simplest case, the researcher randomly assigns cases into two groups
-One group: treatment group
-Other group: control group (no intervention)

Both groups are measured with regard to some outcome or dependent variable and differences between the groups are compared

Front 18

Control of Confounding Variables

Back 18

To ensure the internal validity of research
-i.e., the question of whether the experimental treatment was the sole cause of observed changes in the dependent variable

-Control for the effect of extraneous or confounding variables

Laboratory experiments provide greater control of extraneous variables and are stronger in internal validity

Field experiments provide for more naturalistic experiences and are stronger in external validity (ability to generalize beyond the experimental data to other subjects or groups in the population under study)

Front 19

Ways to control confounding variables

Back 19

Randomization Assignment

Matching

Front 20

Randomization Assignment

Back 20

Randomly assign subjects to various treatment groups to equalize extraneous variables across treatment groups

Assumes experimental subjects differ in a variety of unknown ways that may have an impact on the dependent variables of interest

By assigning subjects to groups at random, researcher hopes to have effects of such variables average out within each group

Front 21

Matching

Back 21

When extraneous variables are known, can be easily assessed by researcher

If gender is an extraneous variable, can have equal number of boys to girls in groups

Similar to stratified sampling

Front 22

Experimental Design Post test

Back 22

This design means:

-Subjects were randomly assigned to groups

-One group was given an experimental stimulus, represented by X

-Members of both groups were measured to see the effect of treatment

Front 23

Posttest Design

Back 23

Posttest only control group design

R x 01
R 02

Where: R = random assignment; X = experimental stimulus or manipulation and 0 = observations or measurements (numbers indicated time-order)

Front 24

Experimental Design Pretest-Posttest

Back 24

This design means:

Subjects are randomly assigned

Between the first and second observation, some type of treatment or manipulation is performed (O1 X 02)

Each line in notation refers to the experience of one group

Effect of experimental treatment is (02-01) – (04-03)

Randomization should help insure 01 = 03

Front 25

Posttest Design

Back 25

Pretest-Posttest Control Group Design

R 01 x 02
R 03 04

Front 26

Solomn's Four Group Design

Back 26

R 01 x 02
R 03 04
R x 05
R 06

Front 27

Solomn's Four Group Design Means

Back 27

Combines the first two methods (posttest-only and pretest-post test control group)

If the treatment had an effect, O2 differs from O1 and O4.

Also, O2 should be different from O6 and O3.

Comparison of O1 and O3 allows checking the randomization efficacy.

This design requires more groups, more money & more time.

Front 28

Statistical Control

Back 28

Use of statistical techniques for controlling extraneous variance

Factorial designs:

-More than one categorical independent variable in a single study

-The variables (factors) can be controlled by factorial design

Analysis of Covariance:

-Interval or ratio level measures along with experimental manipulations

-Analysis of covariance (ANCOVA)

Front 29

Quasi Experiment

Back 29

Situations in which the researcher is unable to randomly assign subjects to groups (no randomization)


Example

To test the impact of television advertising in intact school classes:

One section of a college class as the experimental group and another section as the control group

Use of statistical controls to control extraneous variance

However, this is no substitute for random assignment

Front 30

Field Experiments

Back 30

Take place in natural settings

Allow for generalizability but no control (external validity)

Usually provide a truer picture of a subject’s normal behavior

Useful for studying complex social processes and situations


Example:

Showing Advertisement A in market A and Advertisement B in market B, making sure that both markets are similar.

Actual sales of the product in both markets are monitored.

Front 31

Two Types of Field experiments

Back 31

Those in which researcher manipulates independent variables

Those in which independent variable manipulation occurs naturally as a result of other circumstances

Front 32

Advantages of Field Experiments

Back 32

External validity

Nonreactive (influence of subject’s awareness of being measured or observed has on behavior)

Inexpensive

Front 33

Disadvantages of Field Experiments

Back 33

Some issues do not relate well to this method: ethical considerations

External hindrances

Inability to control all intervening variables