Research Methods - Unit 2

Front 1

modified replication

Back 1

a replication with some major modifications such as examining a new population or using an improved measurement technique

Front 2

strict replication

Back 2

mimics the original study in all important respects to see if the same types of results as in the original study will be obtained

Front 3

literature review

Back 3

usually present in a separate chapter immediately following the chapter that contains the introduction, this is relevant information used by a researcher

Front 4

3 major databases in the social and behavioral sciences

Back 4

1. Sociological Abstracts, Planning/Policy and Development Abstracts
2. PsycINFO containing Psychological Abstracts
3. ERIC

Front 5

thesaurus

Back 5

the first step in searching a database. Used to find a topic and refine a research topic

Front 6

major headings

Back 6

guides readers through a long literature review

Front 7

Sampling

Back 7

first area of critical assessment.

More often than not, samples are less than ideal

Front 8

Instrumentation

Back 8

second area of critical assessment.

It is safe to presume that all instruments are flawed to some extent. Intruments have limitations.

Front 9

Experiments

Back 9

Third area of critical assessment.

Experiments are often flawed by having inappropriate control conditions

Front 10

synthesis

Back 10

provides a "whole" picture of what is known and what is not known as well as an attempt to show how diverse peices of information fit together and make sense

Front 11

APA citing for a jorunal article

Back 11

The title of the article is in lowercase except for the first word (or the first word following a colon), and the title of the journal is in upper-and lowercase and is italicized. It is followed by the volume number, which is also italicized

Front 12

APA citing for a book

Back 12

The title of the book is in lowercase except for the first word (or first word following a colon) and is italicized

Front 13

APA citing for material found on the Web

Back 13

Provide the date on which the material was accessed because the content of a Web site might vary from day to day

Front 14

APA citing for all sources

Back 14

Follow the punctuation used in the examples carefully. For instance, when writing in English, a commoa is not necessary between two elements in a list. However, in APA style, a comma is used between the names of two authors

Front 15

population

Back 15

the group in which researchers are ultimately interested. A population may be large or small

Front 16

sample

Back 16

a small portion of a population

Front 17

unbiased sample

Back 17

every member of a population has an equal chance of being included in the sample

Front 18

simple random sample

Back 18

an unbiased sample where names are placed in a hat and drawn at random

Front 19

samples of convenience

Back 19

a biased sample also known as accidental samples.
Ex: a psych professor wants to study a principle of learning theory as it applies to all college sophomores but only uses those students who happen to be enrolled in his course

Front 20

volunteerism

Back 20

2 forms
1. researchers issue a call for volunteers
2. volunteeris might bias a sample from an entire population

Front 21

simple random sample

Back 21

every member of a population is given an equal chance of being included in a sampe

Front 22

sampling error

Back 22

error created by random sampling (for example: a random sample might contain a disproportionately large number of males, high achievers, and so on)

Front 23

systematic sampling

Back 23

every nth individual is selected
(every 10th person or every 6th person, etc)

Front 24

precision

Back 24

discussing the magnitude of sampling errors

Results are more precise when researchers reduce sampling errors

Front 25

Two major ways to reduce sampling errors

Back 25

1. increase sample size

2. use stratification in conjunction with random sampling (stratified random sampling)

Front 26

stratified random sampling

Back 26

divide the population into strata (ex: men and women) and draw at random

Front 27

Number vs. percentage

Back 27

Researchers usually draw the same percentage of participants, not the same number

Front 28

multiple strata

Back 28

increases precision

Ex: gender AND age

Front 29

cluster sampling

Back 29

researchers draw groups or clusters of participants instead of drawing individuals

Front 30

Major drawback of cluster sampling

Back 30

each cluster tends to be more homogeneous in a variety of ways than the population as a whole

Front 31

purposive sampling

Back 31

researchers purposively select individuals whom they believe will be good sources of information

Front 32

snowball sampling

Back 32

can be useful when attempting to locate participants who are hard to find...

Convince one person to contact others and bring them in

Thus these are biased

Front 33

demographics

Back 33

background characteristics of the participants in research such as gener, age, and income

Front 34

mortality

Back 34

if some participants drop out of the experiment at mid-course, mortality is said to have occurred

Front 35

sample size

Back 35

secondary to bias

Front 36

highly precise results

Back 36

results will vary by only a small amount from sample to sample

Front 37

pilot studies

Back 37

studies designed to obtain preliminary information on how new treatments and instruments work

Front 38

instrument

Back 38

generic term for any type of measurement dvice (e.g. test, questionnaire, interview schedule, or personality scale)

Front 39

instrumentation

Back 39

term used as the heading for the section of the report where the measurement devices used in the research are described

Front 40

validity

Back 40

instrument is valid to the extent that it measures what it is designed to measure and accurately performs the functions it is purported to perform

Front 41

content validity

Back 41

researchers make judgments on the appropriateness of an instrument's contents

For achievement tests, this type of validity is essential

Front 42

face validity

Back 42

judgments are made on whether an instrument appears to be valid on the face of it

In other words, on superficial inspection, does the instrument appear to measure what is purports to measure?

Front 43

criterion

Back 43

"In the empirical approach to validity, researchers make planned comparisons to see if an instrument yields scores that relate to a criterion"

Front 44

predictive validity

Back 44

poses the questions: To what extent does the test predict the outcome it is supposed to predict?

Front 45

validity coefficient

Back 45

a correlation coefficient used to express validity

range from 0.00-1.00

1.00 is perfect validity

Front 46

concurrent coefficient

Back 46

administering the test and collection the criterion data at about the same time

Front 47

criterion-related validity

Back 47

general term for both concurrent and predictive validity

Front 48

construct validity

Back 48

type of validity that relies on objective jidgments AND empirical data (i.e. data based on observations)

Provides indirect evidence

Front 49

construct

Back 49

stands for a collection of related behaviors that are associated in a meaningful way

Ex: depression is a construct that stands for a personality trait manifested by behaviors such as lethargy, flat affect when speaking, loss of appetite, etc.

Front 50

reliable

Back 50

test is said to be reliable if it yields consistent results

Front 51

Validity vs. reliability

Back 51

When evaluating instruments, validity is more important that reliability.

Front 52

Three important principles

Back 52

1. A test with high reliability may have low validity
2. Validity is more important than reliability
3. To be useful, an instrument must be both reasonably valid and reasonably reliable

Front 53

interobserver reliability

Back 53

a researcher observes unobtrusively

Front 54

reliability coefficients

Back 54

correlation coefficients that decribe reliability

Front 55

interobserver reliability coefficients

Back 55

When researchers use reliability coeffiecients to describe the agreement between observers

Front 56

test-retest reliability

Back 56

researchers measure at two different points in time

Front 57

parallel-forms reliability

Back 57

administering one form onf the test to examinees and about a week or two later administering the other form to the same examinees, thus yielding two scores per examinee.
When the sets of scores are correlated, the result indicates the parallel-forms reliability

Front 58

How high should a reliability coefficient be?

Back 58

.80 or higher especially for individuals

For group averages of 25 or more, .50 is serviceable

Front 59

split-half reliability

Back 59

researcher administers a test but scores the items in the test as though they consisted of two separate tests

*Better example in Topic 33*

Front 60

internal consistency

Back 60

reliability estimates

Front 61

Cronbach's alpha

Back 61

method for estimating internal consistency
-based on single admin. of a test
-mathematical procedures are used to obtain the equivalent of the average of all possible split-half reliability coefficients

Front 62

norm-referenced tests (NRTs)

Back 62

tests designed to facilitate a comparison of individual's performance with that of a norm group
Ex: examinee earns percentile rank of 64...this means she scored higer than 64% of individuals in the norm group

Front 63

criterion-referenced tests (CRTs)

Back 63

tests designed to measure the extent to which individual examinees have met performance standards

Front 64

acheievement test

Back 64

measures knowledge and skills individuals have acquired

Front 65

aptitude test

Back 65

designed to predict some specific type of achievement

Example: SAT predicts success in college

Front 66

intelligence test

Back 66

designed to predict acheievement in general, not any one specific type

Front 67

3 approaches to reducing social disrability in participants' responses

Back 67

1. administering personality measures anonymously
2. observe behavior unobtrusively
3. use projective techniques (these provide loosely structured or ambiguous stimuli such as ink blots)

Front 68

Likert-type scales

Back 68

scales that have choices from "Strongly agree" to "strongly disagree"

Front 69

reverse scoring

Back 69

Ex: If 5 points are rewarded for answering "Strongly agree," to a positive statement toward school, then 5 points would be awarded for answering "strongly disagree" with a negative statement toward school

Front 70

pretest-posttest randomized control group design

Back 70

Don't know a definition for this...it's at the beginning of Topic 37

Front 71

pretest sensitization

Back 71

aka reactive effect of testing

changes observed in the experimental group may be the result of a combination of the pretest and the treatment

Front 72

posttest-only randomized control group design

Back 72

There is no pretest...

more info in Topic 37

Front 73

Solomon randomized four-group desidgn

Back 73

combination os pretest-posttest and posttest-only randomized control group design

-4 rows of symbols (4 groups)

-two experiments conducted at the same time

Front 74

true experimental designs

Back 74

random assignment to treatments

Front 75

threats to internal validity

Back 75

Depending on the design of the experiment, there may be explanations for changes other than the treatment....that's these.

Front 76

history threat

Back 76

other environmental influences on the participants between the pretest and the posttest

Front 77

maturation threat

Back 77

participants mature during the period between the pretest and the posttest which causes a threat

Front 78

instrumentation threat

Back 78

this refers to possibly changes in the instrument from the time it was used as a pretest to the time it was used as a posttest

Front 79

testing threat

Back 79

the effects of the pretest on the performance exhibited on the posttest

Front 80

statistical regression threat

Back 80

this occurs only if participants are selected on the basis of their extreme scores

Front 81

intact groups

Back 81

previously existing groups

Front 82

selection

Back 82

researchers do not assign participants to the two groups at random, there is a very stron possibility that the two groups are not initially the same in all important respects

this is a threat

Front 83

selection-history interaction

Back 83

selection of participant for two groups was not at random so they may be syustematically subjected to different life experiences

Front 84

selection-maturation interaction

Back 84

the two groups, on the average, were at somewhat different developmental stages at the time of the pretest, which would have led to different rates of maturation in the two groups, which could affect self-concept

Front 85

mortality

Back 85

differential loss of participants from the groups to be compared

Front 86

generalize

Back 86

assume that the treatment adminstered to the experimental group will work as well in the population as it did in the sample

Front 87

threats to external validity

Back 87

the extent to which the experiment is subject to threats from outside things (like the population)

Front 88

selection bias threat

Back 88

researcher's ability to generalize to a population is greatly limited....no generalizations should be made when this is the case

-you won't know whether the effects of the treatment can be expected if the treatment is administered to the entire population

Front 89

reactive effects of experimental arrangements threat

Back 89

this reminds researchers that if the experimental setting is different from the natural setting in which the population usually operates, the effects that are observed in the experimental settings may not generalize to the natural setting

Front 90

reactive effect of testing (pretest sensitization) threat

Back 90

the possibility that the pretest might influence how the participants respond to the experimental treatment

Front 91

multiple-treatment interference threat

Back 91

occurs when a group of participants is given more than one treatment

Front 92

internal validity vs. external validity

Back 92

external is concerned with "to whom and under what circumstances can the results be generalized?"

internal is concerned with "is the treatment, IN THIS PARTICULAR CASE, responsible for the observed changes?"

Front 93

3 pre-experimental designs

Back 93

-all have poor internal validity

1. one-group pretest-posttest design
2. one-shot case study
3. static-group comparison design

Front 94

one-shot case study

Back 94

gropud is given a treatment (X) followed by a test (O)

X O

Front 95

one-group pretest-posttest design

Back 95

O X O

where O is test and X is treatment

Front 96

static-group comparison design

Back 96

two groups, but participants are not assigned to the groups at random....the dashed line indicates that they are intact groups

X O
-------
O

Front 97

static-group comparison design

Back 97

two groups, but participants are not assigned to the groups at random....the dashed line indicates that they are intact groups

X O
-------
O

Front 98

quasi-experimental designs

Back 98

these are of intermediate value for exploring cause-and-effect

Front 99

nonequivalent control group design

Back 99

widely used quasi-experiment design with two intact groups

O X O
------------
O O

Front 100

equivalent time-samples design

Back 100

has only one group (or possibly one one participant). Treatment conditions are alternated

X0O X1O X0O X1O

Front 101

confound

Back 101

a source of confusion regarding the explanation for a given difference

Front 102

Hawthorne effect

Back 102

"attention effect"

There are two intertwined explanations for the differences observed because participants know they're being watched.

To control, some researchers use an experimental gropu, a control group that receive no special attention and a control group that receives attention

Front 103

John Henry Effect

Back 103

refers to the possibility that the control group might become aware of its "inferior" status and respond by trying to outperform the experimental group

Front 104

placebo effect

Back 104

the tendency of individuals to improve (or at least feel that they are improving) simply because they know they are being treated

Front 105

placebo

Back 105

"pill" that contains only inert ingredients

Front 106

blind procedure

Back 106

researchers do not disclose to the participants whether they are receiving an active or inactive substance

Front 107

double-blind experiment

Back 107

neither the participants nor the individual dispensing the drug know which is the active drug and which is the placebo

-this is done to prevent the possibility that the individual dispensing the drug will subtly communicate to the participants their status as being either "control" or "experimental" participants

Front 108

demand characteristics

Back 108

-source of confounding

-a cue that lets participants know the expect outcome of an experiment

Front 109

descriptive statistics

Back 109

summarize data so they can easily be comprehended

Front 110

frequency distribution

Back 110

shows how the scores are distributed

Front 111

frequencies

Back 111

descriptive statistics...describe how many students earned each score

Front 112

percentages

Back 112

descriptive statistics...describe how many students PER ONE HUNDRED had each score

Front 113

inferential statistics

Back 113

-helps researchers draw inferences about the effect of sampling errors on the results that are described with descriptive statistics
-help researchers make generalizations about the characteristics of populations based on data obtained by studying samples

Front 114

margin of error

Back 114

amount plus/minus

Front 115

significance tests

Back 115

help researchers decide whether the differences in descriptive statistics they identify are reliable

Front 116

parameters

Back 116

statistical values derived from a census

p-opulations yield p-arameters

Front 117

statistics

Back 117

statistical values based on the results obtained from a sample

s-amples yield s-tatistics

Front 118

null hypothesis version A

Back 118

the observed difference was created by sampling error

Front 119

null hypothesis version B

Back 119

there is no true difference between the two groups

Front 120

null hypothesis version C

Back 120

the true difference between the two groups is zero

Front 121

significance tests

Back 121

determine the probability that the null hypothesis is true

Front 122

probability (p)

Back 122

p < 0.05

Front 123

statistically significant

Back 123

states that the null hypothesis has been rejected

Front 124

nominal level

Back 124

lowest level of measurement

-"naming" level

-do not put participants in any particular mathematical category

Front 125

ordinal level

Back 125

-measurements place participants in order frmo high to low

Front 126

interval and ratio levels

Back 126

-have equal distances among the scores they yield

-ratio scale is higher level than the interval scale because the ratio scale has an absolute zero point that researchers know how to measure (weight is on the ratio scale, for instance)
-interval does not have an absolute zero

Front 127

NOIL

Back 127

no oil in rivers

Nominal
Ordinal
Interval
Ratio

Front 128

number of cases

Back 128

"N"
also known as frequencies

Front 129

univariate analysis

Back 129

researcher is analyzing how participants vary on only ONE variable

Front 130

bivariate analysis

Back 130

researchers examine a relationship between two nominal variables

Front 131

proportions

Back 131

percentage 47.8% corresponds to proportion of 0.478