Rm Ch. 4-5: Research Designs And Method

Front 1

research strategy

Back 1

general plan of attack

Front 2

research design

Back 2

specific plan and structure of the investigation. development of a plan for selecting and measuring IVs and DVs in order to answer research questions about their relationships (the purpose of the design)

Front 3

controlling variance in the research design

Back 3

maximize systematic variance associated with IV and DV, and minimize random error or random variance including error of measurement. control extraneous variables' influences on DV

Front 4

difference in purpose between experimental and descriptive research

Back 4

experimental: cause and effect. descriptive: differences between groups as they exist in their natural habitat. both trying to answer the research question empirically and reduce contamination by extraneous variables

Front 5

group designs

Back 5

1 or more groups exposed to 1 or more levels of IV and average performance of group on DV is examined. applies to both experimental and descriptive.

Front 6

single subject design

Back 6

individual performance, but perhaps more than one person in experiment (but each is still compared to themselves). multiple subjects being looked at independently in the study. performance of individual on DV measured. usually multiple measures, time series. baseline segment--> treatment segment (AB).

Front 7

internal validity

Back 7

extent to which question is answered with minimal extraneous variance. have to control for ex variables, random variability, measurement error. too controlled=not generalizable. has to be assured before ex validity.

Front 8

external validity

Back 8

generalizability to other groups, important when doing treatment efficacy research. trying to generalize results from uncontrolled experiments is pointless. difficult to establish from a single study.

Front 9

experimental: between subjects designs

Back 9

two levels of IV (bivalent), manipulation occurring. looking at differences between a group. performance on DV is measured, and the differences is called the INDEX OF EFFECT. randomization and matching to control subject selection and achieve group equivalence.

Front 10

randomization

Back 10

everyone has an equal chance of being in any group. variables to be controlled are distributed randomly. statistical randomization: influence is constant, the larger the "n" the more likely the occurrence

Front 11

matching

Back 11

matching up on all variables except for the one being studied (match aphasics with non-aphasics on age, gender, and edu level). 2 groups matching in freq of extraneous variables. matching pairs (individuals) and randomization are better for smaller "n"s

Front 12

descriptive between subject designs

Back 12

may be used with comparative, cross sectional, and survey. multi or bivalent. cannot randomly assign. equate subjects on extraneous variables

Front 13

within subject designs

Back 13

performance of same subject compared to different conditions. at all levels of IV, no randomization of subjects. caution: sequencing or order effects, learning effect, interference. decrease these by randomization and counterbalancing

Front 14

mixed designs

Back 14

more than one IV, one between-subject and one within-subject. may combine descriptive and experimental methods. used when one group performs poorly, look to see what predicts behavior.

Front 15

characteristics of baseline behavior

Back 15

level=overall value, trend=shape of segment (flat, increasing, decreasing), slope=rate of change over time, variability=range of behavioral fluctuation (error, prefer <5% baseline)

Front 16

red flags for internal validity:history

Back 16

what happens between measurements (long time gap in longitudinal studies). life gets in the way

Front 17

red flags for internal validity: maturation

Back 17

what changes within the subject during the course of the study? changes due to normal development. is the change due to intervention or to normal development?

Front 18

red flags for internal validity: testing or practice effects

Back 18

what effect does taking one test have on taking the next test. did one subtest have clues for another subtest? need to counterbalance the order of subtests.

Front 19

red flags for internal validity: instrumentation

Back 19

what changes are there in calibration of the instruments?

Front 20

red flags for internal validity: statistical regression

Back 20

what change in performance is there in good/bad subjects? what do they do with the outliers?

Front 21

red flags for internal validity: differential selection of subjects

Back 21

what differences exist between groups not related to the study? is there something about those subjects that might have an effect?

Front 22

red flags for internal validity: mortality

Back 22

how many subjects are lost during the course of the study?

Front 23

red flags for internal validity: interaction of factors

Back 23

how many of the threats to internal validity co occur?

Front 24

red flags for external validity: subject selection

Back 24

do the subjects selected represent the whole?

Front 25

red flags for external validity: reactive or interactive effects of pretesting

Back 25

do subjects change just by going through pre-testing?

Front 26

red flags for external validity: reactive arrangements

Back 26

does the testing site itself determine the performance? using sound proof booths, etc. not very generalizable

Front 27

red flags for external validity: multiple treatment interference

Back 27

does the fact that there are multiple treatments change the performance on each individual treatment?

Front 28

improving external validity: random sampling

Back 28

all subjects in study have equal probability of being selected. not always practical, so sample from cluster from specific subpopulation or sample from stratified groups of subjects

Front 29

improving external validity: increase sample size

Back 29

use power analysis or effect size. power analysis determines the probability of detecting a hypothesized degree of effect in a sample size. if the effect size is sufficient, theres a good probability of detecting the effect in the sample size used. the more variable the data, the more participants needed.

Front 30

improving external validity: replication

Back 30

a study that can be repeated and produce the same results over and over is stronger. direct rep: same subjects/different researcher. systematic rep: variation in subjects or treatment.

Front 31

statistics

Back 31

computed estimates of parameters. numeric descriptors of sample population. aids in answering research question by indicating how plausible certain conclusions are in light of the obtained data. statistical regression: always going to have people performing at the extremes. did the researcher base his research on the extremes?

Front 32

nominal level of measurement

Back 32

naming, subjects are not put in an particular order. no category is higher or lower. ex: marital status, employment status, residence.

Front 33

ordinal level of measurement

Back 33

labels placed in order from high to low. ranks do not indiciate the amount of difference between subjects. ex: scale 1-5. not any specific distance between labels

Front 34

interval scale level of measurement

Back 34

equal intervals between ranks, no absolute zero, ex: test score. all points equal.

Front 35

ratio scale level of measurement

Back 35

equal distance between scores (like interval), has absolute zero rank, for ex: weight, time

Front 36

measurement error

Back 36

score depends on true score (that person's true ability) + measurement error (variations). error caused by factors that distort the true score (iq testing, freq counts, transient and stable attributes, situational factors, characteristics of the measure itself, actual mistakes)

Front 37

reliability

Back 37

the degree to which a measure is consistent in producing the same rating when measuring the same things. consistently show differences between individuals who are truly different. 0=no reliability/1=perfect reliability. neither occur in the real world.

Front 38

measures of reliability: internal consistency

Back 38

(coefficient alpha), a reliability measure that shows internal consistency. degree to which scores on items in a scale correlate with each other.

Front 39

split half correlations (reliability)

Back 39

correlating half of the items with the other half

Front 40

test-retest correlation

Back 40

the correlation between scale scores obtained on a test at one time and score on the same scale taken at a different time.

Front 41

inter-rater reliability

Back 41

the ratio of agreements between two or more observers or raters to the total number of observations

Front 42

parallel forms

Back 42

correlation between the scores of two closely related forms of the same scale (two tests on vocab)

Front 43

face validity

Back 43

degree to which a measure appears to assess what it is supposed to assess.

Front 44

content validity

Back 44

degree to which test items correspond to the content of the assessment

Front 45

criterion validity

Back 45

degree to which test correlates to another outside validating criterion (ex: iq test)

Front 46

predictive validity

Back 46

degree to which scores on a test predict later behaviors or scores

Front 47

concurrent validity

Back 47

degree to which test correlates with another conceptually related test

Front 48

construct validity

Back 48

degree to which a test has a pattern of correlation with other variables that would be predicted by a sound theory

Front 49

method

Back 49

structural framework of article. describes the participants, the materials employed, how the materials are used, the procedures. identify the research strategy

Front 50

experimenter bias

Back 50

experimenter attributes interfere to influence behavior of participant, expectancies change how observer rates behavior. can be controlled by blind technique (experimenter doesnt know which is control and which is experimental), or by investigator and experimenter being different people. effects validity.