Psyc 320: Ch 5

Front 1

reliability

Back 1

consistency in measurement

Front 2

reliability coefficient

Back 2

ratio between the true score of variance and the total variance

Front 3

true variance

Back 3

true differences

Front 4

error variance

Back 4

variance from irrelevant, random sources

Front 5

measurement error

Back 5

all of the factors of associated with the process of measuring some variable, other than the variable being measured

Front 6

random error (aka noise)

Back 6

error caused by unpredictable inconsistencies of other variables in the measurement process

ex: unanticipated events in the testing environment

Front 7

systematic error

Back 7

error that is typically constant

ex: ruler is off by 1/10 of an inch, everything measured with that ruler was systematically off by 1/10 of an inch

Front 8

Sources of error variance

Back 8

test construction

test administration

test scoring/interpretation

Front 9

item sampling or content sampling

Back 9

variation among and between tests

ex: differences in wording and content

test developer tries to maximize true variance and minimize error variance

Front 10

examples of error due to test administration

Back 10

environment: temperature, lighting, noise

test-taker variables: physical and emotional problems

examiner-related variables: appearance, demeanor, providing clues

Front 11

examples of error due to test scoring

Back 11

subjectivity in scoring/scorer interpretation

computer glitches

Front 12

other sources of error

Back 12

sampling error (did not obtain sample representative of the population)

methodological error: improper training, wording is ambiguous, biased test items

Front 13

test-retest reliability

Back 13

correlating scores from the same people on two different administrations of the same test

Front 14

coefficient of stability

Back 14

when the interval between tests is longer than 6 months, the estimate of test-reliability is referred to as the coefficient of stability

Front 15

coefficient of equivalence

Back 15

relationship between various forms of a test

Front 16

parallel forms

Back 16

for each test, the means and variance of test scores are equal

Front 17

alternate forms

Back 17

different versions of a test that are equivalent in content and difficulty

Front 18

internal consistency estimate of reliability

Back 18

an estimate of reliability that can be measured without administering the test twice to the same people

Front 19

split-half reliability

Back 19

obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once

Front 20

acceptable ways to split a test

Back 20

1. randomly assign items to one or other half of the test

2. odd-even reliability: assign odd numbered items to one half of the test and even numbered items to the other half

3. divide the test by content so items are equivalent in content and difficulty

Front 21

inter-item consistency

Back 21

correlation among all items on a scale

Front 22

homogeneity

Back 22

degree to which items measure a single trait

the more homogeneous, the more inter-item consistency it is expected to have

straight forward interpretation, but inefficient for intelligence or personality

Front 23

heterogeneity

Back 23

degree to which a test measures different factors-more than one trait

Front 24

inter-scorer reliability

Back 24

degree of agreement or consistency between two or more scorers

Front 25

when inter-scorer reliability is relevant

Back 25

when a researcher wishes to quantify nonverbal behavior

Front 26

rule of thumb for reliability coefficient grading

Back 26

.90- A (.95 used for most important decisions)

.80- B

.65-.70- weak

Front 27

test-retest purpose

Back 27

evaluate the stability of a measure

Front 28

test-retest uses

Back 28

when assessing the stability of various personality traits

Front 29

test-retest sources of error variance

Back 29

administration

Front 30

alternate forms purpose

Back 30

to evaluate the relationship between different forms of a measure

Front 31

alternate forms uses

Back 31

where there is a need for different forms of a test (ex: makeup test)

Front 32

alternate forms sources of error

Back 32

test construction or administration

Front 33

internal consistency purpose

Back 33

to evaluate the extent to which items on a scale relate to one another

Front 34

internal consistency uses

Back 34

when evaluating the homogeneity of a measure (ie all items are tapping a single construct)

Front 35

internal consistency sources of error variance

Back 35

test construction

Front 36

inter-scorer purpose

Back 36

to evaluate the level of agreement between raters

Front 37

inter-scorer uses

Back 37

interviews or coding of behavior

when researchers need to show that there is consensus in the way that different raters view a behavior pattern

Front 38

inter-scorer sources of error variance

Back 38

scoring and interpretation

Front 39

item-response theory (IRT)

Back 39

provides a way to model the probability that person with X ability will be able to perform at a standard of Y

Front 40

two characteristics of items within a IRT framework

Back 40

difficulty and discrimination

Front 41

difficulty

Back 41

attribute of not being easily accomplished, solved, or comprehended

physical difficulty: how hard it is for a person to engage in a particular activity

Front 42

discrimination

Back 42

the degree to which an item differentiates among people with higher or lower levels of the trait or ability being measured