Chapter 5: Reliability

Front 1

reliability coefficient

Back 1

a proportion that indicates the ratio between the true score variance on a test and a total variance

Front 2

variance

Back 2

statistic useful in describing sources of test score variability

Front 3

true variance

Back 3

variance from true differences

Front 4

error variance

Back 4

variance from irrelevant, random sources

Front 5

reliability

Back 5

proportion of the total variance attributed to true variance

Front 6

measurement error

Back 6

all of the factors associated with the process of measuring some variable, except the one being measured

Front 7

random error

Back 7

error in measuring a targeted variable caused bu unpredictable fluctuations and inconsistencies of other variables

Front 8

systematic error

Back 8

error that is typically constant or proportionate to what is presumed to be the true value of the variable

Front 9

test construction

test administration

test scoring and interpretation

Back 9

sources of error variance

Front 10

item sampling

Back 10

(source of error variance) variation among items within a test and between tests

Front 11

test environment

Back 11

(source of error variance) room temperature, level of lighting, amount of ventilation, noise

Front 12

testtaker variables

Back 12

(source of error variance) pressing emotional problems, physical discomfort

Front 13

examiner-related variables

Back 13

(source of error variance) examiner's physical appearance and demeanor

Front 14

- test-retest method

- parallel forms / alternate forms method

- split half method

- KR20

- KR21

- coefficient alpha

Back 14

models of reliability

Front 15

test-retest method

Back 15

used to evaluate error associated with administrring a test on two different times

Front 16

coefficient of stability

Back 16

test-retest coefficient of reliability

Front 17

coefficient of equivalence

Back 17

parallel/alternate forms coefficient of reliability

Front 18

parallel forms

Back 18

when the means and variances of observed test forms are equal

Front 19

parallel forms method

Back 19

compares 2 equivalent forms of a test that measure the same attribute

sometimes the two forms are given at different times

Front 20

alternate forms

Back 20

simply differebt versions of a test

Front 21

alternate forms reliability

Back 21

estimate to which different forms of the same tests have been affected

Front 22

internal consistency estimate of reliability

Back 22

evaluation of the internal consistency of the test items

Front 23

split-half method

Back 23

test is given and divided into halves that are scored separately, then results of the halves will be compared

Front 24

odd-even reliability

Back 24

assigning odd numbered items to one half of the test and even numbered items to the other half

Front 25

Spearman-Brown formula

Back 25

estimate of what the correlation between two halves would have been if each half had been the length of the whole test

Front 26

inter item consistency

Back 26

degree of correlation among all the items on the scale

Front 27

homogeneity

Back 27

degree to which a test measures a single item

Front 28

heterogeneity

Back 28

degree to which a test measures different factors

Front 29

Kuder-Richardson formulas

Back 29

formula used on tests with dichotomous items (scored 0 or 1)

Front 30

coefficient alpha

Back 30

used only in tests with non-dichotomous items (not scored 0 or 1)

Front 31

average proportional distance

Back 31

measure used to evaluate the internal consistency of a test that focuses on the degree of difference that exists between item scores

Front 32

inter-scorer reliability

Back 32

sample of tests is independently scored by two or more examiners then correlated

Front 33

homogenous - high degree of internal consistency

heterogenous - low degree of internal consistency

Back 33

homogenous - ____ degree of internal consistency

heterogenous - ____ degree of internal consistency

Front 34

dynamic characteristics

Back 34

trait, state or ability presumed to be ever-changing as a function of situational and cognitive experiences

Front 35

static characteristics

Back 35

trait, state or ability presumed to be relatively unchanging

Front 36

low

Back 36

if the variance if either variable is restricted by the sampling procedure used, then the resulting correlation coefficient tends to be _____

Front 37

high

Back 37

if the variance if either variable is inflated by the sampling procedure used, then the resulting correlation coefficient tends to be _____

Front 38

power test

Back 38

test when a time limit is long enough and some items are so difficult

Front 39

speed test

Back 39

test that contains items if uniform level of difficulty and generous time limit

Front 40

criterion-referenced test

Back 40

test designed to provide an indication of where a testtaker stands with respect to some criterion

Front 41

domain sampling theory

Back 41

considers the problems created by using a limited number of items to represent a larger, more complicated construct

Front 42

generalizability theory

Back 42

alternative model of measurement based on the idea that a person's test scores vary from testing to testing because of variable in the testing situation

Front 43

generalizability study

Back 43

examines how generalizable scores from a particular test are if the test is administered in different situations

Front 44

coefficients of generalizability

Back 44

influence of particular facets on the test score

Front 45

decision study

Back 45

examine the usefulness of test scores in helping the tests user make decisions

Front 46

item response theory (latent trait theory)

Back 46

computer focuses on the range of difficulty that helps assess the individual's ability level

Front 47

dichotomous test items

Back 47

test items answered with only one of two alternative responses

Front 48

polychotomous test items

Back 48

test items answered with three or more alternative responses

Front 49

standard error of measurement

Back 49

tool used to estimate or infer the extent to which an observed score deviates from a true score

Front 50

standard error of a score

Back 50

an index if the extent to which one individual's scores vary over tests presumed to be parallel

Front 51

confidence interval

Back 51

a range or band of test scores that is likely to contain the true score

Front 52

standard error of difference

Back 52

determines how large a difference should be before it is considered statistically significant

Front 53

test-retest method

Back 53

only of value when measuring traits that do not change over time

Front 54

carryover effect

Back 54

first session influences scores from the second session

Front 55

- careyover effect

- some skills improve with prwctice

- time interval between testing sessions

Back 55

points to consider in test-retest method

Front 56

parallel forms method

Back 56

one of the most rigorous assessments of reliability

Front 57

split-half method

Back 57

test scores gain reliability as the number of items increases

Front 58

coefficient alpha

Back 58

most general reliability coefficient

Front 59

KR20 formula

Back 59

considers all possible ways of splitting the items

Front 60

content sampling and heterogeneity

Back 60

two sources of error variance

Front 61

KR20 formula

Back 61

20th formula presented in the famous article of Kuder and Richardson

Front 62

KR21 formula

Back 62

formula that does not require the calculation of percentage who got every item right and wrong

Front 63

KR21 formula

Back 63

uses an approximation of the sum of the pq products (mean test score)

Front 64

all items must be of equal difficulty or that the average difficulty level is 50%

Back 64

points to consider in KR21 formula

Front 65

larger

Back 65

as the sample gets _____, it represents the domain more and more accurately

Front 66

higher

Back 66

greater number of items means ______ reliability

Front 67

charles spearman (1904)

Back 67

responsible for the advancement of reliability assessment

Front 68

abraham de moivre (1733)

Back 68

introduced the basic notion of sampling error

Front 69

karl pearson (1846)

Back 69

developed the product moment correlation

Front 70

classical test score theory

Back 70

assumes that each person has a true score thay would be obtained if there were no errors in the measurement

Front 71

classical test score theory

Back 71

uses the SD of errors and is usually called standard error of measurement