Test Construction

Front 1

reliability

Back 1

consistency, a test is reliable to the degree it provides repeatable, consistent results.
Types: test-retest, alternate forms, internal consistency

Front 2

validity

Back 2

a test is valid to the degree it measures what it purports to measure.

Front 3

tests of maximum performance vs. typical performance

Back 3

tell about examinee's best possible performance (achievement) vs. what examinee usually does/feels (personality).

Front 4

power vs. mastery test

Back 4

power test assesses level of difficulty a person can attain (no time limits) vs. attaining a pre-established level of acceptable performance (pass/fail - often used for basic skills tests)

Front 5

ipsative vs. normative measures

Back 5

ipsative measure uses self as the frame of reference while normative measures provide strengh of each attribute compared to others

Front 6

true score vs. measurement error

Back 6

-classical test theory
-test reliable to degree that score reflects true score rather than error
-always some degree of error, no test perfectly reliable

Front 7

reliability coefficent

Back 7

-method of estimating test's reliability
-range from 0.0 to +1.0
-.0 means 90% variability in test score due to true score differences among examinees and 10% represents measurement error

Front 8

test-retest reliability

Back 8

a.k.a. coefficient of stability
-affected time factors & provide sources of error
-not typically recommended for reliability testing

Front 9

alternative forms reliability

Back 9

a.k.a. coefficient of equivalence
-administering two equivalent forms of test to same group of examinees then corelating two sets of scores
-considered best one to use although coefficent tends to be lower than test-retest (due to content differences & time passage)
-costly

Front 10

internal consistency reliability

Back 10

a.k.a. coefficient of internal consistency
-obtaining correlations among individual items
-methods: split-half, Cronbach's alpha, Kuder-Richardson

Front 11

split-half reliability

Back 11

dividing test in two and corelating halvs as if two shorter tests
-Spearman-Brown formula can overcome the lowering of coefficient due to decreasing length (thus lowering reliability)

Front 12

Cronbach's coefficient alpha vs. Kuder Richardson

Back 12

-both recommended over split-half
-indicate average degree of inter-item consistency
-Cronbach's used for tests with multiple scored items (likert choices)
-Kuder-Richardson when test items are dichotomously scored (yes/no, T/F)

Front 13

major source of measurement error for internal consistency reliability coefficients?

Back 13

-content sampling or item heterogeneity: degree items are different in terms of content sampled

Front 14

measures of internal consistency good for assessing what and bad for assessing what?

Back 14

good: unstable traits
bad: speed tests (inflated), use test-retest/alternate forms instead

Front 15

interscorer (inter-rater) reliability

Back 15

-calculating correlation coefficient bt scoes of two different raters
-kappa coefficient: measure of agreement bt two judges who rate set of objects using nominal scales

Front 16

standard error of measurement

Back 16

-how much error an individual test score can be expected to have
-used to construct confidence intervals

Front 17

confidence intervals

Back 17

-68% probability true score falls within +- one std error of meas
-95%: within +-2 std error of meas
-98%: within +- 3 std error of meas

Front 18

factors affecting reliability

Back 18

-length of test (longer>reliability)
-more homogeneous group taking test, lower reliability
-floor/ceiling effects decreases reliability
-T/F < Multiple Choice, if can guess than lowers reliability