Eppp Terms Test Construction

Front 1

Alternate Forms Reliability

Back 1

Method of estimating a test's reliability that entails administering two forms of the test to the same group of examinees and correlating the two sets of scores. Forms can be administered at about the same time (coefficient of equivalence) or at different times (coefficient of equivalence and stability). Some see this as the best method for assessing reliability. Difficulty in creating equivalent forms.

Front 2

Classical Test Theory

Back 2

Theory of measurement that regards observed variability in test scores as reflecting two components: true differences between the examinees on the attribute(s) measured by the test and the effects of measurement (random) error. Reliability is a measure of true score reliability.

Front 3

Coefficient Alpha

Back 3

Method for assessing internal consistency reliability that provides an index of average inter-item consistency.

Front 4

Kuder-Richardson 20

Back 4

Can be used as a substitute for coefficient alpha when test items are scored dichotomously.

Front 5

Construct Validity

Back 5

The extent to which a test measures the hypothetical trait (construct) it is intended to measure. Established through use of convergent and discriminant validity on other tests, factor analysis, determining if changes is scores reflect developmental changes, and seeing if experimental manipulations have the effect that is expected.

Front 6

Content Validity

Back 6

The extent to which a test adequately samples the domain of information, knowledge or skill that it purports to measure. Determined primarily by expert judgement. Important for achievement and job tests.

Front 7

Criterion Contamination

Back 7

Refers to bias introduced into a person's criterion score as a result of the knowledge of the scorer about performance on the predictor. Tends to artificially inflate the relationship between the predictor and criterion.

Front 8

Criterion-referenced interpretation

Back 8

Interpretation of a test score in terms of a prespecified standard; in terms of percent of content correct (percentage score) or of predicted performance on an external criterion (regression equation, expectancy table).

Front 9

Criterion-related validity

Back 9

The type of validity that involves determining the relationship (correlation) between the predictor and the criterion. The correlation coefficient is referred to as the criterion-related validity coefficient. Can be either concurrent or predictive.

Front 10

Factor Analysis

Back 10

A multivariate statistical technique used to determine how many factors (constructs) are needed to account for the intercorrelations among a set of tests, subtests, or test items. True score variability consists of communality and specificity. Factors can be orthogonal or oblique.

Front 11

Factor Loading

Back 11

In factor analysis, a factor loading is the correlation between a test ( or other variable included in the analysis) and a factor.

Front 12

Communality

Back 12

The communality is the total amount of variability in scores on the test (or other variable) that accounted for by the factor analysis.

Front 13

Incremental Validity

Back 13

The extent to which a predictor increases decision-making accuracy. Calculated by subtracting the base rate from the positive hit rate.

Front 14

True Positives

Back 14

Those who scored high on the predictor and the criterion.

Front 15

False Positives

Back 15

Those who scored high on the predictor and low on the criterion.

Front 16

True Negatives

Back 16

Scored low on the predictor and on the criterion.

Front 17

False Negatives

Back 17

Scored low on the predictor and high on the criterion.

Front 18

Item Characteristics Curve

Back 18

When using item response theory, an iten characteristic curve is constructed for each item by plotting the proportion of examinees who answered the item correctly against total test score, an external criterion, or a mathematically derived estimate of a latent ability or trait.

Front 19

Item Difficulty

Back 19

An item's difficulty level is calculated by dividing the number of individuals who answered the item correctly by the total number of individuals. Ranges from 0 to 1.0, .5 is preferred.

Front 20

Item Discrimination

Back 20

The extent to which an item differentiates between examinees who obtain a high score versus low scores. Ranges from -1.0 to 1.0.

Front 21

kappa statistic

Back 21

A correlation coefficient used to assess inter-rater reliability.

Front 22

Multitrait-multimethod matrix

Back 22

A systemic way to organize the correlation coefficients obtained when assessing a measure's convergent and discriminant validity. Measure at least two traits with at least two methods.

Front 23

Norm-referenced interpretation

Back 23

Interpretation of an examinee's test performance relatie to the performance of examinees in a normative (standardization) sample. Percentile ranks, standard scores, t-scores, and z-scores are norm referenced.

Front 24

Orthogonal Rotation

Back 24

In factor analysis, an orthogonal rotation of identified factors produces uncorrelated factors.

Front 25

Oblique Rotation

Back 25

In factor analysis, produces correlated factors.

Front 26

Relationship between Reliability and Validity

Back 26

Reliability is a necessary but not sufficient condition for validity.

Front 27

Relevance

Back 27

In test construction, relevance refers to the extent to which test items contribute to achieving the stated goals of testing.

Front 28

Reliability

Back 28

The consistency of test scores across time, across items, or over different forms. Methods include test-retest, alternative forms, split-half, coefficient alpha, and inter-rater.

Front 29

Reliability Coefficient

Back 29

Interpreted directly as a measure of true score variability. .80 means that 80% of the variability is due to true score variability.

Front 30

Split-half Reliability

Back 30

Assessing reliability by splitting the test in half and correlating the scores on two halves.

Front 31

Spearman-Brown Formula

Back 31

Correction to split-half coefficient. which estimates what the test's reliability would be if it were based on the full length of a test.

Front 32

Standard Error of Estimate

Back 32

An index of error when predicting criterion scores from predictor scores. Used to construct a confidence interval. Depends on the criterion's standard deviation and the predictor's validity coefficient.

Front 33

Standard Error of Measurement

Back 33

An index of measurement error. Used to construct a confidence interval around an examinees obtained test score. depends on test's standard deviation and reliability coefficient.

Front 34

Increase Reliability

Back 34

Increase test length, increase heterogeneity of sample, increase the range of obtained scores.

Front 35

Test-retest Reliability

Back 35

A method for assessing reliability that involves administering the same test to the same group of examinees on two different occasions and correlating the two sets of scores. Yields a coefficient of stability.