Tests And Appraisals In Counseling Midterm: Chapter 5

Front 1

Describe why you want reliability in a test (and a man) :)

Back 1

Test scores (or behavior) is dependable, consistent, and stable across items.

It's the *results* we are looking at (just like a man's behavior) not the test itself.

It always refers to a *specific type* of reliability (time, raters, items, etc)

Remember: None are totally consistent or error free. All are subject to some degree of error and fluctuation.

Front 2

Any fluctuations in scores that results from factors related to the measurement process that are irrelevant to what is being measured is

Back 2

Measurement error.

Front 3

True Score + Measurement Error =

Back 3

Observed score.

Front 4

This is associated with the fluctuation in test scores obtained from repeated testing of the same individual.

Back 4

Time-sampling error

Front 5

This is when the first testing session influences the scores on the second session

Back 5

Carryover effect

Front 6

this term is used to label the error that results from selecting test items that inadequately cover the content area that the test is supposed to evaluate.

Back 6

Content-sampling error

Front 7

this considers differences in scorers as a potential source of error

Back 7

Interrater differences

Front 8

List and explain 4 "other sources of error"

Back 8

Quality of test items (not vague and ambiguous)

Test length (the greater the number of items, the greater the reliability)

Test-Taker Variables (motivation, fatigue, illness, physical discomfort, mood, etc.)

Test Administration (room temp, admin instructions, lighting, noise, etc)

Front 9

List 4 methods of estimating reliability

Back 9

Test-Retest
Alternate Forms
Internal Consistency
Interrater

Front 10

This is when the same test is given twice with time interval between testings

Back 10

Test-Retest

Front 11

What source of error is associated with the test-retest model of estimating reliability?

Back 11

Time-sampling error

Front 12

This is when equivalent tests are given at the same time in order to estimate reliability.

Back 12

Simultaneous Administration (main category of this method is Alternate Forms)

Front 13

What source of error could occur with the Simultaneous Administration method of estimating reliability?

Back 13

Content-sampling error

Front 14

This is when equivalent tests are given with a time interval between testings

Back 14

Delayed Administration (main cat = alternate forms)

Front 15

What sources of error are correlated with Delayed Administration?

Back 15

Time sampling and content sampling errors

Front 16

This is when one test is divided into two comparable halves, and both halves are given during one testing session

Back 16

Split half (main cat = internal consistency)

Front 17

What source of error is associated with the split-half method of estimating reliability?

Back 17

Content sampling.

Front 18

This is when one test is given at one time (items compared to other items or to the whole test)

Back 18

KR Formulas and Coefficient Alpha (Cat = Internal consistency)

Front 19

What source of error is associated with KR Formulas and Coefficient Alpha?

Back 19

Content Sampling

Front 20

This is when one test is given and two individuals independently score the test.

Back 20

Interrater

Front 21

What source of error is associated with the Interrater method of estimating reliability?

Back 21

Interrater differences

Front 22

This is an alternative to the split-half method of estimating internal consistency. It calculates the reliability of a test by using a single form and a single test administration, and without arbitrarily dividing the test into halves. Dichotomous items only (true false)

Back 22

Kuder-Richardson Formulas (KR-20 and KR-21)

Front 23

N/N-1(1-(sum of pq / variance)) =

Back 23

KR-20 formula

Front 24

This internal consistency method of estimating reliability is used when items on a test are *not* scored dichotomously (no right or wrong answers), like a rating scale.

Back 24

Coefficient Alpha

Front 25

This is the extent to which two or more raters agree. It assesses how consistently the raters implement the rating scale.

Back 25

Interrater Reliability

Front 26

A test that measures depressed mood assesses a single content domain and thus contains homogenous test items. In this case which reliability coefficient would be appropriate?

Back 26

KR 20 or Coefficient alpha because they assess internal consistency by correlation test items with each other.

Front 27

If a test instrument measured two separate constructs (like depressed mood and anxiety) the heterogeneity of the test items should be computed by this reliability coefficient:

Back 27

Split-half method. (Or homogenous parts of test could be subdivided into appropriate groupings and the KR 20 or coefficient alpha reliability estimates could be used for calculating each group.)

Front 28

Reliability coefficients of .70 or higher are considered

Back 28

acceptable

Front 29

Do coefficients relate to individuals or a group of scores?

Back 29

a group of scores.

Front 30

What measurement is used to estimate the amount of variation in test scores for a single test taker?

Back 30

Standard Error of Measurement (SEM)

Front 31

Do SEMs estimate the accuracy of true scores or observed scores?

Back 31

Observed scores.