Psych Testing Chapter 4

Front 1

What characteristic deems a test relaible?

Back 1

If the test is reasonably free from measurement error, it can be considered this.

Front 2

How can we find a persons true score, or a score without error?

Back 2

We cannot find this score. Classical test theory only assumed that a person has this.

Front 3

(True Score)- (Observed score) = ?

Back 3

The difference between these two scores leads to measurement error

Front 4

According to basic sampling theory, what shape does the distribution of random error take?

Back 4

A normal distribution/bell curve. The centre represents the true score, the distribution about the true score represents the error.

Front 5

How can we estimate a persons true score?

Back 5

we can estimate someones true score by finding the mean of their scores after repeated testing.

Front 6

Why do we use a standard error of measurement?

Back 6

because it is assumed that all people will have the same distribution of random error. The basic rate of error is represented by the standard deviation of error.

Front 7

What is the domain sampling model?

Back 7

this model considers problems that are caused when you use a limited number of items to

represent a larger and complicated construct.

Front 8

What happens to reliability as a sample becomes larger according to the domain sampling model?

Back 8

The reliability increases when sample sizes become larger. The entire domain is better represented.

Front 9

Describe Item Response Theory

Back 9

This testing strategy uses computers, which evaluate a persons skill level after each question, and adjusts the questions accordingly. This creates a more reliable estimate of ability.

Front 10

Which three methods are commonly used to test for reliability?

Back 10

test retest, parallel forms, and internal consistency

Front 11

Describe the test-retest method. Under what conditions is it useful?

Back 11

this method of measuring reliability is used to evaluate the error associated with administering a test at two different times.It can only be used when we are measuring a trait or characteristic that is constant.

Front 12

What are two reasons that test scores might change after a period of time?

Back 12

1) there is a change in the true score

2) measurement error

Front 13

What is the carry over effect?

Back 13

this effect takes place when the first testing session influences the scores from the second testing session. This is only an issue if the changes are random and unpredictable.

Front 14

What are practice effects?

Back 14

this is a type of carry over effect, where the test takers skills have improved with practice and their scores increase.

Front 15

What is parallel forms reliability?

Back 15

this involves comparing two tests that measure the same attribute. They use different items, but the rules used to select the items are the same.

Front 16

What is the Split Half Method

Back 16

this method involves randomly splitting a test into 2 halves that are scored separately, and the scored are compared to each other. The reliability of each half would be less reliable than the whole test. To correct for this we can use the spearman brown formula

Front 17

What is the Kuder-Richardson 20?

Back 17

This method is used in measuring the reliability of a test in which the items are dichotomous. (right or wrong, yes or no etc)

Front 18

Coefficient alpha

Back 18

this is a general way to estimate reliability, similar to the KR20, but can be used when items are not dichotomous

Front 19

How is a difference score obtained, and how do you compare difference scores?

Back 19

these scores are created by subtracting one test score from another. They are compared in standardized units or Z scores

Front 20

Why are behavioural observation methods typically unreliable?

Back 20

These are unreliable because of discrepancies between the true scores, and the scored recorded by the observer.

Front 21

What is interrater/interscorer/interobserver/inderjudge reliability ?

Back 21

this reliability considers the consistency among different judges

Front 22

What does the Kappa statistic measure?

Back 22

this is a method used for assessing the level of agreement among several observers. It includes correction for chance agreement. Uses a scale of 1 to -1

Front 23

What is standard error of measurement? What are they used for.

Back 23

this allows us to estimate the degree to which a test provides inaccurate readings. The larger the SEM, the less accurate the measurement. A small SEM tells us that an individual score is close to the measured value They are used to create confidence intervals around specific observed scores.

Front 24

What is considered a good reliability estimate for tests?

Back 24

It depends on the test, typically though, between .70 and .80 is good enough for most purposes. In clinical settings, higher ratings are needed (.90 and up)

Front 25

What are 3 things we can do if reliability is low?

Back 25

1) increase the number of test items, because longer tests are more reliable.

2) Remove items that reduce reliability by performing factor analysis or discriminability analysis - examine the correlation between each item and the total score

3) Correction for attenuation - estimating what the correlation between 2 items would have been if they had not been measured with error.