Chapter 7 - Reliability And Precision

Front 1

What is reliability

Back 1

Measurement concerning consistency, dependability and reproducibility

Front 2

What does an unreliable score imply

Back 2

An unreliable score implies that the scores are affected by sources of error or measurement error.

Front 3

Is it possible for a test to have total and absolute reliability

Back 3

No. There is always some level of inconsistency aka internal inconsistency.

Front 4

What part of a test does reliability measure

Back 4

Reliability is only concerned with the scores. Not the test itself

Front 5

What is measurement error

Back 5

Measurement error is the degree to which there is variance in scores that is related to the measurement process to things that are not relevant to what is being measured

Front 6

What does more measurement error mean for reliability

Back 6

More meausrement error makes for less reliability so they are negatively correlated

Front 7

What is a true score

Back 7

100% pure and accurate representation of the client's skill or abilities in a test. This would occur if there were no errors. It is impossible to achieve.

Front 8

What is the observed score

Back 8

The observed score is what we actually get from a client after testing. It is a measurement of their true score plus the measurement errors.

Front 9

What is the standard error of measurement SEM

Back 9

SEM is the measurement of how much the observes score differs from a true score

Front 10

What is a time sampling error

Back 10

Time sampling error is seen with repeating the same test over time with a client.

Front 11

How is intelligence tests affects by time sampling error

Back 11

Time sampling error doesn't affect intelligence that much because intelligence doesn't change much over time

Front 12

What kinds of test constructs are more susceptible to time sampling error

Back 12

Tests that measure mood or achievement are susceptible to time sampling error as they are likely to change over time.

Front 13

What is the carry over effect

Back 13

Carry over effect is a time sampling error when there isn't enough time between tests (normally a day or 2). The first sessions experience can effect the scores in the 2nd sessions scoring.

Front 14

What is the practice effect

Back 14

The practice effect is when we are measuring something that can increase over time with practice (i.e. ability in math scores). This is seen when testing sessions are taken too closely together.

Front 15

What is learning or maturation in regards to time sampling error

Back 15

This happens when test taking sessions are too far apart and a client may change or grow as a person or respond to therapeutic interventions and cause change in their scores which affects reliability.

Front 16

What is content sampling error

Back 16

When test items or the content of the test are not good representations of the construct of the test.

Front 17

What is the largest source of measurement error?

Back 17

Content sampling error

Front 18

What is interrater reliability error

Back 18

When 2 different raters are needed for their subjective rating of a client, there can be differences in perceptive of different aspects of a client

Front 19

What kind of quality should test items have

Back 19

They should be clear and unambiguous.

Front 20

How does test length relate to reliability

Back 20

The longer the test the more reliable the test. So more rest items the better

Front 21

What is a reliability coefficient

Back 21

Ratio of the true score variance to the observed score variance

Front 22

Does the reliability coefficient relate to a group of scores or an Individual score

Back 22

Reliability coefficient relates to a group of scores. Not an individual score.

Front 23

What is a perfect reliability coefficient and what does it mean

Back 23

+1.00 the reliability of the test is a perfect measure of the actual true score that could be derived from a client. the higher the reliability coefficient the higher the variance in scores across test takers is due to differences in test takers.

Front 24

what does a score of 0.35 in relability mean

Back 24

it means that 35% of the variation in scores is due to the real differences in test takers and 65% of the score is attitrube to random chance or error

Front 25

what is the test-retest measurement for measuring relaibility

Back 25

test and retest the same person on the same test at different times. the correlation given is called the coefficient of stability

Front 26

what ist he coefficient of stability

Back 26

the coefficient of stability is drawn from the coefficient of the test retest method. it reflects the stability fo the score over time.

Front 27

what sampling error is the test retest measuring for

Back 27

the test retest method is measuring for time sampling errors. this can be effected byt he carry over effect (time between is too short) or the learning/practice/maturation/therapeutic intervention effects (time in between testing is too long)

Front 28

wht kind of test would not work well with a test retest method

Back 28

a test retest method would not work well for a construct that is not stable. like mood or emotional state. things that are stable (like intelligence) test well with the test retest method

Front 29

what is the alternate form or parallel form relaiblity form?

Back 29

the alternative form or parallel form is when we test 2 equivalent forms of a test at the same time. like testing an alternative form of the ASI (i.e. ASI lite) against the normal ASI test. the 2 tests must have the same content domain/construct being tested.

Front 30

what are the 2 types of parallel/alternative form

Back 30

simultaneous and delayed form

Front 31

what is the simultaneous form of alternative reliability

Back 31

both tests are given simultaneously, at the same time on the same day. this reduces issues with memory and practice a sseen in test retest issues.

Front 32

what is the delayed administration of alternative/parallel forms

Back 32

the delayed administration is when we give the 2 equivalent tests at different times.

Front 33

what kind of sampling error is the alternative/parallell forms senseitive to?

Back 33

content sampling error.

Front 34

what kind of coefficient does alternative/paraellel forms give us when calculating for relaiblity

Back 34

this is called the coefficience of equivalence.

Front 35

what is the coefficient of equivalence

Back 35

it tells us how good 2 forms of a test measure a construct.

Front 36

what sampling error does delayed administration capable of detecting

Back 36

delayed administration can detect time sampling errors as well as content sampling errors.

Front 37

what is the limitation of alternate/parallel methods of looking for reliability

Back 37

the issue with alternative/parallel tests as a means of detecting reliability is that the many tests don't have alternate tests as they are expensive and difficult to construct. an equivalent test must be equivalent in content, difficulty, and several other factors.

Front 38

what is internal consistency reliability

Back 38

internal consistency reliability is about the relations of the items in a test.

Front 39

what kind of sampling error is internal consistency reliability concerned with

Back 39

internal consistency reliabilty is concenred with content sampling error

Front 40

how does the internal consistency reliability work

Back 40

it measures a single item in terms of it relates to the overall test score.

Front 41

what does a high internal consistency relability mean in terms of of a single item of the test

Back 41

a high internal consistency means that a single item is homogenous or has a high uniformity in measuring the construct in question

Front 42

what is a pro of using internal consistency reliability

Back 42

the use of intenral consistency relaiblity involves the application of a test only once to gain the calculation so this is cost and time effective.

Front 43

what is the split half relability

Back 43

the split half relaiblity is a form of measuring the internal consistency reliability of a test. it is done so by splitting a test in half and then correlating the scores to the other half of the test when taken.

Front 44

what coefficient is demonstrated by the split half reliability test

Back 44

in measuring for th intenral consistency of a test and one applies the split half reliability measurment, a test is tested for the coefficienty of equivalence. as seen in alternate/parallel form measurements.

Front 45

what is the spearman brown prophecy formula

Back 45

the spearman brown prophecy formula is used to measure for internal consistency reliability as seen in the split half testing method as a measurement of the entirity of the scores between the 2 halves.

Front 46

what is an advantage to using the split half method

Back 46

using the split half method makes for one single test administrationg to be needed to gather the evidence which saves time and money.

Front 47

what is a disadvantage of using the split half method

Back 47

the split half method is sensitive to content sampling error, but it is not sensitive to time sampling error.

Front 48

what is the kuder richardson formula

Back 48

they are aka KR 20 AND KR 21. they are seen in measruments of internal consistency on tests that involve answers that are only right or wrong (forced choice form).

Front 49

what is the coefficient alpha/cronbach's alpha?

Back 49

measurement of internal consistency on questions that don't have a right or wrong answer (selection of choice form). it can be used on likert scales.

Front 50

what is interrater reliability

Back 50

it is the degree to which 2 raters may agree.

Front 51

what kind of error does interrater reliability measure?

Back 51

it does NOT make a measure of content nor time sampling error. it only makes a measurement of the agreement or disagreement between raters.

Front 52

if a test was heterogenous due to measuring sevral concepts, what type of method would be best to detect reliability?

Back 52

the split half test would be best for working with tests that test for multiple constructs. unless you split the tests based on constructs, then any other measure of internal consistency reliability could work.

Front 53

if a test is considered what high stakes, what is the least amount of reliability coefficient that would be allowed

Back 53

a reliability coefficient of 0.90 would be the lowest acceptable point of reliability for a high stakes test.

Front 54

what is the acceptable reliability coefficient for an interrater agreement?

Back 54

0.90

Front 55

what is the standard error of measurement? SEM

Back 55

measurement of a test score for a single person if the test was taken ad nauseum

Front 56

what is a low or unacceptable relability coefficient

Back 56

0.59 or less (a F on a test)

Front 57

what is a moderate or acceptable amount for a reliability coefficient?

Back 57

0.60-0.69 (a D on a test)

Front 58

what is the acceptable amount for a reliability coefficient

Back 58

0.70-0.79 (a C on a test)

Front 59

what is considered a high amount of reliabilty coefficient

Back 59

0.80-0.89 (a B on a test)

Front 60

what is considered a very high amount of reliability coefficient?

Back 60

0.90 or more (an A on a test)

Front 61

what is error distribution

Back 61

since the standard error of measurment of a single score shows a single observed score as a measurement of the standard deviation from the true score, we can see that that a group of observed scores would cluster near to the true score with a mean of zero and an SD that would be that of the SD.

Front 62

what would reliability mean for the error distribution

Back 62

the more reliable the test then the more reliable or precise the observed score would be the true score, so less reliability would mean more error distribituion.

Front 63

what type of measrument does SEM help influence

Back 63

the standard error of measurement is a big influencer of the confidence intrval.

Front 64

what is a confidence interval

Back 64

we use this calculation to build interals around a single observed score to estimate the probability of where their true score would fall. this is the upper and lower limit of their true score

Front 65

what is the 1 SD in terms of the confidence interval

Back 65

1 standard deviation (-/+) is a 68% confidence interval. so we are 68% sure their true score falls within one standard deviation. or a z score of 1.00

Front 66

what is 2 sd in terms of the confidence interval

Back 66

2 standard deviations represents 95% of a confidence interval. or a z score of 1.96

Front 67

what is 3(-/+) SD in regards to a confidence interval

Back 67

3 (-/+) SD makes for a 99.5% confidence interval. or a z score of 2.58.

Front 68

how does one increase the reliability/precision of a test x5.

Back 68

increasing the number of items on a test. using unambigous items that are clear. select response items and not constructed response tiems. a balance of difficulty. clear instructions.