Reading...
Play button
Play button
Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
34 Cards in this Set
- Front
- Back
|
What does reliability mean?
|
means consistency
|
|
|
What are some of the reasons for error that compromise reliability?
|
Person's conditions such as mood, fatigue, distraction
|
|
|
What is a reliable score?
|
Reliable scores are produced by tests that are free from errors of measurement.
|
|
|
What is the most important characteristic of any measurement?
|
Validity
|
|
|
What does i mean when someone makes reference to reliability of a test?
|
Reference is to reliability of SCORES derived from the test.
|
|
|
What is the relationship between reliability and the criterion related validity scores?
|
Criterion related validity coefficient cannot exceed the square root of their reliability. This predetermines a ceiling for the validity of the scores,
|
|
|
What is the meaning of the Pearson product-moment correlation coefficient ?
|
Indicates the direction and the strength of a linear relationship between two variables measured on interval or ratio scales.
|
|
|
What factors affect the size of the Pearson's r?
|
The sample size, range of X and Y scores in the sample, and the nature of the relationship between X and Y.
|
|
|
What are possible reasons for no correlation between scores?
|
The presence of a curvilinear relationship
Restricted range of X and Y values 2 groups in one sample, one with a positive Pearson’s r, and one with a negative. |
|
|
In classical test theory, what is the formula for test scores?
|
X= score client received on the testing instrument, T= client’s true score. E= random error
X=T + E |
|
|
What is the true score of the measure? (T)
|
The true score (T) is a hypothetical concept, consisting of the average of the distribution of scores that would be observed in repeated independent measurements of a person with the same test.
|
|
|
What does Random error ( E) effect?
|
Random error affects reliability,
|
|
|
What are some of the sources of random error?
|
-Fluctuations in participant mood, fatigue, illness or recent experience.
-Variation in measurement conditions, such as noise, or inconsistency in administration. -Differences in scoring such as scoring errors, subjectivity or clerical errors. -Random guessing |
|
|
What is systematic error and what does it effect?
|
remains constant and leads to consistency and not inconsistency. Does NOT affect the reliability of the scores.
|
|
|
What is the Classical Definition of Reliability?
|
the reliability of measurements (rxx) indicates what proportion of the observed score variance is true score variance.
|
|
|
What is the formula for reliability?
|
Rxx=TSV/TSV + EV
→where TSV=True Score Variance and EV=Error Variance Perfect reliability is Rxx=1.00, and can technically occur when TSV=Observed Score and EV= 0. |
|
|
What is the Standard Error of Measurement?
|
The standard deviation of the normal distribution of observed scores a person may obtain under repeated independent testing with the same test
|
|
|
What is the percentage of scores that fall between 2 standard deviations of the mean?
|
95%-confident that a person's true score falls between 2 standard deviations of the mean.
|
|
|
How does one construct a confidence interval at 95%?
|
Formula :X-2(SEM)<T<X +2 (SEM) is a 95% Confidence interval
Example: X (observed Score)=23 SEM=2 So: X-2(SEM) to X +2(SEM) 23-2(2) to 23+2(2) We are 95% confident that the person’s true score lies between 19 and 27 on a given test, when the observed score was 23 and the SEM was 2. |
|
|
What is the relationship between a smaller SEM and the accuracy of the measurement?
|
A smaller SEM will produce smaller confidence intervals for a person's true score, therefore, smaller SEM= more accuracy of the measurement.
|
|
|
Which is more helpful when reporting client scores, the SEM or the reliability coefficients?
|
The SEM because it relates directly to the meaning of the test’s scale of measurement.
|
|
|
What is the problem with interpreting an observed score at the 68% confidence level?
|
The 68% confidence level is only +/- 1 SEM, which means one in three scores true scores will fall out of the suggested range resulting in a mistake rate of 32%
|
|
|
What are the methods for assessing reliability?
|
a. Coefficient of internal Consistency
b. Test-re-test c. Alternate Forms |
|
|
What is Internal Consistency?
|
estimates are based on the average correlation between items within a test or scale. Advantage of internal consistency is that participants need to receive only one administration of a single test on a single occasion.
|
|
|
What are the Methods of Determining Internal Consistency?
|
Split Half
Matched Random Subsets Cronbach’s alpha: Test-Retest Alternate Forms |
|
|
What is the problem with using split half for determining reliability and what formula could be used to increase reliability of split half?
|
Halving the number of test items substantially lowers the correlation as the greater number of items, the higher the correlation.
The Spearman Brown Prophecy Formula- predicts the internal consistency of the items if returned to the original size. |
|
|
On which types of measurements should one use Cronbach's alpha to assess internal consistency?
|
Multi-scaled item formats (e.g. Likert Scale)
|
|
|
When should Kuder Richardson 20 be used to assess internal consistency?
|
when test components are dischotomously scored,
|
|
|
When is it most appropriate to use test-re-test reliability and what is the coefficient used?
|
MOST appropriate for measurements of traits that are stable over time
Coefficient of stability-correlation between same person taking same test at different times. |
|
|
What is Alternate Forms Reliability and what is the coefficient used?
|
two equivalently designed tests that have very similar observed score means, variances, and correlations.
Coefficient of equivalence |
|
|
What do criterion referenced measures show?
|
How the examinees stand with respect to an external criterion.
|
|
|
When are criterion referenced measures used?
|
Tests in the classrooms
When a counselor wants to know whether a client has "enough" of a mental disorder to warrant a diagnosis. |
|
|
When are classification consistency reliability used?
|
With criterion measured tests used for classification of mastery vs. non-mastery on a criterion. Either by the same test on two occasions or two different tests.
|
|
|
What are the two indices of classification consistency reliability and which is more conservative?
|
Po=observed proportion of persons consistently classified as mastery vs. nonmastery. Cohens K=proportion of non-random consistent classifications.
Po can happen by chance. so Cohen K is more conservative. |
