This write-up focusses on three concepts of sound assessment practice namely reliability, feasibility and educational impact.
Reliability
Reliability refers to the ‘reproducibility’, ‘repeatability’ or ‘consistency’ of the scores obtained from an assessment (Memon et al, 2010; van der Vleuten et al, 2005). Reliability is generally expressed as a coefficient ranging from 0 (no reliability) to 1 (perfect reliability). Since it is difficult to achieve perfect reliability in any assessment, a reliability coefficient of 0.80 is regarded as the minimal acceptable value. However, the value could be lower or higher depending on the stake of the examination, with a higher stake examination such as licencing examination requiring a higher reliability …show more content…
The most studied pattern of restriction of range is when raters avoid both extremes of the scale and tend to rate everyone as average. This special case of restriction-of-range effect is called central tendency error (Linn & Gronlund, 2000). Restriction of range can cause two undesirable results. First, it casts doubt on the meaning of ratings. A high or low rating might reflect the individual rating preference of a rater rather than the actual performance of students. Second, it limits the variability of ratings, thus failing to provide reliable discrimination between competent and incompetent students (Anastasi, 1988; Linn & Gronlund, 2000). There may be an overlap between these rating errors. For example, a rater who restricts his or her ratings to only the high end of the score range will not give low ratings to examinees who perform poorly and will seem to be applying a more lenient standard to examinees with poor performance, thus exhibiting rating
Reliability
Reliability refers to the ‘reproducibility’, ‘repeatability’ or ‘consistency’ of the scores obtained from an assessment (Memon et al, 2010; van der Vleuten et al, 2005). Reliability is generally expressed as a coefficient ranging from 0 (no reliability) to 1 (perfect reliability). Since it is difficult to achieve perfect reliability in any assessment, a reliability coefficient of 0.80 is regarded as the minimal acceptable value. However, the value could be lower or higher depending on the stake of the examination, with a higher stake examination such as licencing examination requiring a higher reliability …show more content…
The most studied pattern of restriction of range is when raters avoid both extremes of the scale and tend to rate everyone as average. This special case of restriction-of-range effect is called central tendency error (Linn & Gronlund, 2000). Restriction of range can cause two undesirable results. First, it casts doubt on the meaning of ratings. A high or low rating might reflect the individual rating preference of a rater rather than the actual performance of students. Second, it limits the variability of ratings, thus failing to provide reliable discrimination between competent and incompetent students (Anastasi, 1988; Linn & Gronlund, 2000). There may be an overlap between these rating errors. For example, a rater who restricts his or her ratings to only the high end of the score range will not give low ratings to examinees who perform poorly and will seem to be applying a more lenient standard to examinees with poor performance, thus exhibiting rating