Item Writing
When developing questions for a low, medium or high stakes examination an important consideration is that of legal defensibility. In particular how consistently did the questions measure a students response and did it measure what it was supposed to.
In order to achieve this, its important to understand the structure and approach that should be followed. The first task is to access the course objectives which allows extraction of the key concepts and learning points by topic area. From this, meaningful and valid test questions can be developed which focus on Blooms taxonomy on higher levels of cognition.
Bloom’s taxonomy is based around three hierarchical models which classify learning objectives into levels of complexity and …show more content…
The other levels are more commonly used in the low, medium and high stakes examination questions.
To write multiple choice items requires a structure, where a problem statement known as a stem is constructed. The stem needs to be written based on a definite problem that applies focus on specific learning outcomes. This would be followed by the development of a list of alternative answers or solutions. One of the alternatives would be correct and the others incorrect known as distractors.
Multiple choice test items have a number of advantages. One of these is versatility, where the test item can assess different levels of learning outcome from basic recall, analysis through to evaluation. The other is to do with legal defensibility.
Legal defensibility requires a test to be reliable and valid. Multiple choice items are not prone to true or false question testing and can be written to test specific aspects of a learning outcome. This significantly enhances test reliability. From the validity perspective it is improved with the use of multiple choice items. Mainly because they cover the broader aspects of a course rather than an essay type …show more content…
When developing items, a team of subject matter experts comprising content developers, instructors, technical support and technology design staff should be enlisted to review and assess all items against agreed criteria including setting the passing or cut score. This enables the items to be checked for validity, fairness and reliability to ensure legal defensibility criteria is satisfied.
The Modified Angoff method is a popular approach used to determine passing scores for an examination. It is based on the judgement of a panel of subject experts as mentioned above, and their view on what a minimally qualified practitioner should be able to answer question wise.
Typically the subject matter experts consider how many practitioners in a typical cross section of 100 would be likely to answer the questions correctly. This would be followed by using standard deviation calculations on a question by question basis to determine a pass score. This would be further ratified by running a beta examination to a selected audience of 100 participants. With the results being used to normalise the passing score before being
When developing questions for a low, medium or high stakes examination an important consideration is that of legal defensibility. In particular how consistently did the questions measure a students response and did it measure what it was supposed to.
In order to achieve this, its important to understand the structure and approach that should be followed. The first task is to access the course objectives which allows extraction of the key concepts and learning points by topic area. From this, meaningful and valid test questions can be developed which focus on Blooms taxonomy on higher levels of cognition.
Bloom’s taxonomy is based around three hierarchical models which classify learning objectives into levels of complexity and …show more content…
The other levels are more commonly used in the low, medium and high stakes examination questions.
To write multiple choice items requires a structure, where a problem statement known as a stem is constructed. The stem needs to be written based on a definite problem that applies focus on specific learning outcomes. This would be followed by the development of a list of alternative answers or solutions. One of the alternatives would be correct and the others incorrect known as distractors.
Multiple choice test items have a number of advantages. One of these is versatility, where the test item can assess different levels of learning outcome from basic recall, analysis through to evaluation. The other is to do with legal defensibility.
Legal defensibility requires a test to be reliable and valid. Multiple choice items are not prone to true or false question testing and can be written to test specific aspects of a learning outcome. This significantly enhances test reliability. From the validity perspective it is improved with the use of multiple choice items. Mainly because they cover the broader aspects of a course rather than an essay type …show more content…
When developing items, a team of subject matter experts comprising content developers, instructors, technical support and technology design staff should be enlisted to review and assess all items against agreed criteria including setting the passing or cut score. This enables the items to be checked for validity, fairness and reliability to ensure legal defensibility criteria is satisfied.
The Modified Angoff method is a popular approach used to determine passing scores for an examination. It is based on the judgement of a panel of subject experts as mentioned above, and their view on what a minimally qualified practitioner should be able to answer question wise.
Typically the subject matter experts consider how many practitioners in a typical cross section of 100 would be likely to answer the questions correctly. This would be followed by using standard deviation calculations on a question by question basis to determine a pass score. This would be further ratified by running a beta examination to a selected audience of 100 participants. With the results being used to normalise the passing score before being