9-12

Front 1

test bias

Back 1

different groups can score differently on a test- is it because the test is bias, or are certain groups actually different?

Front 2

BITCH and Chitling - alternative special test

Back 2

while these tests can detect differences between groups, their psychometric properties have not been able to predict useful, real-world outcomes, such as job and academic performance.

Front 3

intercept bias (scenario two)

Back 3

when the slope of the regression line is the same for each group, but they intercept the vertical axis at different places. (the test is biased but correctable).

Front 4

scenario one - group A do better than B on the selection test, this means that

Back 4

either test is not biased, or the performance measure and the test score are equivalently biased.

Front 5

scenario 3 (slope bias)

Back 5

the slope of the regression lines for the groups are different - the test is differentially valid for the two groups. The scatterplot shows more variability for group 2- the test is biased and is not predicative of performance of them.

Front 6

counter arguments to race discrimination and testing

Back 6

1. pygmalion effect

2. minority group members still face many disadvantages, especially where group membership is superficially obvious (black children raised in white families still tend to perform worse at school)

3. construct of race is primarily social and has no biological meaning, especially in Aus and US.

Front 7

Pygmalion experimental design: 1966

Back 7

children given a non-verbal IQ test, teachers were told the child was either spurting or blooming. Teachers were given a list of the children who performed in 20% in this test. In reality children on this list were chosen at random, all children tested again at the end of one year.

Front 8

pygmalion effect

Back 8

school achievement is influenced by teacher expectation. For the earliest grades, students who were identified as the bloomers scored significantly higher at the end of the year.

Front 9

Stereotype affects test performance GRA

Back 9

Group 1 told test measured IQ, group 2 told test measured problem solving skills. African Americans did worse in group 1, but not in group 2 (where they did not differ).

Front 10

Stereotype affects test performance

Back 10

when asian women were primed about their racial identity before a math exam, they performed better than controls. When women primed about their gender before a math exam they did worse than controls.

Front 11

disability discrimination act Australia

Back 11

all tests must measure person for the requirements of the job, and not the person in the abstract. Tests should assess the suitability of an applicant for that specific position, based on selection criteria (information such as person's private life or personality should not be used when making decisions about applicant).

Front 12

Australian Industrial Relations Commission vs. Coms21 1999

Back 12

Coms21 hired recruitment consulting to make decisions about a structural re-organisation. Drake provided 5 individual personality profiles/ psychometric reports before firing the people.

Front 13

fair work commission

Back 13

handles workplace discrimination issues

Front 14

law and psychological testing

Back 14

rulings are inconsistent and commonplace,

Front 15

test utility

Back 15

practical usefulness of a test (usually in terms of financial cost/benefit) even if a test is high in validity and reliability, (a test does not have high utility if it costs 1,000,000 to administer) doesn't mean it is practical to use.

Front 16

example of a test with poor reliability and validity but good test utility

Back 16

when test score is less important, e.g., lie detectors, useful if participants think they work. But don't use tests low in reliability and validity to interpret a test score.

Front 17

utility analysis (statistical decision analysis), helps answer two questions:

Back 17

different techniques used to decide the usefulness of a test/s:

1. which test shall we use?

2. is adding a new test to an existing battery of tests worthwhile?

Front 18

expectancy table, utility analysis

Back 18

1. false positive: test incorrectly identifies person as good when they are not

2. false negative: test incorrectly identifies a person as being no good when they are good.

3. selection ratio: ratio between available job positions and number of applicants

4. cut off: minimum test score needed in the test to be hired

Front 19

utility analysis: if selection test had low criterion validity:

Back 19

test is no better than selection applicants at random

Front 20

limitations to expectancy tables:

Back 20

assumes a linear relationship between job performance and test score, and doesn't take into account other characteristics of the applicant: physical health, minority status

Front 21

when to apply Signal detection theory

Back 21

need to apply SDT whenever you have a task that involves discriminating between two stimuli (such as the recognition memory task)

Front 22

four possible outcomes

Back 22

correct hit

false positive

correct miss

false negative

Front 23

sensitivity and response bias

Back 23

ability to discriminate between stimuli, and criterion for saying yes - if you just looking at sensitivity then your results are confounded with response bias

Front 24

how to disentangle sensitivity from response bias?

Back 24

look at false positives as well as correct hits (i.e., hit rate - false positive rate)

Front 25

d' (d prime)

Back 25

1. The closer d' is to 1, the better the person is at discriminating between stimuli,

2. If it is 0 then the person is guessing

3. If it is negative, the person is recognising words they didn't see, and not recognising words they did see.

Front 26

industrial inspection, response bias or sensitivity issue?

Back 26

decrease in hit rate was due to response bias

Front 27

what strategies have been put in place that detect people cheating on the hazard perception test by clicking all over the screen?

Back 27

1. allowing the measurement of false positives

2. stricter definitions of what a hazard is

3. using a modified task: hazard change detection

4.

Front 28

Item response theory

Back 28

superior alternative to classical test theory, it's score is called Theta: a function of the examinee's response interacting with the characteristics of the items.

Front 29

characteristic curve

Back 29

a plot of ability (level on some trait) vs. the probability of getting a particular question right (item-difficulty index)

Front 30

the three parameter model

Back 30

1. item difficulty: level of ability needed to get the item right 50%

2. item discrimination: point where slope is steepest

3. level of guessing

Front 31

advantages of IRT (4 advantages)

Back 31

1. no need to calculate an overall total score on a test to estimate someone's ability level

2. people don't need to complete the entire test to get an idea of their level of ability

3. much better for computerised adaptive testing (no need to give people items that they will definitely get wrong)

4. can compare people even when they completed different parts of the same test

Front 32

disadvantages of IRT

Back 32

complex to understand

software is still in its infancy

requires large samples to get stable estimates

requires more assumptions than classical test theory

Front 33

why is the accuracy of diagnostic tests often overestimated?

Back 33

people forget to consider the base rate (likelihood of the disease occurring in the population)

Front 34

why do many diagnostic tests adopt a liberal response bias?

Back 34

to maximise the number of correct diagnoses, better to minimise false negatives even if that results in an increase of false positives. just because a test has a high number of correct diagnoses doesn't mean it's any good at diagnosing (have to consider false positives)

Front 35

ROC curve (receiver operating characteristic, developed in WWII)

Back 35

a plot of correct positive rate (sensitivity) versus false positive rate (1 - specificity).

Front 36

the pass mark on the ROC curve that gives the most accurate classification is the:

Back 36

point on curve when the sum of sensitivity and specificity is highest.

Front 37

Thompson effect

Back 37

a visual illusion: an image reduced in contrast looks like it is moving slower

Front 38

cactaracts affect 50% of people over the age of 75, how much more likely are they to crash?

Back 38

2.5 times more

Front 39

method of constant stimuli

Back 39

give people a bunch of trials and vary each scene in what you are testing (e.g., speed) show these scenes in a jumbled order and get people to make judgements on these scenes that come in pairs.

Front 40

in psychometric functions what does a steep line tell us?

Back 40

the better people can tell apart the speeds.

Front 41

what does the point where the line crosses the .5 mark tell us?

Back 41

tells us if there is any systematic speed bias

Front 42

utility analysis, an example.

Back 42

new test had higher validity (.76) than the old non-test procedures (0-.5) and saved the company around 6 million dollars a year in 1979.

Front 43

what has signal detection theory analysis found in regards to jury decision making?

Back 43

that the sort of instructions that are given to jurors regarding the definition of 'reasonable doubt' affects their response bias (willingness to convict) rather than their sensitivity (ability to distinguish guilty from innocent defendants).

Front 44

what is the main point of item characteristic curves?

Back 44

to see how each individual item interacts with level of ability

Front 45

computerised adaptive testing

Back 45

select items to administer based on what the participant previously got right.

Front 46

what can we do about the Thompson effect in terms of driving safety?

Back 46

1. remove cataracts faster

2. increase contrast in road environment

3. give drivers perceptual training to improve their ability to tell different speeds apart