Psychology Of The Workplace - Predictors

Front 1

Predictor

Back 1

Any variable used to forecast a criterion

Front 2

In I/O Psychology, what two psychometric criteria are used to measure the quality of predictors?

Back 2

Reliability & validity

Front 3

Test-retest reliability

Back 3

A type of reliability that reveals the stability of test scores upon repeated applications of the test

Front 4

Equivalent (parallel) form reliability

Back 4

A type of reliability that reveals the equivalence of test scores between two versions or forms of the test (e.g. taking the GRE first you get one version & when you take it 2nd time you get another version)

Front 5

Internal-consistency reliability (Cronbach's alpha reliability)

Back 5

A type of reliability that reveals the homogeneity of the items comprising a test

Front 6

Inter-rater reliability

Back 6

A type of reliability that reveals the extent of agreement among the assessments of two or more raters

Front 7

Construct validity

Back 7

The degree to which a test is an accurate and faithful measure of the construct it purports to measure

Front 8

Criterion-related validity

Back 8

The degree to which a test forecasts or is statistically related to a criterion

Front 9

Validity coefficient

Back 9

A statistical index (correlation coefficient) that reveals the degree of association between 2 variables). Often used in the context of prediction

Front 10

Content validity

Back 10

The extent to which the content of a test includes or represents all of the content of a particular construct, e.g. organizational commitment.

Front 11

Face validity

Back 11

The perception that test items are appropriate for the intended use of the test by the individuals who take the test

Front 12

A tool can be reliable but not valid. Give an example.

Back 12

A bathroom scale which consistently states that your weight is 70lbs (when it's actually 120), i.e. it's measuring something consistently but not accurately.

Front 13

What is one disadvantage of test-retest reliability measures?

Back 13

Some things you want to change over time e.g. children's learning.

Front 14

Disadvantage of parallel (equivalent) form tests.

Back 14

Expensive and time-consuming to develop.

Front 15

What is split-half reliability?

Back 15

A method of measuring reliability where all items that claim to measure the same construct are randomly divided into two sets. The entire instrument is given to a sample of people and total scores are calculated for each randomly divided half. The split-half reliability estimate is the correlation between these two total scores. (Example: Correlating odd numbered items with even numbered items on same test, e.g. OCQ items 1,3,5 should be highly correlated with items 2,4, 6)

Front 16

What is alpha reliability?

Back 16

A type of reliability that reveals how similar each item is on a test. Comparing responses to each item with responses to every other item on the same test should result in a high correlation. Each item in a 100-item test is considered 100 minitests. the response to each item is correlated with the response to every other item. A matrix of inter-item correlations is formed whose average is related to the homogeneity of the test. If the test is homogeneous (the item content is similar), it will have a high internal consistency reliability coefficient.

Front 17

What are two types of criterion-related validity?

Back 17

Predictive (eg. give all job applicants a selection test, hire everybody, then measure their job performance). Correlate scores on test with job performance (expect high correlation b/w those who did well on the test & performance). Good in theory, not done in practice.
Concurrent: give test to current employees & correlate their scores on test with their performance & see correlation. (Disadvantage: restriction of range).

Front 18

Give an example of content validity in a selection test.

Back 18

The job of clerical assistant involves typing, filing, taking messages. If selection test only tested typing speed, it would not have content validity.

Front 19

How would you test whether an IQ test has convergent (construct) validity?

Back 19

Give people the IQ test and see if it correlates with things past research says it should, e.g. GPA. If so, it has convergent validity.

Front 20

How would you test whether an IQ test has divergent (construct) validity?

Back 20

Give people the IQ test. Results would NOT correlate with things you would expect, e.g. IQ scores and sociability. If you find low correlation, test has divergent validity.

Front 21

Give 3 examples of mental capacity tests commonly given in work situations and what is the average validity correlation coefficient?

Back 21

Otis Self-Administering Test of Mental Abilities
Wornderlic Personnel Test
Adaptability Test
Validity correlation coefficient: 0.25 (acceptable)

Front 22

Give 2 examples of mechanical aptitude tests & validity correlation coefficient.

Back 22

Bennett Test of Mechanical Comprehension
Minnesota Spatial Relations Test
0.25 - 0.30

Front 23

List some types of physical ability tests (don't need names) & their validity correlation coefficients.

Back 23

Sensory ability (visual acuity, color vision, hearing sensitivity)
Motor ability (e.g. fine or gross motor coordination)
0.20 - 0.25

Front 24

Give an example of a personality / interest inventory and how it works.

Back 24

Holland's Self-Directed Search. 'The party' - imagine groups are divided into different groups, e.g. realists, enterprisers, etc. You choose which one you want to join. Occupation finder tells you what occupation would be ideal based on which groups you picked.

Front 25

List the Big Five personality traits of the Five Factor Model

Back 25

Openness
Conscientiousness
Extroverted-introverted
Agreeableness
Neuroticism

Front 26

Give some examples of honesty and integrity tests and how do they work?

Back 26

Polygraph
Clear Purpose or Overt Integrity Test
Personality based measures
They directly ask people about theft

Front 27

What is the Polygraph Protection Act (1988)?

Back 27

Polygraph Protection Act (1988) restricted the use of polygraph in selection testing, but there are exceptions eg security services, working with currency or radioactive materials, manufacturing, distribution or dispensing of controlled substances, Federal, State and Govt employees – all these people are not protected by the Act.

Front 28

What is the rationale behind paper and pencil honesty and integrity tests?

Back 28

The typical employee who steals is more tempted to steal, rationalize theft, punish thieves less, think about theft-related activity and attribute more theft to others. (e.g. a question on an integrity test might be how often do others steal and a high response would probably indicate the person steals).

A lot of questions involve whistleblower behavior (because thieves show more inter-thief loyalty).

Front 29

What is the rationale behind personality-based honesty tests? Give an example of one.

Back 29

Employee Reliability Index. Personality-based measures try to weed out ‘bad eggs’, people who are more likely to vandalize or sabotage things, abuse drugs in the workplace or be guilty of subordination. They would measure conscientiousness in particular.

Front 30

What is a problem with honest/integrity tests?

Back 30

40-60% of people will fail, but this doesn't mean they are not honest. Also, job applicants can get irate when if they find out they have failed such a test.

Front 31

Discuss the problem with graphology (handwriting analysis).

Back 31

Rafaeli & Klimoski conducted a research study and found graphologists no better than untrained assessors.

Front 32

What is the difference between a 'general' and a 'specific' job interview?

Back 32

General interviews vague, interviewers come up with own questions.
Specific i/v -i/viewer has list of specific questions - same for all applicants

Front 33

Discuss reliability of general (unstructured) interviews

Back 33

Intra-interviewer reliability is high
Inter-interviewer reliability is low

Front 34

Give some examples of how validity is affected in general interviews.

Back 34

Interviewers place more weight on negative than positive information.
Primacy effect – interviewers make decisions at beginning of interview
Recency effect – the last few minutes of an interview are important too
Primacy effect and recency effect can relate to both within the interview itself (i.e. first and last minutes of interview) as well as between applicants (so first and last applicants are remembered rather than the ones in the middle)
Contrast effects – interviewers tend to compare each applicant with another applicant (‘that one was better than that one’) but don’t compare with a general standard or average (so this can affect validity negatively)
Stereotypes: everybody has stereotypes. If interviewers aren’t given clear directions and they might rely on stereotypes which might well be inaccurate.

Front 35

Give an example of a specific type of structured interview.

Back 35

Situational interview: an applicant is asked what they would do in a certain situation.

Front 36

Describe work samples and their validity.

Back 36

Involves giving somebody a portion of the job and get them do it. E.g. if you have an auto-mechanic coming in looking for a job, get them to fix a car. Typing test for clerical workers.
High validity (0.40)
High face validity

Front 37

Describe 2 types of situational exercises

Back 37

In-basket test: One type of situational exercise is an ‘in basket’ test. The job applicant / manger would be given a desk with messages, emails, phone messages, projects and has to pretend he has to fly out in 3 hours and you have to solve the issues in that period of time. Measured whether they prioritized, did the make the right decisions in terms of what to handle now and put off later, did they delegate appropriately?
In-basket tests are also used for development

2. Leaderless group discussion – might give a group of people applying for a job a problem or issue to work out and you look to see who would become the leader, to what extent people were listening to each other, what interpersonal skills are like.

Validity will vary depending on type of job – range from .15 to .35 on situational exercises.

Front 38

Disadvantage of work samples?

Back 38

They don't measure potential very well, only what the applicant currently knows.

Front 39

What is the validity of situational exercises?

Back 39

0.15 to 0.35

Front 40

Describe biographical data and the rationale behind it

Back 40

Based on the idea that the best predictor of future behavior is past behavior.
Bio data involves asking questions about applicant's life situation, life experiences. Asking questions like did you like science in high school, your parents’ occupations, did you ever have a paper route?

Front 41

How are biodata tests developed?

Back 41

To come up with a bio data item is you would take your current employees, give them questions, ask them about their childhood, hobbies, etc. Then divide your current employees into high and low performers and see if any attributes distinguishes them, e.g. Attending science fairs might be applicable to one particular aspect of the job.

Front 42

Advantages and disadvantages of biodata.

Back 42

High validity (.77 to .79)
Disadv: too empirical/theoretical
unstable validity over time (attending science fair might be good for one job but not another)
Adverse impact (paper routes might be more relevant to boys than girls)
low face validity (drives applicants crazy)
Low generalizability (can't generalize to other jobs)

Front 43

Discuss reliability / validity of letters of recommendation

Back 43

Letters of recommendation are always positive so they have high reliability but low validity. Very common in grad school but part of it is that they want to make it difficult to apply (track people down to write letters).
Companies use them for reference checking.

Front 44

Two types of approaches to drug testing. What are they?

Back 44

Drug education
Drug testing (urine, hair, blood). Paper & pencil drug tests rarely used

Front 45

Problems with urine drug testing

Back 45

Expensive
Oversensitive threshold levels
Caution needed when communicating positive result

Front 46

What 4 standards should apply to predictors?

Back 46

Generalizability
Inexpensive
Validity
Fairness

Front 47

Summarize the validity of different types of job predictors

Back 47

biodata and work samples (high validity)
intelligence tests (moderate)
mechanical aptitude & spatial relations tests (low to moderate)
personality tests (low to moderate)
interview (low to moderate, depending on amount of structure)
recommendations (low)

Front 48

Predictive criterion-related validity

Back 48

The measurement tool's ability to predict something we theorize that it should be able to predict. For instance, we might theorize that GRE scores should predict how well a person will do in a grad program. We could give our measure to grad students and see if there is a high correlation between scores on the GRE and their achievements in grad school. A high correlation would provide evidence for predictive validity -- it would show that our measure can correctly predict something that we theoretically think it should be able to predict.

Front 49

Concurrent criterion-related validity

Back 49

A test's ability to show that your test is valid by comparing it with an already valid test.
Example: A new test of intelligence would have concurrent validity if it had a high positive correlation with the Wechsler Adult Intelligence Scale since the Wechsler is an accepted measure of the construct of intelligence.

Front 50

What is Cronbach's alpha reliability?

Back 50

A measure of how well each individual item in a scale correlates with the sum of the remaining items. It measures consistency among individual items in a scale.

Front 51

What does a selection ratio of .75 mean?

Back 51

Company will hire three out of every four applicants.