Selection And Performance Mgmt

Front 1

Validity: GMA

Back 1

0.51

Schmidt & Hunter (1998)

Front 2

Validity: Work Sample

Back 2

0.54

Schmidt & Hunter (1998)

Front 3

Validity: SJT

Back 3

General job perf: .58;

Managerial perf: .67

Christian, Edwards, & Bradely (2010)

Front 4

Validity: AC

Back 4

0.37

Schmidt & Hunter (1998)

Front 5

Validity: Biodata

Back 5

0.37

Hunter & Hunter (1984)

Front 6

Validity: Interview

Back 6

Structured: .44;

Unstructured: .33

McDaniel, Whetzel, Schmidt, & Mauer (1994)

Front 7

Validity: Personality

Back 7

Conscientiousness: .24;

Emotional Stability: .17

Barrick, Mount, & Judge (2001)

Front 8

Validity: Integrity

Back 8

Overall: .15;

Counterproductive behav: -.32

Van Iddekingee, Roth, Raymark, & Odle-Dusseau (2012)

Front 9

Validity: CSE

Back 9

Self-eteem: .26;

self-efficacy: .45;

Locus of control: .32;

emotional stability: .19

Judge & Bono (2001)

Front 10

What three pieces of evidence can an organization bring forward to answer a disparate impact charge?

Back 10

a. Business necessity
b. BFOQ
c. Validity evidence

Front 11

Stock and flow statistics: What are they and what are they used for?  Can you pick out the warning signs for adverse impact by seeing these metrics? 

Back 11

Stock statistics: compare the percentages of specific internal and external demographic groups of workers at one point in time. (uses relevant labor market (RLM) data.

 

Flow statistics: determine how minority members fared in the selection process in comparison to nonminority members. (use for AI)

Front 12

Burden of proof—how does it work?

Back 12

Disperate Tx:

Plaintiff must provide the following: 

1. Applicant is a member of a protected class
2. Applicant applied and was qualified for a job for which the company was seeking applicants
3. Despite these qualifications, applicant was rejected
4. After this rejection, the position remaiend open and the employer continued to seek applicants from persons with the applicant's applicants

Adverse Impact:

Employers

1. Business necessity (for safety of applicant)
2. BFOQ (justifiable reason why person can't be hired)
3. validity (job relatedness of selection procedure)

Front 13

Disability—how is it defined?

Back 13

Someone who has (a) physical or mental impairment that substantially limits one or more major life activities, (b) has a record of such impairment, or is regarded as having such an impairment.

Front 14

What is the evidence that must be brought forward to show prima facie for disparate treatment?

Back 14

1. Applicant is member of protected class
2. Applicant applied for and was qualified for a job for which the company was seeking applicants
3. Despite these qualifications, the applicant was rejcted
4. After the applicant was rejected, the position remained open and the employer continued to seek applicants from persons with the applicant's qualfications

Front 15

Where do inference leaps in HR occur?

Back 15

1. Job -> specification of job tasks (Sanchez & Levine, 2012) [job crafting,carelessness of raters, high v low performers, job complexity]
2. Job tasks -> KSAOs (Sanchez & Levine; Campion)
3. KSAOs -> selection measures/procedures (construct validity; Shcmidt & Hunter)
4. Job tasks -> Performance eval
5. Selection measures -> performance eval (criterion validity)

 

 

Front 16

Contrast CTT and GT and how they apply to JA and AC

Back 16

• Classical test theory (CTT): Observed score is a function of true standing on the construct plus error.
• Generalizability theory (GT): Breaks up the residual variance into multiple parts, allowing us to partition that variance in ways that suit us (accounting for systematic variance in settings, types of items, judges, etc.).
• Application: GT allows us to partition variance to see what factors contribute to error in AC and JA/selection practices

Front 17

 

 

1. What is meant by the follwing terms:
2. Domain sampling model

1. Criterion domain
2. Construct domain
3. Construct irrelevant variance

Back 17

Domain sampling model: a measurement tool is created by representative sampling from an infinite number of items that together fully represent the construct domain.

Criterion domain: DV

Construct domain: IV

Construct irrelevant variance: error

Front 18

Shrout and Fliess--what is the big picture, conceptually? In what situations would be want to generalize from one or two raters to a pool of similar raters?

Back 18

ICC: Ratio of true score variance among individuals to the sum of true variance plus random error variance

 

ICC tells us the extent the judgements measured are representative of across other judges. We want to generalized when the facets are random.  For example, if only two of your rater are available. 

Front 19

How do you construct a CI around someone's test score?

Back 19

CI = predicted score +/- 1.96(SEE)

OR

CI = observed score +/- 1.96(SEM)

Front 20

Clinical versus mechanical judgment (Highhouse; Meehl):

1. When does intuition serve us well versus not so well?
2. Difference between NDM (Gary Klein) and HR domains when predicting decision accuracy.

Back 20

1) Intuition is not good usually; overestimate accuracy, use irrelevant info, but, can be useful when looking for unique composites or 'broken-leg"

2) ? NEED TO ANSWER

Front 21

Systematic variance

Back 21

Systematic differences; effect of the manipulation of the IV

Front 22

Unsystematic variance

Back 22

Variance that is not systematic (can result from the "expert" judges rather than stats)

Front 23

Error variance

Back 23

Error in scores due to extraneous variables. 

Front 24

Common factor variance

Back 24

BLANK; NEEDS TO BE ANSWERED

Front 25

specific variance

Back 25

BLANK; NEEDS TO BE ANSWERED

Front 26

What combinations of variance would equate to reliability?

Back 26

perfect reliability = 1-proportion of error variance/total variance

Front 27

What are differences between the following and when would you need to use them?

a) standard deviation

b) standard error of the mean

c) standard error of measurement

d) standard error of the estimate

Back 27

a) standard deviation: measure of variance of the scores. Average deviation from mean in sample. 

b) standard error of the mean: SD of the means of all samples that were randomly extracted from a population of units. Give measure of sampling error. 

c) standard error of measurement: SD of the distribution of an individual's scores around his/her true score.

d) standard error of the estimate: SD of the distribution of residuals that result from a simple or multiple regression analysis. 

Front 28

Reliability's relationship with validity (attenuation formula)

Back 28

Front 29

What is reliability conceptually? What are the four types?

Back 29

1. Reliability is a measure of consistency
   1. Across raters
   2. Across time
   3. Across forms
   4. Across items in a measure

Front 30

Inferential leaps in validity and JA: where and why do they occur?

Back 30

1) Work info -> human attributes

2) Human attributes -> selection procedure

3) work related info -> performance measures

4) selection procedures -> performance measures (validity)

Front 31

MTMM--how and why is it used? What is the primary critique of the method and how is it resolved?

Back 31

 Use:

convergent validity = mono-trait, hetero-method 

divergent/discriminant validity = heterotrait, monomethod

Critique:

Front 32

When should you conduct a content validity study? Critierion? Concurrent versus predictive?

Back 32

Content validity: does selection test adequately represents domain sample

Criterion validity: does DV accurately measure domain of outcome?

Concurrent vs. predictive:

Front 33

Validity generalization, synthetic validity, validity specificity. What is the difference and when would you use each one?

Back 33

Validity generalization: is test validity generalizable from one context to another similar?

Synthetic validity:  (job component) infer validity for a job by IDing major functions then choosing tests or other predictors based on research.

Validity specificity: validity is specific to the job or org in which the measure was validated

Front 34

Utility analysis--why is it used? SDy rule of thumb (approximately 40% of mean salary for the job)

Back 34

Shows the degree to which use of a selection measure improves the quality of individuals selected versus the quality of invididuals selected if the measure had not been used. (Cascio; Schmidt). Expected $ payoff due to increased productivity annually.

Front 35

Job evaluation:

a) why is it done?

b) what are some ways it's typically done right now in corporations?

c) does it serve its purposes?

Back 35

Purpose: to demonstrate relative worht of jobs in an org. Results in job structure.

How it's done:

• Point

+ most common and most rigorous

- can become beureaucratic and rule-bound

• Ranking:

+ fast, easy, initially least expensive, easy to explain

- can be subjective and/or misleading (bias not called out)

• Classification:

+  can group a wide range of work together in one system

- descriptions may leave too much room for manipulation

Front 36

What pay/reward structures are typically used in orgs. What are strengths and weaknesses of each? (Lawler & Jenkins, 1992)

Back 36

Skill-based Systems:

• can get expensive; invest in training; how to adjust when skill is no longer relevant

Pay for individual performance:

• Merit pay ( - little relationship btween pay and performance)
• Incentive pay ( - divides workforce, adversarial) 

Pay for org performance:

• Gainsharing
• Profit sharing (form of gainsharing but doesn't have participative mgmt component and focuses on more than just employee driven results; more widely practiced)

Front 37

Reconciling the evidence on the effectiveness or gut instincts. When does it serve us well? When does it not? WHy?

Back 37

Generally, intuition and experts are not as good as stats. Experts overstimate, rely on too few pieces of info.

Experts can:

1. Spot composits/combos of skills
2. broken-leg syndrome

(Highhouse; Meehl)

Front 38

CM vs TJA: similarities, differences, and pros and cons of each.

Back 38

CM: development focused;can be org specific

TJA: task focused; selection; standardized

Front 39

How do you test for AI? (Bobko & Roth, AI book Ch 2)

Back 39

4/5ths rule (practical significance)

Statistical significance

 

 

Front 40

Comprehensive HR strategies to avoid AI

Back 40

Note: these recs are from multiple sources in AI book

a) remove performance irrelevant variance

b) recruit quality minority applicants

c) provide extensive employee development programs

d) define critieria broadly (Murphy)

e) Aguinis software tool for cut scores

f) Tippins recs

Front 41

Describe the Brunswick Lens Model and apply to personnel/performance appraisal processes.

Back 41

Relaity vs perception

cues and cue utlization impact connection between the two and accuracy. 

1) people are good at noting exceptions to the rule and usually give them more weight in the decision than is warranted. 

2) People are better at detecting simple, linear relationships, than nonlinear ones or interactions among two or more cues. 

Front 42

Strategies for decision making and pros and cons of each

Back 42

1) Top down

Pro: valid if there is a linear correlation between predictor and job performance

Con: susceptible to AI

2) Cut scores

Pro: minimize/narrow applicant pool to subset of minimally selective group; easy to explain

Con: can be expensive/resource intensive; all applicants must go through all procedures. No clearcut way to order applicants because they either pass or fail. 

3) Multiple hurdles

Pro: same benefits of cutoff but with reduced cost

Con: establishing validity for each predictor because of range restriction resulting; increased time needed for implementation

4) Fixed banding

5) Sliding banding

Pros for banding: employer has more flexibility in making decisions; select within band; allows employers to take into account factors that are not taking into account in traditional selection systems. 

Cons for banding: legal debate; may not reduce AI by a lot; may lead to loss of economic utility. Expert judgement + stats is problematic. SEM if high makes bands too wide. 

 

Front 43

How do you check for test bias?

a) differing intercepts? reasons why? does it always mean test bias? 

b) differing slopes: why does this happen?

c) statisically, how can we test for differing intercepts and slopes?

Back 43

1. Regress criterion on predictor; gives you main effect for predictor predicting criterion (Y regressed on X); if not significant, look for group differences, but you know you have a problem; the predictor is not predicting performance across groups.
2. Main effects for groups:  don't want this to be significant, if it is, it means different intercepts (making dummy codes). If significant; go through list (4; bias in ratings, reliability, etc).
3. Investigate interaction between group and tests; this is looking for slopes; do this regardless of whether the main effects are significant.Use ALTMMR to check for heteroscedacity; don't want interactions because this would mean the groups interact differently witht eh test.

Front 44

Conceptually, what does unfair discrimination mean? (Guion)

Back 44

Those who have an equal probability of meeting the job performance standard have an unequal probability of being hired.

Front 45

At which points could Type II errors be committed in various HR activities (e.g., range restriction, unreliability, unequal subgropu sizes with MMR for testing for test bias, etc.) How does shared contaminiation create conditions for Type I errors?

Back 45

BLANK; NEEDS TO BE ANSWERED

Front 46

Banding:

1. Why is it used?
2. How do courts view it?
3. Pros and Cons--
4. Logical issues (Schmidt)
5. SEM issue

Back 46

BLANK; NEEDS TO BE ANSWERED

Front 47

What is the efficacy for cognitive ability predicting performance? (Schmidt & Hunter, Hunter & Hunter; Murphy)

Back 47

.51 (Schmidt & Hunter)

Front 48

What is the systemic, historical picture of g and subgroup differnces?

Back 48

First v. second generation models: 

• first doesn't include any antecedents for g other than genetics. second includes SES, environement, etc. 

Psychometric perspective v. debates

• Pscyhom = spearman=race differences are result of intelligence diff
• debate: priv and mutlivariate models explain subgroup differnces

Front 49

Why do we see subgroup differences in g? (3-4)

Back 49

• Entrenched tasks; with more exposure to test content the better we do. Unentrenched tasks even the playing field
• Spearman/Jensen: true race diff exist
• Performance irrelevant race-related variance (Rpriv)
• Second gen model (SES)

Front 50

What are the construct vlaidity issues around g?

Back 50

Spearman/Jensen and psychometric perspective suggest variability in race on intelligence is due to actual differences between groups in intelligence. An alternate explanation (Goldstein et al) suggests that construct validity may be the true culprit. 

Front 51

How has g been conceptualized?

Back 51

1. Cattell: Crystalized versus fluid
2. Working memory (Kyllonen, 1996).
3. Multiple Intelligences (i.e., interpersonal, verbal, math, kinesthetic, etc.; Gardner).
4. Goal-adaptive behavior (Sternberg).
5. Analytical, creative, and practical abilities (Sternberg)

Front 52

Theory of AI (Outtz & Newman). Why does it occur?

Back 52

BLANK; NEEDS TO BE ANSWERED

Front 53

What is the Spearman Hypothesis?

Back 53

Variance in race groups is not error, but corersponds to actual differences. 

Front 54

What is the positive manifold?

Back 54

Suggests that g exists as is evident by the correlation of all tests measuring intelligence. 

 

*Understand the idea that simply observing a Positive Manifold in absence of theoretical justification is insufficient for inferring with a degree of certainty that a latent variable (g) exists

Front 55

According to Campion et al., (1997), what makes an interview structured?

Back 55

Structured = extent to which it meets below criteria 

1. Based on job analysis 

2. Same questions are asked to each candidate 

3. (a) Follow-up questions, (b) requests for elaboration, or (c) prompting of candidate after his/her response are non-existent or limited. 

4.  Most questions are either: (a) situational, (b) past behavior, or (c) background focused or some combination of the three. 

5. For validity reasons (avoiding deficiency) and reliability reasons, more questions are better than less (15-20 minimum is a suggestion). 

6. Interviewer limits his/her exposure to pre-interview info. on candidate’s resume, application forms, etc.  If access to this information is allowed, all interviewers need to look at same data in a standardized way. 

7. Each response on a question should be rated and these summed. Questions can be rated on multiple dimensions (e.g., content conveyed, communication style, clarity, specificity). 

8. Use anchored rating scales that are behaviorally detailed, if possible (Description, evaluation, % of candidates who answered in a similar way). 

9. Interviewers take detailed notes. 

10. Multiple interviewers are employed. 

11. Same individuals interview all candidates. 

12. Interviewers do not discuss their impressions of candidates between interviews. 

13. Interviewers are provided extensive training. 

14. Use statistical, not clinical/subjective judgment in aggregating candidate responses.

Front 56

What is the distinction between construct and method and why is this important?

Back 56

BLANK; NEEDS TO BE ANSWERED

Front 57

Does personality predict performance?

Back 57

Depends on what element of personality and how performance is defined. 

 

1. Conscientiousness .24 

2. Emotional stability .17 

3. Barrick, Mount, & Judge (2001) 

Front 58

What are some issues and challenges with personality testing?

Back 58

1. Issues and challenges with personality testing. (Barrett) 

   
   

   1. MA's can provide overly optimistic results 

   2. Aggregate too many jobs and contexts 

   3. No one universally  used measure facets described with similar names (e.g., conscientiousness); lack of agreement between scales and facets 

Front 59

What are some critiques of personality testing?

Back 59

BLANK; NEEDS TO BE ANSWERED

Front 60

What's Hough & Barrett's perspective on personality testing?

Back 60

Hough & Oswald 

1. Predictive validity can be improved by combining g and personality; personality has incremental increase in predictive validity; must be aware of which measures you're using, the extent to which they are correlated and what the mean differences are.  

2. Context matters 

   
   

   1. G is a lower predictor on tests of low complexity than on tasks of high complexity 

3. Mono-method bias? Personality I self-report; mutli-trait multi-method would be better

Front 61

How do AC's work?

Back 61

BLANK; NEEDS TO BE ANSWERED

Front 62

Do AC's predict job performance?

Back 62

validity = .37 (Schmidt & Hunter, 1998)

Front 63

What is the argument between Lance and Arthur re: AC's?

Back 63

1. Lance: AC's work, but performance is not cross-situational. Tett & Burnett Trait Activation Theory (TAT) suggest that situations will bring out different traits. 

2. Arthur: dimensions need to be constructed with the rigor that would be given to other psychometric instruments 

Front 64

AC's: transparency of dimensions--pros and cons.

Back 64

ransparency of dimensions: pros and cons 

1. Pros:  

   
   

   1. Lessens anxiety 

   2. Levels playing field if all know dimensions 

2. Cons:  

   
   

   1. Doesn't increase construct validity 

   2. Lowers predictive validity 

   3. People may try to fake now that they know what you are testing for.  

Front 65

Job performance/criterion problem

 

Back 65

1. Job performance/’Criterion problem’: (Cleveland & Colella; Campbell, 2012). 

   
   

   1. Criterion deficiency - what have we ignored? 

   
   

   1. Distinctions between performance and effectiveness. 

      
      

      1. Performance includes (Sales Numbers, Customer Service Scores, etc.; Criterion relevance) AND other factors (criterion contamination) we are measuring that are not included in the "ultimate" or conceptual criterion.  

      2. While effectiveness would include Criterion Relevance factors as well as things we may have ignored (Criterion deficiency) 

Front 66

Alignment between business strategy and performance management.

Back 66

lignment between business strategy and performance management (Schiemann) 

1. Holism 

   
   

   1. Performance management systems are not isolated systems, but should be highly integrated into the philosophy, values, and systems of the organization.  

2. Role Modeling 

   
   

   1. Process should be driven by the top team's example. 

3. Cultures that evoke self-accountability 

   
   

   1. Orgs with great performance management systems also tend to have high expectations of their managers and manage those systems in a disciplined way 

4. Don't overcomplicate 

   
   

   1. Not necessarily simple, but avoid unnecessary complexity. 

Front 67

What are some cultural considerations in performance management. (hint: know Hofstede dimensions)

Back 67

1. Hofstede dimensions: 

   
   

   1. Performance orientation 

   2. Future Orientation 

   3. Gender Egalitarianism 

   4. Assertiveness 

   5. Individual vs Collectivism 

   6. Power distance 

   7. Human orientation 

   8. Uncertainty avoidance  

Front 68

What's Hattrup & Roberts' argument about the AI validity tradeoff?

Back 68

Hatrup and Roberts would suggest that AI and validity are actually outcomes that exist at different levels of analysis.  AI and diversity-related factors are group level measures whereas the discussion of validity as it relates to selection is related to individual-level performance, and not group.  Therefore, to consider both as outcomes at a similar level is flawed logic.  It is a (some would argue unrealistic) leap to claim that the aggregate of individual task perf. completely maps onto performance at the org level. 

Front 69

What are some ways to encourage contextual performance in an organization? (Reilly 7 Aronson)

Back 69

1. Antecedents 

   
   

   1. Personality 

   2. National Culture 

   3. Org Culture 

   4. Leadership 

2. Management Tools and Methods 

   
   

   1. Performance Appraisal 

   2. Recognizing and Rewarding 

   3. Feedback 

   4. Career Development 

   5. Legal Considerations 

3. Key Behaviors  

   
   

   1. Interpersonal Facilitation 

   2. Organization Support 

   3. Conscientiousness Initiative 

Front 70

What are some ways to encourage employees ot accept performance feedback in healthy ways? ( Diamante, Sessa, et al)

Back 70

1. Perf MGMT as learning tool 

2. Employee value is created/sculpted by an org and the tools used are imperfect; authentic social exchange; performance negotation (focus on business outcomes)

Front 71

What are some of the disadvantages to a univariate model of job performance? (Murphy)

Back 71

Multivariate allows for more variance to accounted for and more predictors to included (can also include AI as a social outcome)

Front 72

What are some practical suggestions for reducing AI in a selection test? (Schmidtt and Quinn)

Back 72

1. remove biased (construct -contaminated items)
2. weight dimensions of job performance differntly, depending on what hte company values.  The more task perfomrance is weighted, the higher the expected AI, typically.
3. use alternative methods to disentangle construct v. method variance.
4. Reduce stereotype threat (Claudia Steele)
5. Coaching: providing optional test prep sessions
6. remove or extend time

Front 73

Typical (will do) versus maximum (can do) performance.  What's the difference?

Back 73

Typical = will do

Maximum = can do

Increased variability is negatively related to compensation (Barnes & _____)

Front 74

What are the issues with MA corrections and how do they apply to practice?

Back 74

Overestimation of effect sizes provides more optimistic picture

Front 75

What are sources of bias in performance ratings? (Landy; McKay)

Back 75

• race
• gender
• age
• disability

Front 76

What are some ways to predict and manage counterproductive performance behaviors? (Atwater & Elkins)

Back 76

1. ndividual contributors:  

   
   

   1. Examples: 

      
      

      1. Substance abuse  

      2. Family problems (divorce, ill parents, ill child, financial problems) 

      3. Employee personality or psychology issues 

   2. Employers should be careful when diagnosing personal issues because of ADA, state legislation, common law.  Rather than addressing suspicions regarding personal issues, supervisors should document and discuss specific undesirable behaviors.  

2. Job Context Contributors 

   
   

   1. Poor Interpersonal relationships 

      
      

      1. Relationships with supervisors 

      2. Relationships with co-workers 

   2. Feelings of Injustice 

      
      

      1. Distributive justice: perceptions of fairness regarding outcomes employees receive such as promotions, pay, and other types of rewards.  According to equity theory, outcomes perceived as fair, comparing one's own inputs and outcomes relative to others, can contribute to motivation and performance, if viewed as inequitable, can be demotivating. Important to note that equity is a perception and may not be objectively accurate.  

      2. Procedural Justice: perceived fairness associated with procedures used to determine outcomes.  

      3. Interactional justice: perceptions of how employees are treated by others. Two components: informational and interpersonal.  

   3. Job disatisfaction 

   4. Situational constraints 

   5. Organizational Climate

Front 77

What are some artifactual factors that influence performance ratings?

Back 77

BLANK; NEEDS TO BE ANSWERED

Front 78

What are some performance appraisal methods? (Aguinis)

1. Comparative vs. absolute systems
2. Pros and cons of various sources

• supervisors
• peers
• subordinates
• Self

Back 78

BLANK; NEEDS TO BE ANSWERED

Front 79

What are some pros and cons associated with various sources of ratings:

Back 79

Supervisors

Peers

Subordinates

Self

Front 80

What are some rater motivations that are linked to rating inaccuraties, and how can they be mitigated?

Back 80

BLANK; NEEDS TO BE ANSWERED

Front 81

1. Within-person performance variability (a.k.a., dynamic criterion):
   1. Individuals with inconsistent performance are rated poorly (Barnes & Morgeson, 2007).
   2. Understand why Classic Test Theory (CTT) assumptions do always not hold up in the performance domain.
   3. Approximately 70% of variance in performance was within-person [Stewart & Nandkeolyar, (2007)].

Back 81

BLANKS; NEEDS TO BE ANSWERED

Front 82

Criticisms of JE (Lawler & Jenkins)

Back 82

1) psychometric properties

2) unintended consequences ("that's not my job")

3) System congruence

 

Front 83

What factors affect statistical conclusion validity when conducting stat anslyses in the context of selection?

Back 83

1) reliability of criterion and predictor

2) Violation of statistical assumptions

3) Range restriction

4) Criterion contamination