Reading...
Play button
Play button
Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
50 Cards in this Set
- Front
- Back
|
Introduction to Performance Appraisal
Levy & Williams (2004). The social context of performance appraisal: A review and framework for the future. What is contexts role in PA? What are distal factors? |
-Context plays a huge role in the effectiveness of the performance appraisal process & how participants react to it
-Distal factors: effect the rater and rate behaviors (harder to research) ex. culture -Process proximal variables: process or structural definitions |
|
|
Introduction to Performance Appraisal
Levy & Williams (2004). The social context of performance appraisal: A review and framework for the future. What are some rater and ratee issues? |
-Rater issues: rater affect, similar personalities
> Rater purpose: administrative vs development > Raters motivated to be accurate? (low on conscientiousness), less accurate ratings > Rater leniency (high in agreeableness) > Attributions (if not internal, can affect rating) > Rater accountability: being held responsible for decisions (they must be able to justify ratings, increase accuracy & feedback) - Ratee issues > motivation: merit pay systems (few organizations link pay to performance & participation & justice) >Individual reactions Distributive justice: the allocation of rewards/outcomes Procedural: procedures followed to determine the rewards (employee voice) Interactional: how an employee is treated and how the supervisor communicates critical information |
|
|
Introduction to Performance Appraisal
Levy & Williams (2004). The social context of performance appraisal: A review and framework for the future. How do you achieve higher perceptions of fairness with the due process? What are some issues related to leader-member relationship, group dynamics, and structural proximal variables? What are the benefits of multi-source feedback? |
Due process: achieving higher perceptions of fairness
1. adequate notice 2. fair hearing: letting them have a voice, ask questions, comment 3. judgment based on evidence (diary keeping, trust) Leader-member relationship: trust Group dynamics - Politics in PA (like ability) - Impression management (rewarding and punishing subordinates) > giving supervisor ratings (assertive vs. defensive) - Supporting culture feedback Structural proximal variables: make up the organizations design of the PA process (ex. dimensions, frequency of appraisals) - multi-source feedback 1. each adding unique information 2. multiple ratings will add unique information about the target 3. lead to self-awareness and lead to behavioral change - frame of reference training |
|
|
Introduction to Performance Appraisal
Levy & Williams (2004). The social context of performance appraisal: A review and framework for the future. What are some issues with appraisal reactions? |
Appraisal reactions
- thought was important - motivation to use feedback > negative information presented in a constructive way > focus on tasks > goal settings > need to follow-up - acceptability: instrument, feedback process Evaluate performance as a team: hard to measure individual performance |
|
|
Introduction to Performance Appraisal
Murphy (2008). Explaining the weak relationship between job performance and ratings of job performance. What are some characteristics of objective vs subjective job performance ratings? What are some general suggestions to improving measures? |
Job performance ratings
Objective: low reliability, limited to a few jobs, criterion deficiency, and skew reward systems Subjective: ratings (relying on judgment as a rater) Improving measures - Rating scale changes - Identifying specific types of rating errors & removing them (training or statistical control) - Rater training - 360 evaluations - Forced -distribution systems - Group discussion & review systems |
|
|
Introduction to Performance Appraisal
Murphy (2008). Explaining the weak relationship between job performance and ratings of job performance. Describe the one-factor model, the mutli-factor model, and the mediate models of the job performance --> job performance ratings relationship. |
One-factor model
Performance --> Performance Ratings <-- Measurement Error * if measurement error disappeared, the relationship would be simple and direct * Disadvantages: 2 raters don't rate exact same performance, raters agreement cannot be directly linked to ratee performance and ratee disagreement to measurement error, & true scores problematic (not necessarily only based on performance) Multi-factor model Performance, System Characteristics (fairness), & Individual Characteristics (motivation, personality) --> Performance Ratings <-- Measurement Error * Added other non-performance related variables that affect performance ratings * Identify the non performance factors that influence ratings and take steps to reduce their influence Mediate models System Charaterisitcs, Performance, & Individual Characteristics --> Rater Goals & Intentions (distinct from ratee performance) --> Performance <-- Performance Ratings <-- Measurement Error * Rater goals help to understand if relationship will be weak or strong *Performance ratings are poor because raters are not willing to act as neutral instruments * Raters should have the tools, incentives to be accurate, & protection against negative consequences |
|
|
Introduction to Performance Appraisal
DeNisi & Sonesh (2011). The appraisal and management of performance at work. Why conduct appraisals? What would Ideal systems look like? |
Why conduct appraisals
1. Provide documentation for decision making 2. provide performance feedback 3. developing a bases for pay decisions (improve organizational performance) Ideal systems 1. employees told about area of improvement 2. given the appropriate tools 3. show how improved performance is related to rewards (FOR) training: directs raters to various performance dimensions that they should be familiarized with to create a common frame in observing and assessing behavior performance *Shifting focus from making the PA better to PERFORMANCE MANAGEMENT (make up performance improvement) - aligning goals, improve performance (strengths & weaknesses) - creating link of individual performance & organizational goals - goal setting & feedback (increases motivation), focus on task - benefits: management commitment & intrinsic motivation |
|
|
Introduction to Performance Appraisal
DeNisi & Sonesh (2011). The appraisal and management of performance at work. What is the difference between judgment and ratings? |
Judgment: private evaluation formed by a rater after processing all the available performance information
Rating: a public statement of the evaluation Reactions: trust, person rating needs knowledge of ratee & job, justice, employee voice, encouraging feedback, focus on task, timeliness & frequent, and rater training |
|
|
Defining and Measuring Performance
Chapter 5 |
Defining job performance
-Behavior or the results of behavior > Focusing on the results: lead to dysfunctional behavior > More complex than behavior (the person or the situation?) > Lead to ignore a wide range of behaviors - Job performance does not equal task performance > little time spent on specific tasks > PAs include non-relevant to task performance measures (absenteeism) > time constraints on task performance - Conceptual criteria: long & short term goals included Defining the dimensions of Job Performance - Job analysis (does not provide a total solution) - Job element method, critical incident methods - Dimensions for behaviors a) those contained in a formal job description b) those defined by the social context of work and organizations (OCBs) |
|
|
Defining and Measuring Performance
Chapter 5 What is ultimate criterion? What are performance dimensions? What are 3 important things when evaluating criterion? |
-Ultimate criterion/conceptual criterion (job performance as the total domain) & relevance to job performance
-Performance dimensions: specific criterion measures 3 things important when evaluating criterion 1. Criterion relevance: your criterion is what you want to get at with measure 2. Don't want: criterion deficiency: lacking some key of criterion 3. Don't want: criterion contamination: measuring things that are not relevant |
|
|
Defining and Measuring Performance
Chapter 5 Name things to consider about the criterion problem. |
Criterion problem
1. Where use composite vs. separate (multiple) criteria - do you average across job performance scores or use separately? - dimensionality: is it uni or multidimensional? > multidimensional task: the specific functions in job description contextual (OCBs): extra role behaviors, not in job description counterproductive: deviant behaviors --> going against the well-being of organizations 2. objective (non-judgmental) vs subjective (judgmental) 3. Typical (long-term; will do) vs. maximum (short-term; can-do) 4. Dynamic vs. Stable criteria -whether performance changes over time or stays stable Steel-Johnosn et al., (2000) suggest it is not stable over time |
|
|
Defining and Measuring Performance
Chapter 5 What are some reasons why performance might change over time? |
Reasons why performance might change over time
1. way to perform task changes 2. KSAOs you need for task might change 3. KSAOs of employees might change Cognitive abilities stronger for tasks (maximum) than OCBs (contextual) Typical --> personality traits better Cognitive ability - consistency: the more consistent and the less complex, cognitive ability is less of a predictor - need less with skill acquisition and work experience - organizational variables > technology changes, cognitive ability stable > any changes in work processes increases importance for cognitive ability > situational constraints: less predictor of cognitive ability |
|
|
Defining and Measuring Performance
Viswesvaran & Ones (2000). Perspectives on models of job performance. What are some different approaches to defining job performance? |
- Role theory & identity theory to describe job performance
- Task vs. contextual performance - Binning & Barret models > Job performance dimensions can be characterized as 1. stand-alone, specific, can't generalize 2. part of a larger set of dimensions > Developed for specific occupations 1. specific occupations 2. applicable across jobs |
|
|
Defining and Measuring Performance
Viswesvaran & Ones (2000). Perspectives on models of job performance. What are the three pieces to job performance? |
1. Task performance (tasks specified in job descriptions)
- One comprehensive taxonomy of job performance dimensions (homogeneous task cultures applicable across jobs) - 4 approaches: behavior description approach, behavior requirements approach, abilities approach, & task characteristics approach 2. Organizational Citizienship Behavior - dimensions (altruism, courtesty, cheerleading, sportsmanship, civic virtue, & concientuousness) - Organization spontaneity: recognized by reward systems (difference from OCBs) 3. Counter Productive Behaviors - Another dimension of job performance a. interpersonal/organizational b. serious/minor (property deviance, production deviance, personal aggression, and political deviance) Cambell (1990) general latent structure of job performance: job specific task proficiency, non-job specific task proficiency, written & oral communication, demonstrating effort, maintaining personal discipline, facilitating peer and team performance, supervision, & management or administration |
|
|
Defining and Measuring Performance
Viswesvaran & Ones (2000). Perspectives on models of job performance. What is the difference between criterion reference and norm reference? What are some job performance dimensions for entry level and manager positions? |
Criterion reference: have some standard in mind
Norm reference: comparing against a norm group (ranking) *Measurement errors: judgmental ratings vs. organizational outcomes - scale formats do not substantially alter the assessments *Individual differences to predict job performance: cognitive ability & conscientiousness Entry level & service: off-hand tasks, flexibility, industrious, adherence to confrontation rules Managers: leadership/supervision, interpersonal/communication, knowledge to management, useful behaviors & skills |
|
|
Defining and Measuring Performance
Borman, Bryant, & Dorio (2010). The measurement of task performance as criteria in selection research. Describe and give examples of objective vs subjective PAs. |
Objective criteria: can be contaminated & deficient, may only tap into small part of the criterion
- environments need to be held equal - ex. work samples (issues w/ criterion): take as ultimate criteria and are not rather maximum performance > time and equipment constraints - ex. job knowledge: when task is procedural > not applicable to a lot of jobs - ex. sales using production rates (limits the jobs, rates unstable, and environmental factors Subject criteria: performance ratings - BARS: examples of behaviors as anchors - Behavior summary scales: behavioral content of several individual anchors at leach level of effectiveness (behaviors summarized into high, mid, & low effectiveness) >underlying construct made more evident to rate -BOS: how frequently the ratee exhibits behavior - Computerized adaptive rating scales: trying to get at "true performance" > Paired-comparison rating task: presented with two behaviors, pick one, & computer presents two more (IRT) |
|
|
Defining and Measuring Performance
Borman, Bryant, & Dorio (2010). The measurement of task performance as criteria in selection research. Describe the two type of rater training for PA. |
Rater training
- Rater error training: make raters aware of their errors - Frame-of-reference training: performance is multidimensional & to familiarize them with the actual content of each dimension > reduces psychometric error > increases rating accuracy |
|
|
Defining and Measuring Performance
Borman, Bryant, & Dorio (2010). The measurement of task performance as criteria in selection research. |
Dimensionality of Job Performance
- Communicating and interacting - Productivity & proficiency - Useful personal qualities - Problem solving - Organizing & planning Predictors of Task performance - cognitive ability - spatial & perceptual/psychomotor ability - personality - vocational interests |
|
|
Defining and Measuring Performance
Sackett, Zedeck, & Folgi (1988). Relations between measures of typical and maximum job performance. |
Maximum (ability & motivation)
1. explicit awareness of one being evaluated 2. awareness of instructions to maximize effort 3. performance measured over a short enough time that individuals attention is focused on the maximum performance goal - highly ability loaded - more highly correlated with supervisory ratings because observing during "peak periods" Typical: not aware of being evaluated, monitored over extended period of time, & not strictly dichotomous - motivational factors play a role *These two are modestly correlated (.16 & .36) *Not interchangeable |
|
|
Purposes of Performance Appraisals
Chapter 4 Name some purposes/uses to conduct PA. What are the two most popular ones? |
Reasons/Purposes to conduct PAs
1. evaluative vs. developmental components 2. system maintenance uses - evaluating systems in general 3. documentation for legal issues and such 2 main purposes 1. administrative: promotion, salary, firing 2. development feedback: strengths/weaknesses, goal setting, no direct bearing on administrative decisions, must be specific and should be behaviorally oriented 4 Broad Uses of PA in organizations: 1. Between person (comparing people, admin) 2. Within person (more developmental) 3. systems maintenance (workforce planning) 4. documentation Purpose helps determine: who should be evaluated, the raters, how frequently conducted, rating format used, extent of power and politics involved - Performance appraisal purpose effect: ratings obtained for admin purposed tend to be more lenient and less accurate than those for research, feedback, or employee development |
|
|
Purposes of Performance Appraisals
Chapter 4 Name and describe the three relationships between rating and decision making. |
Relationships between rating & decision making
lead- do ratings then decide to fire/promote based on them lag- make decision to fire/promote then fill out ratings based on that lead-lag- make decision, then do ratings, but use ratings to make another decision like remedial training rather than firing |
|
|
Purposes of Performance Appraisals
Williams, DeNisi, Blencoe, & Cafferty (1985). The role of appraisal purposes: Effects of purpose on information acquisition and utilization. How can purpose influence appraisal decision? Research for what people search for given information or little information? - in related to attribution theory. |
- The purpose may influence how information is used to make appraisal decisions
> may cue raters to search for, weight, & combine information in a certain way & raters utilize different type of information (cognitive function) > deservedness (merit, based on individual- training) vs. designation (pick 1 or 2 employees for an outcome- compare to others) different motivational factors: designation has more personal characteristics Attributional theory: consistency, distinctiveness, & consensus information Results: -purpose of deficiencies/deservedness (training) received higher ratings than other purposes -low distinctive & high consistency = higher ratings -low consensus rated higher (that employee is performing higher than others) When have unclear info what will people use to make decision? - naive participants seek distinctiveness info - consensus for designation outcomes (merit, salary purpose) *Make sure managers have access to necessary info |
|
|
Purposes of Performance Appraisals
Cleveleand, Muprhy, & Williams (1989). Multiple uses of performance appraisal: Prevalence and correlates. Why should there be a distinction between using between and within formats? What were the results for the purposes of PA? |
Distinction between using PAs between individuals and within individuals
1. ratings may be different 2. accuracy differs 3. when using multiple purpose: de-emphasizing all others Results - information of PA had greatest impact on salary administration, performance feedback, and identification of strengths and weaknesses - more often used w/ between and w/i then maintenance or documentation - more likely to use between for admin decisions - more likely to use within for development |
|
|
Purposes of Performance Appraisals
Jawahar & Williams (1997). Where all the children are above average: the performance appraisal purpose effect. Why are admin ratings higher then development ratings? What is the PAP effect and describe situations where it occurs (i.e. sig moderators of the PAP effect)? |
- Ratings 1/3 SD larger for admin purposes than employee development purposes
- Why? > avoid giving negative feedback, avoid negative consequences of such, obtain positive consequences, or to motivate a poor performer - Raters are not correctly distinguishing between top and low performers Performance Appraisal Purpose inconsistent results PAP effect 1. Admin ratings more lenient than research ratings (PAP larger in field settings than lab settings) 2. PAP effect larger in organizational raters than student raters 3. PAP effect larger for observe ration of raters than paper people or videotapes 4. PAP effect larger for downward appraisal than upward 5. PAP effect was nearly the same when making perf ratings and when making personnel decisions 6. PAP effect larger for GRS than FCS or BARS 7. PAP effect slightly larger for multiple purposes than for single purposes (not sig.) 8. Weighted vs. unweighted: same conclusions except for paper vs. behavior observation Sig moderators: research setting, type of rater, type of appraisal stimulus, & direction/source of appraisal |
|
|
Purposes of Performance Appraisals
Boswell & Boudreau (2002). Separating the developmental and evaluative performance appraisal issues. |
Development: improving attitudes & effectiveness of employee
Evaluative: comparing performance to a set standard *Make evaluation responsibility of someone other than the supervisor *Conflict between PA uses in terms of rating processes & strategies --> may negatively influence employees' attitudinal reactions to the appraisal - less close to employee, more effective rater (however disconnect with further away supervisor) Results - sig difference between groups on reported future development > less motivated to develop because no clear connection between development and salary *development should come from supervise *evaluative should come from higher-up |
|
|
Scales, measures, and formats
Appendix Describe Graphic Rating Scales. |
*Format has minimal effect on the quality of ratings
*No format consistently better Graphic Rating Scales -Record judgment of performance on a scale that can be used to obtain numeric values that correspond with the rater's evaluation of the ratee -little structure for judgment -very good --> very poor, or brief description at each level -presenting rater with dimensions -adv: simplicity, most common -disv: lack of clarity & definition, little structure, no definition of dimension or ratings (ex. no definition for Time management or what is meant be "poor" or "average" |
|
|
Scales, measures, and formats
Appendix Describe BARS or BES. |
Behaviorally Anchored Rating Scales or BES
-Use behavioral examples of different levels of performance to define both the dimension being rated and the performance levels on the scale in clear, behavioral terms -adv: accepted by users, incorporates a lot of feedback in process in constructing scales, and feel like they (raters & ratees) have personal investment -disv: "more objective" then graphic, but not more accurate |
|
|
Scales, measures, and formats
Appendix Describe Mixed Standard Scales. |
Mixed Standard Scales
-Same: incorporate behavioral examples -Different: designed to measure two performance dimensions (response to questions & speaking style) -Items: 1 item describing good performance, 1 item describing average performance, & 1 item poor performance (& items for different traits) >comparing performance to every behavior described >mixed items -Scoring: algorithm --> overall numeric score of each dimension -adv: may simplify the rater's task, research does not support -disv: complexity of scoring > find it difficult to give meaningful feedback |
|
|
Scales, measures, and formats
Appendix Describe Behavior Observation Scales. |
Behavior Observation Scales
-Same items as MSS -Describe how frequently each behavior occurred over the period covered by appraisal -Removes subjectivity: can be subjective though b/c of cognitive processes -Behavioral orientation appears to be an advantage: however use overall, subjective evaluation to guide behavior ratings |
|
|
Scales, measures, and formats
Appendix Describe Performance Distribution Assessment. |
Performance Distribution Assessment
- BOS features: indicate frequency of outcomes that indicate specific level of performance on a given dimension -adv: allows rater to consider the distribution of the variability of performance as well as the average level of performance in forming an evaluation -complex scoring rules: relative effectiveness of performance, consistency of performance, & the frequency -global evaluations of performance than object behaviors -disv: not more specific or objective |
|
|
Scales, measures, and formats
Appendix Describe Management by Objectives. |
Management by Objectives
- Not a system rather a METHOD for defining goals, objectives, and priorities - Evaluative: ratee evaluated in terms of goals and objectives - Represents most common format for evaluating managerial performance - Danger: goals and objectives will be set that are a. easily quantified b. easily achieved c. not really central to the job > Simplifies process, but also worthless - Will work well with appropriate assigned goals and objectives |
|
|
Scales, measures, and formats
Appendix Describe Employee Comparison Methods. |
Employee Comparison Methods
- as supervisor's span of control increases, ranking all subordinates can become tedious and sometimes arbitrary 1. Full ranking: involves rating all employees 2. Forced-distribution scale: sort into ordered categories - Best two, worst two, all else fall in middle > use if admin rewards correspond to categories > categories is less than # of people > depends on specificity of information required > do NOT use for outcomes need for everyone 3. Pair-comparision method: allows rater to scale subordinates with some precision on a ratio-level scale of overall performance >scoring with large comps: establish both the ranking and extent to which differ on performance (ADV) > Disv: the # of comparisons expands geometrically as the # of subordinates increases >Dilemma: the accuracy of the scaling is a direct function of the comparisons made (more pairs = more accuracy = more time- consuming) |
|
|
Scales, measures, and formats
Bernardin & Smith (1981). A clarification of some issues regarding the development and use of behaviorally anchored ratings scales (BARS). Describe the Smith and Kendall Approach to BARS. What is the major criticism of it? |
The Smith and Kendall Approach to BARS
- enhance and standardize observation and thus prepare raters for summary ratings in the future - need common frame of reference - Different than other BARS formats because > given a set of vertical graphic scales and instructed to record the behavior on each applicable scale throughout the appraisal period *date and detail of the incident *effectiveness level aided by comparisons of behavioral "anchors" >anchoring concrete and irregular intervals >thus, a summary could be made of incident recordings Major criticism of BARS -The BARS rating process is time consuming |
|
|
Scales, measures, and formats
Blanz & Ghiselli (1972). The mixed standard scale: A new rating system. Describe the MSS. Is MSS reliable? What are some positives and negatives of it? |
MSS: designed to minimize the errors of halo & leniency
Halo- general evaluation, gives a "3" on everything leniency- positive ratings to everyone Format: descriptions of different degrees of goodness of performance for each # of separate traits pertaining to job performance & picks the best one that describes the ratee Descriptions: better than the description, fit the description, worse than the description Order is random --> reduce halo effect & allows for reliability assessment Results -MSS worked just as well as the managerial level as the worker level & cross-culture >MSS is not limited by these factors -No greater errors with MSS than other forms |
|
|
Scales, measures, and formats
Wiersma & Latham (1986). The practicality of behavioral observation scales, behavioral expectation scales, and trait scales. Which scale does research provide evidence of use for? |
Four criteria for evaluating instrument: validity, reliability, freedom from bias, & practicality (can affect care & precision)
Participants expressed dissatisfaction with PA - the PA did not discriminate between superior and poor performers - not easy to use, did not yield objective feedback data, & did not yield specific feedback to the employee or suggest training needs Results - BOS format preferred to the BES format on all the criteria - BOS format preferred to the trait scales on 6/8 measures - No significant differences with BOS & trait scales for accuracy - BES & trait judged about the same - Preferred in defending a company: BOS significantly higher than BES & trait - Asset to a case in the courtroom: BOS significantly higher than BES & trait - Focus on practicality --> BOS offers this |
|
|
Scales, measures, and formats
Murphy & Constants (1987). Behavioral anchors as a source of bias in rating. Are behavioral anchors a source of bias? How could they be? |
- Behavioral anchors thought to reduce subjectivity bias in ratings
- Behavioral anchors may serve as a source of rating bias by distorting the observation or recall of ratee behavior - Could affect cognitive processes >Bias: misdirect the rater's observation or recall of behavior * behaviors that are not representative of the rate's overall performance levels * may pay attention to similar behaviors on the scale * interpret ambiguous behaviors as the listed behaviors Results - Found main effect for scale - Do not bias observation, but may bias memory for behaviors - Behavioral anchors should be representative not atypical - Record observation |
|
|
More on Scales, Measures, and Formats
Tziner & Kopelman (2002). Is there a preferred performance rating format? A non-psychometric perspective. |
*No single rating format this is ultimately superior
Overall -Forced choice scales: minimize deliberate rating inflation >good for promotion, pay, termination -Behavior-based formats: superior for fostering performance improvement >performance feedback, clarity work roles, & reduction of work ambiguity & conflict -Personnel comparison methods (paired comparisons & rank ordering) >less susceptible to intentional leniency, central tendency, & range restriction biases *Psychometrics useless unless raters and ratee have positive reactions toward the measurement |
|
|
More on Scales, Measures, and Formats
Tziner & Kopelman (2002). Is there a preferred performance rating format? A non-psychometric perspective. Is there a preferred performance rating format? What is it related to (what org. outcomes)? |
Results
Overarching: BARS never preferred method - BARS less susceptible than the GRS to both halo & leniency effects - BOS yielded significantly higher levels of goal clarity, goal acceptance, and goal commitment - BOS increased satisfaction significantly more that GRS; feedback followed by goal-setting resulted in significantly higher worker satisfaction & organizational commitment than feedback alone; BOS feedback, & goal-setting led to significantly higher work satisfaction than other conditions - BOS PAR process vs GRS PAR, BOS higher levels of goal clarity, acceptance, & commitment; satisfaction with appraisal process; & job performance - BOS significantly higher goal specificity - BOS significantly higher levels of goal acceptance & goal communication - BOS higher ratee satisfaction - BOS --> most observable & specific goals; ratee's satisfaction & perceptions of performance goals most favorable with BOS |
|
|
More on Scales, Measures, and Formats
Fox, Bizman, & Garti (2005). Is distributional appraisal more effective than the traditional performance appraisal method? What are distributional appraisal and traditional performance appraisal methods? Which is more supported by research? |
-Performance is dynamic and varies over time
>TAM doesn't account for fluctuations -DAM: frequencies, BARs at each anchor, rate frequencies Results -DAM advantageous over TAM for interrater agreement on Cronbach's DA & SA -Higher agreement for DAM than TAM in evaluations of ratee's strengths, weaknesses & upon dimensional scores -Strong inverse relationship between measures of agreements & measures of differentiation |
|
|
More on Scales, Measures, and Formats
Roch, Sternburgh, & Caputo (2007). Absolute vs. relative performance ratings formats: Implications for fairness and organizational justice. What is the difference between relative and absolute measurements? Why is relative not fair? |
Relative: individuals being rated are directly compared against others
absolute: individuals being rated are compared against an absolute standard In order for format to be viewed as fair: provide performance & development goals, based on behaviors, & based on detailed information Relative not fair: b/c forcing rates into normal distribution when may not really exist Relative inconsistent with ratee goals of to obtain feedback and to convey information Relative over absolute for psychometrics - relative have larger correlating with a variety of criteria - RPM correlated more with cognitive ability, personality, etc. than BOS Absolute more fair than relative (relative reflects performance of workgroup) |
|
|
More on Scales, Measures, and Formats
Roch, Sternburgh, & Caputo (2007). Absolute vs. relative performance ratings formats: Implications for fairness and organizational justice. Did relative or absolute receive more favorable ratings? |
Results
- Ranking: all absolute formats were given higher ranks than the relative formats - Each absolute format (BOS) perceived more fair than relative - Ratees reported higher procedural justice perceptions when they received feedback on BOS or no format compared to trait based format - Format had a greater influence on procedural justice perceptions when the outcome was relatively fair, than when it was relatively unfair |
|
|
More on Scales, Measures, and Formats
Bartman (2007). Increasing validity with forced-choice criterion measurement formats. Describe ipsative and normative measurements. Which one does the research support more? |
Ipsative: between items from different scales
-adv: resistance to "faking good" (social desirability becomes less of an option) & halo effect Normative: a choice between different levels Results - Ipsative has higher validities and between divergent validity - Clear advantages in terms of observation validity coefficients for forced-choice criterion measurement item formats - Ipsative: reduce the average correlation between scales > decreases the redundancy of scales > less 'halo' - Ipsative: good when # of scales is small - # of scales increase = both normative & ipsative Limitations to Ipsative - more items are needed - necessary to draw from a relatively large # of scales - required considerable expertise in instrument construction |
|
|
Rating Sources and Feedback
Chapter 5 What is directive way to conduct PAs? Describe some benefits and consequences of directive. What are some observation aids in order to get more accurate information? |
1. Directive: individual chunks behaviors & may differ by supervisor (chunks may be just tasks)
- accuracy of performance judgments increases as a function of the rater's opportunity to observe the ratee's behavior & the time actually spent observing behavior - small samples of behavior will produce unreliable results - Barriers: demands to supervisors time, conflicting task demands, & proximity >frequency only relevant for new employees - Observation affects behavior - typical vs maximal - Observation aids 1. training & experience 2. BARS (standard frame of reference) 3. multiple methods: time sample, behavior diaries, self-observation, use of multiple raters, & use of large behavior samples >results justify costs? |
|
|
Rating Sources and Feedback
Chapter 5 Describe indirection observation. What are some factors that can influence results using this? |
Indirect observation: viewing videotapes, reading descriptions of subordinate's behavior, receiving complaints or letters of praise, & hearing wholly unfounded rumors of the subordinate
- Don't mean same thing - Actor vs. observer - Source of the report (who was doing it & why), initiator of the report, form of the report, medium (conversation vs letter), referent, timing (longer time = less accuracy), motive for the report, & consistency of the report observations - descriptive (reactions of what occurred) vs evaluative (individual's subject assessment of the quality or effectiveness of what was done) |
|
|
Rating Sources and Feedback
Chapter 5 Describe the sources of 360 degree feedback and some benefits and consequences of each. What is a barrier to multi-source feedback and a benefit? |
Who should obtain information?
- task acquaintance: the amount & type of work contact the evaluator has with the person being evaluated - barrier to multi-source: doesn't fit with the principle of hierarchal organization - benefit: no one has access to all the information Supervisor - most common Subordinate -little information on task performance & more on interpersonal information - tust & anonymity issues Self adv: well-informed, disv: overrate themselves (will occur less with constant feedback & told will compare to an objective standard) Peers - more consistent with supervisor than self - adv: have more opportunities to observe behaviors, behavior shouldn't be altered as much with peer, & peer ratings can be pooled - disv: friendship bias & range restriction Upper management - know little of behavior - Significant impact in deciding what to observe Customers |
|
|
Rating Sources and Feedback
Chapter 5 Who is better at collecting what kind of information in relation to 360? How do you increase the chance of collecting accurate Pas? |
360: get a comprehensive view of employee
Disagreement among sources - more likely to occur with evaluation instead of if behaviors occurred a) peers and subordinates --> evaluating person's interpersonal interactions b) upper management --> individual is contributing to the goals of the organization c) supervisor --> evaluate overall performance levels d) all sources, but especially peers --> strengths and weaknesses *few rewards and punishments for inaccurate information Accurate PAs? 1. create conditions for direct observation & a plan to collect from other sources 2. create conditions which rater is motivated to see & use information that is needed to from accurate Pas |
|
|
Rating Sources and Feedback
van Hooft, van der Flier, & Minne (2006). Construct validity of multi-source performance ratings: An examination of the relationship of self- supervisor-, and peer- ratings with cognitive and personality measures. Will sources ratings be correlated? Did traits or methods account for more variance? |
- High interrater agreement between sources should not be expected
> not problematic for development, problematic for administrative Results - supervisor ratings significantly lower than self ratings & peer ratings - Model D fit best: both trait & method factors are needed to explain the variance win the multi-source ratings (trait factors- admin skills, human skills, & technical skills) > multi source reflect rater characteristics (source used) more than the performance of the ratees - low correlations found for the external criteria & the related dimension on the multi-source feedback (typical perf.) (in-basket & intelligence test <-- max perf.) - Personality showed stronger correlations - Peer & self ratings show incremental validity with 7/10 personality gaits Overall *Methods exceeds traits *Admin purposes require 360 to be objective, reliable, & the possibility to combine the appraisal info into one global judgment |
|
|
Rating Sources and Feedback
Hannum (2007). Measurement equivalence of 360-assessment data: Are different raters rating the same constructs? What is structural equivalence? And what is the research supporting? And thus, what does that mean? |
- measurement equivalence: the invariance of rating characteristics that are attributable to rater or ratee demographics
> inequivalence= systematic bias or differential understanding of construct or rating metric - structural equivalence: do the ratings differ between sources? Results - marginally adequate structural equivalence (similar structure regardless of the sources) across the three rater types at different organizational level > suggesting various sources scores can be combined > a minimum of 3 raters per rating group |
|
|
Rating Sources and Feedback
Kluger & DeNisi (1998). Feedback interventions: Toward the understanding of a double-edged sword. Describe discrepancies and locus of attention in relation to PAs. |
Feedback interventions improve performance
1. Law of effect: a positive FI with reinforcement & negative FI - inconsistent findings 2. Ammon's: authoritative article on the effectiveness of feedback Discrepancies: behavior is regulated through the control of discrepancies or errors in the system - when detected, motivated to reduce the discrepancy Locus of attention: thinking about something different after the intervention - attention can be directed to the self, the task at had, or even the details of task - if directed at self, 2 factors can help with an enhanced effect 1. the type of task being performed 2. the types of self-goals activated by the FI |
|
|
Rating Sources and Feedback
Smither, London, & Reilly (2005). Does performance improve following multi source feedback? A theoretical model, meta-analysis, and review of empirical findings. |
- Don't expect much of performance change after multi-source feedback
> some incidents should improve more than others 8 broad factors (moderators) 1. Characteristics of feedback a. feedback is positive or negative b. self-other ratings 2. Initial reactions to feedback a. affective reactions - negative reactions --> reject the feedback - more positive with credible source, power (higher = more accepting), & trust 3. Feedback orientation - seek & use feedback - related to propensity for continuous learning 4. Personality - extraversion - conscientiousness - self-monitoring 5. Beliefs about change - believe change is possible of that change will result in some positive outcome 6. Perceived need for change - discrepancies between self-ratings and feedback from others can lead feedback recipients to perceive a need to change their behavior 7. Goal setting & related constructs - discrepancy reduction vs. discrepancy production 8. Taking action - take appropriate action in response to feedback |
