Training Evaluation

Front 1

training evaluation

Back 1

process to assess value of training programs to employees and the organization

Front 2

formative evaluations vs. summative evaluations

Back 2

Form - provides data about various aspects of a training program

Summ - data about worthiness or effectiveness of a training program

Front 3

descriptive evaluations vs. causal evaluations

Back 3

descriptive - provides info that describes the trainee once he/she completes training program

causal - provides info to determine whether training caused the post-training behaviours

Front 4

Models of Training Evaluation

Back 4

1) Kirkpatrick's Hierarchical Model

2) COMA model

3) Decision Based Evaluation

Front 5

Kirkpatrick's model

Back 5

Training effective when:

L1) trainees report positive reactions

L2) trainees learn the material

L3) trainees apply what they learn on job

L4) training has positive effect on organizational outcomes

Front 6

COMA model

Back 6

suggests measurement of variables falls into four categories:

Cognitive variables - declar./proced. learning

Organizational environment - support, culture

Motivation - desire to learn/to transfer learning

Attitudes - self-efficacy, perceptions of control

questionnaires given to assess above components

Front 7

COMA improves on Kirkpatrick's model in 4 ways

Back 7

1) defines new variables with greater precision

2) greater number of measures

3) measures proven to be causally related to improved transfer of training

4) especially useful for formative evaluations

Front 8

Decision-based evaluation

Back 8

3 assessors to custom-fit evaluation to requirements of situation:

1) target of theevaluation (what we wish to find out),

2) focus (what are the variables to bemeasured)

3) methods that would be most appropriate to conductthe evaluation

Front 9

DBE's 3 potential targets

Back 9

1) trainee change

2) organizational payoff

3) program improvement

Front 10

Advantages of DBE

Back 10

- does not specify a single best way for evaluation

-does not compel the use of a single set of variables for all situations

-allows for different variables to be measured depending on goals

-more flexible, used for both formative and summative evals.

Front 11

Reaction measures

Back 11

-Affective reactions - assesses trainee likes and dislikes of a program

-Utility reactions - perceived usefulness of program (more important for transfer)

-questionnaires more efficient, cost effective, consistent, can be used repeatedly

Front 12

Measuring learning

Back 12

Declarative - acquisition of facts, most assessed. Multiple choice, true/false tests

Procedural - organizational of facts (more strongly related to transfer), re-organizing steps into correct order, case scenarios with carefully created answers to detect comprehension levels

Post-training tests can increase motivation to learn. Tests also provide legal defense to prove competency, and indicate which course material is insufficient

Front 13

Measures to measure behaviour

Back 13

1) self reports - trainee indicates how often she/he used new knowledge. easiest to use. accuracy is questionable.

2) observation - others observe whether trainee uses new knowledge. accuracy questionable when supervisor/peers not in direct line of observing

3) production indicators - output assessed through records, i.e. sales or absenteeism. can be highly precise on specific behaviours.

Front 14

Other factors to measure

Back 14

-Motivation - questions based on Valence, instrumentality, expectancy

-Self-efficacy - "how confident are you..."

-Perceived/anticipated support - "I know I can obtain help from my supervisor - agree or not?"

-organizational perceptions - i.e. "do you agree that supervisors set goals for trainees?"

-organizational results - how has org. befitted? hard to measure

Front 15

hard vs. soft data

Back 15

hard - results assessed objectively

soft - results assessed through perceptions and judgements

Front 16

Data collecting designs

Back 16

non-experimental - comparison not made to another group of people

experimental - trained group compared to another group who has received no training, groups are random

quasi-experimental - like above but groups not random

last two stronger evidence of causality, demonstrating changes in trainee behaviour