Module 22 - Evaluation And Roi

Front 1

Why evaluate meetings?

pmm692

Back 1

- determine success of meeting objectives
- identify strengths & weaknesses
- compare costs to benefits
- decide on participants for future meetings
- identify most successful participants
- reinforce major points
- gather data for marketing future meetings
- determine if mtg was appropriate solution

Front 2

Stages in Evolution Process:

Back 2

Draw diagram below:

Front 3

What is strategic meetings management?

cic11

Back 3

- not about the end product
- it is about engaging in a disciplined practice to ensure the end product to
1) maximize resources
2) supports organizational strategy
3) Produces high ROI
- Incorporate fundamental concepts such as:
1) business planning and strategy
2) Organization & execution
3) Trust and transparency

Front 4

What are the six major categories of possible sources of data to provide input on the success of a meeting?

Back 4

- meeting attendees
- stakeholders
- organizational performance records
- attendees supervisors
- attendees subordinates or peers

Front 5

Before a meeting evaluation plan is developed, what are the questions to be answered to ensure the evaluation progresses logically?

Back 5

- purpose of evaluation?
- what data to collect?
- what is desired level of evaluation?
- when to collect data?
- how to collect data?
- who will collect and analyze data?
- who will utilize and receive results?

Front 6

Why do we need to look at ROI?

pmm715

Back 6

- demanding economy/increased competition
- attention to measurement issues
- focus on bottom-line
- business mindset expected
- trend toward accountability
- demand for ROI calculations

Front 7

The ROI process is a 5 step process. Name the 5 steps:

Back 7

1. Stakeholder Analysis: identifies the owners & key stkhldrs. Then prioritize them.
2. Development of event objectives - prioritize their objectives, articulate key strategies, content, etc.
3. Measurement: answer the question, "how do we know the event was effective"?
4. Design, Development and delivery of meeting content: create experiences that reinforce key objectives
5. Demonstrating value: gather data, analyze, and report

Front 8

What are the 10 criteria of effective ROI?

pmm716

Back 8

Simple
Economical
Credible
Theoretically Sound
Account for Variable Factors
Applicable to all types of data
Flexible for pre and post mtg
Operable for a variety of mtgs
Include the costs
Successful Track Record

Front 9

What are ways to isolate the effects of a meeting (ROI)?

pmm718

Back 9

- Use of control groups*
- Trend line analysis*
- Forecasting methods
- participants or supervisors estimate of impact
- Outside experts

*Most effective ways

Front 10

How are ROI data converted to monetary values?

pmm720

Back 10

- converting output or performance to money*
- converting employee's time*
- using historical costs*
- using internal and external experts
- using participant or supervisory/manager estimates
- using staff estimates

*most credible.

Front 11

What is included in tabulating event costs?

pmm721

Back 11

- marketing costs
- developments costs
- program materials
- facilitator/speaker costs
- facility costs
- travel/lodging/meals
- staff and participant salaries and benefits
- administrative / overhead costs
- evaluation costs

Front 12

What are intangible benefits to meetings?

pmm721

Back 12

increased
- job satisfaction
- organizational commitment
- teamwork
- customer service

Reduced complaints and conflict

Front 13

What are the elements to determine the best meeting to set for ROI measurements?

pmm722

Back 13

- long life cycle of the meeting
- linkage of the meeting to operational goals & issues
- importance of the meeting to strategic objectives
- high meeting costs
- significant visibility of the meeting
- high number of meeting participants
- considerable investment of time on the part of the meeting organizer or attendees
- comprehensive needs assessment has been conducted prior to designing and planning the meeting
- senior execs are interested in the ROI measurement.

Front 14

What is a strategic plan?

cic11

Back 14

a long-range plan of action of a company or organization.

Front 15

In the Discovery Stage (know the organization) of strategic meeting management, whare are some questions to provide a profile of the total environment?

cic 12

Back 15

- what are best practices of other companies?
- what stage is our business in? merger upcoming? growth?
- how are we dealing with change?
- where do we think we are going with management and effectiveness?
- how are we currently tracking and planning for our business?

Front 16

What are benefits to including all stakeholders in the change of the company (strategic mtg management)

cic 13

Back 16

- provides a strategic role + integrate into all levels of the business.
- creates a better understanding of process
- Identifies opportunities/disconnects
- provides a better understanding of how the stakeholders describe success
- familiarizes you with the communication style of the company.
- opens the dialogue on the true value of events.
- provides an unbiased review of your own department's real and perceived core competencies.

Front 17

How is the strategic meeting management model derived?

cic15

Back 17

From the discovery, analysis and planning activities.

Front 18

In evaluations, who defines success of the meeting? What will each want to know?

cic 35

Back 18

all stakeholders.

- board of directors: summary of high and low points to ensure proper direction, approval, stimulate interest
- conference coordinator: detailed evals of speakers and topics for subsequent educational programming
- exhibitor: demographics of buyers attending show
- hotel CSM: function room sets, staff reviews
- attendees: networking opportunities avail / enhance results of future mtgs
- Contracted service provider: eval their performance
- attendee mngrs: gin support for the meeting and enhance content

Front 19

Before beginning the evaluation process, what is the first step?

cic 36

Back 19

decide who will use the evaluation and what their objectives are.

Front 20

What are recommendations to evaluate on surveys, etc?

cic 36

Back 20

review the event goals and objectives.

review what was asked at the prior event. by asking the same questions year after year, you can get comparisons between the data.

Front 21

When should you conduct the event evaluation?
cic 36

Back 21

- Before the event, get needs assessment.
- during the event, evaluate education/general sessions immediately after conclusion: info fresh in their minds.
- send an eval a week or 2 after the event. Allows them time to absorb the event.

Front 22

What are the types of data collected with evaluations?

cic 37

Back 22

Quantitative: hard data, most common and easily understood. It is represented numerically, comparing data by assigning scores.

Tells what people think, but not usually why.

Qualitative data: soft data, is a descriptive record of participants observations. open ended responses. Adds the why to the what.

Best to collect both pieces.

Front 23

What are the types of questions in a survey?

cic37

Back 23

- Open-ended: qualitative data
- Checklist: check all that apply
- two-way question: offers alternative response options as yes or no.
- multiple choice questions: respondent selects from a range of responses (Likert scale)
- ranking scale: rank a list of items

Front 24

What is the Likert scale?

Back 24

a multiple choice question

Front 25

Should there be a variety of types of questions used in surveys?

Back 25

not too many, it is frustrating to respondents as they would like to get in a groove.

Front 26

What are common rating scales?

cic38

Back 26

excellent - poor
satisfied - dissatisfied
Likely - not likely
high to low
Agree, disagree
All the time - rarely, never
favor, opposed

Front 27

what are guidelines to survey design?

cic 38

Back 27

- questions and visual appeal are important
- keep short and easy
- ask specific questions, each addressing a single topic
- avoid professional jargon/acronyms
- put difficult or time-consuming questions at the end of the survey
- ask personal questions in ranges
- don't ask for their name.
- include a contact name/fax
- number questions
- space questions adequately
- use diff colors for diff groups
- make results available.
- if online, make easy to understand
- thank them

Front 28

what has technology allowed for data collection and analysis?

cic 39

Back 28

- send & receive surveys by email
- post survey on website
- engage attendees with an onsite, touch screen kiosk
- collect data with wreless PDAs
- use numbers with recorded messages to capture responses entered on phone
- evaluate in real time.

Front 29

Why and how is sampling used?

pmm 706

Back 29

sampling provides general data from a smaller group to respresent the larger.

Can do:
- random sampling
- stratified random sampling (attendees are divided into smaller groupings within demographic and then randomly selected)

Front 30

what are factors that can affect a sample size?

pmm 706

Back 30

size of the total population
data being collected
desired accuracy
desired level of confidence

Front 31

how can evaluation results be reported?

pmm 708

Back 31

face to face, memos, newletters, intranet, email blasts, articles.

Front 32

What is semantec differential?

Back 32

A type of question format in which the respondent selects a response between two opposite terms.