Reading...
Play button
Play button
Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
91 Cards in this Set
- Front
- Back
|
Two types of questions
|
1. Closed-ended
2. Open-ended |
|
|
4 levels of measurement
|
Nominal, ordinal, interval, & ratio
|
|
|
Nominal questions - definition
|
-Used to classify responses & respondents
-Defined by response options that are exhaustive & mutually exclusive |
|
|
Three types of nominal level questions
|
1. Dichotomous questions
2. Multiple-choice questions 3. Checklists |
|
|
Dichotomous Question - definition
|
-simplest type of nominal measurement
-USE TO: classify individuals, objects, attitudes, or other responses into one of two exhaustive, mutually exclusive groups EXAMPLES: -most common form is "yes or no" question -can also be when you have to divide respondents into two groups EX: "do you prefer coke or pepsi?" |
|
|
Advantages of Dichotomous questions
|
-Easy to answer
-Simple to edit, code, tabulate, analyze -Move respondents through the questionnaire (establish skip patterns) |
|
|
What is a "skip pattern"?
|
AKA Skip logic
-Dichotomous questions -Selects appropriate next question based on individual's response |
|
|
Multiple Choice Question - definition
|
-Nominal
-Present 3 or more exclusive & exhaustive categories of response -USE TO: -categorize characteristics or attributes of respondents or responses into smaller, focused categories |
|
|
How to write Multiple Choice questions
|
1. Make sure categories are exhaustive & exclusive
2. Put choices in alphabetical order (Don't bias the respondent) 3. When using M/C to probe behaviors..... ----a. provide a "zero" option ----b. make sure response categories are nearly balanced intervals ----c. "typical" behavior appears near the center of the list. |
|
|
Advantages of Multiple Choice questions
|
1. Flexible format, appropriate to a wide range of situations
2. Similar flexibility to open-ended questions, but without asking respondents to verbalize their thoughts 3. Easier to code, tabulate, edit, & analyze than open-ended |
|
|
Dichotomous vs Multiple Choice
|
Dichotomous is only used when entire range is two categories
|
|
|
Checklist - definition
|
-Nominal
-Combines a series of related dichotomous questions into one single question -Less time-consuming & tedious than asking each question individually |
|
|
How to write a Checklist
|
1. Clearly define the criteria on which items in the checklist should be selected.
2. Do not assume that the respondent will check any specific number of items ("as many or as few or even no items as you like") 3. Reduce bias (put all choices alphabetically. Attitude choices should have equal number of pos and neg) |
|
|
Ordinal Level Questions - Definition
|
-Asks respondents to order a list of items according to an explicit criteria
-Informs researcher about relative ordering of items but NOT distance between items -Rank questions |
|
|
Rank order questions - definition
|
-Present respondent with several characteristics, objects, or attributes & then request that the respondent orders or ranks them with respect to a specific characteristic
|
|
|
How to write a rank order question
|
1. Clearly state respondent's task ("rate X")
2. Identify characteristic to be used in ranking ("overall appeal") 3. Explain method to perform the ranking ("use '1' for most appealing," etc) 4. Put answer choices in alphabetical order 5. Use no more than 6 items |
|
|
Interval Level Questions - Definition
|
-provides information on the rank order of items AND provides distance between items
-Rating scales most common Types: -Rating Scale -Semantic Differential |
|
|
Rating Scales definition
|
-Interval
-Require respondent to select one option from the members of an explicit, well-defined continuum |
|
|
How to write an interval-level question
|
1. Unbiased lead-in (explicitly state both ends of the rating scale
2. Equal intervals (spaces) between scale points. 3. Direct correspondence between information requested in the question itself and the scale used to gather info. (EX: question asks how believable something was, answer choices should be "very believable, not believable, etc) |
|
|
How to write interval-level answer choices
|
1. Determine the # of categories to use. (between 4-7 are ideal)
2. Determine whether the scale should be balanced or unbalanced. (use unbalanced when you generally know the direction of their answer but want finer distinctions on one end) 3. Decide whether your balanced scale will have an even or odd # of options 4. Decide if you will use a frame of reference |
|
|
2 Characteristics of a balanced scale
|
1. Equal number of pos & neg choices
2. Response options mirror each other |
|
|
2 reasons you should NOT use an unbalanced scale
|
1. Bias
2. Problems with data analysis & interpretation |
|
|
What is a noncomparative rating scale?
|
Interval
Asks a respondent to assign a rating without an explicit frame of reference "How believable is X?" |
|
|
What is a comparative rating scale?
|
Interval
Provides a frame of reference "How believable is X compared to Y?" |
|
|
What is a semantic differential?
|
*Interval
-Asks a respondent to rate an object on a number of related, seven-point scales bounded on each end by one of two bipolar adjectives. EX: Trustworthy__ __ __ __ __ __ __ Untrustworthy Sincere__ __ __ __ __ __ __ __ Insincere |
|
|
Two considerations of semantic differential scales
|
1. It may be unfamiliar to respondents. (must have clear instructions)
2. Take care in placing the adjectives (alternate pos & neg to make sure respondent reads all choices) |
|
|
Ratio Level Questions - Definition
|
-Measure objects, behaviors, and beliefs on a continuum with a fixed, zero origin
-Placement reflects degree to which the measured object, behavior, or belief possesses the target characteristic. -Common form: constant sum |
|
|
What is a constant sum question?
|
-Ratio
-Requires respondent to divide a present quantity (the constant sum) among 2+ objects/attributes in a way that reflects the respondent's relative preference for each object |
|
|
How to write a constant-sum question
|
1. Focus for allocation is explicit (EX, "appeal to you")
2. Clearly describe respondent's task (EX: "allocate the points") 3. Make criteria for point allocation clear (EX: "the more points you assign something, the higher importance..") 4. Number of options needs to be manageable (7 or less items is ideal) 5. Determine number of points to be allocated (25 or 100 is ideal) |
|
|
Considerations for open-ended questions
|
1. The question must be truly open-ended. (don't introduce bias)
2. Incorporate probes ("is there anything else?") 3. The question should be single-minded. (EX: don't ask "what are strengths AND weaknesses of X?) |
|
|
8 Rules for ALL question development
|
1. Clearly state the respondent's task in simple language
2. Use simple, active sentences & commonly used language 3. Avoid bias 4. Avoid multiple informational requests in a single question. 5. Avoid ambiguity. 6. Avoid assumptions ("what, IF ANYTHING, ....") 7. Justify requests for personal information. 8. Provide a reasonable time frame for behavioral questions. (ask about the last 24hrs, not month) |
|
|
What is Razorfish & what did they find?
|
-One of the world's largest interactive marketing & technology companies
-Digital Brand Experience Report/2009 -Consumer experience matters!! -Consumers are digitally engaged with brands - |
|
|
What is Cossette & what did they find?
|
-international, full-service communications company with offices in Canada, US, and UK.
-one of top communications companies in the world 2009 Social Media Study |
|
|
Steps in questionnaire design & construction
|
1. Determine data collection method // determine need for disguise.
2. Create questionnaire components 3. Physically prepare questionnaire 4. Internal evaluation 5. Pretest (6. Revision, as needed. Return to 4) 7. Finalize questionnaire 8. Prepare field support materials |
|
|
QD: Determine data collection method
|
4 approaches: interviews, mail, telephone, online
-Not F2F = less complex questionnaire -Self-administered questionnaires = written for the "eye", use pictures if you want -telephone = write for "ear" |
|
|
4 main components of a questionnaire
|
1. Introduction
2. Screener 3. Main body questions 4. Classification questions |
|
|
QD: Introduction (definition)
|
GOAL: motivate respondent to participate
-Explain goals and purpose of the research -Ask for respondent's cooperation & participation in the study -Wording depends on need for disguise, complexity, and sensitive nature of data collected |
|
|
QD: Introduction (components)
|
-Reference to importance
-Information on the rationale & goals of the research -clear request for participation -reassurance that participating is not difficult or time-consuming -Need for truthful answers -promise of confidentiality -reassurance that the research is legit |
|
|
QD: Main body components
|
-Begin with simple, nonthreatening, interesting, easy-to-answer questions
-Group questions on the same topic together, complete one topic before moving on to another -Within a topic, move from general to most specific -Place difficult/sensitive questions at the end -Avoid biasing questions later with questions asked earlier -Address most important topics first |
|
|
QD: Physically prepare the questionnaire - what are the 3 areas to address?
|
Three areas to address:
1. visual appearance 2. transitions between major topic areas and between individual questions 3. interviewer/respondent instructions |
|
|
QD: Physically prepare the questionnaire - Self-administered questionnaire design
|
-Design is most important to self-administered questionnaire
-Should be uncluttered, easy to follow, typed in easy-to-read typeface. -Good visual appearance and layout -Questions visually distinct from responses -Response coding is unobtrusive -Questions do not continue across columns or pages -Columns are use to help maintain the respondent's focus, save space, and simplify response. |
|
|
QD: Physically prepare the questionnaire - 2 considerations in online questionnaire design
|
Two considerations:
1. Form & visual appearance 2. Range of question types available 3. Way in which specific questions are presented |
|
|
QD: Physically prepare the questionnaire - online questionnaire - form & visual appearance
|
-Use of pictures is not trivial. Should have a specific function, real and meaningful link to research
-Carefully consider how many questions to present on a single web page -Understand the relationship between space provided & depth of response in open-ended q's -Input box labels affect the quality of data provided -Not all response formats for closed-ended questions are equivalent |
|
|
QD: Physically prepare the questionnaire - internal evaluation
|
Make certain that:
-the questionnaire is complete & concise -the questions themselves are clear & appropriate -The layout is clear & easy to follow |
|
|
QD: Pretest - Common problem areas
|
1. Problems with administration
2. Problems related to question comprehension 3. Problems related to question demands 4. Problems related to response options 5. Problems related to organization & question sequencing |
|
|
QD: Pretest - best way to pretest?
|
-Cognitive interviews
-Help researchers understand how respondents interpret & construct answers |
|
|
What is cognitive interviewing?
|
administering draft survey questions to a small number of target individuals while collecting additional verbal information ABOUT the survey responses, which is used to evaluate the quality of the response to help determine whether the question is generating the information that the author intends.
|
|
|
What are the two approaches to cognitive interviewing?
|
1. Think-aloud method
---> asks respondents to verbalize their thoughts as they are answering a question. --->Interviewer reads each question aloud and records what the respondent says and does to comprehend the question 2. Active probing ---> after the interviewer asks a question and the respondent replies, the interviewer probes further in order to understand the response |
|
|
5 common types of probes
|
1. Comprehension/interpretation (what does X mean to you?)
2. Paraphrasing (can you repeat X in your own words?) 3. Confidence judgement (how sure are you that you X?) 4. Recall probe (How do you remember that you X?) 5. General probes (how did you arrive at that answer?) |
|
|
Descriptive research -- definition
|
-Provides a look at consumer's attitudes, beliefs, lifestyles, and behaviors at a particular point in time.
EX: survey, personal interviews |
|
|
Experimental research -- definition
|
-Determines CAUSALITY -- the effect of changes in one area on one or more other areas
|
|
|
When you SHOULDN'T use a survey:
|
-- when questions require recall, especially over long periods of time
-- When individuals are asked for introspection in areas of behavior they don't normally think about --When individuals are asked to use memory to attribute cause & effect -- There may be complex, multiple, or interrelated influences on behavior |
|
|
Experiments - 4 basic steps
|
1. Identify what you need to learn
2. Take action (conduct the experiment by manipulating 1+ variables) 3. Observe the effects of those actions on DVs 4. Determine the extent to which the observed effects can be attributed to the IV |
|
|
4 Requirements for causality
|
1. Events must take place in the proper order. (cause must be before effect)
2. Events must take place at the same time and show an explicit relationship. (must see a link between cause & effect) 3. Alternative explanations must be reduced & eliminated whenever possible. 4. Strength of association. (cause & effect must be verified several times) |
|
|
Internal Validity definition
|
-Extent that one can eliminate alternative explanations for the observed experimental results
-More results supporting the causality, higher internal validity --- reduced whenever alternative explanations of the results can be found |
|
|
Internal Validity Problems/Threats
|
--Premeasurement
--Interaction --Testing --Instrumentation --Maturation --Selection --Mortality --History --Researcher Bias |
|
|
IV THREATS: Premeasurement
|
*Problem w/pretest
--- occur whenever an interview administered before the start of an experiment has a direct effect on a respondent's attitudes, actions, or behaviors DURING the experiment --survey influences behavior --occur WITHOUT exposure to indep. variable |
|
|
IV THREATS: Interaction
|
-- Occur when an interview administered before the start of an experiment affects the respondent's sensitivity or responsiveness to the Independent Variable
--survey influences sensitivity to independent variable (ex: survey about ads, start to notice ads more) |
|
|
IV THREATS: Testing
|
--result from repeated administrations of same survey or questionnaire, when one is a pretest and the other is a posttest.
--Changes in scores can be because of repeated testing --EX: a consumer may try to guess more "correctly" the second time |
|
|
IV THREATS: Instrumentation
|
--changes made to the measurement instrument (survey) or data recording techniques during the experiment
4 types --changes in measurement instrument --changes/noncomparability in recording techniques --changes in data collection methods --inconsistent interviewing |
|
|
IV THREATS: Maturation
|
changes in respondent's moods, attitudes, behaviors, and physiology during the experiment
(getting bored, tired, etc) |
|
|
IV THREATS: Selection
|
--At the start of the experiment the test and control groups have different demographics, attitudes, or behaviors.
--Random assignment can eliminate this |
|
|
IV THREATS: Mortality
|
--Respondents drop out of a study between pre and post test
|
|
|
IV THREATS: History
|
--any events or influences that have the potential to affect the experimental outcome as measured by the dependent variables
Can be the result of: -circumstances beyond any person's control -deliberate actions taken by others |
|
|
IV THREATS: Researcher Bias
|
--can be eliminated through raised self-awareness and proper planning
--can occur during any part of the experiment |
|
|
Experimental design: key points
|
--info accuracy & cost increase as more controls are built into the experimental design
--IV threats can be controlled through experimental design --not necessarily that the researcher wants to control for everything... have to have an appropriate balance between cost & accuracy |
|
|
Quasi-Experimental Designs
|
--not true experiments
--attempt to uncover a causal relationship, but do so poorly because they fail to eliminate threats to IV --Results are difficult to interpret and only slight improvements over descriptive studies |
|
|
Three types of Quasi-Experimental Designs
|
-One group post-test only
-One group pretest to post-test -Two group post-test with control |
|
|
Q-ED: One Group Post-Test Only
|
"let's at least do something" design
--takes a single group of individuals, exposes them to the treatment or manipulation, then measures the DVs as part of the post test **NOT randomly selected WEAKNESSES --does not control for maturation, selection, mortality --cannot generalize to pop --cannot attribute ratings to email campaign (historical threat) |
|
|
Q-ED: One Group Pretest to Post-Test
|
"before and after without control"
--premeasure is taken before experimental manipulation of Indep. V --test again after exposure to indep. V WEAKNESSES: -no control for any historical threats to internal validity -does not eliminate problems associated with initial pretest (introducing bias or sensitivity) -does not eliminate problems associated with the sample itself (ex: maturation) |
|
|
Q-ED: Two Group Post-Test With Control
|
Random selection, but not random assignment to control & experimental groups
WEAKNESSES -Because random assignment was not used to form the groups, there is no way to tell any equivalence in pretest scores |
|
|
3 components of a true experiment
|
1. independent variable(s)
2. dependent variable(s) 3. manipulation |
|
|
2 rules of true experimental designs
|
1. have a control group
2. use random assignment to form test and control groups |
|
|
5 most common types of true experimental designs
|
1. Simulated pretest to post-test
2. Post-test only with control 3. Pretest to post-test with control 4. Solomon four-group design 5. Factorial designs |
|
|
What are the two most common forms of experimental research?
|
1. Simulated pretest to post-test
2. Post-test only with control |
|
|
TED: Simulated pretest to post-test
|
*two groups - two measures
--controls for premeasurement and interaction threats to IV --Controls by using two randomly assigned groups for pretest & treatment/post test --Still allows for history, maturation, instrumentation, and selection threats |
|
|
TED: Post-Test Only with Control
|
*two groups - two measures
--uses two groups of respondents --compares post-test measures from the treatment and control groups --CONTROLS FOR: history, maturation, instrumentation, premeasurement, & interaction --DOES NOT CONTROL FOR: mortality |
|
|
TED: Pretest to Post-test With Control
|
*two groups with four measures
--makes certain that the treatment and control groups are equivalent before the research starts --Tests both groups before the research starts --CONTROLS FOR: premeasurement, history, maturation, instrumentation, selection --DOES NOT CONTROL FOR: mortality |
|
|
TED: Solomon Four-Group Design
|
*Four groups - six measures
--most powerful and most resource-intensive experimental design --Controls for ALL threats to IV --combines a pretest to post-test with control design and a post-test only with control design --rarely applied to advertising and marketing research because of its complexity, cost, & timing --all four groups get post-test --two groups get pretest (one control, one treatment) --two groups do not get pretest (one control, one treatment) |
|
|
what is a variable with different levels?
|
Changing the quantity of the independent variable
EX: length of videos watched, number of emails received |
|
|
What is a variable with different aspects?
|
Changing the characteristics of the independent variable
EX: male vs female spokesperson, blog with ads in different spots |
|
|
How to select an experimental design?
|
--based on what information is needed
--based on specific threats to IV that need to be controlled |
|
|
What is a Factorial Design?
|
-Experiments with 2+ independent variables, where each of these variables has 2+ aspects
-Simultaneously measures the effect of two+ IVs -- each with different levels -- on one or more DVs -Diagrammed as squares or cubes |
|
|
What are the 2 components of a factorial design?
|
1. Main effects
--seperate influence of each independent variable on the DVs 2. Interactions --Combinations of main effects on the DVs --simultaneous effect of two+ IVs is different from the sum of their independent effects. (EX: people don't react to a female spokesperson or a funny spokesperson, but they react to a funny female spokesperson) |
|
|
What is PsychData?
|
a company with the capability to conduct online experiments with random assignment.
|
|
|
3 circumstances that improve internal validity
|
1. Using strong predictions in research.
2. When a change in the cause is followed by large change in effect. 3. When the effect reverses a prevailing tendency or condition (as opposed to a change that simply increases more of the same) (EX: an ad that reverses negative opinions vs an ad that improves positive opinions) |
|
|
How to make a strong prediction?
|
-Include directionality (increase or decrease "X will increase/decrease")
-address nature of the change ("X will improve Y in the areas of ___________") |
|
|
3 types of external validity
|
1. Population-related
2. Measurement-related 3. Setting-related |
|
|
Population-related external validity
|
-increases when proper sampling procedures are used to select individuals from the population of interest and assign them to groups
|
|
|
Measurement-related external validity
|
-increases when steps are taken to ensure that measures used to collect the data are collecting the right data
|
|
|
setting-related external validity
|
--Laboratory study will have high internal validity but low external validity
--Field experiments have high external validity but low internal validity |
