Research Methods Exam 2

Front 1

Target Population

Back 1

the population to which the researcher would like to generalize their results

Theoretically chosen aggregation of units of analysis

Front 2

Sampling Frame

Back 2

Actual set of units from which a sample has been drawn
-Ex: If it was a phone survey and you selected names from a phonebook, the phonebook would be your sampling frame.

-the researcher must find a way of making the population operational.
- Not a sample, but the operational definition of population
- denotes the set of all cases from which the sample is actually selected.

Front 3

Probability Sampling

Back 3

Any procedure that gives every member of the population you’re interested in a known chance of being selected. Selection mechanism is based on probability.

It is scientifically more accepted though more costly and inconvenient.

advantage: removal of bias.

Front 4

Non-Probability Sampling

Back 4

we don’t know the probability of selecting a unit into our particular sample. These samples tend to be quick and easy, but not as representative of the population from which they’re drawn.

Front 5

Sample Vs. Population

Back 5

The population is the area in which you are trying to get info from

The Sample is the section of the population that you are actually going to study.

Front 6

Population Parameter

Back 6

A quantity or statistical measure that for a given population, is fixed and that is used as a value of a variable in some general distribution of frequency function to make it descriptive of the population.

Ex: Mean and Variance

Front 7

Why is sample size less important for non-probabilistic samples?

Back 7

It’s not random and its not as accurate, the researcher in the non-probabilistic isn’t worry about finding a result that correlates to the entire population just his/her study.

Front 8

"Representativeness"

Back 8

the degree to which a sample represents the population it is drawn from

Front 9

Are random (probabilistic) samples ALWAYS representative of the population? How does sample size affect this?

Back 9

Not necessarily, if you had 2 students from class it wouldn’t be representative, larger sample sizes increases representativeness for a random sample.

Front 10

What is a good sample size for getting a representative view of the U.S. population (within 3-4 percentage points)?

Back 10

1500 people, if drawn correctly

Front 11

Simple Random Sample

Back 11

Get a list of everybody who is in your population (sampling frame) and draw a set of random numbers to find out who will be part of your sample.

Front 12

Systematic Random Sample

Back 12

Get a list of everybody or every object in the population, then start with a randomly selected person, and take every Nth person after that until you have enough people for your sample.

Front 13

Stratified Random Sample

Back 13

You set up a random sample to ensure that appropriate numbers are drawn from homogenous subsets of that population.

Front 14

Random Digit Dialing

Back 14

One common approach, especially with telephone surveys is to simply dial phone numbers at random.

Front 15

Multi- Stage Cluster Sampling

Back 15

Divide the population into clusters, randomly select clusters, then randomly select clusters within these clusters until you get to a level where you can develop a list of everybody in the population you’re going to sample from

Front 16

Cluster Sampling

Back 16

useful when a sampling frame- a definite list- of elements Is not available, as often is the case for large populations spread out across a wide geographic area or among many different organizations.

multi-stage procedure

the researcher draws a random sample of clusters, then draws a random sample of elements from each cluster

Front 17

Census

Back 17

You include everybody in the population

Not everyone fills it out. Even though it is the law, it is an unenforceable one.

Front 18

Self-Selected Sample

Back 18

Web-site polls or newspaper polls

Front 19

Convenience Sample

Back 19

First 100 (or any other set X) you come to

-aka Snowball Sampling

Front 20

Typical Case

Back 20

Researcher uses expert judgment to select cases that are representative or “typical” of the population.

-aka Purposive sampling

Front 21

Quota Sampling

Back 21

recognizes problems with non-representativnesss of convenience of self-selected samples. It says if you know the approximate demographic makeup of the target population, then you can mimic this in your sample.

Unlike stratified sampling random sampling, the filling of quota is not random but is based on convenience sampling

Front 22

Self-Administered Questionnaire

Back 22

Mailing survey typically and several follow-ups for response.
- Pros: least expensive, sensitivity
- Cons: lower response rates, requires simplicity, impersonal
-Survey Design: Maximize, clear response patterns, have to send out reminders, send out second attempts, second delivers, include stamps envelopes all to achieve higher response rates

Front 23

Face-to-Face Interview

Back 23

Use of survey interview schedule that interviewer reads to subject.
- Pros: high response rate, longer and more complex, few incompletes
- Cons: expensive, bias

Front 24

Phone Interview

Back 24

- Use of survey interview schedule, phone and normally a computer to enter responses with (CATI). A computer aid that helps you with your questionnaire (Computer Assisted Telephone Interview)
- Pros: speedy, medium response rate, easy to sample
- Cons: telemarketing, call-backs, declining response rates
- Survey rates, (ask the same question the same way by the same person) to maximize response rates have to make it attractive, no skip patterns can’t be too complicated.

Front 25

Survey Interview vs. Ordinary Conversation

Back 25

- Survey questions must be asked of many people, not just one.
- The same survey question must be used with each person, not tailored to the specifics of a given conversation. (Can’t change question based on one person’s misunderstanding, but in face to face interview you can modify a question to help conduct your survey)
- Survey questions must be understood in the same way by people who differ in many ways.
- You will not be able to rephrase a survey question if someone doesn’t understand it, because that would result in a different question for that person.
- Survey respondents don’t know you and so can’t be expected to share the nuances of expression that help you and your friends and family communicate.

Front 26

Direct vs. Indirect Questions

Back 26

Direct - what the survey asks is what the investigator wants to know

Indirect- the link between what the survey asks and what the designer wants to know is somewhat unclear or hidden.

Front 27

Open-Ended vs. Close-Ended

Back 27

-Open Ended: Free response or Fixed Choice
-Close ended: you should fit somewhere you should fit in one place, should be a, b, c, d in that questionnaire

Front 28

Likert Scales

Back 28

simply express the range of possible attitudes across a continuum from negative to positive.
- strongly agree, somewhat agree, uncertain, somewhat disagree, strongly disagree

Front 29

Amount or Frequency Responses

Back 29

help you find out the quantity of some type of behavior.
- Never, rarely, sometimes, frequently, always

Front 30

Semantic Differential Scale

Back 30

a way of having the respondent spatially identify relationships between things and concepts:

-God Feminine………..(X)….Masculine

-Job
Excitement….......(X)……Obligation

Front 31

Order of Merit

Back 31

Ask people to rank the importance of several items.

Front 32

Demographic or Objective

Back 32

attempts to extract objective information from the respondent. DIRECT or INDIRECT:

1) Are any of your good friends that you feel close to Black/White? (direct)

2) Do you have any good friends that you feel close to? About how many? How many of them are White/Black? (three-step)

3) Who are your good friends (just tell me first names)?

-Now let’s go back and talk about (EACH)? Is (NAME) Asian, Black, Hispanic, or White? (indirect)

White R Other Race Friend: 1.) 42% 2.) 24% 3.) 6%
Black R Other Race Friend 1.) 62% 2.) 45% 3.) 15%

Front 33

Cafeteria-Style Responses

Back 33

are formatted such that the respondent picks any and all that apply to her situation.

Front 34

problematic/biased wording of questions

Back 34

1. Don’t make unreasonable demands on Resp’s memories
-Ex: How many restaurants have you eaten at in the last year?

2. Don’t use double-negatives
3. Avoid jargon
4. Ensure clarity in wording
5. Floating vs. Fence-sitting
-Floating: ppl who have no idea but choose an answer anyway
-Fence-sitting: always say "not always agree of disagree", never makes decision.

6. Avoid Double-Barreled questions
-"Should Nixon be thrown out of office and Impeached?"- people may believe yes to one but not to the other.

7. Social Desirability Effects
8. Acquiescence Response set:
-a consistent tendency to respond to yes/no questionnaire items by answering Yes, regardless of the content of the questions.

9. Response Position bias
-vary response category or else people will fill out same circle; xmas tree

10. Minimize risk of bias. No biased words or phrases.
-avoid emotionally worded questions that will trigger gut reaction.

Front 35

How may a researcher go about refining questions?

Back 35

1. Use simple, unambiguous language that can be understood in the same way by all respondents.

2. Avoid long and complex sentences.
hard to read and lead to misunderstandings. troublesome for interviewer-administered survey because they make for questions that are hard to grasp when someone is listening to them.

3. Avoid hypothetical questions.
Questions that ask people to say how they would respond or feel in conditions that they have never experienced have been found to be highly unreliable. The answers turn out to be more dependent on the way the question is worded than on how people feel, and they turn out to be poor predictors of peoples’ behavior.

4. Avoid double-barreled questions (asking two question at once) or questions with built-in assumptions.
-“Would like to be rich and famous?” Two questions are being asked at once; a person obviously could want to be rich but not famous.
-“Given the crime problem, how do you feel about walking in your neighborhood?” Assumes there is a crime problem and that it is linked to walking.

5. Do not ask questions that ask respondents for information they do not have.
-Asking people to answer questions that they don’t know anything about is bad question design, unless the goal of a question is to measure level of information.

6. Avoid questions that ask about causality.
-People’s diagnoses of why thing occur are notoriously inconsistent and vary from respondent to respondent.

7. The time frame referred to in questions should be unambiguous.
-The time frame can play a major role in what the actual answer is. Questions need to be specific about what time frame is covered by the question.

8. For fixed response questions response categories must be exhaustive and mutually exclusive.

9. Make sure the context of the question does not inappropriately affect its meaning.
-Preceding questions can add meaning or imply something about the content of a question, which is not actually intended. Such “context effects” may affect some respondents more than others, and they change answers from project to project, when the same question may be in different contexts.

10. Define terms. Complex definitions and instruction should be given in a preamble or introduction, not in the question itself.

Front 36

How can question order impact survey research and can we account for this?

Back 36

At some point throughout the survey the respondent is going to reach a level of comfort, so you need to build rapport so they trust you more and more throughout the survey.

Front 37

Advantages/strengths of survey research

Back 37

Versatile
Almost any social issue can be addressed through surveys, even the most sensitive topics such as sexuality and income
Efficient
Costs are relatively low (one methods text cites a range from $10/$15 for mail surveys to $100-$300 for in-person interviews, but may be a bit low)
The interviews for the sex survey we are reading about were very expensive: $450 per person in the 1990s.
Powerful: Results of surveys can be generalized to large populations
Probability sampling among large populations allows researchers to generalize about large populations with a relatively small number of respondents (e.g., excellent nationally representative surveys in the U.S. can involve as few as 1200 respondents.)

Front 38

Disadvantages/weaknesses of survey research

Back 38

-We have to be careful in assessing the “meaning” of surveys.
-Non-response bias is an especially common and vexing problem in surveys.
-The issue of Non-Response: Response Rates for a mail survey at different stages
•First mailing- 20 percent
•Postcard follow up- 40 percent
•Replaced questionnaire- 50 percent
•Certified Mail- 70 percent

-Survey “context” is artificial. May not allow us to study behavior as it actually exists in the broader world.