Research Methods For Public Administration

Front 1

Positivism

Back 1

Associated with quantitative research. Idea that there is one best way. Apply natural science technology to the social world

Positivists believe that one can seek to explain and predict what happens in the social world by searching for patterns and relationships between people. They believe one can develop hypotheses and test them, and that knowledge is a cumulative process.

Front 2

Burrell and Morgan Paradigms or Org Theory

Back 2

Functionalist, Interpretive, Radical Humanist, Radical Structuralist.

The 4 paradigms define 4 views of the social world based upon different meta-theoretical assumptions wiht regard to the nature of science and society and as such, are incompatible.

Holy Grail of Org Theory in PA

Front 3

Functionalist Paradigm (objective-regulation)

Back 3

This has been the primary paradigm for organizational study. It assumes rational human action and believes one can understand organizational behavior through hypothesis testing.

Assumes people are rational and that things happen for a reason (quant)

Front 4

Interpretive Paradigm (subjective-regulation)

Back 4

This paradigm "seeks to explain the stability of behavior from the individual's viewpoint". Researchers in this paradigm try to observe "on-going processes" to better understand individual behavior and the "spiritual nature of the world".

based on individual interpretations of world/action (qual)

Front 5

Radical Humanist Paradigm (subjective-radical change)

Back 5

Theorists in this paradigm are mainly concerned with releasing social constraints that limit human potential. They see the current dominant ideologies as separating people from their "true selves". They use this paradigm to justify desire for revolutionary change. It's largely anti-organization in scope.

Constraints on human reactions, researchers job is to remove these constraints (qual)

Front 6

Radical Structuralist Paradigm (objective-radical change)

Back 6

Based on this paradigm, theorists see inherent structural conflicts within society that generate constant change through political and economic crises. This has been the fundamental paradigm of Marx, Engles, and Lenin.

constructional constraints (like capitalism) but that it generates change automatically
(quan)

Front 7

Systematic Reviews

Back 7

Review all previous studies, good because reviewing studies next to one another can highlight methodological issues, basically it means to evaluate other people's studies with a set of established criteria.

Front 8

Validity

Back 8

Is what you are saying true?

Front 9

Reliability

Back 9

Can you repeat the study?

Front 10

Face Validity

Back 10

On the face of it, does it make sense

Front 11

Convergent Validity

Back 11

Do most of the varilables have high intercorrelations? Do all variables sort of point in same direction (converge)?

Front 12

Criterion Validity

Back 12

Easy quality indicators to point out, does a variable that is known to be valid match with your findings?

Front 13

Construct Validity

Back 13

Related to theoretical argument of hypothesis: "Does your variable match up with a theoretically derived hypothesis?

Front 14

Scientific Method

Back 14

1. Problem Formulation
2. Formulate research question
3. Define concepts related to research question.
4. Operationalization of concepts
5. Hypothesis formulation
6. Collect data
7. Analyze data
8. Look at results
9. Produce research report
10. Can this be replicated?

Front 15

Ecological Fallacy

Back 15

Attribute things to individuals you don't know can be attributable, can't take crime rates in richmond and then say that crime is higher in cities with a higher number of african americans because your unit of analysis is the city

Front 16

Reductionism

Back 16

Kind of the opposite of Ecological Fallacy. It is where you take broad data, and then inappropriately appply stringent appropriation of variables, like looking at only economic issues when there is more to the story

Front 17

Nominal Variables

Back 17

Gender: generally considered mutually exclusive, there is no rank order (ex, men are not better than women)

Front 18

Ordinal Variables

Back 18

Items that are on a scale like level of agreement with a statement, etc

Front 19

Interval Variables

Back 19

Temperature is an example, because zero doesn't mean there is no heat really, because numbers go below that. There is a meaningful distance between numbers of measurement

Front 20

Ratio Variables

Back 20

Age (0 does have a meaningful distance), Length of residence, income, where there is a relationship to zero?

Front 21

Constructivism

Back 21

Within social science there is almost invariably a rejection of the view that
'truths' about the social world can be established by using natural science
methods. This is essentially because of the nature of the subject matter of the
social sciences – people. People, unlike the objects of the natural world, are
conscious, purposive actors who have ideas about their world and attach meaning
to what is going on around them. In particular, their behaviour depends crucially
on these ideas and meanings. Constructivist researchers tend to use research
methods such as interviews and observation which allow them to acquire multiple
perspectives.

Front 22

"all knowledge is situated"

Back 22

that some types of knowledge are best learnt in one context rather than another and that the more authentic the context, the more effective the learning.

Doing any kind of research is that there is a certain power stance and that it is based upon the knowledge of the researcher and the decisions he makes.

Front 23

Scientific Method, take 2

Back 23

1. Select a topic/formulate a problem.
2. Develop broad questions.
3. As you develop, move more narrow in scope.
4. Formulate a research question.
5. Operationalizing a concept (defining variables in terms of the question)
6. Collect data
7. Analyze data
8. Is your question feasible, can it be replicated?

Front 24

Mutually exclusive and exhaustive

Back 24

Example, gender is mutually exclusive and if you include both male and female, then there is no one that is left out)

Front 25

Grounded Theory

Back 25

Based in qual. research. Says that theory should arise from observation not like in quant. research where researchers start with a priori (already established theories) that force them into certain assumptions

Front 26

Field Research

Back 26

Associated with grounded theory

Front 27

Ethnography

Back 27

Researcher is embedded within certain groups and are observing from their perspective. Shared social experiences.

Front 28

Phenomenology

Back 28

A philisophical term used to focus on people's subjective experience and interpretations of the world. It adds an interpretive peiece to ethnographic studies through an interpretive lends, how people interpret things is associated with philosophy (more theoretical)

Front 29

Heuristic Inquiry

Back 29

Shed the observer role and experience events first hand

Front 30

Hermeneutics

Back 30

Set out to seek patterns from large volume of detail and interpret them and seek meanings

Front 31

Verstaehen

Back 31

Means understand in german, it is important in phenomenology. (person-centered way of doing research)

Front 32

Proventialism

Back 32

"going native" So over-identify with population that are youare no longer doing any kind of objective research

Front 33

Strengths of Qualitative Research

Back 33

1. Capture real stories
2. Understand nuances

Front 34

Weaknesses of Qualitative Research

Back 34

1. Objectivity
2. Less generalizable than quant.
3. Quant is considered more neutral, but this is not necessarily true.

Front 35

Deductive reasoning

Back 35

Come up with a theory after observing something

Front 36

Probability Sampling

Back 36

any method of sampling that utilizes some form of random selection. In order to have a random selection method, you must set up some process or procedure that assures that the different units in your population have equal probabilities of being chosen. The probability of selection of each member of the population is known, which allows for unbiased estimates of population characteristics.

Front 37

Double Sampling

Back 37

Selecting a representative sample, and then at each address, one conducts a very short interview to determine whether there is a member of the desired group present and if there is such a person, an attempt is made to interview her or him.

Front 38

Kish Method

Back 38

Talking to the first person contacted in a phone survey

Front 39

Simple Random Sampling

Back 39

The ultimate goal of sampling that insures that every person in a population has an equal chance of being selected.

Front 40

Population of inference

Back 40

The group from the population from which you wish the results to be generalizable.

Front 41

Population Frame

Back 41

The list from which you can actually can draw your sample. In the case of budget restraints, time, etc. you have a smaller window in which to select your sample.

Front 42

Sampling Frame Problems

Back 42

1. Missing elements or non-coverage.
2. Duplicate listings
3. Clusters (where numbers have some kind of meaning that would shape your study)
4. Blanks or ineligables coming from a database
5. Disconnected numbers, non-residential numbers, fax numbers.

Front 43

Systematic Sampling

Back 43

Selecting every nth number from a list

Front 44

Stratified sampling

Back 44

Pull out certain populations, but then must report them separately

Front 45

Cluster sampling

Back 45

In cluster sampling, we follow these steps:

divide population into clusters (usually along geographic boundaries)
randomly sample clusters
measure all units within sampled clusters

Front 46

Snowball sampling

Back 46

Asking someone to participate in that study and then finding out from them who is like them and then contacting them (non probability sample)

Front 47

Purposeful sampling and quota sampling

Back 47

Sending out 10,000 surveys and then stopping when you get 500 white women responses. Not representative because they may be the most likely to respond

Front 48

Sampling Error

Back 48

A standard deviation is the spread of the scores around the average in a single sample. The standard error is the spread of the averages around the average of averages in a sampling distribution

Front 49

Things to watch out for when asking survey questions.

Back 49

1. Do not have non-responses in middle of scaled question.
2. Beware of hypothetical questions.
3. Beware of causality.
4. Beware of double-barrel questions (asking 2 questions at once).
5. Include definition as part of a question and include it first.
6. Make sure respondent has all info needed to answer question.

Front 50

Confidence Intervals

Back 50

A range around the sample estimate in which the population estimate is expected to fall with a specified degree of confidence, usually 95 percent in social science research.

Front 51

Internal versus external validity

Back 51

Internal validity has to do with the accuracy of the results. Results could be inaccurate if samples are not selected randomly. External validity has to do with the generalizability of the findings to the population. If the sample selected is only hispanics under the age of 25, then it would be hard to generalize the results to the entire US population.

Front 52

Null hypothesis

Back 52

There is no relationship or effect that is coming out of your experiment. Whatever your finding is, it could have occurred by chance.

Front 53

Characteristics of Qualitative Research

Back 53

Detail-oriented, Relies on complete, detailed description, inducive logic, does not limit inquiry, questions can change, researcher may not know in advance what she is looking for, design emerges and study unfolds, researcher acts as data gathering instrument, subjectives, based on interpretation

Front 54

Types of Qualitative Methodology

Back 54

Case Study, Grounded Theory, Phenomenology, Ethnography, Historical, Focus Groups, Interviews (can be both),

Front 55

Characteristics of Quantitative Research

Back 55

Focus on broad trends, uses numbers, deductive logic, specific questions and variables, guiding hypothesis, questions remain unchanged, researcher knows in advance what he/she is looking for, all aspects of study are carefully designed before data is collected, researchers use tools to collect data, objectives, seeks precise measurements and analysis of target concepts

Front 56

Types of Quantitative Research

Back 56

Surveys

Front 57

Ways to assess validity

Back 57

Descriptive: Whether features described are actually held to phenomena, whether they are possessed to the degree indicated.
Explanatory: Validity of what is being explained and the forces cited. Validity between link of cause and effect.
Theoretical: The validity of theoretical conclusions and the validity of any descriptive claims on which they rely

Front 58

Glacer and Strauss

Back 58

Non engaging in deductive pre-something. Constructing theory from inductive qualitative data through using successful analytic strategies. Starts with inductive data and subjecting them to close scrutiny through specific coding and analytical practices while collecting data. Qualitative data has own logic and could be conducted systematically, offering flexible guidelines.

Front 59

Incentives for survey participants

Back 59

Appeal to do good for their own community,
always use the word "help" as in, it will help them and us,
raffles and prizes are not the most efficient

Front 60

How big should my sample size be?

Back 60

It depends, because the determination of sample size is about the homogenaity or lack thereof of your population, which is the most important.

Front 61

Basic Sample Size Equation

Back 61

ss = Z 2 * (p) * (1-p)

--------------------------------------------------------------------------------

c 2


Where:

Z = Z value (e.g. 1.96 for 95% confidence level)
p = percentage picking a choice, expressed as decimal
(.5 used for sample size needed)
c = confidence interval, expressed as decimal
(e.g., .04 = ±4)

Front 62

Incommensurability of Paradigms

Back 62

Thomas Kuhn: Based on scientific tradition, his idea was that scientific knowledge was not really linear, it was normal science, and evidence builds to the paradigm, this is when a crisis occurs, and this is where a revolution occurs.

Paradigms within a field are incompatible and cannot be reconciled. Denied ability to use both methods.

Front 63

What makes a good research question?

Back 63

Is it feasible?
Is it measurable?
Does it matter?
Is it straightforward?
What is already known about your subject?
It is something everyone already knows the answer to?

Front 64

Type I Error

Back 64

Making the error that there is a relationship where one does not exist.

Front 65

Type II Error

Back 65

You find that there is no relationship where one does exist.

Front 66

Internal Validity

Back 66

Knowing whether your treatment caused the effect, if you are doing some kind of opinion study. Incorporating measurement and causal validity.

Front 67

External Validity

Back 67

Can this effect be generalized beyond your sample (external to)?

Front 68

Social Science?

Back 68

P 207-209: Tips of sampling; generalization rooted in ideal type. Tips on qualitative design. Go on about equations because research is about having confidence in your data. Be careful saying x caused y when this is not true.

Front 69

Questions to avoid in surveys

Back 69

1. Straight yes/no answers
2. Ranking objects
3. Percentages (increases respondent burden
4. Having an other in a ranking means that you probably don't have a well defined question
5. Questions should be mutually exclusive and exhaustive.
6. Don't start with demographic questions, you should already know them.
7. Do not attempt to measure knowledge.
8. Avoid ambiguous words that can be taken in different meanings.

Front 70

Cognititve Pretest

Back 70

Think alouds or probes for pretesting.

Front 71

Behavior Coding

Back 71

Pretesting: Monitoring the interviewer and interviewee. High frequencies of behavior need repair.

Front 72

Response Latency

Back 72

Pretesting: The time it takes a respondent to answer a question.

Front 73

Formal Respondent Debriefings

Back 73

Pretesting: Informs respondent about the purpose of the pre-test

Front 74

Vignettes

Back 74

Pretesting: Hypothetical scenarios that respondents evaluate, how people think about concepts, also diagnoses wording question.

Front 75

Exploratory Research

Back 75

a type of research conducted because a problem has not been clearly defined. Exploratory research helps determine the best research design, data collection method and selection of subjects. Given its fundamental nature, exploratory research often concludes that a perceived problem does not actually exist.
Exploratory research often relies on secondary research such as reviewing available literature and/or data, or qualitative approaches such as informal discussions with consumers, employees, management or competitors, and more formal approaches through in-depth interviews, focus groups, projective methods, case studies or pilot studies. The Internet allows for research methods that are more interactive in nature: E.g., RSS feeds efficiently supply researchers with up-to-date information; major search engine search results may be sent by email to researchers by services such as Google Alerts; comprehensive search results are tracked over lengthy periods of time by services such as Google Trends; and Web sites may be created to attract worldwide feedback on any subject.
The results of exploratory research are not usually useful for decision-making by themselves, but they can provide significant insight into a given situation. Although the results of qualitative research can give some indication as to the "why", "how" and "when" something occurs, it cannot tell us "how often" or "how many."
Exploratory research is not typically generalizable to the population at large.

Front 76

Randomized Experiments

Back 76

Random assignment to a treatment or control group-ensures that the only difference bewteen groups is their exposure to a treatment (differences are almost certain to be caused by control block)

Front 77

Non-experiments

Back 77

Do not use random assignment, do not directly manipulate treatment or have observations and comparison groups that are designed for causality

No control over assignment or who participates. Just observes that some treatment has taken place and some group of individuals participated.

Front 78

Quasi-experiments

Back 78

Attempt to mimic randomized experiments in purpose and structure despite the absence of random assignment (RD and ITS) Only some control of conditions of treatment, sometimes includes who receives treatments

Front 79

Regression-Discontinuity

Back 79

Pretest-posttest two group design. Implies that the same measure is administered before and after some program or treatment. Pretest implies that the same measure is given twice while pre-program implies that before or after measures may be the same or different.

Front 80

Interrupted Time Series

Back 80

involves multiple observations over time on the same units (eg, particular individuals) or on different, but similar, individuals (eg, same community or worksite); requires knowing when an intervention took place in order to compare before and after treatment.

Front 81

Independent Variable

Back 81

Just exists, no influenced by research

Front 82

Dependent Variable

Back 82

Could be influenced by study, what you are measuring

Front 83

Chi-Square

Back 83

When IV and DV are categorical, use this test for statistical signficance

Front 84

T-Test

Back 84

When your IV has only two categories, and your DV is continuous, use this test for statistical signficance

Front 85

ANoVA

Back 85

When your IV has more than two categories, and your DV is continuous, use this test for statistical significance

Front 86

Keys for web surveys

Back 86

1. Light color/dark text
2. No distracting graphics or animations
3. Send a compelling email about incentive and why selected
4. Include personalized links
5. Follow up only with non-respondents
6. Don't allow for multiple completion by the same respondent
7. Ensure confidentiality
8. Test on different browsers

Front 87

Response Rate Calculation

Back 87

In reporting a response rate, you need to also say how you calcuated the response. It should be calculated based on complete interviews, partial interviews, refusals and breakoffs, non-contact, and other.

Look at 1. interview
2.eligible, non-interview,
3. unknown eligbility, non-interview, and
4. non-eligible

Front 88

When to use what method:

Back 88

1. Focus group: little is known about the subject
2. Phone surveys: superficial and broad-based, need to know about respondents, not good with recall questions.
3. Mail surveys: bad if you need to use branching
4. Web survey: like a mail survey, except you can do branching in the background. Is good when you have a population you can link to e-mail that have access to computers.

Front 89

Dilman Methodology of mail surveys

Back 89

Most people follow today.
1. How envelopes are stuffed
2. What postage to use
3. Color of paper
4. Binding of paper
5. Follow up methods
6. Pre-test
7. Ensure data is codable

Front 90

IRB

Back 90

A committee, mandated by federal law to evaluate the potential phsyical or psychological risk of research involving human subjects. (the minimum of what you should be doing).