Core Thiel 1

Front 1

Order of most important

Back 1

Klotz - Qualitative Methods in IR
Klotz / Neumann - Discourse Analysis
Klotz / Gusterson - Ethnographic
Klotz / Checkel - Process Tracing
DPK - Approaches
DPK - Causal Explanation
DPK - Concept Formation
DPK - Comparative Analysis
DPK - Case studies and process tracing
Sil Katzenstein - Beyond Paradigms
DPK / Schmitter - Design of Research

Front 2

Klotz - Case Selection
Cases are of something...

Back 2

- Take into account the universe of possible cases and the logic of comparison implied by research question

Front 3

Klotz - Case Selection
What is the ontology of the case

Back 3

- A case of what?
- What are key concepts? Key dimensions to study?
- Ask, what is the opposite of the concept?

Front 4

Klotz - Case Selection
Identify universe of cases

Back 4

- What is a non-case?
- May be gray zone
Does the study seek to test theories?
Are causal claims made in terms of conditions or mechanisms?
- Causal chain narratives – use stats to identify patterns then select cases to illustrate direction of causal effect
o Downplays contingency and contestation
Are constitutive claims adquately distinguished from causal ones?

Front 5

Klotz - Case Selection
Single case

Back 5

- Plausibility probe
o Be careful don’t have easy case
- Least likely scenario
- Depends on status of theory that underpins it

Front 6

Klotz - Case Selection
Typology

Back 6

- Path between extreme of universal generalization and idiosyncratic contextualization
- Be careful – easy to draw up a table for just about anything

Front 7

Klotz/Neumann - Discourse Analysis
First task

Back 7

show affinities and differences between representations to demonstrate where they belong to the same discourse

Front 8

Klotz/Neumann - Discourse Analysis
Point

Back 8

Capture inevitable cultural changes in representations of reality

Front 9

Klotz/Neumann - Discourse Analysis
What it produces

Back 9

Produces preconditions for action

Constrains what is thought of at all, what is possible, natural

Discourse can’t determine action completely – always more than one possible outcome

Specify bandwidth of possible outcomes

May start with an outcome and demonstrate the preconditions for it happening

Front 10

Klotz/Neumann - Discourse Analysis
Need to have this first

Back 10

Cultural competence

Front 11

Klotz/Neumann - Discourse Analysis
Describe method

Back 11

- Have a wide but manageable range of sources and timeframes
o Look at where the conflict is
o “Monuments” – crossroads or anchor points, such as govt doc
o Some are “canonical” – have broad reception, often cited
- Identify representations that comprise the discourse, taking into account censorship etc that shape text
o Usually contains a dominant representation of reality and some alternatives
o Discourse best for studying situations where power is maintained by aid of cultured and challenged only to a limited degree (hegemony)
• Dominant – will position self as “way things have always been” or hark back to idealized beginning
o Start with the representations themselves – the stories of how things have always been like this or that
o Demonstrate that where the carriers of a position see continuity there is almost always change
o Will carry a memory of its own genesis – how each text builds on previous
• Go back to pioneer texts
- Practices
o Formal and informal
• Legal systems, censorship
• Self-censorship of media
• Cultural artifacts in popular culture
o How analyst is situated in relation to the data
- Uncover layering within discourse
o Not all representations are equally lasting
• Differ in historical depth, variation, degree dominance
• Demonstrate this

Front 12

Klotz/Gusterson - Ethnographic

Back 12

- Participant observation
o Spending time with locals while also taking notes
o Sustained contact, build trust
o See for self
- Semi-structured interviews
o Difficult to have precise comparability
o Core questions ask everyone, then let them go off

Front 13

Klotz/Checkel - Process tracing
Mechanism

Back 13

- Analytical level below more encompassing theory
- Increase theory’s credibility by providing more fine-grained explanations
- Set of hypotheses that could be the explanation for some social pheno, the explanation being in terms of interactions between individuals and others or between individuals and some social aggregate

Front 14

Klotz/Checkel - Process tracing
What it does

Back 14

- Identifies a causal chain that links ind and dep variables
- Nuts and bolts for mech-based accounts of social change
- Trace operation of causal mech at work in given situation
- Data – qualitative
o May include historical memoirs, expert surveys, interviews, press accounts, documents
- Strong on questions of interactions
- Weaker at establishing structural context
- Takes a lot of time and data
- Compatible with and complementary to stats, analytic narratives, formal modeling, case studies and content analysis.

Front 15

Klotz/Checkel - Process tracing
Good

Back 15

o Minimize problem of pet theories
o Answers how much data is enough
o Brings mechs back in
o Promotes bridge building

Front 16

Klotz/Checkel - Process tracing
Bad

Back 16

o Proxies are a pain
o Takes a lot of time
o How micro to go?
o Non parsimonious theories
o Missing causal complexity
- Ugly
o Loses big picture
o Loses ethics, normative dimension
o Not so empiricist

Front 17

DPK: Approaches
Kuhn

Back 17

Kuhn - Structure of Scientific Revolutions
- Rely upon paradigm that defines
o What to study (relevance)
o Why to study (hypotheses)
o How to study (methods)
- Paradigm – accepted by whole community of science in discipline

Front 18

DPK: Approaches
KKV

Back 18

- Goal is inference
- Procedures public
- Conclusion uncertain
- Content is the method

Front 19

DPK: Approaches
Ontology

Back 19

existence of real and objective world

what we study, object of investigation
- How world fits together
- Nominalists- categories are arbitrary
- Realists – categories are there to be discovered

Front 20

DPK: Approaches
Epistemology

Back 20

possibility of knowing this world

how we know things; nature, source, limit knowledge, not belief

Front 21

DPK: Approaches
Positivism

Back 21

Objective; realism
Reality is easy to capture
Scholar/object separate
Natural laws/causal

Front 22

DPK: Approaches
Post-Postivist

Back 22

Objective reality
Reality not easy to capture
Knowledge influenced by scholar
Probabilistic law

Front 23

DPK: Approaches
Interpretivist

Back 23

Objective and subjective
Reality not separate from subjective
Role of researcherContextual knowledge

Front 24

DPK: Approaches
Humanistic

Back 24

Subjective, science of spirit
Reality not knowable
No objective knowledge
Empathetic knowledge

Front 25

DPK: Approaches
Goal of postivist

Back 25

aim to single out causal explanation or assumption of cause – effect on relationship between variables
o Structured and context-free
o General, universal laws

Front 26

DPK: Approaches
Inductive

Back 26

o Inductive (pragmatism, behaviorism) –deriving generalizations from specific observations in large number of cases

Front 27

DPK: Approaches
Hypothetico-deductive

Back 27

start with theory, generate hypothesis, test

Front 28

DPK: Approaches
- Interpretive/qualitative

Back 28

understanding events by discovering meanings humans attribute to their behavior and external world
o Not laws, human nature and motivations
o Theory important, not always before research
o Cases not broken down into variables
o Generalization through assigning cases to classes
o Context most important
o Explanations for social outcomes, not universal laws

Front 29

DPK: Approaches
How to decide method to use

Back 29

1. Framing of research question
2. # cases analyzed and criteria for selecting
3. Language – variables or holistic
4. Relation of researcher to object – distance, immersion
5. Value neutrality

Front 30

DPK: Causal Explanation
What is it

Back 30

Answering “why” questions by seeking to identify antecedent factors responsible for event

Assumption about how the world works – regularities, order, objective reality

Front 31

DPK: Causal Explanation
What do strive for

Back 31

Strive for regularity, generalization, reliability, replicability, validity, prediction, parsimony

Front 32

DPK: Causal Explanation
What is regularity/generalization

Back 32

relationship between concepts. Hypotheses

Front 33

DPK: Causal Explanation
What is reliability and replicability

Back 33

steps hypothesis tested

Front 34

DPK: Causal Explanation
What is validity

Back 34

do indicators measure what’s intended

Front 35

DPK: Causal Explanation
What is prediction

Back 35

conclusion from confirmed hypothesis in other cases

Front 36

DPK: Causal Explanation
Deductive

Back 36

explanation of event derived from theoretical hypothesis. Survey theories for best fit.
- Internally consistent – hypotheses of a theory do not contradict
- Logically complete – derive from consistent assumptions
- Falsifiable – testable propositions; unit of analysis, time, space, value-free
- Objective not to describe minutely a reality with details
o Most important aspects of situation
o May have to reduce scope
o Deterministic statements, if a then b, rare in SS

Front 37

DPK: Causal Explanation
Inductive

Back 37

start from empirical research of phenomenon
- See from actor’s perspective
- Systematic patterns in data
- Form hypotheses that can be linked and tested

Front 38

DPK: Causal Explanation
How to assess multiple explanations

Back 38

- Competitive relationship – which theory’s outcome is observed
- Complementary – respective domains under which 2 theories hold are identified
- Sequencing – one theory temporarily depends on the other to explain an outcome
- Subsumption – one theoretical account can be incorporated into another

Front 39

DPK: Causal Explanation
unit homogeneity

Back 39

conditional independence, constant effect

Front 40

DPK: Causal Explanation
endogeneity

Back 40

ind. Variable may be influenced by dependent – BAD

Front 41

DPK: Causal Explanation
multicollinearity

Back 41

explanatory factors should not be derived from each other – BAD

Front 42

DPK: Concept Formation
What is comparison

Back 42

Comparison = explanation
How different factors led to similar or different outcomes

Lessons in developing and understanding comparisons
- Is like being compared
o Are they the same thing?
o Are so different comparison meaningless?
o When like cannot be compared – chalk and cheese

Front 43

DPK: Concept Formation
What is concept stage

Back 43

Focus on crucial stage where initial idea/hypothesis translated into research design

Front 44

DPK: Concept Formation
Sartori's rules of concept formation

Back 44

- Three levels – high, medium, low
- Degree of generality/abstraction related to range of cases covered
- More concrete, narrower the range of cases
- Abstract – wider range
o Lighten to make concept travel further
o If fail to do this – run risk of “concept stretching” – beyond sensible limits to fit range
- When move from wide-ranging to narrow weigh concepts heaviliy and concretize
- See where stand on ladder of abstraction and when go up or down
- Concept is “basic unit of thinking”
o Have concept of A – distinguish from not A

Front 45

DPK: Concept Formation
Question to address

Back 45

First address “what is” question for dependent and independent variable
Later “how much” question

Specify nature of objects of research
Difficult in practice

Front 46

DPK: Concept Formation
Negative identification

Back 46

Sometimes easier to say what concept is not

Front 47

DPK: Concept Formation
More and less comparisons

Back 47

More and less comparisons should only be within same class/category

1. Concept defined
2. Classed
3. Quantification/measurement

Front 48

DPK: Concept Formation
Classification

Back 48

Principle of per genus et differentium
- Classed by genus
- By particular attributes that make it different from other objects in class
- A taxonomy

Classification
- Each is exclusive
o Some item can’t be in more than one
- Exhaustive
o Can’t leave one out because doesn’t fit

Front 49

DPK: Concept Formation
Typology

Back 49

- 2 or more class combined
- Exclusive and exhaustive
- Moving toward explanation
- Lijphart – classed types democracy by elite behavior
o Constitutional demo differs from centralized demo because of patterns of elite behavior

Front 50

DPK: Concept Formation
To compare across class

Back 50

moving up ladder to more abstract
- Regime over demo and nondemo

Front 51

DPK: Concept Formation
Sartori's ladder of abstraction

Back 51

TOP – 30,000 foot view
- One/two properties, wide range, abstract – “regime”

BOTTOM
- Concepts defined by many properties, limited range – “democracy”

The more cases – fewer properties can look at – abstract

Extension – range of cases it covers/denotation
Intension - # attributes/properties/connotation

TOP – broad definition, minimum intension, maximum extension
As you go down, more intensive, less extensive

Middle layers most interesting

Front 52

DPK: Concept Formation
Lessons from ladder

Back 52

Lessons from Sartori’s ladder of abstraction
1. Comparison across classes affected by going up ladder
2. Concept as data containers with different attributes/properties
a. Gerring – aspects
i. Event to be defined
ii. Properties
iii. Label that covers both
iv. Good concept aligns all 3

Front 53

DPK: Concept Formation
Radial categories – Collier and Mahon

Back 53

- Begin with single primary category, like ideal type
o Contain all possible attributes
- Secondary category
o Take single feature from primary alone or with one other

Front 54

DPK: Concept Formation
Wittgenstein – family resemblance categories

Back 54

Related concepts but no single core shared
Turns ladder on head
More attributes = more cases

Front 55

DPK: Comparative Analysis
3 approaches in comparative

Back 55

x- Experimental
o 2 identical groups, control
- Statistical
o Create subsamples when variables kept constant
o Need large samples
- Comparative method
o Low # cases – 2 to 20
o Aims at establishing general empirical relations between 2 variables
o Preferred when investigating institutions or macro phen.

Front 56

DPK: Comparative Analysis
Different aims of science

Back 56

x- Statistics – build law-like propositions
- Durkheim – sociology as science must favor general over detail
- Estimate average effects of independent variable
o “Effects of causes”
- Case-oriented strategy
o Attention to each case as interpretable whole
o Understand complex unity not establish relationship between variables

Front 57

DPK: Comparative Analysis
Statistics

Back 57

based on search for concomitant variations (whether ind or dep variables vary together) with regression as main instrument for measuring causal inference

Front 58

DPK: Comparative Analysis
Agreement

Back 58

– if 2 or more instances of a phen have only one of several possible causal circumstances in common, cause of phen is one circumstance present in all analyzed instances

Front 59

DPK: Comparative Analysis
Difference

Back 59

when 2 or more cases have different values on a phen, look for one circumstance in which they differ

Front 60

DPK: Comparative Analysis
Ferejohn

Back 60

- External explanations – agents doing things because of some configuration of causal influence
- Internal explanations – identify reasons for actions
- Explain is really to justify
- Example - # car burnings during French riot vs rioter motives

Front 61

DPK: Comparative Analysis
Durkheim

Back 61

Durkheim – inductive reasoning on empirical data to reconstruct different social species
- Properties of a social species influences the course of the social phen developing in it
- Search for permanent cause focuses on explanation with patterns of relations among abstract variables that are trans-historical
- An effect cannot have different causes in different contexts

Front 62

DPK: Comparative Analysis
Case-oriented

Back 62

- Generalizations that are temporarily limited to cases studied – wider relevance controlled through further research
- Macro-units are unique and complex
- In qualitative, historical comparison
o Explanations are genetic (based on reconstruction of origins of an event)
o Generalizations are historically concrete
o Theory from ideal types
• Abstract model with internal logic
• Real complex cases can be measured against
• Offers guidance to construction of hypotheses
• Facilitate empirical analysis
• Enable limited generalization about historical divergence
• Beyond uniqueness of events but not to degree of generalization of natural science

COR – case often not determined at start; coalesce during research. “What is this a case of?”

- COR – say increase in # cases increase # third variables (external to hypothesis)

Front 63

DPK: Comparative Analysis
Variable-oriented

Back 63

VOR – homogeneity of units of analysis stated at beginning when defining population of case

Large N

Front 64

DPK: Comparative Analysis
Case selection

Back 64

- VOR – constrained by statistical rules
o Random samples – usually can’t do in polis ci
o Don’t select on dependent variable
- COR – may choose similar cases
o Positive cases – where phen is present
o Relevance
o Deviant
- Both – similar criteria
1. Appropriate to kind of theoretical problem posed
2. Relevant to phen studied
3. Empirically invariant with respect to classificatory criteria
4. Reflect degree of available data for unit
5. Decisions to select and classify units of analysis based on standardized and repeatable procedures

Front 65

DPK: Comparative Analysis
Gerring, trade offs in selection

Back 65

- Plenitude - # cases; large sample = higher confidence
- Boundedness – range of generalizability. Inclusion of relevant cases. Exclusion of irrelevant
- Comparability – similarity among cases on same relevant dimensions
- Independence – autonomy of units – if linked risk studying same unit twice
- Representativeness – capacity of sample to reflect properties of entire pop
- Variation – range of variables registered on relevant variables
- Analytical Utility – with reference to theory to test or sci approach chosen
- Replicability

Front 66

DPK: Comparative Analysis
Similar vs different cases

Back 66

- Most-Similar Systems
o Reduces # disturbing variables
o Ignore common variables
o Focus on variables that are different
o Cannot go beyond middle range theory
o Never similar enough to null influence of environment
o Confirmation of hypothesis
- Most-Different System
o Check if correlation holds true no matter in which country
o Usually individual level
o Random sample of world pop
o Generalizable
o Risk ending up with hypothesis that explains little

Front 67

DPK: Comparative Analysis
Use of time

Back 67

- Historical approach good for cases context-bound
o Long-term processes for ‘internal’ interpretation
- VOR – needs less historical depth
o Periodization – same country in different time periods treated as different cases
- Bartolini – explicit diachronic – based on collection data at several points in history
o Development of some characteristic over time
- Cross-national diachronic – higher level generalization
- Sewell – teleological temporality – compare different historical paths, less to more
o Experimental temporality – compare paths; ie, demo vs nondemo
o Eventful temporality – transfer of structure
• Events – relatively rare subclass of happenings that transform structure
o Eventful sociology – social pressures are inherently contingent, discontinuous, open-ended
• Structures constructed by social action
• Social systems – shaped by creativity of humans
- Mahoney – narrative – address phen such as revolutions as product of unique, temporally ordered and sequentially unfolding events that occur within cases
o Indepth ideographic knowledge
- Periodization
o Define temporal units that determine variance
• Time points, regular interval
• General character of periods
• Significant according to our model
• Account for main changes in dep variable as well as other operative variables

Front 68

DPK: Case studies and process
Definition of case study

Back 68

Case study – research strategy based on an in depth empirical investigation of one or small # phen to explore configuration of each case and find features of larger class of similar phen by developing theoretical explanation
- Case is not a unit of analysis or observation as data. It is theoretical category.
- Case is not a priori spatially delimited
- Does not have to be contemporary
- Data can be qual or quan

Front 69

DPK: Case studies and process
Purpose of case study

Back 69

- Develop/evaluate theory
- Formulate hypothesis
- Explain phen with theory/causal method
- Can be combined with stats or computer sim

Front 70

DPK: Case studies and process
Types of cases

Back 70

- Descriptive (configurative ideographic)
o Systematic description of phen with no explicit theoretical intention
o Shed new light, explore little known subject
- Interpretive (disciplined configurative)
o Use the framework to provide explanation of particular cases, which can lead to evaluation/refinement of theory
- Hypothesis – generating and refining (heuristic)
o Generate new hypothesis inductively or refine existing
o Clarify meaning of variables
o Suggest alternative causal mechs
o Identify overlooked interaction effects
o Deviant case useful
- Theory-evaluating
o Assess whether existing theory accounts for processes/outcomes

Front 71

DPK: Case studies and process
Bachelard's epistemology of science

Back 71

- Why use this
o Epist. Categories we use affect how we evaluate
o Different scientific practice inseparable
- His main concern – creation, revision, rejection of sci theories
- Reconstruct philosophy in practice of scientists to identify their applied rationalism
- Different epist acts at core of sci practice cannot be separated
o A data collection is only as good as the theory it tests
- A sci fact is conquered, constructed and observed

Front 72

DPK: Case studies and process
How a case is made

Back 72

Cases are not waiting out there to be studied
- Researcher makes into a case – define boundaries, object of study
- What is this a case of?

Front 73

DPK: Case studies and process
Within case analysis

Back 73

- Congruence method
- Typological theory
- Process tracing
o Procedure for identifying steps in a causal process leading to outcome of a given dep variable of a particular case in a historical context
o Researcher assess a theory by identifying causal chains that link ind and dep variables
o Widespread
o Have used for
• Discover a causal mech and show that a posited underlying mech connecting causal and dep variable exists
• Demonstrate conjunction and temporal sequence of variables
• Increase # observable implications a theory predicts

Front 74

DPK: Case studies and process
What is PT

Back 74

PT is not a pure narrative
- Focused, only certain parts of a phen; some info/characteristics lost
- Structured – develop an explanation based on theory framework
- Goal to provide a narrative explanation of a causal path leading to an outcome

Case studies have important policy component
PT helps transition from recognition causal patterns – solutions

Front 75

DPK: Case studies and process
Challenges of case and PT

Back 75

- Reliance on pre-existing theories (also theory development)
- Assumption each case can be treated autonomously and distinct (may be linked, embedded)
- Need for empirical data
- Pitfalls of cognitive biases
o Confirmation bias – confirm belief, ignore contradictions (ask what else it can be)
o Results of PT might be consistent with too many theories – overdetermination
o Ignoring negative evidence – what doesn’t happen

Front 76

Sil Katzenstein Beyond Paradigms
Argument

Back 76

possible and necessary for scholars to resist assuming one or other research tradition is inherently superior for solving all problems. Going beyond paradigms does not mean ignoring work done by adherents of paradigms. Means exploring substantive relationships and revealing hidden connections among elements of parardigm-bound theories with an eye to generating insights that bear on policy debates. Requires alternative way of thinking about the relationships among assumptions, concepts, theories, organization of research and real world problems – “analytic eclecticism”. More than call for pluralism. About making connections among analyses normally found in separate paradigms.

Front 77

Sil Katzenstein Beyond Paradigms
Kuhn

Back 77

Kuhn. Challenged Popper – falsification as basis for progress. Kuhn – history of sci as sequence of discrete periods of normal sci separated by relatively short episodes of revolutionary sci. Normal sci – dominant paradigm. Rev sci occurs when communitieis frustrated by increasing anomalies focus on new problems and approaches. Paradigms are incommensurable, can’t integrate.

Front 78

Sil Katzenstein Beyond Paradigms
Lakatos

Back 78

make room for a more pluralistic view of coexisting sci communities each with own research program. Hard core, protective belt of auxiliary assumptions and positive and negative heuristics which protect core assumptions from being challenged. Staying power can be extended through defense, refinement. Progressive/degenerative research program.

Both face limitations capturing debates. In IR there have been enduring fractal distinctions (ie, agent vs structure)

Front 79

Sil Katzenstein Beyond Paradigms
Laudan

Back 79

This book uses concept of research tradition by Laudan. Long enduring epistemological commitments which produce discrete research traditions. Each consists of
- set of beliefs about what sorts of entities and processes make up domain of inquiry
- set of epistemic and methodological norms about how domain is to be investigated, how theories tested, how data collected, etc.
- Coexist, react to each other
- Not succession of paradigms

Front 80

Sil Katzenstein Beyond Paradigms
Eclectic

Back 80

any approach that seeks to extricate, translate and integrate analytic elements of theories or narratives that have been developed in separate paradigms but that address related aspects of substantive problems that have scholarly and practical significance

Front 81

Sil Katzenstein Beyond Paradigms
Markers of eclectic scholarship

Back 81

Markers of eclectic scholarship
- Open-ended problem formulation encompassing complexity of pheno, not intended to advance or fill gaps in paradigm-bound scholarship
- Middle-range causal account incorporating complex interactions among multiple mechs and logics drawn from more than one paradigm
Findings and arguments that pragmatically engage both academic debates and the practical dilemmas of policymakers

Front 82

DPK Schmitter Design of Research
Cycle

Back 82

Idea – topic – conceptualization – hypothesis generation/normative clarification – case selection – proposal writing – variable operationalization – measurement of indicators – test for association – causal inference – self-assessment

Front 83

DPK Schmitter Design of Research
Topics of research

Back 83

Topics of research come in two guises
- Projections
o Confident in approach and methods – new cases or greater precision
- Puzzles
o Something is deficient in the way a topic has been handled

Front 84

DPK Schmitter Design of Research
Optimizing topic choice

Back 84

Optimizing topic choice
- Choose a topic care about
- Topic that interests other social scientists even outside field
- Specify temporal, spatial, cultural boundaries that make research feasible but not trivial
- Acknowledge initial inspiration for topic and personal pref for outcome without apology
- Never justify selection on grounds it has been ‘underexplored’ and do not ignore, trivialize or dismiss what’s been written on the topic
- Reach back into theory for relevance and avoid being manipulated by academic fad
- Listen to advisor/peers – but topic belongs to you

Front 85

DPK Schmitter Design of Research
Potential fallacies: topic choice

Back 85

Potentially damaging fallacies
- Fadism – Topic is being favorably discussed now, so you’ll be less criticized and more likely to find a job
- Wishful thinking – topic has already produced well-publicized results, so if you research it, findings be taken more seriously
- Ambulance chasing – because topic is in crisis will have greater access to data and public more interested
- Presentism – whatever you find associated with some topic today was there in past and will be there in future
- Standing on the shoulders of the past giants – giants might not have been looking at same thing or same direction

Front 86

DPK Schmitter Design of Research
conceptualization

Back 86

Conceptualization
- Translating the words that surround a topic into variables
- All social and poli research is part and parcel of ‘state of theory’ prevailing at that time
- Distinguish between
o Operative – play some discernible role in explanation of outcomes
• Explicans – that which does the explaining OR
• Explicandum – that which is explained
o Inoperative – PRESENT VARIABLES CAN TAKE ON DIFFERENT VALUES BUT NOT EXPECTED to produce difference in outcome

Optimizing conceptualization
- avoid references to specific persons, countries or cultures with upper case names by using only lower case variables to describe them
- Ok to start with hunch, but then identify the generic theory it’s embedded in, switch to it slandguage
- Avoid multicollinearity – closely associated vars
- Make operative, inoperative and constants explicit
- Exercise caution over long periods of time – meaning to actors may change
- Restate argument several times and make it more and more concise

Front 87

DPK Schmitter Design of Research
Potential fallacies:conceptualization

Back 87

Potentially damaging fallacies – concept
- Obscurantism – if cloak in abstractness no one will notice you’re just describing what happened
- Attribute-ism – the more attributes the more significant
- Concept stretching – concepts are valid across all times and places
- Isolation – preferred variable so important it can be measured alone
- Novelty at any price – inventing novel concepts in a bid for originality
- Arbitrariness – makes no difference what concepts I use
- Consensual-ism – if everyone is using some concept you should feel safe to do so too

Front 88

DPK Schmitter Design of Research
Formation of hypotheses

Back 88

Formation of hypotheses
- Perfectly possible to not assign “if… then” to a variable relationship – discovering and not proof
o Recent occurrence
o Only characteristic of small number of cases
o Incite strong emotions
o Fall between disciplines

Optimizing choices – hypo
- Ensure presumed cause is independent of presumed effect
- Specify intervening conditions or prevailing constants that must be present for hypo relation to produce effect
- Prepare to recognize and deal with equifinalities
- Have three hypo to test – positive, negative, null (most probable)
- Differente bet vars that are necessary, sufficient and merely helpful

Front 89

DPK Schmitter Design of Research
Potential fallacies: hypotheses

Back 89

Potential fallacies – hypo
- Scientism – if vars not organized into hypo with ind and dep var research is not scientific
- Fear of failure – if hypo disproved, no contribution to knowledge
- Infinite regress – makes no difference where you choose to break into the var relationship chain

Front 90

DPK Schmitter Design of Research
Selection of cases

Back 90

Selection of cases

- Normal that only some subset of units will enter analysis
- Have to choose number and identity of those to include and criteria to select
- Researcher does not select individual cases but ‘configurations of variables’ that co-habit same unit

Optimizing choices – selection
- If not trying to cover entire universe, consider selecting a sample of cases randomly
- When not randomizing, choose case based on relation to ind var, not dep
- Ensure cases represent a wide range of scores on ind var
- If have to select on dep var keep in mind potential bias
- Can use nested strategies of selection
- Always prefer the lowest level of aggregation, since can move upwards in scale if needed
- Ask first “What is this a case of?” – then can know which units are eligible

Front 91

DPK Schmitter Design of Research
Potential fallacies: case selection

Back 91

Fallacies
- Notoriety – just because a case has been prominent in public discussion, will be more interesting/better received
- Numbers – always better to have large numbers of cases
- Cruciality – because a unit is an outlier it will be a crucial case that will test the causal association
- The illusion of control – selecting cases because they seem to share general characteristics
- Contemporaneousness – in units chosen for comparison within same time frame actors must have similar awareness of relevance of common vars and be capable of acting on them simultaneously – when may be at different points in longer cycle or different time schedules
- Imitation – actors are aware of what others are doing and will learn from others’ success/failure – may in fact be unaware

Front 92

DPK Schmitter Design of Research
Writing the proposal

Back 92

Writing the proposal
- Optional but highly desirable
o If part of established tradition may not be needed/accepted
- Obtaining research funding
o Rarely look at conformity between proposal and actual research
- Give researcher a chance to reflect on choices made

Front 93

DPK Schmitter Design of Research
Operationalization of variables

Back 93

Operationalization of variables
- Early stage – adopt attitude that any var can be operationalized
- Validity – do observations proposed accurately reflect and capture meaning of concepts chosen for explanation?
- Needs to be replicable

Optimizing choices – op
- Pay close attention to correspondence between initial concepts and proposed indicators by comparing to research by others on similar topics
- Be wary of variable/empirical applied routinely over time and across units to measure diff concepts
- Make sure concept and its indicator are applied to same level of analysis/level of abstraction
- When can, use alternative operationalizations and multiple potential indicators and triangulate among them
- Better to use unobtrusive indicators – actors have less chance to respond to your request for info
- Various ways of assessing validity of indicators

Front 94

DPK Schmitter Design of Research
Potential fallacies: op variables

Back 94

Fallacies
- Availability – indicator exists and has been used successfully by others, therefore it will be valid for your topic
- Operationalism – decide to include in analysis only variables for which know a valid indicator exists
- Mimetism – X got away with using data to indicate a concept similar to yours even when drawing on a diff theory; you can use it too
- Ignorance of the uncertainty principle – if op a var by intruding on real world of respondent, can ignore possibility answer will be contaminated

Front 95

DPK Schmitter Design of Research
Measurement

Back 95

Measurement
- At this point, choices will be dictated mostly by choices already made
- Go with the flow
- Use existing techniques of observation and indicators for vars
o Also provides element of internal quality control
o If inventing, must make especially strong justification
- Discussion dominated by quan vs qual
o Quan said to be more reliable, accurate, agreement across units, more compatible with diff ways of testing for association
- Stage best suited for serendipity
o Learning from the research process itself in ways that can feed back to previous choices and introduce improvements

Optimizing choices
- Routinely test for reliability of indicators – alternative sources of data
- Opt for quan over qual if possible – tech advantages are considerable
- Always opt for highest, most informative level of measurement – can later switch to lower level
- Make instructions for assignment of quan scores or qual labels transparent and complete for replicability
- When working at macro level, most vars will have multiple components – pay attention to how aggregated
- When gathering info over time check that changes not due to modifications of instruments of observation
- Many measurement devices only pick up large scale changes in vars
- Estimate before gathering data where errors are likely to come from
- Catch self in adjusting data to make it fit better with hypothesis

Front 96

DPK Schmitter Design of Research
Potential fallacies: Measurement

Back 96

Fallacies
- Compositeness – many concepts are complex and multidimensional and can only be measured by complex indicators
- Longevity – better to use indicator around for some time in a variety of research settings
- Clarity – preferable to give each var specific score even if nature of its concept is fuzzy
- Reification – what measuring is same as what conceptualized which is same as what actors perceive

Front 97

DPK Schmitter Design of Research
Test for association

Back 97

Test for association
- Vars can be associated in diff ways
o Direction – positive, negative or null
o Strength – how much one var affects another
o Significance – likelihood fit could be by chance
o Time, timing and sequence of fit
- Most powerful means for testing fit among variables – storytelling in chronological order

Optimizing choices
- eyeball data and form visual impression of what’s going on
- try diff tests of fit, starting with simpler
- manipulate data by dividing into subsamples – figure out where intervening vars are
- sensitive to extreme cases – try eliminating
- be aware of time dimensions and test successive cross-sections
- test will be more convincing if put more effort into falsifying initial hypo
- don’t ignore deviant cases

Front 98

DPK Schmitter Design of Research
Potential fallacies: test for association

Back 98

Fallacies
- Spuriousness – have found a close association bet two vars and report without considering that if a third var introduced might explain this variation
- Contingency – associations are strong but only when certain usually unspecified contextual vars are present
- Curve-fitting – smooth distribution by transforming raw data or eliminating outliers
- Anachronism – whatever associations within a time period will fit whenever
- Ad-hocracy – each case can be used to explain away observed deviations

Front 99

DPK Schmitter Design of Research
Causal inference

Back 99

Causal inference
- Most hazardous and most rewarding stage
- Widest range of choices
- Many researchers exit before this point
- Controversy over generalizability
- Case selection plays a role
o Single case study rarely convincing for gen
- Most secure way to get respect – put under a covering law
Optimizing choices
- add alt explanatory vars to discover if original fit maintained
- probe data by subtracting subsets of cases
- don’t anchor inferences by relying too much on a single association at the expense of lesser
- don’t privilege findings that were easier to document
- try to predict analogous behaviors in a diff sample know nothing about or predict future performance of units studied

Front 100

DPK Schmitter Design of Research
Potential fallacies: causal inference

Back 100

Fallacies
- Triumphalism – have made a significant finding; work is over; even though it could have been the result of some other var
- Pago-Pago-ism – whenever think found something that applies everywhere there will be someone who will point out it does not
- Exceptionalism – study a particular topic in one country because it’s exceptional but then try to gen to all
- Cross-level replicability – associations will replicate themselves at other levels of aggregation
- Cognitive dissidence – vars that ‘should’ not go together still seem to be associated – still draw inference by excluding
- Temporal proximity – give greater prominence to recent associations

Front 101

DPK Schmitter Design of Research
Self-Assessment

Back 101

Self-Assessment
- Once arrived at exit point, be best critic of own work
- Anticipate all objections – return and make corrections or show aware of defect
- No research is perfect