Research Design

Front 1

Research Design

Back 1

* A plan of what data to gather, from whom, how and when to collect the data, and how to analyze the data obtained.
* a plan outlining how Information is to be gathered that includes identifying the data gathering method(s), the instruments to be used/created, how the instruments will be administered, to whom, and how the information will be organized and analyzed.

Front 2

McGrath’s Circumplex: 3-Horned Dilemma

Back 2

Looking at Actors emitting
Behaviors in Contexts
Want
(a)Generalizability across Actors - populations
(b) Precision in Control and measurement of Behavior, and
(c) Realistic Contexts for observing actors behaving

Front 3

McGrath’s Key Point

Back 3

All research methods are flawed but each is flawed differently. There is NO PERFECT study.
Importance of triangulation to compensate for limitations of each method.
Key question is
* How to combine multiple strategies since it is not
possible to do an unflawed study
* Distinction between inherent flaws of method and using the method badly. Former is a limitation, latter is bad research and unacceptable

Front 4

Four Research Dilemmas (Tradeoffs)

Back 4

(i) Scope vsprecision
(ii)Randomization “noise” vs internal validity
(iii)Standardization vs generalization
(iv)Number of conditions and number of subjects per condition

Front 5

Research Dilemma Levels

Back 5

* Research strategy level (3-horned dilemma)
* Research design level
* Research methods level

Front 6

Research design level dilemmas

Back 6

- Replication vs partitioning
- Hold constant, manipulate, measure, match
- Randomization

Front 7

Research methods level dilemmas

Back 7

* Reliability and validity
* Measures classified along (a) obtrusive vs. unobtrusive
and by (b) researcher vs actor vs recorded in past
* Methods of data collection
* Use multiple methods selected from different classes with
different vulnerabilities to trascend one another’s
methodological weaknesses (triangulation)

Front 8

How do you choose a research design?

Back 8

* Fit with research question:
-Importance of context to phenomenon
- Process (how) vs. variance (what) question
- Pragmatic considerations (access, cost, etc)
* Fit with maturity in existing knowledge about phenomenon (nascent vs mature). Case study more appropriate for more nascent fields of study

Front 9

Dangers of methodological misfit

Back 9

* Unable to convincingly answer research question.
* Do not make a contribution

Front 10

Edmonson & McManus's Four Key Elements of a Field Research Project

Back 10

* Research Question
* Prior Work
* Research Design
* Contribution to literature

Front 11

Research question

Back 11

● Focuses a study
● Narrows the topic area to a meaningful, manageable size
● Addresses issues of theoretical and practical significance
● Points toward a viable research project—that is, the question can be answered

Front 12

Prior work

Back 12

● The state of the literature
● Existing theoretical and empirical research papers that pertain to the topic of the current study
● An aid in identifying unanswered questions, unexplored areas, relevant constructs, and areas of low agreement

Front 13

Research design

Back 13

● Type of data to be collected
● Data collection tools and procedures
● Type of analysis planned
● Finding/selection of sites for collecting data

Front 14

Contribution to literature

Back 14

● The theory developed as an outcome of the study
● New ideas that contest conventional wisdom, challenge prior assumptions, integrate prior streams of research to produce a new model, or refine understanding of a phenomenon
● Any practical insights drawn from the findings that may be suggested by the researcher

Front 15

Field Research as an Iterative Cycle (Edmondson & McManus 2007)

Back 15

Front 16

What’s the most appropriate Research Design to Use?

Introduction of a new quality control process innovation in an organization.

How would you test whether it has any effect on time to completion and number of errors?

Back 16

Since its a contemporary real work phenomenon and a new process, a case study would probably be the most appropriate research design.

Front 17

Research Design?

Examining whether certain interventions successfully disrupt workplace habits with respect to IT.

How would you study?

Back 17

Since its a contemporary real work phenomenon and a new process, a case study would probably be the most appropriate research design.

Front 18

Research Design?

You would like to study the effect of TMT-CIO relationship on strategic alignment.

Back 18

Survey, so that you could gather a lot of data from different organizations to enhance generalizability

Front 19

Research Design?

You would like to examine whether content-personalization or collaborative personalization are more effective.

Back 19

Experiment would probably be the most effective research design here because this would be a behavioral analysis and may difficult to measure cognitively. However, to improve convergent validity and content validity, a follow up questionnaire or in-process question-answer exercise would probably be helpful to determine why people behaved the way they did.

Front 20

You would like to see how mobile devices influence work-life balance and productivity

Back 20

Survey or interview because this is not a

Front 21

You would like to examine how relationships between vendors and suppliers influence organizational value of the relationship.

Back 21

Finish reading about research designs before you answer these

Front 22

Types of Research Design

Back 22

Front 23

Research Design Notation

Back 23

Front 24

Single Group Validity Threats

Back 24

Front 25

Research Design Notation

Back 25

Front 26

How to reduce multiple group threats

Back 26

* Rule out single-group threats to validity only if control and experimental groups are comparable. To do so make them random. Quasi-experimental is the next best thing

Front 27

6 major Selection Threats

Back 27

The only real threat is selection
- Selection-history threat
- Selection-maturation threat
- Selection-testing threat
- Selection-instrumentation threat
- Selection-mortality threat
- Selection-regression threat

Front 28

2 major threats to multiple groups

Back 28

* Selection threats - occurs as a result of non-equivalent groups due to non-random assignment. Similar to single-group threats except they are brought on due to selection bias due to non-random selection. All are preceded by selection.
* Social Threats - Occur when control group doesn't feel it was treated fairly

Front 29

4 Social Threats

Back 29

* Diffusion of imitation of treatment
* Compensatory rivalry
* Resentful demoralization
* Compensatory equalization of treatments

Front 30

Types of Research Design

Back 30

Front 31

Research Design Diagram

Back 31

Front 32

Qualitative Measures

Back 32

Types of Data Collection Approaches
* In-depth interviews
* Direct observation
* Written documents

Front 33

Unobtrusive Measures

Back 33

* Indirect Measures
-E.g. wear and tear museum example
* Content Analysis of existing text documents
-E.g. analyze outsourcing Contracts
* Secondary Data Analysis
- E.g. financial data