Research Methods Final

Front 1

Measurement equivalence

Back 1

-The construct the same in both places? Anything over time development, cultures, etc.
-Whether a given measure is interpreted in a conceptually similar manner by respondents who represent different genders or cultural backgrounds, or over development
-Ex: Test an anxiety scale with young children and older adolescents and elderly adults
-Ex: BDI -- Remove the somatic items from the BDI because elderly individuals may not respond in the same way

Front 2

Reliability

Back 2

Temporal Stability:
-Test-Retest Reliability
-The stability of obtained scores over a specified time
-Can be reflected by a correlation score
-Important to stable constructs and traits -- NOT states
Internal Consistency:
-The degree of consistency of the items or elements; correlation of items with each other
-Cronbach’s alpha
-Can be reflected by split-half reliability or odd-even reliability
Inter-Rater Reliability:
-When two people are consistent when measuring the same thing with the same tool
-Can use intra-class correlation to measure this; you want it to be over .8

Front 3

Validity

Back 3

The degree to which a test measures what it is designed to measure
-You cannot be valid without being reliable. It is not possible to accurately capture what you are trying to capture if you aren’t consistent about it.
-Content, face, criterion-related (concurrent/predictive), construct, internal, external

Front 4

Content Validity

Back 4

-The extent to which a measure represents all facets of a given construct
-Measure of depression should measure the somatic symptoms as well as the mood symptoms

Front 5

Face Validity

Back 5

-the extent to which the measurement method appears “on its face” to measure the construct of interest.
-Ex: does the Rosenberg Self-Esteem scale appear to be measuring self-esteem?
-This is a very weak kind of evidence. The simple fact that a test seems to be measuring a particular construct is no guarantee that it is. However, face validity can be important because it can affect people's attitudes toward a test. For example, people might have negative reactions to an intelligence test that did not appear to them to be measuring their intelligence.
-May lead to response bias

Front 6

Criterion-Related Validity:

Back 6

-The degree to which measures from an assessment instrument correlate with scores from previously validated instruments that measure the phenomena of interest
-Ex: Does your new depression measure correlate highly with the BDI
-Concurrent Criterion-Related Validity:
-Your measure and the BDI correlate at the same time
-Predictive Criterion-Related Validity:
-The gold-standard
-Your measure predicts BDI scores at a later time point

Front 7

Construct Validity

Back 7

-How well does a test measure a construct?
-Convergent Validity:
-When your measure correlates highly with another measure that is intended to measure the same construct
-Divergent/Discriminant Validity:
-When your measure does not correlate with another measure that is intended to measure a different construct

Front 8

Internal/External Validity

Back 8

-Internal Validity:
-Careful method of measurement, rigorious research
-When you don’t have internal validity, you can’t make ANY causal conclusions

-External Validity:
-How well you can generalize to other settings
-When you don’t have external validity, you can’t make GENERALIZABLE causal conclusions
-Without external validity, your treatment may not be able to be replicated

Front 9

Causal language

Back 9

-In direct manipulation studies, you can technically say “the effect of…” but you can’t say that in correlation studies
-Correlation: “the role of, the association between, the relationship between”

Front 10

Moderators

Back 10

-A variable that affects the strength and/or direction of the relation between an independent
-A moderator answers the question, for whom and under what condition
-When you have an interaction effect, it is due to a moderator
-The moderating variable interacts with A to predict B

Front 11

Mediators

Back 11

-Accounts for some or all of the relationship between two variables
-A variable that functions to account for some amount of the relationship between the independent variable and the dependent variable
-Answers the question of how or why

Front 12

Behavioral observation

Back 12

-advantages of a behavioral observation: great for external validity and good for establishing method variance
-disadvantages: bad at capturing low frequency measures, establishing interrater reliability is hard to achieve, expensive and time consuming can’t observe internal behaviors, people act differently when they know they’re observing them
-Response class matrix: detailed way of coding behaviors where you take into account the context (antecedents & consequences), 2D method of coding.

Front 13

Normative Comparisons

Back 13

-Procedure for evaluating the clinical significance for therapeutic intervention by using a control group.

Front 14

Randomized Control Trials

Back 14

To be considered an RCT, you must have:
-Various conditions, potentially including a control group
-Participants are randomly assigned to various conditions

Front 15

Empirically Based Treatment

Back 15

takes into account the data, not that important, doesn’t have as many criteria
-just because it’s empirically based just means they incorporate research, the base level of empirical involvement

Front 16

Empirically Supported Treatment

Back 16

has to be replicated by an independent researcher, random assignment
-comparison group
-random assignment
-uses real patients with exclusion and inclusion criteria
-found in 2 independent groups, by someone other (1 additional) non treatment development
-treatment has to work better than no treatment, better than alt treatment, or a placebo

Front 17

Efficacious & Specific Treatment

Back 17

-empirically supported and statistically superior to other tx and placebo (both pill and psychological)
this is the best tx around!
more than it just works, it works better than everything else out there
also has to be found in 2 independent groups

Front 18

Treatment Research Control Groups

Back 18

A) Attention Control: controls for the passage of time, attention, and maturation
B) Wait-List Control: controls for passage of time, and treatment expectation, but not attention
C) Treatment as Usual: controls for passage of time, attention, and treatment expectation
-This is where bleeding can be a problem
bleeding example: Group of people getting talk therapy vs. Group of people with CBT, the talk therapy people might start getting CBT accidentally

Front 19

Specificity & Sensitivity

Back 19

Sensitivity (also called the true positive rate, or the recall rate in some fields) measures the proportion of actual positives which are correctly identified as such (e.g. the percentage of sick people who are correctly identified as having the condition).

Specificity measures the proportion of negatives which are correctly identified as such (e.g. the percentage of healthy people who are correctly identified as not having the condition, sometimes called the true negative rate).

Front 20

Ethics

Back 20

-Competence - know boundaries, don’t delimit yourself, do research, stay updated, don’t practice outside your training
-Integrity - don’t have dual relationships (eg. being a therapist for a student); if you have a [sexual relationship], must be at least 2 years after
-Professional and Scientific Responsibility - moral standards (eg. promoting alcohol using your psychologist credentials)
-Respect for people’s rights and dignities - privacy and confidentiality, don’t condone unfair discriminating practices
-Do no harm/act responsibly & avoid/minimize harm
-Social responsibility - service, sliding income scale

*(Children In Playgrounds Ride Down Slides)*

Front 21

Efficacy & Effectiveness

Back 21

Efficacious: it works under optimal conditions (works really well when you have tight inclusion/exclusion criteria, methods, etc.)
Effective: treatment is transferable, works in real-world settings

Front 22

Power

Back 22

-Power is the probability that the test will reject the null hypothesis when the null hypothesis is false (AKA the probability of NOT committing Type II error)
- Power =1-beta
- Critical effect size = threshold for clinical significance
- Factors influencing power:
1. statistical significance
2. magnitude of effect (cohen's d)
3. sample size

Front 23

Clinical Significance

Back 23

refers to the meaningfulness of the magnitude of change, the remediation of the presenting problem to the point that it is no longer troublesome.