Advanced Research Methods

Front 1

If there is a true small difference between the two groups, there maybe insufficient participants in both groups for the statistical test to be able to provide reasonable evidence for this difference.

Back 1

Threats to conclusion statistical validity: Low statistical power

Front 2

If a difference is being investigated between diagnostic groups and each negative symptom is measured as being either present or absent, then the use of categorical data for individual negative symptoms in an independent sample t-test will violate the assumption of normality of scores on the DV to such an extent that the result may no longer be valid.

Back 2

Threats to conclusion statistical validity: Violation of statistical assumptions

Front 3

If the homogeneity of variance assumption for total negative symptom scores is being violated, then the unequal sample sizes will most likely mean that any NHST will be incorrect without some correction to the t test being undertaken (same would apply to Cis).

Back 3

Threats to conclusion statistical validity: Lack of robustness

Front 4

If a NHST or a CI was calculated on each individual negative symptom (even if measured continuously), then the actual false rejection error rate over all NHSTs will be much higher than the nominal value of alpha specified for each test.

Back 4

Threats to conclusion statistical validity: Multiplicity of testing

Front 5

If the negative symptoms are rated very poorly by the clinicians, then this will mean that small differences may not be detected (it will reduce statistical power).

Back 5

Threats to conclusion statistical validity: Unreliability of measurements

Front 6

If only a subset of negative symptoms is rated by clinicians, then the total negative symptom score may not have sufficient variability for differences to be detected (also affects reliability).

Back 6

Threats to conclusion statistical validity: Restriction of range

Front 7

Clinicians differ in how well they can diagnose schizophrenia and schizophreniform disorder, or there is poor reliability of diagnosis and/or negative symptom assessment between clinicians.

Back 7

Threats to conclusion statistical validity: Unreliability in manipulations

Front 8

Different interview rooms may affect responses by patients, which results in the ratings of negative symptoms being done poorly in some instances.

Back 8

Threats to conclusion statistical validity: Extraneous variance in settings

Front 9

If multiple psychiatric hospitals were being used in obtain a large number of participants, there will be some dependency in responses of patients within each hospital because of differences in patient types or differences in treatments etc between hospitals.

Back 9

Threats to conclusion statistical validity: Homogeneity in units of observations

Front 10

For instance, Hedges’ g was used to estimate a standardized difference when scores were non-normal and heterogeneity of variance between diagnostic groups was present.

Back 10

Threats to conclusion statistical validity: Biased Estimation

Front 11

Do the negative symptoms cause the disorders, or are they an outcome of the underlying disorders? (timing)

Back 11

Threats to internal validity: Ambiguous temporal precedence

Front 12

Group allocation is based on an arbitrary dichotomising of duration of illness. Different triaging of Sz and Sf at admission may result in exacerbation/reduction of negative symptoms in one group relative to the other.

Back 12

Threats to internal validity: Selection

Front 13

Differential changes due to the course of illness before admission (e.g.,housing, employment, etc) may result in differences being observed that are not due to fundamental differences between disorders.

Back 13

Threats to internal validity: History

Front 14

Differential application of treatments may mean that differences are observed between groups unrelated to the underlying disorders.

Back 14

Threats to internal validity: Maturation

Front 15

If people with Sz are admitted because their psychosis is more severe on average than those with Sf, then levels of negative symptoms in Sz are likely to be less severe on average than those for Sf when people in both groups are subsequently measured a second time.

Back 15

Threats to internal validity: Regression to the mean

Front 16

Severity of negative symptoms, which may be greater in Sz than in Sf, may mean that those patients are less likely to consent to participate, or they are more likely to drop out after diagnosis but before negative symptoms are measured. This may lessen the observed differences between groups.

Back 16

Threats to internal validity: Attrition

Front 17

Having already been assigned a diagnosis, and both the patient and the clinicians knowing that diagnosis, may change the measurement of negative symptoms (compared to circumstances in which both patient and clinician are blind to diagnosis).

Back 17

Threats to internal validity: Testing

Front 18

If a printing error meant that the numbers on the response categories of the SANS for one group differed from the other group, then this may result in differences being observed even if subsequently corrected when entering the data.

Back 18

Threats to internal validity: Instrumentation

Front 19

The diagnostic criteria may be missing an important determinant of inherent differences between the groups.

Back 19

Threats to construct validity: Incomplete explanation of constructs

Front 20

Diagnosis includes the presence of negative symptoms, which mimic symptoms of depression, that may affect diagnosis or measurement.

Back 20

Threats to construct validity: Confounding among constructs

Front 21

The assessment of negative symptoms is only undertaken once, and/or at different time points subsequent to admission.

Back 21

Threats to construct validity: Bias from single measurement

Front 22

Only one negative symptoms inventory is used, or a global assessment of negative symptomatology is used rather than an inventory that assesses different types of negative symptoms.

Back 22

Threats to construct validity: Bias from single measurement methodology

Front 23

Symptom assessment relies on thresholds being inferred about thestrength of each symptom being exhibited. Differences betweengroups may reflect incorrect assessment of thresholds by clinicians.

Back 23

Threats to construct validity: Confounding constructs with level or amount of construct

Front 24

The process of undertaking any diagnostic assessment, and the assignment of a diagnosis, may differentially influence the exhibiting or observance of negative symptoms.

Back 24

Threats to construct validity: Manipulation changes construct measure

Front 25

Patients with greater insight into their disorder may be more or lesswilling to indicate their level of severity of negative symptoms.

Back 25

Threats to construct validity: Subjective measurement motivation

Front 26

Admission to a psychiatric clinic, and unfamiliarity with a threatening hospital environment, may prompt feelings of despondency, panic, etc that may affect measurement of negative symptoms.

Back 26

Threats to construct validity: Reactivity to experimental context

Front 27

If patients receive a particular diagnosis, e.g., schizophrenia, they may then act up and exhibit greater symptomatology because that it is the way they think they are supposed to be.

Back 27

Threats to construct validity: Transference of expectancies

Front 28

Changes to daily practices because of hospital regimes may affect level of symptomatology.

Back 28

Threats to construct validity: Novelty, disruption, and routines

Front 29

Patients who receive a diagnosis of Sz may get more intensive attention and/or initial treatment that may subsequently affect the ratings of negative symptoms.

Back 29

Threats to construct validity: Compensatory equalisation

Front 30

Patients who receive a diagnosis of Sf may observe preferential treatments or attention from clinician, which may subsequently mean they exaggerate their symptoms.

Back 30

Threats to construct validity: Compensatory rivalry

Front 31

Patients who receive a diagnosis of Sz may become less engaged withtreatment or measurement because of the label and they maytherefore exhibit greater “false” negative symptoms.

Back 31

Threats to construct validity: Demoralisation

Front 32

Some people with Sf diagnosis may subsequently have Sz diagnosis.

Back 32

Threats to construct validity: Diffusion of manipulation

Front 33

Symptom severity observed in Sz, when compared to Sf, may not be found if a different comparison group had been used (e.g., MDE with psychotic features).

Back 33

Threats to external validity: Interaction of causal relationships with experimental units

Front 34

Clinicians making diagnoses in one facility may be more experienced or reliable than those in another facility.

Back 34

Threats to external validity: Interaction of causal relationships over variations in manipulations

Front 35

Clinicians in one facility may have different cultural interpretations of negative symptoms to other facilities.

Back 35

Threats to external validity: Interaction of causal relationships with outcomes

Front 36

The results obtained from one psychiatric facility may reflect differences in treatment practices or in catchment areas that do not replicate in other clinics.

Back 36

Threats to external validity: Interaction of causal relationships with settings

Front 37

Usage of a different inventory of negative symptoms in another facility may result in different levels of negative symptoms being observed in Sz and Sf.

Back 37

Threats to external validity: Interaction of causal relationships with context-dependent mediators