Testing A Test

Front 1

Screening test

Back 1

- Performed on patients who have no symptoms however the results indicate if a disease/condition might be present
- need further testing

Front 2

Diagnostic test

Back 2

Performed on patients with symptoms with the intention of identifying (or ruling out) the presence a disease

Front 3

Index test

Back 3

- the test that is being investigated
- compared to the reference test

Front 4

Reference or standard test

Back 4

- the most definitive test available
- reference standard
- gold standard

Front 5

1000 or more sample size and statistical power

Back 5

screening tests with a low pretest probability of disease

Front 6

100-200 sample size and statistical power

Back 6

diagnostic tests with a moderate/high probability of disease

Front 7

Patients assigned to test groups

Back 7

- Include all patients who fulfill entry and exclusion criteria (signs/symptoms) prior to their undergoing the reference or index tests
- Patients undergo the index test AFTER they have been identified as having, or not having, a disease with the reference test
- Patients who undergo the reference test AFTER they have taken the index test

Front 8

Spectrum bias

Back 8

Failure to include individuals with other diseases which might also test positive

Front 9

Verification bias

Back 9

- Patients might not choose to undergo further testing
- Index test “negative”: No desire to be subjected to another procedure
- Clinician has to make intervention

Front 10

Cutoff point

Back 10

a specific point or level which is used to separate positive from negative

Front 11

Determining a reference interval

Back 11

- Identify a group of disease free individuals (reference sample group)
- Perform index test on this group
- Calculate the reference interval
- Include 95% of the reference sample group
- Mean +/- 2SD
- 2.5% below and 2.5% above the +/- 2SD

Front 12

1 SD

Back 12

68%

Front 13

2 SD

Back 13

95%

Front 14

3 SD

Back 14

99.7%

Front 15

Reference range

Back 15

- 5% will by definition will test outside the reference range: A reflection of the method used to determine the reference range
- It must be determined using individuals for whom the test is being used
- It may not be the desirable range for a particular test
- It may be necessary to determine reference ranges for different subpopulations
- Changes within the reference interval may be pathologic

Front 16

Regression Analysis

Back 16

- Simple linear regression
- Defines the relationship between two variables: X, Y
- Independent variable & dependent variable
- Does not imply causality
- “least squares best fit line”
- Provides information regarding statistical significance of the relationship
- Provides the rationale for the comparison of
two tests

Front 17

Dependent variable

Back 17

Y, index test

Front 18

Independent variable

Back 18

X, reference test

Front 19

Simple linear regression

Back 19

- to compare results of a new (index) text to those of an established procedure (the reference test)
- Used when only a single explanatory variable exists
- Method of least squares:
X = reference test
Y = index test
- line of best fit
- The expected variability in y for any fixed x value
- y = bx + a
- r = between -0.7 and 0.7 = not linear

Front 20

Assumptions for simple linear regression

Back 20

- The values for x are pre-selected by the investigator
- The error associated with x is negligible
- For each value of x, there is a population of y values assumed to be Gaussian
- The means of the populations of y values lie on the regression line

Front 21

Comparison of index and reference tests

Back 21

1.) Sensitivity
2.) Specificity

Front 22

Sensitivity

Back 22

- Positive in disease
- The percentage of participants with the disease, as defined by the reference test, which are correctly identified by the index test

Front 23

Specificity

Back 23

- Negative in health
- The percentage of participants free of the disease as defined by the reference test, which are correctly identified by the index test

Front 24

False positive

Back 24

- A test is positive for a disease, however the patient does not have the disease
- increases when diagnostic specificity decreases

Front 25

False negative

Back 25

- A test is negative for a disease, however the patient does have the disease
- increases when diagnostic sensitivity decreases

Front 26

Calculating sensitivity

Back 26

- % (proportion) of individuals with a disease that test positively with a test
- # of individuals with the disease who test positive/total # of diseased individuals tested
= TP/(TP + FN) x 100

Front 27

Calculating specificity

Back 27

- % (proportion) of individuals without a disease that test negatively with a test
- # of individuals without the disease who test negative/total # of individuals tested without the disease
= TN/(TN + FP) x 100

Front 28

Discriminant ability

Back 28

- A measure of the information provided by the index test when compared to the reference test
- Average of sensitivity and specificity for the index test
= (sensitivity + specificity)/2
- Reference test = 100%
- Assumption: the reference test provides perfect information

Front 29

ROC Curve

Back 29

- receiver operator characteristics curve
- Y-axis: sensitivity, true positive rate
- X-axis: 100%-specificity, false positive rate
- perfect test: upper left corner, sensitivity and specificity each 100%, what your index test (ROC line) is being compared to

Front 30

Line of no discrimination

Back 30

- "zero information"
- anything on or below line is bad

Front 31

Determination of cut-off point

Back 31

- Choose several sets of cutoff points
- Calculate sensitivity and specificity for each
- Calculate discriminant ability for each
- Choose cutoff point with greatest discriminant ability

Front 32

When you increase diagnostic sensitivity, what happens to diagnostic specificity?

Back 32

decreases

Front 33

Diagnostic ability of a test

Back 33

- Consider relative importance of false positives and false negatives
- Calculation of discriminant ability
- Goal: to maximize discriminant ability
- False positive and false negative results are not always equally undesirable
- Consider impact of false positives and false negatives to the patient

Front 34

Testing a test: Interpretation

Back 34

0 Rule in and rule out disease
- Posttest chances of disease: Baye’s theorem
- Clinical performance:Acceptance by patients and clinicians
- Safety
- Cost

Front 35

Superior discriminant ability on ROC curve

Back 35

top left quadrant

Front 36

Superior for ruling-out

Back 36

top right quadrant

Front 37

Superior for ruling-in

Back 37

bottom left quadrant

Front 38

Inferior discriminant ability

Back 38

bottom right quadrant

Front 39

Likelihood ratios

Back 39

- Expression of the chances that an index test will be correct compared to the chances that it will be incorrect
- Calculated from sensitivity and specificity
- Use like ROC curves to compare two diagnostic tests to determine which is best in ruling in and ruling out disease

Front 40

The test with the greatest likelihood ratio of a positive test is best to rule in or rule out a disease?

Back 40

best to rule in a disease, 1 to infinity, the larger the better

Front 41

The test with the smallest likelihood ratio of a negative test is the best test to rule in or out a disease?

Back 41

best to rule out a disease, 1 to 0, the smaller the better

Front 42

Positive predicted value (PV+)

Back 42

percent of patients with positive results who are diseased

Front 43

Negative predictive value (PV-)

Back 43

percent of patients with negative test results who are nondiseased

Front 44

Extrapolation

Back 44

- Conditions under which test is used in practice
- Similarity of clinical populations
- Combining tests: 1st test positive, then follow with a second test
- Baye’s theorem: Use not recommended UNLESS the test sequence is examined in a clinical trial

Front 45

Screening

Back 45

Detecting disease in asymptomatic individuals as part of a testing strategy to to diagnose disease

Front 46

Criteria of screening

Back 46

1.) The disease screened for leads to death or disability
2.) Early detection of the disease improves outcome
3.) Feasibility: Ability to use with a high risk group
- Recommended pretest probability of disease > 1:1000
- Availability of a test with the desired discriminant ability
4.) Acceptability: Minimal harm, cost, patient acceptance

Front 47

Lead time bias

Back 47

Overestimating survival time due to earlier diagnosis of disease, the actual time of death does not change