Eppp Statistics, Test Construction

Front 1

Mu

Back 1

population mean

Front 2

Sigma

Back 2

population standard deviation

Front 3

Discriminant Validity

Back 3

when a test does NOT correlate significantly with measures of different constructs

evidence of a test's construct validity

Front 4

Cluster Sampling

Back 4

identifying naturally occurring groups or clusters & then randomly selecting certain of these clusters

typically all subjects within selected clusters are sampled; but subjects may be randomly selected from the selected clusters

Front 5

Differential Validity

Back 5

a test's validity coefficient for one group is different from its validity coefficient for another group

Front 6

Congruent Valiidty

Back 6

when a test correlates highly with an established test that measures the same trait

Front 7

Bayes Theorem

Back 7

theory re: statistical probability

describes the likelihood of certain occurences given the likelihood of other occurences

Front 8

Latin Square

Back 8

most sophisticated counterbalancing design, controls for carryover effects when repeated measures are used

Front 9

Solomon 4 Group Design

Back 9

controls for effects of testing/practice

Group 1: pretest/tx/posttest
Group 2: pretest/ posttest
Group 3: tx/posttest
Group 4: postest

Front 10

Item Response Theory (IRT)

Back 10

aka latent trait theory; used to establish a uniform scale for individuals of varying ability with items of varying difficulty

used to calculate to what extent a specific item on a test correlates with an underlying construct

subject's performance on a test represents degree to which subject has a latent trait

can be used to compare a subjects's performance on 2 measures that are diferent in scoring or # of items

Front 11

Interval Sampling

Back 11

behavioral sampling used when a behavior has no distinct beginning or end (record whether a behavior occurred during each of a series of time intervals)

Front 12

Standard Error of Measurement

Back 12

average amount of error in each data point of a certain variable

SEmeas = SD x square root of 1- rxx

example: average amount of error in each person's IQ score

Front 13

Central Limit Theorem

Back 13

derived from probability theory

states that the sampling distribution of the mean:

1. will approach a normal shape as sample size increases regardless of the shape of the population distribution of individual scores

2. has a mean equal to the population mean

3. has a SD equal to the population SD divided by the square root of the sample size

Front 14

Power

Back 14

1 - beta; the ability to reject a false Ho

Front 15

Ways to increase power:

Back 15

increasing alpha
increasing N
increasing effect size (by strengthening the IV)
minimizing error
using a one tailed test
using a parametric test

Front 16

Type II Error

Back 16

retaining Ho when it is false

probability = beta

more likely when alpha is low, when N is small, & when the IV isn't intense enough

Front 17

Biserial Correlation Coefficient

Back 17

used when 1 variable is an artificial dichotomy (made from a continuous variable) & the other variable is continuous

Front 18

Point Biserial Correlation

Back 18

used when 1 variable is a true dichotomy & the other is continuous

Front 19

Spearman Rho

Back 19

used to measure association between measures expressed as ranks

Front 20

summative evaluation

Back 20

type of program evaluation

conducted after a program has been administered to determine if the program goals were achieved

Front 21

Formative Evaluation

Back 21

type of program evaluation

conducted during the development of a program to determine how the program should be altered to make it more effective

Front 22

standard error of the mean

Back 22

estimate of how much a sample mean can be expected to differ from the population mean as the result of sampling error

calculated by dividing the population SD by the square root of the sample size

Front 23

Incremental Validity

Back 23

benefits that use of a test provides to decision-making accuracy

Front 24

p-value
in item response theory

Back 24

characteristics of each item are described with an item response curve

p-value: probability of getting the item correct (# of examinees who answered an item correctly / total # of examinees)

Front 25

Coefficient of Determination

Back 25

proportion of variance shared between 2 variables

formula for variability shared between 2 variables = correlation coefficient squared

Front 26

Eigenvalues

Back 26

can be calculated for each component extracted in a principal components analysis

indicates the total amount of variability in a set of tests or other variables that is explained by an identified component or factor

Front 27

Trend Analysis

Back 27

type of ANOVA used to assess linear & nonlinear trends when the IV is quantitative

Front 28

Spearman Brown formula

Back 28

used to estimate the effects of increasing or decreasing the length of a test on its reliability coefficient

Front 29

KR-20

Back 29

Kuder Richardson Formula 20

a method for assessing internal consistency reliability when test items are scored dichotomously

a higher score = a more homogeneous test

Front 30

Mediators

Back 30

explain why there is a relation between the predictor & criterion

when controlled for, the correlation between the DV & IV goes down close to 0

Front 31

Moderators

Back 31

variables that influence the strength of the relation between 2 other variables

Front 32

Homoscedasticity

Back 32

similar variability among groups or data

an assumption of parametric tests & bivariate correlation coefficients

Front 33

formula for the relation between validity & reliability

Back 33

validity is less than or = square root of reliability

(reliability is a decimal, square root of a decimal is a larger number)

a test with reliability of .25 could have a validity of up to .50

Front 34

Rosenthal Effect

Back 34

self fulfilling prophecy; refers to the tendency of experimenters to inject their bias into the experiment so that it comes out fulfilling their hypotheses

Front 35

Empirical Criterion Keying

Back 35

items are chosen based on their ability to discriminate group membership

used in development of the original MMPI

Front 36

Cluster Analysis

Back 36

gathering data on a number of DV's & statistically looking for naturally occurring subgroups without any prior hypotheses

used to identify homogeneous groups from a collection of observations

Front 37

ways to increase a test's reliability:

Back 37

* more items on the test
* more homogeneous items
* unrestricted range of scores (results from a more heterogeneous sample)
* difficulty of guessing

Front 38

Construction of Confidence Intervals

Back 38

99% = +/- 3 SEM's
95% = +/- 2 SEM's
68% = +/- 1 SEM

Front 39

ANOVA

Back 39

used when there is 1 IV & 1 DV

Front 40

ANCOVA

Back 40

used to control for or partial out a confounding variable

Front 41

Factorial ANOVA

Back 41

used when there are 2 or more IV's

Front 42

MANOVA

Back 42

used when there is >1 DV

less powerful than running separate ANOVA's (i.e., it's easier to find significance with separate ANOVA's but also have a greater chance of Type I error)

Front 43

Kuder Richardson

Back 43

measures internal consistency by analyzing all possible split halves of a test; split half reliability creates 2 shorter tests, therefore, Spearman-Brown is needed to correct for the decreased number of items

Front 44

measures of inter-rater reliability

Back 44

Pearson r, percentage agreement, Kappa, Yule's Y

Front 45

Standard Error of the Estimate

Back 45

measure of the accuracy of predictions made with a regression line:

SEest = SDy√1-(rxy)2

ranges from 0 (no error) to SD of y (lots of error)

Front 46

Correction for attenuation formula

Back 46

used to determine how much the criterion-related validity coefficient would increase if both the predictor (test) & criterion (outcome) were perfectly reliable

Front 47

Split plot ANOVA

Back 47

used with mixed design of within- & between-subjects variables (e.g., time & treatment type)

Front 48

Tetrachoric coefficient

Back 48

measures asociation between 2 artificial dichotomous variables

Front 49

Phi coefficient

Back 49

measures association between 2 true dichotomous variables

Front 50

Multiple correlation (Multiple R)

Back 50

measures association between 2 or more predictors and 1 continuous criterion

Front 51

Canonical correlation

Back 51

measures association between 2 or more predictors and 2 or more criterions

Front 52

Orthogonal

Back 52

variables are not correlated

Front 53

Oblique

Back 53

variables are correlated

Front 54

Orthogonal Rotation

Back 54

in factor analysis, results in uncorrelated factors

Front 55

Oblique Rotation

Back 55

in factor analysis, results in correlated factors

Front 56

F ratio

Back 56

in ANOVA, equals the variance between groups (due to treatment + error) divided by the variance within groups (error)

* when groups are not different, the F ratio = 1, no significance
* when an F ratio gets above 2, there is generally significance

Front 57

Eta

Back 57

coefficient to use to measure a curvilinear relationship

Front 58

Multi-trait Multi-method Matrix

Back 58

used to determine a test's construct validity (both divergent & convergent)

Front 59

Communality

Back 59

the total amount of variability accounted for in a test by the identified factors and is a reflection of the amount of variance that the test has in common with the other measures included in the factor analysis

Front 60

Latent Class Analysis

Back 60

used to identify the underlying latent structure of a set of observed data

Latent trait analysis (LTA) is also used to identify the underlying latent structure of a set of observed data. A primary difference between the two techniques is that, in LCA, the latent variable that determines the structure is nominal; while, in LTA, the latent variable is continuous.