Group Wiki 2-Validity Part 1

Front 1

What does validity imply historically?

Back 1

-an inherent characteristic of a test score
-A test is valid to measure anything with which it correlates.
-“Validity coefficients” were often used.
-The extent to which a test measures what it purports to measure.

Front 2

What is the current definition of validity?

Back 2

Validity is an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment.”

Front 3

What are the three important characteristics of validity?

Back 3

-Using the test reasonably for a particular purpose
-There can be many aspects of valdity depending on the use of the test score
-Tests do not have a single validity

Front 4

What is one of the simplest ways to obtain evidence for the validation of a test?

Back 4

to examine the content of the test.

Front 5

What is content validity?

Back 5

-Content validity is then the extent to which the questions on a test are representative of the trait, behavior, or attribute being measured.
-In other words, does the test constitute a representative sample of the items assessing the objectives the test was originally designed to measure?
-Do the judges constitute a representative sample as well?

Front 6

What is the first step for the development of any test?

Back 6

to determine the testing universe

Front 7

What is the testing universe?

Back 7

-The set of knowledge or behaviors the test represents.
-This step involves locating theoretical or empirical research, talking with experts, or reviewing similar instruments.

Front 8

What is some info about content domains?

Back 8

-Content domains often have defined boundaries, and can usually be structured into distinct subcategories.
-Describing the boundaries and categories of a content domain facilitates test question development and is crucial in evaluating content validity.

Front 9

What happens when a test shows evidence of content validity?

Back 9

It representatively samples the testing universe.

Front 10

What is the general procedure for evaluating content validity?

Back 10

1) Describe the content domain and subcategories of the testing universe.
2) Determine where each test item fits with respect to the testing universe.
3) Compare the test structure with that of the structure of the testing universe.
-Evaluated by showing how well the content tested samples the subject matter

Front 11

Test blueprint/table of specifications?

Back 11

-Content objectives
-Taxonomy of levels of skills
-Test that provide more detail about the structure and boundaries of content domains generate more confidence about content validity.

Front 12

What are some things to keep in mind about content validity?

Back 12

Content validity by itself cannot guarantee the validity of a measure, and it cannot simply be assumed. Publishers should provide evidence of content validity in the test manual.

Front 13

How does content validity seem conceptually similar to reliability?

Back 13

-Content validity places an emphasis on providing a description of the content domain.
-Reliability assumes the domain exists, but does not define it.
-Thus, you could have a very reliable test with little content validity.

Front 14

What is something interesting about face validity?

Back 14

-Face validity is a term we hear associated with content validity.
-It tells us nothing about the validity of the test

Front 15

What is the definition of face validity?

Back 15

Face validity refers to how test takers perceive the attractiveness and appropriateness of a test.

Front 16

What would test takers consider tests with face validity?

Back 16

If test takers consider the test to have face validity, they may give a better effort to complete the test. If a test does not have face validity, they might hurry through the test and take it less seriously. (IE: If a test taker is in the middle of the tests and feels it to be too easy or to hard (inappropriate) they may not take it seriously enough to finish it well.

Front 17

What is construct validity?

Back 17

Construct validity is defined as the extent to which a test measures some theoretical construct.

Front 18

How do you establish construct validity?

Back 18

The process of establishing construct validity is tedious and requires the gradual accumulation of evidence that illustrates that test scores relate to observable behaviors in a way that is predicted by theory.
If you accept the evidence provided by construct validity, then you accept the definition of the construct provided by the developers of the test.

Front 19

What is a construct?

Back 19

-Attributes that exist in the theoretical sense.
-We do not observe the construct, but we observe behaviors that provide evidence of a construct underlying them.
-We observe behaviors that are theoretically related to the construct.

Front 20

How are constructs defined?

Back 20

Constructs are also defined by the relations they hold with other constructs and their behavioral universes.
A pictoral description of the relations between constructs and behavioral universes is called a nomological network.
This defines constructs by illustrating their relation to as many other constructs and behaviors as possible.
This becomes the starting point for establishing construct validity – gives a large number of hypotheses about behaviors.

Front 21

What are the two main ways to obtain evidence of construct validity?

Back 21

-Gathering theoretical evidence
-Gathering psychometric evidence

Front 22

What are the two types of gathering theoretical evidence?

Back 22

-Nomological network
-Proposal of experimental hypotheses

Front 23

What are the three types of Gathering psychometric evidence?

Back 23

-Evidence of reliability
-Convergent and discriminant validity evidence
-Experimental interventions

Front 24

When considering the ways to gather psychometric evidence, reliability?

Back 24

Reliability:
-Recall: reliability is a necessary characteristic for a psychological test.
-High reliability scores generally indicate that a single theoretical construct is present.
Why?
-Test theory suggests that a test should not have a stronger correlation with any other variable than it does with itself.
-Thus, reliability estimates may be used to evaluate the relative strength of a test’s correlations with other variables related to the construct.

Front 25

Convergent Validity evidence?

Back 25

-If a test has construct validity, we should expect that our test’s scores will correlate strongly with the scores on other tests measuring the same construct.
-This is convergent validity evidence

Front 26

Divergent validity evidence?

Back 26

-If different constructs are not considered to be related, then we should expect to find no correlation between test scores measuring different constructs.
-This is divergent validity evidence.

Front 27

Experimental Intervention?

Back 27

-When a test is used as an independent or dependent variable in a research study, the results can make a substantial contribution to the argument of construct validity.
-Example: Think of a significant difference between pre- and post-test scores on some construct that was predicted to change due to experimental treatment.

Front 28

What is a criterion?

Back 28

A criterion is a measure of performance that is correlated with test scores.

Front 29

When we collect criterion-related validity evidence what are we interested in?

Back 29

how well our tests predict behavior or events.

Front 30

The stronger our test scores correlate with independent behaviors, attitudes, or events, the better our decisions will be and what?

Back 30

the greater evidence of criterion-related validity we will have.

Front 31

There are two methods for demonstrating criterion-related validity, what are they?

Back 31

-Predictive validity
-Concurrent validity

Front 32

Predictive Valdity evidence?

Back 32

-Used when you are trying to show a relation between test scores and some future behavior.

General procedure:
-A large group of people take the test.
-The scores for those people are held for a predetermined period of time.
-Once the time period elapses, a measure of some behavior (i.e., criterion) is taken.
-The test scores are correlated with the criterion scores.
-If the scores correlate, the test has predictive validity.

Front 33

Concurrent validity evidence?

Back 33

Concurrent validity evidence:
-This is the “practical” alternative to the predictive method.
-You obtain the test scores and criterion scores at roughly the same time in the predetermined population.
-Once this is accomplished, you determine the correlation between the scores.