Reading...
Play button
Play button
Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
63 Cards in this Set
- Front
- Back
|
assessment
|
a mechanism for evaluating an attribute, defect, factor, element, etc. in a human being
|
|
|
test
|
focus is on evaluation
|
|
|
inventory
|
focus is on providing information
|
|
|
assessment helps us work through the following with clients
|
1. assessing the client problem
2. conceptualizing and defining the client problem 3. selecting and implementing effective treatment 4. evaluating counseling |
|
|
measurement scales
|
nominal, ordinal, interval, ratio
|
|
|
nominal
|
focuses on providing categorical information; label and categorize observations, but do not make any quantitative distinctions between observations; party affiliations
|
|
|
ordinal
|
rank ordering observations; indicates only a relative amount; ranking students
|
|
|
interval
|
presenting ordered categories as actual quantities that are separated by an equal distance. There is a zero point but it is not a true zero; degrees F
|
|
|
ratio
|
presenting ordered categories as actual quantities that are separated by an equal distance and has a true zero point (none of the variable is present); MPH
|
|
|
descriptive statistics
|
used to tabulate, depict, describe, and organize data
|
|
|
inferential statistics
|
using techniques with sample data to make generalizations about the selected sample
|
|
|
central tendency
|
statistical measure to determine a single score that defines the center of a distribution; to find the single score most typical or representative of the entire group
|
|
|
measures of central tendency
|
mean, median, mode
|
|
|
mode
|
most frequently occurring score
|
|
|
median
|
50% percentile or middle point of the data
|
|
|
mean
|
arithmetic average; most commonly used measure of central tendency; the only measure of central tendency that utilizes every score or data point
|
|
|
variability
|
describes the amount that scores differ from each other; how much the scores are spread out around the mean
|
|
|
measures of variability
|
variance and standard deviation
|
|
|
standard deviation
|
depicts how scores vary from the mean; most popular and commonly used measure of variability
|
|
|
variance
|
mean squared deviation
|
|
|
degrees of freedom
|
number of scores in a sample that are free to reflect the variability in the population; most often calculated when estimating the population variance or standard deviation
|
|
|
z-score
|
standard score; calculated to identify and describe the exact location of every score in a distribution based on standard deviations; allows to see the relative standing of one individual score in comparison to other scores; + is above the mean, - is below the mean
|
|
|
why is the quality of an instrument critical?
|
1) the instrument we are using needs to actually measure what we want/expect it to measure
2) the quality of an instrument is often times closely tied to the quality of research/results that produced 3) it helps our clients |
|
|
reliability
|
consistency of the test, assessment, or measurement tool to produce similar results
|
|
|
reliability coefficients
|
concerned with: the amount of measurement error and producing accurate interpretations as a result of the standard error of measurement
|
|
|
standard error of measurement (SEM)
|
an estimation of the range of scores that would be obtained if someone were to take an instrument over and over again
|
|
|
classical test theory/classical reliability theory
|
person's observed score is composed of a true score plus some unsystematic error of measurement; obs= T+E
|
|
|
range of reliability coefficients
|
.00 (none) to +/- 1.00
|
|
|
what is an acceptable range of reliability coefficients?
|
what is acceptable depends on what you are measuring
|
|
|
types of error associated with reliability (and measurement in general)
|
systematic error and unsystematic erro
|
|
|
systematic error
|
an error that is accounted for; it is ordered and methodical
EX: when everyone taking the test experiences the same error; typo |
|
|
unsystematic error
|
an error that is unaccounted for and occurs at random
EX: when at least one or more (but not all) individuals experience something randomly while taking the test; one individual not given all the instructions |
|
|
test-retest coefficient
|
reliability of a psychometric instrument from one time to another; not always successful as a result of contamination between time points
|
|
|
parallel-form coefficient (a.k.a. alternative form)
|
correlating scores on one form to another; there must be two forms of the instrument; or instrument can be dividided into halves equally and the two scores compared (split-half reliability)
|
|
|
interrater reliability
|
multiple people score the same test and then compare their scores for consistency; most often used in qualitative research (Gottman)
|
|
|
generalizability theory
|
a test score is viewed as a "single sample from a universe of possible scores" (Aiken, 1999, p. 81); an alternative to classical test theory
|
|
|
validity
|
degree to which a test measures what it is intended to measure
|
|
|
types of validity
|
Content
Criterion -concurrent -predictive Construct -convergent -discriminate |
|
|
content validity
|
"concerned with whether the appearance or content of a measuring instrument supports the assertion that it is a measure of a certain psychological variable" (Aiken, 1999, p. 82)
often determined by having experts review the instrument |
|
|
criterion validity
|
"extent to which an instrument was systematically related to an outcome criterion" (Whiston, 2009, p. 67); correlating test scores with external criterion of whatever test intended to measure
|
|
|
concurrent validity
|
a measure of criterion validity; how well a current test relates to a previously validated test
EX: Counselor Burnout Inventory with burnout inventory that it was adapated from |
|
|
predictive validity
|
a measure of criterion validity; predict scores on some criterion measure
EX: GRE with GPA in grad school |
|
|
cross-validation
|
process of administering the instrument on a different set of individuals and then comparing the subsets for accuracy; should especially be done with predictive measures
|
|
|
incremental validity
|
"how much more accurate predictions and/or diagnoses are when a psychometric instrument is adminstered than when it is not" (Aiken, 1999, p. 85).
|
|
|
construct validity
|
"extent to which the instrument may measure a theroetical or hypothetical construct or trait" (Whiston, 2009, p. 67); measured through "factor loading" (internal consistency), correlating test w/ others, questioning examinees about their process taking the test
|
|
|
convergent validity
|
form of construct validity; how well instrument "correlates with other measures (or methods of measuring) the same construct" (Aiken, 1999, p. 86).
|
|
|
discriminate validity
|
form of construct validity; showing that instrument has low correlations with instrument measuing other constructs
|
|
|
Taxonomy for understanding validity; 4 major categories
|
created by Cook and Campbell (1979)
1)statistical conclusions validity 2)threats to internal validity 3)threats to construct validity of causes and effects 4)threats to external validity |
|
|
Define statistical conclusions validity
|
"threats to valid inference making that results from random error and the ill-adivised selection of statistical procedures" (Kirk, 1995, p. 16)
|
|
|
threats regarding statistical conclusions validity
|
1. low stat power/effect size (inadequate sample size)
2. violated assumptions of statistical test (ex: using t-test when should be anova) 3. fishing for significant results and the error rate problem 4. reliability of measures 5. reliability of treatment implementation (how it is administered) 6. random irrelevancies in experimental setting (any variations) 7. random heterogeneity of respondents (how alike/different) |
|
|
Define internal validity
|
confidence one can have in "concluding that an independent variable is...responsible for variation in the dependent variable" (Kirk, 1995, p. 16)
|
|
|
threats regarding internal validity
|
1. history (events occurred in past)
2. maturation (what happens between assessments) 3. testing (testing increases familiarity/learning) 4. instruments (inconsistencies) 5. statistical regression (scores gravitating towards mean) 6. selection (how individuals acquired) 7. mortality (loss of individuals over time) 8. interactions with selection (selection impacting variable) 9. ambiguity about direction of causal influence 10. diffusion or imitation of threats (communication b/w particpants) 11. compensatory rivalry by respondents receiving less desirable treatments (ex: motivated to "prove" that they can have successful treatment even if they did not get selected for drug that others were selected for) 12. resentful demoralization of respondents receiving less desirable treatments (ex: less motivated in treatment because they were not selected to receive new drug while others were) |
|
|
Define construct valdity of causes and effects
|
"how well the independent and dependent variables represent the constructs they were intended to measure" (Heppner, et al 1999, p. 71)
|
|
|
Threats regarding construct valdity of causes and effects
|
1. hypothesis guessing w/n experimental conditions (ex: subjects figure out hypothesis and answer in such a way to confirm hypothesis)
2. experimenter expectations (personal wants obstruct accuracy) 3. interaction of different treatments (cannot isolate variables to determine relationship) |
|
|
Define external validity
|
generalizability to other individuals/situations outside of study
|
|
|
Threats regarding external validity
|
1. interaction of testing and treatment (testing informs individual about treatment)
2. interaction of selection and treatment (ex: those who agree to participate in study more motivated, thus not generalizable to others who may be less motivated) 3. interaction of setting and treatment (results obtained in one setting may not be obtained in another) 4. interaction of history and treatment (testing for fear of airplanes may look very different for those who experienced 9/11) 5. reactive arrangements ("Hawthorne effect"; awareness of testing influences bx) 6. multiple-treatment interference (multiple variables make it difficult to isolate effects; how can you be sure that a causes b if group also received c? |
|
|
standardized vs. nonstandardized
|
a standardized instrument has:
1. fixed instructions for administering and scoring 2. content is constant and developed according to professional standards nonstandardized insturment has not met these guidelines |
|
|
individual vs. group
|
how the instrument is administered - to a group or to an individual
pro/con of group: +convenient +less time -difficult to observe all clients and their behaviors |
|
|
objective vs. subjective
|
methods used to score instrument
objective = predeterimined methods for scoring that does not require judgment subjective = professional judgments will be made |
|
|
speed vs. power
|
difficulty level of the items
power = items vary in difficulty with more credit given for more difficult items speed = # of items completed in certain time period |
|
|
verbal vs. nonverbal (nonlanguage)
|
degree to which instrument requires understanding and use of language
|
|
|
cognitive vs. affective
|
whether instrument assesses thoughts or feelings
cognition = perceiving, processing, thinking, remembering ex: IQ, achievement tests affective = interest, attitude, values, motives, temperaments ex: MMPI, projective tests |
|
|
norm-referenced vs. criterion-referenced
|
how score is interpreted
norm = score is compared with scores of other individuals who have taken same intsrument criterion = score is compared to established standard |
