Psych 304-2

Front 1

1. Define and be able to apply the broad definition of validity.

Back 1

a. The degree to which a test actually measures what it purports to measure

Front 2

2. What are the three main types of validity evidence?

Back 2

Construct-related
• Does the test measure what it purports to measure?
– Criterion-related
• Does the test correlate with/predict one’s score on an external criterion?
– Content-related
• How adequately do test items measure the construct domain?

Front 3

3. What prerequisites exist for validity?

Back 3

a. Reliability is NECESSARY (but not sufficient) for validity
Validation studies are necessary to justify inferences that the test is valid (measures what it purports to ` measure).

Front 4

4. Define Face Validity. How does it differ from other aspects of validity?

Back 4

Mere Appearance that a measure measures what it is designed to measure; it differs because it does not offer evidence to support conclusions drawn from test scores

Front 5

5. Define Content Validity: How is it measured?

Back 5

Degree to which a test or measure adequately represents the conceptual domain (universe) it is designed to measure
Measured by the CVR and uses a panel of judges on a 3-point scale.

Front 6

6. How do construct underrepresentation and construct-irrelevant variance relate to content validity?

Back 6

Underrepresentation: Failure to capture important components of a construct; Construct Irrelevant variance: Scores are influenced by factors irrelevant to the construct.

Front 7

7. What is the content validity ratio and how is it calculated?

Back 7

a. uses a panel of judges to rate items on a 3-point
b. scale (Essential, Useful but not essential, Not necessary)
c. – >50% of panelists consider an item “essential”, it has some content validity
d. – The more judges who rate an item “essential” = higher its content validity
e. – The fewer judges who rate an item as “essential” = lower its content validity

CVR= (#of panelist rating item as essential- (N:total number of panelist/2))/ (N/2)

Front 8

8. What is a criterion? What is Criterion-related Validity?

Back 8

the standard against which a test is compared; How well test scores correspond with a particular criterion

Front 9

9. Name and define the three subtypes of criterion related validity. Be able to give examples.

Back 9

Predictive validity: accuracy with which test scores predict a criterion OBTAINED AT A LATER TIME
Ex: SAT is predictor; college GPA is criterion
Concurrent validity: Degree to which the test scores are related to the criterion and can be measured AT THE SAME TIME (Job samples)
Postdictive validity: accuracy with which a test score predicts a previously obtained criterion (test of antisocial behavior to predict past delinquent acts)

Front 10

10. What is the validity coefficient? What is the meaning of a squared validity coefficient?

Back 10

a. Relationship between a test and a criterion: Extent the test is valid for making statements about the criterion
b. Percentage of variation in the criterion we can expect to know in advance because of our knowledge of test scores.

Front 11

11. What is incremental validity?

Back 11

a. If test has unique predictive power

Front 12

12. What is a construct? What is Construct-related Validity?

Back 12

Construct: Something built by mental synthesis
Construct related Validity: Degree to which a set of measurement operations measures hypothesized constructs.

Front 13

13. What are the two types of evidence in construct validity?

Back 13

Convergent and discriminate
Convergent is a demonstration of similarity: measures of same construct converge or narrow in on the same thing. Ex: show a test measures same things as the other tests do for the same purpose. Or Demonstrate specific relationships one can expect if the test is doing its job.
Discriminant evidence: Demonstration of uniqueness: Low correlations with measures of unrelated constructs or evidence for what the test does NOT measure
Measure doesn’t represent a construct other than its devised purpose

Front 14

14. How the two types of evidence for construct-validity defined and how are they different from one another?

Back 14

a. Convergent and discriminate
i. Convergent is a demonstration of similarity: measures of same construct converge or narrow in on the same thing. Ex: show a test measures same things as the other tests do for the same purpose. Or Demonstrate specific relationships one can expect if the test is doing its job.
b. Discriminant evidence: Demonstration of uniqueness: Low correlations with measures of unrelated constructs or evidence for what the test does NOT measure
Measure doesn’t represent a construct other than its devised purpose

Front 15

15. What is the relationship between reliability and validity?

Back 15

Reliability is necessary but not sufficient in validity

Front 16

16. Which two types of validity are logistical and not statistical? Why?

Back 16

Face validity and content validity are logistical. They require good logic, intuitive skills and perseverance rather than statistics.

Front 17

17. Which type of validity has been referred to as “the mother of all validities”, or “the big daddy” and why?

Back 17

a. Construct related Validity: subsumes all activities used in the other types of validity; all validation is one and in a sense, all is construct validation.

Front 18

19. What role does the relationship between examiner and test taker play?

Back 18

a. Test scores are affected by examiner behavior and relationship.

Front 19

20. What is the relationship between test examiner race and intelligence scores?

Back 19

a. Race doesn’t really affect IQ but one can still communicate a hostile/friendly atmosphere…etc

Front 20

21. Why would examiner race effects be smaller on IQ tests than on other psychological tests?

Back 20

There is high structure to IQ tests; test procedures are very specific and not real any places to deviate.

Front 21

22. What is the standard for test takers who are fluent in two languages?

Back 21

Give test in their BEST language

Front 22

23. Define expectancy effects and know whose name is associated with these effects

Back 22

Data is affected by what an experimenter EXPECTS to find. Rosenthal is associated with effect.

Front 23

24. A review of many studies showed that expectancy effects exist in ________ situations

Back 23

Some, BUT NOT ALL

Front 24

25. What types of situations might require the examiner to deviate from standardized testing procedures

Back 24

When the person is blind/deaf or have other sorts of misunderstandings

Front 25

26. What advantages and disadvantages were mentioned in lecture and the text regarding computer administered tests?

Back 25

a. ADVANTAGES
i. NO JUDGEMENT PROBLEM
ii. Excellence of standardization.
iii. – Individually tailored sequential administration.
iv. – Precision of timing responses.
v. – Release of human testers for other duties.
vi. – Test taker not rushed.
vii. – Control of bias.
viii. – Objective and cost effective.
b. Disadvantage
i. Computer generated reports
ii. Hard to detect error if validation is inadequate
iii. People let the computer do too much of the thinking
iv. Lack of personal interaction

Front 26

27. What subject variables can impact testing?

Back 26

Test Anxiety
Worry; Emotionality; Lack of self-confidence
Illness; Elderly; Medications; Stress; Fatigue; Hormonal variations

Front 27

28. What are the three major problems in behavioral observation studies?

Back 27

Reactivity, Drift, and Expectancies

Front 28

29. What is reactivity? How does performance change when people are not being observed or checked?

Back 28

a. Observing the observers; it increases the accuracy and is done when people are coding actual behavior

Front 29

30. What is drift? How does it relate to contrast effect? How can drift be addressed?

Back 29

People tend to drift away from the protocol.
Contrast effect is the tendency to rate the same behavior differently when observations are repeated in the same context. It is one of the most common behaviors of drift. It can be addressed through repeated checks

Front 30

32. How well do people do in detecting deception/lies?

Back 30

Not very well

Front 31

33. What is the Halo Effect? Be able to think of examples where the Halo Effect applies.

Back 31

Attribute positive attributes independently of the observed behavior. Linked with general standoutishness-or attractiveness

Front 32

34. What does a good interviewer know how to do?

Back 32

Keep interview answers flowing

Front 33

35. How are interviews similar to tests?

Back 33

Both used to gather information about the participant

Front 34

36. Define interpersonal influence and interpersonal attraction. How are they interrelated?

Back 34

Interpersonal Influence: Degree to which one person can influence another
Interpersonal attraction: Degree to which people share a feeling of understanding, mutual respect, similarity, etc.

Front 35

37. Define and note what types of statements should be avoided to elicit as much information as possible?

Back 35

Judgmental statements:
Evaluative Statements: GOOD, BAD, excellent, terrible
Probing Statements: Like using Why?
Hostile Responses: directs anger toward the interviewee
False reassurance: reassuring subject when they shouldn’t be reassured.

Front 36

38. What is the main goal in interviewing?

Back 36

Gather information

Front 37

39. Define transitional phrase. If it fails, what responses should be used to continue the theme? Define.

Back 37

a. Once a response dies, the interviewer responds to keep the interaction flowing with minimum effort.
Transitional phrase (“Yes”, “I see”)
If it fails,
– Verbatim Playback – Repeats the exact words
– Paraphrasing and Restatement – Repeats the response using different words
– Summarizing – Pulls together the meaning of several responses
– Clarification Response – Clarifies the response
– Empathy & Understanding – Communicates interviewer knows how they feel

Front 38

40. When should direct questions be used in an interview?

Back 38

When you need to fill in important details; maybe at end of interview

Front 39

41. What are the advantages and disadvantages of using structured clinical interviews?

Back 39

a. Everyone gets same questions in same order
b. – Uses specified rules for probing so all interviewees are handled the same.
c. – They offer reliability, but sacrifice flexibility.
d. – Frequently used in research
Disadvantage:
Requires cooperation (particularly hard in some psychiatric and forensic settings).
– Relies exclusively on the respondent making the assumptions questionable.

Front 40

42. What is the purpose of a mental status examination? What areas are typically covered?

Back 40

It is given at the hospital and areas are psychosis, brain damage, neurologic, and mental status

Front 41

43. Define general standoutishness. How does appearance play a role?

Back 41

Attraction to subject you are interviewing
Take one characteristic and take it out on all their characteristics.

Front 42

44. How much higher is interview reliability for structured interviews?

Back 42

Double

Front 43

45. What is a major criticism of structured interviews?

Back 43

It sacrifices flexibility

Front 44

46. What is social facilitation? Be prepared for examples.

Back 44

We act like the people around us

Front 45

47. What is the largest source of error in interviews?

Back 45

JUDGING

Front 46

48. What were the three independent research traditions identified by Taylor to study human intelligence?

Back 46

Psychometrics, Information Processing, and Cognitive

Front 47

49. Through what 3 facilities did Binet believe intelligence expressed itself? What two major concepts guided him?

Back 47

Judgment, Attention, and Reasoning; guided by two principles: General Mental Ability and Age Differentiation

Front 48

50. Know age differentiation, mental age, general mental ability, and positive manifold.

Back 48

Age differentiation: deferentiating older from younger children by older’s greater capacity
Mental age: equivalent age capabilities of a child regardless of his/her chronological age: obtained through age differentiation
General Mental Ability: Total product of the various separate and distinct elements of intelligence: sum of the parts
Positive manifold: good in one, good in others

Front 49

51. Binet searched for tasks that could be completed by what percentage of children in a particular age group?

Back 49

66.67% to 75% (2/3 to ¾)

Front 50

52. What concept did Spearman introduce? What does this concept mean?

Back 50

The “g”: general mental ability, which he got from positive manifold.

Front 51

53. What statistical method did Spearman develop to support his notion of g?

Back 51

factor analysis

Front 52

54. According to the gf-gc theory, what are the two basic types of intelligence? How do they differ?

Back 52

Fluid and crystallized

Front 53

55. Know and be able to calculate IQ using mental age and chronological age

Back 53

IQ is a unit of measure for expressing the results of intelligence tests and it is a ratio score. It is found by dividing mental age/ chronological age.

Front 54

56. What is a deviation IQ and how was it used in the Stanford-Binet scale?

Back 54

Standard score with M=100; SD=15; used to compare age groups
Got rid of MA/CA

Front 55

57. Define and differentiate basal and ceiling. Be able to give an example of each.

Back 55

Basal: minimum number of correct responses obtained ex: go down an exam and say that person gets five correct in a row. Then they are at that level.
Ceiling: Number of wrong answers indicating items too difficult
Ex: go up an exam and see which items are too hard… keep on going until you get five incorrect.

Front 56

58. What factors did Wechsler focus on that those before him had not?

Back 56

Non-intellectual factors

Front 57

59. What were some criticisms of the Binet scale by Wechsler?

Back 57

No performance, it was designed for children, didn’t take into account non-intellectual factos, and was a one-score test.

Front 58

60. What is the age range of the Wechsler scales?

Back 58

Goes from 16 to age 90 and 11 months

Front 59

61. Why is the inclusion of a point scale a significant improvement? What did a performance scale add?

Back 59

Credits or points are assigned to each item

Individuals receive credit for each item passed
Allows analysis of individual in several areas
Allows you to sum together items together so you can score it in a similar ways

Performance scale added nonverbal measure of intelligence
Attempt to overcome cultural and language barriers and education.

Front 60

62. Know which subtests are verbal and nonverbal. Know how they are divided into index scores

Back 60

Four Index: Verbal Comprehension Index; Perceptual Organization Index; Working Memory Index; Processing Speed Index

Verbal subtests: Vocabulary, Similarities, Arithmetic, Digit Span, Information, Comprehension, Letter-number sequencing
Nonverbal subtests: Picture completion, Digit symbol-coding, Block Design, matrix Reasoning, Picture arrangement, symbol search, Object Assembly.

Verbal Comprehension Index: Has Vocabulary, Similarities and Information
Perceptual Organization Index: Picture Completion, Block Design, and Matrix Reasoning
Working Memory: Digit Span, Arithmetic, and Letter Number Sequencing
Processing Speed Index: Digit Symbol-Coding, and Symbol Search

Front 61

63. Be able to know and differentiate the major functions measured by each subtest

Back 61

Vocabulary: Vocabulary Level
Similarities: Abstract thinking
Arithmetic: Concentration
Digit Span: Immediate Memory, Anxiety
Information: Range of Knowledge
Comprehension: Judgment
Letter-number sequencing: Freedom from Distractibility
Picture Completion: Alertness to detail
Digit Symbol-Coding: Visual-Motor Functioning
Block Design: Nonverbal reasoning
Matrix Reasoning: Inductive reasoning
Picture Arrangement: Planning ability
Symbol Search: Information Processing Speed
Object Assembly: Analysis of Part-Whole relationships

Front 62

64. Understand and be able to differentiate each WAIS subtest from the others

Back 62

Vocab subtest: Ability to define words; most stable, and best single measure of intelligence
Similarities: Consists of paired items of increasing difficulty; most sensitive of verbal subtest to cerebral dysfunction
Arithmetic: Contains 15 relatively simple story problems; need to retain info in memory and manipulate it; concentration, motivation, and memory are the underlying main factors in performance
Digit Span: Requires subject to repeat digits forwards and backwards; measures short-term auditory memory; Non-intellectual factors easily influence this subtest: like anxiety, distraction, and inattention
Information: Items increase in difficulty; includes abilities to comprehend, follow directions and respond; VERY culturally specific
Comprehension: More a measure of social awareness; what should be done in this type of situation, provide a logical explanation of a rule or phenomenon; Define proverbs; emotional difficulties play a role
Letter-number sequencing: Asked to reorder lists of numbers and letters; related to working memory and attention
Picture Completion: Subject is shown a picture with something missing
Digit Symbol-Coding: Requires subjects to copy symbols and code numbers with the symbols in the box.
Block Design: Uses 9 colored blocks and a stimulus booklet and reproduce design; design increases in difficulty; Block design and Matrix reasoning are most sensitive in Performance IQ to cerebral dysfunctions.
Matrix Reasoning: Subject presented with a nonverbal figure and stimuli and must find the relationship between the two; good measure of abstract reasoning and information processing
Picture Arrangement: Requires subject to notice details and cause and effect relations. Have pictures and are asked to put the pictures into order.
Symbol Search: Ability to rapidly scan and process info; this shows the role of speed of information-processing in intelligence.
Object Assembly: Subject assembled to put together the puzzle quickly as possible; requires visual analysis and its coordination with simple assembly skill.

Front 63

65. What are the mean, standard deviation, and range for scaled scores, standard scores, and index scores?

Back 63

Scaled Scores: M=10, SD=3, Range=1-195
Standard Scores: M=100, SD=15, Range=45-155
Index Scores: M=100, SD=15, Range=50-150

Front 64

How is IQ calculated

Back 64

a. Verbal IQ (VIQ): The five raw verbal subtests scores are converted to age-corrected scaled scores which are summed. They are then compared with the standardization sample.
b. • Performance IQ (PIQ): The five raw performance subtests scores are converted to age-corrected scaled scores which are summed. They are then compared with the standardization sample.
c. • Full-Scale IQ (FSIQ): Follows the same principles of VIQ and PIQ. Obtained by summing the age-corrected scaled scores of the verbal and performance subtests and comparing the subject to the standardization sample.

Front 65

67. Name the index scores and what the purpose of each index is.

Back 65

a. Verbal Comprehension Index
i. Measures acquired knowledge and verbal reasoning; measure of crystallized intelligence
b. Perceptual Organization Index
i. Measure of Fluid Intelligence; Attention to detail and visual motor integration influence this
c. Processing Speed Index
i. How quickly your mind works
d. Working Memory Index
i. Information we actively hold in our minds, NOT stored information

Front 66

68. What is pattern analysis? What are the concerns when using such a method?

Back 66

When one looks for large differences in subtest scaled scores; Results of diseases using pattern analysis were inconclusive and contradictory; Doesn’t take into account individual variability well; Need to proceed WITH EXTREME CAUTION

Front 67

69. What was one of the major changes of the WISC-IV from previous versions?

Back 67

No PIQ and VIQ. Just 4 index scores

Front 68

70. What is a hold subtest?

Back 68

Intelligence in that test is retained after brain damage

Front 69

71. Which verbal and performance subtests are most sensitive to cerebral dysfunction? Which are considered “hold” subtests?

Back 69

Most sensitive are the Similarities, Block-design, and Matrix reasoning; Hold subtests: Vocab

Front 70

72. Where do traditional intelligence tests fail in the study “normal” abilities?

Back 70

In extreme populations; those with physical, sensory, mental, or social disabilities

Front 71

73. What are the disadvantages of alternative intelligence tests when compared to Binet and Wechsler?

Back 71

Weaker psychometrics; less validity and documentation; weaker standardized sample; Test manual instructions; less stable

Scores are NOT interchangeable with Binet and Wechsler and specificity limits what abilities the test measures.

Front 72

74. What are the advantages of alternative intelligence tests when compared to Binet and Wechsler?

Back 72

It hits extremes, a specialized test, not as culturally dependent, and not quite crystallized

Front 73

75. What theme in relation to future intelligence do you notice about infant development tests?

Back 73

They do NOT predict future intelligence; only can tell with developmental features

Front 74

76. Compare and contrast surveillance and screening. Be able to identify either.

Back 74

Surveillance: Survey wide screen; intervene ONLY when you see those who aren’t developing.
Screening: You look for a specific area of concern… You NEED additional evaluation more monitoring

Front 75

77. Which two infant development tests were discussed in class? What are some disadvantages of each of these tests?

Back 75

Bayley: Disadvantages are there are that the psychometrics breakdown at lower age groups. IT does NOT predict later IQ, BUT extreme low scores predict MR later in life.
Brazelton Neonatal Assessment Scale: Disadvantages are that there aren’t any norms and has poor test reliability. Also, does NOT predict future IQ.

Front 76

78. Know sensitivity, specificity, false positives, true negatives, etc.

Back 76

Sensitivity: Accuracy of test finding developmental delays (Find True positives)
Specificity: Accuracy of test finding those who are NOT delayed (Find False Negatives)
False Positives: Is not sick but is diagnosed
True Negatives: Is not sick, is not diagnosed
False Negatives: Is sick, NOT diagnosed
True Positives: Is sick, is diagnosed
IF you are testing sensitivity, you will get many false positives: goal to get all diseased ones
If you are testing for specificity, you will only get true positives, and you hence have false negatives.

Front 77

79. What are acceptable sensitivity and specificity levels for developmental screening tests?

Back 77

70-80%

Front 78

80. How is a learning disability currently defined in the school systems? Is this a good method? Why or why not?

Back 78

Learning disability is 1.5 standard deviations of IQ and achievement …and NO not good method…

Front 79

81. For what was the Woodcock Johnson-III designed? (i.e., what does it assess?)

Back 79

Achievement (academic performance):
Woodcock-Johnson (WJ-III)
• Cognitive (IQ) and achievement batteries
– Most testers use only achievement
• Great psychometrics and validity research
• Used to define Learning Disabilities, based on
discrepancy from IQ