Psych 258 Ch 8

Front 1

Hypothesis Test

Back 1

- is a statistical method that uses sample data to evaluate a hypothesis about a population

- sample data --> inference about population

Front 2

Steps to Hypothesis Test (simple)

Back 2

1. State a hypothesis about population (usually concerning the value of a parameter)

2. Use hypothesis to predict that characteristics that the sample should have (the sample should be reflective of population)

3. Select a random sample from population

4. Compare sample data with the prediction that was made from the hypothesis ... sample mean is consistent with prediction we conclude the hypothesis is reasonable vice versa

Front 3

What are the known and unknown populations?

Back 3

- known = exist before treatment

- unknown = after the treatment is administered; hypothetical population if treatment were administered

Front 4

What is one basic assumption made about the effect of the treatment condition?

Back 4

- If the treatment condition has any effect, it is simply to add/subtract a constant amount from each individual score

Front 5

What is the consequence of this basic assumption?

Back 5

- the adding or subtracting of the constant after the treatment condition does not change the shape of the population or the standard deviation

Front 6

What is the goal the hypothesis test?

Back 6

- to determine whether the treatment has any effect on the individuals in teh population

Front 7

What does a research study involve?

Back 7

1. selecting a sample

2. treating the sample

3. recording scores

Front 8

What two "real" components allow is to make inferences about the unknown population/ hypothetical population?

Back 8

- original population

- treated sample

Front 9

Hypothesis Test

Back 9

1. State the Hypothesis

2. Set the Criteria for a Decision

3. Collect Data and Compute Sample Statistics

4. Make a Decision

Front 10

State the Hypothesis

Back 10

- state two opposing hypothesis about the population parameters

Front 11

Null Hypothesis

Back 11

- most important of the two hypothesis

- treatment has no effect

- no change/effect/difference

(population mean stays the same)

Front 12

Alternative (scientific) Hypothesis

Back 12

- treatment had an effect

- unknown population mean does not equal the known population mean

Front 13

Directional vs Non Directional Test

Back 13

a difference in one direction vs any difference

Front 14

Step 2: Set the Criteria for a Decision

Back 14

- Data from sample will be used to evaluate the credibility of the null hypothesis

Front 15

What does it mean to formalize the decision making process?

Back 15

- use the null hypothesis to predict the kind of sample mean that ought to be obtained

- sample means consistent with the null and vice versa

- provide boundaries for what is "near the mean" what is "far away from the mean"

- Dividing the DSM into two sections

Front 16

The DSM is divided into...

Back 16

(2 sections)

1. Sample means that are likely to be obtained if null is true; that is, sample means that are close to the null hypothesis; body

2. Sample means that are very unlikely to be obtained if null is true; that is, sample means that are very different from the null hypothesis; tails

Front 17

If the null is true where does the sample mean fall?

Back 17

- inside the high probability section of the distribution "close" to the population mean

Front 18

The Alpha Level

Back 18

- level of significance

- small probability used to identify low probability samples

Front 19

Alpha Levels by the Numbers

Back 19

- = 0.05 (aka 5%) z = ±1.96

- = 0.01 (aka 1%) z = ±2.58

- = 0.001 (aka 0.1%) z = ±3.30

Front 20

Critical Region

Back 20

- extremely unlikely values, as defined by the alpha level

- reflect values not consistent with the null

- sample mean located in critical region; reject the null

Front 21

If the alpha level is increased from a = .01 to a = .05, the size of the critical region also increases. True or False?

Back 21

- True, a larger alpha means that the boundaries for the critical region move closer to the center of the distribution

Front 22

Step 3: Collect Data and Compute Sample Statistics

Back 22

- find z score that describes exactly where the sample mean is located relative to the hypothesized population mean from null

(this hypothesized population mean is the DSM of n = sample size where null were true)

- plot z score of sample mean relative to null mean which is a DSM where n = sample size and the mean remains the same from the known population

Front 23

Step 4: Make a Decision

Back 23

ONLY 2 CHOICES

- reject the null (treat has an effect // z score in the critical region)

- accept the null ( treat has no effect // z score not in the critical region)

Front 24

THE ONLY TWO CONCLUSIONS

Back 24

- reject the null hypothesis

- fail to reject the null hypothesis

Front 25

Trial Metaphor

Back 25

- not trying to prove innocent or guilty; your prove guilty or not guilty

~ reject the Null or fail to reject the null

Front 26

Freestyle Hypothesis Test

Back 26

- we are reorganizing the population into a population of samples in order to make a statement about the population with out testing every individual ; we are then finding efficacy of a treatment by comparing the average our sample to what would be expected of a sample without the treatment; we then reject or fail to reject a null hypothesis

Front 27

Generally, a large value for a test statistic indicates...

Back 27

- large discrepancy between sample and null hypothesis

Front 28

what is the null hypothesis ...

Back 28

- its reinterpretation of the known population in terms of the treatment sample size

Front 29

Large deviation / Critical Hits / far from 0 / in the tails tell the researcher...

Back 29

- its is likely the treat condition had an effect

- the null hypothesis should be rejected

Front 30

What can get fucked in the research study to lead to an erroneous conclusion?

Back 30

- picking an extreme sample not representative the population can cause the a strong treatment effect

ex if you randomly chose 20 English lit graduates to show the effect of flashcards on vocabulary improvement

Front 31

Type I Error

Back 31

- occurs when a researcher rejects a null hypothesis that is actually true

- typically means that the researcher is claiming a functional relationship when their is not one

- mislead by sample

Front 32

Type I Error occurs when the researcher unknowingly obtains a ...

Back 32

- extreme sample

Front 33

Can a sample mean be obtained that falls in the critical region from a sample that does not receive the treatment condition?

Back 33

yes, in fact statistically it is improbable that it will never occur given that the critical region is apart of the null hypothesis

Front 34

Alpha Level

Back 34

- the probability that the test will lead to a Type I error ( a failure to accept the null hypothesis)

- determines the probability of obtaining sample data in the critical region even though the null hypothesis is true

Front 35

Essentially the alpha level is used to

Back 35

- establish the probability that a sample was effected by the treatment condition by providing a line of demarcation

Front 36

The probability that Type I error will occur is equal to ...

Back 36

- the alpha level/ critical region

Front 37

Type II Error

Back 37

- failed to reject the null

- hypothesis test failed to detect a real treatment effect

- unlike Type I error whose probability is factored into the null hypothesis there is not absolute measure of probability for Type II error

- influenced by a variety f factors

- researcher can refit and refire

- represented by greek letter betta ~ (B)

Front 38

Two Possible Outcomes of Hypothesis Test

Back 38

1. the sample data provide sufficient evidence to reject the NULL and conclude that treatment has an effect

2. the sample data does not provide enough evidence to reject the NULL; fail to reject the NULL and treatment does not appear to have an effect

Front 39

By choosing the alpha level the researcher directly determines ...

Back 39

- the probability to Type I error (sample scoring critical hit --> false functional relationship)

- researchers determine their own risk of a false report

Front 40

Selecting an alpha level has two direct consequences...

Back 40

1. boundaries for critical region

2. probability of Type I Error

Front 41

State the consequences of raising the alpha level and decreasing the alpha level..

Back 41

- raising the alpha level increases Type I error by expanding the critical region

- by lowering the alpha level you decrease the chances of concluding that the treatment has an effect thus increasing Type II error

Front 42

what type of error would a small treatment effect produce ...

Back 42

Type II... failure to reject the NULL

Front 43

Reporting the Conclusion

Back 43

- report whether the treatment was statistically significant

- z = raw score, p < or > alpha

Example

The treatment with alcohol had a significant effect on the birth weight of newborn rats.

z= 3.00, p < .05

Front 44

Result is said to be Significant or Statistically Significant

Back 44

- unlikely to occur when the null hypothesis is true

- result is sufficient to reject the null hypothesis

Front 45

what does p > .01 entail?

Back 45

it means that the probability of that that treatment condition had an effect is greater than the alpha 1%

- a significant result will always be reported as a probability less than the alpha level

Front 46

Factors that Influence A Hypothesis Test

Back 46

- size between sample mean and original score (numerator of z score)

- variability of scores (deviations or variance and influences the denominator of the z score)

- number of scores in sample (denominator)

Front 47

Size of Mean Difference

Back 47

- larger the difference the larger the treatment effect, which produces a larger z score and increases the likelihood of finding a significant effect

Front 48

The Variability of Scores

Back 48

- high variability makes it hard to see patterns generally

- increasing the variability of the scores produces a larger standard error and a smaller value (closer to zero) for the z score

Front 49

The Number Scores in the Sample

Back 49

- increasing the number of scores in the sample produces a smaller standard error and a larger value for the z score

Front 50

The Fundamental Assumptions Behind Hypothesis Testing

Back 50

1. Random Sampling (equal chance)

2. Independent Observations (with replacement)

3. The Value of the standard deviation is unchanged by treatment (+/- a constant)

4. Normal Sampling Distribution (shape)

Front 51

Random Sampling

Back 51

- selected randomly to ensure that the sample is representative of the population

Front 52

Independent Observations

Back 52

- the probability of occurrence of scores are affected by each other

(ex with replacement) `

Front 53

The Value of the standard deviation is unchanged by treatment

Back 53

- in order to find standard error and to make inferences about the treatment population you need to know the deviation of the treatment population

- the assumption is made by researchers that the deviation does not change thus allowing them to find standard error and find a z score for the treatment sample

- also the implication that the treatment will act as a constant on the population

Front 54

Normal Sampling Distribution

Back 54

- in order to identify the critical region the distribution must be normal

Front 55

What is the most critical factor in the deciding whether the treatment has an effect/ reject the null?

Back 55

- the size difference between the treated sample and the original population

Front 56

What is the major difference between a one tailed test and a two tailed test?

Back 56

- a one tailed test allows you to reject the NULL when the difference between the sample and the pop. is relatively small, provided the difference is in a specific direction

- a two tailed test requires relatively large difference independent of direction

Front 57

Alpha boundary difference between one tailed and two tailed test at .05

Back 57

- one tail = 1.64 (in one direction

- two tail = 1.96 (+/-)

Front 58

Benefits and Downfalls of Two Tailed (non directional) Test

Back 58

- more rigorous/ more evidence needed to reject NULL

- stronger demonstration of treatment effect

- best when no direction is specified or their are competing predictions

- might fail to prove treatment significance (Type II Error)

Front 59

Benefits and Downfalls of One Tailed (directional) Test

Back 59

- more sensitive (smaller treatment effect can produce statistical significance)

- more precise because they test a directional effect

- should be used when their is strong justification for making a directional hypothesis

- can lead to Type I Error

Front 60

Two Major Limitations of Hypothesis Testing (in regards to establishing significance)

Back 60

- focus of hypothesis test is on data rather than the hypothesis

- demonstrating significance does not indicate magnitude of the treatment effect

Front 61

The First Limitation of the Hypothesis Testing

Back 61

- rejecting the NULL makes a statement about the data not the hypothesis

- Rejecting the NULL with a = .05 does not justify a conclusion that the probability of the NULL being true is less 5%

- conclusion reports a conditional statement about the probability about the sample in relationship to the NULL but not a statement about the NULL

Front 62

The Second Limitation of Hypothesis Testing

Back 62

- Statistical significance does not provide any real information about the absolute size of a treatment effect

- the test is making a relative comparison: the size of the treatment effect is being evaluated relative to standard error

Front 63

"The test is making a relative comparison: the size of the treatment effect is being evaluated relative to standard error...." what are the implications behind this fact?

Back 63

- if the sample size is large enough, any treatment effect, no matter how small, can be enough for us to reject the NULL

Front 64

Absolute size of treatment effect is ...

Back 64

- not evaluated.

Front 65

Effect Size

Back 65

- intended to provide a measurement of the absolute magnitude of a treatment effect, independent of the size of the sample being used

Front 66

Cohen's D

Back 66

- mean difference/ deviation

- treatment(unknown) population mean - no treatment(known) population mean / deviation

* remember deviation is assumed constant

Front 67

Cohen's D is an expression of effect size in terms of ...

Back 67

- deviations

- deviation standardizes Cohen's D

Front 68

why is a actual Cohen's D impossible in research?

Back 68

- you need the population mean after treatment

Front 69

Actual Calculations of Cohen's D

Back 69

M(treatment) - U(treatment) / deviation

Front 70

d= 0.4 indicates ...

Back 70

- treatment effect size was 0.4 standard deviations

Front 71

Even if their is substantial significance in treatment effect their can still substantial _____ between non treatment and treatment distributions.

Back 71

- Overlap

Front 72

Cohen's D measures...

Back 72

- degree of separation between two distributions in terms of standard deviation

- does not consider sample size

Front 73

Power

Back 73

- the probability that the test will correctly REJECT the NULL if the treatment has an effect

- find a treatment effect

Front 74

Power (by the numbers)

Back 74

1 - beta

Front 75

Power is...

Back 75

- the probability of correctly rejecting the NULL

Front 76

If the power of a test is 70% what is the probability of Type II error

Back 76

- 30%

Front 77

Type I Error is to the Critical Region as Type II Error is to ....

Back 77

- Power

Front 78

Factors that Effect the Power

Back 78

- Sample Size

- Alpha Level

Front 79

To calculate the power of a test you must determine...

Back 79

~what portion of the distribution on the right-hand side~

1. Locate the boundary for the critical region

2. Find the probability value in the unit table

multiply the standard error by the boundary of the critical region

(any point value over this number in the critical region)

3. find what portion of the treated DSM is greater than the critical region boundary point score

4. Find z score of the critical region boundary point score on the treated DSM

5. use the unit table and report the probability corresponding to shaded area

Front 80

The most critical step in determining the power ...

Back 80

- restating the alpha boundary as a z score of the unknown/treated DSM

Front 81

As the effect size increases....

Back 81

- the DSM of the right hand size moves farther to the right

- more samples are beyond the critical region boundary

- probability of rejecting the NULL goes up ---> power goes up

Front 82

Sample size and Power

Back 82

- directly related

- sample size effect standard error which in turn can increase or decrease the point score of the alpha boundary

- when the sample size is small the standard error is large the point score of the alpha boundary is large

- this will give the alpha boundary a high z score on the treated DSM thus reducing its Power

Front 83

If you increase the sample size the power will ...

Back 83

- increase

Front 84

Alpha Level

Back 84

- lowering the alpha lowers the power

- it moves the alpha boundary from the mean; giving it a higher point score; moving it farther into the treated DSM

Front 85

Changing a 2 tailed test to a one tail test does what to the power?

Back 85

increases the power by expanding the critical region (moving the alpha boundary to the left and closer to the known mean)