Statistics Midterm - Spring 2017

Front 1

The General Linear Model equation

Back 1

Yi = β0 + β1X1i + εi

Front 2

The Generalized Linear Model equation

Back 2

Logit(Y) = β0 + β1X1 + ε

Front 3

What is the difference between the Generalized Linear Model and The General Linear Model wrt outcome?

Back 3

Outcome is Yi for GLM and Logit(Y) for Generalized Linear Model

Front 4

What are the types of logistic regression? (based on types of outcomes)

Back 4

○ Binary logistic (2 categories)

○ Multinomial outcomes (3 or more categories)

Front 5

Binary logistic regression

Back 5

Categorical or continuous variables and outcome is 2 categories

Front 6

When do we use a logistic regression?

Back 6

-Categorical/continuous/nominal variables (binary logistic or multinomial outcomes)

-Can look at interactions (same way you would in a linear regression) by computing product terms. DIFFERENCE is outcome is categorical/nominal.

-Can look at interactions same way you would in a linear regression by computing product terms.

Front 7

Why do we use logistic regression?

Back 7

"1. Prediction is going to be based on likelihood, not levels

2. Assumptions of ordinary least squares regression are violated by having a dichotomous variable.
3. Thus, we need to expand the General Linear Model (which only can be used for continuous outcomes) and make it even more general and call it the Generalized Linear Model (which can be used for categorical, ordinal, and very skewed or weird distributions)"

Front 8

Non-Parametric Analyses: What are the Two Types of Chi-Square Tests?

Back 8

Chi-square Goodness of Fit & Chi-square Test for Independence

Front 9

Major violations of OLS assumptions that call for use of logistic regression

Back 9

-Measurement assumption

-Linear assumption

-Heteroscedastity and non-normal residuals

-Scale of predicted score of the model needs to match the scale of the observed model

Front 10

Major violations of OLS: measurement assumption

Back 10

Outcome is not on interval or ratio scale

Front 11

Major violations of OLS: linear assumption

Back 11

relationship between predicted values is assumed to be NON-linear.

Front 12

Major violations of OLS: Heteroscedastity and non-normal residuals
assumption

Back 12

Variance in residuals is not consistent across the predictor. At extreme levels, the violation is bigger (see logistic regression curve)

Front 13

Major violations of OLS: assumption that scale of predicted score of the model matches the scale of the observed model

Back 13

Violated in logistic regression because we are predicting probabilities and not values of the outcome

Front 14

Logistic Regression Curve

Back 14

Variance in the residuals is not consistent across the predictor. At extreme levels, violation is bigger…. That represents heteroscedastity (violation of homoscedastity)

○ Distance between predicted and observed is not constant
"

Front 15

Logit(Y) = β0 + β1X1 + ε <--what is being predicted?

Back 15

Predicting likelihood that outcome is either 0 or 1. Very similar to GLM and linear regression equation BUT uses the Logit(Y)

Front 16

Probabilities/Odds: What is the range that probability can take?

Back 16

"Expressed in a ratio from 0 to 1

□ 0 - definitely not going to happen

□ 1 - definitely will happen

Front 17

Probabilities/Odds: What does P(Y=1) refer to?

Back 17

Probability that event will happen (aka probabilty that Y will be classified as 1 on the DV)

Front 18

Probabilities/Odds: Express P(Y=0) in terms of P(Y=1)

Back 18

P(Y=0) = 1 - P(Y=1)

Front 19

Probabilities/Odds: Why don't we use P(Y=1) = β0 + β1X1 + ε equation for logistic regression?

Back 19

Problems with this are based on mathematical characteristics of probability --- observed and predicted values are restricted between 0 and 1. Predicted values may fall less than 0 (i.e. could get a negative probability).

Front 20

Probabilities/Odds: What are the Odds?

Back 20

Probability that Y = 1 relative to probability of Y =/= 1
P(Y=1)/(1- P(Y =1))

Front 21

Probabilities/Odds: What does it mean if Odds are less than 1?

Back 21

You're less likely to be categorized as a 1 than 0 on the DV

Front 22

Probabilities/Odds: What is range of Odds?

Back 22

Ranges from 0 --> 1 --> infinity.

Front 23

Probabilities/Odds: What happens as the difference between P(Y) = 1 and P(Y)=/=1 gets bigger?

Back 23

The bigger the diff between the two, the closer the odds ratio gets to infinity or 0!

Front 24

Probabilities/Odds: What does it mean if Odds = 1, is > 1, or is <1, respectively?

Back 24

= 1 equal likely to be in either condition

>1 = more likley to be in category coded as 1

<1 = more likley to be coded as a 1 than a 0

Front 25

Probabilities/Odds: Why are odds better than Probability? Why are they still nonoptimal?

Back 25

Better than probability because odds have no upper limit!

Nonoptimal because lower limit of 0 -- could create problems because you could get predicted values below 0.

Front 26

Probabilities/Odds: What is Logit?

Back 26

Logit = Natural Logarithm of the odds

Front 27

General Linear Model Equation

Back 27

Yi = β0 + β1X1i + εi

Front 28

General Linear Model: what is Yi?

Back 28

Yi = the value of the DV or outcome; sometimes referred to as the expected value and denoted as Ŷ

Subscript “i” indicates that we are dealing with one score or case"

Front 29

General Linear Model: What is β0?

Back 29

β0 = intercept term, constant effects, value of Y when all X’s are equal to zero.

Front 30

General Linear Model: What is β1?

Back 30

β1 = the regression coefficent for variable X, how much does X influence Y

Front 31

General Linear Model: What is X1i?

Back 31

X1i = the score of x for person or case i

Front 32

General Linear Model: What is εi?

Back 32

εi = residual, error, term. How far is the person’s actual score from the expected score

Front 33

What is the logistic regression equation?

Back 33

Logit(Y) = β0 + β1X1 + ε

Front 34

What is the Generalized Linear Model?

Back 34

-Used for analyses when OLS assumptions are not met

-Flexible generalization of ordinary linear regression that allows for response variables that have error distribution models other than a normal distribution

- Allows the linear model to be related to the response variable via a link function (Logit)

- Allows the magnitude of the variance of each measurement to be a function of its predicted value

Front 35

Non-Parametric Analyses: What are non-parametric tests & why are they used?

Back 35

Make no assumptions about the parameters of the population the sample is drawn from

Front 36

Non-parametric analyses: When are they used?

Back 36

Used when assumptions of parametric analyses are violated (outcome variable is on interval/ratio scale, residuals are normally distributed/demonstrate homoscedasticity, linear, scale of predicted score is the same as scale of observed variable)

Front 37

Non-parametric analyses: alternative to one-sample t-test

Back 37

Chi-square Goodness of Fit

Front 38

Non-parametric analyses: alternatives to independent samples t-test

Back 38

Chi-square Test for Independence, Mann-Whitney (M-W) U Test

Front 39

Non-parametric analyses: alternative to Paired-samples t-test

Back 39

Wilcoxon Signed-Rank Test

Front 40

Non-parametric analyses: alternative to Between Subjects Factorial ANOVA

Back 40

Kruskal-Wallis Test

Front 41

Non-Parametric Analyses: alternative to Repeated Measures Factorial ANOVA

Back 41

Friedman Test

Front 42

Non-Parametric Analyses: alternatives to Pearson’s Correlation Coefficient

Back 42

Chi-square test for independence, logistic regression

Front 43

Non-Parametric Analyses: alternative to Ordinary Least Squares regression

Back 43

Logistic regression

Front 44

Non-Parametric Analyses: What is Chi-square Goodness of Fit used for?

Back 44

Uses sample data to test hypotheses about the shape or frequencies/proportions of a population distribution. Evaluates how well the obtained data fit the population proportions specified by the null (i.e., what is expected)

Front 45

Non-Parametric Analyses: What are Chi-square Test for Independence used for?

Back 45

Uses the frequency data from a sample to evaluate the relationship between two (nominal) variables in the population

Front 46

Non-Parametric Analyses: How to calculate df for goodness of fit chi-square test

Back 46

df = c – 1(c = number of categories)

Front 47

Non-Parametric Analyses: How to calculate df for chi-square test of independence

Back 47

df = (R-1)(C-1)(R = categories in the row variable; C = categories in the column variable)

Front 48

Why do we use logistic regresion?

Back 48

Having a dichotomous variable violates many of the assumptions of OLS regression --> Prediction is going to be based on likelihood, not levels

Front 49

Probabilities/Odds: How to convert from probabilities to logit?

Back 49

P(Y=1)/P(Y=0) = P(Y=1)/(1-P(Y=1)<--odds

LN(P(Y=1)/(1-P(Y=1)) <--logit

Front 50

Probabilities/Odds: How to convert from logit to probability?

Back 50

Odds (Y) = elogit(y)

P (Y=1) = elogit(y)/(1+elogit(y))

Front 51

Probabilities/Odds: Probabilities/Odds: LN(0) = LN(small#) = LN(.98) =
LN(1) = LN(larger numbers) =

Back 51

"LN(0) = undefined. "the limit does not exist!"

LN(.00001) is really negative

LN(.01) is negative

LN(.98) = increasing towards 0

LN(1) = 0

LN(larger numbers) = gets larger"

Front 52

What are the primary assumptions of logistic regression?

Back 52

Shared with OLS assumptions:

1. specificity assumption - model contains all relevant predictors and no irrelevant predictors. This is easier said than done, and often violated.

2. multicollinearity - still bad.

Specific to logistic regression

3. Outcomes must be statistically independent - a case can only be in one category (e.g. one cannot be both male and female).

4. Mutually exclusive and collectively exhaustive - a case cannot be in more than one category AND every case must be in one category (e.g. alive or dead)

Front 53

What are the three broad pieces of information that are provided by a logistic regression analysis?

Back 53

Shared with OLS assumptions:

1. Model fit - How well do all of the variables in the model together predict the outcome

2. Regression coefficient - What is the influence of each individual predictor in accounting for the outcome

Specific to logistic regression

3. Classification - How well does the model correctly classify cases into target (1) and non-target (0) group. We'll talk about this more later….

Front 54

What is an odds ratio & how do you interpret it in logistic regression?

Back 54

• analogous to a standardized regression coefficient in OLS regression

• Estimates the increase (or decrease) in odds of membership in the target group for a one-unit increase in the predictor while controlling for the other predictors in the model

Front 55

What does an odds ratio of greater than 1 mean in logistic regression?

Back 55

Increase in the likelihood of being in the target group as the predictor increases

Front 56

What does an odds ratio of 1 mean in logistic regression?

Back 56

No impact of the predictor on the outcome

Front 57

What does an odds ratio of less than 1 mean in logistic regression?

Back 57

Decrease in the likelihood of being in the target group as the predictor increases

Front 58

"Classification: Sensitivity = "

Back 58

Sensitivity = 100 * (d/(b+d))

Front 59

Classification: Sensitivity is...

Back 59

Correct identification of true positives

Front 60

Classification: Specificity is...

Back 60

correct identification of true negatives

Front 61

"Classification: Specificity = "

Back 61

specificity = 100 * (a/(a + c))

Front 62

"Classification: Positive Predictive Value = "

Back 62

Positive Predictive Value = 100 * d/(c+d)

Front 63

Classification: Positive Predictive Value is...

Back 63

percent of predicted 1 that are 1

Front 64

"Classification: Negative Predictive Value = "

Back 64

Negative Predictive Value = 100 * (a/(a + b))

Front 65

Classification: Negative Predictive Value is...

Back 65

percent of predicted 0 that are 0

Front 66

EFA/PCA: What are some design issues to consider when planning a PCA or EFA study?

Back 66

1. Sample size

2. Number of variables/indicators per common factor (EFA): 3-5 per factor

3. Content validity4. Measurement

Front 67

EFA/PCA: what are sample size recommendations?

Back 67

At least 100 participants and STV >= 5(Subject-to-variables ratio = STV ratio).

Front 68

EFA/PCA: what are recommended number of variables/indicators per common factor?

Back 68

3-5 per factor

Front 69

EFA/PCA: what are content validity considerations?

Back 69

-Inadequately covering all domains of interest could lead to the failure to identify relevant common factors

-Including irrelevant variables could lead to the false identification of irrelevant common factors to the construct of interest

Front 70

EFA/PCA: what are measurement considerations?

Back 70

Variables should be measured at interval or ratio level

Front 71

EFA/PCA: What are the four steps of conducting a PCA/EFA?

Back 71

1. Choose an estimator/extraction method

2. Determine number of factors

3. Select a rotation

4. Interpret solution (may need to repeat steps 2 and 3)

Front 72

EFA/PCA: Most commonly used extraction procedures

Back 72

Maximum likelihood

Principle axis factoring

Front 73

EFA/PCA: criteria for choosing the correct number of factors

Back 73

1. Kaiser-Guttman Rules (eigenvalues over 1) =>eigenvalues measure total variance

2. Scree test (“scree plot”, really) => identify where the plot levels off

3. Interpretability"

Front 74

EFA/PCA: What is the purpose of rotation?

Back 74

-The goal of rotation is to redefine the factor loadings to obtain simple structure

-Each factor should have indicators with strong loadingso Each indicator should load strongly on only one factor

-Helps with interpretability

Front 75

EFA/PCA: difference between orthogonal and oblique rotation

Back 75

• Orthogonal - uncorrelated factors

• Oblique - correlation among factors"

Front 76

EFA/PCA: What two things that guide interpretation of factors?

Back 76

• Theoretical interpretation => do the factors and constructs make sense from theory?

• Obtain simple structure

Front 77

EFA/PCA: What is simple structure?

Back 77

• Each factor should have indicators with strong loadings

• Each indicator should load strongly on only one factor

Front 78

EFA/PCA: What is the characteristic of a bad factor?

Back 78

A good factor has AT LEAST three items that significantly load onto it and no other factors; bad factor has 2 or less

Front 79

EFA/PCA: What are two types of bad indicators/items?

Back 79

• Items that do not load onto any factor

• Items that load onto two or more factors

Front 80

CFA/SEM: What is the primary distinctions between EFA and confirmatory factor analysis (CFA)?

Back 80

• EFA is a “bottom-up” or data-driven process. The analyst submits the data to EFA, and EFA dictates the results.

• CFA is a “top-down” or theory-driven process. The analyst a-prior specifies the model to be tested, and the CFA provides information about how well the data fit the specified model.

Front 81

CFA/SEM: What are the primary advantages of CFA over EFA?

Back 81

• More methodologically rigorous

• CFAs are theory driven and more specific (and more parsimonious) than EFAs

• Likelihood of obtaining good model fit is reduced; HOWEVER, we can have more confidence in CFA solutions than EFA solutions and they are more likely to replicate across samples

• Provides ABSOLUTE and RELATIVE goodness of fit indicators

• More flexible (can do a lot more)• Can evaluate INVARIANCE

• Maps onto construct validity very well

• Produces UNSTANDARDIZED, as well as standardized solution

• Can incorporate error theory

Front 82

CFA/SEM: CFA/Structural Equation Modeling (SEM) Graphical Notation - what does square represent? what about circle?

Back 82

Squares for indicators, circles for factors

Front 83

CFA/SEM: CFA/Structural Equation Modeling (SEM) Graphical Notation - what do bidirectional arrows indicate? unidirectional?

Back 83

bidirectional arrows for correlations, unidirectional arrows for factor loadings/directional relationships

Front 84

CFA/SEM: CFA/Structural Equation Modeling (SEM) Graphical Notation - Double Arrow Curve with both ends pointing to same latent variable

Back 84

variance of latent variable

Front 85

CFA/SEM: Factor Loadings

Back 85

o The effect of the factor on the observed variableo Indicates how well the indicator “represents” or loads onto the factor

o CFA provides significance tests for factor loadings and provides standardized (-1 to 1) and unstandardized estimates"

Front 86

CFA/SEM: error variances

Back 86

Error variance = the residual, error term, or unique variance. How much of the observed variable or indicator is NOT accounted for by the factor

Front 87

CFA/SEM: error covariances

Back 87

Associations among residual terms

Front 88

CFA/SEM: Factor variance

Back 88

How much variability is there in the factor across participants

Front 89

CFA/SEM: Factor covariance

Back 89

The degree to which factors are associated (correlated)

Front 90

CFA/SEM: what is the Structural Component of SEM?

Back 90

the factor relationships to each other

Front 91

CFA/SEM: what is the Measurement Component?

Back 91

the relationship of indicators to factors