Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
21 Cards in this Set
- Front
- Back
|
Quantitative Evaluation Methods |
User performance data - system use data collected - specific aspects should be targeted Controlled experiments - scientific method |
|
|
Desired outcome of controlled Experiment |
statistical inference |
|
|
Experimental Steps |
- Testable Hypothesis (+ null hypothesis) - State independent variables - Choose Dependent Variables - Nuisance Variables - Tasks - Protocol - Formal Explicit Experiment Design - Group participants - Analyze/interpret |
|
|
Goal of experiment Design |
Guard against ambiguous results |
|
|
Reasons for less distinguishable results |
poorly chosen task/maybe no difference |
|
|
Poor experiment design effects |
misleading results large spread in values |
|
|
Individual subject difference may ____ |
pose as nuisance variable |
|
|
Within Subject comparison |
- subjects get multiple treatment conditions - comparison is internal to each subject - fewer subjects required - greater statistical power warning: exposure to one condition may contaminate subject for another condition |
|
|
Between Subject comparison |
- subject exposed to one condition - less power - more subjects (more individual differences) |
|
|
Statistical vs Practical Significance - importance?- consequences on design? |
When N is large, a small change might be statistically significant. large vs small keyboard - takes up real estate but is only slightly faster |
|
|
Statistical Significance : |
The probability that our claims are correct |
|
|
Variations in collected data could be due to: |
Normal Variation Real Differences in data |
|
|
Types of T-Tests |
- Comparing sets of independent observations - Paired Observations - non-directional (two-tailed) - Directional (one-tailed) |
|
|
ANOVA lets you |
examine multiple independent variables at the same time |
|
|
Factor Factor Level |
Independent variable specific value of independent variable |
|
|
Counterbalancing |
- do factor levels in different orders |
|
|
Mixed factor example |
Between and within subjects within - keyboard type between - size |
|
|
Types of threats to validity |
Construct - measuring what we think? Internal - nuisance? Hawthorne? Statistical - fluke results? External - generalize? Ecological - representative of real tasks? |
|
|
Experiment tasks Natural vs Artificial |
Natural : more transferable Artificial : more control |
|
|
Main effect (ANOVA) |
effect of variable collapsed across all levels of other variables
|
|
|
Interaction Effect |
Effect of variable differs depending on level of another variable |
