Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
30 Cards in this Set
- Front
- Back
|
goal of usability testing |
check that the system being deveoped is usable by the intended user population for their tasks |
|
|
goal of experiments |
test hypothesis to discover new knowledge by investigating the relationship between two or more things |
|
|
user testing's aim and facts n stuff |
aim:improve products few participants results inform design not perfectly replicable controlled conditions procedure planned results reported to developers |
|
|
research experiemnts aim and facts and stuff |
aim: discover knowledge many participants results validated statistically replicable strongly controlled conditions experimental design scientific paper reports results to community |
|
|
Whats the experimental process |
formulate hypothesis identify independant, dependant variables design a controlled experiment check for: - confounds - validity - reliability select representative participants randomly assign to conditions run experiment, collect data analyze results |
|
|
whats a hypothesis |
a suggest explanation of a phenomenom in experimentation you want the hypotehsis to be as specific as possible |
|
|
what you need to identify when testing hypothesis |
must identify what variables we think will lead to expected outcome must identify how manipulating these variables will result in expected outcome - if i provide keyboard shortcuts, users will be able to complete the tasks faster than with just menus clearly identify which variables will influence what outcomes, and how |
|
|
what are independant variables
|
those direcly manipulated as part of the experiment examples: - menu type: pie or vertical context - keyboard shortcuts: available,not available everything else should be kept constant |
|
|
dependant variables |
those that change in response to the independant variables examples: - completion time - error rates - user preference - quality of user response |
|
|
relationships |
independant variables are assumed to produce an effect on dependant variables values when manipulated |
|
|
nuisance variables |
any other factor that can affect the dependant variables examples: - time of day - handedness goal is to have as few of these as possible |
|
|
what are we seeking to reject in hypothesis testing? |
we are seeking to reject the null hypothesis |
|
|
null hypothesis |
there exists no relationship between manipulating the independant variables and teh resultant changes in the depedant variables |
|
|
what are the two conditions you need in experimental design and what do you do after you have tested both of them |
control condition experimental condition compare results |
|
|
control condition |
no experiment manipulations performed |
|
|
experimental condition |
experimental variable is manipulated |
|
|
validity |
are we measuring what we say we are measuring |
|
|
reliability |
if we run the experiemnt multiple times will we get the same results |
|
|
confounds |
are there variables we didnt control for which may be influencing the results we are obtaining |
|
|
types of validity |
internal validty: the changes in the dependant variables are caused by the independant variables external validty: results can be generalized to other settings, populations, tasks etc there is often tradeoff between the two. meaning the more tightly you control the experiment (to increase internal validty) the less generalizeable the results ecological validty: to what extent do the study conditions mimic those in real word |
|
|
what should you ask yourself when creating the participant pool |
are the study participants representative of the inteded user population? |
|
|
two options for assignging participants to conditions |
between-subjects within-subjects |
|
|
between-subjects |
each participant does one of the experimental conditions doesn't account for individual variability need more participants no learning effects (good) also known as "randomized experiements" |
|
|
within-subjects |
each participant completes all experimental conditions better able to account for individual differences requires fewer participants allows participants to make direct comparative statements learning effects are possible |
|
|
for large effects you need --------- participants, and for smaller effects you need ------- participants |
for larger effects you need less participants to gather info and for smaller effects you need more participants |
|
|
the more the participants the----- |
better because it increases confidence of findings |
|
|
ways to collect data |
software loggin: instrument software to cpature detailed timing, error information questionnaires administered before, during and/or after tasks structured interviews |
|
|
when analyzing results its good to take into account |
the variable of random chance is represented as the value p p= 0.05 means 5% chance it was just pure luck |
|
|
controlled experimentation advantages and disadvantages |
advantages: - high confidence in the phenomena that we observed disadvantage: - confidence applies to only narrow phenomena - can be difficult to create the right motivation for participants - cant see things like workarounds that participants develope over time in the field - results can be statistically significant without being practically significant |
|
|
fa |
ggot |
