Probability - Jim Pittman

Front 1

As a fish swimming in waters leaves no mark behind,

Back 1

So a free soul moves in the world unnoticed by others.

Front 2

What is the formula for the the Probability of A given B?

Back 2

Prob of A times Prob of B all over Prob of B - given independence of A and B.

[ P(A=m)*P(B=n) ] / P(B=n)

Front 3

What is the multiplication rule for independent events?

Back 3

P(AB) = P(A)P(B)

Front 4

What is the formula for Binomial Distribution?

Back 4

(N Ch k) p^k q^n-k

Where:

N = Size of overall group
k = Size of desired success or, what you are looking for
p = Probability or Rate of success
q = Probability or Rate of failure

Front 5

What is the binomial expansion?

Back 5

(p+q)^n = Sum from k=0 to n of (n C k) p^k q^n-k

Front 6

When is Mew = np?

Back 6

When it's binomial (n,p) distribution

Front 7

What are the two parameters of the normal curve?

Back 7

Mew and the standard deviation.

Front 8

What is the range of x on the normal curve?

Back 8

(-I , I)

Front 9

What is the equation for the normal curve in x? (w/o normalizing the values)

Back 9

y = 1/ (rt(2 pi sigma))*e^((-1/2))*(x-M)^2 / sigma ^2)

Where sigma is the SD

Front 10

What is the equation for the normal curve in z using normalized (standardized) values?

Back 10

y = 1 / (rt(2 pi sigma))*e^ (-1/2)*z^2

Where sigma is the SD

Front 11

What is the formula for standardizing a value?

Back 11

Z = ( x - M) / sigma

sigma = SD

In other words, you standardize a variable so that you can use the Normal Table and values of Phi.

Take your variable, subtract the Mean (Mew), then divide by the Standard Deviation (Sigma)

Front 12

What area of the curve does your CDF give you?

Back 12

The area to the left of z under the standard normal curve.

Front 13

What is the equation for Poisson approx?

Back 13

P( k successes) ~~ L^k * e^ -L / k!

Where L = lamda

Front 14

How can the mean of a probability distribution be defined?

Back 14

Over a finite set of numerical values x, it is the average of the values x weighted by their probabilities.

Front 15

What is the definition of expectation of a random variable X?

Also called the mean.

Back 15

average of all possible values of X, weighted by their probabilities.

Sum over all x, x * P( X = x)

Front 16

What is the addition rule for expectation for any two random variables X and Y?

Back 16

E(X +Y) = E(X) + E(Y)

No matter whether X and Y are independent or not.

Front 17

What is the definition for the expectation of an indicator?

Back 17

The expectation of the indicator of an event is the probability of the event.

E(I sub A) = P(A)

Front 18

What is the rule for multiplication fo random variables X and Y?

Back 18

IF they are Independent, then;

E(XY) = E(X)*E(Y)

But we need independence for this to occur, otherwise you cannot use this method.

Front 19

What kind of distributions can you have a CDF for?

Back 19

ANY KIND!!!!

Front 20

What can you do with a CDF?

Back 20

Calculate the probability of any event determined by X.

Front 21

Because probabilities must be nonnegative, what is a characteristic of the CDF?

Back 21

It must be a nondecreasing function of X.

Front 22

How do you get the density function from the CDF?

Back 22

Differentiate the CDF.

Front 23

How do you get the CDF from the density function?

Back 23

Integrate the density function.

Front 24

How do you find the Max of a collection of ind. rand. var?

Back 24

Look at it as an inequality:

If the max value of the set is less than or equal to, say x, then that can only happen if the entire set is less than or equal to x.

Front 25

How do you find the Min of a collection of ind. rand. var?

Back 25

Look at this as an inequality:

The min of the collection of variables is greater than a limit x only if all the variables in that set are greater than x.

Front 26

What do you get when you add two ind rand variables together?

Back 26

A new density function.

Front 27

How can you find the new density function formed from adding two ind rand variables?

Back 27

You can integrate the product of their density funcs over the correct area, then differentiate to get back down or you can plug both the density functions into the convolution formula.

Front 28

When working with joint densities of X and Y, you might not always have to get the distribution, sometimes getting the expectation is enough. How can you express the expectation of a joint function of X and Y?

Back 28

E(X*Y) = double integral of: x*y*f(x,y) dxdy

=E(X)*E(Y) <--- IF X and Y are independent!

Any other type of relationship, i.e. E( X + Y) , E( X / Y ), E ( sqrt( X^2 + Y^2 )) can also be expressed in that integral form using the density functions for the variables.

Front 29

What would be the CDF method of finding the distribution of Z = g( X , Y )?

Back 29

You'd want to FIND the CDF P(Z <_ z) by integrating the PRODUCT of the DENSITY functions fx(x) and Fy(y) over the region in the xy-plane where g(x,y) <_ z.

Then, after doing that you have the CDF of your variables so , if you differentiate that thing you get the new density function of the joint variables that you were looking for.

Front 30

What is the addition rule for expectation?

Are there any necessary conditions for this rule to hold?

Back 30

E( X + Y ) = E(X) + E(Y)

No, even if X and Y are dependent variables you can always do this.

Front 31

What distribution does the phrase "n independent Bernoulli (p)" make you think of?

Back 31

Binomial ( n, p )

n = outcome space

p = rate or probability

Expectation for the variable = n*p

Front 32

What is the density convolution formula?

Back 32

f x+y (z) = integral of f( x, z-x ) dx

This integral is from neg infinity to infinity.

Subsitute in (z-x) for every y in the density function for y, make sure x is plugged in as the variable for the density function for x.

You'll end up with integral of: fx(x) * fy(z-x) dx

NOTE: If X and Y are non-negative, then the lower limit of integration in the convolution formula can be changed to zero, and the upper limit to z, since fy(z-x) = 0 for all x > z.

Front 33

What is the convolution formula a special case of?

Back 33

It is a special case of the formula for the density of X + Y when f(x*y) = fx(x) * fy(y) by INDEPENDENCE.

This formula leads to a new density which is the density of the sun of the two random variables which are assumed to be independent.

Front 34

For ind. normal variables X and Y, what is the relationship for the density functions of X / Y and Y / X and what is the integral for them?

Back 34

f x/y (z) = f y/x (z) by symmetry between X and Y.

Integral from negative infinity to infinity of: abs valu (x) * f x,y ( x , xz ) dx

Plug in your appropriate density functions and variables in their proper places and change the bounds of integration according to the range of your density functions.

Front 35

So, we have two uniformly distributed variables over a region, what is the formula for the marginal density of f x (x) ?

Back 35

Well, since we have a DENSITY that we are looking for, we will be using an integral.

For f x (x) we'll have integral from negative infinity to infinity of: f(x,y) dy

So, the MARGINAL DENSITY for f x(x) of two joint variables is an INTEGRAL with respect to Y.

Just remember that for marginal density it will be an integral of f( x,y ) but integrated with respect to the opposite variable of the marginal density that you are looking for.

Front 36

What is the formula for the marginal density of f y(y) if X and Y are two uniformly distributed joint variables?

Back 36

Integral from negative infinity to infinity of: f( x,y ) dx <-- Notice that this is w/ respect to x and you are looking for the marginal density of f y(y).

So, this is for the joint density function - if you don't have it already you may have to get it by the CDF or Convolution method. CDF meaning integrate the product of the two density functions then differentiate back down for density, Convolution meaning to plug into the Convolution formula and integrate.

Remember: Take the integral with respect to the variable opposite of the variable for the marginal density that you are looking for.

Front 37

How can you check if uniformly dist variables X and Y are independent or not?

Back 37

if f ( x,y ) = f x(x) * f y(y) then yes, they are independent.

If that equation is not satisfied, then those variables in question are NOT independent.

Front 38

Say we have X and Y independent exponentially distributed rand variables with respective parameters lamda and Mew.....

What exactly does that mean and how can you calculate P ( X < Y ) ?

Back 38

If we are given two ind. exp. rand variables w/ parameters. Then this means that each variable can be represented by its own density function.

So, then their joint density is the PRODUCT of those two functions. (By independence).

Then, P( X < Y ) is going to be found by integrating a double integral of this product over the appropriate area.

Front 39

If you are asked what the probability is that a standard normal variable has value between a and b, what it the procedure to solve this?

Back 39

Since it is given as standard normal you only have to use the Normal Table and subtract Phi(a) from Phi(b) since the table will give you everything to the left, so Phi(b) - Phi(a) gives the prob that your stand. norm. variable has value between a and b.

Front 40

If you are given that X has normal density f(x) = c/x^4 for x > 1 and f(x) = 0 otherwise, then how can you find out what c is, if c is a constant?

Back 40

If the density function is zero for all values of x other than x > 1, then we know that the area under the function from 1 to infinity must add up to 1. Integrate c/x^4 from 1 to infinity, set the result equal to 1 and solve for c.

Front 41

X has normal density f(x) = 3/x^4 for x > 1 and 0 otherwise. How can you find the expectation of the variable X?

What about the Variance of X?

Back 41

E(X) is easy enough with a normal density function, we'll just want to use the definition to integrate it (also times x) over the area where it is not 0.

So, integrate from 1 to infinty: (x)* 3/x^4 dx.

Variance can be calculated the same way using the definition so E(x^2) = integral from 1 to infinty of (x^2) * 3/x^4 dx, then finish the formula for Variance.

Front 42

If you are given normal variable X with prob dens f(x) = 1/(1+x)^2 for x > 0 and f(x) = 0 otherwise, how would you go about finding P ( X > 3) ?

Back 42

We have the density function already, and know that it is normal, so we can just integrate this from 3 to infinity, the resulting number is P ( X > 3)

Front 43

If you are given normal variable X with prob dens f(x) = 1/4 for x > 3 , and then you have X1, X2, X3, and X4 each with the same dist as X and independence, how could you find the chance that exactly 2 of those four variables is greater than 3?

Back 43

Well, they all have identical probabilities and are all independent events - this makes a simple success or failure type of situation and hence binomial with n = 4 , and p = 1/4

So we have (4 Ch 2)*(1/4)^2 * (3/4)^2

Front 44

When finding probabilities and expectations for normal variables, what is the mechanical difference between E(x) and P( X > some value) ?

Back 44

From the def of expectation, we know that when integrating we first multiply by the variable to the first power.

When finding the probability, we just have to integrate over the appropriate area, no tweaks are mandatory.

Front 45

The probability of a random time falling in any interval can be found from what?

Back 45

The survival function - this pertains to Exponential and Gamma distributions.

Front 46

What is the formula for the survival function of the exponential function?

How about Mean and Standard Deviation?

Back 46

P(X > t) = e^-Lt ( t >_0)

Both the Mean and the Standard Deviation will be 1/L for the Exponential Distributions.

Front 47

What do we generally call E[ X^n ] ?

Back 47

The nth moment of X

Front 48

What is the formula for the moment generating function?

Back 48

M x(t) = Sum from n = 0 to infinity of: E ( X^n ) *( t^n / n! )

Which by linearity is equal to E [ e^xt ]

Front 49

What does M x^n (0) equal?

Back 49

E [ x^n ]

By Taylor's Theorem.

Front 50

For moment generating function M x (t) = E [ e^xt ] what does the sum formula look like and how do you FIND the expectation on that?

Back 50

You can just use the definition:

Sum from k = 1 to whatever you are given since it is discrete, of e^kt * P [ x = k ]

You should an exponential polynomial in terms of t.

Like 1/3 * e^3t + 4/5 * e^5t + ... which is your moment generating function M x(t), the first derivative of this function evaluated at 0 is E [ x^1 ], the second derivative at zero is E [ x^2 ], the second moment of x, and it goes on in that fashion.

Front 51

Say you have a variable x with density function f(x) = 5e^-5x for x > 0 and zero otherwise. What is M x (t)?

Back 51

We know that M x(t) = E [ e^xt ]

So, take an integral from x = 0 to infinity of e^xt * the density function.

So we have int. from x = 0 to infinity of e^xt * 5e^-5x dx

Front 52

Say that you're given that M x(t) = 5 / 5-t and you want to find a formula for E [ x^n ]. How might you go about that?

Back 52

Take the definition, and tweak it into a recognizable sum formula, THEN tweak that sum formula into t^n / n! with a coefficient and whatever the coefficient of t^n / n! is will be the E [ x^n ].

For this one in particular, divide everything by 5 and you have the sum of x^n, so you have sum from 0 to infinity of (t/5)^n. Once you have that it can be tweaked into the form you want and your desired coefficient becomes n! / 5^n.

Front 53

Say that X is uniform on [ a, b ], how can you find a formula for E [ x^n ] ?

Back 53

Since x is uniform, set up the integral from a to b of: 1 / (b -a) * x^n dx

Then integrate that, and whatever you get is E [ x^n ].

Front 54

How can you transform a probability generating function in a moment generating function?

Back 54

Probability Generating Func = G x(t)

Moment Generating Func = M x(t)

Prob uses "z" and Moment uses "e^t"

Prob = sum of z^k * P [ x = k ]

Moment = sum of e^kt * P [ x = k ]

Front 55

Say that you have x as a discrete variable with G x(z) = z / (4 - 3z)^2.

What is this and how can you find E [(X) falling 3] ?

(That falling 3 means that you want a MOMENT btw)

Back 55

This is a Probability Generating Function.

We are looking for the third moment of G x(z).

This can be found by taking the third derivative of the function and evaluating it at "1".

That's it.

Front 56

Say that you have x as a discrete variable with G x(z) = z / (4 - 3z)^2.

How can you find E [ X^3 ] ?

Back 56

We are given a probability generating function and we want to get to the moment generating function.

This can be done by replacing "z" with "e^t" and then for X^3 you take the third derivative of that function and evaluate it at zero.

Front 57

Say that you have x as a discrete variable with G x(z) = z / (4 - 3z)^2.

What is this thing?

Back 57

This is a probability generating function.

Front 58

Say that you have x as a discrete variable with G x(z) = z / (4 - 3z)^2.

How can you find P [ X = 3 ] ?

Back 58

Remember what the PGF is, it's a power series whose coefficients are probabilities.

Here, we need the coefficient of z^3 in the power series for this function.

by Taylor's Theorem, that coefficient should be the third derivative of G at 0, divided by 3 factorial.

Front 59

How does the method of tailsums work?

Back 59

Use an inequality, say, you are looking for the min of something, or you want to know P [ X < 4 ] - for that particular case, you can just add up all the probabilities for 1, 2, and 3.

Don't forget that when using tailsums you must start your series at k = 1, you cannot include zero or things will be counted too many times.

Find them, add them , done.