Probability Midterm 2

Front 1

Expected Value (definition)

Back 1

Process of averaging when random variable is involved

Front 2

Expected Value EX[g(x)] (continuous equation)

Back 2

EX[g(x)]
= integral(g(x)f(x)dx,-inf.,inf.)

Front 3

Expected Value EX[g(x)] (discrete equation)

Back 3

EX[g(x)]
= sum(g(xi)P(xi),all i)

Front 4

Mean

Back 4

same as expected value but when g(x)=X.

1)mean is weighted average value of all xi values

2) the mean is center of gravity of the probability density function

Front 5

different notations for mean

Back 5

E[X]
=muX
_
=X

Front 6

Mean (discrete equation)

Back 6

sum(xi*PX(xi),all i)

Front 7

Mean (continuous equation)

Back 7

integral(x*fX(x)dx,-inf.,inf.)

Front 8

Variance (definition)

Back 8

spread of r.v. around mean values. average values for error. Larger variance = larger spread. Smaller variance means closer around the center.

Front 9

Variance (notation)

Back 9

var[X]
=sigmaX^2

Front 10

Variance (discrete equation)

Back 10

=EX[(X-muX)^2]
=sum((xi-muX)^2*P(xi),all i)

Front 11

4 properties of variance

Back 11

1) variance is a measure of the spread about the mean value

2) variance is a positive value

3) variance is moment of inertia of the P.d.f. about the mean

4) if σX²=0 then P(X=μX)=1 (ie X is a constant)

Front 12

If probability density fu is even symmetric about x=a

Back 12

then μX=a since f(a-x)=f(a+x)

Front 13

example: random variable X is uniformly distributed on interva [a,b], findmean and variance of random variable X

find: mean and variance

Back 13

mean=μX=E[X]=
∫(x*(1/(b-a)),x,a,b)=(a+b)/2 <-- midpoint

var[X]=σX² = E[(x-μX)²]
=∫((x-μX)²*(1/(b-a)),x,+a,b)
=(1/12)*(b-a)²

Front 14

example ***

you are playinga game with a friend and throwing a pair of dice, if the sum of the up faces is a prime number you win corresponding number of dollars. and if it is not a prime number you lose corresponding number of dollars. how much do you expect to win or to lose?

Back 14

X={2,3,4,...,12}

E[X]=∑(xi*P(xi),all i)
= (1/36)*2+(2/36)*3-(3/36)*4+(4/36)*5-(5/36)*6+(6/36)*7-(5/36)*8-(4/36)*9-(3/36)*10+(2/36)*11-(1/36)*12
=-1.89

LOSE

Front 15

example:

given X={2,3,5,6}

f(x)=0.2*∂(x-2)+0.1*∂(x-3)+0.4*∂(x-5)+0.3*∂(x-6)

find: mean and variance

Back 15

_
X
=∑(xi*P(xi),all i)
=2*0.2+3*0.1+5*0.4+6*0.3
=4.5


σ²=E[(x-4.5)²]
=∑(xi*P(xi),all i)
=(2-4.5)²*0.2+(3-4.5)²*0.1+(5-4.5)²*0.4+(6-4.5)²*0.3
=2.25

Front 16

mode (definition)

Back 16

value of X where f(x) is maximum. for normal distribution function mode and mean are the same quantities

Front 17

median (definition)

Back 17

the value of X such as xm where:
FX(xm)=P(X≤xm)
=∫(f(x),x,-∞,xm)
=1/2

Front 18

Standard Deviation (def and eqn)

Back 18

positive square root of variance of random variable

sdv[X]=σX=√(σX²)

Front 19

moments (around origin)

Back 19

mk
=E[X^k]
=∫(x^k*f(x),x,-∞,∞) (continuous)
=∑(xi^k*P(xi),all i) (discrete)

Front 20

central moments (definition)

Back 20

moment around mean values

Front 21

Central Moments (equation)

Back 21

λk
=E[(x-μX)^k]
=∫((x-μX)^k*f(x),x,-∞,∞) (continuous)
=∑((x-μX)^k*P(xi),all i) (discrete)

Front 22

m1

Back 22

μ=mean

first central moment is the mean

Front 23

m2

Back 23

E[X²]
=average power
=mean squared value

Front 24

λ2

Back 24

=σX²
=variance

Front 25

λ3

Back 25

=E[(x-μX)³]
which is a measure of the asymmetry of f(x) about mean.

Front 26

λ3/σX³

Back 26

normalized third central moment is called skewness or coefficient of skewness

Front 27

λ3 if p.d.f. is symmetric about x=μX

Back 27

λ3=0

Front 28

EX[c]

where c is a constant

Back 28

EX[c]=c

Front 29

E[X1+X2]

Back 29

E[X1+X2]
=E[X1]+E[X2]

Front 30

when does

E[X1*X2]
=E[X1]*E[X2}

Back 30

for two independent r.v. X1 and X2

Front 31

E[X1*X2] for two independent r.v. X1 and X2

Back 31

E[X1*X2]=E[X1]*E[X2]

Front 32

under what conditions does
E[(X1+X2)²]=E[X1²]+E[X2²]

Back 32

when they are both independent random variables whose mean values are zero

Front 33

variance (another equation)

Back 33

var[X]
=σX²
=E[(x-μX)²]
=E[X²]-μX²

Front 34

exmaple

consider two random variables X and Y with the following relation:

Y=aX+b

find: mean and variance of Y in terms of mean and variance of X

Back 34

mean:
μY=E[Y]=E[aX+b]=E[aX]+E[b]=a*μX+b

variance:
σY²=E[Y²]-μY²
with:
E[Y²]=E[(aX+b)²]=a²E[X²]+b²+2abE[X]
and μY² defined above
thus:
σY²
=( a²E[X²]+b²+2abE[X])-(a*μX+b)²
=a²*σX²

Front 35

consider bernouli r.v. X whihc takes two values of 0 or 1 with: P(x=1)=p and P(x=0)=q

find mean and variance

Back 35

mean:
μX=E[X]=∑(xi*P(xi),all i)=1*p+0*q=p

variance:
σX²=E[X²]-μX²

with second moment
m2=E[X²]
=∑(xi²*P(xi),all i)=1²*p+0²*q=p
thus
σX²=p-p²=p*(1-p)=p*q

Front 36

example:

for binomial random variable X calculate the mean and variance

[recall P(x=k)=(n,k)p^k*q^(n-k) with k=0,1,...,n]

Back 36

mean:
μX=E[X]
=∑(xi*P(xi),all i)
=∑(k*PX(k),k,0,n)
=∑(k*(n,k)p^k*q^(n-k),k,0,n)
=∑(k*{n!/[(n-k!)*k!]}p^k*q^(n-k),k,0,n)
=n*p*∑({(n-1)!/[(n-k!)*(k-1)!]}*p^(k-1)*q^(n-k),k,1,n)
=n*p*∑((n-1,k-1)p^(k-1)*q^(n-k),k,1,n)
=n*p*(p+q)^(n-1)=np

variance:
m2=E[X²]
=∑(k²*(n,k)p^k*q^(n-k),k,0,n)
=n*(n-1)*p²*+np

σX²
=m2-m1²
=[n*(n-1)*p²*+np]-[np]²
=np(1-p)
=npq

Front 37

mean and variance of binomial random variable X

Back 37

mean=np

variance=npq

Front 38

example:

a fair coin is flipped ten times

what is the mean and variance?

Back 38

mean=5

variance=2.5

Front 39

in which conditions would we use moments of normal

Back 39

for zero mean gaussian random variable X or for gaussian random variable X with mean μX

Front 40

moments of normal equation
for r.v. X=N(0,σ) (ie zero mean gaussian r.v. X)

Back 40

nth moment mn=
when n is even:
=(1)(3)…(n-1)*σX^n
when n is odd:
=0

Front 41

moments of normal equation for gaussian random variable X with mean μX and variance σX²

Back 41

λn=E[(x-μX)^n]
when n is even:
=(1)(3)…(n-1)*σX^n
when n is odd:
=0

Front 42

Chebychev's Inequality
upper band

Back 42

for any arbitrary density function f(x)
P(|x-μX|>k)≤σX²/k²

which gives the upper band on probability of x deviated from mean by k

Front 43

Chebychev's Inequality
lower band

Back 43

for any arbitrary density function f(x)
P(|x-μX|≤k)≥ 1-σX²/k²
ie 1- P(|x-μX|>k)≥1-σX²/k²

which gives the lower band on probability of x deviated from mean by k

Front 44

example

consider r.v. X with σX²=3

1)find upper band on the r.v. X deviated from its mean by 2

2)find lower band on probability of X to be in the range of 2 from its mean

Back 44

1) P(|x-μX|>2)≤σX²/2²
-->P(|x-μX|>2)≤ 3/4

2) P(|x-μX|≤2)≥ 1-σX²/k²
--> P(|x-μX|≤2)≥ 1/4

Front 45

characteristic function
(symbol and equation)

Back 45

ΦX(ω)
=E[exp(jωx)]
=∫(exp(jωx)*f(x),x,-∞,∞) (x is continuous)
=∑(exp(j*ω*xi)*PX(xi),all xi) (x is discrete)

Front 46

relation of ΦX(ω) to the probability density fu.

Back 46

ΦX(ω) is the "fourier transform" of the probability density function with sign change on ω as:

FX(ω)=F.T.(fX(x))=ΦX(-)

Front 47

inversion formula (uses

Back 47

finds pdf from characteristic eqn

Front 48

inversion formula (eqn)

Back 48

fX(x)=1/(2*π) * ∫( ΦX(ω)*exp(-jωx),ω,-∞,∞)

Front 49

Uses of characteristic eqn

Back 49

1) determination of higher order moments

2)determination of densities of sums of independent ranom

Front 50

ΦX(0)

Back 50

ΦX(0)=1

Front 51

|ΦX(ω)|

Back 51

|ΦX(ω)|≤1

Front 52

euler’s eqn

Back 52

|exp(jωx)|=|cos(ωx)+jsin(ωx)|=√(cos²ωx + sin²ωx)=1

Front 53

if X and Y are two independent r.v. and Z=X+Y then ΦZ(ω)=?

Back 53

ΦZ(ω)=ΦX(ω)*ΦY(ω)

Front 54

if Y=aX+b
ΦY(ω)=?

Back 54

ΦY(ω)=exp(jωb)ΦX(aω)

Front 55

under what conditions will
ΦZ(ω)=ΦX(ω)*ΦY(ω)

Back 55

Z=X+Y
with X and Y as independent random variables

Front 56

moments in terms of characteristic function:
mn=?

Back 56

mn=E[X^n]=(1/j^n)*nth_derivative(ΦX(ω),ω)|ω=0

Front 57

j^n*E[X^n]=?

Back 57

nth_derivative(ΦX(ω),ω)|ω=0

Front 58

example:

calculate the c.f. for poisson r.v. X using c.f.

find mean and variance of X

[recall poisson eqn: P(X=k)=exp(-b)*b^k/k! with k=0,1,...]

Back 58

characteristic function:
ΦX(ω)
=E[exp(jωx)]
=∑(exp(jωk)*exp(-b)*b^k/k
=exp[b*(exp(jω)-1)]

mean:
E[X]=(1/j)*∂(ΦX(ω),ω)|(ω=0)
=(1/j)∂(exp[b*(exp(jω)-1)])

variance:

E[X²]=2nd_derivative(ΦX(ω)|(ω=0))
=b+b²

thus σX²=b+b²-b²=b

Front 59

example:

find c.f. for standard normal r.v. [ie N(0,1)]

Back 59

ΦX(ω)
=E[exp(jωx)]
=∫(exp(jωx)*f(x),x,-∞,∞)
=(1/√(2*π))*∫(exp(jωx)*exp(-x²/2),x,-∞,∞)
=exp(-ω²/2)

Front 60

example:

find c.f. for normal r.v. with mean μ and variance σ² by using the following transformaton

Y=σX+μ

Back 60

c.f. of Y is:

ΦY(ω)
=exp(jωμ)*ΦX(σω)
=exp(jωμ)*exp((jω)²/2)
=exp(jωμ-ω²σ²/2)

Front 61

example:

find mean and variance of Y for standard normal r.v. [ie N(0,1)] using ΦY(ω)

Back 61

E[Y]
=(1/j)*∂(Φ(ω),ω)|(ω=0)
=(1/j)*(jμ-2σ²ω/2)*ΦY(ω)|(ω=0)
=μ

E[Y²]
=(1/j²)*2nd_derivative(Φ(ω),ω)|(ω=0)
=σ²-μ²

thus
σ²=E[Y²]-{E[Y]}²
=σ²-μ²+μ²
=σ²

Front 62

exercise:

find c.f. for geometric r.v.with the following probability mass function

calculate mean and variance using c.f.

geometric probability:
P(X=k)=p*q^(k-1)
k=1,2,...

Back 62

ΦX(ω)
=∑(exp(jωk)*p*q^(k-1),k,1,∞)
=p*exp(jω)/(1-q*exp(jω))

since |exp(jω)*q|<1:

E[X]
=(1/j)*∂(ΦX(ω),ω)|(ω=0)
=1/p

σX²=p/q²

Front 63

expected value of a function of random variables

Back 63

E[g(x1,…,xn)]
=∫…(∫(g(x1…xn)*f(x1,…,xn),x1,-∞,∞)…,xn,-∞,∞)

if g(x1,…,xn)= ∑(ai*xi,i,1,n)
then
_
g=E[g(x1…xn)]
=∑(ai*E[xi],i,1,n)

ie the mean value of a weighted sum of r.v.s equals the weighted sum of mean values

Front 64

joint moments of two r.v.

mnk=?

Back 64

mnk
=E[x^n*y^k]
=∫(∫((x^n)*(y^k)*f(x,y),x,-∞,∞),y,-∞,∞)

Front 65

order of joint moment

Back 65

n+k

Front 66

joint moments of two r.v.
m01

Back 66

_
m01=Y

Front 67

joint moments of two r.v.
m10

Back 67

_
m10=X

Front 68

joint moments of two r.v.
m11

Back 68

m11
=E[xy]
=Rxy
=∫(∫(xy*f(xy),x,-∞,∞),y,-∞,∞)

Front 69

correlation (notation)

Back 69

m11
=E[xy]
=Rxy

Front 70

uncorrelated (definition)

Back 70

Rxy=E[X]E[Y]

or if two random variables are independent

Front 71

orthogonal r.v. (definition)

Back 71

Rxy=0

Front 72

if two r.v. are independent they are also ____

Back 72

if two r.v. are independent they are also uncorrelated. (Vice-versa is not true in general.)

Front 73

joint central moments (notation)

Back 73

λnk
=E[(X-μX)^n*(Y-μY)^k]

Front 74

joint central moments (equation)

Back 74

λnk
=E[(X-μX)^n*(Y-μY)^k]
=∫(∫( [(x-μX)^n*(y-μY)^k]*f(x,y),x,-∞,∞), y,-∞,∞)

Front 75

λ20=?
λ02=?

Back 75

λ20=E[(x-μx)²]=σx²
λ02=E[(y-μy)²]=σy²

Front 76

covariance (definition)

Back 76

covariance of X and Y is defined by σxy or Cxy and defined as:

Cxy
=E[(x-μX)*(y-μY)]
=E[XY]-μX*μY

Front 77

if Cxy=0

Back 77

then X and Y are uncorrelated

Front 78

when X and Y are two independent r.v. the covariance is

Back 78

Cxy=0 thus the two variables are uncorrelated

Front 79

λ11

Back 79

λ11=Cxy

Front 80

correlation coefficient (definition)

Back 80

correlation coefficient ρxy is a measurement of linear dependency of two r.v. X and Y

Front 81

correlation coefficient
(eqn)

Back 81

ρxy=Cxy/(σx*σy)

Front 82

properties of correlation coefficient (2)

Back 82

1)|ρxy|≤1

2)if Y=aX+b with a and b as constants,
then ρxy=1 if a>0
ρxy=-1 if a<0

Front 83

relationship of correlation and independence

Back 83

if two r.v. are independent they are also uncorrelated. Howerver, the converse is not necessarily true.

Front 84

what does the correlation between r.v. X and Y measure?

Back 84

measures their tendency to lie in a straight line

Front 85

example:

consider y=x² when X is N(0,1). X and Y are dependent.
find covariance.

Back 85

covariance:

Cxy
=[XY]-μx*μy
=E[X³]-0*μy
=E[X³]
=0 (b/c odd moment)

thus X and Y are uncorrelated but certainly not independent

Front 86

conditional expectation

E[g(x)|B]=?

Back 86

E[g(x)|B]
=∫(g(x)*f(x|B),x,-∞,∞)
=∑(g(xi)*P(X=xi|B),all i)

Front 87

conditional expectation (notation)

Back 87

E[g(x)|B]
=EX|B[g(x)|B]

Front 88

conditional mean value

Back 88

E[x|B]
=∫(x*f(x|B),x,-∞,∞)
=∑(xi*P(X=xi|B),all i)