Likelihood Function

Let us consider a normally distributed random variable \(X.\)

Probability density function of a normal random vriable \(X\) is written as:

\(f(X=x)=\frac{1}{\sqrt{2 \pi} \sigma} \exp ^{-\frac{1}{2} \left( \frac{x-\mu}{\sigma }\right)^{2}}\)

where \(-\infty < \mu < \infty\) and \(0< \sigma < \infty.\)

Example : The textbook you need to buy for your Statiatics class is expensive at the college bookstore, so you consider buying it on Ebay instead. A look at the past auctions suggest that the prices of that statistics textbook have an approximately normal distribution with standard deviation $15.

We wish to acertain which value of \(\mu\) is most consistent with sample data.

Consider \(\mu= \$89.\) Let us consider three random samples from the sales history : $115, $120, and $127.

If \(\mu\) were $89 the sample observations would be in the right tail. These are unlikely occurances.

• Consider \(\mu= \$118\), now the observations would be at the center of the distribution. Our current choice of \(\mu\) is more likely than the previous one.

par(mfrow=c(1,2)) # divide ploting window into a matrix of 1 by 2
## left figure
sigma= 15
mu= 89
x<- seq(40, 140,length=200)
f<- 1/(sqrt(2*pi)*sigma)*exp(-(x-mu)^2/(2*sigma^2))
plot(x,f,type="l",lwd=2,col="red", xlab="price(x)", ylab="probability")
abline(v=115, lty=3, col="blue")
abline(v=120, lty=3, col="blue")
abline(v=127, lty=3, col="blue")
######## right figure
sigma= 15
mu= 118
x<- seq(70, 180,length=200)
f<- 1/(sqrt(2*pi)*sigma)*exp(-(x-mu)^2/(2*sigma^2))
plot(x,f,type="l",lwd=2,col="red", xlab="price(x)", ylab="probability")
abline(v=115, lty=3, col="blue")
abline(v=120, lty=3, col="blue")
abline(v=127, lty=3, col="blue")

Calculation of likelihood values

when \(\mu=89\)

sigma= 15
mu= 89
f1<- 1/(sqrt(2*pi)*sigma)*exp(-(115-mu)^2/(2*sigma^2))
f2<- 1/(sqrt(2*pi)*sigma)*exp(-(120-mu)^2/(2*sigma^2))
f3<- 1/(sqrt(2*pi)*sigma)*exp(-(127-mu)^2/(2*sigma^2))
 c(f1, f2, f3) # list all the probabilities 

## [1] 0.005921231 0.003143104 0.001074524

 #P(X= 115, 120, 127)= P(X=115)* P(X=120)*P(X=115)= f1*f2*f3
L=f1*f2*f3  # L(mu)
L

## [1] 1.999801e-08

when \(\mu=118\)

sigma= 15
mu= 118
f1<- 1/(sqrt(2*pi)*sigma)*exp(-(115-mu)^2/(2*sigma^2))
f2<- 1/(sqrt(2*pi)*sigma)*exp(-(120-mu)^2/(2*sigma^2))
f3<- 1/(sqrt(2*pi)*sigma)*exp(-(127-mu)^2/(2*sigma^2))
 c(f1, f2, f3) # list all the probabilities 

## [1] 0.02606951 0.02636079 0.02221497

 #P(X= 115, 120, 127)= P(X=115)* P(X=120)*P(X=115)= f1*f2*f3
L=f1*f2*f3  # L(mu)
L

## [1] 1.526642e-05

Likelihood Function

like<- rep(0,280)
for(j in 70: 180){
  like[j]= (1/(sqrt(2*pi)*sigma))^3*exp(-(115-j)^2/(2*sigma^2))* exp(-(120-j)^2/(2*sigma^2))*exp(-(127-j)^2/(2*sigma^2))
  
}

plot(like, type="l",lwd=2,col="red", xlab=expression(mu),ylab=expression(L(mu)), xlim = c(70, 180), ylim=c(0, 0.00002))
abline(v=115, lty=3, col="blue")
abline(v=120, lty=3, col="blue")
abline(v=127, lty=3, col="blue")

Summary

Mathematically, for \(\sigma= 15\), the likelihood function is:

\(L(\mu)= \left[\frac{1}{\sqrt{2\pi} *15 } \right]^3 exp{\left[ -\frac{1}{2} \left( \frac{115-\mu}{15}\right)^2\right]}exp{\left[ -\frac{1}{2} \left( \frac{120-\mu}{15}\right)^2\right]}exp{\left[ -\frac{1}{2} \left( \frac{127-\mu}{15}\right)^2\right]}\)