Homework 3

Due Friday February 7 at 5:00pm

Exercise 1

Let \(Y_1, \ldots Y_n | \theta\) be an i.i.d. random sample from a population with pdf \(p(y|\theta)\) where

\[ p(y|\theta) = \frac{2}{\Gamma(a)} \theta^{2a} y^{2a -1} e^{-\theta^2 y^2} \]

and \(y > 0\), \(\theta > 0\), \(a > 0\).

For this density,

\[ \begin{aligned} E~Y|\theta &= \frac{\Gamma(a + \frac{1}{2})}{\theta \Gamma(a)}\\ E~Y^2|\theta &= \frac{a}{\theta^2} \end{aligned} \]

Call this density \(g^2\) such that \(Y_1, \ldots Y_n | \theta \sim g^2(a, \theta)\).

  1. Find the joint pdf of \(Y_1, \ldots Y_n | \theta\) and simplify as much as possible.

  2. Suppose \(a\) is known but \(\theta\) is unknown. Identify a simple conjugate class of priors for \(\theta\). For any arbitrary member of the class, identify the posterior density \(p(\theta | y_1, \ldots y_n)\).

  3. Obtain a formula for \(E~ \theta | Y_1, \ldots Y_n\) and \(Var~\theta | Y_1, \ldots Y_n\) when the prior is in the conjugate class.

Exercise 2

Physicists studying a radioactive substance measure the times at which the substance emits a particle. They will record \(n+1\) emissions and set \(Y_1\) to be the time elapsed between the first and second emission, \(Y_2\) to be the time elapsed between the second and third emission and so on. They will model the data as \(Y_1, \ldots Y_n | \theta \sim \text{i.i.d. } \text{exponential}(\theta)\). The pdf of the exponential(\(\theta\)) distribution is

\[ p(y |\theta) = \theta e^{-\theta y} \ \text{ for } \ y>0, \ \theta>0. \]

For this distribution, \(E[Y|\theta] = \frac{1}{\theta}\).

(a). Write out the corresponding joint density \(p(y_1, \ldots, y_n | \theta)\) and simplify as much as possible. Justify each step of your calculation.

(b). Compute the maximum likelihood estimate \(\hat{\theta}_{MLE}\), i.e. the value \(\hat{\theta}_{MLE}\) that maximizes \(p(y_1,\ldots y_n | \theta)\). Hint: it’s easier to work with the log-likelihood.

(c). Choose a prior \(p(\theta)\) that is conjugate to the likelihood. Hint: look at kernels of densities on the distribution sheet. Write out the formula for \(p(\theta | y_1, \ldots y_n)\), up to a proportionality in \(\theta\), and simplify as much as possible. From this, identify explicitly the posterior distribution of \(\theta\) (i.e., write “the posterior is a blank distribution with parameter(s) blank)”.

(d). Obtain the formula for \(E[\theta | y_1, \ldots y_n]\) as a function of the prior parameters, and \(n\), and \(y_1, \ldots y_n\), and try to write this as a function of the estimator \(\hat{\theta}\) you found in part (b). What does \(E[\theta | y_1,\ldots,y_n]\) get close to as \(n\) increases?

(e). Assume you observe \((y_1, \ldots, y_5) = (0.13, 0.31, 0.15, 0.12, 0.29)\) and let \(\theta \sim \text{gamma}(1, 1)\). Report a 95% posterior confidence interval for \(\theta\).

Exercise 3

Suppose \(Y|\theta \sim \text{binary}(\theta)\) and we believe \(\theta \sim \text{Uniform}(0, 1)\) describes our uninformed prior beliefs about \(\theta\). However, we are really interested in the log-odds \(\gamma = f(\theta) = \log \frac{\theta}{1 - \theta}\).

  1. Find the prior distribution for \(\gamma\) induced by our prior on \(\theta\). Is the prior informative about \(\gamma\)? Verify \(p(\gamma)\) using Monte Carlo sampling (i.e. sampling from \(p(\theta)\)) and then plotting the empirical density of the transformed samples along with the closed-form solution.

  2. In general, is the mean of the transform the same as the transform of the mean? In other words, is \(E f(\theta) = f(E[\theta])\)? Why or why not?

  3. Assume some data come in and \(\sum y_i = 7\) out of \(n = 10\) trials. Report the posterior mean and 95% posterior confidence interval for \(\gamma\). Is the transform of the quantile the quantile of the transform? Why or why not?