PROB_AS2

Vasudeva S N

Problem 1

Question

if $X$ is a continuous random variable with distribution function $F$ and density function $f$, show that the random variable $Y=|X|$ is also continuous and express (with proof) its cumulative distribution function and density in terms of $F$ and $f$. Find the density of $Y$ when $X$ has (i) normal distribution (ii) exponential distribution and (iii) Cauchy distribution.

For $Y$ to be a random variable, $\{\omega \in \Omega |Y(\omega )\le r\}\in \mathcal{A}$ for all $r$.

$$\begin{array}{rl}\{Y\le r\}& =\{|X|\le r\}\\ & =\{X\le r\}\cap \{X\ge -r\},\end{array}$$

both of which are in $\mathcal{A}$ since $X$ is a random variable. Hence, $Y$ is a random variable.

FTSOC, assume $Y$ is not continuous, that is, there exists $r\in {\mathbb{R}}^{+}$ such that $P(Y=r)\ne 0$. This implies $P(X=r,X=-r)=P(X=r)+P(X=-r)\ne 0$. However, this is not possible since $X$ is a continuous random variable.

For $y\ge 0$,

$$\begin{array}{rl}{F}_Y(y)& =P(|X|\le y)\\ & =P(-r\le X\le y)\\ & =F_X(y)-F_X(-y)\end{array}$$

We can prove that the function defined by

$$f_Y(y)=\{\begin{array}{ll}{f}_X(y)+f_X(-y)& y\ge 0\\ 0& \text{otherwise}\end{array}$$

is the density of $|X|$ by integration: for $y\ge 0$,

$$\begin{array}{rl}{\int }_{-\infty }^y{f}_Y(u)du& ={\int }_0^y{f}_X(u)+f_X(-u)du\\ & =F_X(y)-F_X(0)-{\int }_0^{-y}{f}_X(u)du\\ & =F_X(y)-F_X(0)-[F_X(-y)-F_X(0)]\\ & =F_X(y)-F_X(-y)\\ & =F_Y(y).\end{array}$$

If $X\sim n(0,1)$,

$$f_{|X|}(x)=\{\begin{array}{ll}\frac{2}{\sqrt{2\pi }}{e}^{-x^2/2}& x\ge 0\\ & \\ 0& \text{otherwise}\end{array}$$

If $X\sim \exp (\lambda )$, its density remains unchanged, since it is a positive random variable.

If $X\sim \text{Cauchy}$,

$$f_{|X|}(x)=\{\begin{array}{ll}\frac{2}{\pi (1+x^2)}& x\ge 0\\ & \\ 0& \text{otherwise}\end{array}$$

Problem 2

Question

Show that the integral

$${\int }_{-\infty }^{\infty }|x-\mu |f(x)dx$$

becomes minimum when $\mu$ is the median of the distribution with density $f$. (For continuous distributions, the median is the point $x_0\in \mathbb{R}$ such that $F(x_0)=1/2$.)

$$G(\mu )\equiv {\int }_{-\infty }^{\infty }|x-\mu |f(x)dx={\int }_{-\infty }^{\mu }(\mu -x)f(x)dx+{\int }_{\mu }^{\infty }(x-\mu )f(x)dx.$$

Assume $f$ is continuous. Then, both integrands on the right are continuous, and the partial derivative with respect to $\mu$ of both exist on $\mathbb{R}$. Thus, we have

$$\frac{d}{d\mu }{\int }_{-\infty }^{\mu }(\mu -x)f(x)dx={\int }_{-\infty }^{\mu }f(x)dx+(\mu -\mu )f(\mu ),$$ $$\frac{d}{d\mu }{\int }_{\mu }^{\infty }(x-\mu )f(x)dx=-{\int }_{\mu }^{\infty }f(x)dx-(\mu -\mu )f(\mu ).$$

So

$$G^{\prime }(\mu )=F(\mu )-(1-F(\mu ))=2F(\mu )-1$$

Setting the derivative equal to zero, we get

$$F(\mu )=\frac{1}{2}.$$

Thus, the median point is a stationary point, denoted by $x_0$. Next, notice that $G^{\prime }$ inherits the properties of continuity and monotonicity form $F$. Thus, from the mean value theorem, for $x<x_0$, $G(x_0)-G(x)=(x_0-x)G^{\prime }(h)<0$ for some $h<x_0$, and for $x>x_0$, $G(x)-G(x_0)=(x-x_0)G^{\prime }(h)>0$ for some $h>x_0$. Thus, $G$ attains its minimum value at $x_0$.

Problem 3

Question

Show that

$$F(x,y)=\{\begin{array}{ll}0& x+y<1\\ 1& x+y\ge 1\end{array}$$

is not a joint distribution function.

Consider the region $[0,1]\times [0,1]$. The probability of attaining this region is given by

$$F(1,1)+F(0,0)-F(1,0)-F(0,1)=1+0-1-1=-1.$$

Thus, $F$ cannot be a probability distribution function.

Problem 4

Question

If $\log X$ is normally distributed, find the density of $X$.

Let $Y=\log X$. It is given that

$$f_Y=\frac{1}{\sqrt{2\pi }}{e}^{-x^2/2}.$$

We with to find the distribution of $e^Y$. Since $e^x$ is a differentiable and strictly monotonic function on $\mathbb{R}$, we have $f_X(x)=f_{e^Y}(x)=0$ for $x\le 0$ and

$$\begin{array}{rl}{f}_X(x)& =f_Y(\log x)\left|\frac{d}{dx}\log x\right|\\ & =\frac{1}{x\sqrt{2\pi }}{e}^{-(\log x{)}^2/2}\end{array}$$

for $x>0$.

Problem 5

Question

Suppose $X$ has a symmetric distribution about $a$. Show that $E(X)=a$, provided it exists.

$$\begin{array}{rl}E(X)& ={\int }_{-\infty }^{\infty }xf(x)dx\\ & ={\int }_{-\infty }^{\infty }(u+a)f(u+a)du\\ & ={\int }_{-\infty }^{\infty }\underset{\text{odd}}{\underbrace{uf(u+a)}}du+{\int }_{-\infty }^{\infty }af(u+a)du\\ & =a\end{array}$$

Problem 6

Question

If $E(X)=E(X^2)=0$, show that $P(X=0)=1$.

$\text{Var}(X)=E(X^2)-E(X{)}^2=0$. From Chebyshev’s inequality, it follows that

$$\begin{array}{r}P(|X|\ge t)=0\text{ for all }t>0.\end{array}$$

Now,

$$\begin{array}{rl}& P(|X|\ge t)=0\\ ⟹& P(|X|>t)=0\\ ⟹& 1-F_{|X|}(t)=0\\ ⟹& F_{|X|}(t)=1\text{ for all }t>0.\end{array}$$

From right continuity of the distribution function, it follows that $F(0)=1$. So,

$$P(X=0)=P(|X|=0)=\underset{x\to 0^{+}}{\lim }F(x)-\underset{x\to 0^{-}}{\lim }F(x)=1.$$

Problem 7

Question

Let $X$ and $Y$ have the joint density

$$f(x,y)=c{x}^{n_1-1}(y-x{)}^{n_2-1}{e}^{-y}\text{ for }0<x<y<\infty .$$

Find $c$ and the marginal distributions of $X$ and $Y$.

$$\begin{array}{rl}& {\int }_0^{\infty }{\int }_x^{\infty }f(x,y)dydx\\ & ={\int }_0^{\infty }{e}^{-y}{\int }_0^y{x}^{n_1-1}(y-x{)}^{n_2-1}dxdy.\end{array}$$

For the inner integral, use the substitution $t=x/y$, which yields $x=ty$ and $dx=ydt$.

$$\begin{array}{rl}{\int }_0^y{x}^{n_1-1}(y-x{)}^{n_2-1}dx& ={\int }_0^1(ty{)}^{n_1-1}(y-ty{)}^{n_2-1}ydt\\ & =y^{n_1+n_2-1}{\int }_0^1{t}^{n_1-1}(1-t{)}^{n_2-1}dt.\end{array}$$

The integral is the $\beta$ function $B(n_1,n_2)$, which is equal to $\frac{\Gamma (n_1)\Gamma (n_2)}{\Gamma (n_1+n_2)}$. Thus, the inner integral becomes

$$y^{n_1+n_2-1}\frac{\Gamma (n_1)\Gamma (n_2)}{\Gamma (n_1+n_2)}.$$

Substituting this back, we get

$$\frac{\Gamma (n_1)\Gamma (n_2)}{\Gamma (n_1+n_2)}{\int }_0^{\infty }{y}^{n_1+n_2-1}{e}^{-y}dy=\frac{\Gamma (n_1)\Gamma (n_2)}{\Gamma (n_1+n_2)}\Gamma (n_1+n_2)=\Gamma (n_1)\Gamma (n_2).$$

Thus, $c=\frac{1}{\Gamma (n_1)\Gamma (n_2)}$.

$$\begin{array}{rl}{F}_X(u)=\underset{v\to \infty }{\lim }{F}_{X,Y}(u,v)& =c{\int }_0^u{x}^{n_1-1}{\int }_x^{\infty }(y-x{)}^{n_2-1}{e}^{-y}dydx\\ & =c{\int }_0^u{x}^{n_1-1}{\int }_0^{\infty }{z}^{n_2-1}{e}^{-x-z}dzdx\\ & =c\Gamma (n_2){\int }_0^u{x}^{n_1-1}{e}^{-x}dx\\ & =\frac{1}{\Gamma (n_1)}{\int }_0^u{x}^{n_1-1}{e}^{-x}dx.\end{array}$$

Thus, $X\sim \Gamma (n_1,1)$.

$$\begin{array}{rl}{F}_Y(v)=\underset{u\to \infty }{\lim }{F}_{X,Y}(u,v)& =c{\int }_0^v{\int }_0^y{x}^{n_1-1}(y-x{)}^{n_2-1}{e}^{-y}dxdy\\ & =c{\int }_0^v{e}^{-y}{\int }_0^y{x}^{n_1-1}(y-x{)}^{n_2-1}dxdy\\ & =\frac{1}{\Gamma (n_1+n_2)}{\int }_0^v{y}^{n_1+n_2-1}{e}^{-y}dy.\end{array}$$

Thus, $Y\sim \Gamma (n_1+n_2)$.

Problem 8

Question

Calculate the characteristic function of a Gamma distribution with parameters $\alpha$ and $\lambda$ and deduce the characteristic function of ${\chi }^2$.

Let $X\sim \Gamma (\alpha ,\lambda )$.

$$\begin{array}{rl}{\phi }_X(t)=E(e^{itX})& =\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}{\int }_0^{\infty }{x}^{\alpha -1}{e}^{-(\lambda -it)x}dx.\\ & =\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}\frac{\Gamma (\alpha )}{(\lambda -it{)}^{\alpha }}\\ & =\frac{{\lambda }^{\alpha }}{(\lambda -it{)}^{\alpha }}.\end{array}$$

Let $Z_1\sim n(0,1)$. If $d$ is the density of $Z_1^2$, then we can compute $d$ to be

$$d(x)=\frac{1}{\sqrt{2\pi x}}{e}^{-x/2},x>0$$

which is $\Gamma (1/2,1/2)$. If $Z_2,\dots ,Z_n\sim n(0,1)$, then

$$\begin{array}{rl}{\phi }_{{\chi }_n^2}={\phi }_{Z_1^2+\cdots +Z_n^2}(t)& ={\phi }_{Z_1^2}(t)\dots {\phi }_{Z_n^2}(t)\\ & =\frac{{\left(\frac{1}{2}\right)}^{n/2}}{{\left(\frac{1}{2}-it\right)}^{n/2}}\end{array}$$

The above the characteristic function of $\Gamma \left(\frac{n}{2},\frac{1}{2}\right)$. Hence, ${\chi }_n^2\sim \Gamma \left(\frac{n}{2},\frac{1}{2}\right)$.

Problem 9

Question

Let $X_1$ and $X_2$ be independent exponential variables with parameter $\lambda$. Find the joint density function of $(Y_1,Y_2)$, where $Y_1=X_1+X_2$ and $Y_2=X_1/X_2$, and show that they are independent.

$$\begin{array}{rl}{F}_{(Y_1,Y_2)}(y_1,y_2)& =P(Y_1\le y_1,Y_2\le y_2)\\ & =P(X_1+X_2\le y_1,X_1/X_2\le y_2)\\ & ={\iint }_S{f}_{X_1}(x_1)f_{X_2}(x_2)d{x}_{1}d{x}_2,\end{array}$$

where $S=\{(x_1,x_2)|x_1+x_2\le y_1\}\cap \{(x_1,x_2)|x_1/x_2\le y_2\}$. The integral can now be expressed as the sum of the following two integrals:

$$\begin{array}{rl}& {\int }_0^{y_1/(1+y_2)}{\int }_0^{y_2{x}_2}{f}_{X_1}(x_1)f_{X_2}(x_2)d{x}_{1}d{x}_2\\ & ={\int }_0^{y_1/(1+y_2)}{f}_{X_2}(x_2){\int }_0^{y_2{x}_2}{f}_{X_1}(x_1)d{x}_{1}d{x}_2\\ & =\lambda {\int }_0^{y_1/(1+y_2)}(e^{-\lambda x_2})(1-e^{-\lambda y_2{x}_2})d{x}_2\\ & ={\left[\frac{e^{-\lambda x_2(1+y_2)}}{1+y_2}-e^{-\lambda x_2}\right]}_0^{y_1/(1+y_2)}\\ & =\frac{e^{-\lambda y_1}}{1+y_2}-e^{-\lambda y_1/(1+y_2)}-\frac{1}{1+y_2}+1,\end{array}$$ $$\begin{array}{rl}& {\int }_{y_1/(1+y_2)}^{y_1}{\int }_0^{y_1-x_2}{f}_{X_1}(x_1)f_{X_2}(x_2)d{x}_{1}d{x}_2\\ & ={\int }_{y_1/(1+y_2)}^{y_1}{f}_{X_2}(x_2){\int }_0^{y_1-x_2}{f}_{X_1}(x_1)d{x}_{1}d{x}_2\\ & =\lambda {\int }_{y_1/(1+y_2)}^{y_1}(e^{-\lambda x_2})(1-e^{-\lambda (y_1-x_2)})d{x}_2\\ & =\lambda {\int }_{y_1/(1+y_2)}^{y_1}{e}^{-\lambda x_2}-e^{-\lambda y_1}d{x}_2\\ & =\lambda {\left[-\frac{e^{-\lambda x_2}}{\lambda }-x_2{e}^{-\lambda y_1}\right]}_{y_1/(1+y_2)}^{y_1}\\ & =e^{-\lambda y_1/(1+y_2)}+\frac{\lambda y_1}{1+y_2}{e}^{-\lambda y_1}-e^{-\lambda y_1}-\lambda y_1{e}^{-\lambda y_1}.\end{array}$$

Thus,

$$F_{(Y_1,Y_2)}(y_1,y_2)=\frac{e^{-\lambda y_1}}{1+y_2}-e^{-\lambda y_1/(1+y_2)}-\frac{1}{1+y_2}+1+e^{-\lambda y_1/(1+y_2)}+\frac{\lambda y_1}{1+y_2}{e}^{-\lambda y_1}-e^{-\lambda y_1}-\lambda y_1{e}^{-\lambda y_1}$$

Compute the joint density:

$$f_{(Y_1,Y_2)}(y_1,y_2)=\frac{{\partial }^2}{\partial y_1\partial y_2}{F}_{(Y_1,Y_2)}(y_1,y_2)=\frac{{\lambda }^2{y}_1{\mathrm{e}}^{-\lambda y_1}}{{\left(y_2+1\right)}^2}.$$

Now,

$$\begin{array}{rl}{f}_{Y_1}(y_1)& ={\int }_0^{y_1}{f}_{X_1}(x)f_{X_2}(y_1-x)dx\\ & ={\lambda }^2{\int }_0^{y_1}{e}^{-\lambda y_1}dx\\ & ={\lambda }^2{e}^{-\lambda y_1}{y}_1\end{array}$$

and

$$\begin{array}{rl}{f}_{Y_2}(y_2)& ={\int }_0^{\infty }x{f}_{X_1}(x)f_{X_2}(x{y}_2)dx\\ & ={\lambda }^2{\int }_0^{\infty }x{e}^{-\lambda x(1+y_2)}dx\\ & ={\lambda }^2\left(\frac{1}{(\lambda +\lambda y_2{)}^2}\right)\\ & =\frac{1}{(1+y_2{)}^2}.\end{array}$$

Since

$$\begin{array}{r}{f}_{(Y_1,Y_2)}=f_{Y_1}{f}_{Y_2},\end{array}$$

it follows that $Y_1$ and $Y_2$ are independent random variables.