PROB_L5

Discrete Random Vector

Definition 1(Definition).

Let $(\Omega ,\mathcal{F},P)$ be a probability space. $\mathbf{X}:\Omega \to {\mathbb{R}}^n$ is called a discrete random vector if $|\mathrm{Im}\mathbf{X}|\le {\aleph }_0$ and $\{\omega \in \Omega |\mathbf{X}(\omega )=\mathbf{x}\}\in \mathcal{F}$ for all $\mathbf{x}\in {\mathbb{R}}^n$.

If $f:{\mathbb{R}}^n\to {\mathbb{R}}^m$ is any function and $\mathbf{X}:\Omega \to {\mathbb{R}}^n$ is a random vector, $f\circ \mathbf{X}$ is also a random vector. In particular, if $m=1$, $f\circ \mathbf{X}$ is a random variable.

Theorem 2(Theorem).

Let $\mathbf{X}:\Omega \to {\mathbb{R}}^n$, $\omega \mapsto (X_1(\omega ),X_2(\omega ),\dots ,X_n(\omega ))$ where $X_i:\Omega \to \mathbb{R}$. $\mathbf{X}$ is a random vector iff each $X_i$ is a random variable.

Proof of $⟹$
Let $f_i:{\mathbb{R}}^n\to \mathbb{R}$ be the projection map $(x_1,x_2,\cdots ,x_n)\mapsto x_i$. Then, $x_i=f_i\circ \mathbf{X}$ is a random variable.

Proof of $⟸$

$$(X=(x_1,x_2,\cdots ,x_n))=\bigcap _{i=1}^n(X_i=x_i),$$

which is in the $\sigma$ algebra since it is closed under countable intersections.

Definition 3(Definition).

Let $\mathbf{X}=(X_1,X_2,\dots ,X_n):\Omega \to {\mathbb{R}}^n$ be a random vector. Then, $f_{\mathbf{X}}:{\mathbb{R}}^n\to [0,1]$ defined by $f_{\mathbf{X}}(\mathbf{x})=P(\mathbf{X}=\mathbf{x})$ is called a joint probability mass function of the random variables $X_1,X_2,\dots ,X_n$, or just the probability mass function of $\mathbf{X}$. The density function $f_{X_i}$ of $X_i$ is called the $i$th marginal density of $\mathbf{X}$ or $f_{\mathbf{X}}$.

As in the one dimensional case, $f_{\mathbf{X}}$ has the following properties:

• $f_{\mathbf{X}}(\mathbf{x})\ge 0$ for all $\mathbf{x}\in {\mathbb{R}}^n$, and $f_{\mathbf{X}}(\mathbf{x})>0$ for at most countably many $\mathbf{x}\in {\mathbb{R}}^n$, which may be denoted by $\{{\mathbf{x}}_1,{\mathbf{x}}_2,\dots \}$.
• ${\sum }_{i}f({\mathbf{x}}_i)=1$.

Also, any real valued function $f$ on ${\mathbb{R}}^n$ having these properties is the density function for some $n$ dimensional random vector, and is called a discrete $n$ dimensional density function.

The marginal densities of $\mathbf{X}$ can be obtained from the density of $\mathbf{X}$:

Theorem 4(Theorem).

Let $f_{\mathbf{X}}$ be the density function of a random vector $\mathbf{X}$ defined on $(\Omega ,\mathcal{F},P)$. If $\mathbf{X}=(X_1,X_2,\dots ,X_n)$, we know that each $X_i$ is a random variable. The probability mass functions $f_{X_i}$ can be expressed in terms of $f_{\mathbf{X}}$ like so:

$$f_{X_i}(c)=P\left(\bigcup _{\mathbf{x}\in \mathrm{Im}\mathbf{X}|x_i=c}(\mathbf{X}=\mathbf{x})\right)=\sum _{\mathbf{x}\in \mathrm{Im}\mathbf{X}|x_i=c}P(\mathbf{X}=\mathbf{x}).$$

---

Independent Random Variables

The concept of independent random variables is very similar to independent events. Recall that two events $A$ and $B$ are independent if $P(A\cap B)=P(A)P(B)$. We say discrete random variables $X$ and $Y$ are independent if $P(\{X=x\}\cap \{Y=y\})=P(X=x)P(Y=y)$ for all $x,y$. In other words, the joint density of $X$ and $Y$ should be given by $f(x,y)=f_X(x)f_Y(y)$.

Definition 5(Definition).

Let $X_1,X_2,\dots ,X_r$ be $r$ discrete random variables having densities $f_1,f_2,\dots ,f_r$ respectively. These random variables are said to be mutually independent if their joint density function $f$ is given by

$$f(x_1,x_2,\dots ,x_r)=f_1(x_1)f_2(x_2)\dots f_r(x_r).$$

If $A_1,A_2,\dots ,A_r$ are any $r$ subsets of $R$, then

$$P(X_1\in A_1,\dots ,X_r\in A_r)=P(X_1\in A_1)\dots P(X_r\in A_r).$$

To see this, note that

$$\begin{array}{rl}P(X_1\in A_1,\dots ,X_r\in A_r)& =\sum _{x_1\in A_1}\cdots \sum _{x_r\in A_r}P(X_1=x_1,\dots ,X_r=x_r)\\ & =\sum _{x_1\in A_1}\cdots \sum _{x_r\in A_r}P(X_1=x_1)\dots P(X_r=x_r)\\ & =\prod _{i=1}^r\left(\sum _{x\in A_i}P(X_i=x)\right)\\ & =\prod _{i=1}^{r}P(X_i\in A_i).\end{array}$$

Theorem 6(Proposition).

Let $X$ and $Y$ be independent random variables, and let $f,g:\mathbb{R}\to \mathbb{R}$. Then, $f\circ X$ and $g\circ Y$ are independent random variables.

Theorem 7(Proposition).

Let $f_1$ and $f_2$ be two probability mass functions. Then, there exists $(\Omega ,\mathcal{F},P)$ and $X,Y:\Omega \to \mathbb{R}$ such that $f_X=f_1$, $f_Y=f_2$, and $X$ and $Y$ are independent.

Example 14 here is pertinent.

The multinomial distribution

We have previously seen that if $X_1,X_2,\dots ,X_n$ have a common Bernoulli density with parameter $p$ ($f(1)=p$, $f(0)=1-p$), then $S_n=X_1+X_2+\dots X_n$ is binomially distributed with parameters $n$ and $p$. Trials that result in either success or failure are called Bernoulli trials, and the above situation is described by saying we perform $n$ Bernoulli trials with common probability $p$ for success.

More generally, consider an experiment, such as rolling a die, that can result in a finite number $r$ of distinct possible outcomes. Let $Y$ be a random variable which assumes the values $1,\dots ,r$ so that $\{Y=i\}$ represents the fact that the experiment yielded the $i$th outcome. Let $p_i=P(Y=i)$. An n-fold independent repetition of the experiment can be represented by the random vector $(Y_1,Y_2,\dots ,Y_n)$, where $Y_1,Y_2,\dots ,Y_n$ have the same distribution as $Y$ and are independent. Now, let $X_i$, $i=1,2,\dots ,r$, denote the number of trials that yield the $i$th outcome. The joint density of $X_1,X_2,\dots ,X_r$ follows the multinomial distribution with parameters $n$ and $p_1,p_2,\dots ,p_r$:

$$\begin{array}{r}f(x_1,x_2,\dots ,x_r)=\{\begin{array}{ll}\frac{n!}{(x_1!)\dots (x_r!)}{p}_1^{x_1}\dots p_r^{x_r}& x_i\in {\mathbb{Z}}_{\ge 0},\sum x_i=n\\ 0& \text{otherwise.}\end{array}\end{array}$$

Note that the random variables $X_1,\dots ,X_r$ are not independent. In fact, knowing any $r-1$ of them determines the $r$th.

Poisson Approximation to the binomial distribution

Given ${\lim }_{n\to \infty }n{p}_n\to \lambda$,

$$\underset{n\to \infty }{\lim }\left(\genfrac{}{}{0ex}{}{n}{k}\right)(p_n{)}^k(1-p_n{)}^{n-k}=\frac{{\lambda }^k}{k!}{e}^{-\lambda }.$$

So, the limit of the binomial distribution with parameters $n$ and $p_n$ as $n\to \infty$ is the poisson distribution with parameter $\lambda$.