PROB_L2

More properties of the probability function

$P(A\cup B)=P(A)+P(B)-P(A\cap B)$.
Proof: $A\cup B=A\bigsqcup (B\setminus A)=A\bigsqcup (B\setminus (A\cap B))$.

Theorem 1(The inclusion-exclusion principle).

Let $A_1,A_2,\dots ,A_n\in \mathcal{F}$ for some $(\Omega ,\mathcal{F},P)$. Define

$$S_1=\sum _{i=1}^{n}P(A_i)$$ $$S_2=\sum _{1\le i<j\le n}P(A_i\cap A_j)$$ $$S_3=\sum _{1\le i<j<k\le n}P(A_i\cap A_j\cap A_k)$$ $$⋮$$

Then,

$$P\left(\bigcup _{i=1}^n{A}_i\right)=S_1-S_2+S_3-\cdots +(-1{)}^{n+1}{S}_n$$

Proof by induction (assignment).

Theorem 2(Sub-additivity).

$$P\left(\bigcup _{i=1}^n{A}_i\right)\le \sum _{i=1}^{n}P(A_i)=S_1.$$

Proof
Define $B_1=A_1$, $B_2=A_2\setminus B_1$, $B_3=A_3\setminus (B_1\bigsqcup B_2)$, $B_i=A_i\setminus {⨆}_{j=1}^{i-1}{B}_j$.
and note that ${\bigcup }_{i=1}^n{A}_i={⨆}_{i=1}^n{B}_i$. Now,

$$P\left(\bigcup _{i=1}^n{A}_i\right)=P\left(\underset{i=1}{\overset{n}{⨆}}{B}_i\right)=\sum _{i=1}^{n}P(B_i)\le \sum _{i=1}^{n}P(A_i).$$

Theorem 3(More sub-additivity).

$$P\left(\bigcup _{i=1}^n{A}_i\right)\ge S_1-S_2.$$

Proof
Consider the $B_i$s defined in the previous proof.

$$\begin{array}{rl}P(A_i)& =P(B_i\bigsqcup (A_i\setminus B_i))\\ & =P(B_i)+P(A_i\setminus B_i)\\ & \le P(B_i)+P\left(\bigcup _{j\ne i}(A_i\cap A_j)\right)\\ & \le P(B_i)+\sum _{j\ne i}P(A_i\cap A_j)\\ & \\ ⟹& P(B_i)\ge P(A_i)-\sum _{j\ne i}P(A_i\cap A_j)\end{array}$$

Summing both sides from $i=1$ to $n$ yields the property.

---

Conditional probability

We will consider events to be subsets of the sample space from now.

Definition 4(Conditional probability).

Let $A,B\subset \Omega$. The conditional probability of $A$ given $B$ is

$$P(A/B)=\frac{P(A\cap B)}{P(B)}.$$

Notes:

• $P(A\cap B)=P(A)P(B/A)=P(B)P(A/B)$.
• $P(C)=P(C/A)P(A)+P(C/A^c)P(A^c)$.
• If $A_1,A_2,\dots ,A_n$ are disjoint and cover $\Omega$, $P(C)={\sum }_{i=1}^{n}P(C/A_i)P(A_i)$.
  Also, $P(A_i/C)=\frac{P(C/A_i)P(A_i)}{{\sum }_{j=1}^{n}P(C/A_j)P(A_j)}$. ← Bayes’ Formula

Example: Polya’s Urn scheme

An urn contains $r$ red balls and $b$ blue balls. One ball is selected at random and $c$ balls of the same color are added into the urn. What is the probability of drawing a red ball in the second draw?

$$\begin{array}{rl}P(R_2)& =P(R_2/R_1)P(R_1)+P(R_2/B_1)P(B_1)\\ & =\left(\frac{r+c}{r+b+c}\right)\left(\frac{r}{r+b}\right)+\left(\frac{r}{r+b+c}\right)\left(\frac{b}{r+b}\right)\\ & =\frac{r}{r+b}\end{array}$$

You can prove that $P(R_n)=\frac{r}{r+b}$ using induction.