PROB_L1

Probability spaces

The sample space ($\Omega$) is the set of all possible outcomes of an experiment. An event is a subset of $\Omega$.

A probability space consists of three elements:

1. The sample space, $\Omega$.
2. An event space, which is a set of events $\mathcal{F}$ we would like to consider.
3. A probability function $P$ which assigns to each event in $\mathcal{F}$ a probability in $[0,1]$.

The event space and probability function must satisfy some requirements, as we will see.

Finite probability space

When $|\Omega |\le {\aleph }_0$, we can take $\mathcal{F}$ to be the power set of $\Omega$. This will not be possible for continuous probability spaces; more on that in a bit.

Let $\Omega$ be a finite set.
Let $P(\Omega )$ denote the power set of $\Omega$.
Let $\mathbf{P}:P(\Omega )\to [0,1]$ be a function satisfying $P(\varnothing )=0$, $P(\Omega )=1$, and $P(A_1\bigsqcup A_2)=P(A_1)+P(A_2)$ for disjoint subsets $A_1$ and $A_2$ of $\Omega$ (This property can be validated by our intuitive understanding of probability, and is called the additivity of the probability function).

$(\Omega ,P(\Omega ),\mathbf{P})$ is called a finite probability space. Note that in finite probability spaces, it is possible to assign a probability to the elementary events (the elements of $\Omega$, or singleton events). $\mathbf{P}$ could have been alternatively defined from $\Omega$ to $[0,1]$ such that ${\sum }_{\omega \in \Omega }\mathbf{P}(\omega )=1$, and the probability of any event $E$ in $P(\Omega )$ calculated like so: ${\sum }_{\omega \in E}\mathbf{P}(\omega )$.

Using induction, we can show that $\mathbf{P}\left({⨆}_{i=1}^n{A}_i\right)={\sum }_{i=1}^n\mathbf{P}(A_i)$ if $A_i\cap A_j=\varnothing$ if $i\ne j$.

Discrete probability space

Let $|\Omega |={\aleph }_0$.
$\mathbf{P}$ is defined much the same way as in a finite probability space, with the additional restriction that it is countably additive (aka $\sigma$-additive): if $\{A_n{\}}_{n=1}^{\infty }$ is a countable subset of $P(\Omega )$ such that $A_i\cap A_j=\varnothing$ if $i\ne j$, then $\mathbf{P}\left({⨆}_{i=1}^{\infty }{A}_i\right)={\sum }_{i=1}^{\infty }\mathbf{P}(A_i)$.

$(\Omega ,P(\Omega ),\mathbf{P})$ is called a discrete probability space. It is again possible to assign probabilities to elementary events here, since it is possible for a countably infinite set of real numbers to have a finite sum. For example, if $\Omega =\mathbb{N}$, we can define the probability function from $\Omega$ to $[0,1]$ by $P(n)=\frac{1}{2^n}$.

Continuous probability space

Let $|\Omega |>{\aleph }_0$.

An example: Let $N_0$ be the initial number of atoms in a radioactive sample. The number of atoms that have not decayed at time $t$ is given by $N=N_0{e}^{-\lambda t}$. So, the probability of an atom decaying in the time interval $t_1$ to $t_2$ is $e^{-\lambda t_1}-e^{-\lambda t_2}$. Notice that the probability of an atom decaying at an instant $t$ is $0$, but is non-zero over an interval.

This tells us that probability is not uncountably additive. In a continuous probability space, we cannot assign a probability to the elementary events and hope to the calculate the probability of all other events by adding up the probabilities of their constituent elementary events. Also, when $\Omega$ is uncountable (in this case, $\Omega$ is the set of all real numbers $[0,\infty )$, representing the time of decay), we cannot assign a probability to every set in the power set of $\Omega$ (this is a result you’ll have to just believe for now).

So what do we take the event space to be?

Definition 1(Definition).

$A\subset P(\Omega )$ is said to be a $\sigma$-algebra if

• $\Omega \in A$ and $\varnothing \in A$;
• Whenever a countable collection of sets $\{A_n{\}}_{n=1}^{\infty }$ is in $A$, their union ${\bigcup }_{n=1}^{\infty }{A}_n$ and intersection ${\bigcap }_{n=1}^{\infty }{A}_n$ is also in $A$, i.e. $A$ is closed under countable unions and countable intersections;
• for all $S\in A$, $S^c\in A$.

In a continuous probability space $(\Omega ,\mathcal{F},\mathbf{P})$, the event space $\mathcal{F}$ is a $\sigma$-algebra, and $\mathbf{P}:\mathcal{F}\to [0,1]$ is countably additive.

We will mostly be working with finite and discrete probability spaces in this course.

Info

When working with finite and discrete probability spaces, the event space is almost always the power set of $\Omega$, since we are able to assign probabilities to the elementary events and all events can be expressed as at most countable unions of elementary events. So, the probability space presentation is usually shortened from $(\Omega ,P(\Omega ),\mathbf{P})$ to $(\Omega ,\mathbf{P})$.

---

Properties of the probability function

Theorem 2(The monotone property).

If $A\subseteq B$, then $P(A)\le P(B)$.

Proof
$B=(B\cap A)\cup (B\cap A^c)$. So, $P(B)=P(B\cap A)+P(B\cap A^c)=P(A)+P(B\cap A^c)$.

Theorem 3(Property).

Let $(\Omega ,\mathcal{F},\mathbf{P})$ be a probability space. Let $\{A_n{\}}_{n=1}^{\infty }$ be a countable collection in $\mathcal{F}$ such that $A_n\subseteq A_m$ if $n\le m$. Let $A={\bigcup }_{n=1}^{\infty }{A}_n$. Then, $\mathbf{P}(A)={\lim }_{n\to \infty }\mathbf{P}(A_n)=\sup \mathbf{P}(A_n)$.

Proof
From the previous property, $(P(A_n))$ is a monotone increasing sequence. Since it is also bounded, it must converge to its supremum, i.e, ${\lim }_{n\to \infty }\mathbf{P}(A_n)=\sup \mathbf{P}(A_n)$.
Now, define $B_1=A_1$, $B_n=A_n\setminus A_{n-1}=A_n\cap A_{n-1}^c$ for $n>1$. Note that the collection of all $B_n$s is mutually disjoint. Also note that $A_n={⨆}_{i=1}^n{B}_i$. So,

$$P(A)=P\left(\underset{n=1}{\overset{\infty }{⨆}}{B}_n\right)=\sum _{n=1}^{\infty }P(B_n)=\underset{n\to \infty }{\lim }P(A_n).$$

Use complements to obtain the dual property: Let $\{B_n{\}}_{n=1}^{\infty }$ be a countable collection in $\mathcal{F}$ such that $B_n\supseteq B_m$ if $n\le m$. Let $B={\bigcap }_{n=1}^{\infty }{B}_n$. Then, $\mathbf{P}(B)={\lim }_{n\to \infty }\mathbf{P}(B_n)=\inf \mathbf{P}(B_n)$.