3.3 Independent \(\sigma\)-algebras and random variables
On the most fundamental, and general, level, independence is formulated for \(\sigma\)-algebras. This notion then naturally extends to random variables through their generated \(\sigma\)-algebras (Definition 2.25).
Definition 3.11
- The \(\sigma\)-algebras \(\mathcal B_1, \dots, \mathcal B_n \subset \mathcal A\) are independent if for all \(A_1 \in \mathcal B_1, \dots, A_n \in \mathcal B_n\) we have \(\mathbb{P}(A_1 \cap \dots \cap A_n) = \mathbb{P}(A_1) \cdots \mathbb{P}(A_n).\)
- The random variables \(X_1, \dots, X_n\) are independent if \(\sigma(X_1), \dots, \sigma(X_n)\) are independent.
Recalling the convention (2.6).
The following result is very convenient: it gives a concrete characterisation of independence of random variables that is very useful when working with independent random variables.
Proposition 3.12
The random variables \(X_1, \dots, X_n\) are independent if and only if the law of \((X_1, \dots, X_n)\) is the product of the laws of \(X_1,\) …, \(X_n,\) i.e. \[\tag{3.6} \mathbb{P}_{(X_1, \dots, X_n)} = \mathbb{P}_{X_1} \otimes \cdots \otimes \mathbb{P}_{X_n}\,.\] In this case we have \[\mathbb{E}[f_1(X_1) \cdots f_n(X_n)] = \mathbb{E}[f_1(X_1)] \cdots \mathbb{E}[f_n(X_n)]\] for any measurable and nonnegative functions \(f_i.\)
Proof. Let \((E_i, \mathcal E_i)\) be the target space of \(X_i.\) Let \(F_i \in \mathcal E_i\) for all \(i.\) Then we have \[\begin{aligned} \mathbb{P}_{(X_1, \dots, X_n)}(F_1 \times \cdots \times F_n) &= \mathbb{P}(X_1 \in F_1, \dots, X_n \in F_n)\,, \\ \mathbb{P}_{X_1} \otimes \cdots \otimes \mathbb{P}_{X_n}(F_1 \times \cdots \times F_n) &= \mathbb{P}(X_1 \in F_1) \cdots \mathbb{P}(X_n \in F_n)\,. \end{aligned}\] Using (3.5), we conclude that \(X_1, \dots, X_n\) are independent if and only if \(\mathbb{P}_{(X_1, \dots, X_n)}\) and \(\mathbb{P}_{X_1} \otimes \cdots \otimes \mathbb{P}_{X_n}\) coincide on all rectangles of the form \(F_1 \times \dots \times F_n.\) The family of such rectangles, \[\mathcal C = \{F_1 \times \dots \times F_n \,\colon F_i \in \mathcal E_i \, \forall i\}\] is stable under finite intersections (Definition 3.7) and it satisfies \(\sigma(\mathcal C) = \mathcal E_1 \otimes \cdots \otimes \mathcal E_n\) (recall Example 1.3 (ii)). By Corollary 3.9 we therefore conclude that \(X_1, \dots, X_n\) are independent if and only if \(\mathbb{P}_{(X_1, \dots, X_n)} = \mathbb{P}_{X_1} \otimes \cdots \otimes \mathbb{P}_{X_n}.\)
For the last statement, we use the Fubini-Tonelli theorem (Proposition 1.14) to conclude \[\begin{gathered} \mathbb{E}\Biggl[\prod_i f_i(X_i)\Biggr] = \int \prod_i f_i(x_i) \, \mathbb{P}_{X_1}(\mathrm dx_1) \cdots \mathbb{P}_{X_n}(\mathrm dx_n) \\ = \prod_i \int f_i(x_i) \, \mathbb{P}_{X_i}(\mathrm dx_i) = \prod_i \mathbb{E}[f_i(X_i)]\,. \end{gathered}\]
Proposition 3.12 shows how to construct independent random variables \(X_1, \dots, X_n\) with given laws \(\mu_1, \dots, \mu_n\) on the spaces \((E_1, \mathcal E_1), \dots (E_n, \mathcal E_n),\) respectively. Indeed, simply choose \(\Omega = E_1\times \cdots \times E_n,\) \(\mathcal A = \mathcal E_1 \otimes \dots \otimes \mathcal E_n,\) \(\mathbb{P}= \mu_1 \otimes \dots \otimes \mu_n,\) and set \(X_i(\omega_1, \dots, \omega_n) :=\omega_i.\) Clearly, (3.6) holds.
Let us record some obvious but important properties of independent random variables.
Remark 3.13
- If \(X_1, \dots, X_n\) are independent random variables with values in \(\mathbb{R},\) then \(\mathbb{E}[X_1 \cdots X_n] = \mathbb{E}[X_1] \cdots \mathbb{E}[X_n]\) provided that \(\mathbb{E}[\lvert X_i \rvert] < \infty\) for all \(i.\) In particular, if \(X_1, \dots, X_n \in L^1\) then \(X_1 \cdots X_n \in L^1.\) Without independence, this is in general false. For instance, for \(X_1 = X_2 = X \in L^1\) in general we have \(X \notin L^2\) (i.e. \(X^2 \notin L^1\)).
- If \(X_1, X_2 \in L^2\) are independent then \(\mathop{\mathrm{Cov}}(X_1, X_2) = 0.\) In words: independent random variables are uncorrelated. The reverse implication (uncorrelated random variables are independent) is in general wrong.
If \(X\) is independent of itself then \(\mathop{\mathrm{Var}}(X) = 0.\) Hence, by Chebyshev, \(\mathbb{P}(\lvert X - \mathbb{E}[X] \rvert > t) = 0\) for all \(t > 0,\) which implies that \(\mathbb{P}(X \neq \mathbb{E}[X]) = 0.\)
Remarkably, if the law of \((X_1, X_2)\) is Gaussian, then independence of \(X_1\) and \(X_2\) is equivalent to them being uncorrelated. This is a consequence of Wick’s theorem (Exercise 3.2).
Remark 3.15
Let \(X, Y, Z\) be independent random variables. Then \(X\) and \(f(Y,Z)\) are independent for any measurable function \(f.\) Indeed, by Proposition 3.12, \[\begin{gathered} \mathbb{P}(X \in A, f(Y,Z) \in B) = (\mathbb{P}_X \otimes \mathbb{P}_Y \otimes \mathbb{P}_Z)(A \times f^{-1}(B)) \\ = \mathbb{P}_X(A) \cdot \mathbb{P}_Y \otimes \mathbb{P}_Z(f^{-1}(B)) = \mathbb{P}(X \in A) \cdot\mathbb{P}(f(Y,Z) \in B)\,. \end{gathered}\] This principle of regrouping random variables can easily be generalised in the obvious way to more random variables.