ALG1_L23

Bilinear forms

We will now generalize the notion of the inner product. We will see that the inner product, as defined in the previous lecture, is a positive definite symmetric (bilinear) form on a real vector space, and a positive definite Hermitian form on a complex vector space.

Definition 1(Definition).

A bilinear form on a real vector space $V$ is a map $V\times V\to \mathbb{R}$. Given a pair of vectors $\mathbf{v}$ and $\mathbf{w}$, the form returns a real number denoted from now on by $⟨\mathbf{v},\mathbf{w}⟩$. (This is no longer the inner product from the previous lecture!) A bilinear form is linear in each variable.

$$⟨\sum x_i{\mathbf{v}}_i,\mathbf{w}⟩=\sum x_i⟨{\mathbf{v}}_i,\mathbf{w}⟩$$ $$⟨\mathbf{v},\sum {\mathbf{w}}_j{y}_j⟩=\sum ⟨\mathbf{v},{\mathbf{w}}_j⟩y_i$$

Let $⟨,⟩$ be a bilinear form on $V$. Let $V$ have a basis $\mathcal{B}=({\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_n)$. Let $\mathbf{v},\mathbf{w}\in V$. Then,

$$⟨\mathbf{v},\mathbf{w}⟩=⟨\sum _i{x}_i{\mathbf{v}}_i,\sum _j{\mathbf{v}}_j{y}_j⟩=\sum _{i,j}{x}_i⟨{\mathbf{v}}_i,{\mathbf{v}}_j⟩y_j.$$

Observe that this sum can be expressed as

$$\left[\begin{array}{cccc}{x}_1& x_2& \dots & x_n\end{array}\right]\left[\begin{array}{cccc}⟨{\mathbf{v}}_1,{\mathbf{v}}_1⟩& ⟨{\mathbf{v}}_1,{\mathbf{v}}_2⟩& \dots & ⟨{\mathbf{v}}_1,{\mathbf{v}}_n⟩\\ ⟨{\mathbf{v}}_2,{\mathbf{v}}_1⟩& ⟨{\mathbf{v}}_2,{\mathbf{v}}_2⟩& \dots & ⟨{\mathbf{v}}_2,{\mathbf{v}}_n⟩\\ ⋮& ⋮& \ddots & ⋮\\ ⟨{\mathbf{v}}_n,{\mathbf{v}}_1⟩& ⟨{\mathbf{v}}_n,{\mathbf{v}}_2⟩& \dots & ⟨{\mathbf{v}}_n,{\mathbf{v}}_n⟩\end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right]$$ $$=[\mathbf{v}{]}_{\mathcal{B}}^{T}A[\mathbf{w}{]}_{\mathcal{B}}$$

The matrix $A$ is known as the matrix of the bilinear form in the basis $\mathcal{B}$ (may be denoted as $[⟨,⟩{]}_{\mathcal{B}}$. This is non-standard.) Note that the bilinear form itself is independent of the basis chosen; what changes with the choice of basis is its matrix representation. Thus, given a bilinear form $⟨,⟩$ on $V$, $⟨\mathbf{v},\mathbf{w}⟩$ can be calculated by using any basis $\mathcal{B}$ to be $[\mathbf{v}{]}_{\mathcal{B}}^T[⟨,⟩{]}_{\mathcal{B}}[\mathbf{w}{]}_{\mathcal{B}}$.

Similarly, any bilinear form $⟨,⟩$ on ${\mathbb{R}}^n$ is given by

$$⟨\mathbf{v},\mathbf{w}⟩={\mathbf{v}}^{T}A\mathbf{w}$$

for some $n\times n$ matrix $A$. Note that we didn’t have to choose a basis to arrive at the above equation (in a sense, it is “canonical”); the elements of ${\mathbb{R}}^n$ are column vectors, and naturally allow a bilinear form to be described in the above manner. Of course, if we treat ${\mathbb{R}}^n$ as an abstract vector space and choose an arbitrary basis $\mathcal{B}$, we can write elements in terms of their coordinates in $\mathcal{B}$, leading again to the formula: $[\mathbf{v}{]}_{\mathcal{B}}^T[⟨,⟩{]}_{\mathcal{B}}[\mathbf{w}{]}_{\mathcal{B}}$. The distinction between treating ${\mathbb{R}}^n$ as a coordinate space with its canonical basis and as an abstract vector space with a chosen basis is subtle but important.

Change of basis

The matrix of a bilinear form $⟨,⟩$ depends on our choice of basis, as must be evident from the above discussion. How does this matrix change when we change the basis?

Let $⟨,⟩$ be a bilinear form on a real vector space $V$, and let $A$ and $A^{\prime }$ be its matrices with respect to bases $\mathcal{B}$ and ${\mathcal{B}}^{\prime }$. Let $P$ be the the change of basis matrix from $\mathcal{B}$ to ${\mathcal{B}}^{\prime }$ (i.e, $[\mathbf{v}{]}_{\mathcal{B}}=P[\mathbf{v}{]}_{{\mathcal{B}}^{\prime }}$ for all $\mathbf{v}\in V$). Then,

$$\begin{array}{r}⟨\mathbf{v},\mathbf{w}⟩=[\mathbf{v}{]}_{\mathcal{B}}^{T}A[\mathbf{w}{]}_{\mathcal{B}}=(P[\mathbf{v}{]}_{{\mathcal{B}}^{\prime }}{)}^{T}A(P[\mathbf{w}{]}_{{\mathcal{B}}^{\prime }})=[\mathbf{v}{]}_{{\mathcal{B}}^{\prime }}^T(P^{T}AP)[\mathbf{w}{]}_{{\mathcal{B}}^{\prime }}\end{array}.$$

Thus, we have $A^{\prime }=P^{T}AP$.

Theorem 2(Theorem).

If $A$ is the matrix of a bilinear form is a basis, and you change the basis such that the change of basis matrix is $P$, the new matrix of the bilinear form is $P^{T}AP$.

Important

For a real vector space $V$ with dimension $n$, when a basis is given, both linear operators and bilinear forms are described by $n\times n$ matrices. However, the theories of linear operators and bilinear forms are not equivalent. When one makes a change of basis, the matrix $A$ of a bilinear form changes to $P^{T}AP$, while the matrix $A$ of a linear operator changes to $P^{-1}AP$.

Symmetric forms

A bilinear form is symmetric if $⟨\mathbf{v},\mathbf{w}⟩=⟨\mathbf{w},\mathbf{v}⟩$ for all $\mathbf{v},\mathbf{w}\in V$. “symmetric form” is short for “symmetric bilinear form”.

Let $A$ be an $n\times n$ matrix. Say the form defined by ${\mathbf{v}}^{T}A\mathbf{w}$ on ${\mathbb{R}}^n$ is symmetric. Then, we must have $⟨{\mathbf{e}}_i,{\mathbf{e}}_j⟩=⟨{\mathbf{e}}_j,{\mathbf{e}}_i⟩$ for all $i,j$. Note that ${\mathbf{e}}_i^{T}A{\mathbf{e}}_j=a_{ij}$. Thus, we have $a_{ij}=a_{ji}$ for all $i,j$, i.e, $A$ is symmetric! Is the converse true? Let $A$ be symmetric. Thinking of ${\mathbf{v}}^{T}A\mathbf{w}$ as a $1\times 1$ matrix, it is equal to its transpose. Thus, we have $⟨\mathbf{v},\mathbf{w}⟩={\mathbf{v}}^{T}A\mathbf{w}={\mathbf{w}}^T{A}^T\mathbf{v}={\mathbf{w}}^{T}A\mathbf{v}=⟨\mathbf{w},\mathbf{v}⟩$.

Theorem 3(Theorem).

Let $A$ be an $n\times n$ matrix. The form ${\mathbf{v}}^{T}A\mathbf{w}$ on ${\mathbb{R}}^n$ is symmetric if and only if $A$ is symmetric.

Now consider a bilinear form $⟨,⟩$ on an abstract vector space $V$. Pick an arbitrary basis $\mathcal{B}$. Let the matrix of the bilinear form in this basis be $A$. Then, if the form is symmetric, $[\mathbf{v}{]}_{\mathcal{B}}^{T}A[\mathbf{w}{]}_{\mathcal{B}}=[\mathbf{w}{]}_{\mathcal{B}}^{T}A[\mathbf{v}{]}_{\mathcal{B}}$, i.e, the form on ${\mathbb{R}}^n$ defined by ${\mathbf{x}}^{T}A\mathbf{y}$ is symmetric, which implies $A$ is symmetric.

Theorem 4(Theorem).

A bilinear form $⟨,⟩$ is symmetric if and only if its matrix with respect to an arbitrary basis is a symmetric matrix.

Positive definite forms

A bilinear form is positive definite if $⟨\mathbf{v},\mathbf{v}⟩>0$ for all nonzero vectors $\mathbf{v}$. The dot product is a symmetric, positive definite form on ${\mathbb{R}}^n$. The matrix of the dot product on ${\mathbb{R}}^n$ is the identity matrix. Thus, if $⟨,⟩$ is the dot product, $⟨\mathbf{v},\mathbf{w}⟩={\mathbf{v}}^T\mathbf{w}$. If we change basis to $\mathcal{B}$ using a change of basis matrix $P$, then $⟨\mathbf{v},\mathbf{w}⟩=[\mathbf{v}{]}_{\mathcal{B}}{P}^{T}P[\mathbf{w}{]}_B$, i.e, the matrix of the dot product becomes $P^{T}P$. If the change of basis is orthogonal, $P^{T}P$ is the identity matrix, and $⟨\mathbf{v},\mathbf{w}⟩=⟨[\mathbf{v}{]}_{\mathcal{B}},[\mathbf{w}{]}_{\mathcal{B}}⟩$.

Analogously to the terminology for positive forms, we say a matrix $A$ is positive definite if the form defined by $A$ on ${\mathbb{R}}^n$ is positive definite, i.e, ${\mathbf{x}}^{T}A\mathbf{x}>0$ for all nonzero column vectors $\mathbf{x}$. Evidently, if the form ${\mathbf{x}}^{T}A\mathbf{x}$ is equivalent to the dot product (which is positive definite), $A$ must be positive definite.

Consider the following properties of a real $n\times n$ matrix:

1. The form ${\mathbf{x}}^{T}A\mathbf{x}$ on ${\mathbb{R}}^n$ represents the dot product with respect to some basis of ${\mathbb{R}}^n$.
2. There is an invertible matrix $P$ such that $A=P^{T}P$.
3. The matrix $A$ is symmetric and positive definite.

We have seen that 1 and 2 are equivalent, and that 1 implies 3. We will see that 3 implies 1 in a bit. So, the above statements are equivalent.

---

Hermitian forms

Definition 5(Definition).

A Hermitian form on a complex vector space $V$ is a map $V\times V\to \mathbb{C}$ denoted by $⟨\mathbf{v},\mathbf{w}⟩$. A Hermitian form is

1. Conjugate linear in the first variable
2. Linear in the second variable
3. Hermitian symmetric

• Because of Hermitian symmetry, $⟨\mathbf{v},\mathbf{v}⟩=\overline{⟨\mathbf{v},\mathbf{v}⟩}$, so $⟨\mathbf{v},\mathbf{v}⟩\in \mathbb{R}$ for all $\mathbf{v}\in V$.
• The transpose operation is replaced by the adjoint here: $A^{\ast }=\overline{A^T}$.
• The standard hermitian form is the complex analogue of the the dot product: $⟨\mathbf{v},\mathbf{w}⟩={\mathbf{v}}^{\ast }\mathbf{w}$.

The matrix of a Hermitian form with respect to a basis is defined as for bilinear forms. The matrix of the standard hermitian form on ${\mathbb{C}}^n$ is the identity matrix.

$A$ is a hermitian matrix $⟺$ the form defined by $A$ is a hermitian form.
Change of basis: $A^{\prime }=P^{\ast }AP$.

Eigenvalues (and so trace and determinant) of a Hermitian matrix are real numbers.
Cor: eigenvalues of real symmetric matrix are real numbers.

P is a unitary matrix (analog to orthogonal matrices): $P^{\ast }P=I$. It’s columns are orthonormal with respect to the standard hermitian form.

Change of basis in ${\mathbb{R}}^n$ preserves dot product if and only if the change of basis matrix is orthogonal. Similarly, a change of basis in ${\mathbb{C}}^n$ preserves the standard Hermitian form $X^{\ast }Y$ if and only if the change of basis matrix is unitary.

A form may be degenerate on a subspace, though it is nondegenerate on the whole space, and vice versa. Example:

$$\left[\begin{array}{ccc}0& 1& 1\\ 1& 0& 1\\ 1& 1& 0\end{array}\right]$$

non degenerate on R3. Degenerate on the span of $[1,0,0]$.

A vector $\mathbf{v}$ is a null vector iff its coordinate vector $Y$ solves the homogeneous equaiton $AY=0$.
The form in nondegenerate iff the matrix mKA is invertible.

Let $⟨,⟩$ be a symmetric form on a re