Completing the Square

This operation, while useful in elementary algebra, also arises frequently when manipulating Gaussian random variables. I review and document both the univariate and multivariate cases.

In elementary algebra, we learned about completing the square. This operation has many uses. For most people, the first use of completing the square is finding roots of quadratic equations, and the operation is one way of re-deriving the quadratic formula. I have recently found completing the square useful because it also arises when manipulating Gaussian random variables, since every normal distribution is the exponential of a quadratic function. This post is a review of the univariate case and an extension to the multivariate case.

Univariate

To review, given a variable $x \in \mathbb{R}$, we want to write a quadratic polynomial

in the form

This second form is often easier to work with. For example, the real roots of the polynomial are immediately obvious from $(2)$. To see how to complete the square, let’s expand the squared term in $(2)$,

And now we just solve for $d$ and $k$,

The reason this trick is called “completing the square” is because it can be viewed as adding a term to $(1)$ such that you can square the equation. For example,

Putting our results together, we get

Multivariate

Now let’s consider the multivariate extension. Let $\mathbf{x}$ and $\mathbf{b}$ be $d$-dimensional vectors, and let $M \in \mathbb{R}^{d \times d}$ be a symmetric invertible matrix. Then

We can easily verify this by multiplying out the quadratic form,

Of course, the $\mathbf{b}^{\top} M^{-1} \mathbf{b}$ term cancels with the one in $(4)$, and we’re done.