Quadratic form

Paraphrased from Chaudhury 2024, ch. 4 (linear algebraic tools for ML)

Given a square symmetric matrix , the scalar quantity is a quadratic form of Observe that if is a column vector, then will be a row vector. Consequently, is scalar.

Principal component analysis (PCA) can be formulated as the minimization of a quadratic form of the covariance matrix, where the first principal component is the vector that minimizes the form and its magnitude is the variance along that vector.