Covariance Matrix Example

Before getting off the topic of data fits I thought I'd give a simple example to illustrate how a set of points is formed into a matrix and how its covariant matrix is calculated. If the data points are represented by a set of column vectors then one places them side by side in a data matrix, x. One then averages these points to find their center, x_ctr, and subtracting it from each of the points gives Δx which in this case is a 2×6 matrix. Its transpose is 6×2.



Multiplying Δx by its transpose gives the covariance matrix. Some people define the covariance matrix as an expected value which involves averaging but doing this is not necessary in order to find the eigenvector and so the calculation is simpler.

To multiply two matrices together one multiplies a row of the of the first by a column of the second for all possible combinations and in this case gets a 2×2 matrix for the covariance matrix. Notice that the covariance matrix is equal to its transpose which makes it symmetric.