To see the "functional regression" formula in action we can solve for the Fourier coefficients of a simple square wave.
The Variance appears large but this is due to the large number of terms. The fit is shown below and the ringing is the Gibbs phenomenon due to the limitation in the number of terms used.
The set of functions use are orthogonal so the function correlation matrix is diagonal. The first few "fit" coefficients compare well with the computed Fourier coefficients.