Generalized least squares

[lindonroberts]

Allow option for generalized least squares (i.e. instead of ||r(x)||^2, have ||r(x)||_A^2, where A is the inverse of the correlation matrix; see https://en.wikipedia.org/wiki/Generalized_least_squares). This captures correlation/heteroskedacity in the residuals.

[SimonTett]

Hi Lindon,
this would be useful! I implement it by translating into (and out of ) the eigenvector space of the covariance matrix.

[SimonTett]

And as I gave been having a hard time with DFOLS and "noise" in my trials so have been looking at that bit of my code I put it here!
Part 1 --generate transformation matrix from Covariance matrix. You might want to filter this
evalue, evect = np.linalg.eigh(Cov)
transMatrix = np.diag(evalue ** (-0.5)).dot(evect.T)
Part 2 -- apply
resid = transMatrix @ fn(params)