QR decomposition method in GLM.

[mousum-github]

It is well known that the Cholesky decomposition is fast but numerically unstable for ill-conditioned design/model matrix. Also, some users prefer the QR decomposition method in the linear (or generalized linear) model.

In this PR, we have added a keyword argument method in the lm, glm and related functions to control which decomposition method will be used. The method argument takes two values :cholesky (the default value and it uses Cholesky) and :qr (it uses QR).

Using QR decomposition in lm:
Issue #426
lm(model, data; method=:qr)

Coefficients:

• Intercept: -2.18759e6
• Year: 1096.15
• Mileage: -0.0238368

Loglikelihood: -917.7449640609644
r2: 0.46431799718947153
adjr2: 0.454016420212346

Issue #375
lm(@formula(y ~ x), df; method=:qr, dropcollinear=false)

Coefficients:

• Intercept: 65.7485
• x: 9.94455

Summary of additions/changes:

• DensePredQR - now supports both pivoted and unpivoted QR decomposition
• invqr - compute the inverse of X’WX from QR decomposition
• inverse - a wrapper of invqr and invchol, used inside the vcov function.
• delbeta!(p::DensePredQR{T,<:QRCompactWY}, r::Vector{T}, wt::Vector{T})
• delbeta!(p::DensePredQR{T,<:QRPivoted}, r::Vector{T})
• delbeta!(p::DensePredQR{T,<:QRPivoted}, r::Vector{T}, wt::Vector{T})
• Added dropcollinear and method keyword arguments in negbin function
• All test cases related to the linear model (lm) are replicated for Cholesky and QR decompositions
• All test cases related to the glm are replicated for Cholesky and QR decompositions
• Added one example of a linear model with the QR method.

To check and compare the performance of GLM with the QR method, we performed a logistic regression with 3,481,280 rows × 24 columns. Logistic regression took approximately (Intel i7 11th generation with 16GM RAM, Windows 11 OS),

• 56 seconds in R GLM
• 10 seconds in Julia GLM with Cholesky
• 30 seconds in Julia GLM with QR

Finally, @harsharora21 is one of the main contributors to this PR.

[bkamins]

@mousum-github - I can review this PR, but please note that I am not a maintainer of GLM.jl, so someone from the team working with this package should approve and merge this PR.

[droodman]

As a user who is getting wrong results where R and Stata get it right, I think this is a fantastic addition.
One suggestion: rename the method option to something less generic like invmethod (as in inversion method). Consider that FixedEffectModels.jl has a method option, also maybe too generically named, to control whether the CPU or GPU is used. I think that FixedEffectModels.jl also should gain the ability to request the QR; what should that option be called?

[ararslan]

For whatever it's worth, I think the use of method to choose the solution method is more appropriate than using method to choose where a computation is performed. The method keyword here in GLM.jl is the same as in R's lm. I don't know whether that was the motivator behind the name but it is pleasantly discoverable for folks coming from R.