Development of Julia module and R-package for SR1 and sum-modeling facilities of Lancelot (and Galahad) about galahad

[nimgould]

Dear Jim

Thanks for all of this. We are currently using the C interfaces we developed last year as building blocks onwards into python and julia (not R, but that is possible, it simply doesn't cross our horizon and we have no experience at all). There is a caveat, namely that Lancelot is the one package from Galahad that we have not translated as yet, partially because we didn't expect much demand, and also because the partially-separable interface is significantly more complicated than the more-traditional function-gradient-(maybe )Hessian one that most users expect.

One issue though, for machine learning I might be wrong but I thought that each of the sub functions in the sum involved all parameters, in which case the individual Hessians are dense. This doesn't fit well with partial separability, where the underlying assumption is that the Hessians are sparse, or at least structured low rank. Fine if there aren't too many parameters, but difficult/impossible for machine learning with millions of parameters. As I said, maybe this is just a misunderstanding on my part.

The python/julia interfaces are taking all of our time at present, but we will keep this in mind once we have some free time ... and yes, finding interested, knowledgeable collaborators is always helpful.

Nick

[JimEBlevins]

Hi Nick!

It is great to hear about the work towards a Julia interface, which would be more accessible to R-users. R-contributors could develop explicit interfaces using the existing Julia--R interfaces [link to ongoing StackOverflow discussion with leading contributors].

Regarding partial separability:

I agree that statistical uses are negligible compared to the FEM applications of partial separability. I found only 38 statistical citations that are concerned with Hessians (Fisher information matrix) and mention partial separability. Some of these papers mention functional-MRI statistics (which may be related to FEM).

Other statistical applications include spline-based density estimation, following J. W. Schmidt.

Validation and experimentation

For dense problems (regardless of additivity), your code would be useful as a benchmark for evaluating other iterative-methods; such validation of statistical software is important for FDA and EMA submissions.

A second interest is to use any additivity-facilities for experimentation with, e.g., stochastic [quasi-]gradient methods for interesting problems (rather than impressing people with the number of variables or the size of the data). A Julia-laboratory (developed with an eye towards Turing and JuMP) would be very useful to researchers in machine learning.

Thanks

Above, I mentioned the spatial-median problem. It is fitting to note that the spatial median problem is related to the Fermat--Weber location problems, to which the late Professor Andrew R. Conn made many contributions (besides partial separbility, Lancelot, Galahad, etc.). I am very grateful for your team's work.