Cholesky solver about rsparse

[david-cortes]

But that loss is not the same function that is being minimized. The solver minimizes (or is supposed to minimize) the values over all entries, whereas the loss function calculates it over the present entries.

[dselivanov]

Thanks for reporting, I will take a look. Actually I expect the problem is the same as in #35 - XtX.

Which is certainly not what I'd expect as the Cholesky is a more exact method and should lead to better results.

Loss with Cholesky should be lower, but hit rate might be worse since we don't optimize it directly...

[dselivanov]

@david-cortes could set logger to debug and check the loss? From what I see cholesky achieves smaller (or about the same) loss:

library(rsparse)
library(lgr)
lg = get_logger('rsparse')
lg$set_threshold('debug')
data('movielens100k')

train = movielens100k
train = log1p(train)

for (seed in c(1, 2, 3)) {
  for (solver in c('cholesky', 'conjugate_gradient')) {
    model = WRMF$new(rank = 50,  lambda = 1, solver = solver)
    message(glue::glue("seed {seed}, solver {solver}"))
    user_emb = model$fit_transform(train, n_iter = 10, convergence_tol = -1, cg_steps = 3)
  }
}

[dselivanov]

@david-cortes you are right! starting to forget all the details...

[david-cortes]

Well I did some experiments calculating over all entries and they do indeed end up with similar loss. Weird that the resulting from Cholesky then performs worse at other metrics, and that it takes so little time.

[david-cortes]

Did some more experiments this time, from the same movielens. After running for 10 iterations with seed=1, I get these losses:

• CG: 74202.05
• Cholesky: 74295.25

Whereas with the other package I was comparing, I get these:

• CG: 126483.4
• Cholesky: 78421.88

Wonder where the difference comes from...

[dselivanov]

And if you remove arma::solve_opts::fast argument from the solver?

[david-cortes]

Then the loss does end up lower than for CG, even though in a test set it didn't improve HR@5 or AUC. I guess technically it is working as it should - that is, it minimizes the function it is intended to - so I'll close this.