rjagerman / shoelace

Neural Learning to Rank using Chainer

Interests ✨

• Machine Learning 🤖
• Counterfactual Learning 🌓
• Information Retrieval 🔎

Contact ✉️

• @RolfJagerman 🐦
• www.jagerman.nl 🌐

shoelace's Issues

How did you compute ndcg?

Hi, thanks for your library.
the accuracy of output example code, for listmle, listpl and listnet are the same. you computed ndcg in your article.
how did you compute ndcg?

Question about LogCumsumExp

Hi Rolf,

I'm reading your code about ListMLE loss and found that in LogCumsumExp the max is subtracted and added back later,

shoelace/shoelace/functions/logcumsumexp.py

Line 19 in 39085b9

y = x - m

, what is the reason for doing that?

Thx
-Yu

How do I use shoelace for prediction?

Hi,

Thank you for making your code available!

I was wondering if you could give me some directions for how to actually use the trained neural network for predicting query scores. That is, supose I have a training file where each line looks like this:

  1 qid:25662 1:5.0 2:1.0 3:1.0

and I trained your model on it following your example code in README:

from shoelace.dataset import LtrDataset
from shoelace.iterator import LtrIterator
from shoelace.loss.listwise import listnet
from chainer import training, optimizers, links, Chain
from chainer.training import extensions

# Load data and set up iterator
with open('./path/to/ranksvm.txt', 'r') as f:
    training_set = LtrDataset.load_txt(f)
training_iterator = LtrIterator(training_set, repeat=True, shuffle=True)

# Create neural network with chainer and apply loss function
predictor = links.Linear(None, 1)
class Ranker(Chain):
    def __call__(self, x, t):
        return listnet(self.predictor(x), t)
loss = Ranker(predictor=predictor)

# Build optimizer, updater and trainer
optimizer = optimizers.Adam()
optimizer.setup(loss)
updater = training.StandardUpdater(training_iterator, optimizer)
trainer = training.Trainer(updater, (40, 'epoch'))
trainer.extend(extensions.ProgressBar())

# Train neural network
trainer.run()

Now I have a test file where each line looks like this:

  qid:34562 1:2.0 2:3.0 3:5.0  

How do I use the trainer above to predict the query score?

Thanks very much!

Format of input dataset?

Nowhere in the docs is the format of input dataset mentioned.
In what format should data be fed to the model?

Thank you?

Official Docs out of date

Hey there, thanks for your library, it seems very useful. I noticed an issue with the docs located at:
https://rjagerman.github.io/shoelace/datasets.html

The section on datasets contains the snippet
`from shoelace.dataset import LtrIterator

iterator = LtrIterator(dataset)`

Which appears to contain an incorrect import statement, as LtrIterator is located in shoelace.iterator.

The Readme.MD contains the correct snippet.

Thanks!