xinyandai / string-embed

string embed for fast edit distance computation, codes for [Convolutional Embedding for Edit Distance (SIGIR 20)].

Hey, I'm Xinyan DAI!

• 🔭 I was awarded a Ph.D. at CUHK, in machine learning and similarity search

stats

  

string-embed's Issues

Extract embeddings/predict for new data after trained the model?

Hi,

Thanks for the great work and the ingenious paper!

I trained a model based on the line that is given on the project's page, specifically:

python main.py --dataset word --nt 1000 --nq 1000 --epochs 20 --save-split --recall --save-model
Now I have the trained model (I know it is not the version that is presented in the paper), and a list of two words, how can I use the model in order to predict their edit distance?
I got lost going through the preprocessing stage.

Thanks again, and sorry for the noob question :)

Arguments for running on other datasets?

I believe I found other datasets that your paper reference (eg Enron, Uniref, gen50ks) here: https://github.com/kedayuge/Embedjoin/blob/master/README.md

I was wondering if you can provide the command line for the recommended arguments to run on these other datasets, as well as the expected run time for them? I noticed that a table of edit distances needs to be made, which can run in quadratic time with dataset size.

View is across batch instances?

string-embed/networks.py

Lines 71 to 72 in 61e0347

	x = x.view(-1, self.mtc_input, self.M)
	x = self.conv(x)

It seems to be this code is making x to mix values from different batch instances. Instead, I think it should be x = x.view(x.size(0), self.mtc_input, -1)

Duplicate self.Ls. Merge error?

Hi, thanks for sharing the code from the paper! I think I found a bug, looks like a git merge issue:

string-embed/networks.py

Lines 412 to 425 in 8a8a2ea

	self.Ls = {
	step * 0: (0, 10),
	step * 1: (10, 10),
	step * 2: (10, 1),
	step * 3: (5, 0.1),
	step * 4: (1, 0.01),
	}
	self.Ls = {
	step * 0: (15, 0),
	step * 1: (10, 0),
	step * 2: (8, 0),
	step * 3: (5, 0.0),
	step * 4: (1, 0.0),
	}

Should be only one of the self.Ls, right?
And it seems the current master version is not using the mse_loss, since it's all zero.

您好，请问recall-item这个指标的实际效果意义是？

就是retrieve topN的里面更有可能包含正确答案？

@xinyandai 多谢多谢多谢！

您好，请问下我的这个理解对么？

用编辑距离得到的数据作为训练数据训练一个模型，然后recall-item会提升？
正负例是如何构造的？比BM25效果怎样？
@xinyandai 多谢多谢多谢！