guoyinwang / leam Goto Github PK

Att_v = tf.contrib.layers.conv2d(G, num_outputs=opt.num_class,kernel_size=[opt.ngram], padding='SAME',activation_fn=tf.nn.relu) #b * s * c
The implementation code above is a conv2D operation on the match matrix G.
while the formulations in the paper below seems only one filter with the size(2r+1) to produce a further match matrix (K*L). I think they are little different. Is it True?

where W ∈ R2r+1 and b ∈ R K u l ∈ R K

Hi,

I read your paper and it's an interesting approach. I'm more interested in the applications to clinical text - assigning ICD codes for medical documents. Specifically, I am attempting to reproduce the results on the MIMIC dataset. I have already setup the system for this dataset using the code provided by Mullenbach et al., 2018.

I went through your code - is there a pipeline to try this model on the MIMIC3 dataset and also produce visualizations for learnt attention across documents?

Looking forward to hearing from you.

Best,
Vijay

Can you send me a new data set that you have trained?Because the address you sent is no longer available.
And I always like to make mistakes when I train data sets myself, because I am a novice

Hi,
I loaded your dataset yelp.p by using cPickle followed by your code:
data = cPickle.load(open('yelp.p', 'wb'))
It raises an error like this: "Value Error: could not convert string to int". Can you check this dataset file again. I think there are some problems with this file.

id_list = [ [ wordtoidx[i] for i in l] for l in name_list]
value_list = [ [ opt.W_emb[i] for i in l] for l in id_list]

I'm confused in somewhere，but got no reply from email，so i'm here.

In section 4.1 Model, when get the matrix G, i thought each element in G can be seen as the score between words and labels. But when you do conv and pooling, what's the meaning of result? max-pooling will lose the position information, which means you can't know the score represent which labels

Besides, how do you get the test accuracy in your paper? Choose the max valid accuracy as the final model then test in Test set?

How to get the cosine similarity(eq2) in paper from the function att_emb_ngram_encoder_maxout?
AND what is the following code Att_v = tf.contrib.layers.conv2d(G, num_outputs=opt.num_class,kernel_size=[opt.ngram], padding='SAME',activation_fn=tf.nn.relu) #b * s * c refer to in this paper?

Can you share the code that handles multi-label classification in this paper
Thank you very much

你好，embedding的损失函数（9），只有K个训练样本的意思吗？ 如何嵌套进training batch呢？

Thanks

Hi,
after correcting a few small typos, I'm running into a real error, apparently related to a wrong embedding size:

Traceback (most recent call last):
File "/home/mat/repos/tmp/LEAM/main.py", line 275, in
main()
File "/home/mat/repos/tmp/LEAM/main.py", line 151, in main
opt.W_class_emb = load_class_embedding( wordtoix, opt)
File "/home/mat/repos/tmp/LEAM/utils.py", line 90, in load_class_embedding
value_list = [ [ opt.W_emb[i] for i in l] for l in id_list]
IndexError: index 514556 is out of bounds for axis 0 with size 300

This is using your latest checkout and the data downloaded from your google drive.

Thanks,
-Mathias

d:\users\yamia\eclipse-workspace\zp\leam\preprocess_yahoo.py(182)()->None
-> pdb.set_trace()
(Pdb)
我自己的数据，出现这个错误，怎么解决？我数据格式如下：标签 （tab键） 一行文本数据

能不能把你的原始数据格式给我看下？我这下不了那些数据，谢谢了！

Hi， Dr. Guo.I’m confused when I run your codes, there are several details I want to consult you.

Firstly, the class 'emb_classifier' in 'main_multiclass.py' as follows, why you only calculate the most relevant class's embedding in multi_class and what is the significance of y_emb, which you didn't use in following codes?

​    y_pos = tf.argmax(y, -1) #？
    y_emb, W_class = embedding_class(y_pos, opt, 'class_emb') # b * e, c * e
    y_emb=tf.cast(y_emb,tf.float32)
    W_class=tf.cast(W_class,tf.float32) # c * e
    W_class_tran = tf.transpose(W_class, [1,0]) # e * c

​

Secondly, in the class of 'att_emb_ngram_encoder_cnn' in 'model.py', the input of convolution you designed here absolutely impossible to get the output size of 'b * s * c' and is not the same with your paper. What is your thinking of here for this convolution and the detailed design for padding, kernel, and output?

    x_emb_0 = tf.squeeze(x_emb,) # b * s * e
    x_emb_1 = tf.multiply(x_emb_0, x_mask) # b * s * e

   H = tf.contrib.layers.conv2d(x_emb_0, num_outputs=opt.embed_size,kernel_size=[10], padding='SAME',activation_fn=tf.nn.relu) #b * s *  c

​Thirdly, why you only calculate the most relevant class as accuracy in your 'main_multiclass.py' as follow? Shouldn't it calculate the whole classes?

​correct_prediction = tf.equal(tf.argmax(prob, 1), tf.argmax(y, 1))

​

hi, i am very interested in your paper. I tried to do experiment on my own Finance News dataset to predict finicial event type given the finicial news but, the model doesn't converge that the loss on training set and development set are stuck on a value and change slowly. I am looking forward to your reply for deeper discussion. Thanks

你好，代码里面的数据集链接进不去，可以更新一下链接或者重新给个地址吗，谢谢！

d:\users\yamia\eclipse-workspace\zp\leam\preprocess_yahoo.py(182)()->None
-> pdb.set_trace()
(Pdb)
我自己的数据，出现这个错误，怎么解决？我数据格式如下：标签 （tab键） 一行文本数据

Hi, there seems to be some issues with the version I cloned:

Traceback (most recent call last):
File "/home/mat/repos/tmp/LEAM/main.py", line 15, in
from model import *
File "/home/mat/repos/tmp/LEAM/model.py", line 2, in
from utils import normalizing
ImportError: cannot import name normalizing

Indeed, there's no function "normalizing" in the utils package.

Thanks,
-Mathias

main.py 和 multiclass.py 查看发现网络结构有区别，这两个是具体解决的问题不一样吗

You take cosine similarity of each vector with each class. so how you know that max similarity is with right class or wrong one?

as in #6 (comment), the implementation is not same as what the paper shows in equation 3.

i think line50 in model.py should be replaced as follows:
G = tf.expand_dims(G, axis=-1) # bsc1 filter: ngram1 filter_num:1
Att_v = tf.contrib.layers.conv2d(G, num_outputs=1, kernel_size=[opt.ngram,1], padding='SAME',activation_fn=tf.nn.relu) #b * s * c * 1
Att_v = tf.squeeze(Att_v,)