codermusou / mect4cner Goto Github PK

Hello, when will the source code be made public?

AdaptionSelfAttention.py中
if dataset == 'weibo':
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 320, 320), requires_grad=True)
if dataset == 'msra':
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 310, 310), requires_grad=True)
if dataset == 'resume':
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 344, 344), requires_grad=True)
中的320 ，310，344是由什么决定的？若是换成自己的语料需要改哪些部分的代码？

When running"python main.py --dataset weibo" there is an "IndexError: list index out of range", which is directly caused by "return list(c_info[3])" in \MECT4CNER\Modules\CNNRadicalLevelEmbedding.py, line 26, in char2radical (a function defined in the certain file).

I'm wondering why choose the number"3", and how I should do to solve the error.

您好，请问可以开源带BERT的MECT代码吗，想学习一下，感谢

请问下作者分割数据集ontonotes与msra时怎么限制到长度200呢，是直接比较分割的吗？谢谢

限制样本长度为200的msra数据集使用的是单独处理后的数据集吗，能否分享一份处理好的数据集，我对数据集进行处理后16G显存依旧不足，batch为5
邮箱：[email protected]
非常感谢！！！

请问我想在torch1.6的版本中训练模型，我需要怎么修改代码呢，或者修改哪里的代码呢

您好，看了您的文章收获很大，十分感谢。请问实验部分中BERT-MECT是怎么联合起来的，是把bert的输出放在了lattice embedding中吗。后续可以在原有基础上加对应的模块吗

代码呢？

您好，请问AdaptSelfAttention.py中，第39-47行的
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 477, 477), requires_grad=True)
里面的477是怎么算出来的呢

您好，看见论文中有提及MECT与BERT结合后性能有提升，请问方便告知是如何做的结合么？可以的话不知能否提供代码呢？
感激不尽！

微博数据集
FitlogCallback evaluation on data-test:
SpanFPreRecMetric: f=0.577723, pre=0.591479, rec=0.564593
label_acc: acc=0.957026
Evaluation on dev at Epoch 50/50. Step:6750/6750:
SpanFPreRecMetric: f=0.627097, pre=0.629534, rec=0.624679
label_acc: acc=0.961509

In Epoch:48/Step:6480, got best dev performance:
SpanFPreRecMetric: f=0.640327, pre=0.681159, rec=0.604113
label_acc: acc=0.964209
Reloaded the best model.

resume数据集
FitlogCallback evaluation on data-test:
SpanFPreRecMetric: f=0.947818, pre=0.942927, rec=0.952761
label_acc: acc=0.966689
Evaluation on dev at Epoch 50/50. Step:19150/19150:
SpanFPreRecMetric: f=0.938288, pre=0.932103, rec=0.944556
label_acc: acc=0.969474

In Epoch:22/Step:8426, got best dev performance:
SpanFPreRecMetric: f=0.941294, pre=0.934783, rec=0.947896
label_acc: acc=0.966883
Reloaded the best model.

您好，我跑了一下这两个数据集，可是数据结果并没有达到论文的水平，resume的第22个epoch为f=0.944733, pre=0.940426, rec=0.94908，weibo的第48个epoch为f=0.597132, pre=0.65616, rec=0.547847。请问可能是什么原因呢？是因为我用的不是标准的在线新华字典的原因嘛？

File "D:\pythonProject\MECT4CNER\model.py", line 122, in forward
char_encoded = self.char_encoder(components_embed, embedding, embedding, seq_len, lex_num=lex_num, pos_s=pos_s,
File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\pythonProject\MECT4CNER\Modules\TransformerEncoder.py", line 47, in forward
output = self.transformer_layer(query, key, value, seq_len, lex_num=lex_num,
File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\pythonProject\MECT4CNER\Modules\TransformerEncoderLayer.py", line 47, in forward
output = self.attn(query, key, value, seq_len, lex_num=lex_num,
File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\pythonProject\MECT4CNER\Modules\AdaptSelfAttention.py", line 87, in forward
attn_score_raw = A_C + B_D + self.randomAttention[:, :, :max_seq_len, :max_seq_len]
RuntimeError: The size of tensor a (364) must match the size of tensor b (310) at non-singleton dimension 3

dataset: msra
The rest is configured according to the readme section,
Why the result is not good?
^_^ please~

FitlogCallback evaluation on data-test:
SpanFPreRecMetric: f=0.536364, pre=0.553125, rec=0.520588
label_acc: acc=0.957234
Evaluation on dev at Epoch 27/100. Step:3429/12700:
SpanFPreRecMetric: f=0.536364, pre=0.553125, rec=0.520588
label_acc: acc=0.957234

File "/root/miniconda3/envs/torch1.5/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MECT4CNER-master/Modules/TransformerEncoderLayer.py", line 48, in forward
rel_pos_embedding=rel_pos_embedding)
File "/root/miniconda3/envs/torch1.5/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MECT4CNER-master/Modules/AdaptSelfAttention.py", line 87, in forward
attn_score_raw = A_C + B_D + self.randomAttention[:, :, :max_seq_len, :max_seq_len]
RuntimeError: The size of tensor a (454) must match the size of tensor b (310) at non-singleton dimension 3

调试好之后使用weibo数据集训练到二十多epoch的时候会出现loss = nan的情况并且在此后的训练都是loss=nan且在test和dev的evaluation都是0，请问这是为什么呢？

单个GPU运行没有问题。尝试使用多GPU，但是运行时总是说tensor的维度不对。
修改了模型中的lex_mask和mask中的参数，添加了max_len参数，还是报如上的错误

请教一下，请问为什么weibo数据集会报错一些字符不是有效bio格式啊，谢谢！
AssertionError: 淘 is not a valid tag in encoding type:bio. Please check your encoding_type.

一 O
节 O
课 O
的 O
时 O
间 O
真 O
心 O
感 O
动 O
了 O
李 B-PER.NAM
开 I-PER.NAM
复 I-PER.NAM
感 O
动 O

请问有完整代码嘛

是在哪个地方用到了flat模型

请教一下，请问为什么weibo数据集会报错一些字符不是有效bio格式啊
AssertionError: 淘 is not a valid tag in encoding type:bio. Please check your encoding_type.

一 O
节 O
课 O
的 O
时 O
间 O
真 O
心 O
感 O
动 O
了 O
李 B-PER.NAM
开 I-PER.NAM
复 I-PER.NAM
感 O
动 O

hello，when i use Weibo2018 dataset for training,It always shows me an error: "RuntimeError: The size of tensor a (341) must match the size of tensor b (320) at non-singleton dimension 3",in AdaptSelfAttention.py , line 87, what should i do to fix this problem

Trainer训练完成后，使用torch.load，无法加载保存下来的模型
报错:

ModuleNotFoundError: No module named 'model'

我查看了Fastnlp中Trainer保存best model的代码，是

model.cpu()
torch.save(model,model_path)

使用torch.load应该没有问题的，请问作者大大怎么加载训练完成的best model

I want to know on line 26 of CNNRadicalLevelEmbedding.py： return list(c_info[3])，is there a problem？I generated more than a list in the running code.So I changed this to：return list(c_info[:3]),I don't know if this is right. I hope to answer it,thk

I want to know why the author chose to use cross transformer instead of a single transformer or other? Which paper did you get your ideas from? Can you share them？

您好，我使用爬虫获取了汉字部件构造，为什么在weibo数据集resume数据集上都不能达到论文中所描述的结果？
weibo数据集上的f1值平均在0.59左右，resume数据集上f1值也只能在0.94-0.95。
还有个问题，数据集中存在繁体字，这部分没有匹配的汉字部件构造，原实验中是否对这部分也进行了拆分，会不会是因为这部分影响了实验的结果？

请问大佬是如何得到新华字典的部首呢？有什么方法可以获得授权么？如果可以的话可以单独发一份给我嘛~

请问，使用漢語拆字字典得到weibo数据集的精度如下，是正常的么？
[tester]
SpanFPreRecMetric: f=0.574386, pre=0.625352, rec=0.5311
label_acc: acc=0.957296
{'SpanFPreRecMetric': {'f': 0.574386, 'pre': 0.625352, 'rec': 0.5311}, 'label_acc': {'acc': 0.957296}}

离论文中63.30，相差很大。