codermusou / mect4cner Goto Github PK
View Code? Open in Web Editor NEWCode for ACL 2021 paper. MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition.
Code for ACL 2021 paper. MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition.
Hello, when will the source code be made public?
AdaptionSelfAttention.py中
if dataset == 'weibo':
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 320, 320), requires_grad=True)
if dataset == 'msra':
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 310, 310), requires_grad=True)
if dataset == 'resume':
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 344, 344), requires_grad=True)
中的320 ,310,344是由什么决定的?若是换成自己的语料需要改哪些部分的代码?
When running"python main.py --dataset weibo" there is an "IndexError: list index out of range", which is directly caused by "return list(c_info[3])" in \MECT4CNER\Modules\CNNRadicalLevelEmbedding.py, line 26, in char2radical (a function defined in the certain file).
I'm wondering why choose the number"3", and how I should do to solve the error.
您好,请问可以开源带BERT的MECT代码吗,想学习一下,感谢
请问下作者分割数据集ontonotes与msra时怎么限制到长度200呢,是直接比较分割的吗?谢谢
限制样本长度为200的msra数据集使用的是单独处理后的数据集吗,能否分享一份处理好的数据集,我对数据集进行处理后16G显存依旧不足,batch为5
邮箱:[email protected]
非常感谢!!!
请问我想在torch1.6的版本中训练模型,我需要怎么修改代码呢,或者修改哪里的代码呢
您好,看了您的文章收获很大,十分感谢。请问实验部分中BERT-MECT是怎么联合起来的,是把bert的输出放在了lattice embedding中吗。后续可以在原有基础上加对应的模块吗
代码呢?
您好,请问AdaptSelfAttention.py中,第39-47行的
self.randomAttention = nn.Parameter(torch.empty(1, self.num_heads, 477, 477), requires_grad=True)
里面的477是怎么算出来的呢
您好,看见论文中有提及MECT与BERT结合后性能有提升,请问方便告知是如何做的结合么?可以的话不知能否提供代码呢?
感激不尽!
微博数据集
FitlogCallback evaluation on data-test:
SpanFPreRecMetric: f=0.577723, pre=0.591479, rec=0.564593
label_acc: acc=0.957026
Evaluation on dev at Epoch 50/50. Step:6750/6750:
SpanFPreRecMetric: f=0.627097, pre=0.629534, rec=0.624679
label_acc: acc=0.961509
In Epoch:48/Step:6480, got best dev performance:
SpanFPreRecMetric: f=0.640327, pre=0.681159, rec=0.604113
label_acc: acc=0.964209
Reloaded the best model.
resume数据集
FitlogCallback evaluation on data-test:
SpanFPreRecMetric: f=0.947818, pre=0.942927, rec=0.952761
label_acc: acc=0.966689
Evaluation on dev at Epoch 50/50. Step:19150/19150:
SpanFPreRecMetric: f=0.938288, pre=0.932103, rec=0.944556
label_acc: acc=0.969474
In Epoch:22/Step:8426, got best dev performance:
SpanFPreRecMetric: f=0.941294, pre=0.934783, rec=0.947896
label_acc: acc=0.966883
Reloaded the best model.
您好,我跑了一下这两个数据集,可是数据结果并没有达到论文的水平,resume的第22个epoch为f=0.944733, pre=0.940426, rec=0.94908,weibo的第48个epoch为f=0.597132, pre=0.65616, rec=0.547847。请问可能是什么原因呢?是因为我用的不是标准的在线新华字典的原因嘛?
File "D:\pythonProject\MECT4CNER\model.py", line 122, in forward
char_encoded = self.char_encoder(components_embed, embedding, embedding, seq_len, lex_num=lex_num, pos_s=pos_s,
File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\pythonProject\MECT4CNER\Modules\TransformerEncoder.py", line 47, in forward
output = self.transformer_layer(query, key, value, seq_len, lex_num=lex_num,
File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\pythonProject\MECT4CNER\Modules\TransformerEncoderLayer.py", line 47, in forward
output = self.attn(query, key, value, seq_len, lex_num=lex_num,
File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\pythonProject\MECT4CNER\Modules\AdaptSelfAttention.py", line 87, in forward
attn_score_raw = A_C + B_D + self.randomAttention[:, :, :max_seq_len, :max_seq_len]
RuntimeError: The size of tensor a (364) must match the size of tensor b (310) at non-singleton dimension 3
dataset: msra
The rest is configured according to the readme section,
Why the result is not good?
^_^ please~
FitlogCallback evaluation on data-test:
SpanFPreRecMetric: f=0.536364, pre=0.553125, rec=0.520588
label_acc: acc=0.957234
Evaluation on dev at Epoch 27/100. Step:3429/12700:
SpanFPreRecMetric: f=0.536364, pre=0.553125, rec=0.520588
label_acc: acc=0.957234
File "/root/miniconda3/envs/torch1.5/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MECT4CNER-master/Modules/TransformerEncoderLayer.py", line 48, in forward
rel_pos_embedding=rel_pos_embedding)
File "/root/miniconda3/envs/torch1.5/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MECT4CNER-master/Modules/AdaptSelfAttention.py", line 87, in forward
attn_score_raw = A_C + B_D + self.randomAttention[:, :, :max_seq_len, :max_seq_len]
RuntimeError: The size of tensor a (454) must match the size of tensor b (310) at non-singleton dimension 3
调试好之后使用weibo数据集训练到二十多epoch的时候会出现loss = nan的情况并且在此后的训练都是loss=nan且在test和dev的evaluation都是0,请问这是为什么呢?
单个GPU运行没有问题。尝试使用多GPU,但是运行时总是说tensor的维度不对。
修改了模型中的lex_mask和mask中的参数,添加了max_len参数,还是报如上的错误
请教一下,请问为什么weibo数据集会报错一些字符不是有效bio格式啊,谢谢!
AssertionError: 淘 is not a valid tag in encoding type:bio. Please check your encoding_type.
一 O
节 O
课 O
的 O
时 O
间 O
真 O
心 O
感 O
动 O
了 O
李 B-PER.NAM
开 I-PER.NAM
复 I-PER.NAM
感 O
动 O
请问有完整代码嘛
是在哪个地方用到了flat模型
请教一下,请问为什么weibo数据集会报错一些字符不是有效bio格式啊
AssertionError: 淘 is not a valid tag in encoding type:bio. Please check your encoding_type.
一 O
节 O
课 O
的 O
时 O
间 O
真 O
心 O
感 O
动 O
了 O
李 B-PER.NAM
开 I-PER.NAM
复 I-PER.NAM
感 O
动 O
hello,when i use Weibo2018 dataset for training,It always shows me an error: "RuntimeError: The size of tensor a (341) must match the size of tensor b (320) at non-singleton dimension 3",in AdaptSelfAttention.py , line 87, what should i do to fix this problem
Trainer训练完成后,使用torch.load,无法加载保存下来的模型
报错:
ModuleNotFoundError: No module named 'model'
我查看了Fastnlp中Trainer保存best model的代码,是
model.cpu()
torch.save(model,model_path)
使用torch.load应该没有问题的,请问作者大大怎么加载训练完成的best model
I want to know on line 26 of CNNRadicalLevelEmbedding.py: return list(c_info[3]),is there a problem?I generated more than a list in the running code.So I changed this to:return list(c_info[:3]),I don't know if this is right. I hope to answer it,thk
I want to know why the author chose to use cross transformer instead of a single transformer or other? Which paper did you get your ideas from? Can you share them?
您好,我使用爬虫获取了汉字部件构造,为什么在weibo数据集resume数据集上都不能达到论文中所描述的结果?
weibo数据集上的f1值平均在0.59左右,resume数据集上f1值也只能在0.94-0.95。
还有个问题,数据集中存在繁体字,这部分没有匹配的汉字部件构造,原实验中是否对这部分也进行了拆分,会不会是因为这部分影响了实验的结果?
请问大佬是如何得到新华字典的部首呢?有什么方法可以获得授权么?如果可以的话可以单独发一份给我嘛~
请问,使用漢語拆字字典得到weibo数据集的精度如下,是正常的么?
[tester]
SpanFPreRecMetric: f=0.574386, pre=0.625352, rec=0.5311
label_acc: acc=0.957296
{'SpanFPreRecMetric': {'f': 0.574386, 'pre': 0.625352, 'rec': 0.5311}, 'label_acc': {'acc': 0.957296}}
离论文中63.30,相差很大。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.