Comments (6)
huseinzol05
commented on May 12, 2024
I cut the dataset to use 500 sentences only. you might want to use the whole dataset for better experiments.
from nlp-models-tensorflow.
katherinelyx
commented on May 12, 2024
Actually, I used the whole dataset but the perplexity shows too low (lower means better).
from nlp-models-tensorflow.
huseinzol05
commented on May 12, 2024
Oh, I believe thats good? how about test some BLEU or word position accuracy on dev / test split? I implemented word position accuracy in the notebooks, can you paste some logs here?
from nlp-models-tensorflow.
katherinelyx
commented on May 12, 2024
Epoch 0 Batch 50/225 - Loss: 3.2396 - perplextity: 25.5224
Epoch 0 Batch 50/225 - Test Loss: 3.1811 - Test perplextity: 24.0732
Epoch 0 Batch 100/225 - Loss: 3.1357 - perplextity: 23.0053
Epoch 0 Batch 100/225 - Test Loss: 2.7565 - Test perplextity: 15.7445
Epoch 0 Batch 150/225 - Loss: 2.6570 - perplextity: 14.2531
Epoch 0 Batch 150/225 - Test Loss: 2.6090 - Test perplextity: 13.5860
Epoch 1 Batch 50/225 - Loss: 2.5518 - perplextity: 12.8302
Epoch 1 Batch 50/225 - Test Loss: 2.5145 - Test perplextity: 12.3599
Epoch 1 Batch 100/225 - Loss: 2.7359 - perplextity: 15.4241
Epoch 1 Batch 100/225 - Test Loss: 2.4095 - Test perplextity: 11.1285
Epoch 1 Batch 150/225 - Loss: 2.3386 - perplextity: 10.3663
Epoch 1 Batch 150/225 - Test Loss: 2.3367 - Test perplextity: 10.3469
Epoch 2 Batch 50/225 - Loss: 2.2795 - perplextity: 9.7714
Epoch 2 Batch 50/225 - Test Loss: 2.2351 - Test perplextity: 9.3470
Epoch 2 Batch 100/225 - Loss: 2.4809 - perplextity: 11.9526
Epoch 2 Batch 100/225 - Test Loss: 2.1800 - Test perplextity: 8.8461
Epoch 2 Batch 150/225 - Loss: 2.1127 - perplextity: 8.2705
Epoch 2 Batch 150/225 - Test Loss: 2.1392 - Test perplextity: 8.4927
Epoch 3 Batch 50/225 - Loss: 2.0933 - perplextity: 8.1114
Epoch 3 Batch 50/225 - Test Loss: 2.0807 - Test perplextity: 8.0101
Epoch 3 Batch 100/225 - Loss: 2.3165 - perplextity: 10.1398
Epoch 3 Batch 100/225 - Test Loss: 2.0538 - Test perplextity: 7.7971
Epoch 3 Batch 150/225 - Loss: 1.9877 - perplextity: 7.2991
Epoch 3 Batch 150/225 - Test Loss: 2.0336 - Test perplextity: 7.6416
Epoch 4 Batch 50/225 - Loss: 1.9937 - perplextity: 7.3424
Epoch 4 Batch 50/225 - Test Loss: 2.0070 - Test perplextity: 7.4412
Epoch 4 Batch 100/225 - Loss: 2.2090 - perplextity: 9.1067
Epoch 4 Batch 100/225 - Test Loss: 1.9855 - Test perplextity: 7.2825
Epoch 4 Batch 150/225 - Loss: 1.9120 - perplextity: 6.7669
Epoch 4 Batch 150/225 - Test Loss: 1.9765 - Test perplextity: 7.2174
Epoch 5 Batch 50/225 - Loss: 1.9217 - perplextity: 6.8324
Epoch 5 Batch 50/225 - Test Loss: 1.9546 - Test perplextity: 7.0608
Epoch 5 Batch 100/225 - Loss: 2.1339 - perplextity: 8.4479
Epoch 5 Batch 100/225 - Test Loss: 1.9412 - Test perplextity: 6.9672
Epoch 5 Batch 150/225 - Loss: 1.8519 - perplextity: 6.3719
Epoch 5 Batch 150/225 - Test Loss: 1.9381 - Test perplextity: 6.9458
Epoch 6 Batch 50/225 - Loss: 1.8610 - perplextity: 6.4300
Epoch 6 Batch 50/225 - Test Loss: 1.9214 - Test perplextity: 6.8304
Epoch 6 Batch 100/225 - Loss: 2.0644 - perplextity: 7.8804
Epoch 6 Batch 100/225 - Test Loss: 1.9103 - Test perplextity: 6.7549
Epoch 6 Batch 150/225 - Loss: 1.7991 - perplextity: 6.0440
Epoch 6 Batch 150/225 - Test Loss: 1.9102 - Test perplextity: 6.7543
Epoch 7 Batch 50/225 - Loss: 1.8036 - perplextity: 6.0717
Epoch 7 Batch 50/225 - Test Loss: 1.8949 - Test perplextity: 6.6519
Epoch 7 Batch 100/225 - Loss: 2.0062 - perplextity: 7.4351
Epoch 7 Batch 100/225 - Test Loss: 1.8878 - Test perplextity: 6.6046
Epoch 7 Batch 150/225 - Loss: 1.7517 - perplextity: 5.7647
Epoch 7 Batch 150/225 - Test Loss: 1.8894 - Test perplextity: 6.6155
Epoch 8 Batch 50/225 - Loss: 1.7498 - perplextity: 5.7532
Epoch 8 Batch 50/225 - Test Loss: 1.8740 - Test perplextity: 6.5143
Epoch 8 Batch 100/225 - Loss: 1.9525 - perplextity: 7.0464
Epoch 8 Batch 100/225 - Test Loss: 1.8725 - Test perplextity: 6.5045
Epoch 8 Batch 150/225 - Loss: 1.7382 - perplextity: 5.6873
Epoch 8 Batch 150/225 - Test Loss: 1.8799 - Test perplextity: 6.5528
Epoch 9 Batch 50/225 - Loss: 1.7048 - perplextity: 5.5004
Epoch 9 Batch 50/225 - Test Loss: 1.8631 - Test perplextity: 6.4438
Epoch 9 Batch 100/225 - Loss: 1.9123 - perplextity: 6.7690
Epoch 9 Batch 100/225 - Test Loss: 1.8637 - Test perplextity: 6.4479
Epoch 9 Batch 150/225 - Loss: 1.6760 - perplextity: 5.3441
Epoch 9 Batch 150/225 - Test Loss: 1.8560 - Test perplextity: 6.3980
Epoch 10 Batch 50/225 - Loss: 1.6487 - perplextity: 5.2003
Epoch 10 Batch 50/225 - Test Loss: 1.8510 - Test perplextity: 6.3661
Epoch 10 Batch 100/225 - Loss: 1.8625 - perplextity: 6.4397
Epoch 10 Batch 100/225 - Test Loss: 1.8534 - Test perplextity: 6.3814
Epoch 10 Batch 150/225 - Loss: 1.6387 - perplextity: 5.1486
Epoch 10 Batch 150/225 - Test Loss: 1.8477 - Test perplextity: 6.3454
Epoch 11 Batch 50/225 - Loss: 1.6005 - perplextity: 4.9557
Epoch 11 Batch 50/225 - Test Loss: 1.8444 - Test perplextity: 6.3245
Epoch 11 Batch 100/225 - Loss: 1.8206 - perplextity: 6.1756
Epoch 11 Batch 100/225 - Test Loss: 1.8453 - Test perplextity: 6.3298
Epoch 11 Batch 150/225 - Loss: 1.6047 - perplextity: 4.9765
Epoch 11 Batch 150/225 - Test Loss: 1.8449 - Test perplextity: 6.3275
Epoch 12 Batch 50/225 - Loss: 1.5533 - perplextity: 4.7272
Epoch 12 Batch 50/225 - Test Loss: 1.8412 - Test perplextity: 6.3038
Epoch 12 Batch 100/225 - Loss: 1.7814 - perplextity: 5.9379
Epoch 12 Batch 100/225 - Test Loss: 1.8413 - Test perplextity: 6.3046
Epoch 12 Batch 150/225 - Loss: 1.5708 - perplextity: 4.8104
Epoch 12 Batch 150/225 - Test Loss: 1.8434 - Test perplextity: 6.3181
Epoch 13 Batch 50/225 - Loss: 1.5063 - perplextity: 4.5098
Epoch 13 Batch 50/225 - Test Loss: 1.8406 - Test perplextity: 6.3002
Epoch 13 Batch 100/225 - Loss: 1.7443 - perplextity: 5.7221
Epoch 13 Batch 100/225 - Test Loss: 1.8420 - Test perplextity: 6.3090
Epoch 13 Batch 150/225 - Loss: 1.5340 - perplextity: 4.6365
Epoch 13 Batch 150/225 - Test Loss: 1.8408 - Test perplextity: 6.3016
Epoch 14 Batch 50/225 - Loss: 1.4640 - perplextity: 4.3233
Epoch 14 Batch 50/225 - Test Loss: 1.8399 - Test perplextity: 6.2960
Epoch 14 Batch 100/225 - Loss: 1.7052 - perplextity: 5.5023
Epoch 14 Batch 100/225 - Test Loss: 1.8406 - Test perplextity: 6.3000
Epoch 14 Batch 150/225 - Loss: 1.5005 - perplextity: 4.4838
Epoch 14 Batch 150/225 - Test Loss: 1.8395 - Test perplextity: 6.2931
Epoch 15 Batch 50/225 - Loss: 1.4342 - perplextity: 4.1965
Epoch 15 Batch 50/225 - Test Loss: 1.8393 - Test perplextity: 6.2924
Epoch 15 Batch 100/225 - Loss: 1.6726 - perplextity: 5.3259
Epoch 15 Batch 100/225 - Test Loss: 1.8384 - Test perplextity: 6.2866
Epoch 15 Batch 150/225 - Loss: 1.4723 - perplextity: 4.3592
Epoch 15 Batch 150/225 - Test Loss: 1.8417 - Test perplextity: 6.3072
Epoch 16 Batch 50/225 - Loss: 1.3969 - perplextity: 4.0426
Epoch 16 Batch 50/225 - Test Loss: 1.8398 - Test perplextity: 6.2955
Epoch 16 Batch 100/225 - Loss: 1.6442 - perplextity: 5.1768
Epoch 16 Batch 100/225 - Test Loss: 1.8409 - Test perplextity: 6.3021
Epoch 16 Batch 150/225 - Loss: 1.4423 - perplextity: 4.2305
Epoch 16 Batch 150/225 - Test Loss: 1.8445 - Test perplextity: 6.3247
Epoch 17 Batch 50/225 - Loss: 1.3791 - perplextity: 3.9713
Epoch 17 Batch 50/225 - Test Loss: 1.8484 - Test perplextity: 6.3497
Epoch 17 Batch 100/225 - Loss: 1.6208 - perplextity: 5.0572
Epoch 17 Batch 100/225 - Test Loss: 1.8447 - Test perplextity: 6.3261
Epoch 17 Batch 150/225 - Loss: 1.4237 - perplextity: 4.1524
Epoch 17 Batch 150/225 - Test Loss: 1.8416 - Test perplextity: 6.3069
Epoch 18 Batch 50/225 - Loss: 1.3451 - perplextity: 3.8385
Epoch 18 Batch 50/225 - Test Loss: 1.8453 - Test perplextity: 6.3297
Epoch 18 Batch 100/225 - Loss: 1.5864 - perplextity: 4.8861
Epoch 18 Batch 100/225 - Test Loss: 1.8483 - Test perplextity: 6.3489
Epoch 18 Batch 150/225 - Loss: 1.3997 - perplextity: 4.0540
Epoch 18 Batch 150/225 - Test Loss: 1.8428 - Test perplextity: 6.3144
Epoch 19 Batch 50/225 - Loss: 1.3163 - perplextity: 3.7295
Epoch 19 Batch 50/225 - Test Loss: 1.8469 - Test perplextity: 6.3401
Epoch 19 Batch 100/225 - Loss: 1.5570 - perplextity: 4.7445
Epoch 19 Batch 100/225 - Test Loss: 1.8510 - Test perplextity: 6.3664
Epoch 19 Batch 150/225 - Loss: 1.3776 - perplextity: 3.9655
Epoch 19 Batch 150/225 - Test Loss: 1.8449 - Test perplextity: 6.3277
Model Trained and Saved
from nlp-models-tensorflow.
katherinelyx
commented on May 12, 2024
According to the results showed above, the test perplexity achieves 6.3277 after 20 epochs, which is too much lower than usual results from other published papers. I will try some other metrics.
from nlp-models-tensorflow.
huseinzol05
commented on May 12, 2024
skeptical much? haha. Maybe you read old research papers? maybe they use old seq2seq APIs?
from nlp-models-tensorflow.
Related Issues (20)
- TF version HOT 1
- Explain the time taken column
- embedded for data? HOT 1
- Could you please share example of code on how to use trained models for the text classification? HOT 1
- package requirements HOT 2
- missing embed_seq() HOT 2
- 1.lstm-seq2seq-greedy.ipynb In [17] missing 1 required positional argument: 'maxlen'
- Tensorflow 1.1not compatible with cuda 9 or 10 HOT 1
- Could you code predict function transfer-learning-albert-base.ipynb
- Could you please provide the installation guide for augmentation in Speech2text parts? HOT 1
- Spelling Correction- Shape must be rank 2 but is rank 3 for 'cls/predictions/MatMul' (op: 'MatMul') with input shapes: [?,?,768], [768,30522]. HOT 3
- why you have so many Chatbot notebooks?? HOT 2
- Could you add more details for the spell correction section? HOT 1
- Can't run with python 3.6 and Tensorflow 2.3
- spelling-correction/4.bert-accurate.ipynb get_score Method is not defined HOT 1
- attention/1.bahdanau.ipynb文件中引用了utils中的函数，但是没找到utils相关文件
- 从文本分类中测试结果看，好像fasttext的性价比最高，acc：0.76，耗时：0.49499；为啥fasttext直接训练会达到如此好的效果？
- Where can I see the paper corresponding to the code？
- problem in data download in OCR
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nlp-models-tensorflow.