I have issue with this code, there are several condition, that explain from several statements here
Issue 1
because it's using benepar, and download the benepar_en2 or benepar_en2.gz, we must make file to setup first like
import nltk, benepar
nltk.download('punkt')
benepar.download('benepar_en2')
after that, when I want to run python config.py. there's error says we must change several statements in benepar library such as /usr/local/lib/<python_version>/dist-packages/benepar/base_parser.py, changed it into like this syntax: (i prefer using nano editor to change the root directory or nltk directories)
graph = tf.Graph()
graph_def = tf.compat.v1.GraphDef()
but there's an error appear again, we must using tf2 disable behaviour like this syntax:
import tensorflow.compat.v1 as tf
tf..disable_v2_behavior()
it's work, as my suggestion..
but we should thinking twice, when there's an update from benepar library next future issue..
Issue 2
because we're using glove pretrained models, I suggest when we're using load_word2vec_format.
we must convert first from glove to word2vec format, using this command line
from gensim.scripts.glove2word2vec import glove2word2vec
...
GLOVE_TXT_PATH = DATA_PATH + "original/Glove/glove.6B.300d.txt"
GLOVE_OUT_PATH = DATA_PATH + "original/Glove/word2vec.txt" # whatever format do you want
...
glove2word2vec(GLOVE_TXT_PATH, GLOVE_OUT_PATH)
GLOVE = gensim.models.KeyedVectors.load_word2vec_format(GLOVE_OUT_PATH, binary=True, encoding= 'utf-8', unicode_errors= 'ignore')
# or
GLOVE = gensim.models.KeyedVectors.load_word2vec_format(GLOVE_OUT_PATH, binary=False)
and don't forget to make folder as mentioned in common/constants.py such as
1. FQG/src/model/Fsctorized
2. FQG/output
3. FQG/output/checkpoint in
4. FQG/output/figure
5. FQG/output/log
6. FQG/output/pkl
7. FQG/output/result
Issue 3
if we have CUDA device, I suggest you to use cuda channles that available like
DEVICE = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
or
DEVICE = torch.device("cuda:1" if torch.cuda.is_available() else "cpu")
but there are several common problems that have been solved such as..
model/encoder.py - in line 50
emb = pack(sorted_input_emb, sorted_lengths.to('cpu'), batch_first=False)
we must change the lenghts device value into cpu, and thenmodel/beam_searcher.py - in line 138
self.attn.append(attnOut.index_select(0, father_beam_idx))
actually this problem, because father beam is not great to calculate, so we must change the variable '/' into '//'
so the syntax this:
father_beam_idx = best_output_accumulate_scores_id // vocab_plus_input_size
Issue 4
because spacy doesn't support for "en" format, so we must change it into "en_core_web_sm"
don't forget to use this command line for download that module as reference here
python -m spacy download en_core_web_sm
Issue 5
When it doesn't work after install allennlp==0.8.3
from allennlp.modules.elmo import batch_to_ids
and appears errors like this one
TypeError: Highway.forward: return type <class 'torch.Tensor'>is not a<class 'NoneType'>.
So how to solve it, we should changed the version as this reference
pip uninstall overrides
pip install overrides==3.1.0
it's works fine to me..
Issue 6
there's an errors about several steps from this references that from file in /util/nlp_utils.py in line 225 because we have mentioned that GLOVE as our model with keyedvectors from glove to word2vec as mentioned in Issue 1
if token in GLOVE.vocab:
token_in_glove = token
elif token.lower() in GLOVE.vocab:
token_in_glove = token.lower()
to be
if token in GLOVE.key_to_index:
token_in_glove = token
elif token.lower() in GLOVE.key_to_index:
token_in_glove = token.lower()
Issue 7
I've got so many issue when I'm using the run_glue.py
For my best solution, we don't take risk about local_rank,
because of that, I can't use the CUDA for only one channel..
So I have been inactivate all the syntax that cause of problems..
and also about config, tokenizer, and model from pretrained data.
I recommed you to download the pretrained data from internet, because
When we are not able to download the model from offline, This is the best solution so far..
we should consider to use proper syntax like this:
MODEL_CLASSES = {
'bert': (BertConfig, BertForSequenceClassification, BertTokenizer, BertModel, 'bert-base-uncased'),
'xlnet': (XLNetConfig, XLNetForSequenceClassification, XLNetTokenizer, XLNetModel, 'xlnet-base-cased'),
'xlm': (XLMConfig, XLMForSequenceClassification, XLMTokenizer, XLMModel, 'xlm-mlm-enfr-1024'),
}
....
config_class, model_sequence_class, tokenizer_class, model_class, model_name = MODEL_CLASSES[args.model_type]
...
config = config_class.from_pretrained(model_name)
tokenizer = tokenizer_class.from_pretrained(model_name)
model = model_sequence_class(config)
or
model = model_class.from_pretrained(model_name)
but there's several error during this training this..
so we must using code like..
Example code to resolve the issue
from torch import nn
from pytorch_transformers import XLNetConfig, XLNetModel
Load the configuration
config = XLNetConfig.from_pretrained("xlnet-base-cased")
Check if config.n_token and config.d_model are positive integers
if config.n_token <= 0 or config.d_model <= 0:
# Modify config.n_token and config.d_model to be positive integers
config.n_token = 32000
config.d_model = 768
Create the XLNet model
model = XLNetModel(config)
After I've done with this syntax, no more conflict after that..
because if we are looking from offline pretrained, sometimes it have several errors..
Issue 8
Because I can't use several metada that I can't find it..
please give me proper metadata link, that I can download for now..
here's several link.. that I can use so far,
- SQUAD dataset here
- download_glue_data here
- wiki datasets here
Issue 9
When I debug the run_glue.py, there are several errors appears like:
- There's an error an unexpected keyword argument 'labels', because we must to erase the labels variable into this syntax
inputs = {'input_ids': batch[0],
'attention_mask': batch[1],
'token_type_ids': batch[2] if args.model_type in ['bert', 'xlnet'] else None} # XLM don't use segment_ids
outputs = model(**inputs)
loss = outputs[0] # model outputs are always tuple in pytorch-transformers (see doc)
The error message is indicating that the forward method of your model is being passed an unexpected keyword argument labels. This argument is likely being passed from the inputs dictionary, but it is not being handled correctly in the forward method.
To solve this, you need to make sure that the forward method has the necessary arguments to handle all the values in the inputs dictionary. If labels is not a necessary argument, you should remove it from the inputs dictionary before passing it to the forward method. If labels is a necessary argument, you need to add it to the method signature of the forward method.
This error occurs because the argument labels is not expected by the forward method of the model. The error message states "forward() got an unexpected keyword argument 'labels'".
You should remove the labels argument from the inputs dictionary and the corresponding calculation of the loss function.
and also change variable outputs if we have errors like
raise RuntimeError("grad can be implicitly created only for scalar outputs")
so we change syntax like this:
loss = outputs[0].mean()
The error message "grad can be implicitly created only for scalar outputs" means that the loss should be a scalar tensor, but it seems to be a tensor of shape (batch_size,), which is not a scalar tensor. You'll need to reduce the tensor to a scalar by taking the mean over the batch_size.
After that, if we meet the error about this
AttributeError: 'tuple' object has no attribute 'detach'
so we must to change the variable into like this:
outputs = model(**inputs)
logits = outputs[0]
preds = logits.detach().cpu().numpy()
The error message suggests that the output from the model is a tuple, but the .detach() method is only supported for tensors, not tuples. To fix this, you need to extract the tensor component from the tuple that you want to use for computing the preds.
Note that the specific index used for outputs[0] may vary depending on the structure of the model output, so you may need to modify this based on your specific model.
Expected behavior
I would expect the tests to pass.
Environment
- OS: Ubuntu 18.04.3 LTS
- Python version: 3.7
- PyTorch version: 1.8.0 + CUDA using this pip installation:
pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
- PyTorch Transformers version (or branch): 2.0.0
- Using GPU ? Yes, channel 1 "cuda:1"
- Distributed of parallel setup ? Yes, But I have been changed into single GPU
- Any other relevant information: System is completely clean before pip installs
Additional context
Please let me know if there is any more information I can provide.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent