seominjoon / piqa Goto Github PK
View Code? Open in Web Editor NEWPhrase-Indexed Question Answering (PIQA)
Home Page: https://pi-qa.com
License: Apache License 2.0
Phrase-Indexed Question Answering (PIQA)
Home Page: https://pi-qa.com
License: Apache License 2.0
Collate fn uses self._elmo to check the use of elmo, but the cached Processor's _elmo is set to False.
(Cache was saved when processing SA+Elmo)
Error log is as follows:
$ python main.py baseline --cuda --mode embed_question --iteration 501 --test_path $SQUAD_DEV_QUESTION_PATH --elmo --num_heads 2 --batch_size 32 --cache
...
'train_path': '/home/jinhyuk/data/squad/train-v1.1.json',
'train_steps': 0,
'word_vocab_size': 10000}
Model loaded from /tmp/piqa/squad/save/501/model.pt
Saving embeddings
Traceback (most recent call last):
File "main.py", line 277, in <module>
main()
File "main.py", line 258, in main
embed(args)
File "main.py", line 240, in embed
question_output = model.get_question(**test_batch)
File "/home/jinhyuk/github/piqa/squad/baseline/model.py", line 285, in get_question
q = self.question_embedding(question_char_idxs, question_glove_idxs, question_word_idxs, ex=question_elmo_idxs)
File "/home/jinhyuk/anaconda3/envs/p3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/jinhyuk/github/piqa/squad/baseline/model.py", line 98, in forward
elmo, = self.elmo(ex)['elmo_representations']
File "/home/jinhyuk/anaconda3/envs/p3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/jinhyuk/anaconda3/envs/p3.6/lib/python3.6/site-packages/allennlp/modules/elmo.py", line 133, in forward
original_shape = inputs.size()
AttributeError: 'NoneType' object has no attribute 'size'
$ python piqa_evaluate.py $SQUAD_DEV_PATH /tmp/piqa/context_emb/ /tmp/piqa/question_emb/
Traceback (most recent call last):
File "piqa_evaluate.py", line 151, in
progress=args.progress)
File "piqa_evaluate.py", line 123, in get_predictions
m = sim.max(1)
File "/home/jinhyuk/anaconda3/envs/p3.6/lib/python3.6/site-packages/numpy/core/_methods.py", line 28, in _amax
return umr_maximum(a, axis, None, out, keepdims, initial)
numpy.core._internal.AxisError: axis 1 is out of bounds for array of dimension 1
Changing codes from
119 else:
120 q_emb = q_emb['arr_0']
121 c_emb = c_emb['arr_0']
122 m = np.matmul(c_emb, q_emb.T)
123 # m = sim.max(1)
124
125 argmax = m.argmax(0)
simply solves the problem, but don't know why the original code was in that form.
q_emb.shape = (1024,), c_emb.shape = (1008, 1024) leads to sim.shape = (1008),
and just taking argmax(0) seems fine.
Hey,
The arxiv paper links to this repo for the leaderboard, yet none is available. Have no valid submissions been received yet?
Thanks
Suggest using Conda for faiss installation.
Maybe we need to remove faiss from requirements.txt.
Docker image works fine.
This did not happen previously, so I am currently investigating the issue.
Error occurs if nltk.download('punkt') was not called.
Maybe edit README.md or download.sh ?
Performances of two evaluation scripts differ as follows:
$ python evaluate.py $SQUAD_DEV_PATH /tmp/piqa/pred.json
{"exact_match": 52.81929990539262, "f1": 63.28879733489547}
$ python piqa_evaluate.py $SQUAD_DEV_PATH /tmp/piqa/context_emb/ /tmp/piqa/question_emb/
{"exact_match": 52.28949858088931, "f1": 62.72236634535493}
Difference is about 0.5~0.6, and tested model is LSTM+SA+ELMo.
Training using
$ python main.py baseline --cuda
and testing with
$ python main.py baseline --cuda --mode test --iteration 501
makes an error as follows:
...
Model loaded from /tmp/piqa/squad/save/501/model.pt
Traceback (most recent call last):
File "main.py", line 257, in <module>
main()
File "main.py", line 236, in main
test(args)
File "main.py", line 162, in test
test_dataset = tuple(processor.preprocess(example) for example in test_examples)
File "main.py", line 162, in <genexpr>
test_dataset = tuple(processor.preprocess(example) for example in test_examples)
File "/home/jinhyuk/github/piqa/squad/baseline/processor.py", line 124, in preprocess
context_word_idxs = tuple(map(self._word2idx, context_words))
File "/home/jinhyuk/github/piqa/squad/baseline/processor.py", line 276, in _word2idx
return self._word2idx_dict[word] if word in self._word2idx_dict else 1
TypeError: argument of type 'method' is not iterable
Code version: fa113ab
Error occurs due to no initialized model.elmo when loading the models with elmo.
model.init(metadata) should be called in test(), embed() functions like in train() function.
Hi,
Thanks for uploading the code!
It seems that in model.py you are maintaining the scale and mixing parameters for computing the averaged Elmo representation. However, from my understanding of allennlp.modules.elmo.Elmo
this should be already taken care of by the Elmo
class.
Is there any reason for re-mixing the already mixed Elmo representations returned? Or am I misunderstanding something?
Thanks!
Bhuwan
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.