alexyangli / tf_cfo

Tensorflow implementation of CFO, a knowledge graph based QA system

tf_CFO

Tensorflow implementation of CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases

Requirement

• python >= 3.5
• tensorflow >= 1.4
• pickle
• numpy
• gensim
• nltk
• fuzzywuzzy

Preprocessing

Goto data_preprocess directory and run the scripts (might take some time):

1. fetch dataset

sh fetch.sh

Update: For those who are blocked, here's the link of BaiduYunPan to download the dataset: https://pan.baidu.com/s/1RBFQHuQDHuuESu4fGkmeMw (password: 6mpi)

2. preprocess data

sh preprocess.sh

Training

• The QA system consists of 3 components: entity detection, relation network and subject network.
• Goto corresponding directory and refer to README.md to finish training and testing.

Inference

Goto inference folder and refer to README.md to get test result.

Reference

• https://github.com/castorini/BuboQA
• https://github.com/ZichaoHuang/TransE
• https://github.com/zihangdai/cfo

Download processed data

Processed data are also available, feel free to download: https://pan.baidu.com/s/1M_674aselMx8LtLagr0AcQ (password：azen)

tf_cfo's Issues

MemoryError

What is your memory size?

My cofiguration: 64bits + 16GB.

store knowledge triples in pickle format...
Traceback (most recent call last):
File "create_kg.py", line 138, in
fetch_triple(sys.argv[1], sys.argv[2], sys.argv[3])
File "create_kg.py", line 111, in fetch_triple
pickle_save(triples, os.path.join(out_dir, basename+'_triple.pkl'))
File "/home/iii/tf_CFO/data_preprocess/utils.py", line 67, in pickle_save
pickle.dump(obj, writer)
MemoryError

hello, i find the net can not reach when i run the command sh fetch.sh

hello, i find the net can not reach when i run the command sh fetch.sh,and my internet is normal,so could you check the source ? it will be very thanking for receiving your answer