Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, Liang Lin. “GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning”. Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2021). [PDF]
Download GeoQA benchmark: [Dropbox]
python=3.6
allennlp==0.9.0
Document for allennlp
git clone https://github.com/chen-judge/GeoQA.git
cd GeoQA
pip install requirements.txt
Download the data.zip, move it to GeoQA path, and unzip it.
allennlp train config/NGS_Aux.json --include-package NGS_Aux -s save/test
Evaluate your trained model:
allennlp evaluate save/test data/GeoQA2.2/test.pk --include-package NGS_Aux_test --cuda-device 0
Or, you can use our checkpoint NGS_Aux_CKPT.zip, move it to save path, unzip it, and run:
allennlp evaluate save/NGS_Aux_CKPT data/GeoQA2.2/test.pk --include-package NGS_Aux_test --cuda-device 0
The result of our checkpoint should be:
Method | Acc | Angle | Length | Other |
---|---|---|---|---|
NGS-Auxiliary | 60.7 | 72.0 | 47.0 | 44.4 |