tarun-bl / geoqa

Official Implement of ACL 2021 paper “GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning”.

GeoQA

Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, Liang Lin. “GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning”. Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2021). [PDF]

Download GeoQA benchmark: [Dropbox]

Environment

python=3.6

allennlp==0.9.0

Document for allennlp

Usage of NGS-Auxiliary

Preparing

git clone https://github.com/chen-judge/GeoQA.git

cd GeoQA

pip install requirements.txt

Download the data.zip, move it to GeoQA path, and unzip it.

Training

allennlp train config/NGS_Aux.json --include-package NGS_Aux -s save/test

Evaluation

Evaluate your trained model:

allennlp evaluate save/test  data/GeoQA2.2/test.pk --include-package NGS_Aux_test --cuda-device 0

Or, you can use our checkpoint NGS_Aux_CKPT.zip, move it to save path, unzip it, and run:

allennlp evaluate save/NGS_Aux_CKPT  data/GeoQA2.2/test.pk --include-package NGS_Aux_test --cuda-device 0

The result of our checkpoint should be:

Method	Acc	Angle	Length	Other
NGS-Auxiliary	60.7	72.0	47.0	44.4