vis-nlp / chartqa Goto Github PK

Thanks for the great work. Is it possible to provide metadata for question type (data retrieval, visual, compositional, etc) and image source for each data instance?

Hi @vis-nlp,

Thank you for the useful benchmark and insightful paper !
Could you point me to a reference implementation for the relaxed accuracy computation ?
In particular I am interested in the logic that allows to discriminate between numeric and non-numeric answer types (based on casting behavior ?)

Have a great day !

Yana

Hi,

Sorry to disturb. I am trying to retrain a chart visual features extractor on other charts datasets. Based on the instruction, it seems we need the train_dict.pickle and val_dict.pickle files which contain the ground-truth mask annotations.

Could you mind sharing your preprocessing codes to obtain these pickle files on ChartQA datasets (or other charts datasets)?

Best

Hi,
thank you for sharing the code and dataset of your interesting research. I am currently doing some research that will also require extracting table from chart images.
In the paper that, you mentioned you extended the chartOCR method to extract the full table output.
I wonder if you can also share this code which will help my current research a lot.
Thank you for your time to consider this request.

There are a few queries that we have, for which your help is needed.

1. In section 4.1, you mentioned that gold data tables are not available therefore you set up the extraction mechanism to get the underlying tables. We have downloaded the dataset from (https://drive.google.com/file/d/17-aqtiq_KJ16PIGOp30W0y6OJNax6SVT/view) which has table annotations present. Are these extracted tables referred to as 'Gold Data Tables' in the paper?

2. If there is a separate set of 'Gold Data Tables' not available from the link mentioned above, can you also share those for reproducibility purposes?

3. And if the extracted tables are the same as the gold data tables, what are the results implying in Table 5 of the paper? How can TaPas predict answers if the tables itself is not provided?

Hi,

Since the checkpoint for VisionTapas is not available, I am trying to train this model from scratch. The code directory does not have any instructions to run the code. It would be great if you can a small readme file with steps to train this model. If you can add the required versions for libraries, that would also be appreciated.

Currently, I am using torch==1.13.0 with CUDA==11.6.2, and transformers==4.24.0. When I run the training script, it seems the fixed_vocab_file is missing. Can you please add how to get that?

I understand you might be short on time, but I would appreciate if you can at least give one of the training commands you used. Currently, my command looks like this:

python train_question_answering.py --train-folder ../data/chartqa/train/ --validation-folder ../data/chartqa/val/ --qa-train-file ../data/chartqa/train/train_augmented.json --qa-val-file ../data/chartqa/val/val_augmented.json --out-dir vision-tapas-model

Hoping for a quick reply.

Thanks,
-- ashim

This paper is a great work.
The bounding boxes annotation of the chart elements will be helpful in training the chart element detection model or an end-to-end ChartQA model.
I wonder when will the annotations be released?

Hello, I have a question regarding the results on pre-trained VLT5 and VisionTapas reported in Table6. How did you do pre-training? Do you just do pre-training on all the other chartQA dataset with the QA objectives and then fine-tuned on ChartQA?

I see that the repo contains a GPL-3.0 license.

Does that applies to the dataset or only to the source code?

As the compute and time requirement for the traning these models is high it would be of great help if you can share a trained model checkpoint on ChartQA data of the following models:
VisionTAPAS, T5 and VLT5