hibb-bb / lattice

Lattice

Lattice

Code for our paper Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning at NAACL 2022. If you find the code useful, please cite the following paper.

@inproceedings{wang2022robust,
  title={Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning},
  author={Wang, Fei and Xu, Zhewei and Szekely, Pedro and Chen, Muhao},
  booktitle={Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year={2022}
}

Quick Links

• Overview
• Requirements
• Train
• Robustness Evaluation

Overview

Previous table-to-text generation methods suffer from the loss of structural information and are brittle to table layout change. Lattice alters Transformer with a structure-aware self-attention mechanism, and a tranformation-invariant positional encoding mechanism to address the aforementioned problems.

Requirements

pip install -r requirements.txt

Train

You can train your own model following these steps.

1. Download ToTTo dataset

 wget https://storage.googleapis.com/totto-public/totto_data.zip
 unzip totto_data.zip

2. Preprocess data

python preprocess/preprocess_data.py --input_path="totto_data/totto_dev_data.jsonl" --output_path="totto_data/dev_linearized.jsonl"
python preprocess/json_to_csv.py -i totto_data/dev_linearized.jsonl -o totto_data/dev.csv

python preprocess/preprocess_data.py --input_path="totto_data/totto_train_data.jsonl" --output_path="totto_data/train_linearized.jsonl"
python preprocess/json_to_csv.py -i totto_data/train_linearized.jsonl -o totto_data/train.csv

3. Train model

bash run.sh

After running the script, you will get trained model weights, training logs, and results in the output folder, including the predicted summaries in generated_predictions.txt.

Note that the evaluation process for each checkpoint during training are simplified for efficiency, so the BLEU scores are lower than the final evaluation results. You can change the setting according to this post. To get accurate scores, please use the official evaluation tool. You can also submit your predictions to the official leaderboard through this form.

Robustness Evaluation

To get the hard version of ToTTo introduced in section 3.3 of our paper, a simple way is to perturb the row/column indexes of highlighted_cell at the end of the get_highlighted_subtable function. We apply the following three content-neutral table transformations:

• row shuffling -- map row indexes to a different permutation;
• column shuffling -- map column indexes to a different permutation;
• table transposing -- swap row and column indexes.