bhavyagera10 / code_summarization_public

source code for 'Improving automatic source code summarization via deep reinforcement learning'

Requirement

This repos is developed based on the environment of:

• Python 2.7
• PyTorch 0.2

Data folder structure

/media/BACKUP/ghproj_d/code_summarization/github-python/ is the folder to save all the data in this project, please replace it to your own folder. The data files are organized as follows in my computer:

|- /media/BACKUP/ghproj_d/code_summarization/github-python

|--original (used to save the raw data)

|----data_ps.declbodies data_ps.descriptions

|--processed (used to save the preprocessed data)

|----all.code all.comment

|--result (used to save the results)

|--train (get the data files before training)

You need to get these files before you starting to train our model. Here I put the original folder in the dataset foler of this project. You'd better copy them to your own folder.

Data preprocess

cd script/github

python python_process.py -train_portion 0.6 -dev_portion 0.2 > log.python_process

Training

Back to the projector folder

cd ../..

Get the data for training

python run.py preprocess

Training

python run.py train_a2c 10 30 10 hybrid 1 0

Testing

python run.py test_a2c hybrid 1 0

TODO

• To build the AST, on the data preprocessing, I parse the AST into a json and then parse the json into AST on training. This kind of approach is not elegant.
• On training, I don't know how to batchify the ASTs, so I have to put the ASTs into a list and encode them one by one. It's unefficient, making the training of one epoch takes about 2-3 hours. Please let me know if you have a better way to accelerate this process.
• On the encoder side, I am working on applying Tree-CNN and GraphCNN to represent the code in a better way.
• On the decoder side, GAN network will also be considered for the code summarization task.

Acknowledgement

This repos is based on https://github.com/khanhptnk/bandit-nmt

Please cite our paper if you use this repos.

Bibtex:
@Inproceedings{wan2018improving,
title={Improving automatic source code summarization via deep reinforcement learning},
author={Wan, Yao and Zhao, Zhou and Yang, Min and Xu, Guandong and Ying, Haochao and Wu, Jian and Yu, Philip S},
booktitle={Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering}
pages={397--407},
year={2018},
organization={ACM}
}