Welcome to the homepage of
Compiler testing is an important task for assuring the quality of compilers, but investigating test failures is very time-consuming. This is because many test failures are caused by the same compiler bug (known as bug duplication problem). In this work, we propose a novel technique (called
Figure 1 presents the workflow of
Figure 1: Overview of D3
To evaluate the effectiveness of
Table 1: Overall experimental results
The failing test programs and their coverage can be downloaded from the link, the coverage data of test suites (used in the execution feature extraction) is in the
passing testsuite coverage
folder, the generated passing programs and the corresponding differences extracted from pairs of failing program and passing programs (used in the program feature extraction) can be downloaded from this link
If you don't want to process the row data by youself, we provide the pre-calculated distances in folder distances
.
Unzip the data.zip under your project directory first, then just run the following command to get the result, which will be saved in the results
folder.
python test.py --dataset llvm280 --loop_time 100
Note that --dataset
indicates which data set to test on. The options are 'gcc430', 'gcc440', 'gcc450', and 'llvm280'. --loop_time
indicates the number of times the test is repeated, and the results are averaged. We recommend at least 100 times to avoid random factors.
Here we take the data processing of llvm-2.8.0 as an example.
First download the generated passing programs and the differences and unzip it, then place it in the mutation-data
folder.
To get the three features, namely program features, optimization features, and exexcution features, just run the following three commands.
# Program features
python get-program-features.py
# Optimization features
python get-optimization-features.py
# Execution features
python get-execution-features.py
The required configuration for each feature is in the header of the file, such as get-execution-features.py
. The first few lines of the file look like this:
names = './data/llvm280/names'
wrongat = './data/llvm280/wrongat.txt'
cov_prefix = './data/llvm280/'
ep_file = './passing-testsuite-coverage/testsuite_280_ep.txt'
np_file = './passing-testsuite-coverage/testsuite_280_np.txt'
dis_url = './coverage.npy'
If you want to extend to other data sets, you only need to modify the path here (the required files are provided in our data).
├── distances : the calculated distances
├── figures : figures in the README.md
├── passing-testsuite-coverage : preprocessed coverage information of testsuites
├── results : testing results
├── Config.py : Configurations and dataset descriptions
├── get-execution-features.py : get execution features and save the distance
├── get-program-features.py : get program features and save the distance
├── get-optimization-features.py : get optimization features and save the distance
└── test.py : use the calculated distances to reproduce the results