Dicos is an approach for discovering insecure code snippets in Stack Overflow. Principles and experimental results are discussed in our paper, which will be published at the Annual Computer Security Applications Conference (ACSAC 2021).
※ Exception handling parts may be insufficient due to many modifications in the code refactoring process; we will improve them.
-
Linux: Dicos is designed to work on any of the operating systems. However, currently, this repository only focuses on the Linux environment.
-
Git
-
Python 3
-
Universal-ctags: for function parsing.
-
Joern: for generate code property graph. (only applied in C/C++)
-
cpgqls-client: for communicating with an instance of a Code Property Graph server. (only applied in C/C++)
- How to install cpgqls-client:
pip3 install cpgqls-client
- How to setting Joern server:
- Installation: refer to Joern document
- After following the installer instructions, by default, joern will be installed at
~/bin/joern
. - Create a sh file in
~/bin/joern/joern-cli
. (filename:joern_running.sh
)./joern --server
- How to install cpgqls-client:
Our utilized versions: Python 3.9.1, and universal-ctags p5.9.20210620.0 on Ubuntu 18.04.
- Google Bigquery: Dicos requires post information of Stack Overflow. In the paper, Dicos uses SOTorrent dataset (version "2020-12-31") and obtained information using SQL query.
- We recommend 24 GB RAM.
※ If you have problems related to path information, try testing with absolute paths.
-
Extract posts information through queries from the Bigquery, SOTorrent dataset.
- collecting_comments.sql: collect post comments including security-related keywords. (Output sample :comment_Sample.json)
- collecting_allhistoryPost.sql: collect post code snippets and post descriptions from all history of each post.(Output sample :post_Sample.json)
-
Store all result json files at "src/bigquery_json"
- Note that the result json file name of collecting_comments.sql query to begin with "comment_". (Please refer to the sample file)
- Extract code snippets from the oldest and the latest versions of all posts.
- Execute PostCodeSnippet_Collector.py
python3 PostCodeSnippet_Collector.py
- Check the outputs (description based on the default paths).
- ./dataset/code_raw/: Directory for storing the latest and the oldest versions of code snippets for each post.
- Check the similarity of all code snippets between two versions and diffs in the order of high similarity.
- Execute Pairing_CodeSnippet.py
python3 Pairing_CodeSnippet.py
- Check the outputs (description based on the default paths).
- ./dataset/code_pairs/: Directory for storing two paired code snippets and diff files.
- Two ways to analyze the Control-Flow change, one of Dicos detection features.
- Execute Post_Analyzer.py.
python3 Post_Analyzer.py
- Execute Post_Analyzer_usingJoern.py.
python3 Post_Analyzer_usingJoern.py
- Specify the path regarding Joern to suit the users. (Please change the global variables)
- ./ouput/: Directory for storing results.
- ./ouput/insecure_posts.txt: List of insecure post IDs. (two or more features detected.)
- ./ouput/Analyzing_codesnippet.json: Result of the code snippets analysis.
- ./ouput/Analyzing_keyword.json: Result of the post descriptions and comments analysis.
This repository is authored and maintained by Hyunji Hong. For reporting bugs, you can submit an issue to the GitHub repository or send me an email ([email protected]).