joaohenggeler / uc-masters-software-vulnerabilities

This repository contains any code and documents developed for the master thesis "Building and Evaluating Software Vulnerability Datasets" (2020/2021).

Building and Evaluating Software Vulnerability Datasets

This repository contains any code and documents developed for the master thesis "Building and Evaluating Software Vulnerability Datasets" (2020/2021).

Background

Software vulnerabilities can have serious consequences when exploited, such as unauthorized authentication, data losses, and financial losses. Although there exist techniques for detecting these vulnerabilities by analyzing the source code or executing the software, these suffer from both false positives (misidentified vulnerabilities) and false negatives (undetected vulnerabilities). One other way of identifying vulnerabilities is to combine certain source code properties (software metrics) with machine learning techniques. A previous study has shown this to be feasible, although the data that was collected is now out of date. In a similar fashion, security alerts (i.e. potential vulnerabilities) may be found directly by using Static Analysis Tools (SATs), though these also present a high number of false positives.

Contributions

• Implemented an automated process capable of collecting vulnerability metadata from the CVE Details website, retrieving any affected code units (files, functions, classes) from a project's version control system, generating software metrics and security alerts for each one, storing the collected information in a MySQL database, and building robust datasets capable of being fed to machine learning algorithms.

• Built datasets of vulnerable code units for five large open-source C/C++ projects: Mozilla, Linux Kernel, Xen Hypervisor, Apache HTTP Server, and GNU C Library (Glibc).

• Validated the function samples by exploring various machine learning configurations and investigating whether it is possible to detect vulnerable function code in current versions using static data from previous commits.

Publications

• José D'Abruzzo Pereira, João Henggeler Antunes, and Marco Vieira. On Building a Vulnerability Dataset with Static Information from the Source Code. 2021 10th Latin-American Symposium on Dependable Computing (LADC), 2021, pp. 1-2, doi: 10.1109/LADC53747.2021.9672589.

Authors

• João Henggeler Antunes - Student
• José Alexandre D'Abruzzo Pereira - Supervisor
• Marco Vieira - Supervisor

uc-masters-software-vulnerabilities's Issues

Write the Software Metrics section (State of the Art chapter)

Import the complete dataset to the remote machine

Load Cppcheck's output files into the database

Write the Work Plan chapter

Design the structure of the tables in the database that will store the information generated by the SATs

Generate metrics for the Mozilla, Linux Kernel, and Xen projects

Design the flowchart that specifies the process of retrieving information from external websites and storing it in the database

Compile the Mozilla Source Code

Compile the mozilla source code in order to be able to run SATs:
https://developer.mozilla.org/pt-BR/docs/Mozilla/Developer_guide/Intrucoes_Build

We should aim at compiling old versions of the source code, so we can enrich the database with the SAT alerts.
It would be better if we have the code compiled in a Virtual Machine, so we can replicate it easier.

Load Flawfinder's output files into the database

Insert the collected vulnerabilities into the database

Write the Related Work section (State of the Art chapter)

Analyze the advantages and disadvantages of using the information in the CVE Details and MFSA websites

Write the Preliminary Work chapter

Validate the dataset using additional configurations

Investigate the MySQL storage engines used in the original dataset import scripts

The tables that were imported from the original dataset use MyISAM as their storage engine. In MySQL 5.5 and later, the default is InnoDB. Since the Python scripts don't specify an engine when creating any new tables, these will use InnoDB while the tables from the original dataset use MyISAM.

The InnoDB engine provides foreign key referential integrity constraints, while MyISAM does not. If you try to define a foreign key when creating a new table (which will use InnoDB by default), you won't be able to relate it to a column in a table that uses MyISAM. For example, when creating a relationship between the table 'alert' (InnoDB) and 'patches' or 'files_*' (MyISAM).

For more information on MySQL storage engines, refer to this page.

How important is this issue? Should only one type of engine be used (use MyISAM for new tables, or convert any old ones to InnoDB)?

Write the Software Vulnerability Discovery Techniques section (State of the Art chapter)

Generate software metrics using Understand

Create initial README.md page

• The page is from the main repository: https://github.com/joaohenggeler/estagio-software-vulnerabilities-2020-2021
• It should list the main goals of this work
• It should have the list of authors responsible for that

(Check an example here: https://github.com/nmsa/tma-framework)

Fix the foreign key relationship between the 'repository_samples' and 'vulnerabilities' tables in the database

Validate the metrics from the original dataset against the metrics generated by our scripts

Load Understand's file-level metric reports into the database

Write the Methodology and Objectives chapter

Validate the retrieved vulnerabilities and commit hashes from the original dataset against the ones collected by our scripts

Retrieve reported vulnerabilities from online platforms

Implement the automated collection mechanism proposed in the first step of the methodology. The reported vulnerabilities should be collected for the five C/C++ projects.

Find the location of functions and classes in the source code using its AST