nasa / scrub

SCRUB is a platform for orchestration and aggregation of static code analysis tools.

SCRUB

SCRUB is an orchestration and aggregation platform for static code analysis tools.

SCRUB allows users to run multiple static code analysis tools, collect the results, and export them to external tools. The results from each analyzer are post-processed to a standard warning format that provides information about the location of the warning as well as a brief description of the warning. These warnings can then reviewed and assessed by experienced developers to determine their merit.

A full description of the origins of SCRUB, written by the original author (Gerard Holzmann), can be found here.

SCRUB Documentation

scrub's Issues

Collaborator upload fails when no warnings are included

Collaborator target integration generates an empty-ish file for warnings when none are included in the filter. The ccollab integration will error on this file, causing the upload to terminate and the review to be deleted.

Empty SARIF results can generate parsing error

Update the SARIF parser to handle valid SARIF results files that don't contain any static analysis results.

Update gcc parser for duplicate output

Certain types of warnings can generate the same warning on the same line of the same file. The parser should be updated to ensure that duplicate warnings are not reported.

Update Collaborator logging for failures

When Collaborator fails during the upload process (after review creation) the log will provide users with an incorrect path to the log file.

Add debug information for clean process

With large codebases the process of recursively cleaning previous SCRUB artifacts from the repository can take a non-trivial amount of time. Adding debugging information would be a good way to indicate progress.

Transition SCRUB results away from using a hidden directory

Storing SCRUB results in a hidden directory can be misleading for some users. Storing analysis in a non-hidden directory makes it easier to locate SCRUB results and log information.

Pylint analysis fails when executing below source root

If no .py files are located at the source root, pylint analysis fails. Execution should be modified to support this.

Improve results filtering performance

SCRUB results filtering contains many redundancies and needs to be simplified to improve performance of the filtering step.

Add support for JavaScript analysis

Add JavaScript analysis templates for all applicable tools.

Set up action to push build to PyPi on release creation

Create new mechanism for filtering SonarQube results

Using the projects/export_findings does not allow for tailoring the results that are pulled down from the server. Need to determine how results can be filtered before parsing.

Updated micro filtering algorithm to support template analysis

The update to template based analysis is causing issues with the filtering algorithm in the micro filtering logic. This affects the detection of misspellings in micro suppressions and can cause certain suppressions to be erroneously ignored.

Add option to adjust stdout logging level

Users should be able to run SCRUB quietly as a background task. Suppressing stdout logging facilitates this.

Investigate SARIF viewer options

VS Code, CodeSonar, SonarQube, etc.

Disable results distribution by deafult

Distribution of results is closely tied to operation of the legacy GUI. It should be disabled by default and only activated if the legacy GUI is being used.

API calls for SonarQube should use api/projects/export_findings

Making calls to api/issues/search has the potential to miss some findings. The correct API call should be used to ensure that all findings are exported.

Create release checklist

Add support for pylint

Introducing Python as a supported language opens the opportunity to support pylint analysis. This is analogous to compiler analysis for compiled languages.

SARIF parsing error when root is '/'

Path combinations should be performed using built-in python functions, not manually.

Improve module_helper filtering

Currently when the module_helper executes, all of the results are filtered again even if no new results have been generated for certain tools. This can take a long time for larger sets of results. The module_helper should only filter the newly generated results files.

Implement file level warning suppression

Current mechanism for individual warning suppression is to place an inline comment on the source code line where the warning occurs. This doesn't work for file level warnings that are marked as occurring on line 0.

Create flag to exit on any tool failure

Nominal operation is for SCRUB to continue running if a tool fails, but users may want to exit on first failure. There should be a flag to enabled this. (--strict, --fail-fast, etc.)

SonarQube templates need to check for analysis completion

Larger SonarQube analyses have a delay period after analysis is completed before the results may be retrieved from the server. SCRUB should check to make sure the analysis is finished before attempting to retrieve the results.

SonarQube API call only works for sys admin permissions

The API call 'projects/export_findings' only works if the user has sys admin permissions. Need to revert back to the 'issues/search' API call and find another solution for exporting security hotspots.

Update SonarQube templates for max retrieval

SonarQube has a max limit of 10,000 results. Results retrieval should be capped at 20 pages at 500 results per page.

Update exit code for existing working directory

When SCRUB is using a user-defined working directory to store analysis artifacts, it will fail if the user specifies a directory that already exists. This currently generates an exit code of 1, similar to an individual tool failure. This should have a unique exit code (10) for debugging purposes.

Update Coverity to output and parse json

The Coverity json output is versioned and controlled better than the emacs style output.

Update JavaScript analysis templates to support TypeScript as well

Most tools group JavaScript and TypeScript analysis together, SCRUB should support this pattern.

Add performance test case

There should be a set of performance test(s) that have the intent of monitoring SCRUB runtime. This will help to ensure that new features/changes don't significantly affect runtime.

Remove cov-configure step from Coverity analysis

SCRUB operates under the assumption that all tools have been properly configured before SCRUB is executed. The cov-configure command should be executed before SCRUB execution, not during.

Split CodeSonar analysis into 'build' and 'analyze' steps

Currently CodeSonar analysis implicitly performs the codesonar build step as part of the codesonar analyze step. This can be broken into two steps to allow for more flexibility in capturing the build process.

Provide additional information for missing required parameters

There are certain sets of parameters that are required for analysis to be performed. Currently the error message does indicate what parameter is missing. It would be helpful to inform the user which required parameters are missing as part of the error message.

Update error messages for command execution to be more clear

The command execution error messages that are returned to the user are vague and should provide better information to the user.

Attempt automated builds

If no build instructions are provided, SCRUB should attempt common build instructions for the language of choice.

SARIF filepath are improperly normalized in some cases

Set encoding when parsing SARIF files

Encoding should be set to UTF-8 when importing SARIF files for parsing. Parsing issues may occur when not set.

Add default test case

SCRUB should have a default test case that can be executed to ensure that it has been installed properly.

Create GitHub action for CodeQL analysis of SCRUB repo

Run tests on a per tool basis

There should be a testing utility to allow for developers to run all tests associated with a particular tool in order to focus testing activities.

Support for clang compiler results

Fix regex filtering algorithm

The regex filtering functionality currently allows for filtering patterns that can cause files to appear multiple times. This can cause performance issues when uploading to an external system such as Collaborator.

Capture custom configuration options

The documentation should provide details on how users can use custom tool configurations as part of SCRUB execution (Coverity, CodeSonar, etc...)

Deprecate support for Semmle

The Semmle team has indicated that license will not be renewed beyond FY21. The Semmle tool and references should be removed from SCRUb to support this decision. Code associated with Semmle will be stored as archive if needed for future release.

Update CodeSonar templates to allow for SARIF or XML output format

Add support for gbuild compiler level warnings

Update the gbuild parser to capture gbuild warnings in addition to DoubleCheck warnings.

Decorate GitHub pull requests

SCRUB should be able to interface with GitHub to decorate pull requests with output data from SCRUB analysis.

Add backwards compatibility for 'scrub run-tool'

The command 'scrub run-tool' was supported in previously deployed versions of SCRUB. This command should be supported under backwards compatibility.

Add support for SonarQube

Add C/C++ template for SonarQube analysis

Add support for Python analysis