Coder Social home page Coder Social logo

softwipe's Introduction

SoftWipe

A tool for automatically checking the software quality of a C/C++ program and giving it a score. It is developed as part of my Hiwi project at the Heidelberg Institute for Theoretical Studies (HITS). The background for doing this is the appearent lack of quality in evolutionary biology software as found in this paper: https://academic.oup.com/mbe/article/35/5/1037/4828033

SoftWipe runs various checks to produce a software quality score: it compiles your program using clang and checks for compiler warnings; it activates almost all available warnings here. It runs your program with clang sanitizers activated, which detect memory errors and undefined behavior. It checks for the usage of assertions. It runs the following static code analyzers: cppcheck, clang-tidy, KWStyle, and lizard. Lizard is also used to find your programs average cyclomatic complexity and code duplication.

Installation & Usage

We highly recommend to use the conda package (see below) for a cleaner installation of the dependencies.

SoftWipe was mainly developed on Ubuntu 20.04 LTS. Some of the used code analysis tools might be affected on other OS versions. Clone this repository to download SoftWipe:

git clone https://github.com/adrianzap/softwipe.git

Dependencies

There are some Python package dependencies, which need to be installed before running SoftWipe:

And there are also some tool dependencies, but fear not! For macOS and Debian-based systems, SoftWipe can automatically install the following dependencies for you. Just run SoftWipe (sudo ./softwipe.py - sudo is most likely required for installing the dependencies); if a dependency cannot be found, it will ask you whether it should automatically install it. Note for Mac users: Homebrew must be installed for the automatic installation of dependencies.

The following tools must be installed:

For make-based projects, you also need:

Also, if the program you want to check uses make or CMake as build system, make and CMake must be installed respectively.

Conda

There is a conda package for SoftWipe, which can be installed using:

conda install softwipe -c angtft -c conda-forge

On the first execution of SoftWipe, it will install Infer locally (if Infer is enabled), as there is no conda package for this tool yet. We recommend installing SoftWipe in a separate conda environment, which can be done with:

conda create --name softwipe_env
conda install softwipe -c angtft -c conda-forge -n softwipe_env

Then, before running SoftWipe, you need to activate the environment with:

conda activate softwipe_env

Close the environment again with:

conda deactivate

Basic usage

SoftWipe can be run as follows:

softwipe.py [-c | -C] [-M | -m | -l target [target ...]] [-e EXECUTEFILE] programdir

Where:

-c tells SoftWipe that your program is C, and -C tells SoftWipe that you program is C++.

-M tells SoftWipe to build your program using CMake, -m using make, and -l using clang. The -l option takes as arguments the files to compile.

-e specifies the path to an "execute file" which contains a command line that executes your program.

programdir specifies the root directory of your target program.

Example command line for a CMake-based C++ program:

softwipe.py -CM path/to/program -e path/to/executefile

For more options and further information, run softwipe.py --help.

Docker usage

Docker enables an awesome way to use a out-of-the-box installation by calling

docker run -it --rm -u <USER_ID>:<GROUP_ID> -w /work -v $PWD:/work softwipe/softwipe softwipe.py <SOFTWIPE_ARGS>

where USER_ID/GROUP_ID should be the owner of the host directory to have write permissions within the docker container. The same image can also be used for continuous integration, shown here for the example of a Jenkins pipeline:

stage('Softwipe') {
  agent {
    docker {
      image 'softwipe/softwipe:0.1'
    }
  }
  steps {
    sh 'softwipe.py -CM -e run_softwipe.sh . 2>&1 |tee softwipe_general.txt'
  }
}

Citation

A. Zapletal, D. Höhler, C. Sinz, A. Stamatakis (2021) The SoftWipe tool and benchmark for assessing coding standards adherence of scientific software Sci Rep 11, 10015 (2021). https://doi.org/10.1038/s41598-021-89495-8

softwipe's People

Contributors

adrianzap avatar angtft avatar bernddoser avatar computations avatar dnbaker avatar josephwb avatar stweil avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

softwipe's Issues

kwstyle is not found when named KWStyle

For example on Ubuntu 20.04, KWStyle is written with the first three letters in caps. SoftWipe looks only for kwstyle and therefore cannot find the binary.

unexpected cppcheck score

while testing swarm, I noticed that:

 --- Running: CPPCHECK --- 
Total weighted Cppcheck warning rate: 0.0 (0/6884)
Detailled results have been written into softwipe_cppcheck_results.txt
Cppcheck Score: 9.2/10

cppcheck returns zero warnings and softwipe_cppcheck_results.txt is empty. Nonetheless, the score is 9.2, and not 10.0 has one would expect. Either the warning rate is not really zero, and some warnings are not logged (an independent cppcheck run reports 4 errors), or there is something wrong with the cppcheck score.

softwipe needs a versioning system

I used conda to download softwipe, then ran into bugs, but I was not sure which version I'm running.
Its really hard to know if a bug has been fixed already if there is no version information anywhere :/

Readme should list requirements

There's some dependencies, but fear not! For macOS and Debian-based systems, softwipe can automatically install these dependencies for you. Just run softwipe (sudo ./softwipe.py - sudo is most likely required for installing the dependencies); if a dependency cannot be found, it will ask you whether it should automatically install it.

README.md should indicate that numpy and scipy are required, and need to be installed before running sudo ./softwipe.py. These are not available by default on a fresh Ubuntu 20.04. Also, running sudo ./softwipe.py doesn't install missing dependencies on Ubuntu (Debian-derived), but provides a list of them.

CMake options (via `-O`) chokes on quotes in input

I used
softwipe -C -M -O softwipe_cmake.cmd ../my_repo
and ran into errors when Softwipe ran CMake with my custom arguments from softwipe_cmake.cmd.
The reason was that softwipe_cmake.cmd contained a -DSOME_PATH="/path/here", which had quotes in it.
When running this on the command line, it works fine. When running it via softwipe, the variable SOME_PATH is not defined in CMakeCache.txt.
So something goes wrong when parsing the arguments in the -O flag (which is not documented in the readme :/)

duplicated compilation options (make, C++)

hi, when running softwipe on swarm's source code (C++, make), the reported compilation commands contain several duplicated options. Here is a list of duplicates:

-fno-omit-frame-pointer
-fsanitize=address
-fsanitize-recover=address
-fsanitize=undefined
-g
-Weverything
-Wno-c++11-extensions
-Wno-c++98-compat
-Wno-c++98-compat-pedantic
-Wno-c99-compat
-Wno-newline-eof
-Wno-padded
-Wno-source-uses-openmp

[usability] Issue a warning if the user passes ctest via -e | Add CTest support to SoftWipe

If the user passes the ctest call via -e, SoftWipe uses the runtime sanitizers only on ctest, not on the user program. This does not come with a warning and might therefore hide bugs.

Ideally, SoftWipe could extract the necessary commands from CTest and directly execute them using the runtime sanitizers. And the very least SoftWipe should issue a warning if ctest is passed via -e. A middle ground would be to allow multiple executables to be passed in via -e.

Feature request: allow -x to take a file

I have a lot of directories I want to ignore. Could -x (or -X, maybe?) take a file listing files/directories to ignore? Ideally this would be one per line rather than a comma separated list (like with x). It is just a bit cumbersome to write these anew with each repeated test.

No support for libaries which are using -Werror

If a libary uses -Werror and the additional warning flags introduced by SoftWipe cause the compiler to issue a warning, the compilation and therefor the SoftWipe run will not finish. This issue raises the barrier for new user substantially, as (1) you have to find out what the issue is and (2) you have to manually edit the build process of the 3rd party library you're using. It would be nice if SoftWipe would include some kind of workaround.

updated code quality benchmark results for swarm 3.0

hi @adrianzap would you consider updating swarm's results in your Code Quality Benchmark?

We've released swarm 3.0 a few days ago. The software works the same, but way faster thanks to @lczech and @torognes, so you should be able to run your pipeline with the new swarm without any modification. @torognes also did a fantastic job fixing warnings and modernizing our code base, so we expect an overall better quality value for swarm.

Thanks,

Failed to find compiledb

Installed with:

pip3 install compiledb

Works fine:

compiledb --help
Usage: compiledb [OPTIONS] COMMAND [ARGS]...
...

Not installed in a weird location (i.e., it is in my $PATH):

$ which compiledb
/home/josephwb/.local/bin/compiledb

I'm not sure what the issue here is. Is SoftWipe assuming everything is root (as suggested by #11)?

[feature-request] Enable exclusion of unit tests from duplicate code metric

In unit test, duplicate code does often not point to a problem as
(1) you rarely ever change unit tests
(2) unit test will tell you if you break them
(3) de-duplicating the code often leads to very unreadable code

I think it would be sensible to exclude the unit tests from the duplicate code metric (but still use the other linters on them).

FileNotFoundError: [Errno 2] No such file or directory: 'cd': 'cd'

Hi there!

when trying to run softwipe on EPIK, using the softwipe command file

mkdir -p bin
cd bin
cmake ..
make -j4

(and after fixing some issues related to OpenMP not working under conda...)

I get the following error:

/home/lucas/Software/miniconda/envs/softwipe/bin/softwipe . --cc --cmake --no-execution -f build.txt 
 --- Running: COMPILER --- 
Traceback (most recent call last):
  File "/home/lucas/Software/miniconda/envs/softwipe/bin/softwipe", line 33, in <module>
    sys.exit(load_entry_point('SoftWipe==0.1.1', 'console_scripts', 'softwipe')())
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/site-packages/softwipe.py", line 437, in main
    args, lines_of_code, program_dir_abs, use_cpp, excluded_paths, args.no_execution)
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/site-packages/softwipe.py", line 315, in compile_and_execute_program_with_sanitizers
    weighted_sum_of_compiler_warnings = compile_program(args, lines_of_code, cpp, compiler_flags, excluded_paths)
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/site-packages/softwipe.py", line 251, in compile_program
    additional_args=additional_args)
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/site-packages/compile_phase.py", line 335, in compile_program_cmake
    excluded_paths)
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/site-packages/compile_phase.py", line 267, in parse_make_command_file_and_run_all_commands_in_it
    subprocess.run(split_command, cwd=working_directory, stdout=subprocess.DEVNULL)
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/home/lucas/Software/miniconda/envs/softwipe/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'cd': 'cd'

I think I have specified the command file as described in the help though.

Cheers
Lucas

Softwipe not compiling tool correctly?!

Hey there!

Using this commit of a tool called EPIK, I run into compilation errors, as I described in this issue. These cause the compilation to fail.

However, when compiling with softwipe, none of them are reported, and instead I get a perfect

  Number of level 3 warnings (must be fixed): 0/61743
  Number of level 2 warnings (should be fixed): 0/61743
  Number of level 1 warnings (could be fixed): 0/61743

which seems weird to me. I think that softwipe is not compiling EPIK correctly. Hence, I am not sure that I can trust the reported score :-(

Cheers
Lucas

wiki: wrong link

In the wiki, in the section:

Contents

    Installation
    Usage
    Code Quality Benchmark

Code Quality Benchmark links to the wrong page (https://github.com/adrianzap/softwipe/wiki/Code-score-comparison). It should link to https://github.com/adrianzap/softwipe/wiki/Code-Quality-Benchmark.

links for vsearch and swarm

Hi,

on the Code Quality Benchmark wiki page, the swarm and vsearch entries are not correctly formatted (wrong markdown).

I suggest:

[paper](https://www.ncbi.nlm.nih.gov/pubmed/26713226) [github](https://github.com/torognes/swarm)
[paper](https://www.ncbi.nlm.nih.gov/pubmed/27781170) [github](https://github.com/torognes/vsearch)

Thanks,

Softwipe Conda package is missing dependencies

If the software project under test using using link-time-optimization, e.g.
here for CMake-based projects like so:

set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE) # Enable IPO (LTO) by default

, then softwipe will crash with

/usr/bin/ld: /home/mi/bielow/miniconda3/envs/softwipe_env/bin/../lib/LLVMgold.so: error loading plugin: /home/mi/bielow/miniconda3/envs/softwipe_env/bin/../lib/LLVMgold.so: cannot open Shared-Object-file: file or directory not found

since the LLVMgold.so is not shipped with the Conda package.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.