owasp / glue Goto Github PK

Application Security Automation

License: Other

HTML 4.37% Ruby 90.49% DIGITAL Command Language 0.03% Shell 2.40% Groovy 0.18% JavaScript 0.16% Dockerfile 2.37%

ci-cd devsecops tool owasp

glue's Introduction

OWASP Glue

Glue is a framework for running a series of tools. Generally, it is intended as a backbone for automating a security analysis pipeline of tools.

Recommended Usage

For those wishing to run Glue, we recommend using the docker image because it should have the other tools it uses available already and configured. See the documentation for more info. Glue Docker Documentation

For those interested in how to use Glue in a DevOps context, see Glue DevOps Integration Options

Checkout the Playground to get a better understanding of Glue's features and how you can use them.

Local run

docker run owasp/glue

Installation for Development

git clone https://github.com/owasp/glue
cd glue                     -- RVM will set to 2.3.1 with Gemset Glue
gem install bundler
bundle install

Running in Development

cd lib
../bin/glue -h

Extending Glue

Glue is intended to be extended through added "tasks". To add a new tool, copy an existing task and tweak to make it work for the tool in question.

Usage

Glue <options> <target>

Options

Common options include:

-d for debug
-f for format (takes "json", "csv", "jira")

For a full list of options, use Glue --help or see the OPTIONS.md file.

Target

The target can be:

Filesystem (which is analyzed in place)
Git repo (which is cloned for analysis)
Other types of images (.iso, docker, etc. are experimental)

Dependencies

clamav
hashdeep
rm (*nix)
git
mount (*nix)
docker

Development

To run the code, run the following from the root directory:

>ruby bin/Glue <options> target

To build a gem, just run:

gem build Glue.gemspec

Integration

Git Hooks

First, grab the hook from the code.

meditation:hooks mk$ cp /area53/owasp/Glue/hooks/pre-commit .

Then make it executable.

meditation:hooks mk$ chmod +x pre-commit

Make sure the shell you are committing in can see docker.

meditation:hooks mk$ eval "$(docker-machine env default)"

Now go test and make a change and commit a file. The result should be that Glue runs against your code and will not allow commits unless the results are clean. (Which is not necessarily a reasonable expectation)

Configuration files

For advanced usage scenarios, you can save your configuration and use it at runtime.

Authors

Matt Konda Alex Lock Rafa Perez

License

Apache 2: http://www.apache.org/licenses/LICENSE-2.0

glue's People

Contributors

Stargazers

Watchers

Forkers

vasiab pombredanne 10dot runako ryrizo jjediny jjhauck 3rawkz samyoyo olivierh59500 gibbsjosh tardummy01 lucabongiorni br3ign wflk kaicastledine rajivraj alexxnica kryndex pardotkynes methos2016 gsavastano omerlh mendickxiao amitfun bcfore souravbadami magnologan r3p3r nixworks rbramwell ro9ueadmin sevatecinc shrikant224 reality9 nhajiyani fullstackenviormentss ch4p34un0ir security-prince lepoli mmg1 ajaynegi-websecresearch szliuyujie turcan-vasile xiaolushuo bitwisebytefoolish opt9 k1llm0ng34 hackerup owaspmemphis mzielinski1978 afrovera bobthesecurityguy shridharme danbadds38 danilocaruso jaguasch yjerez rahmiy tdshipley lnaphade ramos-dev securestep9 michaelmworthington greyvish akurtasinski sinsixx yangwangvungle moveaxlab security-lab arthurakay neatnerdprime kniersbach ascdso2020 devopsascode 5l1v3r1 arukaminado ci-cd-security leaderdevops hackerinchief pacificdynamics anigasan kshitij-agrawal2010 miguelfreitas93 darmanex jorson-chen apython123 technology-development yosiat mca-digital paulaalarconstd hartl3y94 umutbb naveen12 anubisant erdal-pb be-secure sumodgeorge ankityadavv2014 ekmixon

glue's Issues

Consider support for Bandit (for python projects)

https://github.com/openstack/bandit

installation using launchpad/.deb would be easy but may bloat the image with python dependencies

Add support for find security bugs.

Languages supported by glue

Hi,

What are the languages supported by glue at this time?

Passing authMethodName/authMethodConfigParams to ZAP

Is there a way to pass these arguments to ZAP through glue to authenticate with a website you're testing against?

https://github.com/zaproxy/zaproxy/wiki/FAQformauth

Bare docker image?

Currently, the docker image contains a lot of security tools, which make it pretty big. I think it would be nice, in addition to the existing image, to have another image, with the tag "bare", containing only Glue. This will allow me to build custom Glue images, containing only the tools that I need - which will result in smaller docker images.

Please review your use of stale bot

At the time of writing this project has has 47 issues raised. 23 of them have been closed by the stale bot.
This might keep your open issues down, but I think its a bad decision.
Issues raised are contributions, even if they arent actually improvements - they are suggestions as to how the project could be improved.
They are an indication to other people that an improvement has been suggested and they can indicate that they also want this enhancement or even actually work on it.
OK, so you probably dont want the ~600 open issues that we have on https://github.com/zaproxy/zaproxy/issues ;) We do close stale issues, but its a manual process and its much more conservative (possibly too conservative). But that means that we do have bugs that are fixed after 2 years (eg zaproxy/zaproxy#765) - I'd much rather have this than have a bug or feature request forgotten about.
I think that the aggressive closing of issues sends a signal that enhancement requests are not really valued.
I'm expecting this issue to be closed by stale bot :P

Managing Secrets

Add support for pulling config / secrets from tiers of locations:
0. Defaults

CLI
ENV
Vault

Basically, when run, set defaults, apply CLI options, then read from ENV, then pull from Vault if configured.

The purpose is to make it so that it can all work without putting secrets in stored CLI in Jenkins (for example).

Task that exit with error should set the exit code

Task should be able to report a fatal error by setting the exit code. Currently I tried Glue.error, but it only reports the error and not setting the exit code to 1. I think that if task encounter fatal error, it should be detectable by the CI tool and the build should fail.

Document and update JIRA integration.

Add tools to process chef config

Add S3 mounter.

Make it easy to use Glue to run AV and DLP (hashdeep) against an S3 bucket.

Checkmarx results filter

Hey
First, really great project, thank you!
I noticed you already have Checkmarx integration, and I thought I could share from the experience I have with Checkmarx. We are using it in our CI process, and I did some work to filter the results from Checkmarx. This is what I did so far:

In Checkmarx's portal, you can mark the state of the finding - this is used mainly to mark false positives finding. The relevant field in the XML is state.
Another interesting feature of Checkmarx: It is keeping the state of the finding, and report if it a new or old finding. The relevant field in the XML is Status. This is a bit tricky though - for example, I would like to fail the build also on existing results, but I want to receive Slack notification only about new findings (I know that currently there is no Slack integration, but I think the example is still valid).

To conclude, I think that the first thing I mentioned (the state) should be part of the Checkmarx task. For the other 2, I think it should be controlled by the user, although I am still not sure how. I would love to hear your opinion about that.
Thanks,
Omer

add support for Snyk

http://snyk.io

Write a queue and worker flow.

Consider building a simple front end that could allow people to create jobs through a UI, then put them in a queue, then let Pipeline pull from the queue.

Add tools to mount AMI.

Write a Filter for ZAP findings to consolidate to 1 / site for http header related items.

Add support for Dawnscanner

I'm in a conflict of interest here, close this one if not appropriated.
It might be a good idea to add support for dawnscanner ruby static analysis tool I wrote also. The main difference with brakeman is that dawnscanner, is MVC framework agnostic so it works also for Sinatra, Padrino, pure rack applications and Lotus in the future.

Could be a great way to improve the tool and improve the pipeline as well.

Test for fingerprinting headers

There are a few fingerprinting headers, for example:

X-Powered-By
X-AspNet-Version
X-AspNetMvc-Version

That expose information about the server. We can easily add a test for those headers using curl and grep, and report (if found) on those headers.

Add support for gauntlt.

Consider looking at S3 bucket contents and catching change over time.

Getting Segmentation fault error

owasp-pipeline-0.8.7/lib/pipeline/util.rb:13: [BUG] Segmentation fault at 0x00000000000000
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0003 p:---- s:0008 e:000007 CFUNC :gets
c:0002 p:0036 s:0005 e:000004 BLOCK /usr/local/rvm/gems/ruby-2.3.0/gems/owasp-pipeline-0.8.7/lib/pipeline/util.rb:13 [FINISH]
c:0001 p:---- s:0002 e:000001 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
/usr/local/rvm/gems/ruby-2.3.0/gems/owasp-pipeline-0.8.7/lib/pipeline/util.rb:13:in block (2 levels) in runsystem' /usr/local/rvm/gems/ruby-2.3.0/gems/owasp-pipeline-0.8.7/lib/pipeline/util.rb:13:in gets'

-- Machine register context ------------------------------------------------
RIP: 0x00007f9c8fed6e21 RBP: 0x00000000000000b0 RSP: 0x00007f9c3827ffc0
RAX: 0x00007f9c3828004f RBX: 0x00007f9c2c000020 RCX: 0x0000000000000007
RDX: 0x00000000000000b0 RDI: 0x00007f9c2c000020 RSI: 0x00000000000136a0
R8: 0x0000000000000000 R9: 0x6f6e203a766e652f R10: 0x0000000000000000
R11: 0x0000000000000206 R12: 0x00007f9c2c47cb00 R13: 0x000000000000000b
R14: 0x00007f9c2c000078 R15: 0x0000000000002710 EFL: 0x0000000000010283

-- C level backtrace information -------------------------------------------

System configuration:
ubuntu 14.04, 4 core ,16GB RAM,Ruby 2.3.0,Rails 4.2.5

Add support for Slack Reporter

Could be nice to have the reports sent directly to Slack.

Integrate with ZAP via API.

Define process for doing CI builds

Produce:

ruby gem
docker image
GitHub release

Write a reporter for github issues. Include filter for existing issues to prevent dupes.

Filter to remove duplicate dependency findings.

Some tools retire, nodesecurityproject and owasp-dependency-check and bundler-audit for example produce potentially overlapping findings. Write a filter to unify those.

CSV Reports Don't Escape Properly

Some of the columns in the CSV contain commas, which makes CSV a fairly inappropriate choice for output format.

Suggestion: consider removing CSV support.

Add support for PMD.

Document and update github issues integration.

Update Checkmarx section under Tools.md

Add tools to process ansible config

Create hubot for interacting with a pipeline instance.

Allow to choose which tasks to run

Currently, glue tries to run all the tasks, and if the tool exists the task run. It has a few problems:

Build will not fail if there is an issue with the tool
Running irrelevant tools
It would be nice to be able to specify which tasks to run.

Ignoring results

I want to be able to mark results as irrelevant, mainly for false positive. Also, the ability to postpone results should be nice. I am thinking about something like Snyk interactive CLI that iterate over the results and let you decide what to do about them.
I am thinking to implement this using file or DB, but I am open for ideas about that.

Document and Test Jenkins Integration (Plugin?)

Question: How is this different then OWASP OWTF

How is this project different then OWASP OWTF.

OWTF also is automating web testing tools and i suppose pipeline is also doing the same. Please carify.

Add support for FindBugs + FindSecurityBugs

I'll add support for findbugs w/ findsecuritybugs plugin next week, so marking here as a reference.

Unless anyone has a better idea, it will likely be a "maven projects only" situation since findbugs will only scan bytecode and the only way I can think of to reliably build an unknown java app would be 'mvn compile'.

Consider Sample Integration with Drone.io Pipeline

https://github.com/drone/drone
Drone is a CI/CD solution that aims to be a lightweight replacement for jenkins. It is written in Golang uses Docker containers to run builds. Builds are run based on a single instruction file in drone.yml

v0.5 of drone.io includes the concept of a pipeline which expands drones support of services which are docker containers that are accessable to the build localhost, samples include a selenium/selenium grid which could be used to proxy auth for OWASP zap, carry out any future Gauntlt tasks, or provide a storage service to upload the output of the tests to a non-emphemeral external service like s3 or DB

Use nmap to test for weak ssl ciphers

Hey
NMap can be used to test for weak ssl ciphers using:
nmap --script ssl-cert,ssl-enum-ciphers -p 443 -oX output.xml host
It will be nice to run it from glue and process the XML so Glue will be able to report on weak ciphers (nmap rank them, so we can report on each cipher below A grade, and have the minimum grade configurable).
I've created a little script that does exactly that - you can find the code here. It is in Ruby, so it will be very easy to move the code into Glue.

Add dummy task

Create a simple dummy task, to make it easier to play with Glue and understand it's features. The task should generate some findings, with different levels.

Glue retrieved all alerts from Zap, not only the one related to the target URL

I am using:

ruby bin/glue -t zap --zap-host http://localhost --zap-port 1234 --zap-passive-mode -f text --exit-on-warn 0 http://localhost:4004 --finding-file-path glue.json

To run Glue with Zap task, and it returns irrelevant results, for example:

Detail:  Url: http://localhost:8000/login Param: X-Content-Type-Options 
Reference: http://msdn.microsoft.com/en-us/library/ie/gg622941%28v=vs.85%29.aspx
https://www.owasp.org/index.php/List_of_useful_HTTP_headers
Solution: Ensure that the application/web server sets the Content-Type header appropriately, and that it sets the X-Content-Type-Options header to 'nosniff' for all web pages.
If possible, ensure that the end user uses a standards-compliant and modern web browser that does not perform MIME-sniffing at all, or that can be directed by the web application/web server to not perform MIME-sniffing.
CWE: 16	WASCID: 15
	** Consolidated ** - Potentially identified in > 1 spot.

	Description: Web Browser XSS Protection is not enabled, or is disabled by the configuration of the 'X-XSS-Protection' HTTP response header on the web server

	Timestamp: 2017-11-01 11:28:53 +0200

	Source: ZAPhttp://localhost:8000/login

	Severity: 1

	Fingerprint:  ZAPhttp://localhost:8000/loginWeb Browser XSS Protection Not EnabledX-XSS-Protection

	Found by:  Zap

Which is on a different URL. The bug is in Glue code - I've tested manually Zap API and it returns only the relevant results.