Coder Social home page Coder Social logo

depthsecurity / armory Goto Github PK

View Code? Open in Web Editor NEW
414.0 20.0 71.0 1.28 MB

Armory is a tool meant to take in a lot of external and discovery data from a lot of tools, add it to a database and correlate all of related information.

License: GNU General Public License v3.0

Python 83.37% HTML 16.14% CSS 0.49%
hacktoberfest

armory's Introduction

       _                                                          
      dM.                                                         
     ,MMb                                                         
     d'YM.   ___  __ ___  __    __     _____  ___  __ ____    ___ 
    ,P `Mb   `MM 6MM `MM 6MMb  6MMb   6MMMMMb `MM 6MM `MM(    )M' 
    d'  YM.   MM69 "  MM69 `MM69 `Mb 6M'   `Mb MM69 "  `Mb    d'  
___,P____Mb___MM______MM____MM____MM_MM_____MM_MM_______YM.__,P___
   d'    YM.  MM      MM    MM    MM MM     MM MM        MM  M    \ 
__,MMMMMMMMb__MM______MM____MM____MM_MM_____MM_MM________`Mbd'_____\
  d'      YM. MM      MM    MM    MM YM.   ,M9 MM         YMP      
_dM_     _dMM_MM_    _MM_  _MM_  _MM_ YMMMMM9 _MM_         M      
                                                          d'      
                                                      (8),P       
                                                       YMM        
                                                          

Description

This is somewhat stable now, but is still evolving. The original 'master' branch is now armory1

Armory is a tool meant to take in a lot of external and discovery data from a lot of tools, add it to a database and correlate all of related information. It isn't meant to replace any specific tool. It is meant to take the output from various tools, and use it to feed other tools.

Additionally, it is meant to be easily extendable. Don't see a module for your favorite tool? Write one up! Want to export data in just the right format for your reporting? Create a new report!

Installation

Prerequisites

First, set up some kind of virtual environment. I like virtualenvwrapper:

http://virtualenvwrapper.readthedocs.io/en/latest/install.html

Actually installing (without Docker)

  1. Clone the repo: git clone https://github.com/depthsecurity/armory

  2. Run: sudo apt install libmariadb-dev

  3. Install requirements: pip install -r requirements.txt

  4. Install the module: python setup.py install

  5. You will want to run armory2 at least once in order to create the default config directory (~/.armory by default with the default settings.py and settings for each of the modules).

  6. Optionally, edit settings.py and modify the base_path option. This should point to the root path you are using for your current project. You probably should change this with every project, so you will always be using a clean database. If you don't want to change the base_path with every new project, you could also simply delete or rename the db.sqlite3 file from your base_path. All files generated by modules will be created in here, as well as the sqlite3 database. By default it will be within the current directory-. For additional information on setings.py, see documentation here.

  7. Finally, run armory2-manage migrate to setup the database.

Actually installing (with Docker)

  1. See https://github.com/depthsecurity/armory-docker

Usage

Usage is split into modules and reports.

Modules

Modules run tools, ingest output, and write it to the database. To see a list of available modules, type:

armory2 -lm

To see a list of module options, type:

armory2 -m <module> -M

Reports

Reports are similar to modules, except they are meant to pull data from the database, and display it in a usable format. To view all of the available reports:

armory2 -lr

To view available report options:

armory2 -r <report> -R

Interactive Shell

There is also an interactive shell which uses IPython as the base and will allow you to run commands or change database values. It can be launched with: armory-shell. By default, the following will be available: Domain, BaseDomains, IPAddresses, CIDRs, Users, Creds, Vulns, Ports, Urls, ScopeCIDRs.

armory's People

Contributors

absolomb avatar awsmhacks avatar bryanks avatar cclauss avatar ccsplit avatar dependabot[bot] avatar fang0654 avatar jk-mayne avatar markoh17 avatar mister-joe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

armory's Issues

[Feature] [Armory2.0] Improved Gowitness Functionality

Within the host summary webapp portion could we provide a way to filter out entries that have no Gowitness data? i.e. if I have 100 hosts but only 5 had a successful Gowitness run I don't want to scroll through all of them.

A solution that would be even more ideal would be to have a dedicated gowitness page that is a display of all of or a portion of the screenshots (A page that tries to load 1000 screenshots probably isn't ideal). That way you can glance at one page and see interesting webapps. We could implement a link when you click on a screenshot it directs you to a display of that host and all the services, ffuf data, etc.

Certain Reports could use Scope Filtering

For instance, running a DomainSummaryReport or DomainOwnerReport has no option to include or exclude passive/active scope items. Would be really handy for analysis when you want to stay in scope.

Add Database Reports

Add some quick ways to show things like:

  • State of the current scope (All out of scope, in scope, and passive scope)

  • All domains

  • All IPs (individuals)

  • All IPs and their open services

--rescan often doesn't work

Expected Behavior

Rerun module on systems that have already been processed

Actual Behavior

Nothing happens.

Steps to Reproduce the Problem

  1. Run: armory -m Gowitness -i
  2. Cancel the process before it finishes (ctrl+c)
  3. run: armory -m Gowtiness -i --rescan

Specifications

  • Python Version: Python 3.7.6
  • Pip packages: argcomplete==1.10.0
    backcall==0.1.0
    beautifulsoup4==4.8.0
    bs4==0.0.1
    certifi==2019.6.16
    chardet==3.0.4
    Click==7.0
    colorama==0.4.1
    configparser==3.8.1
    decorator==4.4.0
    depth-armory==1.0.0
    dnspython==1.16.0
    Flask==1.0.2
    future==0.17.1
    fuzzywuzzy==0.17.0
    geoip2==2.9.0
    idna==2.8
    ipaddr==2.2.0
    ipwhois==1.1.0
    ipython==7.7.0
    ipython-genutils==0.2.0
    itsdangerous==1.1.0
    jedi==0.15.0
    Jinja2==2.10.3
    lxml==4.4.1
    MarkupSafe==1.1.1
    maxminddb==1.5.1
    mysqlclient==1.4.4
    netaddr==0.7.19
    parso==0.5.1
    pexpect==4.7.0
    pickleshare==0.7.5
    praw==5.4.0
    prawcore==0.14.0
    prompt-toolkit==2.0.9
    ptyprocess==0.6.0
    pycurl==7.43.0.3
    Pygments==2.4.2
    pyparsing==2.4.2
    pyperclip==1.7.0
    python-docx==0.8.10
    python-Levenshtein==0.12.0
    pytz==2019.3
    requests==2.21.0
    requests-file==1.4.3
    six==1.12.0
    sopel==6.6.9
    soupsieve==1.9.2
    SQLAlchemy==1.3.6
    sqlalchemy-mixins==1.1
    termcolor==1.1.0
    thready==0.1.5
    tld==0.9.3
    tldextract==2.2.1
    traitlets==4.3.2
    typing==3.7.4
    update-checker==0.16
    urllib3==1.24.3
    wcwidth==0.1.7
    Werkzeug==0.16.0
    wfuzz==2.4
    whois==0.8
    xmltodict==0.12.0
  • Platform: Fedora 31

Binary not found

Multiple packages produce the following error:
LinkedInt binary not found. Please explicitly provide path with --binary

Could an appropriate work around be posted within the README? or is this a valid issue?

sqlite3.OperationalError: no such table: armory_main_domain when trying to import domains with Ingestor

Expected Behavior

Importing domain and IP data directly into the database as well as managing target scoping via the Ingestor module

Actual Behavior

Armory2 throws an sqlite3 error and exits.

Steps to Reproduce the Problem

1.$ echo 'google.com'>>domains.txt
2.$ armory2 -m Ingestor -d domains.txt -a -p

Traceback (most recent call last):
File "/home/sfttw/pythons/3.7/bin/armory2", line 33, in
sys.exit(load_entry_point('depth-armory==2.0.0', 'console_scripts', 'armory2')())
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_cmd.py", line 511, in main
run_module(Module, mod_args, modules[0])
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_cmd.py", line 330, in run_module
m.run(args)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_main/includsfttw/modules/Ingestor.py", line 137, in run
self.process_domain(line.strip())
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_main/includsfttw/modules/Ingestor.py", line 164, in process_domain
"passive_scope": self.passive_scope,
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/models/query.py", line 581, in get_or_create
return self.get(**kwargs), False
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/models/query.py", line 431, in get
num = len(clone)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/models/query.py", line 262, in len
self._fetch_all()
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/models/query.py", line 51, in iter
results = compiler.execute_sql(chunksfttw_fetch=self.chunksfttw_fetch, chunk_size=self.chunk_size)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/backends/utils.py", line 98, in execute
return super().execute(sql, params)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/home/sfttw/pythons/3.7/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: armory_main_domain

Specifications

  • Python Version: Python 3.7.8
  • Pip packages: argcomplete==1.12.3 asttokens==2.0.5 backcall==0.2.0 beautifulsoup4==4.10.0 black==21.12b0 bs4==0.0.1 certifi==2021.10.8 charset-normalizer==2.0.9 click==8.0.3 colorama==0.4.4 configparser==5.2.0 decorator==5.1.0 depth-armory==1.0.0 dnspython==2.0.0 executing==0.8.2 filelock==3.4.2 fuzzywuzzy==0.18.0 greenlet==1.1.2 idna==3.3 importlib-metadata==4.10.0 ipaddr==2.2.0 ipwhois==1.2.0 ipython==8.0.0b1 jedi==0.18.1 Jinja2==3.0.3 lxml==4.7.1 MarkupSafe==2.0.1 matplotlib-inline==0.1.3 mypy-extensions==0.4.3 mysqlclient==2.1.0 netaddr==0.8.0 parso==0.8.3 pathspec==0.9.0 pexpect==4.8.0 pickleshare==0.7.5 platformdirs==2.4.1 prompt-toolkit==3.0.24 ptyprocess==0.7.0 pure-eval==0.2.1 Pygments==2.11.1 pyperclip==1.8.2 python-docx==0.8.11 python-Levenshtein==0.12.2 requests==2.26.0 requests-file==1.5.1 six==1.16.0 soupsieve==2.3.1 SQLAlchemy==1.4.29 sqlalchemy-mixins==1.5 stack-data==0.1.3 termcolor==1.1.0 thready==0.1.5 tld==0.12.6 tldextract==3.1.2 tomli==1.2.3 traitlets==5.1.1 typed-ast==1.5.1 typing_extensions==4.0.1 urllib3==1.26.7 wcwidth==0.2.5 whois==0.9.13 xmltodict==0.12.0 zipp==3.7.0
  • Platform: OpenSuse Leap 15.3 (also tested on the latest versions of Fedora and MacOS)

LinkedInt Module Error

When running the LinkedInt module with options --top and --auto_keyword I get the following error:

File "/usr/local/lib/python3.7/dist-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_main/included/modules/LinkedInt.py", line 117, in run for user in domain.users AttributeError: 'BaseDomain' object has no attribute 'users'

This is after making a regular LinkedInt run and there is a CSV with users already.

ipwhois/asn.py error handling

Looks like the ipwhois/asn.py function is failing on an improper registry lookup, which crashes armory.

Error

  File "/usr/local/bin/armory", line 11, in <module>
    load_entry_point('depth-armory==1.0.0', 'console_scripts', 'armory')()
  File "/usr/local/lib/python2.7/dist-packages/armory/armory.py", line 427, in main
    run_module(Module, cmd_args, base_args.module)
  File "/usr/local/lib/python2.7/dist-packages/armory/armory.py", line 249, in run_module
    m.run(args)
  File "/usr/local/lib/python2.7/dist-packages/armory/included/ModuleTemplate.py", line 164, in run
    self.process_output(targets)
  File "/usr/local/lib/python2.7/dist-packages/armory/included/modules/Sublist3r.py", line 98, in process_output
    domain=new_domain
  File "/usr/local/lib/python2.7/dist-packages/armory/database/repositories.py", line 235, in find_or_create
    ip_address=i,
  File "/usr/local/lib/python2.7/dist-packages/armory/database/repositories.py", line 312, in find_or_create
    res = IPWhois(ip_str).lookup_whois()
  File "/usr/local/lib/python2.7/dist-packages/ipwhois/ipwhois.py", line 166, in lookup_whois
    get_asn_description=get_asn_description
  File "/usr/local/lib/python2.7/dist-packages/ipwhois/asn.py", line 498, in lookup
    raise ASNRegistryError('ASN registry lookup failed. '
ipwhois.exceptions.ASNRegistryError: ASN registry lookup failed. Permutations not allowed.

Commands Executed
$ armory -m Ingestor -p -d domain.com
$ armory -m Sublist3r -i

[Bug] Armory2.0 UserReport Error

Describe the bug
In armory 2.0 branch the UserReport displays an error.

To Reproduce
Steps to reproduce the behavior:

  1. Have users in database from another module.
  2. Run armory2 -r UserReport -u3

Error:
Traceback (most recent call last): File "/usr/local/bin/armory2", line 11, in <module> load_entry_point('depth-armory==2.0.0', 'console_scripts', 'armory2')() File "/usr/local/lib/python3.7/dist-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_cmd.py", line 485, in main run_report(Report, cmd_args, reports[0]) File "/usr/local/lib/python3.7/dist-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_cmd.py", line 300, in run_report m.run(args) File "/usr/local/lib/python3.7/dist-packages/depth_armory-2.0.0-py3.7.egg/armory2/armory_main/included/reports/UserReport.py", line 51, in run qry, model = self.BaseDomain.get_query() AttributeError: 'Report' object has no attribute 'BaseDomain'

Expected behavior
Expected user emails as output.

Desktop:
Docker on OS X.

SSLScan Module Connects to "Filtered" Ports

I noticed that the SSLScan module will spend time attempting to connect to ports which NMAP has marked as "Filtered". This slowed down some of the bigger scans I have done. I did not immediately notice a way to prevent this in the options.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.