stefanoj3 / dirstalk Goto Github PK

Modern alternative to dirbuster/dirb

License: MIT License

Makefile 2.07% Go 93.14% Dockerfile 0.40% Shell 4.40%

enumeration dirbuster dirb url-bruteforcer dictionaries bruteforce bruteforce-wordlist web-content-scanner auditing security

dirstalk's People

Contributors

Stargazers

Watchers

Forkers

0x01-tools marciopocebon enumeration-tools inspectordidi bbhunter benmcewan7e cybersealops shahid1996 ncmarian jontargaryen kappernie shizonic ingramali arukaminado zxyp9x polling-repo-continua w3b573r tehseensagar shelld3v fadinglr bonedaddy tubbz-alt iloveicedgreentea hartl3y94 j5s sec-nux do0x-ccc jamiejackherer 5l1v3r1 infosec-le anmolfid1 bicsolutions-xyz cpt-jack-a-castle castledev2022 franndzs irsl tommalvoriddle madusec imbas007 elamaran619 lusterx blyaha84 mvandermeulen tehmasta chrisemoulton vcore8 vladertel

dirstalk's Issues

dictionary error

Hello,

When I try to run a scam, this is what I get. I tried moving into the directory and calling the file and it has been to no avail.

docker run stefanoj3/dirstalk dirstalk scan "https://example.com/" -d "/usr/share/dirbuster/wordlists/directory-list-1.0.txt" Error: failed to build dictionary: dictionary: failed to get/usr/share/dirbuster/wordlists/directory-list-1.0.txt`: Get /usr/share/dirbuster/wordlists/directory-list-1.0.txt: unsupported protocol scheme ""
Usage:
dirstalk scan [url] [flags]

Flags:
--cookie stringArray cookie to add to each request; eg name=value (can be specified multiple times)
-d, --dictionary string dictionary to use for the scan (path to local file or remote url)
--header stringArray header to add to each request; eg name=value (can be specified multiple times)
-h, --help help for scan
--http-cache-requests cache requests to avoid performing the same request multiple times within the same scan (EG if the server reply with the same redirect location multiple times, dirstalk will follow it only once) (default true)
--http-methods strings comma separated list of http methods to use; eg: GET,POST,PUT (default [GET])
--http-statuses-to-ignore ints comma separated list of http statuses to ignore when showing and processing results; eg: 404,301 (default [404])
--http-timeout int timeout in milliseconds (default 5000)
--out string path where to store result output
--scan-depth int scan depth (default 3)
--socks5 string socks5 host to use
-t, --threads int amount of threads for concurrent requests (default 3)
--use-cookie-jar enables the use of a cookie jar: it will retain any cookie sent from the server and send them for the following requests
--user-agent string user agent to use for http requests

Global Flags:
-v, --verbose verbose mode

level=fatal msg="Execution error" err="failed to build dictionary: dictionary: failed to get /usr/share/dirbuster/wordlists/directory-list-1.0.txt: Get /usr/share/dirbuster/wordlists/directory-list-1.0.txt: unsupported protocol scheme """
`

Makefile release* commands should use a reproducible environment

https://hub.docker.com/r/goreleaser/goreleaser/tags

regexp filter

Add possibility to filter results based on content body and regexp

failed to build dictionary: dictionary

Hello @stefanoj3 ,

When running the below command, specifically placing single quotes in the 2nd to last flag I got the aforementioned error., Interestingly if I removed the quotes it worked and later in testing i was able to get it to successfully run with the single quotes. Its not a major issue but am raising it so you know it exists

docker run stefanoj3/dirstalk dirstalk scan http://example.com -d "https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt" --no-check-certificate --http-statuses-to-ignore '404,301' -t 10
Error: failed to build dictionary: dictionary: failed to get `https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt`: Get https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Usage:
  dirstalk scan [url] [flags]

Flags:
      --cookie stringArray             cookie to add to each request; eg name=value (can be specified multiple times)
  -d, --dictionary string              dictionary to use for the scan (path to local file or remote url)
      --header stringArray             header to add to each request; eg name=value (can be specified multiple times)
  -h, --help                           help for scan
      --http-cache-requests            cache requests to avoid performing the same request multiple times within the same scan (EG if the server reply with the same redirect location multiple times, dirstalk will follow it only once) (default true)
      --http-methods strings           comma separated list of http methods to use; eg: GET,POST,PUT (default [GET])
      --http-statuses-to-ignore ints   comma separated list of http statuses to ignore when showing and processing results; eg: 404,301 (default [404])
      --http-timeout int               timeout in milliseconds (default 5000)
      --no-check-certificate           to skip checking the validity of SSL certificates
      --out string                     path where to store result output
      --scan-depth int                 scan depth (default 3)
      --socks5 string                  socks5 host to use
  -t, --threads int                    amount of threads for concurrent requests (default 3)
      --use-cookie-jar                 enables the use of a cookie jar: it will retain any cookie sent from the server and send them for the following requests
      --user-agent string              user agent to use for http requests

Global Flags:
  -v, --verbose   verbose mode

level=fatal msg="Execution error" err="failed to build dictionary: dictionary: failed to get `https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt`: Get https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"

scripting target processor

Could use otto as a scripting language to add more complex processing rules for dictionaries

provide aur package

Add support to specify any number of custom headers to the http calls

sigint / ctr-c

handle signint: do not jus tkill the application, stop the scan and print the summary of what was found so far

provide blackarch package

https://blackarch.org/tools.html

stricter linter

Make golangci-lint stricter by enabling more linters and fix all the reported issues.

EG golangci-lint run --enable-all

Add support for cookie jar in http client

can decide whether or not to retain some cookies
can add cookies for every request - value can now be added to the jar, consider having a "null jar" for cases when the normal cookie jar is not provided (the default behavior)

Error requests

dial tcp: lookup site.com on [::1]:53: read udp [::1]:58403->[::1]:53: read: connection refused" method=GET path="~user3" DEBU terminating worker: producer channel closed ERRO failed to perform request depth=3

what is your solution with this?

scan result feature

add flag to specify file where to store a scan result
add command to view scan results
~~add flag to execute another command for each scan result~~ canceled
add command to diff between 2 scan results

Asciinema & update readme

Update readme with the latest functionalities (cookie jar, user agent, headers)
Add Asciinema example of dirstalk usage

Add support for calling a custom command for each result found

Optional `/` at the end of URL when scanning

dictionary.combine

Command to combine dictionaries together or to create a dictionary made of absolute paths made of the combination of all the parts in another dictionary

Certificate signed by Unknown

Hello,

It seems to fail if it doesn't trust the cert from the target web server. Below is sample output from when I ran it. Is there anyway a -k (curl) or --no-check-certificate (wget) flag can be added?

docker run stefanoj3/dirstalk dirstalk scan "https://redacted.example.com" -d https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt
level=info msg="Starting scan" cookie-jar=false cookies= dictionary-length=141694 headers= scan-depth=3 socks5="<nil>" threads=3 timeout=5000 url="https://redacted.example.com" user-agent=
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/cgi-bin: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=cgi-bin
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/education: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=education
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/betsie: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=betsie
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/accessibility: x509: certificate signed by unknown authority" method=GET path=accessibility
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/go: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=go
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/accesskeys: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=accesskeys
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/-: x509: certificate signed by unknown authority" method=GET path=-
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/toolbar: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=toolbar
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/radio: x509: certificate signed by unknown authority" method=GET path=radio
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/tv: x509: certificate signed by unknown authority" method=GET path=tv
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/whereilive: x509: certificate signed by unknown authority" method=GET path=whereilive
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/talk: x509: certificate signed by unknown authority" method=GET path=talk
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/homepage: x509: certificate signed by unknown authority" method=GET path=homepage
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/a-z: x509: certificate signed by unknown authority" method=GET path=a-z
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/bb: x509: certificate signed by unknown authority" method=GET path=bb
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/int: x509: certificate signed by unknown authority" method=GET path=int
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/textonly: x509: certificate signed by unknown authority" method=GET path=textonly
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/oth: x509: certificate signed by unknown authority" method=GET path=oth
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/mobile: x509: certificate signed by unknown authority" method=GET path=mobile
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/t: x509: certificate signed by unknown authority" method=GET path=t
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/help: x509: certificate signed by unknown authority" method=GET path=help
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/search: x509: certificate signed by unknown authority" method=GET path=search
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/pl1: x509: certificate signed by unknown authority" method=GET path=pl1
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/makehomepage: x509: certificate signed by unknown authority" method=GET path=makehomepage
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/text: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=text

Add support for specifying a custom UA

result screenshot

https://github.com/mafredri/cdp
https://godoc.org/github.com/tebeka/selenium
https://hacks.mozilla.org/2017/12/using-headless-mode-in-firefox/

could use headless chrome or selenium to integrate and capture screenshots of all the results found and create a nice html page to view them

scan output to file

add possibility to save the result to a file
add command to compare 2 results and print out diff (useful for automated scans)

add cewl example to readme

https://github.com/digininja/CeWL

Add support for prefixes and suffixes to be added to each word of the dictionary when scanning

Docker stopping issue

Hello @stefanoj3 , we meet again lol!

Humor aside, when I ran dirstalk in docker i noticed if i tried do to either ctrl+z or ctrl+c it didnt stop the program but instead moved to the next procedural phase. I had to hit it a few times to eventually make it stop. I am not sure if you were aware but I figured I let you know. Its a minor issue but ya.

https://pastebin.com/LHJrNpnV
https://github.com/danielmiessler/SecLists/tree/master/Discovery/Web-Content (raft)
add entries from common open source projects