devanshbatham / paramspider

Mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing

---

Hey, It's Devansh ⚡

Projects 🏗️

• ParamSpider - Mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing
• FavFreak - Making Favicon.ico based Recon Great again
• OpenRedireX - A fuzzer for detecting open redirect vulnerabilities
• CertEagle - Weaponizing Live CT logs for automated monitoring of assets
• Quaithe - Quaithe empowers you to execute multiple commands in parallel for blazing-fast performance.
• DNSleuth - DNSleuth allows you to spy on the DNS queries your machine is making.
• Watson - Watson is a utility for note management and search from your terminal.
• ip2cloud - Check IP addresses against known cloud provider IP address ranges
• getresolvers - A simple utility to fetch freshly updated DNS resolvers
• revwhoix - A simple utility to perform reverse WHOIS lookups using whoisxml API
• heaptruffle - Mine URLs from Browser's Heap Snapshot for fun and profit
• ip2asn - A utility to quickly map IP addresses to their respective ASN
• revit - A command-line utility for performing reverse DNS lookups
• getsan - A utility to fetch and display dns names from the SSL/TLS cert data
• headerpwn - A fuzzer for finding anomalies and analyzing how servers respond to different HTTP headers
• dnsaudit - A command-line utility for auditing DNS configuration using Zonemaster API
• rayder - A lightweight tool for orchestrating and organizing your bug hunting recon / pentesting command-line workflows
• realm - A utility for recursively traversing SSL/TLS certificates for collecting DNS names

Knowledge Bases 💡

• Awesome Bugbounty Writeups - A curated list of bugbounty writeups (Bug type wise)
• Vulnerabilities Unmasked - Learn complex security vulnerabilities in simple terms that even a five-year-old can understand!
• Everything About DNS - Explains the basics of DNS at different levels of complexity for readers with various technical backgrounds.
• Solidity Gas Optimization Tips - Solidity Gas Optimization Tips and Tricks

Contact 📞

• Twitter
• LinkedIn

paramspider's Issues

coded with <3 by Devansh Batham

I can’t run this scrip with a 'coded with <3 by Devansh Batham' error.
My env:Python3.7

i have problem

when i run gf i have error

I get a bug

it's show "Can not get target information If you think this is a bug or unintentional behaviour."
and i change other domain it also bug。thanks

error!! ##Traceback

Traceback (most recent call last):
File "paramspider.py", line 1, in
from core import requester
File "/home/user/Downloads/ParamSpider/core/requester.py", line 1, in
import requests
ModuleNotFoundError: No module named 'requests'
how to fix this bug can you please help me out

hi

Excuse me, this script not support the format of IP? such as 192.168.1.1:8080/admin/

error

Can not get target information
If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues
Traceback (most recent call last):
File "paramspider.py", line 73, in
main()
File "paramspider.py", line 37, in main
response = unquote(response)
File "/usr/lib/python3.7/urllib/parse.py", line 629, in unquote
if '%' not in string:
TypeError: argument of type 'bool' is not iterable

tool not working

There is no output from the tool after scanning any domain

Import Request Error

HI I`m using Python 3.7.4 v

when i tried to run this script I'm getting this below mentioned error

Traceback (most recent call last):
File "/root/XSS/ParamSpider/core/requester.py", line 1, in
import requests
ModuleNotFoundError: No module named 'requests'

Tool not working on any of the domain

python3 paramspider.py --domain bugcrowd.com --exclude woff,css,js,png,svg,php,jpg --output bugcrowd.txt

Can not get target information If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues

For any of the domain it throws this error please help

Can not get target information If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues

exclude subdomains using -s is not working

Service Temporary Unavailable

503 Server Error: Service Temporarily Unavailable for url:

OOPS!! Timeout Error. Retrying in 2 seconds.

Hello
Every time I check the site with "paramspider", this message will be displayed to me!
Is there a problem or are those sites not vulnerable?

It doesn't work, man

why do not you answer? It does not work!!!!!
error: OOPS!! Timeout Error. Retrying in 2 seconds.

Cannot get target information

root@vivo1606:~/Downloads/ParamSpider-master# python3 paramspider.py --domain bugcrowd.com --exclude woff,css,js,png,svg,php,jpg

     ___                               _    __       
    / _ \___ ________ ___ _  ___ ___  (_)__/ /__ ____
   / ___/ _ `/ __/ _ `/  ' \(_-</ _ \/ / _  / -_) __/
  /_/   \_,_/_/  \_,_/_/_/_/___/ .__/_/\_,_/\__/_/   
                              /_/                    
                        
                        - coded with <3 by Devansh Batham 

Can not get target information
If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues
root@vivo1606:~/Downloads/ParamSpider-master#

gf+paramspider on window

i installed gf and paramspider on windows machine, can you explain how can use in windows env?
gf is in C:\Users\nath\go\src\github.com\tomnomnom\gf
ps is in D:\Dropbox\bug\Asus\Web\Tools\OSINT\ParamSpider

do not work correctly

Whatever target you enter, the prompt is shown in the figure below:

unintentional behaviour

Can not get target information while scan the sites at sometimes

it doesn't work

python paramspider.py -d xxx.com
DOS print shbatham/ParamSpider//issues

version:python3.8

what happened? how can I do for it？
Longing for your valuable suggestions

Not working on mac os and ubuntu ( all the requirement is instaled)

python paramspider.py --domain yahoo.com
File "paramspider.py", line 35
url = f"http://web.archive.org/cdx/search/cdx?url=*.{args.domain}/*&output=txt&fl=original&collapse=urlkey&page=/"
^
SyntaxError: invalid syntax

q:

Can not get target information
If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues
Traceback (most recent call last):
File "paramspider.py", line 73, in
main()
File "paramspider.py", line 37, in main
response = unquote(response)
File "D:\Python38\lib\urllib\parse.py", line 614, in unquote
if '%' not in string:
TypeError: argument of type 'bool' is not iterable

Not run paramspider

![Capture](https://user-images.githubusercontent.com/37695035/83180599-7c648800-a145-11ea-8395-ed1842299d9e.JPG

Path Related Issue

How can I find path of GF in my System?

'/User/levi/go/bin/gf'

Error with domain name

Hi,

Each time i try a domain name that contain a "-" i have this error:
Can not get target information

invalid syntax

on line 35 using Python 3.5

ParamSpider/paramspider.py

Line 35 in 3e1c5b1

url = f"http://web.archive.org/cdx/search/cdx?url=*.{args.domain}/*&output=txt&fl=original&collapse=urlkey&page=/"

does not find parameters

I have problem becouse paramspider not fint paramaters:( why?

I have website:

└─# cat /var/www/html/index.php

<form action="index.php" method="get">
<input type="text" name="name" value="name">
<input type="submit" name="send" value="send">
</form>

<a href="index.php?name=michas">aaa</a>
Look this shit
<?php
echo $_GET['name'];
?>

i run paramspider:

┌──(root💀za)-[/opt/ParamSpider]
└─# python3 /opt/ParamSpider/paramspider.py --domain  http://mydomian/index.php


         ___                               _    __
        / _ \___ ________ ___ _  ___ ___  (_)__/ /__ ____
       / ___/ _ `/ __/ _ `/  ' \(_-</ _ \/ / _  / -_) __/
      /_/   \_,_/_/  \_,_/_/_/_/___/ .__/_/\_,_/\__/_/
                                  /_/

                            - coded with <3 by Devansh Batham

Can not get target information
If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues




[+] Total unique urls found : 0
[+] Output is saved here   : output/http://maindomain/index.php.txt

[!] Total execution time      : 0.48279s

┌──(root💀za)-[/opt/ParamSpider]

CURL:

┌──(root💀za)-[/opt/ParamSpider]
└─# curl http://mydomain/index.php
<form action="index.php" method="get">
<input type="text" name="name" value="name">
<input type="submit" name="send" value="send">
</form>

<a href="index.php?name=michas">aaa</a>
Look this shit

┌──(root💀za)-[/opt/ParamSpider]
└─# **curl http://mydomainl/index.php?name=XSS**
<form action="index.php" method="get">
<input type="text" name="name" value="name">
<input type="submit" name="send" value="send">
</form>

<a href="index.php?name=michas">aaa</a>
Look this shit
**XSS**                                                                                                                                                   
┌──(root💀za

TypeError: argument of type 'bool' is not iterable

    
Can not get target information
If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues
Traceback (most recent call last):
  File "paramspider.py", line 73, in <module>
    main()
  File "paramspider.py", line 37, in main
    response = unquote(response)
  File "/usr/lib/python3.7/urllib/parse.py", line 609, in unquote
    if '%' not in string:
TypeError: argument of type 'bool' is not iterable

not work with subdomains

when i'm trying to do with subdomain max time it says
Can not get target information
If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues

[+] Total unique urls found : 0
[+] Output is saved here : output/b.txt

[!] Total execution time : 2.0016s

query used :-

python paramspider.py --domain engage.abc.com -o b.txt

Python Syntax Error

File "paramspider.py", line 40
url = f"http://web.archive.org/cdx/search/cdx?url=*.{args.domain}/*&output=txt&fl=original&collapse=urlkey&page=/"
^
SyntaxError: invalid syntax

Unable to enumerate parameters

SyntaxError: line 40

Hi, I have just noticed that you updated your script 5 days ago, so I tested out your script but somehow there's an error in line 40.

RAM memory saturation

Some domains, if they are large, saturate the RAM memory and linux kills the process

For exaxmple: python3 paramspider.py -d crd.auctions.yahoo.co.jp

OS: Linux Manjaro 5.4.62-1-MANJARO

Update Request

Hello friend, I just uploaded an update on Paramspider just updating the request

if you can check thank you

Request 2.27.0

@devanshbatham @vzamanillo @rotemreiss @RoninNakomoto @storenth @CowlingBanana @rotemplay

Not able to featch info

when i am giving command like python3 paramspider.py --domain hackerone.com, always showing target info not showing.

Possibility to save in any other directory the results?

What about changing in this file core/save_it.py:

This:

if outfile:
        filename = f'output/{outfile}'

to this:

if outfile:
        filename = f'{outfile}'

So this way the output file can be saved in other root directory.

please , help me

erro: {e} Can not get target information
If you think this is a bug or unintentional behaviour. Report here : https://github.com/devanshbatham/ParamSpider/issues

To run an error

Traceback (most recent call last):
File "paramspider.py", line 73, in
main()
File "paramspider.py", line 37, in main
response = unquote(response)
File "C:\python3\lib\urllib\parse.py", line 634, in unquote
if '%' not in string:
TypeError: argument of type 'bool' is not iterable

Question?

This only supports .com?

Tool runnning but showing no results

Everytime i run the tool its not working , using ubuntu wsl
there are no errors or warning in runtime but still its not giving any results. Count is always 0 for every domain i use it for .

[+] Total unique urls found : 0
[+] Output is saved here : output/google.com.txt

command : python3 paramspider.py --domain google.com

please resolve this issue.

PC freeze when using paramspider

It does not show any response when scanning for domain. It freezes everything.
Please help

tools is not running

The tools throws the following error
Can not get target information
If you think this is a bug or unintentional behaviour.

Steps to reproduce:

1. clone the repo
2. install all requirements using pip3
3. then ran python3 paramspider.py -d www.tesla.com

my system specs are in attachment

Encoding issue with non-unicode characters

In this case the letter 'ć' or '\u0107'.

Traceback (most recent call last): File "paramspider.py", line 67, in <module> main() File "paramspider.py", line 52, in main print('\n'.join(final_uris)) File "AppData\Local\Programs\Python\Python37\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\u0107' in position 2259: character maps to <undefined>

something wrong

can't use it ,always say Can not get target information. maybe i use wrong ...

problem

gf redirect domain.txt
As you mentioned i follwed but its not working

no such pattern

not work

can not target get information error message !!

Help

How can to use gf in windows im struggling following your steps i already install go located in C:\Users\juan\go\bin\gf. Im confused in this part Note : Replace '/User/levi/go/bin/gf' with the path where gf binary is located in your system.

PLEASE ADD file option in Param Spider

Means instead of add -d example.com
we can add test file of url in paramaspider please add in next update

Can not connect to server. Check your internet connection

I have installed the tool and found the following prompt while running it:

"Cannot connect to server. Check your internet connection"

Please find below the snapshot:

pip3 install -r requirements.txt
Collecting certifi==2020.4.5.1
Using cached certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
Requirement already satisfied: chardet==3.0.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 2)) (3.0.4)
Collecting idna==2.9
Using cached idna-2.9-py2.py3-none-any.whl (58 kB)
Collecting requests==2.23.0
Using cached requests-2.23.0-py2.py3-none-any.whl (58 kB)
Collecting urllib3==1.25.8
Using cached urllib3-1.25.8-py2.py3-none-any.whl (125 kB)
Installing collected packages: urllib3, idna, certifi, requests
Attempting uninstall: urllib3
Found existing installation: urllib3 1.24.3
Uninstalling urllib3-1.24.3:
Successfully uninstalled urllib3-1.24.3
Attempting uninstall: idna
Found existing installation: idna 2.8
Uninstalling idna-2.8:
Successfully uninstalled idna-2.8
Attempting uninstall: certifi
Found existing installation: certifi 2021.5.30
Uninstalling certifi-2021.5.30:
Successfully uninstalled certifi-2021.5.30
Attempting uninstall: requests
Found existing installation: requests 2.21.0
Uninstalling requests-2.21.0:
Successfully uninstalled requests-2.21.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
search-that-hash 0.2.8 requires click<8.0.0,>=7.1.2, but you have click 8.0.1 which is incompatible.
search-that-hash 0.2.8 requires requests<3.0.0,>=2.25.1, but you have requests 2.23.0 which is incompatible.
metafinder 1.1 requires beautifulsoup4>=4.9.3, but you have beautifulsoup4 4.6.3 which is incompatible.
metafinder 1.1 requires requests>=2.25.1, but you have requests 2.23.0 which is incompatible.
metafinder 1.1 requires urllib3>=1.26.4, but you have urllib3 1.25.8 which is incompatible.
emailfinder 0.2.3b0 requires beautifulsoup4>=4.9.3, but you have beautifulsoup4 4.6.3 which is incompatible.
emailfinder 0.2.3b0 requires requests>=2.25.1, but you have requests 2.23.0 which is incompatible.
emailfinder 0.2.3b0 requires urllib3>=1.26.4, but you have urllib3 1.25.8 which is incompatible.
censys 2.0.0 requires requests>=2.25.1, but you have requests 2.23.0 which is incompatible.
Successfully installed certifi-2020.4.5.1 idna-2.9 requests-2.23.0 urllib3-1.25.8
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv

┌──(root💀kali)-[~/ParamSpider]
└─# python3 paramspider.py --domain target.com --exclude woff,png,svg,php,jpg --output ~/target.com/DOMAINS/params_result.txt

     ___                               _    __                                                                                                                                                                                           
    / _ \___ ________ ___ _  ___ ___  (_)__/ /__ ____                                                                                                                                                                                    
   / ___/ _ `/ __/ _ `/  ' \(_-</ _ \/ / _  / -_) __/                                                                                                                                                                                    
  /_/   \_,_/_/  \_,_/_/_/_/___/ .__/_/\_,_/\__/_/                                                                                                                                                                                       
                              /_/                                                                                                                                                                                                        
                        
                        - coded with <3 by Devansh Batham 

Can not connect to server. Check your internet connection.

cli tool

create setup.py and publish to pypi

not work

OOPS!! Timeout Error. Retrying in 2 seconds.