Coder Social home page Coder Social logo

wivd / httpx Goto Github PK

View Code? Open in Web Editor NEW

This project forked from projectdiscovery/httpx

1.0 0.0 0.0 1.82 MB

httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Home Page: https://projectdiscovery.io/open-source

License: MIT License

Go 99.62% Dockerfile 0.38%

httpx's Introduction

httpx

License Go Report Card contributions welcome GitHub Release Follow on Twitter Docker Images Chat on Discord

httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Resources

Features

httpx

  • Simple and modular code base making it easy to contribute.
  • Fast And fully configurable flags to probe mutiple elements.
  • Supports multiple HTTP based probings.
  • Smart auto fallback from https to http as default.
  • Supports hosts, URLs and CIDR as input.
  • Handles edge cases doing retries, backoffs etc for handling WAFs.

Supported probes:-

Probes Status Default check
URL ✔ true
Title ✔ true
Status Code ✔ true
Content Length ✔ true
TLS Certificate ✔ true
CSP Header ✔ true
HTTP2 ✔ false
HTTP 1.1 Pipeline ✔ false
Virtual host ✔ false
Location Header ✔ true
Web Server ✔ true
Web Socket ✔ true
Path ✔ false
Ports ✔ false
Request method ✔ false
IP ✔ true
CNAME ✔ true
CDN ✔ false
Response Time ✔ true

Installation Instructions

From Binary

The installation is easy. You can download the pre-built binaries for your platform from the Releases page. Extract them using tar, move it to your $PATHand you're ready to go.

Download latest binary from https://github.com/projectdiscovery/httpx/releases

▶ tar -xvf httpx-linux-amd64.tar
▶ mv httpx-linux-amd64 /usr/local/bin/httpx
▶ httpx -h

From Source

httpx requires go1.14+ to install successfully. Run the following command to get the repo -

▶ GO111MODULE=auto go get -u -v github.com/projectdiscovery/httpx/cmd/httpx

From Github

▶ git clone https://github.com/projectdiscovery/httpx.git; cd httpx/cmd/httpx; go build; mv httpx /usr/local/bin/; httpx -version

Usage

httpx -h

This will display help for the tool. Here are all the switches it supports.

Flag Description Example
H Custom Header input httpx -H 'x-bug-bounty: hacker'
follow-redirects Follow URL redirects (default false) httpx -follow-redirects
follow-host-redirects Follow URL redirects only on same host(default false) httpx -follow-host-redirects
http-proxy URL of the proxy server httpx -http-proxy hxxp://proxy-host:80
l File containing HOST/URLs/CIDR to process httpx -l hosts.txt
no-color Disable colors in the output. httpx -no-color
o File to save output result (optional) httpx -o output.txt
json Prints all the probes in JSON format (default false) httpx -json
vhost Probes to detect vhost from list of subdomains httpx -vhost
threads Number of threads (default 50) httpx -threads 100
http2 HTTP2 probing httpx -http2
pipeline HTTP1.1 Pipeline probing httpx -pipeline
ports Ports ranges to probe (nmap syntax: eg 1,2-10,11) httpx -ports 80,443,100-200
title Prints title of page if available httpx -title
path Request path/file httpx -path /api
content-length Prints content length in the output httpx -content-length
ml Match content length in the output httpx -content-length -ml 125
fl Filter content length in the output httpx -content-length -fl 0,43
status-code Prints status code in the output httpx -status-code
mc Match status code in the output httpx -status-code -mc 200,302
fc Filter status code in the output httpx -status-code -fc 404,500
tls-probe Send HTTP probes on the extracted TLS domains httpx -tls-probe
content-type Prints content-type httpx -content-type
location Prints location header httpx -location
csp-probe Send HTTP probes on the extracted CSP domains httpx -csp-probe
web-server Prints running web sever if available httpx -web-server
sr Store responses to file (default false) httpx -sr
srd Directory to store response (optional) httpx -srd httpx-output
unsafe Send raw requests skipping golang normalization httpx -unsafe
request File containing raw request to process httpx -request
retries Number of retries httpx -retries
silent Prints only results in the output httpx -silent
timeout Timeout in seconds (default 5) httpx -timeout 10
verbose Verbose Mode httpx -verbose
version Prints current version of the httpx httpx -version
x Request Method (default 'GET') httpx -x HEAD
method Output requested method httpx -method
response-time Output the response time httpx -response-time
response-in-json Include response in stdout (only works with -json) httpx -response-in-json
websocket Prints if a websocket is exposed httpx -websocket
ip Prints the host IP httpx -ip
cname Prints the cname record if available httpx -cname
cdn Check if domain's ip belongs to known CDN httpx -cdn
filter-string Filter results based on filtered string httpx -filter-string XXX
match-string Filter results based on matched string httpx -match-string XXX
filter-regex Filter results based on filtered regex httpx -filter-regex XXX
match-regex Filter results based on matched regex httpx -match-regex XXX

Running httpx with stdin

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

▶ cat hosts.txt | httpx 

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|   v1.0  
             /_/            

		projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.

https://mta-sts.managed.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://docs.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Running httpx with file input

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

▶ httpx -l hosts.txt -silent

https://docs.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.managed.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Running httpx with CIDR input

▶ echo 173.0.84.0/24 | httpx -silent

https://173.0.84.29
https://173.0.84.43
https://173.0.84.31
https://173.0.84.44
https://173.0.84.12
https://173.0.84.4
https://173.0.84.36
https://173.0.84.45
https://173.0.84.14
https://173.0.84.25
https://173.0.84.46
https://173.0.84.24
https://173.0.84.32
https://173.0.84.9
https://173.0.84.13
https://173.0.84.6
https://173.0.84.16
https://173.0.84.34

Running httpx with subfinder

▶ subfinder -d hackerone.com -silent | httpx -title -content-length -status-code -silent

https://mta-sts.forwarding.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://mta-sts.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://mta-sts.managed.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://docs.hackerone.com [200] [65444] [HackerOne Platform Documentation]
https://www.hackerone.com [200] [54166] [Bug Bounty - Hacker Powered Security Testing | HackerOne]
https://support.hackerone.com [301] [489] []
https://api.hackerone.com [200] [7791] [HackerOne API]
https://hackerone.com [301] [92] []
https://resources.hackerone.com [301] [0] []

📋 Notes

  • As default, httpx checks for HTTPS probe and fall-back to HTTP only if HTTPS is not reachable.
  • For printing both HTTP/HTTPS results, no-fallback flag can be used.
  • vhost, http2, pipeline, ports, csp-probe, tls-probe and path are unique flag with different probes.
  • Unique flags should be used for specific use cases instead of running them as default with other flags.
  • When using json flag, all the information (default probes) included in the JOSN output.

Thanks

httpx is made with 🖤 by the projectdiscovery team. Community contributions have made the project what it is. See the Thanks.md file for more details. Do also check out these similar awesome projects that may fit in your workflow:

Probing feature is inspired by @tomnomnom/httprobe work ❤️

httpx's People

Contributors

ehrishirajsharma avatar ehsandeep avatar erjanmx avatar ice3man543 avatar jimen0 avatar jsav0 avatar lc avatar melardev avatar mzack9999 avatar st3rv04ka avatar timoles avatar vzamanillo avatar

Stargazers

avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.