proxy-scraper-checker

HTTP, SOCKS4, SOCKS5 proxies scraper and checker.

Can determine if the proxy is anonymous.
Supports determining the geolocation of the proxy exit node.
Can sort proxies by speed.
Uses regex to find proxies of format protocol://username:password@ip:port on a web page or in a local file, allowing proxies to be extracted even from json without code changes.
Supports proxies with authentication.
It is possible to specify the URL to which to send a request to check the proxy.
Supports saving to plain text and json.
Asynchronous.

You can get proxies obtained using this project in monosans/proxy-list.

Installation and usage

Standalone executable

This is the easiest way, but it is only available for x86-64 Windows, x86-64/arm64 macOS and x86-64 Linux. Just download the archive for your OS from nightly.link, unzip it, edit config.toml and run the proxy_scraper_checker executable.

If Windows antivirus detects the executable file as a virus, please read this.

Docker

Install Docker Compose.
Download and unpack the archive with the program.
Edit config.toml to your preference.

Run the following commands:

docker compose build --pull
docker compose up --no-log-prefix

Running from source code

Desktop

Install Python. The minimum version required is 3.8.
Download and unpack the archive with the program.
Edit config.toml to your preference.
Run the script that installs dependencies and starts proxy-scraper-checker:
- On Windows run start.cmd
- On Unix-like operating systems run start.sh

Termux

To use proxy-scraper-checker in Termux, knowledge of the Unix command-line interface is required.

Download Termux from F-Droid. Don't download it from Google Play.
Run the following command (it will automatically update Termux packages, install Python, and download and install proxy-scraper-checker):
```
bash <(curl -fsSL 'https://raw.githubusercontent.com/monosans/proxy-scraper-checker/main/install-termux.sh')
```
Edit ~/proxy-scraper-checker/config.toml to your preference using a text editor (vim/nano).
To run proxy-scraper-checker use the following command:
```
cd ~/proxy-scraper-checker && sh start-termux.sh
```

Something else?

All other info is available in config.toml file.

License

MIT

This product includes GeoLite2 Data created by MaxMind, available from https://www.maxmind.com.

n0rv3ll / proxy-scraper-checker Goto Github PK