Coder Social home page Coder Social logo

nightshift2k / binance-proxy Goto Github PK

View Code? Open in Web Editor NEW

This project forked from adrianceding/binance-proxy

112.0 8.0 30.0 6.23 MB

A websocket proxy written in GoLang, that caches the endpoints: klines, depth, ticker/24hr, and exchangeInfo. It resembles the Binance API behaviour. And has a primary usecase to eliminate ratelimits when querying the Binance API from a single IP.

License: MIT License

Go 93.97% Dockerfile 0.91% Makefile 5.12%
go golang binance proxy ccxt freqtrade crypto bitcoin websockets rest-api

binance-proxy's Introduction

Binance Proxy

A fast and simple Websocket Proxy for the Binance API written in GoLang. Mimics the behavior of API endpoints to avoid rate limiting imposed on IP's when using REST queries. Intended Usage for multiple instances of applications querying the Binance API at a rate that might lead to banning or blocking, like for example the Freqtrade Trading Bot, or any other similar application.

latest version go version  github issues license

⚡ Quick Start

You can download the pre-compiled binary for the architecture of your choice from the relaseses page on GitHub.

Unzip the package to a folder of choice, preferably one that's in $PATH

tar -xf binance-proxy_1.2.4_Linux_x86_64.tar.gz -C /usr/local/bin 

Starting the proxy:

binance-proxy

That's all you need to know to start! 🎉

🐳 Docker-way to quick start

If you don't want to install or compile the binance-proxy to your system, feel free using the prebuild Docker images and run it from an isolated container:

docker run --rm -d nightshift2k/binance-proxy:latest

ℹ️ Please pay attention to configuring network access, per default the ports 8090 and 8091 are exposed, if you specify different ports via parameters, you will need to re-configure your docker setup. Please refer to the docker network documentation, how to adjust this inside a container.

⚒️ Installing from source

First of all, download and install Go. Version 1.17 or higher is required.

Installation is done by using the go install command and rename installed binary in $GOPATH/bin:

go install github.com/nightshift2k/binance-proxy/cmd/binance-proxy

📖 Basic Usage

The proxy listens automatically on port 8090 for Spot markets, and port 8091 for Futures markets. Available options for parametrizations are available via -h

Usage:
  binance-proxy [OPTIONS]

Application Options:
  -v, --verbose                Verbose output (increase with -vv) [$BPX_VERBOSE]
  -p, --port-spot=             Port to which to bind for SPOT markets (default: 8090) [$BPX_PORT_SPOT]
  -t, --port-futures=          Port to which to bind for FUTURES markets (default: 8091) [$BPX_PORT_FUTURES]
  -c, --disable-fake-candles   Disable generation of fake candles (ohlcv) when sockets have not delivered data yet [$BPX_DISABLE_FAKE_CANDLES]
  -s, --disable-spot           Disable proxying spot markets [$BPX_DISABLE_SPOT]
  -f, --disable-futures        Disable proxying futures markets [$BPX_DISABLE_FUTURES]
  -a, --always-show-forwards   Always show requests forwarded via REST even if verbose is disabled [$BPX_ALWAYS_SHOW_FORWARDS]

Help Options:
  -h, --help                   Show this help message

🪙 Example Usage with Freqtrade

Freqtrade needs to be aware, that the API endpoint for querying the exchange is not the public endpoint, which is usually https://api.binance.com but instead queries are being proxied. To achieve that, the appropriate config.json needs to be adjusted in the { exchange: { urls: { api: public: "..."} } } section.

{
    "exchange": {
        "name": "binance",
        "key": "",
        "secret": "",
        "ccxt_config": {
            "enableRateLimit": false,
            "urls": {
                "api": {
                    "public": "http://127.0.0.1:8090/api/v3"
                }
            }
        },
        "ccxt_async_config": {
            "enableRateLimit": false
        }
    }
}

This example assumes, that binance-proxy is running on the same host as the consuming application, thus localhost or 127.0.0.1 is used as the target address. Should binance-proxy run in a separate 🐳 Docker container, a separate instance or a k8s pod, the target address has to be replaced respectively, and it needs to be ensured that the required ports (8090/8091 per default) are opened for requests.

➡️ Supported API endpoints for caching

Endpoint Market Purpose Socket Update Interval Comments
/api/v3/klines
/fapi/v1/klines
spot/futures Kline/candlestick bars for a symbol ~ 2s Websocket is closed if there is no following request after 2 * interval_time (for example: A websocket for a symbol on 5m timeframe is closed after 10 minutes.

Following requests for klines can not be delivered from the websocket cache:
  • limit parameter is > 1000
  • startTime or endTime have been specified
  • /api/v3/depth
    /fapi/v1/depth
    spot/futures Order Book (Depth) 100ms Websocket is closed if there is no following request after 2 minutes.

    The depth endpoint serves only a maximum depth of 20.
    /api/v3/ticker/24hr spot 24hr ticker price change statistics 2s/100ms (see comments) Websocket is closed if there is no following request after 2 minutes.

    For faster updates the values for
  • lastPrice
  • bidPrice
  • askPrice
    are taken from the bookTicker which is updated in an interval of 100ms.
  • /api/v3/exchangeInfo
    /fapi/v1/exchangeInfo
    spot/futures Current exchange trading rules and symbol information 60s (see comments) exchangeInfo is fetched periodically via REST every 60 seconds. It is not a websocket endpoint but just being cached during runtime.

    🚨 Every other REST query to an endpoint is being forwarded 1:1 to the API at https://api.binance.com !

    ⚙️ Commands & Options

    The following parameters are available to control the behavior of binance-proxy:

    binance-proxy [OPTION]
    Option Environment Variable Description Type Default Required?
    -v $BPX_VERBOSE Sets the verbosity to debug level. bool false No
    -vv $BPX_VERBOSE Sets the verbosity to trace level. bool false No
    -p $BPX_PORT_SPOT Specifies the listen port for SPOT market proxy. int 8090 No
    -t $BPX_PORT_FUTURES Specifies the listen port for FUTURES market proxy. int 8091 No
    -c $BPX_DISABLE_FAKE_CANDLES Disables the generation of fake candles, when not yet recieved through websockets. bool false No
    -s $BPX_DISABLE_SPOT Disables proxy for SPOT markets. bool false No
    -f $BPX_DISABLE_FUTURES Disables proxy for FUTURES markets. bool false No
    -a $BPX_ALWAYS_SHOW_FORWARDS Always show requests forwarded via REST even if verbose is disabled bool false No

    Instead of using command line switches environment variables can be used, there are several ways how those can be implemented. For example .env files could be used in combination with docker-compose.

    Passing variables to a docker container can also be achieved in different ways, please see the documentation for all available options here.

    🐞 Bug / Feature Request

    If you find a bug (the proxy couldn't handle the query and / or gave undesired results), kindly open an issue here by including a logfile and a meaningful description of the problem.

    If you'd like to request a new function, feel free to do so by opening an issue here.

    💻 Development

    Want to contribute? Great!🥳

    To fix a bug or enhance an existing module, follow these steps:

    • Fork the repo
    • Create a new branch (git checkout -b improve-feature)
    • Make the appropriate changes in the files
    • Add changes to reflect the changes made
    • Commit your changes (git commit -am 'Improve feature')
    • Push to the branch (git push origin improve-feature)
    • Create a Pull Request

    🙏 Credits

    ⚠️ License

    binance-proxy is free and open-source software licensed under the MIT License.

    By submitting a pull request to this project, you agree to license your contribution under the MIT license to this project.

    🧬 Third-party library licenses

    binance-proxy's People

    Contributors

    adrianceding avatar nightshift2k avatar

    Stargazers

     avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

    Watchers

     avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

    binance-proxy's Issues

    Having issues to configure binance-proxy over freqtrade in docker? (Docker networking problems)

    Describe your environment

    • Operating system: Debian 10
    • Python Version: 3.8.10 (python -V)
    • CCXT version: 1.59.2 (pip freeze | grep ccxt)
    • Freqtrade Version: 2021.10 (freqtrade -V or docker-compose run --rm freqtrade -V for Freqtrade running in docker)

    Your question

    Well, I'm trying to understand docker networking, because I'm having problems communicating two services (freqtrade + binance-proxy) without luck...

    I created two containers for this, the proxy with a static IP. The gateways are the following ones:

    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_1 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.23.0.1"
    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_2 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.24.0.1"
    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_3 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.25.0.1"
    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_4 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.26.0.1"
    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_5 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.27.0.1"
    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_6 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.28.0.1"
    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_7 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.29.0.1"
    root@DESKTOP-74LHUDF:/mnt/e/Python/freqtrade-docker# docker network inspect bot_8 | grep '"Gateway"' | head -n 1
                        "Gateway": "172.30.0.1"
    

    The proxy has the following number: 172.x.0.2. (Freqtrade I cannot know, continue reading and you will know why)

    I can reach the internet using docker exec -it <container_name> ping google.com, but I cannot reach a proxy created using binance-proxy although I can ping it, neither I cannot httping it nor from the host machine or from freqtrade container.

    I already tried several things but I cannot see where is the problem...

    I used docker inspect and docker network inspect, also I tried to execute docker exec -it ... /bin/bash to check ifconfig and iptables without luck, ping works correctly:

    exec ping

    It's working for only the first bot, I don't know if I used a command before to enable networking (on the first bot, because it works)

    I don't know why I can't ping on the docker container but at least the proxy is reaching google DNS.

    Also I tried static IPs without luck because gateway was missing on the docker inspect command:

    docker inspect


    This is how I run it:

    bash file

    #!/bin/bash
    
    cd /mnt/e/Python/freqtrade-docker/bot_2
    docker run -w="/mnt/e/Python/freqtrade-docker/bot_2" -d --name proxy_2 --net bot_2 -p 8092:8092 nightshift2k/binance-proxy:latest
    docker-compose up -d

    This is my docker-compose.yml file:

    docker-compose.yml

    ---
    version: '3'
    services:
      freqtrade:
        image: freqtradeorg/freqtrade:stable
        restart: unless-stopped
        container_name: freqtrade_2
        volumes:
          - "./user_data:/freqtrade/user_data"
        # Expose api on port 8082 (localhost only)
        # Please read the https://www.freqtrade.io/en/stable/rest-api/ documentation
        # before enabling this.
        ports:
          - "127.0.0.1:8082:8082"
          # - "0.0.0.0:8082:8082"
        networks:
              - default
        # Default command used when running `docker compose up`
        command: >
          trade
          --logfile /freqtrade/user_data/logs/freqtrade.log
          --db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
          --config /freqtrade/user_data/config.json
          --strategy xxx
    networks:
      default:
        external: true
        name: bot_2
    

    This is my docker-compose.yml for the bot... I also tried networks, 0.0.0.0 with http://172.x.0.2:8092/api/v3 instead of using localhost ip (127.0.0.1)


    And also I configured correctly Freqtrade config as stated here:

    config.json

    Maybe the problem is with the proxy, but I cannot debug correctly the bot because it's ending every time it tries to connect...

    crashes

    (Just see what does on the status column)

    This exception causes the program to crash and restart:

    log exception

    log

    Is there any way to prevent the bot to restart every time it fails?

    Also, what can I try more to see where is the problem? Because I have one bot working but the other ones cannot start and the config I'm using is the same. So what's going here?

    I inspected the two proxy + freqtrade containers and I don't find any differences...

    configs.zip

    The only thing is that when I change the cctx json config to the gateway IP it's not having any problems (in the first bot) (in this case 172.x.0.1:809x/api/v3). But in the rest of the bots I'm having this error:

    But if I do the same on the second bot, I'm having this other problem: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

    connetion error

    Hi, I got this message:

    level=error msg="SPOT XTZUSDT@5m kline websocket connection error: websocket: close 1006 (abnormal closure): unexpected EOF connected."

    How can i fix it? proxy could stop working due to this error?
    Thanks.

    Replace api.binance.com to api.binance.us

    Hi There,

    I would like to use proxy for Binance US. I have replaced a hostname, but it works for USDT, but not for USD.

    pnmice@f27b938

    proxy log
    level=debug msg="SPOT HBARUSD@5m kline websocket connected."

    bot log
    freqtrade.exchange.common - WARNING - fetch_l2_order_book() returned exception: "Could not get order book due to RequestTimeout. Message: binanceus GET http://binance-proxy:8090/api/v3/depth?symbol=HBARUSD&limit=5"
    freqtrade.exchange.common - WARNING - retrying fetch_l2_order_book() still for 4 times

    Not working with USD
    curl 'https://172.29.0.11:8090/api/v3/depth?symbol=HBARUSD&limit=5'
    it's stuck

    If I use USDT, it works
    curl -I 'http://172.29.0.11:8090/api/v3/depth?symbol=HBARUSDT&limit=5'
    HTTP/1.1 200 OK
    Content-Type: application/json
    Data-Source: websocket

    Good
    curl -I 'https://api.binance.us/api/v3/depth?symbol=HBARUSD&limit=5'
    HTTP/2 200

    Could you give a hint? Are there some limitations in code for USD tickers? Thanks!

    Restarting every one minute

    The contrainer restarts every 1 minute. The log is saying
    level=fatal msg="unknown flag e' - unknown flag"`

    I use this

      exchange-proxy:
        image: nightshift2k/binance-proxy:latest
        restart: unless-stopped
        container_name: exchange-proxy
        command: -verbose 0
    

    one my docker-compose.yml

    A way to contact you

    Hello,

    I would like to know if there any chance ti contact you I would like to understand a little bit more.

    binance-proxy keeps getting banned by binance every time it starts

    I'm using binance-proxy 1.2.4 as a proxy for 3 freqtrade bots that run on a lot of pairs on multiple timeframes on futures.
    When I restart binance-proxy, I get banned by binance. There's no direct link between the bots and binance, they all go through binance-proxy.

    time="2022-08-15T08:19:57Z" level=error msg="FUTURES C98USDT@5m kline initialization via REST failed
    error: code=-1003, msg=Way too many requests; IP() banned until 16605565089
    Please use the websocket for live updates to avoid bans.."

    I have no ideea why binance-proxy would get banned as I think it's its responsibility to ratelimit the queries.

    binance proxy kill program

    Hi,

    Will try to reproduce with verbose mode

    time="2021-11-29T21:16:22Z" level=info msg="SIGINT received, aborting ..."

    got this error message and then proxy is killed.

    Have a nice day.

    Edit : strange behavior seems like it dosent apply docker restart policy -> i use restart always but it don't work

    Recommend Projects

    • React photo React

      A declarative, efficient, and flexible JavaScript library for building user interfaces.

    • Vue.js photo Vue.js

      🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

    • Typescript photo Typescript

      TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

    • TensorFlow photo TensorFlow

      An Open Source Machine Learning Framework for Everyone

    • Django photo Django

      The Web framework for perfectionists with deadlines.

    • D3 photo D3

      Bring data to life with SVG, Canvas and HTML. 📊📈🎉

    Recommend Topics

    • javascript

      JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

    • web

      Some thing interesting about web. New door for the world.

    • server

      A server is a program made to process requests and deliver data to clients.

    • Machine learning

      Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

    • Game

      Some thing interesting about game, make everyone happy.

    Recommend Org

    • Facebook photo Facebook

      We are working to build community through open source technology. NB: members must have two-factor auth.

    • Microsoft photo Microsoft

      Open source projects and samples from Microsoft.

    • Google photo Google

      Google ❤️ Open Source for everyone.

    • D3 photo D3

      Data-Driven Documents codes.