textileio / bidbot Goto Github PK

View Code? Open in Web Editor NEW

31.0 5.0 7.0 762 KB

A Filecoin Network sidecar for storage providers to bid in storage deal auctions.

License: MIT License

Makefile 4.07% Shell 0.55% Dockerfile 0.38% Go 95.00%

filecoin filecoinminer

bidbot's People

Contributors

Stargazers

Watchers

Forkers

cerblue sealstorage magik6k isabella232 xrayzx pinkdiamond1

bidbot's Issues

Lotus and bidbot displays error 'Filecoin.SectorsSummary' not found

Running Lotus v1.16.1 and Boost 1.2.0 along with CID-Gravity integration, I am seeing these errors very often.
I assume this is due to a change in Boost or Lotus.

Here is my bidbot run command:
nohup bidbot daemon --storage-provider-id f01278 --cid-gravity-key <token redacted> --sealing-sectors-limit 8 >> /market/logs/bidbot.log 2>&1 &

Boost log

2022-07-17T03:34:49.420Z ERROR rpc [email protected]/server.go:95 RPC Error: method 'Filecoin.SectorsSummary' not found

Bidbot log

2022-07-17T03:34:49.420Z ERROR bidbot/service service/service.go:340 fail to get number of sealing sectors, continuing: getting lotus sectors summary: RPC error (-32601): method 'Filecoin.SectorsSummary' not found

bidbot version

version: git
build date: 2022-06-28T12:08:11Z
git summary: v0.1.14-7-ge68a7ee
git branch: main
git commit: e68a7ee
git state: clean

Clarify permission of api-token

In the CLI, maybe clarify what kind of permission the --lotus-miner-api-token string Lotus miner API authorization token token needs. If it's read, write, sign or admin 😃

Transaction Conflict in tests

This happens in tests here as well as the auction system CI

setting proposal cid: committing txn: Transaction Conflict. Please retry

It's caused by a race between txn.Commit at https://github.com/textileio/bidbot/blob/main/service/store/store.go#L372 and the call to s.saveAndTransitionStatus in the fetch worker https://github.com/textileio/bidbot/blob/main/service/store/store.go#L615 which triggered by sending to jobCh https://github.com/textileio/bidbot/blob/main/service/store/store.go#L536

In reality it shouldn't be a problem because it takes some time to download and import data, but the mocked calls return immediately, allow them to run nearly the same time.

Bidbot failed to import deals to lotus-miner

Installed bidbot this morning and saw some deals coming in. It's great!!!

However, Bidbot reported the following errors:

2021-08-14T17:02:24.706-0700    INFO    bidbot/store    store/store.go:589      importing /media/data/bidbot/bafybeig2ohx5vgy3zkjtbfzmuc6frlt6f7j27wel2xaduxun4y7pf5prmy_01fd3az5ef5s8w8jta7fk147nn to lotus with proposal cid bafyreiemvqoyyfzxnooafx62cqa736frakosc3wzxn4pudv2caegnosc6i
2021-08-14T17:02:24.709-0700    INFO    bidbot/store    store/store.go:602      finished fetching data cid 01fd3az5ef5s8w8jta7fk147nn (status=queued_data error=calling storage miner deals import data: failed to open file: open /media/data/bidbot/bafybeig2ohx5vgy3zkjtbfzmuc6frlt6f7j27wel2xaduxun4y7pf5prmy_01fd3az5ef5s8w8jta7fk147nn: no such file or directory)
2021-08-14T17:02:24.709-0700    INFO    bidbot/store    store/store.go:574      fetching sources {url: https://cargo.web3.storage/deal-cars/bafybeibh4odehtgqkebzql57vavtofdx4mzecnfrzbtabnftlkpzecowte_baga6ea4seaqhkrbvrnybfv45tb47ow6exo7bggmk3nyufhquwsscsq3ajzc6wha.car,}

I went to the directory and checked, the file doesn't exist.

Searching the file in miner logs, found that miner also reported the file doesn't exist:

2021-08-14T17:02:06.109-0700    INFO    miner   miner/miner.go:478      completed mineOne       {"tookMilliseconds": 17, "forRound": 1022645, "baseEpoch": 1022644, "baseDeltaSeconds": 6, "nullRounds": 0, "lateStart": false, "beaconEpoch": 1118489, "lookbackEpochs": 900, "networkPowerAtLookback": "10460691787091902464", "minerPowerAtLookback": "296846329282560", "isEligible": true, "isWinner": false, "error": null}
2021-08-14T17:02:17.157-0700    INFO    net/identify    identify/id.go:376      failed negotiate identify protocol with peer    {"peer": "12D3KooWPXd8KSfgy5s6qR6RvMPoXm5gErHFGAuNmgVBqPR61ZZA", "error": "stream reset"}
2021-08-14T17:02:24.706-0700    WARN    rpc     [email protected]/handler.go:279  error in RPC call to 'Filecoin.MarketImportDealData': failed to open file:
    github.com/filecoin-project/lotus/node/impl.(*StorageMinerAPI).MarketImportDealData
        /home/filecoin/filecoin/lotus/node/impl/storminer.go:399
  - open /media/data/bidbot/bafybeig2ohx5vgy3zkjtbfzmuc6frlt6f7j27wel2xaduxun4y7pf5prmy_01fd3az5ef5s8w8jta7fk147nn: no such file or directory

Lotus-miner version:

Daemon:  1.11.1-rc2+mainnet+git.40449f1cc+api1.2.0
Local: lotus-miner version 1.11.1-rc2+mainnet+git.40449f1cc

Bidbot setup:
lotus fullnode, lotus-miner, bidbot are all running separate machines.

both bidbot and lotus-miner have access to a shared nfs path:
/media/data/bidbot

in the bidbot machine, this env is set: BIDBOT_DEAL_DATA_DIRECTORY=/media/data/bidbot

BIDBOT have error with CID GRAVITY for the last 24-48 Hours.
"ERROR bidbot/pricing pricing/cid_gravity.go:157 loading rules from API: unmarshalling rules: json: cannot unmarshal string into Go struct field .PricingRules.Verified of type bool"
seems like BIDBOT can't parse API response.
Create ticket here and for CID Gravity.
can work bidbot with this error or all bids will be rejected?

FR: add standardized way to provide pricing info to bidbot from thirdparty tools

Currently bidbot is only able to connect to cid-gravity.

To allow other solutions to provide pricing info a standardized way is needed these solutions can adapt to provide pricing information to bidbot.

Print specific errors when bid is rejected

ERROR sounds a bit alarmist for simple not winning an auction - it would be better to use “INFO” to prevent undue concern.
2021-08-19T19:42:40.150Z    ERROR  bidbot/service service/service.go:328 making bid: publishing bid; auctioneer 12D3KooWGDBaVz45c5d9VEtF4eM7Pgj71DSzB3HHAfpjc8fb5EGe returned error: bid rejected

Being rejected is different from not winning, so it's better if the reason can be well communicated. And there should be an INFO for not winning.

Another GC / Symlink error

NVM :) being dumb

Add a mechanism to cancel deal / clean up the queue

I have received a lot of bids I could not accomplish because of networking problems. I have so far fixed that, but deal list is full and lagging behind, so all finished downloads fail on import because of missed deadline.

AFAIK there is no remove / cancel / cleanup command and I haven't found anything in the docs.

At the current moment for me the only way is to reinstall bidbod or wait for all the files in queue to fail ( but not after download completes )

Nerpa/Calibration Network support

Is it possible to test bidbot out on either nerpa or calibration networks, which I am guessing would require that there is a SB setup on either of those networks? Testing infrastructure configurations with sidecar applications in production is less than ideal.

installation error "quic-go doesn't build on Go 1.19 yet."，How to solve it?

CGO_ENABLED=0 go install -ldflags="-X github.com/textileio/bidbot/buildinfo.GitBranch=main -X github.com/textileio/bidbot/buildinfo.GitState=clean -X github.com/textileio/bidbot/buildinfo.GitSummary=v0.1.14-7-ge68a7ee -X github.com/textileio/bidbot/buildinfo.Version=git -X github.com/textileio/bidbot/buildinfo.BuildDate=2022-11-06T04:26:20Z -X github.com/textileio/bidbot/buildinfo.GitCommit=e68a7ee" .

github.com/lucas-clemente/quic-go/internal/qtls

/root/go/pkg/mod/github.com/lucas-clemente/[email protected]/internal/qtls/go119.go:6:13: cannot use "quic-go doesn't build on Go 1.19 yet." (untyped string constant) as int value in variable declaration
make: *** [Makefile:21：install] error 2
root@chaiwenyu-System-Product-Name:/opt/bidb

deal_data folder symlink gets removed

I'm not sure how to debug this, but at one point my ~/.bidbot/deal_data symlink got removed and it turned back into a regular folder..
Trying to reproduce it now.

[Feature Request] Ability to change the URI Fetch Timeout through the config file

For Storage Provider with lower bandwidth, it would be great to be able to change the download timeout (DataURIFetchTimeout) through the configuration file.

Add terms ack and onboarding to bidbot setup steps

It would be nice that while auctions are still experimental, we have a better line of contact with providers running the network. Additionally, bidbot is providing enough benefit to providers at this point in time, that we should also ask them to commit to providing free retrieval for all deals they store from the system. So:

Update Readme.

I think we can more or less copy the language from here: https://github.com/filecoin-project/filecoin-discover-dealer#terms-and-eligibility

Update bidbot setup

It would be nice if bidbot required any provider to do 2 things before they submit their first bid.

Ack a terminal print out of the above terms.
Submit an email address in the terminal that we can collect and use for technical communication (probably requiring confirmation that we can use it that way at the same time)

[Feature Request] Ability to stop the deamon via CLI

Since I'm running bidbot daemon with nohup, having the ability to gently stop the daemon with a command like bidbot daemon stop, would be quite nice 😄

bidbot doing nothing, files stalled

I just restarted bidbot again, and the logs show nothing special;

cat /var/log/lotus/bidbot.log
{"severity":"info","ts":"2021-11-03T20:02:34.454Z","logger":"bidbot","caller":"bidbot/main.go:319","msg":"bidbot \tversion:\tgit\n\tbuild date:\t2021-10-30T12:35:11Z\n\tgit summary:\tv0.1.5\n\tgit branch:\tmain\n\tgit commit:\t2b4218e\n\tgit state:\tclean"}
{"severity":"info","ts":"2021-11-03T20:02:34.454Z","logger":"bidbot","caller":"bidbot/main.go:341","msg":"loaded config from /home/master/.bidbot/config: {\"announce-multiaddr\":[],\"ask-price\":4e-9,\"bootstrap-multiaddr\":[\"/ip4/34.83.3.108/tcp/4001/p2p/12D3KooWGDBaVz45c5d9VEtF4eM7Pgj71DSzB3HHAfpjc8fb5EGe\",\"/ip4/34.83.3.108/udp/4001/quic/p2p/12D3KooWGDBaVz45c5d9VEtF4eM7Pgj71DSzB3HHAfpjc8fb5EGe\",\"/ip4/34.105.101.67/tcp/4001/p2p/12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD\",\"/ip4/34.105.101.67/udp/4001/quic/p2p/12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD\",\"/ip4/34.83.36.118/tcp/4001/p2p/12D3KooWQSf4SMyWPSqLN23KxhcLWYhshWb34pYv65cr85jGpNrR\",\"/ip4/34.82.221.249/tcp/4001/p2p/12D3KooWBGyJbDmjjvEzfsgb3nE9JsfcYRqPxpnQagxbb1PyxBrb\",\"/ip4/34.83.88.62/tcp/4001/p2p/12D3KooWHpxr8BTd3R6kAqvtfn77PKW7WRqJ4cbnrT59K2rU44WM\"],\"cid-gravity-key\":\"***\",\"cid-gravity-strict\":false,\"concurrent-imports-limit\":0,\"conn-grace\":120000000000,\"conn-high\":512,\"conn-low\":256,\"deal-data-fetch-attempts\":3,\"deal-data-fetch-timeout\":\"3h\",\"deal-duration-max\":1468800,\"deal-duration-min\":525600,\"deal-size-max\":34359738368,\"deal-size-min\":57344,\"deal-start-window\":1200,\"discard-orphan-deals-after\":86400000000000,\"fake-mode\":false,\"fast-retrieval\":true,\"group\":\"\",\"http-port\":\"9999\",\"json\":true,\"listen-multiaddr\":[\"/ip4/0.0.0.0/tcp/4001\",\"/ip4/0.0.0.0/udp/4001/quic\"],\"log-debug\":false,\"log-json\":true,\"log-plaintext\":false,\"lotus-api-conn-retries\":\"2\",\"lotus-gateway-url\":\"https://api.node.glif.io\",\"lotus-market-api-maddr\":\"\",\"lotus-market-api-token\":\"***\",\"lotus-miner-api-maddr\":\"/ip4/172.16.0.1/tcp/2345/http\",\"lotus-miner-api-token\":\"***\",\"mdns\":false,\"miner-addr\":\"f01392893\",\"nat\":false,\"private-key\":\"***\",\"quic\":false,\"running-bytes-limit\":\"1 tb/24h\",\"sealing-sectors-limit\":700,\"status\":\"\",\"storage-provider-id\":\"f01392893\",\"user\":\"\",\"verified-ask-price\":0,\"wallet-addr-sig\":\"***\"}"}
{"severity":"info","ts":"2021-11-03T20:02:35.079Z","logger":"bidbot/lotus","caller":"lotusclient/lotusclient.go:112","msg":"MarketListIncompleteDeals call took 618.817584ms"}
{"severity":"info","ts":"2021-11-03T20:02:35.079Z","logger":"bidbot","caller":"bidbot/main.go:691","msg":"limit total running bytes to 1000000000000 bytes over 24h0m0s"}
{"severity":"info","ts":"2021-11-03T20:02:35.103Z","logger":"psrpc/peer","caller":"peer/peer.go:154","msg":"marketpeer 12D3KooWDKfYWp8dyRvVgqcwHc8ys1ZNs8nEpGGBcw36VbPJJrqJ is online"}
{"severity":"info","ts":"2021-11-03T20:02:35.560Z","logger":"bidbot/lotus","caller":"lotusclient/lotusclient.go:112","msg":"MarketListIncompleteDeals call took 454.529744ms"}
{"severity":"info","ts":"2021-11-03T20:02:36.579Z","logger":"bidbot/service","caller":"service/service.go:251","msg":"service started"}
{"severity":"info","ts":"2021-11-03T20:02:36.768Z","logger":"psrpc/peer","caller":"peer/peer.go:211","msg":"peer was bootstapped"}
{"severity":"info","ts":"2021-11-03T20:02:36.768Z","logger":"bidbot/service","caller":"service/libp2p_pubsub.go:131","msg":"subscribed to the deal auction feed"}
{"severity":"info","ts":"2021-11-03T20:02:36.768Z","logger":"bidbot/api","caller":"httpapi/httpapi.go:45","msg":"http server started at :9999"}
{"severity":"info","ts":"2021-11-03T20:07:08.089Z","logger":"bidbot/service","caller":"service/service.go:315","msg":"auction details:\n{\n  \"id\": \"01fkkpk2mk4ndy20xeq6z8ra3g\",\n  \"payload_cid\": \"bafybeighevg5x246olzuzgcct4j3adg47w32pxwix7knnnofnaqugxo3ji\",\n  \"deal_size\": 34359738368,\n  \"deal_duration\": 1468800,\n  \"fil_epoch_deadline\": 1259774,\n  \"sources\": {\n    \"car_url\": {\n      \"URL\": \"https://cargo.web3.storage/deal-cars/bafybeighevg5x246olzuzgcct4j3adg47w32pxwix7knnnofnaqugxo3ji_baga6ea4seaqo2w6ikje6zfgst4exeaky23qm7qm4pqio64kbujinden2b3shipa.car\"\n    }\n  },\n  \"ends_at\": {\n    \"seconds\": 1635970058,\n    \"nanos\": 21169966\n  },\n  \"client_address\": \"f144zep4gitj73rrujd3jw6iprljicx6vl4wbeavi\"\n}"}
{"severity":"info","ts":"2021-11-03T20:07:09.002Z","logger":"bidbot/lotus","caller":"lotusclient/lotusclient.go:112","msg":"MarketListIncompleteDeals call took 453.919482ms"}
{"severity":"info","ts":"2021-11-03T20:07:09.465Z","logger":"bidbot/pricing","caller":"pricing/cid_gravity.go:184","msg":"loading rules from API took 314.279983ms"}
{"severity":"info","ts":"2021-11-03T20:07:09.466Z","logger":"bidbot/service","caller":"service/service.go:357","msg":"pricing engine result valid for auction 01fkkpk2mk4ndy20xeq6z8ra3g?: true, details: {AllowBidding:false UnverifiedPriceValid:false UnverifiedPrice:0 VerifiedPriceValid:false VerifiedPrice:0}"}

Yet there's a bunch of files in the deal_data folder which seem to be stalled;

total 1.8T
drwxrwxr-x  2 master master  12K Nov  3 20:12 .
-rw-rw-r--  1 master master  21G Nov  3 09:25 bafybeigtctpv6kpgxigojynuroromvpcj6kxyzv4cmra5bw3gkynaunjo4_01fkjgrfzcze7bnt7ktj7rvhy5
-rw-rw-r--  1 master master  30G Nov  3 09:19 bafybeie4yoofzscufymdvojxrfbm5kncj5nsz5p3sfcrljacwpsx44543q_01fkjg0w0gb4xpkyaas1e118mt
-rw-rw-r--  1 master master  23G Nov  3 09:01 bafybeif7kymjnrstnh3wxy43lwy4bydbbz5s5xpep6pwpl5vac4den5ghu_01fkjf2xv5t64jt18nb1h5vvxw
-rw-rw-r--  1 master master  25G Nov  3 08:55 bafybeie7m62ibw2j6zmfxp5nnfkegdodserri74ayaxbj3mbxbxtyvan5a_01fkje7e5kw5177vmfj47gfzvq
-rw-rw-r--  1 master master  20G Nov  3 08:40 bafybeibfn6ra2dlznvfiu5ydarob5x3gfve7hwsc6cjkfoiiwlhj57tdny_01fkjdxz88w27598655707c64t
-rw-rw-r--  1 master master  21G Nov  3 08:33 bafybeigs5u2tctg2lpajqnezwnuh2viqhy3xtuto7wta42a2jphaecaova_01fkjdctf5pnajdfx69y898eyn
-rw-rw-r--  1 master master  21G Nov  3 08:22 bafybeichgaotuheoo7fcym7kllsfax33yeishbxbrwcgvlejrbvwzkxq7m_01fkjctg2begdgw5mfdfa54z41
-rw-rw-r--  1 master master  21G Nov  3 07:59 bafybeig6p7t4qoeidgrvo2zatsq3yftgso7pjwewypo2swbrvoygbowq2y_01fkjbmyrhsbp39vv0xbbq8qhr
-rw-rw-r--  1 master master  21G Nov  3 07:52 bafybeidpkkb4onzyfy5xghkgnaakyc5bmuktdlvrrif4v6d63heuapt5ei_01fkjatbt4r7dgvfcwe6z9q177
-rw-rw-r--  1 master master  32G Nov  3 07:34 bafybeibtnl6uerfdp2edfanv5ayqqyuor5zsmcgvmdnva4xftuoi4gnk4y_01fkj948g8437cmt1hc3mw5pq5
-rw-rw-r--  1 master master  21G Nov  3 07:33 bafybeibycryjmhy2rjv2ncewakpyfi5karafst4pbc2mzrnmg362kdykf4_01fkj9wdavmfh8wwshda15hgvr
-rw-rw-r--  1 master master  26G Nov  3 07:24 bafybeidpmrjjemn72wwlvmtyashwdtuc4dl5gp54egk67iwgktfivhctim_01fkj92d4wcgez2xp02rn6b8vc
-rw-rw-r--  1 master master  24G Nov  3 07:01 bafybeih73ymmvp3aibipph6i62vv356lfkgl6x7mto2ydz57nffsj66ria_01fkj85das83dnj57hww2h5m0c
-rw-rw-r--  1 master master  24G Nov  3 06:52 bafybeifn5iprkayex6d3pltfxfl4fsi2k3ahuje2jyehmgmhufwxbefbcu_01fkj7hjf1tvqe8zj3gpd13m0z
-rw-rw-r--  1 master master  26G Nov  3 06:38 bafybeibwj3gfyzs4s746icytkcoocg4xdsreb7yj5hw5lhqy7dfuwl652e_01fkj6ebq6svtfpt7g2d64p71f
-rw-rw-r--  1 master master  27G Nov  3 06:23 bafybeidqsfelbbi6ouxeuk27ppe4auxsvhmkieyknppozty3vxcxqayzpe_01fkj5p0dnsrx1wprtxte60h7t
-rw-rw-r--  1 master master  31G Nov  3 06:08 bafybeiaxhndwzd6notby3wrz3shxkvege7ewkya3jmtms6d26jom72h4ae_01fkj5xm9er4dq8534nhebnyqq
-rw-rw-r--  1 master master  25G Nov  3 05:57 bafybeigp67cltya2nzbah4p37i7teiswms4klz3xtfdjui5wl54xcasfym_01fkj4wasjbc99d8xw73gejg3w
-rw-rw-r--  1 master master  23G Nov  3 05:30 bafybeibjupbc43d3oepxxmo52ih4nuleepj6tahnwn4pcxetrtuuvg5qsi_01fkj30axxdba3cnmtbrebtve7
-rw-rw-r--  1 master master  25G Nov  3 05:24 bafybeign6orhkudpbon5hpej5y6fbd4lnwv57wx2e7yta2jhdmvh3mn564_01fkj2ky55rjjv024aa8n2x43r
-rw-rw-r--  1 master master  30G Nov  3 05:22 bafybeifzcgobadnptmbdtgwenns5efjjqqfgcsqpltddhbmfa2vfhwf4ou_01fkj296702tkbmxq1nver0m92
-rw-rw-r--  1 master master  24G Nov  3 04:45 bafybeib7knpefe73xztvjajl6su2twxwevbx45lcrzjn5j4n4yl2t7c6km_01fkj1chjyr50wrgk83j75ecb3
-rw-rw-r--  1 master master  26G Nov  3 04:29 bafybeievwsqk4lcw2fgocosq424fdylnnxz3zxlvdqlm5ijwdapbleacg4_01fkj0dr0m5xjm2h10w2mybjrb
-rw-rw-r--  1 master master  32G Nov  3 04:20 bafybeih3ibsb5q2zc2f772ptch3dlm2vkplqnjsact3aq4devxmes3werq_01fkhzq096qvfgcrpr80dtct0w
-rw-rw-r--  1 master master  16G Nov  3 03:58 bafybeia4ipunfeylgzawpfrk27ipjv4gk4toe7g3liiqxkeqkqxa4dmjae_01fkhyprqna5qhxs68djhe5231
-rw-rw-r--  1 master master  28G Nov  3 03:57 bafybeiflfvthxfeizyalpgkrszo37svu6hzagbiobbw6sx6o7gthac6ddm_01fkhyc9xs2w1pntf11kj6fds6
-rw-rw-r--  1 master master  32G Nov  3 03:48 bafybeiciconuoskg2eiquwjyo6dlx6hj7vgp4ovfpjznh3wgjgsyishdsu_01fkhxv5hzpmwgsqf5t9yjpscm
-rw-rw-r--  1 master master  24G Nov  3 03:30 bafybeic4htnyipgnmqxtv3gpcc544le6p563yuhmsgnplzdoa4ynwokyzm_01fkhwzp8yd06zbg6e02te3ayj
-rw-rw-r--  1 master master  25G Nov  3 03:16 bafybeibxpuiq6zripfdbrzpyra5r4f3czxsii77m5tn3p4rw7tjoq2pn2u_01fkhw46cv1th83221akd66mdj
-rw-rw-r--  1 master master  15G Nov  2 13:37 bafybeiezu7jl2s5m6qvficq5va6p5n7xet5ehr6v5pyqwwscarhe5hekte_01fkfv18x4x8y9nrnv75f9xcd0
-rw-rw-r--  1 master master  19G Nov  2 08:13 bafybeiadyrw74ixov77qkls7vixa3kyujwwr4ctpc2jiv47cf6mymamtsq_01fkftffvzaa6s8djfw45edx63
-rw-rw-r--  1 master master  26G Nov  2 08:08 bafybeigs2fpewvo5sdqp57jcrik4f6dtwfryvefgejaskk6rlg4g2cjxjm_01fkfth2cgbaa74394f14g5ptr
-rw-rw-r--  1 master master  32G Nov  2 08:07 bafybeiad6br7apkwkroygzapt6fyh2srvhu6lggbhmpioty7hnooulvhe4_01fkfsjgp4aq1bjcrb8mbpcqtj
-rw-rw-r--  1 master master  31G Nov  2 07:56 bafybeie4rngzw7sokov2lhztatahe2rach6syttkbvc5ltbwmromd7wr5m_01fkfs7eqh90e5ysm6eaksbqe4
-rw-rw-r--  1 master master  30G Nov  2 07:47 bafybeicxe6cyx6ptckpgc6zpaj6egd44mhqw5rlm5lweltuounb6uxggfm_01fkfrrhs0hc12n75sy5m5cmtb
-rw-rw-r--  1 master master  32G Nov  2 07:36 bafybeihtxvsq47wjybmlko5pmh4d56ntjeohu3hcn5h5tfy4uqqi3n2u6m_01fkfr8wkah9npscbrbm45xg4h
-rw-rw-r--  1 master master  32G Nov  2 07:07 bafybeihi5hyalhuim5qawyszgamysvlbmw5i4yyesgowb3i7tlmxnqbqhu_01fkfnpynkbs3qbd6zf6aj5esp
-rw-rw-r--  1 master master  32G Nov  2 07:03 bafybeiffhxc3s4g5hy7mu4reacpscfwwazhyknnfdmlvwradjxeb77mfa4_01fkfnnexw1t96z42b19cx29y9
-rw-rw-r--  1 master master  27G Nov  2 06:55 bafybeidqolrw2u4wxbzgw3rcnwyfhb5im4grq34h4hgf6wh53ztxcqp5ba_01fkfmwasf11gn6c5w1p8bqxqc
-rw-rw-r--  1 master master  30G Nov  2 06:48 bafybeiadkkcggreyh6jgnqejbojoaqojgd3mkmw2ceqw3lvp2pj4m7krgi_01fkfmk62wnfrrth9hcg16bcgh
-rw-rw-r--  1 master master  30G Nov  2 06:40 bafybeiesqwzpwzxwmkzpyhosbkv3q5u65mx35abt63zqvwom2epm7wem7q_01fkfmbzc9zff7dcdqcmef0f6v
-rw-rw-r--  1 master master  27G Nov  2 06:33 bafybeihi57l6ejwi77vjzj6rdt3xoddc7rcyyop5vrokjb6v5qxj63toru_01fkfkzty2vjjtarhz1hzbwd44
-rw-rw-r--  1 master master  32G Nov  2 06:09 bafybeicf5763fastfcrmb2xam46a6p2i6axprzsdzwcgh6a5gwajxlcgla_01fkfkrbrajj0ays7dqgy8h8zq
-rw-rw-r--  1 master master  31G Nov  2 05:49 bafybeifubqh44kdnw63ihev4hfmuthy3x4xfkdzlolwnimkhdjvzrmtv7e_01fkfhmyyejjqfw9xk3mdr721h
-rw-rw-r--  1 master master  29G Nov  2 05:32 bafybeifb5qnm6d3acthjzmyyo7fge3z6dxai4bkt7qaxzbxxzqr74rypae_01fkfgnxdbkkrbt60hb4wfk5ak
-rw-rw-r--  1 master master  28G Nov  2 05:17 bafybeidacltsyubmjdh4wjwcg35z7ke24fovomeuezk7prj6vw74rh4vma_01fkffncahj1sdrb7q801xwq7s
-rw-rw-r--  1 master master  32G Nov  2 05:15 bafybeifr6ej64ajzrpa6ita3urqt2sjoe6ojcfpnyfr4xhyhn62bjvwixe_01fkff6qgpt3h9sq225b0d91kc
-rw-rw-r--  1 master master  30G Nov  2 04:35 bafybeihflqh3zppx24inolgsyknfnjqcta5s4f74julp2wugiv2sisr33a_01fkf9yw07wbhm3hmm9b8dgb93
-rw-rw-r--  1 master master  30G Nov  2 04:25 bafybeihmvwf6lptxpthk4tzrxztl6cl6dq5ybmdk4t4iztlxwekrdhgnwu_01fkf9v9q757y97fpj6www1ncn
-rw-rw-r--  1 master master  24G Nov  2 04:11 bafybeigsglehzur27m4hsfmxgm5ldsr7zn7w2yz2x6apivywlmghc7vhei_01fkf9djwzt7b2gfwj0pszrg99
-rw-rw-r--  1 master master  32G Nov  2 03:53 bafybeielycmao4o32ghipnklvyzbxx5b6hmx6na5mxxtinus5qap6swcl4_01fkf7e4sketq4b0v0zs64tkab
-rw-rw-r--  1 master master  27G Nov  2 03:37 bafybeid6ottb74zbvgainzysf77uyn7hezm5ivrpra2awklttvxkkbwzz4_01fkf46taqx937bfq765dh4fr0
-rw-rw-r--  1 master master  31G Nov  2 03:32 bafybeiavrxmxbsf3ygoubocar25chsjqikgts3aqgzt537mmygt7zln37q_01fkf3qhz26v9rcwrqhmygy8pd
-rw-rw-r--  1 master master  31G Nov  2 03:06 bafybeidn7zxobnm6ocnis4xp5s6nj3ecghfxqtq62f35ydr5lw2r67r7ja_01fkf0xmpbxah298dqycn6rncs
-rw-rw-r--  1 master master  32G Nov  2 02:59 bafybeifhgay6wr6wmzbk2u4q333otzc2ifbg6hyru6rdrgacoca7zqjdme_01fkf0nd333rj3510ta6kzas9q
-rw-rw-r--  1 master master  32G Nov  2 02:51 bafybeiggenrxj65eexfqrak3ro6snt64ijv4dxpqzh5vwxiqtvmfh22vju_01fkezpmczvp7hhh0schnnjmv4
-rw-rw-r--  1 master master  29G Nov  2 02:20 bafybeicmbycsbqxixhf646ncx4fhagcr2qqg4gadrrg2luoh5a2zfjlxpu_01fkezgz4x0ej9k6yfc8ejqdzg
-rw-rw-r--  1 master master  30G Nov  2 02:09 bafybeid36ox7ha6e4bwraepwkjy2elbdhwrdu7ixn6z2fpyfh6b6rsltau_01fkez29drs9c1d942vs5zpyay
-rw-rw-r--  1 master master  32G Nov  2 02:02 bafybeibgshxykdtbss3dggofw4cx3j63zqoe4yxmb27vdvtkaxygteiqii_01fkeyr5r1m72jec2fj7ev866d
-rw-rw-r--  1 master master  30G Nov  2 01:32 bafybeie5uuza34jj44gc3klyh74jpdzkfmbo772xsthtrfwjkabamiognu_01fkeyhb7eq0ykp4p9jpr66hje
-rw-rw-r--  1 master master  32G Nov  2 01:16 bafybeibvkah2lxq73iaajhatieznggv327p4spx4akqmiprg25vchlzbua_01fkeyapx3j5dkmma8rf6efhze
-rw-rw-r--  1 master master  30G Nov  2 01:12 bafybeibikijje4knqpyezsbaiqdhnv5yoahg3onh3tnlp76z273xrf433y_01fkey91fngjk4dr4nfpabsk35
-rw-rw-r--  1 master master  29G Nov  2 00:48 bafybeibz4phsxhxrjfvhpzfwh36mogbgdsvseyyx3fuvnakjfo47rizzvy_01fkewzpryt515g7jtj4fksw1j
-rw-rw-r--  1 master master  23G Nov  2 00:29 bafybeierwok5eqomh4gpinfgat2r4chq6dagvkxxqjagneosjfkh7dsxs4_01fkexw4c7fxnxx3dve692ppag
-rw-rw-r--  1 master master  31G Nov  2 00:25 bafybeic2qe3yaz237u53xaugfkcsijs3x5dehxnbmrbse33s25sxsmg3ta_01fkexn7fa096br0nqj42afbes
-rw-rw-r--  1 master master  31G Nov  2 00:05 bafybeiaqv5e4m2mk6zc237qph73xjez7vujuzw77pz5j6frkzsby6rtb3a_01fkewy7v156a3j34hgbfcfbra
-rw-rw-r--  1 master master  32G Nov  1 23:56 bafybeidskwu6jjr4pa2zt3grkc5fvm2b4sn5dcv3f4tnuqvansbh3xuv24_01fkewdbe3yesnynvy0p2qgxhj

The top one for example is this deal;

{
        "ID": "01fkjgrfzcze7bnt7ktj7rvhy5",
        "AuctionID": "01fkjgrdwzhvk1xdg52jh6pcej",
        "AuctioneerID": "12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD",
        "PayloadCid": {
                "/": "bafybeigtctpv6kpgxigojynuroromvpcj6kxyzv4cmra5bw3gkynaunjo4"
        },
        "DealSize": 34359738368,
        "DealDuration": 1468800,
        "ClientAddress": "f144zep4gitj73rrujd3jw6iprljicx6vl4wbeavi",
        "Sources": {
                "CARURL": {
                        "URL": {
                                "Scheme": "https",
                                "Opaque": "",
                                "User": null,
                                "Host": "cargo.web3.storage",
                                "Path": "/deal-cars/bafybeigtctpv6kpgxigojynuroromvpcj6kxyzv4cmra5bw3gkynaunjo4_baga6ea4seaqiatabkeg2ehng4fpj6rr3wr6urnowpy3tyysveqcswgysecneofq.car",
                                "RawPath": "",
                                "ForceQuery": false,
                                "RawQuery": "",
                                "Fragment": "",
                                "RawFragment": ""
                        }
                },
                "CARIPFS": null
        },
        "Status": 5,
        "AskPrice": 0,
        "VerifiedAskPrice": 0,
        "StartEpoch": 1255331,
        "FastRetrieval": true,
        "ProposalCid": {
                "/": "bafyreiahqvz4g66st7jdsq5kf56kq3fm3qthvbu3dsrbbhvymdjmpso7vu"
        },
        "DataURIFetchAttempts": 1,
        "CreatedAt": "2021-11-03T09:05:58.258399976Z",
        "UpdatedAt": "2021-11-03T09:28:34.305728628Z",
        "ErrorCause": ""
}

What is it doing? Should it not continue to download and then pack the data?

Clarify corresponding lotus conf var for deal-start-window

The README says:

You should set correctly your --deal-start-window value to be a bit larger than your miner node configuration, to cover the time required to download and import data. If this value is incorrectly set, your miner node will reject proposals because the DealStartEpoch attribute is too soon for you to accept.

it is not immediately clear what lotus config var this refers to. needs clarification.

bidbot version tries to read config and connect over http

./bidbot version
2022-01-11T10:34:46.264Z        ERROR   bidbot  bidbot/main.go:222      loading config from : Config File "config" Not Found in "[/home/filecoin/.bidbot]"
local   version:        git
        build date:     2022-01-11T10:32:46Z
        git summary:    v0.1.6-4-g8dff41e
        git branch:     main
        git commit:     8dff41e
        git state:      clean
2022-01-11T10:34:46.265Z        FATAL   bidbot  bidbot/main.go:466      Get "http://127.0.0.1:9999/version": dial tcp 127.0.0.1:9999: connect: connection refused

it should just print the version. urfav/cli has a dedicated mechanism for printing versions - that might be worth a look

Update contact details in the README

points to slack and slack then points to discord. should be directly pointing to discord

'invalid miner address or signature' in logs

I'm seeing a lot of invalid miner address or signature error messages in logs, along with verifying miner address: calling full node state miner info: RPC client error: sendRequest failed: http status 503 Service Unavailable unmarshaling response: invalid character '<' looking for beginning of value. My miner (f010088) was able to get some deals recently, so the error may be happening intermittently. I've tried re-initializing the bidbot repo (signing a new verification message), and changing Lotus API gateway URL in the bidbot config, but this error persists.

How to run bidbod with --fake-mode option

I want test bidbot now .
here is my operation
1: init the bidbot

bidbot init --fake-mode

got this output

Initialized configuration file: /home/chenwei/.bidbot/config

Bidbot needs a signature from a miner wallet address to authenticate bids.

1. Sign this token with an address from your miner owner Lotus wallet address:

    lotus wallet sign [owner-address] 0024080112208b8633e130734dd1d675292b86d48eff47705140867dfecee8b7c20680ec6ff1

2. Start listening for deal auctions using the wallet address and signature from step 1:

    bidbot daemon --storage-provider-id [id] 
                  --wallet-addr-sig [signature] 
		  --lotus-miner-api-maddr [lotus-miner-api-maddr] 
		  --lotus-miner-api-token [lotus-miner-api-token-with-write-access]
		  --deal-start-window [correct-deal-start-epoch-window-for-your-miner]

Note: In the event you win an auction, you must use this wallet address to make the deal(s).

Good luck!

and then I run the bidbot daemon with this command:

bidbot daemon --fake-mode

got this error

2021-09-28T17:51:33.333+0800 INFO bidbot bidbot/main.go:299 bidbot version: git
build date: 2021-09-28T08:57:38Z
git summary: v0.1.0
git branch: main
git commit: 9e37e26
git state: clean
2021/09/28 17:51:33 --storage-provider-id is required. See 'bidbot help init' for instructions

upgrading to v0.1.10 breaks quic setup

After upgrading to v0.1.10 starting bid bot with the old config that used to work for months dies with :

2022-03-17T11:37:34.434+0200	FATAL	cli	[email protected]/cli.go:142	starting service: 1 error occurred:
	* creating peer: 1 error occurred:
	* creating peer: 1 error occurred:
	* setting up libp2p%!(EXTRA *errors.errorString=transports already registered for protocol(s): quic)

When a disable quic in the config : "quic": true, to "quic": false, bidbot starts and seems to run fine.

Bidbot: Command not found

Hey! Tried to install the bidbot by building from source. I followed the installation docs, but get bidbot: command not found, when I try to use bidbot init.

go version:
go version go1.16.4 linux/amd64

Install logs:

make install
(re)installing /home/sealworker3/go/bin/govvv-v0.3.0
go: downloading github.com/ahmetb/govvv v0.3.0
CGO_ENABLED=0 go install -ldflags="-X github.com/textileio/bidbot/buildinfo.BuildDate=2021-07-07T10:48:55Z -X github.com/textileio/bidbot/buildinfo.GitCommit=1b9df01 -X github.com/textileio/bidbot/buildinfo.GitBranch=main -X github.com/textileio/bidbot/buildinfo.GitState=clean -X github.com/textileio/bidbot/buildinfo.GitSummary=v0.0.2-2-g1b9df01 -X github.com/textileio/bidbot/buildinfo.Version=git" .
go: downloading github.com/joho/godotenv v1.3.0
go: downloading github.com/spf13/cobra v1.1.3
go: downloading github.com/spf13/viper v1.8.1
go: downloading github.com/textileio/go-log/v2 v2.1.3-gke-1
go: downloading github.com/multiformats/go-multiaddr v0.3.3
go: downloading github.com/libp2p/go-libp2p-core v0.8.5
go: downloading github.com/hashicorp/go-multierror v1.1.1
go: downloading github.com/oklog/ulid v1.3.1
go: downloading google.golang.org/protobuf v1.27.1
go: downloading github.com/spf13/pflag v1.0.5
go: downloading go.opentelemetry.io/contrib v0.21.0
go: downloading go.opentelemetry.io/otel v1.0.0-RC1
go: downloading github.com/gogo/status v1.1.0
go: downloading google.golang.org/grpc v1.39.0
go: downloading github.com/filecoin-project/lotus v1.10.0
go: downloading github.com/textileio/go-ds-badger3 v0.0.0-20210324034212-7b7fb3be3d1c
go: downloading github.com/textileio/go-ds-mongo v0.1.4
go: downloading github.com/textileio/go-datastore-extensions v1.0.1
go: downloading github.com/multiformats/go-multiaddr-dns v0.3.1
go: downloading github.com/sanderpick/ipfs-lite v1.1.20-0.20210603231246-4c7bb79224a9
go: downloading github.com/ipfs/go-ipfs-blockstore v1.0.4
go: downloading github.com/ipfs/go-ipfs-config v0.14.0
go: downloading github.com/libp2p/go-libp2p v0.14.3
go: downloading github.com/libp2p/go-libp2p-peerstore v0.2.7
go: downloading github.com/libp2p/go-libp2p-pubsub v0.4.2-0.20210517161200-e6ad80cf4782
go: downloading github.com/oklog/ulid/v2 v2.0.2
go: downloading github.com/libp2p/go-libp2p-quic-transport v0.11.1
go: downloading go.opentelemetry.io/contrib/instrumentation/runtime v0.21.0
go: downloading go.uber.org/zap v1.18.1
go: downloading github.com/multiformats/go-multihash v0.0.15
go: downloading github.com/ipfs/go-log v1.0.5
go: downloading go.opencensus.io v0.23.0
go: downloading go.opentelemetry.io/otel/exporters/prometheus v0.21.0
go: downloading go.opentelemetry.io/otel/metric v0.21.0
go: downloading go.opentelemetry.io/otel/sdk v1.0.0-RC1
go: downloading github.com/google/uuid v1.2.0
go: downloading github.com/gogo/googleapis v1.4.0
go: downloading github.com/gogo/protobuf v1.3.2
go: downloading github.com/dgraph-io/badger/v3 v3.2011.1
go: downloading google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c
go: downloading github.com/golang/protobuf v1.5.2
go: downloading github.com/miekg/dns v1.1.41
go: downloading github.com/benbjohnson/clock v1.1.0
go: downloading go.mongodb.org/mongo-driver v1.4.0
go: downloading go.opentelemetry.io/otel/sdk/export/metric v0.21.0
go: downloading go.opentelemetry.io/otel/sdk/metric v0.21.0
go: downloading github.com/fsnotify/fsnotify v1.4.9
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/magiconair/properties v1.8.5
go: downloading github.com/pelletier/go-toml v1.9.3
go: downloading github.com/mitchellh/mapstructure v1.4.1
go: downloading github.com/spf13/afero v1.6.0
go: downloading github.com/spf13/cast v1.3.1
go: downloading github.com/spf13/jwalterweatherman v1.1.0
go: downloading github.com/subosito/gotenv v1.2.0
go: downloading gopkg.in/ini.v1 v1.62.0
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/hashicorp/golang-lru v0.5.5-0.20210104140557-80c98217689d
go: downloading github.com/ipfs/interface-go-ipfs-core v0.4.0
go: downloading github.com/ipfs/go-bitswap v0.3.4
go: downloading github.com/ipfs/go-ipfs-provider v0.5.1
go: downloading github.com/ipfs/go-unixfs v0.2.6
go: downloading github.com/libp2p/go-libp2p-kad-dht v0.12.1
go: downloading github.com/btcsuite/btcd v0.21.0-beta
go: downloading github.com/minio/sha256-simd v1.0.0
go: downloading github.com/klauspost/compress v1.11.7
go: downloading github.com/libp2p/go-netroute v0.1.6
go: downloading github.com/lucas-clemente/quic-go v0.21.1
go: downloading github.com/prometheus/client_golang v1.11.0
go: downloading golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf
go: downloading go.opentelemetry.io/otel/internal/metric v0.21.0
go: downloading github.com/multiformats/go-multistream v0.2.2
go: downloading github.com/libp2p/go-libp2p-mplex v0.4.1
go: downloading github.com/libp2p/go-libp2p-noise v0.2.0
go: downloading github.com/libp2p/go-libp2p-yamux v0.5.4
go: downloading github.com/libp2p/go-tcp-transport v0.2.3
go: downloading github.com/libp2p/go-ws-transport v0.4.0
go: downloading github.com/whyrusleeping/mdns v0.0.0-20190826153040-b9b60ed33aa9
go: downloading golang.org/x/net v0.0.0-20210423184538-5f58ad60dda6
go: downloading golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40
go: downloading golang.org/x/text v0.3.6
go: downloading github.com/cenkalti/backoff v2.2.1+incompatible
go: downloading go.opentelemetry.io/otel/trace v1.0.0-RC1
go: downloading github.com/klauspost/cpuid/v2 v2.0.4
go: downloading github.com/libp2p/go-libp2p-transport-upgrader v0.4.2
go: downloading github.com/libp2p/go-libp2p-autonat v0.4.2
go: downloading github.com/libp2p/go-conn-security-multistream v0.2.1
go: downloading github.com/libp2p/go-libp2p-swarm v0.5.0
go: downloading github.com/google/gopacket v1.1.19
go: downloading github.com/libp2p/go-mplex v0.3.0
go: downloading github.com/libp2p/go-yamux/v2 v2.2.0
go: downloading github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd
go: downloading github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b
go: downloading github.com/flynn/noise v1.0.0
go: downloading github.com/prometheus/common v0.26.0
go: downloading github.com/prometheus/procfs v0.6.0
go: downloading github.com/dgraph-io/ristretto v0.0.4-0.20210122082011-bb5d392ed82d
go: downloading github.com/huin/goupnp v1.0.1-0.20210310174557-0ca763054c88
go: downloading github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc
go: downloading github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c
go: downloading github.com/golang/snappy v0.0.3-0.20201103224600-674baa8c7fc3
go: downloading github.com/google/flatbuffers v1.12.0
go: downloading github.com/marten-seemann/qtls-go1-16 v0.1.3
go: downloading github.com/go-stack/stack v1.8.0
go: downloading github.com/aws/aws-sdk-go v1.32.11
go: downloading github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
go: downloading github.com/xdg/stringprep v0.0.0-20180714160509-73f8eece6fdc
go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
go: downloading github.com/ipfs/go-ipfs-http-client v0.1.0
go: downloading github.com/ipfs/go-ipfs-cmds v0.6.0
go: downloading github.com/rs/cors v1.7.0
go: downloading github.com/jmespath/go-jmespath v0.4.0

Add log message when pause/resume daemon

Add some log message when these actions happen to provide signal of the change in status.

BIDBOT_DEAL_DATA_DIRECTORY doesn't always work

When running bidbot, sometimes new data downloads are put in the directory I specify in the env variable, and other times it just puts them in the default .bidbot/deal_data folder.

Auctioneer returned error: response not received

Recieved an auction, but when the bidbot was trying to publish the bid it errored out with a response not recieved.

2021-07-08T05:58:30.452+0200	INFO	bidbot/service	service/service.go:317	received auction 01fa248pyev3hzjvh722aw10hm from 12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD: 
{
  "id": "01fa248pyev3hzjvh722aw10hm",
  "payload_cid": "bafybeidmmgrnlo3bg4earr43i3tvsjvn4kn2vtbdmenkxlaldrnwuvimku",
  "deal_size": 4294967296,
  "deal_duration": 1468800,
  "sources": {
    "car_url": {
      "URL": "https://broker.staging.textile.dev/car/bafybeidmmgrnlo3bg4earr43i3tvsjvn4kn2vtbdmenkxlaldrnwuvimku"
    }
  },
  "ends_at": {
    "seconds": 1625716720,
    "nanos": 350875922
  }
}
2021-07-08T05:58:40.443+0200	INFO	bidbot/service	service/service.go:317	received auction 01fa248pyev3hzjvh722aw10hm from 12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD: 
{
  "id": "01fa248pyev3hzjvh722aw10hm",
  "payload_cid": "bafybeidmmgrnlo3bg4earr43i3tvsjvn4kn2vtbdmenkxlaldrnwuvimku",
  "deal_size": 4294967296,
  "deal_duration": 1468800,
  "sources": {
    "car_url": {
      "URL": "https://broker.staging.textile.dev/car/bafybeidmmgrnlo3bg4earr43i3tvsjvn4kn2vtbdmenkxlaldrnwuvimku"
    }
  },
  "ends_at": {
    "seconds": 1625716730,
    "nanos": 362617129
  }
}
2021-07-08T05:58:50.454+0200	INFO	bidbot/service	service/service.go:317	received auction 01fa248pyev3hzjvh722aw10hm from 12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD: 
{
  "id": "01fa248pyev3hzjvh722aw10hm",
  "payload_cid": "bafybeidmmgrnlo3bg4earr43i3tvsjvn4kn2vtbdmenkxlaldrnwuvimku",
  "deal_size": 4294967296,
  "deal_duration": 1468800,
  "sources": {
    "car_url": {
      "URL": "https://broker.staging.textile.dev/car/bafybeidmmgrnlo3bg4earr43i3tvsjvn4kn2vtbdmenkxlaldrnwuvimku"
    }
  },
  "ends_at": {
    "seconds": 1625716740,
    "nanos": 373591008
  }
}
2021-07-08T05:59:00.933+0200	INFO	bidbot/service	service/service.go:431	bidding in auction 01fa248pyev3hzjvh722aw10hm from 12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD: 
{
  "auction_id": "01fa248pyev3hzjvh722aw10hm",
  "miner_addr": "f023467",
  "wallet_addr_sig": "ApGrZvAGoxp6e9eZzTttipFXfuGQMl+jthccfYNYsgWUPDa+WaRfR7TU5r/sl6uC2BPN8ygyG//spbkzwhCygYVc8eYch0iGt2ZtBBEn04k1/5jw664JtDzr7kzxYfHcaQ==",
  "start_epoch": 916557,
  "fast_retrieval": true
}
2021-07-08T05:59:10.934+0200	ERROR	bidbot/service	service/service.go:321	making bid: publishing bid; auctioneer 12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD returned error: response not received
2021-07-08T05:59:13.674+0200	INFO	bidbot/service	service/service.go:431	bidding in auction 01fa248pyev3hzjvh722aw10hm from 12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD: 
{
  "auction_id": "01fa248pyev3hzjvh722aw10hm",
  "miner_addr": "f023467",
  "wallet_addr_sig": "ApGrZvAGoxp6e9eZzTttipFXfuGQMl+jthccfYNYsgWUPDa+WaRfR7TU5r/sl6uC2BPN8ygyG//spbkzwhCygYVc8eYch0iGt2ZtBBEn04k1/5jw664JtDzr7kzxYfHcaQ==",
  "start_epoch": 916558,
  "fast_retrieval": true
}
2021-07-08T05:59:20.885+0200	ERROR	bidbot/service	service/service.go:321	making bid: creating bids topic: creating topic: joining topic: topic already exists
2021-07-08T05:59:23.675+0200	ERROR	bidbot/service	service/service.go:321	making bid: publishing bid; auctioneer 12D3KooW9wsxrkCx6CnsWb1gBxZWAjzVK5Hif9FLXKoQZYLewXoD returned error: response not received

FR: Add max download speed option

G'day
I think it would be fantastic to have a max download speed config option to allow SPs to limit the max a single BB download can pull down at one time.

The running-bytes-limit flag lets you set it over a period of time but doesn't let you limit any downloads to a max of 50MB/s etc etc

Feature similar to the --max-overall-download-limit flag in aria2c