white-whale-defi-platform / white-whale-bots Goto Github PK

add a setup logging to the liquidation loop initializer

Steps to Reproduce

The skip bid rate is not used in the message generation: increasing or lowering it in the env file has no impact

Proposed fix

Use the environment variable "SKIP_BID_RATE" in the skipLoop.skipTrade() function

This would be really helpful.

Currently we are also querying poolstates of pools that are not used in any of our constructed paths. Although this is done in async its unnecessary overhead and should be removed.

As you all probably know, there are currently Txs in the Juno Mempool that would trigger arbitrage opportunitys if succesful.

Im currently working on a way to ignore these addresses and prevent fake swap transaction spam.

Steps to Reproduce

1. Start Bot
2. See Fake Transaction
3. Try Arb
4. waste tx fees if using mempoolloop

We should be able to store the pools setup we query in the bot's filesystem and check for a previously saved pool setup if the user demands so. This would prevent us from querying all pools again when we experience small downtime.

Some transactions that should be blocked based on the igno addresses list still arent blocked by the bot. Doublecheck this.

Some borrow/lending protocols use bidding systems to sell off liquidated assets, implement a way for the liquidation bot to place a bid when needed

make sure its possible to run the/a liquidation bot without mempool usage

The IGNORE_ADDRESSES config parameter is not present in the Modules.ts file that states all input variables. Add it and also doublecheck if other env variables used in the botconfig are missing.

Improvement description

Have the bot log on exit. For now, the bot just calls process.exit on a error so the bot exists.
To see the error that actually occurred and made the bot exit, make sure it also logs the obtained error and then exits using process.exit.

Instead of using a profit thresholds based on fees that should be passed before the bot executes a trade, we prefer a profit threshold set to an absolute amount of the BASE_ASSET. e.g. we want the bot to trade if profit > 1 juno instead of 10 * paid_fees_in_juno.
Makes it easier to setup for people and less confusing.

Steps to Reproduce

• I accidentally used yarn to get the latest packages so i ened up to all latest versions. (NOTE: with npm install it worked as its using the package-lock.json )

Additional context

This commit in skipjs break the current implementation of the skipLoop at:
https://github.com/White-Whale-Defi-Platform/white-whale-bots/blob/main/src/types/arbitrage/skipLoop.ts#L137

skip-mev/skipjs@73ae642#diff-a2a171449d862fe29692ce031981047d7ab755ae7f84c707aef80701b3ea0c80

• https://github.com/skip-mev/skipjs

tried this implementation but the mthod to get the key getAccountsWithPrivkeys is private.

// broken
const privKey = (await this.skipSigner.getAccountsWithPrivkeys())[0].privkey

const txToArbRawString = Buffer.from(txToArbRaw).toString('base64')
const txRawString = Buffer.from(txRaw).toString('base64')
const signed = await this.skipClient.signBundle([txToArbRawString, txRawString], privKey);

The tests are lacking on the bot code.
write tests for classes and methods, create a new issue for the remainders.

Currently we miss all swap messages through skip

If you want to report a security issue, please follow our security policy: https://github.com/White-Whale-Defi-Platform/migaloo-core/blob/main/SECURITY.md

Steps to Reproduce

Run Bot on terra, wait for console log

Expected results:

Actual results:

Additional context

Add any other context about the problem here, code snippets, json responses and so on.

Steps to Reproduce

1. Just running the bot as is, with skip turned off
2. It appears the bot attempts to arbitrage through the same pool

Expected results:
The bot will only swap through different pools

Actual results:
Bot appears to attempt to swap through the same pool

Additional context

Recently i discovered a trade that send more denom to skip as a bid than the trade made as a profit.
To prevent this we should limit the amount of slippage we tolerate on all hops.

At the moment we are creating logging messages at multiple places in the code, mostly in the Loop - types. We want to be able to call a function that either creates the logmessage for us and even sends it. Instead of fully creating a 'skip logging message' like we do now, we simply want to call 'createSkipLogResult( res: SkipResult ) and this function handles it for us. This will greatly increase readability of the code.

When the bot tries to apply mempool transactions on local state, it tries to decode 'MsgSend' objects among other.
Usually, these send messages happen on a cw20 token and result in a swap of the token, but apparently sometimes they also result in a 'withdraw_liquidity' message. When this occurs, the bot tries to swap an LP token with the assets of that specific pool and will result in an error:

TypeError: Cannot read properties of undefined (reading 'amount')
    at applyTradeOnPool (/usr/src/app/out/core/types/base/pool.js:51:28)
    at applyMempoolTradesOnPools (/usr/src/app/out/core/types/base/pool.js:177:33)
    at SkipLoop.step (/usr/src/app/out/core/types/arbitrageloops/skipLoop.js:62:58)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async main (/usr/src/app/out/index.js:118:9)

This can be fixed to tighten up the check isSendMsg() in mempool.ts

A lot of pools on the Junoswap Dex will be moved to the new Wynd Dex, developed on the Juno Chain. Make sure we integrate this new dex in the bot. This means:

• implementing the required queries
• implementing the used messages
• implementing the used messages in our flashloan message

The bot logs every 150 iterations of "blocks" to either console or slack with a sign of life. This number 150 is rather arbitrary since the blocktime differs a lot between chains. E.g. on Injective this means the bot logs every 2/3 minutes whereas on Juno it means it logs every 15 minutes. The amount of iterations between each sign of life log should be added as an Environment Variable in the .env configuration file

When creating a new MergeRequest in this Repo, a Checklist is shown, but the link to the CONTRIBUTING.md file is broken as the file now is in a renamed repo under /docs.

## Checklist:

- [x ] I have read [Migaloo's contribution guidelines](**https://github.com/White-Whale-Defi-Platform/migaloo-bots/blob/main/CONTRIBUTING.md**).

✅ https://github.com/White-Whale-Defi-Platform/white-whale-bots/blob/main/docs/CONTRIBUTING.md

Currently the bot supports a skipLoop and a mempoolLoop that are invoked based on environment variable.
The mempool loop scans mempool for trades, applies them on local states and calculates potential arbitrage (MEV).
The skipLoop does all of the mempoolLoop, but bids on blockspace using skipprotocol's method.

Although it can be assumed that most arbitrage is extracted before the opportunity hits the actual blockchain state (through MEV), it is possible that arbitrage exists on the blockchain state. To allow arbitrage bots to only search for these opportunities, without risking failed transactions by using MEV or being reliant on Skip, we want to implement a NoMempoolLoop. This loop will not be using memory pool and only look at actual states of the blockchain, therefore incurs less risk but will most likely also see less opportunities (since other searcher will be running MEV and execute first).

Currently the account sequence is incremented even when a transaction failed. This results in all subsequent transactions to fail

Currently the auction house addresses are being updated by the Skip team. Make sure to update to the correct addresses in our config examples.

The ignored addresses list in memory should be stored in an object with attributes instead of a Set for performance reasons.

Tried to add the Posthuman - Juno pool and got this output:
Error: Query failed with (18): Error parsing into type cw20_base_mintable::msg::QueryMsg: unknown variant info, expected one of balance, token_info, minter, allowance, all_allowances, all_accounts, marketing_info, download_logo: query wasm contract failed: invalid request
at QueryClient.queryUnverified (.../node_modules/@cosmjs/stargate/build/queryclient/queryclient.js:107:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.queryContractSmart (.../node_modules/@cosmjs/cosmwasm-stargate/build/modules/wasm/queries.js:59:34)
at async initPools (.../out/juno/queries/getPoolState.js:42:25)
at async main (.../out/index.js:82:19)

Improvement description

The optimizer now uses a analytically derived calculation for the optimal tradesize. However this calculation is based on the AMM formulas in which the fee (usally 0.3%) is deducted from the returned_asset. For uniswap-like AMMs however, the fee is deducted from the input asset that is offered in a swap. The optimizer needs to be able to handle this.

Additional context

Use this paper to implement the cyclic arbitrage optimal tradesize calculation:
https://arxiv.org/pdf/2105.02784.pdf

Improvement description

To improve quality of PRs and simplify code reviews, this repo should have a simple CI pipeline including:

• checks for linting
• building the docker file
• running tests

Refactor the mempool message handling and applying them on the pools to a central point.

Currently the flashloan router contract address is hardcoded in the code (at flashloanmessage.ts) for each Juno and Terra. To support future expansions towards new chains and make sure we dont have hardcoded values like contract addresses in our code, we should move those contract addresses to the environment variables. Also make sure they are added to the docs

the SkipLoop uses hardcoded juno address as bid wallet insead of the walletaddress we provide via environment variables. This obviously is a bug and needs to be changed. If we currently run the bot on different chains then juno the skip bid will fail resulting in an error.

see skipLoop.ts

Since the update the bot encounters this error for me:
TypeError: Cannot read properties of undefined (reading 'contract_addr')
at isMatchingAssetInfos (.../migaloo-bots/out/types/core/asset.js:47:71)
at applyTradeOnPool (.../migaloo-bots/out/types/core/pool.js:48:42)
at applyMempoolTradesOnPools (.../migaloo-bots/out/types/core/pool.js:73:17)
at SkipLoop.step (.../migaloo-bots/out/types/arbitrage/skipLoop.js:51:58)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async main (.../migaloo-bots/out/index.js:112:9)

The user zJ on Discord confirmed that it errors on the pool osmo/wynd and I speculated that it might be related to the combination of cw20 assets with ibc assets on wynd. In favor for this theory speaks that the error seems to occur more often since wynd/usdc and wynd/atom haven been integrated on the dex. Against it speaks the fact that successful arbitrage on those pools is possible as seen in the following tx: https://www.mintscan.io/juno/txs/7720031934817D68D1701CB7FB6440473E15FEA0CBE88308D8B90F6565C54CB9

The bot currently assumes all trading fees (usually .3% on dexs) remain in the pools for the LP rewards. However, most protocols use something called "protocol fee" or "burn fee", which is a portion of the trading fees. This portion leaves the pool and therefore influences the expected state. This should be handled properly

When people are reliant on the availability of public nodes, they get usually blocked out a lot by specific rpc clients. We'd like to be able to provide a list of known public rpc's, of which the bot selects one. When the bot errors on querying the node because of rate limits, we switch the client to a different rpc endpoint and reinitialise the clients.

Write decent unit tests for the optimzers we use.

Improvement description

It would be helpful to have a way to get the input and output fees without having to manually enter them for every pool defined within a config.

Some options on how this could be accomplished:

• Create centralized location with all input and output fees for DEXes on a give chain.
• Add some automated process integrated into the bot that queries the DEXes/pools to determine the proper fees.

Additional context

Example of current config where inputfee and outputfee must be definied.

POOLS = '
 {"pool": "juno1sg6chmktuhyj4lsrxrrdflem7gsnk4ejv6zkcc4d3vcqulzp55wsf4l4gl","inputfee": 0.3, "outputfee": 0},
 {"pool": "juno124d0zymrkdxv72ccyuqrquur8dkesmxmx2unfn7dej95yqx5yn8s70x3yj","inputfee": 0.4,"outputfee": 0}'

Implement first version of the orderbook arbitrage between a AMM dex and an Orderbook. Make sure it uses proper abstractions so it could be used for different orderbooks if possible.

If you want to report a security issue, please follow our security policy: https://github.com/White-Whale-Defi-Platform/migaloo-core/blob/main/SECURITY.md

Steps to Reproduce

1. See Log
2. npm start and wait around 350-600 Blocks
3. tendermint-rpc Bad status response: 500

Expected results:
App waits x seconds and tries again.

Actual results:
App is crashing. (Log)

Additional context

Add any other context about the problem here, code snippets, json responses and so on.

/home/dietpi/white-whale-bots/node_modules/@cosmjs/tendermint-rpc/build/rpcclients/http.js:10
throw new Error(`Bad status on response: ${res.status}`);   
^

Error: Bad status on response: 500
at filterBadStatus (/home/dietpi/white-whale-bots/node_modules/@cosmjs/tendermint-rpc/build/rpcclients/http.js:10:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v19.6.0

Instead of brute forcing all paths, 2, 3 and 4-hop potentially, we can implement a graph search algorithm that increases initialisation speed significantly. This will preserve more uptime after the bot crashes and restarts.

At the moment the 'inj' token on Injective holds 18 decimals. Although we handle this well on querying and broadcasting messages, we do not yet handle it properly when we apply mempool trades on our pools to project future chain states. This causes incorrect arbitrage opportunities that the bot will try to trade. Note: only relevant when using mempool-loop or skip-loop.

For further expanding across the cosmos we need to store the decimals somewhere in the bot. The asset seems the most logic place since its only relevant to that specific asset. Add the decimals field but make sure the query initPools and getPoolStates still work by reading an assetinfo from chain.

Improve logging of the config file

In my desperate quest to work on one of my ideas I channeled the coding skills of a 5y old and added the following on line 98 in ../src/types/arbitrage/skipLoop.ts
if (arbTrade.profit > 10000000) {
this.botConfig.skipBidRate = 0.51;
}
if (arbTrade.profit > 100000000) {
this.botConfig.skipBidRate = 0.81;
}
This resulted in some unprofitable arbs. This came with no surprise, because arbTrade.profit obviously doesn't calculate in the new skipBidRate like this.
However, many examples indicated that changes in the skipBidRate did not depend on the profit, but on the amount of the flash loan taken. Two examples:
https://www.mintscan.io/juno/txs/945A11F0698FFE3DDF1DB19A55313862C89570D25BC63D053915AC9444782FE2
https://www.mintscan.io/juno/txs/83E183BF08BB10C9000D008EE78F97B608AD0E8A5960A42A4129C3F6864CB6F5

Once in a while this assessment gets contradicted though.
https://www.mintscan.io/juno/txs/329EB8791B5C1544BBBD365AF9C94CC93067F9769195E9CFD35BA6344037EC09
https://www.mintscan.io/juno/txs/D189034E1BAECDCAA6727B76BF52B37C9F40A862DBB503D35496C9C6703D8D88

It is possible to estimate fee usage (based on amount of messages) and the therefore required profit margin for each path.
make sure that each path contains a Fee/Gas object and a profit_margin object, rework this from the config

https://github.com/White-Whale-Defi-Platform/migaloo-bots/blob/98ac905a6fbb36df88e5f2976164e99abecbe870/src/arbitrage/optimizers/analyticalOptimizer.ts#L120-L134

Sorry, I have no clue about Typescript and this is most likely not helpful, but what's up with 'maxProfit' in line 127. Isn't it always 0?