The etherbalance's discuss from gnosis

Use tini?

We should probably wrap this inside tini. We had an issue with our dex-services binary not handling signals well.

cf. https://github.com/gnosis/dex-services/blob/16ef8cf025c41b80318544df9a45890cb764deff/docker/rust/Dockerfile#L41-L44_

Don't alert on Error. Instead record metric.

As proposed in slack, I think we should merely increment a metric for success/failure counts here instead of alerting on every RPC connection timeout like this one

ERROR (pod: dev-dfusion-etherbalance-xdai-64db97c847-rt4vj):
failed to get balance for address 0xf794…5a70 token ether: Transport error: Error(Connect, Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })

Looking at the following code block, it seems we are only actually printing the statement about failing to get balance so I suspect eliminating this error in our alert channel will also involve some communication with DevOps as well.

etherbalance/src/main.rs

Lines 132 to 145 in b6bf3e2

    
           match params.balance { 
        
               Ok(balance) => balance_metric 
        
                   .with_label_values(&[ 
        
                       params.address_name, 
        
                       params.token_name, 
        
                       &format!("{:#x}", params.address), 
        
                       params.tag, 
        
                   ]) 
        
                   .set(u256_to_f64(balance)), 
        
               Err(err) => println!( 
        
                   "failed to get balance for address {} token {}: {}", 
        
                   params.address, params.token_name, err 
        
               ), 
        
           }

Account specific balance alert threshold

Proposal

Allow to specify low balance threshold for each monitored account.

Motivation

We have reached a point where we are attempting to monitor several accounts with this service. Each of these accounts is submitting different types of transactions for several different purposes (i.e. with varying expected transaction fees at varying frequencies).

Example/Use Case

For example solvers submit solutions and spend lots of gas doing so, while the synthetix liquidity provision bot sparsely places orders on an exchange. This implies that that the classification of a "low balance" in one account might be drastically different from that of another. In order to safely ensure a solver is running with sufficient balance over a long weekend we would consider 4 ETH to be a low balance, while a safety net for synthetix bot is 1 ETH.

Batch requests

We are making a lot of requests in a row. It would be more efficient to batch them together. There is a BatchTransport in web3.

Still sending Error alerts

Somehow, this code block (even though it is just a print statement) is still appears as an error in our alerts channel.

I have also checked with devops to see if they have some of their own triggers for such alerts (like any string containing "failed" or something).

etherbalance/src/main.rs

Lines 151 to 161 in c13d061

    
           Err(err) => { 
        
               success_metric 
        
                   .with_label_values(&["failure", &format!("{:#x}", params.address)]) 
        
                   .inc(); 
        
               println!( 
        
                   "failed to get balance for address {} token {}: {}", 
        
                   &format!("{:#x}", params.address), 
        
                   params.token_name, 
        
                   err 
        
               ); 
        
           }

gnosis / etherbalance Goto Github PK

etherbalance's Issues

Use tini?

Don't alert on Error. Instead record metric.

Account specific balance alert threshold

Batch requests

Still sending Error alerts

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	match params.balance {
	Ok(balance) => balance_metric
	.with_label_values(&[
	params.address_name,
	params.token_name,
	&format!("{:#x}", params.address),
	params.tag,
	])
	.set(u256_to_f64(balance)),
	Err(err) => println!(
	"failed to get balance for address {} token {}: {}",
	params.address, params.token_name, err
	),
	}

	Err(err) => {
	success_metric
	.with_label_values(&["failure", &format!("{:#x}", params.address)])
	.inc();
	println!(
	"failed to get balance for address {} token {}: {}",
	&format!("{:#x}", params.address),
	params.token_name,
	err
	);
	}