Coder Social home page Coder Social logo

safe-global / safe-transaction-service Goto Github PK

View Code? Open in Web Editor NEW
180.0 17.0 247.0 4.38 MB

Keeps track of transactions sent via Safe contacts and confirmed transactions. It also keeps track of Ether and ERC20 token transfers to Safe contracts.

License: MIT License

Python 99.82% Shell 0.10% Dockerfile 0.05% HTML 0.03%
ethereum gnosis gnosis-safe multisig backend

safe-transaction-service's Introduction

CI/CD Coverage Status pre-commit Python 3.12 Django 5 Docker Image Version (latest semver)

Safe Transaction Service

Keeps track of transactions sent via Safe contracts. It uses events and tracing to index the txs.

Transactions are detected in an automatic way, so there is no need of informing the service about the transactions as in previous versions of the Transaction Service.

Transactions can also be sent to the service to allow offchain collecting of signatures or informing the owners about a transaction that is pending to be sent to the blockchain.

Swagger (Mainnet version) More networks

Index of contents

Setup for development

Use a virtualenv if possible:

python -m venv venv

Then enter the virtualenv and install the dependencies:

source venv/bin/activate
pip install -r requirements-dev.txt
pre-commit install -f
cp .env.dev .env
./run_tests.sh

Setup for development using docker

docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

Setup for production (event indexing)

Since version 3.0.0 transaction service can be configured to rely on event indexing when SafeL2 version is used. Only contracts from v1.3.0 onwards with L2 events will be indexed.

An example environment file can be used for the L2 setup:

cp .env.l2.sample .env

Edit .env file to add ETHEREUM_NODE_URL (on the example a Polygon public node is used) and remember to modify DJANGO_SECRET_KEY to use a strong key. The rest of the configuration does not need to be modified. Then:

docker-compose build --force-rm
docker-compose up

For more parameters check base.py file.

Setup for a custom network

  • If the network is not supported yet contracts can be deployed using the deployment instructions and then a PR should be provided adding the deployment block number and the address (address will be the same for every network).
  • Only ProxyFactory and GnosisSafeL2 must be configured. +L2 must be added to the Safe L2 contract versions, so the service knows the contract can be indexed using events. For us to accept the PR network must be on https://github.com/ethereum-lists/chains .
  • You can always set this up later using the admin panel if your network is not supported, going to the Master Copies and Proxy Factories.
  • We recommend using event indexing for every network where transaction fees are not relevant, so a tracing node is not required and everything can be indexed using events with the Safe L2 version.

Setup for production (tracing mode)

This is the recommended configuration for running a production Transaction service. docker-compose is required for running the project.

bash cp .env.tracing.sample .env

Configure the parameters needed on .env. These parameters need to be changed:

  • DJANGO_SECRET_KEY: Use a strong key.
  • ETHEREUM_NODE_URL: Http/s address of a ethereum node. It can be the same than ETHEREUM_TRACING_NODE_URL.
  • ETHEREUM_TRACING_NODE_URL: Http/s address of an OpenEthereum node with tracing enabled.

If you don't want to use trace_filter for the internal tx indexing and just rely on trace_block, set:

  • ETH_INTERNAL_NO_FILTER=1

For more parameters check base.py file.

Then:

docker-compose build --force-rm
docker-compose up

The service should be running in localhost:8000. You can test everything is set up:

curl 'http://localhost:8000/api/v1/about/'

You can go to http://localhost:5555/ to check the status of the task queue, also you can configure prometheus metrics.

For example, to set up a Göerli node:

Run an OpenEthereum node in your local computer:

openethereum --chain goerli --tracing on --db-path=/media/ethereum/openethereum --unsafe-expose

Edit .env so docker points to the host OpenEthereum node:

ETHEREUM_NODE_URL=http://172.17.0.1:8545
ETHEREUM_TRACING_NODE_URL=http://172.17.0.1:8545

Then:

docker-compose build --force-rm
docker-compose up

Use admin interface

Services come with a basic administration web ui (provided by Django) by default on http://localhost:8000/admin/

A user must be created to get access:

docker exec -it safe-transaction-service-web-1 python manage.py createsuperuser

Safe Contract ABIs and addresses

Service maintenance

Service can run into some issues when running in production:

Indexing issues

You can tell there are indexing issues if:

  • Executed transactions are missing from the API (all-transactions, multisig-transactions, module-transactions... endpoints). If you use the Safe{Wallet} Web client you should check what is the current state of the Safe Client Gateway cache as it might have outdated data.
  • Asset transfers (ERC20/721) are missing from all-transactions or transfers endpoints.
  • You see error logs such as "Cannot remove owner" or similar inconsistent errors when worker-indexer is processing decoded data.

There are multiple options for this. Connect to either web or worker instances. Running commands inside of tmux is recommended (installed by default):

  • python manage.py check_index_problems: it will try to automatically fix missing transactions. Tokens related transactions (ERC20/721) will not be fixed with this method. This method will take a while, as it needs to compare database data with blockchain data for every Safe.
  • python manage.py reindex_master_copies --from-block-number X --addresses 0x111 0x222: if you know the first problematic block, it's faster if you trigger a manual reindex. --addresses argument is optional, but if you know the problematic Safes providing them will make reindexing way faster, as only those Safes will be reindexed (instead of the entire collection).

If you see ERC20/ERC721 transfers missing:

  • python manage.py reindex_erc20 --from-block-number X --addresses 0x111 0x222: same logic as with reindex_master_copies.

FAQ

Why /v1/safes/{address} endpoint shows a nonce that indicates that a transaction was executed but the transaction is not shown or marked as executed in the other endpoints?

/v1/safes/{address} endpoint uses eth_call from the RPC to get the current information for a Safe, so there's no delay and as soon as a transaction is executed it will be updated. The other endpoints rely on polling, indexing decoding and processing of traces/events and take longer (shouldn't be more than half a minute).

How do you handle reorgs?

When indexed every block is marked as not confirmed unless it has some depth (configured via ETH_REORG_BLOCKS environment variable). Not confirmed blocks are checked periodically to check if the blockchain blockHash for that number changed before it reaches the desired number of confirmations, if that's the case, all blocks from that block and the transactions related are deleted and indexing is restarted to the last confirmed block.

If I add my chain to safe-eth-py will you support it?

No, for a chain to be supported we need to set up a dedicated infra for that network and have a proper RPC

How can I interact with service?

Aside from using standard HTTP requests:

What chains do you officially support?

https://docs.safe.global/api-supported-networks

What means banned field in SafeContract model?

The banned field in the SafeContract model is used to prevent indexing of certain Safes that have an unsupported MasterCopy or unverified proxies that have issues during indexing. This field does not remove the banned Safe and indexing can be resumed once the issue has been resolved.

Troubleshooting

Issues installing grpc on an Apple silicon system

If you face issues installing the grpc dependency locally (required by this project) on a Apple silicon chip, set GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 and GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 and then try to install the dependency again.

Contributors

See contributors

safe-transaction-service's People

Contributors

bijianing97 avatar denisgranha avatar dependabot[bot] avatar falvaradorodriguez avatar fmrsabino avatar germartinez avatar giacomolicari avatar gmonty030 avatar gzeoneth avatar hectorgomezv avatar honzadajc avatar inomurko avatar itsfridaythen avatar jpalvarezl avatar luarx avatar mhxw avatar mmv08 avatar moisses89 avatar oscar-gross avatar philipappiah avatar pkakelas avatar rafaeltorres77 avatar rmeissner avatar scream4ik avatar simonzg avatar svanegmond avatar t0mcr8se avatar titandac avatar uxio0 avatar xiaoxianboy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

safe-transaction-service's Issues

Improve testing/coverage

As tracing is difficult to test, some parts of the code have not automatic testing. That should be fixed using mocking

Incoming NFT transfers

In order to display incoming ERC-721 transfers we need to adapt the transactions endpoint or have a seperate endpoint for incoming NFT transfers.

List of trusted tokens

Balance endpoint is showing all available tokens for an address, even the scammy ones. A trusted list is desired

BUG: Can't POST execution confirmation with removed owner after removal tx is confirmed

What is needed?

If you execute transaction for .removeOwner method with the owner account it is going to remove, the service will return 422 status code when trying to POST execution tx. (I guess replaceOwner is affected too)

Since we don't do optimistic updates on the frontend, we send a POST request to the service only when TX is confirmed, because there are too much possible failures: tx failed, user rejected tx, etc, we need some workaround for this.

Why is it needed?

So the frontend can know if the TX is executed.

When is it needed?

We need this for v0.1.0 to function properly. I'd say we need this as soon as possible, but let's decide it collectively.

Screenshot with the request:
export

Add more creation details

Currently we return created, transactionHash and creator, but sometimes the safe creation is done via an internal transaction (e.g. when using the secure Safe factory which triggers the normal proxy factory). For this is would be nice to also decode the details of the transaction creation:

factory address, mastercopy, setup data. In most cases this should correspond to the function parameters when calling the Proxy factory.

Factory dependent params: saltNonce

See is an extension of #65

Add support for webhooks

What

It would be nice to be able to define webhooks that should be triggered when new confirmation, transactions, etc are added.

Why

Our transactions service handles quite some useful information and it would be nice if it is possible to build services around this. Webhooks would allow other services to better interact with the transactions service.

How

Add a webhooks model, so that we can add webhooks via the admin interface.
The webhooks should be triggered each time any of the following actions occur:

  • When a new pending outgoing transaction is added to the database
  • When a new confirmation is added to the database
  • When a new executed outgoing transaction is added to the database
  • When a new incoming transaction is added to the database
  • TBD

Fix Confirmations matching another Safe's Transactions

Currently the confirmations are matched with the multisig txs just using the SafeTxHash. It could happen that a confirmation from one "fake" Safe matches to another Safe transaction. It's not probable but it should be be addressed.

First would be blocking all non trusted master copies of the Safe contract

Can't send etag header within GET requests due to CORS

If I try to send a get request to fetch the transactions list, I got an CORS error because Access-Control-Allow-Headers is missing in preflight response.

I think it should be fixed just adding on the response headers
Access-Control-Allow-Headers

Failed transactions not showing

For Safes with version >= 1.1.1 the event for failed txs has been updated to:

event ExecutionFailure(
        bytes32 txHash, uint256 payment
    );

Service must update it and process again all txs

Add state information for executed transactions

It would be nice to display some information about a transaction once it has been executed:

  • Confirmation required on time of execution.
  • Confirmation used on-chain to execute the transaction.
  • Fees paid to execute the transaction and receiver of the fees (if applicable)
  • ...

Feature: Return safe tx hash

What is needed?

Return safe tx hash like in relay service with get request

Why is it needed?
On the frontend we need something unique to distinguish transactions

When is it needed?
The faster the better 🙈

Allow filtering transactions by modified

See 5afe/safe-react#517 for details.

Create a new filter for transactions/ endpoint to filter transactions modified after/before a timestamp:

  • modified__lt
  • modified__gt
  • modified__gte
  • modified__lte

It will allow ISO 8601 datetimes like:

  • 2020-02-27T10:46:15Z
  • 2020-02-27T10:46:15+00:00
  • 2020-02-27 10:46:15Z

Backend needs to be adjusted to update modified on transactions when a new confirmation is added to the database.

Support incoming txs

Endpoint to return incoming txs with eth and tokens

URL

GET /api/v1/safes/<str:address>/incoming-transactions/

Response

Paginated json with:

[
{
      "transactionHash": "0x412c19413503174f1497d23a4e2412238610cf52cc0ccc414eae681ad6d2e7e6",
      "to": "0xAB080c90D99095FfAc2AdAc5405e93C4517A0b6D",
      "value": 1000000000000000000,
      "tokenAddress": "0x62f25065BA60CA3A2044344955A3B2530e355111",
      "from": "0x5aC255889882aCd3da2aA939679E3f3d4cea221e"
    },
    {
      "transactionHash": "0xd0ba226d438e37b64f11d088302cbdac78fa6d515d4d3847ed8c16c7cb3632d8",
      "to": "0xAB080c90D99095FfAc2AdAc5405e93C4517A0b6D",
      "value": 1240000000000000,
      "tokenAddress": null,
      "from": "0xAB080c90D99095FfAc2AdAc5405e93C4517A0b6D"
    },
    {
      "transactionHash": "0xdf06ff513ee187e9228f658a272c7054b4e0cc6bfe7e3aab9d2ca19d786155eb",
      "to": "0xAB080c90D99095FfAc2AdAc5405e93C4517A0b6D",
      "value": 5000000000000000,
      "tokenAddress": null,
      "from": "0xAB080c90D99095FfAc2AdAc5405e93C4517A0b6D"
    },
    {
      "transactionHash": "0x80e80c65a5b62029b332a549c11f25a36a11a9a61fddda7eef654723cbf298a6",
      "to": "0xAB080c90D99095FfAc2AdAc5405e93C4517A0b6D",
      "value": 10000000000000000,
      "tokenAddress": null,
      "from": "0x5aC255889882aCd3da2aA939679E3f3d4cea221e"
    }
]

Ether transfers have tokenAddress: null

Add flags to endpoint to only return trusted or non spamming token

Service is returning balance for every ERC20 token detected for a Safe address, even for the scammy tokens.

  • Add flags spam and trusted flag to tokens endpoint to filter with a boolean by these values.
  • Add spam field to token model to mark a token a spam token.
  • Add access for product managers to have the possibilty to mark tokens as spam tokens.

Depends on #37

Make deployment/ maintenance plan

Currently we deploy whenever required (same for reindexing etc). This might interrupt the users and prevent the usage of the web interface.

We should make a plan when we deploy and chose a reasonable timeframe for the deployment (or other maintenance work).
This timeframe depends on:

  • How long does the change take
  • How does it influence the user
  • How critical is the change

Another option is to provide mirror services that can be used during the update.
Also it would be good to announce bigger chances / maintenance work via telegram (and/or twitter)

Long term it would be interesting if we want to have something like https://status.github.com for our services.

Endpoint for retrieving connected Safe(s) for owner address

  • We are planning on improving the recovery flow (safe#180).
  • As part this, we would like to remove the step where a user has to enter the Safe address.
  • Therefore we need an endpoint where the apps can retrieve Safe addresses for a given owner address.

Safe creation transaction details

We want to display additional information about the Safe creation transaction:

  • Creation date/time
  • Tx hash of Safe creation transaction
  • Initiator of Safe creation

We need an endpoint that provides this information for a given Safe address

Related to 5afe/safe-react#554

Add simple authentication to transaction proposal

Allow an owner to appoint delegate addresses that can propose transactions to the service without confirmations (perform a POST with transaction meta).

This would allow us to improve interaction with dapps by providing sdks or an chrome extension or mobile apps to propose transactions.

Notes:

  • ANY owner should be able to add an delegate (backend should probably remember which owner did this) that is annotated with a label
  • ANY owner should be able to remove ANY delegate
  • ANY owner should be able to retrieve a list of all delegates and their label

Add support for off chain signatures

Now POST of confirmations accept signature as an hexadecimal. If signature is provided, transactionHash are not required.

Then GET will return confirmations and they will have a field signature

Store tx hash for not mined multisig transactions

Currently txs are stored on the tx service without txHash, just the safeTxHash. When tx is mined and the safeTxHash is detected the tx will have the txHash stored.

This feature would allow txs to have a txHash even though they are not mined. Currently we are treating the txs with txHash as mined, so logic would need to be changed

Add authentication to retrieve pending transactions

Currently it is possible to retrieve pending transactions for any Safe. It should only be possible for owners to retrieve these information.

Open questions:

  • Do users care?
  • How to we perform authentication?
  • Can non-owners be authenticated?

Make services available for xDai

  • contracts are required on xDai chain
    • ProxyFactory and MasterCopy should be on the same address as on other networks
  • xDai tracing node
  • xDai transaction service

Support off chain Contract Signatures

Currently just on chain contract signatures are supported.

  • Service should parse EIP1271 signatures in Gnosis Safe format.✔️
  • Service should allow EIP1271 without Gnosis Safe format (contractAddress, messageHash and signature), even if service returns it using Gnosis Safe format❌
  • When returning all the signatures, signature must be crafted depending on the other signatures so the pointer to the dynamic part matches.❌

Backend returns same transaction separately

Paul reported via Telegram that their Safe (https://gnosis-safe.io/safes/0x71B94163b085cc2ea7432D8cC96A94989Da4FD6e/transactions) shows the tx with nonce 18 as pending even though Etherscan shows it as successful.

https://safe-transaction.gnosis.io/api/v1/safes/0x71B94163b085cc2ea7432D8cC96A94989Da4FD6e%2F/transactions/ returns it twice:

  • Once with 1 confirmation and pending
  • Once with 2 confirmation and successful.

Why is that? Why isn't the backend able to connect both txs?
That might confuse the frontend.

Add "origin" to transaction

It is planned to allow Safe apps to submit transactions to the interface. As an owner of a Safe I would like to know from which service the transaction provided. For this we should add a "origin" field when sending the transaction meta to the service.

Notes:

  • origin field is not part of the safe tx hash and therefore is not 100% trustworthy (maybe this should be mentioned in swagger) -> malicious owner could abuse this, but the advantages outweight this small risk
  • origin field should be a string

Show incoming module interactions

Currently we only show incoming value transfers (e.g. ERC20 and ETH). Therefore the user currently doesn't see if a module performs a transaction via a Safe (call to execTransactionFromModule or execTransactionFromModuleReturnData).

We should expose this somehow (TBD how)

Create endpoint for balance of tokens

Endpoint to return balance of tokens (and eth)

URL

GET /api/v1/safes/<str:address>/balances/

Response

It will return the following for tokens with value > 0:

[{"tokenAddress": "<address>", 
  "balance": "<int>"
 },
 {"tokenAddress": "<address>", 
  "balance": "<int>"
 }
]

Ether balance is always returned with tokenAddress: null, even if balance == 0

Endpoint for incoming and outgoing tx

Proposal

  • Sorting
    • The endpoint should only show outgoing transactions where the nonce < currentSafe nonce
    • The endpoint should only show incoming transactions that have been mined
    • The transactions should be sorted by execution date. If an outgoing transaction doesn't have an execution date the execution date of the transaction with the same nonce that has been executed should be taken.
  • Transaction types
    • Ethereum transaction - These are transactions that have not been triggered by the requested Safe
    • Safe transaction - These are internal transactions that have been triggered by the requested Safe
    • Module transaction - These are internal transactions that have been triggered via the requested Safe by a Safe module
  • Transfers triggered by Safe transactions
    • Transfers should be nested into the transaction type (see above) that triggered them. If it is unknown how this transfer was triggered it should fallback to an Ethereum transaction

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.