Coder Social home page Coder Social logo

l2beat / starkex-explorer Goto Github PK

View Code? Open in Web Editor NEW
37.0 37.0 5.0 8.88 MB

Decode calldata stored by the dYdX rollup. Perform force exits and trades.

Home Page: https://dydx.l2beat.com

License: MIT License

JavaScript 0.42% TypeScript 99.21% CSS 0.18% Solidity 0.13% Procfile 0.01% Dockerfile 0.05%
dydx ethereum explorer layer2

starkex-explorer's Introduction

L2BEAT 💓

L2BEAT (Layer 2 Beat) is a website dedicated to providing research and statistics about the very exciting layer 2 technology on Ethereum.

You can visit the site yourself at https://www.l2beat.com/.

Contributing

We welcome and encourage contributions. To learn about the project structure and contributions please read CONTRIBUTING.md.

🔍 Here is also a visual step-by-step guide for creating a Pull Request - link

License

We believe in knowledge sharing and open access. Our code is and will remain open source with the permissive MIT license.

Versioning and publishing (tooling)

Some of the sub-packages are published to NPM. We use a tool called changesets to manage versioning, cross-dependency versioning and publishing new versions to NPM. When you make a change, before mergin to master:

  • run yarn changeset and mark the packages you wish to publish, select what kind of a change it is (major,minor,patch) and provide the summary of the changes

When you want to publish all previous changes to NPM:

  • now run yarn changeset version, this will change the generated .changeset/file.md into an entry into CHANGELOG.md and package.json in changed packages

After your PR with changed CHANGELOG.md and package.json is merged into main, a CI step will run which will try to publish all change changes.

starkex-explorer's People

Contributors

adamiak avatar hasparus avatar krzkaczor avatar mateuszradomski avatar michalsidzej avatar sin6pi7 avatar sz-piotr avatar torztomasz avatar wendrowiec13 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

adamiak gunthardeniro subs4cs vairamzen cbiniamow

starkex-explorer's Issues

Project update 02-03-2022

Progress

  1. Display the computed state in the UI
    • A lot of groundwork has been put in to display the computed state
    • State changes, position details and more are now displayed in the ui
  2. Offboard Piotr & onboard Jacek
    • Piotr, who has been helping me up to this point has left the project to work on other things
    • Jacek has joined and I have onboarded him to the project
  3. UI designs
    • I eventually decided not to use an external company to help with the design of the project
    • Instead I went about it myself and I'm attaching the design of the home page below:

image

Plan for the next sprint

  1. Actually style the UI
  2. Potentially connect ethereum addresses and stark keys

Queries for views

Update:
Home /

  • list of position assets
  • sum in USD (1 USDC = 1 USD)
  • * list of latest forced txs
  • * list of latest offers
  • list of latest state updates
    • id
    • hash
    • number of affected positions
    • * number of included forced txs

State update view /state-updates/:id

  • state update id
  • state update hash
  • state update timestamp
  • * list of included forced txs
  • list of position updates

Position view /positions/:id

  • by position id
  • stark key (public key)
  • * ethereum address
  • list of position assets
  • sum in USD
  • * list of forced txs associated with this position
  • position update history (paginated)
    • state update id
    • state update hash
    • state update timestamp
    • ?

Position update view /positions/:id/updates/:id

  • state update id
  • state update hash
  • state update timestamp
  • position id
  • position public key
  • position ethereum address
  • list of changes:
    • 10 BTC -> 2 BTC
    • 100 USC -> 80_000 USDC

Old:

  • getStateChangeByTransactionHash
  • getStateChangeList (paginated)
  • getPositionById
  • getPositionList (paginated)
  • getPositionChangeHistoryById (paginated)
  • getPositionIdByStarkKey
  • getPositionIdByEthereumAddress (there is a mapping eth address -> stark key but not for every stark key)
  • getMerkleProofByPositionId

Project update 31-12-2022

We have stumbled upon weird state update. In the update from state update 11506 there's this position:

{
  positionId: 263516n,
  starkKey: '<redacted>',
  collateralBalance: 0n,
  fundingTimestamp: 0,
  balances: [ { assetId: 'ETH-9', balance: 1000000000n } ]
}

As you can see, fundingTimestamp is 0, which as far as we know has not happened before and doesn't look right. We need correct funding of each position in the merkle tree to calculate the merkle root. For this case we have used the most recent funding index (merkle root matched) so syncing could go ahead.

Project update 30-06-2022

Finishing touches

  1. Frontend updates
    • Clicking on your address takes you to your position
    • Make the site mobile friendly
    • Various consistency fixes
  2. Forced trade offers
    • Allow for cancelling offers
    • Send forced trade transaction
    • Show pending offers on the position
    • Show pending transactions on the position
  3. Deployment
    • Improve error reporting so that we are notified if something goes wrong
    • Deploy the site! Yay!

Maintenance work

  1. Fixed a bug where when viewing a list of all offers the backend returned 500 because of an incorrect database query
  2. Fixed a bug where the synchronization process would sometimes miss upgrades when reorgs happened under very specific conditions

Project update 27-04-2022

Progress

  1. Forced actions
    • Connect forced transactions coming from chain to those included in the onchain data
  2. Metamask connection
    • You can now connect to the website with metamask which will allow submitting forced txs
  3. The Layer Two Amsterdam conference
    • There was a conference which distracted our attention away from the project
  4. Kris joins the project
    • We've onboarded Kris to the project in the hopes that he'll help us implement the final features

Plan for the next sprint

  1. Start work on forced trade offers
  2. Submit forced transactions through the UI

Project update 19-01-2022

Progress

  1. Piotr (another one!) joins the project
  2. Complete on-chain data decoding
    • This includes decoding the first page of data
  3. Configure Postgres database
  4. Add logic for event fetching
    • Continuously fetch new events
    • Save fetched events to the database
  5. Add a merkle tree implementation
    • It uses the pedersen hash function
    • It can hold position states
    • The hashes match the python implementation
  6. Reach out to a design agency
    • Currently talking about the possibility of them helping with the design

Plan for the next sprint

  1. Implement the state transition function
  2. Store the full state in the database

Project update 08-06-2022

Progress

  1. Forced trade offers
    • add offer filtering
    • add ui for viewing offers
    • add buttons to cancel, accept and finalize offers
    • add active offers to position UI
    • add a countdown for finalizing offers
  2. Forced exits
    • add endpoint for the frontend to submit a forced withdrawal tx to
    • disallow exiting USDC when your balance is negative
    • add pending transactions to position UI
  3. Local test network
    • set up a mock local network to test the interactions on
    • add a way to encode data in the same way dydx does
  4. Other UI fixes
    • make naming and columns consistent across screens
    • add links to etherscan for most things
    • view your position when you click your address
    • make the UI usable on mobile
    • add proper meta tags to pages

Plan for the next sprint

  1. Fixes and tweaks if needed

Project update 31-07-2022

Maintenance work

  1. Fixed a bug where pagination resulted in an invalid offset
  2. Help a user recover funds after forced exit
  3. Add a warning for people to temporarily exercise caution when using forced exits
  4. Begin work on adding a final withdraw step to forced exits

Project update 31-08-2022

Maintenance work

  1. Add the withdraw step to forced exits
  2. Do some small refactoring in preparation of possible future extensions
  3. Update the TVL display on the site

Project update 30-11-2022

Work has begun on having the explorer also support other starkex instances. The same codebaase would be used across all of them. All new features and bugfixes will automatically benefit the existing dydx explorer

Project update 16-02-2022

Progress

  1. Finish handling chain reorganisations
    • This was almost ready the last time and finally completed in this sprint
  2. Store the full rollup state in the database
    • All the merkle tree nodes and positions are saved into the database
    • This is also compatible with chain reorganisations, so if that happens we'll get the correct state
  3. Start building the UI to display the state
    • The home page can now display a list of state changes
    • Even though the frontend can display the data and the database has the data the queries are not yet implemeted
  4. Technical changes
    • Fixed the script that hashes the asset to perform cache busting so that it respects file references
    • Update coverage reporting to only concern itself with the total coverage

Plan for the next sprint

  1. Display the computed state in the UI
  2. Potentially move on to actually styling the UI

Project update 05-01-2022

Progress

  1. Christmas & time off
  2. On-boarded another person to the project so the next sprint is going to be with 2 people

Plan for the next sprint

  1. Save the memory pages to the database
  2. Build the merkle tree

Save offers in database

https://github.com/starkware-libs/starkex-contracts/blob/StarkExchange-v3.0/scalable-dex/contracts/src/perpetual/interactions/ForcedTrades.sol#L31

interface ForceTradeInitialOffer {
  starkKeyA: StarkKey,
  positionIdA: bigint, // vaultIdA
  syntheticAssetId: AssetId,
  amountCollateral: bigint,
  amountSynthetic: bigint,
  aIsBuyingSynthetic: boolean,
}

interface ForceTradeAcceptedOffer extends ForceTradeInitialOffer {
  starkKeyB: StarkKey,
  positionIdB: bigint, // vaultIdB
  submissionExpirationTime: Timestamp // B enforces that A acts quickly
  nonce: bigint // TODO: figure out
  premiumCost: boolean // TODO: figure out

  // https://github.com/starkware-libs/starkex-contracts/blob/StarkExchange-v3.0/scalable-dex/contracts/src/perpetual/interactions/ForcedTrades.sol#L138-L181
  signature: string // HEX string signature of all parameters
}

Project update 31-01-2023

  • Transaction handling was refactored, so that its easier to add new transactions
  • All tests were moved next to the implementation. This way it's easy to see what functionality is covered
  • Add decoding for last two fields of perpetual cairo output

Syncing Events 

// one at a time, in order
function sync(from, to) {
   const oldVerifiers = db.getExistingVerifiers(to)
   const newVerifiers = blockchain.getVerifierChanges(from, to)
   db.saveVerifiers(from, to, newVerifiers)
   const verifiers = [...oldVerifiers, ...newVerifiers]
   const events = blockchain.getEvents(verifiers, from, to)
   const data = parseOnChainData(events)
   db.saveData(data)
}

Project update 22-12-2021

Progress

  1. Repository setup
  2. Working SSR frontend
    • Use React for SSR, but don't ship it to client
    • Compile CSS with Sass
    • Compile TypeScript to frontend scripts
    • Implement Cache Busting with file hashes
  3. A live backend server
  4. A pedersen hash implementation
    • Uses different code than StarkWare (https://github.com/l2beat/dydx-state-explorer/blob/8f3ca42d9c42f5f79ea36305d736b0e5dab782ed/packages/crypto/src/pedersenSync.ts#L37)
    • It was actually less hassle to rewrite than try to reuse StarkWare code, because of license restrictions, different language (JS / TS), the fact that the StarkWare code was only published on npm by third parties and a lot of parts not related to this project being present in the original code
    • The hash function is really slow (>10ms per hash) so I might explore other options like a native implementation in the future, but it works for now. This was also the case for the original StarkWare JS code
  5. On chain data decoder
    • can fetch data like the original python script
    • decodes funding index and position updates
    • decodes asset ids
    • (On a PR) decodes the first page with the state. It is unclear at this point what the remaining data in the page is for or how to decode it
    • the Graph was actually nice but didn't help much except for providing another reference point. Both The Graph and the original python script don't decode the first page which contains important information like prices
  6. Start work on merkle trees
    • the work there focused on building an understanding of how the state is constructed
    • the implementation has started but doesn't yet contain a working tree with many positions

Missed items

  1. Figma designs
    • I was really excited about how the work was progressing in the decoding/hash/merkle tree side that I decided to postpone this to a later sprint

Plan for the next sprint

  1. Christmas time
    • Since it is the holidays I do not expect much work to happen in the next sprint
  2. Continue work on decoding
  3. Continue work on the merkle tree
  4. Figma designs
    • Still not high priority but maybe

Table design for state values 

interface state_updates {
  id: number // PK, doesn't auto-increment, calculated in backend
  fact_hash: Hash256 // INDEX
  root_hash: PedersenHash // INDEX
  fact_timestamp: UnixTime // time of the fact event
  data_timestamp: UnixTime // time specified in the pages
}

interface prices {
  state_update_id: number // FK to state_updates
  asset_id: AssetId // INDEX
  price: bigint
}

interface positions {
  state_update_id: number // Joint PK, FK to state_updates
  position_id: number // Joint PK
  public_key: StarkPublicKey
  collateral_balance: bigint 
}

interface position_assets {
  state_update_id: number // FK to position_assets
  position_id: number // FK to position_assets
  asset_id: AssetId // INDEX
  balance: bigint
  funding_index: bigint
}

Project update 11-05-2022

Progress

  1. Forced actions
    • UI for forced exits
    • force action buttons visible on position that you own
    • form for creating new exits / offers
  2. Forced trade offers
    • code for managing offers on the backend almost ready
  3. Search
    • working search
  4. Jazzicons
    • when you connect with metamask you now see a jazzicon for your account

Plan for the next sprint

  1. Finish work on forced trade offers
  2. Submit forced transactions through the UI

Project update 13-04-2022

Progress

  1. Safer code
    • Add more checks during data syncing to ensure that we calculate state correctly
    • Add a dev mode where the backend only syncs up to a certain block
  2. Eth addresses
    • Map between ETH addresses and stark keys
  3. Forced actions
    • Download forced action events from on chain
    • Decode forced action data from on chain data
    • Begin work on connecting the two
  4. Fixes
    • Include state updates which do not change any positions in the list
    • Unifty timestamp handling in the code

Plan for the next sprint

  1. Finish handling forced actions on the backend
  2. Submit forced transactions through the UI

Project update 02-02-2022

Progress

  1. Technical updates
    • Add a parameter to clear the local database before system start
    • Add automatic code coverage tracking
    • Simplify code by replacing ad-hoc algorithm with lodash
    • Replace string types with domain specific ones like Hash or AssetId
  2. Implement the state transition function
    • Carefully checked against the python implementation
  3. Handle chain reorganizations
    • Almost done, finishing touches on the PR are in progress
    • Will allow the system to always display an up to date state seconds after it happens on chain
    • It turned out to be a bit more complex then I thought initially, so I started to question my decision about not using the graph, but then realized that the problem wouldn't go away even if we used the graph, because we still have to compute the merkle tree and would still need to handle reorgs, but this time on the data provided by the graph.

Plan for the next sprint

  1. Store the full state in the database
  2. Add a basic unstyled UI for the data

Project update 30-03-2022

Progress

  1. More UI progress
    • Value of the position before the state update is now calculated
    • Asset icons are now added
    • State update page is added with details
    • Position update page is added with details
  2. DevOps stuff
    • We have configured review apps on heroku
  3. Fixes
    • We found out that the state updates were calculated incorrectly
    • It turns out that the verifiers were emitting events before being registered through the proxy and picked up by our backend
    • Additionally the logic of updating positions didn't update the fundingIndex on assets that weren't mentioned in the on chain data update
    • After fixing those issues we added a check to the resulting hashes to make sure they match the hashes present in the submitted on chain data

Plan for the next sprint

  1. Connect ethereum addresses and stark keys
  2. Maybe start work on forced transactions

Project update 16-03-2022

Progress

  1. Finish design work
  2. UI implementation
    • Some initial queries that can read data from the database and display it on the web were built
  3. Geopolitical issues
    • unfortunately due to the current world situation the progress wasn't as large as we expected

Plan for the next sprint

  1. Continue working on the UI implementation
  2. Fix some bugs that we found in the state transition code

Project update 25-05-2022

Progress

  1. Forced trade offers
    • manage offers in the database
    • require user signature to create offer
    • require user signature to cancel offer
    • check signatures for create, accept & cancel
    • create offers in the UI
    • show latest offers on the home page
  2. Forced exits
    • can now be sent from the UI
    • the form for exits is served from the backend with correct data
  3. UI changes
    • negative USD amounts are now properly formatted
    • price of USDC when viewing the position is now $1 instead of $0.01
    • long lists are now paginated on the client side
  4. User transactions
    • the backend now remembers transactions sent by users and tracks their status (e.g mined/reverted)

Plan for the next sprint

  1. Add UI for displaying and accepting offers and sending forced trade transactions
  2. General polishing

Add search (KISS - just check format and build link)

Search options:

  • state root hash - redirects to state update by id
  • state transition fact hash - redirects to state update by id
  • state update id - redirects to state update by id
  • position id - redirects to position by id (problem: positions and state updates share same id space, maybe we should prefix, e.g. states are s1, s5123 and positions are p1, p5123123)
  • ethereum address - redirects to position by id
  • stark key - redirects to position by id
  • transaction hash - redirects to forced transaction by hash
  • offer id - redirects to offer id (same problem, maybe o1, o2, o123)
    When nothing matches we display a 404 saying that the following search term was not found.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.