Comments (12)
I've set up basic structure using PythonOperator
, proceeding with moving all exports to python. It's tricky to use click.Command
s that are exported from ethereumetl.cli
: they are not callable in itself (pallets/click#40), which requires either setting up context manually and invoking it or using CliRunner
from testing utilities. Both approaches are rather unreliable and/or complicated, so I'm going to use instead undecorated functions which can be accessed using command.callback
. This will disable click parameter validation, but overall should be more reliable way to run jobs.
from ethereum-etl-airflow.
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
This issue now has a funding of 300.0 DAI (300.0 USD @ $1.0/DAI) attached to it as part of the Ethereum Foundation fund.
- If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
- Want to chip in? Add your own contribution here.
- Questions? Checkout Gitcoin Help or the Gitcoin Slack
- $34,861.24 more funded OSS Work available on the Gitcoin Issue Explorer
from ethereum-etl-airflow.
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
Work has been started.
These users each claimed they can complete the work by 1 week, 6 days ago.
Please review their action plans below:
1) evgeniuz has been approved to start work.
Will reimplement using PythonOperator, add logging and documentation and test it.
Learn more on the Gitcoin Issue Details page.
from ethereum-etl-airflow.
@evgeniuz send me an email to [email protected] from your Google email. I will give you access to GCP project.
from ethereum-etl-airflow.
@evgeniuz Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- reminder (3 days)
- escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
from ethereum-etl-airflow.
@evgeniuz Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- reminder (3 days)
- escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
from ethereum-etl-airflow.
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
@evgeniuz due to inactivity, we have escalated this issue to Gitcoin's moderation team. Let us know if you believe this has been done in error!
- reminder (3 days)
- escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
from ethereum-etl-airflow.
Reimplemented according to specs. Couple of notes:
- since
ethereumetl
uses default logger no logging configuration is needed: default logger is captured by airflow; - only
data
folder is mounted from bucket, all others are discarded when task ends, so using/tmp/
for temporary files is ok (no need to do manual cleanup); - in the PR there are few TODOs that make DAG run only once and export output into
export-dev
folder instead ofdev
- these are for easier debugging and to avoid overwritingexport
folder until changes are complete.
Steps left to do:
- update README;
- test
load_dag.py
.
from ethereum-etl-airflow.
@medvedev1088 Fixed minor issue with template formatting in load_dag
, will update README, but otherwise ready for review.
from ethereum-etl-airflow.
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
Work for 300.0 DAI (300.0 USD @ $1.0/DAI) has been submitted by:
@ceresstation please take a look at the submitted work:
- Learn more on the Gitcoin Issue Details page
- Want to chip in? Add your own contribution here.
- Questions? Checkout Gitcoin Help or the Gitcoin Slack
- $44,518.51 more funded OSS Work available on the Gitcoin Issue Explorer
from ethereum-etl-airflow.
|
⚡️ A *Polyglot Programmer* Kudos has been sent to @evgeniuz for this issue from @ceresstation. ⚡️
The sender had the following public comments:
Nice work @evgeniuz! |
from ethereum-etl-airflow.
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
The funding of 300.0 DAI (300.0 USD @ $1.0/DAI) attached to this issue has been approved & issued to @evgeniuz.
- Learn more on the Gitcoin Issue Details page
- Questions? Checkout Gitcoin Help or the Gitcoin Slack
- $44,114.00 more funded OSS Work available on the Gitcoin Issue Explorer
from ethereum-etl-airflow.
Related Issues (20)
- Add contract parsers for SushiSwap HOT 6
- Add DAG that queries data from BigQuery and saves it to csv
- Support parsing anonymous events
- Support parsing anonymous events
- etl & bigquery fee
- Chainlink parsing pointed at the wrong contracts HOT 2
- composer-1.8.3-airflow-1.10.3 not exist HOT 2
- Why is ethereum_load_dag's start date different from the first run date? HOT 2
- Missing logs_by_topic table HOT 3
- BigQuery Aggregator_ETH_USD_event_AnswerUpdated event not functioning
- Issue: Permission bigquery.tables.create denied on dataset blockchain-etl-internal:common (or it may not exist). HOT 1
- Create a function that generates SQL for a table definition that can be run in BigQuery HOT 1
- Missing `blocks` table when running first time HOT 1
- Failure building polygon_etl
- Add verification of empty fields
- add `contracts.bytecode_hash` field
- Deployment on Airflow fails due to Dependencies HOT 4
- Missing`_history` table
- DAG import error HOT 1
- tokens table and amended tokens view missing some tokens like PEPE. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ethereum-etl-airflow.