yearn / ydata

yData

Explore yearn through the realm of data analysis

• Live Link: https://ydata.vercel.app/

About

yData is a web app with analytics for the yearn ecosystem. These are the available visualizations:

Yearn V2 Vaults

You can scroll through many views in this section, including:

• Group vaults by many factors like Chain, Assets Under Management (AUM) size, Cumulative Return, Asset Type, and others.
• Daily Variation of Deposits and Withdraws, helps you track how much Total Value Locked (TVL) is entering and leaving the yearn protocol.
• Overview of cumulative return since inception plotted against monthly returns.

Efficient Frontier

An overview of yearn's vaults performance that considers risk factors so you can compare them with other DeFi assets. This table will plot yield options in the following axes:

• Return: How much yield does this option generates (higher is better)
• Volatility: How much does the yield vary over time (lower means less risk)

In this graph you will find a region where the most risk-efficient assets are, that is the "Efficient Frontier" (highlighted with the red crayon square):

Set up and run

1. Clone this repo: git clone [email protected]:yearn/ydata.git
2. Install dependencies: yarn
3. Run dev environment: yarn dev

Available scripts

• yarn dev: start dev environment
• yarn build: build for production
• yarn format: apply prettier formatting
• yarn lint: run linter

More data resources

• Yearn official docs
• About yVaults V2
• About yearn risk scores
• yDaemon: unified yearn data API

ydata's Issues

add prod link to yearn-devdocs when deployed

I think adding it in this page is a nice fit: https://github.com/yearn/yearn-devdocs/pull/262/files#diff-4f2dce047c7c113c15bef324dcbc7152a5995bdd7dcaae01e1419ce0abd32ec8

[Risk Score]: Update Maker-Lev Group

Update the following scores to Maker-Lev group here.

Code Review: 2

[Risk Score]: Create New Group Strategy Lender (Optimism)

Create new group:

"id": "strategylender",
"network": 10, // Optimism
"label": "Gen Lender",
"codeReviewScore": 3,
"testingScore": 3,
"auditScore": 4,
"protocolSafetyScore": 3,
"complexityScore": 3,
"teamKnowledgeScore": 3,
"criteria": {
"nameLike": ["strategylender", "genericlender"],
"strategies": [],
"exclude": []
}

Please, add it to the Others group too.

Refactor urls and constants into a single file

we have these capital-cased constants everywhere! (e.g. URLs)
we could start gathering them in a single file

Sync the risk framework data

• there is a mismatch between the risk framework for legacy.yearn.watch and yearn.watch, check if the difference can be explained by yearn/yearn-watch#3, if not note the extra for further discussion
• add the groups that exist on multiple chains but are only addressed for a single chain (e.g. vesper)
• create a spreadsheet that lists all the missing strategies and the corresponding actions
• add the missing strategies to API
• [ ] create PRs for the two ywatch repos to automatically give the 'others' group scores of 5 (excl. tvl and longevity impacts)

Gas fee optimization

Optimizing timing of transactions to minimize gas fees

Item No. 4 in the team spreadsheet

Source of yields

Related to Item No. 1 and 4 in the Additional Metrics section of the vault risk google docs

Break down the sources of yields for each of our strategies.
I believe that within this process we should also be able to detect whether a strategy holds a non-expected token, i.e., a token that the strategy did not plan to hold

Some research objectives related to this topic are:

• detect non-expected tokens interacting with a strategy
• protocol concentration
• detect like:like pairs on liquidity pools, i.e., pools that have different names but essentially hold the same tokens
• classify the sources of profit for our vaults, e.g. liquidity mining, lending, trading fees
• describe how much of a vault's profit can be attributed to each of these profit sources

[Risk Score]: Update Maker-Lev Group (part 2)

Update the following scores to Maker-Lev group here.

Team Knowledge Score: 2 (FP, Slim and Mil0x + flashfish)

[Risk Score] - Modify Audit Score in Goldfinch

Please, modify the audit score in this group.

New value: 5
Previous Value: 4

[Risk Score] Angle - Update Scores

Modify some scores in the Angle group here

Team Knowledge Score: 2 (Charles, slim, and mil0x)
Code Review Score": 2 (1 internal sec rev + 2 ext sec rev -Dave and Dedaub-)

Change in share prices not reflected in the TVL distribution

https://github.com/yearn/yearn-data-analytics/blob/master/research/tvl_concentration.ipynb

the calculation assumes that the share prices stay the same (or equivalently the APY = 0) throughout the whole period

Timing of YFI buybacks

Item No. 2 in the team spreadsheet and possibly related to #1

We want to optimize the timing of the YFI buybacks in the sense of minimum energy control, i.e., bring the value of YFI higher while spending less.

Create Inactive group

The risk framework json file: https://github.com/yearn/yearn-data-analytics/blob/master/src/risk_framework/risks.json
how the keys in the json file are parsed in yearn.watch: https://github.com/yearn/yearn-watch/blob/36ad03627b9e81f71b70414771a2de95ecf735cd/utils/filters.tsx#L60-L76

It would be more informative if we could differentiate between the strategies that are inactive vs. strategies that have tvl but do not belong to a risk group.

Currently all strategies in the Others group (for 3 all chains) except StrategyHECStakerBoo are inactive.
Thus I suggest we create another group with label "Inactive" for all 3 chains, and move all the nameLikes from the Others groups to these new groups respectively except for StrategyHECStakerBoo.

More specifically the expected actions for closing this issue are the following:

• Check if all strategies in the Others groups are indeed inactive (e.g. we can query their labels in yearn.watch, or read their properties directly from etherscan)
• Create new groups named Inactive for all chains
• Move all nameLikes from the Others group except for those that are still active

Investigate on the "Max rate limit reached" error when fetching erc20 transaction history

api_cache_1  | 2022-06-04 14:22:53.012 ERROR network: Failed to fetch erc20 transfers from address=0x9AA08BA012B2f2c002BbC56a8c3ECE5b3E8299d3; Retrying in 1.0 seconds (1/5)
api_cache_1  | 2022-06-04 14:22:54.014 ERROR web3: Max rate limit reached
api_cache_1  | 2022-06-04 14:22:54.014 ERROR network: Failed to fetch erc20 transfers from address=0x9AA08BA012B2f2c002BbC56a8c3ECE5b3E8299d3; Retrying in 2 seconds (2/5)
api_cache_1  | 2022-06-04 14:22:56.017 ERROR web3: Max rate limit reached
api_cache_1  | 2022-06-04 14:22:56.017 ERROR network: Failed to fetch erc20 transfers from address=0x9AA08BA012B2f2c002BbC56a8c3ECE5b3E8299d3; Retrying in 4 seconds (3/5)
fastapi_1    | INFO:     172.31.28.198:47434 - "GET / HTTP/1.1" 200 OK
api_cache_1  | 2022-06-04 14:23:00.021 ERROR web3: Max rate limit reached
api_cache_1  | 2022-06-04 14:23:00.022 ERROR network: Failed to fetch erc20 transfers from address=0x9AA08BA012B2f2c002BbC56a8c3ECE5b3E8299d3; Retrying in 8 seconds (4/5)
fastapi_1    | INFO:     172.31.44.38:55070 - "GET / HTTP/1.1" 200 OK
api_cache_1  | 2022-06-04 14:23:08.030 ERROR web3: Max rate limit reached
api_cache_1  | 2022-06-04 14:23:08.030 ERROR network: Failed to fetch erc20 transfers from address=0x9AA08BA012B2f2c002BbC56a8c3ECE5b3E8299d3; Retrying in 16 seconds (5/5)
fastapi_1    | INFO:     172.31.28.198:47480 - "GET / HTTP/1.1" 200 OK
fastapi_1    | INFO:     172.31.44.38:55108 - "GET / HTTP/1.1" 200 OK
api_cache_1  | 2022-06-04 14:23:24.035 ERROR web3: Max rate limit reached
api_cache_1  | 2022-06-04 14:23:24.035 ERROR network: Failed to fetch erc20 transfers from address=0x9AA08BA012B2f2c002BbC56a8c3ECE5b3E8299d3

The retries and backoff delays for the erc20 transfers do not seem to be solving this rate limit problem, I think this may be coming from another reason.

The token txns query for the address works fine on etherscan:
https://etherscan.io/address/0x9AA08BA012B2f2c002BbC56a8c3ECE5b3E8299d3#tokentxns

Write better tests for the vaults and strategies

Now since we are loosening up the API loop by adding error handlers, I believe it is time to start writing stricter tests for our vaults and strategies.
We should make sure that the values for each vault and for each strategy match the values shown in yearn.watch, and also that the risk scores are calculated as intended.
I think we could first write tests for one vault/strategy per chain.

The following test files should be modified (but ofc more the better):

• tests/yearn/test_vaults.py
• tests/yearn/test_strategies.py
• tests/risk_framework/test_analysis.py

[Risk Score]: Update IDLE strategy score

Add nameLike argument to existing score.

https://github.com/yearn/yearn-data-analytics/blob/0ee5ee086ec2f2f0a632b005282cac3d2c3c0661/src/risk_framework/risks.json#L82

Update to:

{

	"criteria": {
		"nameLike": ["StrategyIdleV2", "idle"],
		...
	}
},

Fetch the wallet balance from the subgraph

Thanks to @bensams0
We can fetch the balance of the depositors directly from the subgraph,
e.g. for the Mainnet we could use the following query

{
  accountVaultPositions(first: 5,
    where:{vault:"0xdcd90c7f6324cfa40d7169ef80b12031770b4325"},
    orderBy: balanceShares,
    orderDirection: desc) {
    id
    balanceShares
    account {
      id
    }
  }
}

this removes the need to track sharePrices for the calculation of the NPV of deposits since the balance naturally tracks the accumulated profits

Upload csv files for easier research

Should crawl and upload data related to our vaults and strategies from the exporter

Change the primary key for the sqlmodels

currently the sqlmodels use just the address as the primary key,
but theoretically different vaults on different chains could have the same address
so to be safe I think we should either just create another integer primary key or use both the network id and the address to create a unique key

Change the data source to the new API

Still Under Progress!!!

I am opening up this issue to share the upcoming change beforehand, so no need to switch it up right now.
If anyone is working on incorporating additional data sources make sure to check what the new API has to offer :)

yDaemon

We now have an awesome next-gen API that integrates and sanitizes data from a number of different data sources

We could now fetch both the vaults and strategies directly from this new endpoint, instead of fetching the vault data from the Yearn SDK cache and the strategy data from the Yearn Meta separately.

The following is a copy pasta of some brief description from Major, pls refer to the yDaemon repo for more details

• new route https://api.ycorpo.com/[chainID]/vaults/[address] to get the info for one specific vault
• new route https://api.ycorpo.com/info/chains to list the supported chainID
• new route https://api.ycorpo.com/info/vaults/blacklisted to list the vaults ignored by the API

Also added a few arguments on the vaults/all page:

• ?skip=N will skip N vault from the graphQL query. Default is 0
• ?first=N will limit the result to N vaults on the graphQL query. Default is 1000
• ?orderBy=S will order the result by S on the graphQL query. Default is activation
• ?orderDirection=asc|desc will order the result by ascending or descending on the graphQL query. Default is desc. (also available for one specific address)

[Risk Score]: Create New Group Generic Lev Aave v3 (Optimism)

Create new group:

"id": "genlevaavev3-optimism",
"network": 10, // Optimism
"label": "Generic Lev Aave v3",
"codeReviewScore": 3,
"testingScore": 3,
"auditScore": 5,
"protocolSafetyScore": 3,
"complexityScore": 4,
"teamKnowledgeScore": 2,
"criteria": {
"nameLike": ["genlevaavev3-optimism"],
"strategies": [],
"exclude": []
}

Add it into the Others group.

[Risk Score]: Update Angle Group

Update the following score to the Angle group here.

Audit Score: 4 (1 audit 3 months ago). More info

[Risk Score] - Improve Angle Protocol Scores

New scores

Protocol Safety Score: 2 (DD, multisig, bug bounties, verified contracts, audited at least by 2 firms).

Historical, current and forward modeled APY

Item No. 12 in the team spreadsheet

It would be nice to start off by calculating the APY from our historical data using the exporter.
The tentative goals for this item would be the following:

• Calculate historical APY for various window sizes (e.g. 1 day, 30 days, 3 months) for each vault
• Identify noise levels in the time series
• Identify (short-term) trend factors in the time series if there are any

[Risk Score]: Updates to joing provider balancer and adding UniV3 Joint Provider

comment out this one since it may interfere with the other jointProvider and does not have any live strategy in this group at the moment, also check the others group for global exclude on string JointProvider

{
"id": "jointproviderbalancer",
"network": 1,
"label": "Joint Provider Balancer",
"codeReviewScore": 2,
"testingScore": 2,
"auditScore": 4,
"protocolSafetyScore": 1,
"complexityScore": 4,
"teamKnowledgeScore": 3,
"criteria": {
"nameLike": ["JointProvider"],
"strategies": [],
"exclude": []
}
},

Add this new risk score

{
"id": "NoHedgeUniV3StablesJoint",
"network": 1,
"label": "NoHedgeUniV3StablesJoint",
"codeReviewScore": 2,
"testingScore": 3,
"auditScore": 5,
"protocolSafetyScore": 1,
"complexityScore": 4,
"teamKnowledgeScore": 4,
"criteria": {
"nameLike": ["NoHedgeUniV3StablesJoint"],
"strategies": [],
"exclude": []
}
},
NoHedgeUniV3StablesJoint

GenLender FTM: Update Score

Update the score for the group "GenLender Ftm"

Update:

Audit Score: 5 (instead of 4)

Bug in /src/services/*/Dockerfile

https://github.com/yearn/yearn-data-analytics/blob/6344b0f1ce8e7b71e5b9efa98248108e1793b3b0/src/services/api_cache/Dockerfile#L8

https://github.com/yearn/yearn-data-analytics/blob/6344b0f1ce8e7b71e5b9efa98248108e1793b3b0/src/services/fastapi/Dockerfile#L8

pip install poetry >=1.0.0 installs poetry==1.1.13 and saves the installation log into ./'=1.0.0'.
Desired command would be pip install 'poetry>=1.0.0'.
But this still installs the same version (1.1.13) with the above command.
Wonder if the poetry version matters?

root@yda:/app# pip install poetry>=1.0.0 
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
root@yda:/app# ls
'=1.0.0'   README.md   data   docker-compose.yml   examples   poetry.lock   pyproject.toml   research   src   tests
root@yda:/app# cat '=1.0.0' 
Collecting poetry
  Using cached poetry-1.1.13-py2.py3-none-any.whl (175 kB)
Requirement already satisfied: pexpect<5.0.0,>=4.7.0 in /usr/local/lib/python3.9/site-packages (from poetry) (4.8.0)
Requirement already satisfied: requests-toolbelt<0.10.0,>=0.9.1 in /usr/local/lib/python3.9/site-packages (from poetry) (0.9.1)
Requirement already satisfied: requests<3.0,>=2.18 in /usr/local/lib/python3.9/site-packages (from poetry) (2.28.0)
Requirement already satisfied: cachy<0.4.0,>=0.3.0 in /usr/local/lib/python3.9/site-packages (from poetry) (0.3.0)
Requirement already satisfied: shellingham<2.0,>=1.1 in /usr/local/lib/python3.9/site-packages (from poetry) (1.4.0)
Requirement already satisfied: clikit<0.7.0,>=0.6.2 in /usr/local/lib/python3.9/site-packages (from poetry) (0.6.2)
Requirement already satisfied: poetry-core<1.1.0,>=1.0.7 in /usr/local/lib/python3.9/site-packages (from poetry) (1.0.8)
Requirement already satisfied: crashtest<0.4.0,>=0.3.0 in /usr/local/lib/python3.9/site-packages (from poetry) (0.3.1)
Requirement already satisfied: virtualenv<21.0.0,>=20.0.26 in /usr/local/lib/python3.9/site-packages (from poetry) (20.14.1)
Requirement already satisfied: cleo<0.9.0,>=0.8.1 in /usr/local/lib/python3.9/site-packages (from poetry) (0.8.1)
Requirement already satisfied: cachecontrol[filecache]<0.13.0,>=0.12.9 in /usr/local/lib/python3.9/site-packages (from poetry) (0.12.11)
Requirement already satisfied: html5lib<2.0,>=1.0 in /usr/local/lib/python3.9/site-packages (from poetry) (1.1)
Requirement already satisfied: keyring>=21.2.0 in /usr/local/lib/python3.9/site-packages (from poetry) (23.6.0)
Requirement already satisfied: packaging<21.0,>=20.4 in /usr/local/lib/python3.9/site-packages (from poetry) (20.9)
Requirement already satisfied: tomlkit<1.0.0,>=0.7.0 in /usr/local/lib/python3.9/site-packages (from poetry) (0.11.0)
Requirement already satisfied: pkginfo<2.0,>=1.4 in /usr/local/lib/python3.9/site-packages (from poetry) (1.8.3)
Requirement already satisfied: msgpack>=0.5.2 in /usr/local/lib/python3.9/site-packages (from cachecontrol[filecache]<0.13.0,>=0.12.9->poetry) (1.0.4)
Requirement already satisfied: lockfile>=0.9 in /usr/local/lib/python3.9/site-packages (from cachecontrol[filecache]<0.13.0,>=0.12.9->poetry) (0.12.2)
Requirement already satisfied: pastel<0.3.0,>=0.2.0 in /usr/local/lib/python3.9/site-packages (from clikit<0.7.0,>=0.6.2->poetry) (0.2.1)
Requirement already satisfied: pylev<2.0,>=1.3 in /usr/local/lib/python3.9/site-packages (from clikit<0.7.0,>=0.6.2->poetry) (1.4.0)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.9/site-packages (from html5lib<2.0,>=1.0->poetry) (1.16.0)
Requirement already satisfied: webencodings in /usr/local/lib/python3.9/site-packages (from html5lib<2.0,>=1.0->poetry) (0.5.1)
Requirement already satisfied: importlib-metadata>=3.6 in /usr/local/lib/python3.9/site-packages (from keyring>=21.2.0->poetry) (4.11.4)
Requirement already satisfied: SecretStorage>=3.2 in /usr/local/lib/python3.9/site-packages (from keyring>=21.2.0->poetry) (3.3.2)
Requirement already satisfied: jeepney>=0.4.2 in /usr/local/lib/python3.9/site-packages (from keyring>=21.2.0->poetry) (0.8.0)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.9/site-packages (from packaging<21.0,>=20.4->poetry) (3.0.9)
Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.9/site-packages (from pexpect<5.0.0,>=4.7.0->poetry) (0.7.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry) (1.26.9)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry) (2022.5.18.1)
Requirement already satisfied: platformdirs<3,>=2 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry) (2.5.2)
Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry) (0.3.4)
Requirement already satisfied: filelock<4,>=3.2 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry) (3.7.1)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/site-packages (from importlib-metadata>=3.6->keyring>=21.2.0->poetry) (3.8.0)
Requirement already satisfied: cryptography>=2.0 in /usr/local/lib/python3.9/site-packages (from SecretStorage>=3.2->keyring>=21.2.0->poetry) (37.0.2)
Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.9/site-packages (from cryptography>=2.0->SecretStorage>=3.2->keyring>=21.2.0->poetry) (1.15.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=2.0->SecretStorage>=3.2->keyring>=21.2.0->poetry) (2.21)
Installing collected packages: poetry
Successfully installed poetry-1.1.13

After pip uninstall poetry,

root@yda:/app# pip install 'poetry>=1.0.0'
Collecting poetry>=1.0.0
  Using cached poetry-1.1.13-py2.py3-none-any.whl (175 kB)
Requirement already satisfied: packaging<21.0,>=20.4 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (20.9)
Requirement already satisfied: poetry-core<1.1.0,>=1.0.7 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (1.0.8)
Requirement already satisfied: requests-toolbelt<0.10.0,>=0.9.1 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (0.9.1)
Requirement already satisfied: crashtest<0.4.0,>=0.3.0 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (0.3.1)
Requirement already satisfied: cachecontrol[filecache]<0.13.0,>=0.12.9 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (0.12.11)
Requirement already satisfied: cachy<0.4.0,>=0.3.0 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (0.3.0)
Requirement already satisfied: html5lib<2.0,>=1.0 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (1.1)
Requirement already satisfied: pkginfo<2.0,>=1.4 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (1.8.3)
Requirement already satisfied: tomlkit<1.0.0,>=0.7.0 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (0.11.0)
Requirement already satisfied: virtualenv<21.0.0,>=20.0.26 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (20.14.1)
Requirement already satisfied: clikit<0.7.0,>=0.6.2 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (0.6.2)
Requirement already satisfied: shellingham<2.0,>=1.1 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (1.4.0)
Requirement already satisfied: pexpect<5.0.0,>=4.7.0 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (4.8.0)
Requirement already satisfied: keyring>=21.2.0 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (23.6.0)
Requirement already satisfied: requests<3.0,>=2.18 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (2.28.0)
Requirement already satisfied: cleo<0.9.0,>=0.8.1 in /usr/local/lib/python3.9/site-packages (from poetry>=1.0.0) (0.8.1)
Requirement already satisfied: msgpack>=0.5.2 in /usr/local/lib/python3.9/site-packages (from cachecontrol[filecache]<0.13.0,>=0.12.9->poetry>=1.0.0) (1.0.4)
Requirement already satisfied: lockfile>=0.9 in /usr/local/lib/python3.9/site-packages (from cachecontrol[filecache]<0.13.0,>=0.12.9->poetry>=1.0.0) (0.12.2)
Requirement already satisfied: pylev<2.0,>=1.3 in /usr/local/lib/python3.9/site-packages (from clikit<0.7.0,>=0.6.2->poetry>=1.0.0) (1.4.0)
Requirement already satisfied: pastel<0.3.0,>=0.2.0 in /usr/local/lib/python3.9/site-packages (from clikit<0.7.0,>=0.6.2->poetry>=1.0.0) (0.2.1)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.9/site-packages (from html5lib<2.0,>=1.0->poetry>=1.0.0) (1.16.0)
Requirement already satisfied: webencodings in /usr/local/lib/python3.9/site-packages (from html5lib<2.0,>=1.0->poetry>=1.0.0) (0.5.1)
Requirement already satisfied: jeepney>=0.4.2 in /usr/local/lib/python3.9/site-packages (from keyring>=21.2.0->poetry>=1.0.0) (0.8.0)
Requirement already satisfied: SecretStorage>=3.2 in /usr/local/lib/python3.9/site-packages (from keyring>=21.2.0->poetry>=1.0.0) (3.3.2)
Requirement already satisfied: importlib-metadata>=3.6 in /usr/local/lib/python3.9/site-packages (from keyring>=21.2.0->poetry>=1.0.0) (4.11.4)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.9/site-packages (from packaging<21.0,>=20.4->poetry>=1.0.0) (3.0.9)
Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.9/site-packages (from pexpect<5.0.0,>=4.7.0->poetry>=1.0.0) (0.7.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry>=1.0.0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry>=1.0.0) (1.26.9)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry>=1.0.0) (2022.5.18.1)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/site-packages (from requests<3.0,>=2.18->poetry>=1.0.0) (2.0.12)
Requirement already satisfied: filelock<4,>=3.2 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry>=1.0.0) (3.7.1)
Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry>=1.0.0) (0.3.4)
Requirement already satisfied: platformdirs<3,>=2 in /usr/local/lib/python3.9/site-packages (from virtualenv<21.0.0,>=20.0.26->poetry>=1.0.0) (2.5.2)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/site-packages (from importlib-metadata>=3.6->keyring>=21.2.0->poetry>=1.0.0) (3.8.0)
Requirement already satisfied: cryptography>=2.0 in /usr/local/lib/python3.9/site-packages (from SecretStorage>=3.2->keyring>=21.2.0->poetry>=1.0.0) (37.0.2)
Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.9/site-packages (from cryptography>=2.0->SecretStorage>=3.2->keyring>=21.2.0->poetry>=1.0.0) (1.15.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=2.0->SecretStorage>=3.2->keyring>=21.2.0->poetry>=1.0.0) (2.21)
Installing collected packages: poetry
Successfully installed poetry-1.1.13
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
root@yda:/app# poetry --version
Poetry version 1.1.13

Profile and optimize the number of web3 calls + api calls

I have temporarily removed the caching from our API to allow us to host the service in a lightweight instance in AWS,
but this may increase the number of web3 calls and API calls we make.
I believe that this could be remedied by exporting the query responses we know for sure that they would not change, to a database.

An example workflow for solving this issue is the following:

• profile the average number of web3 calls to the web3 provider, estimate its pricing
• do the same for the block explorers e.g. etherscan
• compare the pricing with the price increase for bumping up the memory size (e.g. t2.micro -> t2.medium)
• if it is profitable to scale up, add caching with fixed sizes and bump up the instance
• export the cache to a database
• scale down the instance

Vault TVL distribution across wallets

Item No. 14 in the team spreadsheet

There is some interest in analyzing the wallets that deposit to yearn, to gain customer insights, and to enhance the performance of our vaults.
We would like to be able to provide answers to the following questions:

• How much of the TVL is concentrated in the top 10 wallets?
  * [ ] Are the withdrawals from last month largely from a specific chain or a specific vault?
  * [ ] How many are retail customers and how many are our b2b partners?
  * [ ] Which event (e.g. protocol migration) affected the recent increase/decrease in TVL?

Stargate Group: Improve Code Review Score

Stargate group:

Ethereum:
Fantom

Update "codeReviewScore": 2

Handle timeout errors for requests

I believe we should be retrying the request if the SDK returns the timeout error (522)
and skipping the data if yearn meta returns the timeout

TVL Data Analysis

Complete analysis on TVL drop off

B2B Dashboard

Develop a B2B dashboard to support integrators

poetry install fails in docker container

Problem

poetry install fails because of pygments and ruamel.yaml.clib

Environment

HW: MacBook Pro (14-inch, 2021) / Apple M1 Pro
OS: macOS Monterey 12.2.1
Docker: 20.10.14
Image: python:3.9
Python: 3.9.13

Log

docker run -it -v ~/repos/yearn-data-analytics/:/app -h yda --name yda --rm python:3.9 /bin/bash

root@yda:/# pip install poetry
Collecting poetry
  Downloading poetry-1.1.13-py2.py3-none-any.whl (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 KB 8.0 MB/s eta 0:00:00
Collecting clikit<0.7.0,>=0.6.2
  Downloading clikit-0.6.2-py2.py3-none-any.whl (91 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 91.8/91.8 KB 12.1 MB/s eta 0:00:00
Collecting requests-toolbelt<0.10.0,>=0.9.1
  Downloading requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.3/54.3 KB 9.3 MB/s eta 0:00:00
Collecting html5lib<2.0,>=1.0
  Downloading html5lib-1.1-py2.py3-none-any.whl (112 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 112.2/112.2 KB 9.2 MB/s eta 0:00:00
Collecting poetry-core<1.1.0,>=1.0.7
  Downloading poetry_core-1.0.8-py2.py3-none-any.whl (425 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 425.0/425.0 KB 11.1 MB/s eta 0:00:00
Collecting pkginfo<2.0,>=1.4
  Downloading pkginfo-1.8.3-py2.py3-none-any.whl (26 kB)
Collecting tomlkit<1.0.0,>=0.7.0
  Downloading tomlkit-0.11.0-py3-none-any.whl (34 kB)
Collecting virtualenv<21.0.0,>=20.0.26
  Downloading virtualenv-20.14.1-py2.py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 11.1 MB/s eta 0:00:00
Collecting cachy<0.4.0,>=0.3.0
  Downloading cachy-0.3.0-py2.py3-none-any.whl (20 kB)
Collecting cachecontrol[filecache]<0.13.0,>=0.12.9
  Downloading CacheControl-0.12.11-py2.py3-none-any.whl (21 kB)
Collecting keyring>=21.2.0
  Downloading keyring-23.6.0-py3-none-any.whl (34 kB)
Collecting packaging<21.0,>=20.4
  Downloading packaging-20.9-py2.py3-none-any.whl (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 KB 6.8 MB/s eta 0:00:00
Collecting pexpect<5.0.0,>=4.7.0
  Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.0/59.0 KB 9.7 MB/s eta 0:00:00
Collecting requests<3.0,>=2.18
  Downloading requests-2.28.0-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 KB 11.8 MB/s eta 0:00:00
Collecting shellingham<2.0,>=1.1
  Downloading shellingham-1.4.0-py2.py3-none-any.whl (9.4 kB)
Collecting crashtest<0.4.0,>=0.3.0
  Downloading crashtest-0.3.1-py3-none-any.whl (7.0 kB)
Collecting cleo<0.9.0,>=0.8.1
  Downloading cleo-0.8.1-py2.py3-none-any.whl (21 kB)
Collecting msgpack>=0.5.2
  Downloading msgpack-1.0.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (313 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 313.6/313.6 KB 9.9 MB/s eta 0:00:00
Collecting lockfile>=0.9
  Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Collecting pylev<2.0,>=1.3
  Downloading pylev-1.4.0-py2.py3-none-any.whl (6.1 kB)
Collecting pastel<0.3.0,>=0.2.0
  Downloading pastel-0.2.1-py2.py3-none-any.whl (6.0 kB)
Collecting six>=1.9
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting webencodings
  Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting importlib-metadata>=3.6
  Downloading importlib_metadata-4.11.4-py3-none-any.whl (18 kB)
Collecting SecretStorage>=3.2
  Downloading SecretStorage-3.3.2-py3-none-any.whl (15 kB)
Collecting jeepney>=0.4.2
  Downloading jeepney-0.8.0-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.4/48.4 KB 10.3 MB/s eta 0:00:00
Collecting pyparsing>=2.0.2
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 KB 10.1 MB/s eta 0:00:00
Collecting ptyprocess>=0.5
  Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2022.5.18.1-py3-none-any.whl (155 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.2/155.2 KB 12.6 MB/s eta 0:00:00
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.0/139.0 KB 13.4 MB/s eta 0:00:00
Collecting idna<4,>=2.5
  Downloading idna-3.3-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 KB 16.1 MB/s eta 0:00:00
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting filelock<4,>=3.2
  Downloading filelock-3.7.1-py3-none-any.whl (10 kB)
Collecting platformdirs<3,>=2
  Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB)
Collecting distlib<1,>=0.3.1
  Downloading distlib-0.3.4-py2.py3-none-any.whl (461 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 461.2/461.2 KB 10.9 MB/s eta 0:00:00
Collecting zipp>=0.5
  Downloading zipp-3.8.0-py3-none-any.whl (5.4 kB)
Collecting cryptography>=2.0
  Downloading cryptography-37.0.2-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 10.9 MB/s eta 0:00:00
Collecting cffi>=1.12
  Downloading cffi-1.15.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (447 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 447.6/447.6 KB 11.2 MB/s eta 0:00:00
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 10.5 MB/s eta 0:00:00
Installing collected packages: webencodings, pylev, ptyprocess, msgpack, lockfile, distlib, zipp, urllib3, tomlkit, six, shellingham, pyparsing, pycparser, poetry-core, platformdirs, pkginfo, pexpect, pastel, jeepney, idna, filelock, crashtest, charset-normalizer, certifi, cachy, virtualenv, requests, packaging, importlib-metadata, html5lib, clikit, cffi, requests-toolbelt, cryptography, cleo, cachecontrol, SecretStorage, keyring, poetry
Successfully installed SecretStorage-3.3.2 cachecontrol-0.12.11 cachy-0.3.0 certifi-2022.5.18.1 cffi-1.15.0 charset-normalizer-2.0.12 cleo-0.8.1 clikit-0.6.2 crashtest-0.3.1 cryptography-37.0.2 distlib-0.3.4 filelock-3.7.1 html5lib-1.1 idna-3.3 importlib-metadata-4.11.4 jeepney-0.8.0 keyring-23.6.0 lockfile-0.12.2 msgpack-1.0.4 packaging-20.9 pastel-0.2.1 pexpect-4.8.0 pkginfo-1.8.3 platformdirs-2.5.2 poetry-1.1.13 poetry-core-1.0.8 ptyprocess-0.7.0 pycparser-2.21 pylev-1.4.0 pyparsing-3.0.9 requests-2.28.0 requests-toolbelt-0.9.1 shellingham-1.4.0 six-1.16.0 tomlkit-0.11.0 urllib3-1.26.9 virtualenv-20.14.1 webencodings-0.5.1 zipp-3.8.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
root@yda:/# cd /app
root@yda:/app# poetry install
Creating virtualenv yearn-data-analytics-9TtSrW0h-py3.9 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 116 installs, 0 updates, 0 removals

  • Installing toolz (0.11.2)
  • Installing cytoolz (0.11.2)
  • Installing eth-hash (0.3.2)
  • Installing eth-typing (2.3.0)
  • Installing eth-utils (1.10.0)
  • Installing six (1.16.0)
  • Installing asttokens (2.0.5)
  • Installing base58 (2.1.1)
  • Installing certifi (2022.5.18.1)
  • Installing charset-normalizer (2.0.12)
  • Installing eth-keys (0.3.4)
  • Installing executing (0.8.3)
  • Installing frozenlist (1.3.0)
  • Installing hexbytes (0.2.2)
  • Installing idna (3.3)
  • Installing multidict (6.0.2)
  • Installing netaddr (0.8.0)
  • Installing parsimonious (0.8.1)
  • Installing parso (0.8.3)
  • Installing ptyprocess (0.7.0)
  • Installing pycryptodome (3.14.1)
  • Installing pure-eval (0.2.2)
  • Installing rlp (2.0.1)
  • Installing sniffio (1.2.0)
  • Installing traitlets (5.2.2.post1)
  • Installing urllib3 (1.26.9)
  • Installing varint (1.0.2)
  • Installing wcwidth (0.2.5)
  • Installing aiosignal (1.2.0)
  • Installing anyio (3.6.1)
  • Installing async-timeout (4.0.2)
  • Installing attrs (21.4.0)
  • Installing backcall (0.2.0)
  • Installing bitarray (1.2.2): Installing...
  • Installing decorator (5.1.1)
  • Installing distlib (0.3.4)
  • Installing decorator (5.1.1)
  • Installing distlib (0.3.4)
  • Installing bitarray (1.2.2)
  • Installing decorator (5.1.1)
  • Installing distlib (0.3.4)
  • Installing entrypoints (0.4)
  • Installing eth-abi (2.1.1)
  • Installing eth-keyfile (0.5.1)
  • Installing eth-rlp (0.2.1)
  • Installing filelock (3.7.1)
  • Installing greenlet (1.1.2)
  • Installing jedi (0.18.1)
  • Installing jupyter-core (4.10.0)
  • Installing matplotlib-inline (0.1.3)
  • Installing multiaddr (0.0.9)
  • Installing nest-asyncio (1.5.5)
  • Installing pexpect (4.8.0)
  • Installing pickleshare (0.7.5)
  • Installing platformdirs (2.5.2)
  • Installing prompt-toolkit (3.0.29)
  • Installing pygments (2.12.0): Failed

  TooManyRedirects

  Exceeded 30 redirects.

  at /usr/local/lib/python3.9/site-packages/requests/sessions.py:191 in resolve_redirects
      187│             except (ChunkedEncodingError, ContentDecodingError, RuntimeError):
      188│                 resp.raw.read(decode_content=False)
      189│ 
      190│             if len(resp.history) >= self.max_redirects:
    → 191│                 raise TooManyRedirects(
      192│                     f"Exceeded {self.max_redirects} redirects.", response=resp
      193│                 )
      194│ 
      195│             # Release the connection back into the pool.

  • Installing pyparsing (3.0.9)
  • Installing pyrsistent (0.18.1)
  • Installing python-dateutil (2.8.2)
  • Installing pyzmq (23.1.0)
  • Installing requests (2.27.1)
  • Installing ruamel.yaml.clib (0.2.6): Failed

  TooManyRedirects

  Exceeded 30 redirects.

  at /usr/local/lib/python3.9/site-packages/requests/sessions.py:191 in resolve_redirects
      187│             except (ChunkedEncodingError, ContentDecodingError, RuntimeError):
      188│                 resp.raw.read(decode_content=False)
      189│ 
      190│             if len(resp.history) >= self.max_redirects:
    → 191│                 raise TooManyRedirects(
      192│                     f"Exceeded {self.max_redirects} redirects.", response=resp
      193│                 )
      194│ 
      195│             # Release the connection back into the pool.

  • Installing stack-data (0.2.0)
  • Installing tornado (6.1)
  • Installing typing-extensions (4.2.0)
  • Installing yarl (1.7.2)

[Risk Score]: Create Auto-Compounding Aura Strategy Group

Create new group:

"id": "auto-compounding aura",
"label": "Auto-Compounding Aura",
"codeReviewScore": 3,
"testingScore": 3,
"auditScore": 4,
"protocolSafetyScore": 4,
"complexityScore": 3,
"teamKnowledgeScore": 3
"nameLike": ["auto-compounding aura"]

Please, add it to the Others group too.

Customer visualization

Customer visualization through clustering

https://hackmd.io/FIlgq9b-TCGiID1DJeHbww
https://chart-studio.plotly.com/~borrito/3/#/

Should answer the following questions (moved from #2)

• Are the withdrawals from last month largely from a specific chain or a specific vault?
• How many are retail customers and how many are our b2b partners?~~
• Which event (e.g. protocol migration) affected the recent increase/decrease in TVL?

Refactor env variables, enum, dataclass and typed dict

Would be good to have these typings and constants organized into a config folder together with the constant file in #24, so that they can be shared throughout the app.

Add retries to web3 calls and api calls

Some requests could be dropped due to temporary bugs or API rate limiting, we should create a max retries constant and repeat our calls if the response was not successful.
We should decide on the number of retries probably after #22

Vault-level aggregation

As a minimal proof of concept:

1. let's show what aggregation looks like at the vault level
2. start with existing metrics: yearn strategy evaluations aggregated by TVL
3. how do we combine future metrics in a robust fashion?

As a way forward, my idea for yearn strategy evals is:

1. accept that they're qualitative judgements, and don't try to solve for an optimal quantitative aggregation
2. as such, simply collect several "common sense" weights to allow aggregation, maybe from a few different sources: yearn teams, protocol partners, etc.
3. find a way to display these while preserving the "uncertainty" around their aggregation, maybe by displaying a range of risk score (4.4-4.6) instead of a point estimate (4.5), or allowing users to select which subjective weighting they prefer. perhaps a user of Protocol Z would prefer to use the subjective weightings contributed by Protocol Z.

[Risk Score]: Update maker lev group

Update the following scores to Maker-Lev group here.

Audit Score: 4

Maker + Arrakis Finance + Gelato Network

https://security.makerdao.com/audit-reports
https://github.com/ArrakisFinance/vault-v1-core/blob/main/audit/Arrakis_Finance_Audit_Report_by_WatchPug.pdf
https://github.com/gelatodigital/gelato-network/blob/master/docs/audits/G0Group-Gelato2020Jun.pdf
Gelato is used by Keepers

Volatility metrics

Related to Item No. 6 and 8 in the team spreadsheet

Another way to use our historical data from the exporter.
The tentative goals for this item would be the following:

• Calculate realized volatility of the underlying tokens for each vault, for various window sizes (e.g. 1 day, 30 days, 3 months)
• Calculate realized volatility of the share prices for each vault
• Calculate implied volatility of the underlying tokens using additional data sources (e.g. derivatives)

[Risk Score]: Create ConvexTempleDAO-U Group

Create new group:

Network: Ethereum
"id": "convextempledao-u",
"label": "Convex and TempleDAO",
"codeReviewScore": 3,
"testingScore": 4,
"auditScore": 4,
"protocolSafetyScore": 3,
"complexityScore": 2,
"teamKnowledgeScore": 1,
"nameLike": ["convextempledao-u"]
"strategies": ["0x06aee67AC42473E9F0e7DC1A6687A1F26C8136A3"]

Please, add it to the Others group too.

This strategy doesn't use TempleDAO technically, but it uses Convex. So the scores are a mix. Also, it is a partnership between TempleDAO and Yearn Finance.
More info about this, please check internal issue 339

Risk Score: Create stETH accumulator v2

Create new group:
{
"id": "StrategystETHAccumulator_v2",
"network": 1 , // mainnet
"label": "stETH accumulator v2",
"codeReviewScore": 3,
"testingScore": 3,
"auditScore": 5,
"protocolSafetyScore": 2,
"complexityScore": 1,
"teamKnowledgeScore": 2,
"criteria": {
"nameLike": ["StrategystETHAccumulator_v2"],
"strategies": [],
"exclude": ["StrategystETHAccumulator"]
}

Add it into the Others group.

Labels from Etherscan do not match the names in DeFi Safety

should check if there is any protocol that is missing from the current vault/strategy descriptions

e.g. "Curve.fi" (Etherscan label) is not a substring of "Curve Finance" (DeFi Safety)

[Risk Score] - Leverage Maker strat new score

Add new score:

{
"id": "maker-lev",
"network": 1,
"label": "Maker-lev",
"codeReviewScore": 3,
"testingScore": 3,
"auditScore": 5,
"protocolSafetyScore": 1,
"complexityScore": 5,
"teamKnowledgeScore": 4,
"criteria": {
"nameLike": ["maker-lev"],
"strategies": [],
"exclude": []
}
},

Update maker score here to exclude the string "maker-lev" also [here](https://github.com/yearn/yearn-data-analytics/blob/4456405915c2ec2aa286c41679f6431f1c07065a/src/risk_framework/risks.json#L47) 

[Risk Score]: Create Goldfinch Group (Ethereum)

Create new group:

"id": "strategygoldfinch",
"label": "Goldfinch",
"codeReviewScore": 3,
"testingScore": 4,
"auditScore": 4,
"protocolSafetyScore": 4,
"complexityScore": 3,
"teamKnowledgeScore": 3
"nameLike": ["strategygoldfinch"]

Please, add it to the Others group too.

Refactor the main loop for the database crawler

https://github.com/yearn/yearn-data-analytics/blob/5f4bfa0cd6a74e30fec767090a0720c7ebcdb95e/src/database/main.py#L24-L30

we could split the functionalities as separate functions above the main entrypoint, and use them to describe the loop more clearly

Collect the associated protocols from the yearn metadata storage

Collect the associated protocols also from the yearn metadata storage alongside what we currently get from Etherscan, to make sure that we are not missing any protocols

We could get the data from the following endpoint
https://meta.yearn.network/strategies/${chainID}/all

Get USDC values of vault shares

• Calculate the share price in USDC for shares' withdrawals and deposits of all vaults.
• Store the calculated share price in the database.
• Share price should be calculated for all transfers (withdrawals/deposits) of a vault at a particular block.
• Use ypricemagic to get the prices.

Originally posted by @kyhoon in #36 (comment)