Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stablecoin balances evms #6826

Merged
merged 9 commits into from
Oct 3, 2024
Merged

Stablecoin balances evms #6826

merged 9 commits into from
Oct 3, 2024

Conversation

Synthquest
Copy link
Contributor

Thank you for contributing to Spellbook 🪄

Update!

Please build spells in the proper subproject directory. For more information, please see the main readme, which also links to a GH discussion with the option to ask questions.

Contribution type

Please check the type of contribution this pull request is for:

  • New spell(s)
  • Adding to existing spell lineage
  • Bug fix

Note: You can safely discard any section below which doesn't apply based on selection above


For new spell(s)

If you are building new spell(s), please provide the following information:

  • Spell name(s): stablecoins_{chain}.balances
  • Description: materialized table of daily balances of stablecoins
  • Who are the new spell(s) for? Internal team
  • How will the new spell(s) be used downstream? Using solely the daily_balances table hits runtime errors & recreating the logic for daily views of stablecoin balances among wallets hits query complexity/runtime errors.

This query is for anyone interested in network views of stablecoin balances

  • **Implementation details:**The stablecoins are defined in the tokens_{network}.erc20_stablecoins tables

  • Test instructions: unique address/balance/day

  • Related issue(s): [Link to related issues, if any]


For adding to existing spell lineage

If you are adding to an existing spell lineage, please provide the following information:

  • Description: [Description of the changes made]

For bug fixes

If you are fixing a bug, please provide the following information:

  • Description: [Description of the bug fix]
  • Steps to reproduce: [How to reproduce the bug]
  • Implementation details: [Information on how the bug was fixed]
  • Test instructions: [How to test the fix]
  • Related issue(s): [Link to related issues, if any]

Additional information

Please provide any additional information that might help us review your pull request:

  • [Any additional information]

Thank you for your contribution!

@dune-eng
Copy link

Workflow run id 11021984662 approved.

@dune-eng
Copy link

Workflow run id 11021984661 approved.

@Synthquest Synthquest marked this pull request as draft September 24, 2024 21:11
@dune-eng
Copy link

Workflow run id 11022225657 approved.

@dune-eng
Copy link

Workflow run id 11022225565 approved.

@dune-eng
Copy link

Workflow run id 11022303136 approved.

@dune-eng
Copy link

Workflow run id 11022303425 approved.

@dune-eng
Copy link

Workflow run id 11025108461 approved.

@dune-eng
Copy link

Workflow run id 11025108358 approved.

@dune-eng
Copy link

Workflow run id 11025108493 approved.

@dune-eng
Copy link

Workflow run id 11025419582 approved.

@dune-eng
Copy link

Workflow run id 11025419438 approved.

@dune-eng
Copy link

Workflow run id 11025419584 approved.

@dune-eng
Copy link

Workflow run id 11036468370 approved.

@dune-eng
Copy link

Workflow run id 11036468425 approved.

@dune-eng
Copy link

Workflow run id 11036468080 approved.

@Synthquest
Copy link
Contributor Author

@jeff-dude
I'm having trouble finding the macro or model that creates the balances_raw table

https://github.com/duneanalytics/spellbook/blob/main/dbt_subprojects/tokens/macros/balances/balances_daily_agg.sql

https://github.com/duneanalytics/spellbook/blob/main/dbt_macros/shared/balances_incremental_subset_daily.sql

Specifically on optimism - I just want to make sure it accounts for legacy transfers

@Synthquest
Copy link
Contributor Author

Otherwise these should be good to go.

Note for myself: To add more chain in the future we have to create the base daily balances table for the network first

@jeff-dude
Copy link
Member

@jeff-dude I'm having trouble finding the macro or model that creates the balances_raw table

https://github.com/duneanalytics/spellbook/blob/main/dbt_subprojects/tokens/macros/balances/balances_daily_agg.sql

https://github.com/duneanalytics/spellbook/blob/main/dbt_macros/shared/balances_incremental_subset_daily.sql

Specifically on optimism - I just want to make sure it accounts for legacy transfers

raw balances is done outside of spellbook due to compute requirements (v heavy). they are based on transfers spells though, which can be found here:
https://github.com/duneanalytics/spellbook/tree/main/dbt_subprojects/tokens/models/transfers_and_balances/optimism

base_transfers for raw data
transfers for enriched with metadata

you can help us validate the optimism transfers there 🙏

@jeff-dude jeff-dude added the WIP work in progress label Sep 25, 2024
@Synthquest
Copy link
Contributor Author

@jeff-dude I'm having trouble finding the macro or model that creates the balances_raw table
https://github.com/duneanalytics/spellbook/blob/main/dbt_subprojects/tokens/macros/balances/balances_daily_agg.sql
https://github.com/duneanalytics/spellbook/blob/main/dbt_macros/shared/balances_incremental_subset_daily.sql
Specifically on optimism - I just want to make sure it accounts for legacy transfers

raw balances is done outside of spellbook due to compute requirements (v heavy). they are based on transfers spells though, which can be found here: https://github.com/duneanalytics/spellbook/tree/main/dbt_subprojects/tokens/models/transfers_and_balances/optimism

base_transfers for raw data transfers for enriched with metadata

you can help us validate the optimism transfers there 🙏

thanks for this!
It looks like it's missing the legacy chain transfers. I'm not sure how I can contribute here with it being off the spellbook but two approaches i can think of is

  1. to initialize the current query with a "snapshot" of the address/token/balances taken at the last block of the legacy version
  2. include the legacy tables in the model
select  
      'optimism' as blockchain
    , log.block_time
    , log.block_date
    , coalesce(bytearray_to_uint256(bytearray_substring(log.data,1,32)),0) as value
    , bytearray_substring(topic1,1+12,20) as "from"
    , bytearray_substring(topic2,1+12,20) as "to"
    , log.contract_address
    , log.tx_hash
    from optimism_legacy_ovm1.logs log

@jeff-dude
Copy link
Member

I'm not sure how I can contribute here with it being off the spellbook but two approaches i can think of is

  1. to initialize the current query with a "snapshot" of the address/token/balances taken at the last block of the legacy version
  2. include the legacy tables in the model

i think best you can do is look to modify the transfers pipeline for optimism, then balances will pick it up. with this being only legacy, i would have to check with the team on how we'd handle the history for balances, may require a new backfill

@dune-eng
Copy link

Workflow run id 11097032228 approved.

@dune-eng
Copy link

Workflow run id 11097032400 approved.

@dune-eng
Copy link

Workflow run id 11097032385 approved.

@Synthquest
Copy link
Contributor Author

I'm not sure how I can contribute here with it being off the spellbook but two approaches i can think of is

  1. to initialize the current query with a "snapshot" of the address/token/balances taken at the last block of the legacy version
  2. include the legacy tables in the model

i think best you can do is look to modify the transfers pipeline for optimism, then balances will pick it up. with this being only legacy, i would have to check with the team on how we'd handle the history for balances, may require a new backfill

@jeff-dude I removed OP for now. I'll look to handle that it in a separate PR

@dune-eng
Copy link

Workflow run id 11107294711 approved.

@dune-eng
Copy link

Workflow run id 11107294478 approved.

@dune-eng
Copy link

Workflow run id 11107294721 approved.

@Synthquest Synthquest marked this pull request as ready for review September 30, 2024 19:23
@dune-eng
Copy link

Workflow run id 11114862885 approved.

@dune-eng
Copy link

Workflow run id 11114863081 approved.

@dune-eng
Copy link

Workflow run id 11114863097 approved.

@Synthquest
Copy link
Contributor Author

@jeff-dude
Pinging

#6826 (comment)

@jeff-dude
Copy link
Member

@jeff-dude Pinging

#6826 (comment)

okay, revisiting OP after works for me.

do you think this PR is in merge-ready state?
i notice we had a short timeframe run a few commits ago, but haven't seen it run again since. i guess the commits are mostly just metadata fixes?

@jeff-dude jeff-dude self-assigned this Oct 2, 2024
@jeff-dude jeff-dude added question Further information is requested in review Assignee is currently reviewing the PR and removed WIP work in progress labels Oct 2, 2024
@Synthquest
Copy link
Contributor Author

@jeff-dude Pinging
#6826 (comment)

okay, revisiting OP after works for me.

do you think this PR is in merge-ready state? i notice we had a short timeframe run a few commits ago, but haven't seen it run again since. i guess the commits are mostly just metadata fixes?

Yea the changes since then are just to the tokens_erc20_stablecoin tables. I ran through similar checks on the balance tables and picked up some issues with them.
These balance tables should be good to go though🤞

Copy link
Member

@jeff-dude jeff-dude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, let's give this a try.

with this macro gaining more traction (seeing it used in multiple spots), i will catch up with @0xRobin next week on a full plan for these spells

@jeff-dude jeff-dude removed question Further information is requested in review Assignee is currently reviewing the PR labels Oct 3, 2024
@jeff-dude jeff-dude merged commit a02c461 into duneanalytics:main Oct 3, 2024
2 of 3 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants