Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ape store spellbook #6850

Merged
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
6d24c81
feat: add ape store trades spellbook query
whalehunting Jun 28, 2024
7e3addc
chore: add spellbook schema
whalehunting Jun 28, 2024
71f7925
chore: rerun CI
whalehunting Jun 30, 2024
2969b0f
Revert "chore: rerun CI"
whalehunting Jun 30, 2024
3f68918
fix: convert source to double quotes
whalehunting Jun 30, 2024
d8669d1
fix: add missing sources file
whalehunting Jun 30, 2024
d33b21b
fix: add missing source
whalehunting Jun 30, 2024
2a80579
fix: change ref tag
whalehunting Jul 1, 2024
59963ed
fix: change to source tag
whalehunting Jul 1, 2024
d5cd092
fix: use correct colume name for evt_index
whalehunting Jul 1, 2024
7cdc3a9
fix: remove duplicates
whalehunting Jul 2, 2024
ff4d863
fix: remove duplicates from dex trades
whalehunting Jul 2, 2024
8b09856
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Jul 2, 2024
9f57c06
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Jul 3, 2024
dcb28c9
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Jul 4, 2024
e751b8b
chore: add expose post_hook
whalehunting Jul 5, 2024
738bda4
Merge branch 'main' into feat-add-ape-store-spellbook
Hosuke Jul 5, 2024
067a3fe
Merge branch 'main' into feat-add-ape-store-spellbook
Hosuke Jul 10, 2024
893ef5b
Fix config block
Hosuke Jul 10, 2024
80e3edb
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Jul 15, 2024
ca1db33
fix: remove order by
whalehunting Jul 15, 2024
afd828f
fix: filter on block_month
whalehunting Jul 15, 2024
c2976f8
fix: do not expose spellbook in data explorer
whalehunting Jul 15, 2024
f4578d6
chore: add block_month to uniqueness tests
whalehunting Jul 15, 2024
24799ec
perf: read from dex.trades before joining deployments CTE
whalehunting Jul 15, 2024
fbd9aad
fix: add incremental filter to Router_evt_swap table
whalehunting Jul 15, 2024
2d9daef
fix: add missing endif
whalehunting Jul 15, 2024
ccd4be1
perf: filter dex.trades incrementally
whalehunting Jul 15, 2024
44f5578
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Jul 15, 2024
6cd0993
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Jul 17, 2024
d0c1f91
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Jul 24, 2024
73a780b
Merge remote-tracking branch 'upstream/main' into feat-add-ape-store-…
whalehunting Sep 28, 2024
7a56687
chore: move model files into new directory structure
whalehunting Sep 28, 2024
09b3358
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Sep 28, 2024
ee9c6c4
perf: add incremental filter to prices.usd.minute
whalehunting Sep 28, 2024
c1cb814
feat: add ape store ethereum trades spellbook
whalehunting Sep 28, 2024
0579b8f
feat: add unified ape store trades spellbook across all chains
whalehunting Sep 28, 2024
17924b8
fix: fix typo
whalehunting Sep 28, 2024
2b23e0a
chore: add ape_store_base deployments spellbook
whalehunting Sep 28, 2024
027da2f
fix: remove duplicated schema column annotations
whalehunting Sep 28, 2024
8654a1e
fix: fix schema
whalehunting Sep 28, 2024
aff5596
fix: fix schema
whalehunting Sep 28, 2024
8d3503b
perf: use upstream deployments spellbook in base trades
whalehunting Sep 28, 2024
e3ba7b2
fix: remove non-existent column in ethereum deployments
whalehunting Sep 28, 2024
a13003c
perf: add incremental filter to base deployments spellbook
whalehunting Sep 28, 2024
87bce5c
fix: add missing project_start_date in base deployments
whalehunting Sep 28, 2024
23b6129
perf: add ape_store_ethereum_deployments spellbook
whalehunting Sep 28, 2024
36035f8
fix: use correct column for incremental filter
whalehunting Sep 28, 2024
5143000
fix: use correct syntax for source table
whalehunting Sep 28, 2024
e8dfef2
chore: move start date to the future to make CI run successful
whalehunting Oct 3, 2024
e2344e4
perf: use contains() syntax instead of or condition in join
whalehunting Oct 4, 2024
fed9035
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Oct 4, 2024
463ddfd
Update ape_store_ethereum_trades.sql
whalehunting Oct 7, 2024
226864d
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Oct 7, 2024
cf96e47
Update ape_store_base_deployments.sql
whalehunting Oct 8, 2024
43ef01f
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Oct 8, 2024
6bb0ef5
Merge remote-tracking branch 'upstream/main' into feat-add-ape-store-…
whalehunting Oct 15, 2024
1e97f49
perf: query dex.trades twice for each side + union all
whalehunting Oct 15, 2024
0da4af3
reset start date for base deployments
jeff-dude Oct 15, 2024
259b963
reset start date for base trades
jeff-dude Oct 15, 2024
bb861ad
Merge branch 'main' into feat-add-ape-store-spellbook
whalehunting Oct 17, 2024
fd6f3e2
fix: add more columsn to uniqueness test
whalehunting Oct 17, 2024
6c49512
fix: add more columsn to uniqueness test
whalehunting Oct 17, 2024
d4a13d5
fix: use union instead of union all to filter out duplicates
whalehunting Oct 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
version: 2

models:
- name: ape_store_trades
meta:
blockchain: base, ethereum
project: ape_store
contributors: whale_hunter
config:
tags: ["base", "ethereum", "ape_store", "dex", "whale_hunter"]
description: >
Trades for tokens that get deployed through ApeStore across all chains
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- block_month
- blockchain
- tx_hash
- tx_index
columns:
- &block_time
name: block_time
description: "UTC timestamp of the trade"
- &block_date
name: block_date
description: "UTC date of the trade"
- &block_month
name: block_month
description: "UTC month of the trade"
- &blockchain
name: blockchain
description: "Blockchain on which the trade occured"
- &platform
name: platform
description: "Platform on which the trade occured. e.g. Bonding Curve or DEX"
- &type
name: type
description: "Wether the trade is a buy or a sell"
- &amount_usd
name: amount_usd
description: "USD value of the trade"
- &token_bought_amount
name: token_bought_amount
description: "Value of the token bought at time of execution in the original currency"
- &token_bought_symbol
name: token_bought_symbol
description: "Token symbol for token bought in the trade"
- &token_bought_address
name: token_bought_address
description: "Contract address of the token bought"
- &token_sold_amount
name: token_sold_amount
description: "Value of the token sold at time of execution in the original currency"
- &token_sold_symbol
name: token_sold_symbol
description: "Token symbol for token sold in the trade"
- &token_sold_address
name: token_sold_address
description: "Contract address of the token sold"
- &user
name: user
description: "Address which initiated the trade"
- &tx_hash
name: tx_hash
description: "Unique transaction hash value tied to each transaction on the DEX"
- &tx_index
name: tx_index
description: "Index of the corresponding trade event"
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{{
config(
schema = 'ape_store',
alias = 'trades',
materialized = 'view',
post_hook = '{{ expose_spells(
blockchains = \'["base", "ethereum"]\',
spell_type = "project",
spell_name = "ape_store",
contributors = \'["whale_hunter"]\') }}'
)
}}

{% set ape_store_models = [
ref('ape_store_base_trades')
, ref('ape_store_ethereum_trades')
] %}

{% for model in ape_store_models %}
SELECT block_time,
block_date,
block_month,
blockchain,
platform,
type,
amount_usd,
token_bought_amount,
token_bought_symbol,
token_bought_address,
token_sold_amount,
token_sold_symbol,
token_sold_address,
user,
tx_hash,
tx_index
FROM {{ model }}
{% if not loop.last %}
UNION ALL
{% endif %}
{% endfor %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{{
config(
alias='deployments',
schema='ape_store_base',
materialized='incremental',
file_format='delta',
incremental_strategy='merge',
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')],
unique_key='token'
)
}}

{% set project_start_date = '2024-04-04' %}
{% set blockchain = 'base' %}

select
evt_block_time as block_time,
date_trunc('day', evt_block_time) as block_date,
date_trunc('month', evt_block_time) as block_month,
'{{blockchain}}' as blockchain,
token,
id,
evt_tx_from as deployer,
evt_tx_hash as tx_hash
from {{ source("ape_store_base", "Router_evt_CreateToken") }}
where
{% if is_incremental() %}
{{ incremental_predicate('evt_block_time') }}
{% else %}
evt_block_time >= timestamp '{{project_start_date}}'
{% endif %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
version: 2

models:
- name: ape_store_base_trades
meta:
blockchain: base
project: ape_store
contributors: whale_hunter
config:
tags: ["base", "ape_store", "dex", "whale_hunter"]
description: >
Trades for tokens that get deployed through ApeStore on Base
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- block_month
- blockchain
- tx_hash
- tx_index
columns:
- &block_time
name: block_time
description: "UTC timestamp of the trade"
- &block_date
name: block_date
description: "UTC date of the trade"
- &block_month
name: block_month
description: "UTC month of the trade"
- &blockchain
name: blockchain
description: "Blockchain on which the trade occured"
- &platform
name: platform
description: "Platform on which the trade occured. e.g. Bonding Curve or DEX"
- &type
name: type
description: "Wether the trade is a buy or a sell"
- &amount_usd
name: amount_usd
description: "USD value of the trade"
- &token_bought_amount
name: token_bought_amount
description: "Value of the token bought at time of execution in the original currency"
- &token_bought_symbol
name: token_bought_symbol
description: "Token symbol for token bought in the trade"
- &token_bought_address
name: token_bought_address
description: "Contract address of the token bought"
- &token_sold_amount
name: token_sold_amount
description: "Value of the token sold at time of execution in the original currency"
- &token_sold_symbol
name: token_sold_symbol
description: "Token symbol for token sold in the trade"
- &token_sold_address
name: token_sold_address
description: "Contract address of the token sold"
- &user
name: user
description: "Address which initiated the trade"
- &tx_hash
name: tx_hash
description: "Unique transaction hash value tied to each transaction on the DEX"
- &tx_index
name: tx_index
description: "Index of the corresponding trade event"
- name: ape_store_base_deployments
meta:
blockchain: base
project: ape_store
contributors: whale_hunter
config:
tags: ["base", "ape_store", "dex", "whale_hunter"]
description: >
Tokens that get deployed through ApeStore on Base
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- token
columns:
- *block_time
- *block_date
- *block_month
- *blockchain
- &token
name: token
description: "Contract address of the deployed token"
- &id
name: id
description: "ID of the deployed token"
- &deployer
name: deployer
description: "Address which initiated the deployment"
- *tx_hash
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
{{ config(
alias = 'trades',
schema = 'ape_store_base',
partition_by = ['block_month'],
materialized = 'incremental',
file_format = 'delta',
incremental_strategy = 'merge',
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')],
unique_key = ['block_month', 'blockchain', 'tx_hash', 'tx_index']
)
}}

{% set project_start_date = '2024-04-04' %}
{% set blockchain = 'base' %}
{% set weth_contract_address = '0x4200000000000000000000000000000000000006' %}

with
deployments as (
select * from {{ref('ape_store_base_deployments')}}
),
bondingcurvetrades as (
select
evt_block_time as block_time,
date_trunc('day', evt_block_time) as block_date,
date_trunc('month', evt_block_time) as block_month,
'{{blockchain}}' as blockchain,
'Bonding Curve' as platform,
if(amount0out > 0, 'Buy', 'Sell') as type,
(amount1in + amount1out) / 1e18 * price as amount_usd,
(amount0out + amount1out) / 1e18 as token_bought_amount,
if(amount0out > 0, token.symbol, 'WETH') token_bought_symbol,
if(
amount0out > 0, token, {{ weth_contract_address }}
) as token_bought_address,
(amount0in + amount1in) / 1e18 as token_sold_amount,
if(amount0out > 0, 'WETH', token.symbol) as token_sold_symbol,
if(
amount0out > 0, {{ weth_contract_address }}, token
) as token_sold_address,
sender as user,
evt_tx_hash as tx_hash,
evt_index as tx_index
from {{ source('ape_store_base', 'Router_evt_swap') }} as swaps
left join
{{ source('tokens', 'erc20') }} as token
on (
token.blockchain = '{{blockchain}}'
and token.contract_address = swaps.token
)
left join
{{ source('prices', 'usd') }} as ethPrice
on (
ethPrice.blockchain = '{{blockchain}}'
and ethPrice.contract_address = {{ weth_contract_address }}
and ethPrice.minute = date_trunc('minute', evt_block_time)
and (
{% if is_incremental() %}
{{ incremental_predicate('ethPrice.minute') }}
{% else %}
ethPrice.minute >= timestamp '{{project_start_date}}'
{% endif %}
)
)
where
{% if is_incremental() %}
{{ incremental_predicate('evt_block_time') }}
{% else %}
evt_block_time >= timestamp '{{project_start_date}}'
{% endif %}
),
dextrades as (
select distinct
trades.block_time,
date_trunc('day', trades.block_time) as block_date,
date_trunc('month', trades.block_time) as block_month,
trades.blockchain,
'DEX' as platform,
if(token_sold_address = {{ weth_contract_address }}, 'Buy', 'Sell') as type,
amount_usd,
token_bought_amount,
token_bought_symbol,
token_bought_address,
token_sold_amount,
token_sold_symbol,
token_sold_address,
tx_from as user,
trades.tx_hash,
evt_index as tx_index
from {{ source('dex', 'trades') }} as trades
join
deployments
on (
trades.blockchain = deployments.blockchain
and trades.block_month >= deployments.block_month
and (token_bought_address = token or token_sold_address = token)
whalehunting marked this conversation as resolved.
Show resolved Hide resolved
)
where
trades.blockchain = '{{blockchain}}'
and
{% if is_incremental() %}
{{ incremental_predicate('trades.block_time') }}
{% else %}
trades.block_time >= timestamp '{{project_start_date}}'
{% endif %}
)
select *
from bondingcurvetrades
union all
select *
from dextrades
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{{
config(
alias='deployments',
schema='ape_store_ethereum',
materialized='incremental',
file_format='delta',
incremental_strategy='merge',
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')],
unique_key='token'
)
}}

{% set project_start_date = '2024-09-06' %}
{% set blockchain = 'ethereum' %}

select
evt_block_time as block_time,
date_trunc('day', evt_block_time) as block_date,
date_trunc('month', evt_block_time) as block_month,
'{{blockchain}}' as blockchain,
token,
id,
evt_tx_from as deployer,
evt_tx_hash as tx_hash
from {{ source("ape_store_base", "Router_evt_CreateToken") }}
join {{source(blockchain, 'transactions')}} on (
evt_block_time = block_time
and evt_tx_hash = hash
and (
{% if is_incremental() %}
{{ incremental_predicate('block_time') }}
{% else %}
block_time >= timestamp '{{project_start_date}}'
{% endif %}
)
)
where
{% if is_incremental() %}
{{ incremental_predicate('evt_block_time') }}
{% else %}
evt_block_time >= timestamp '{{project_start_date}}'
{% endif %}
Loading
Loading