diff --git a/apps/docs/docs/how-oso-works/impact-metrics/index.mdx b/apps/docs/docs/how-oso-works/impact-metrics/index.mdx index ea8142b00..2594ee7fe 100644 --- a/apps/docs/docs/how-oso-works/impact-metrics/index.mdx +++ b/apps/docs/docs/how-oso-works/impact-metrics/index.mdx @@ -11,6 +11,18 @@ import TabItem from '@theme/TabItem'; An **impact metric** is a quantitative measure of impact over a discrete period of time. Impact metrics are most commonly queried by project (eg, `uniswap`), although they can also be queried by individual artifact or at the collection level. ::: +## Principles + +--- + +Impact metrics should be designed with the following principles in mind: + +- Verifiability: Metrics should be based on public data that can be independently verified. They should not rely on proprietary data sources or private APIs. +- Reproducibility: Metrics should be easy to reproduce, simulate, and audit to ensure they are achieving the intended results. They should not have a "black box" element that makes them difficult to understand or replicate. +- Consistency: Metrics should be consistent across projects and artifacts. They should be calculated using the same methodology and data sources to ensure that they are comparable. +- Completeness: Metrics should be comprehensive and cover all projects and artifacts in the OSO database that fulfill basic requirements. They should not be highly sector-specific. +- Simplicity: Metrics should have business logic that is easy to understand. They should not require a deep understanding of the underlying data or complex statistical methods to interpret. + ## Requirements --- @@ -39,6 +51,8 @@ Some may also include a `namespace` field to be explicit about the source of the --- +The following are examples of impact metrics that can be queried from the OSO database. These examples are illustrative and do not represent an exhaustive list of all possible impact metrics. They make use of two "intermediate" tables that aggregate event data into timeseries buckets: `int_events_monthly_to_project` and `int_events_daily_to_project`. These tables are derived from the consolidated `int_events` table. + ### Forks in the Last 6 Months (Project Level) The following is an example of a valid impact metric, expressed in SQL: @@ -46,184 +60,143 @@ The following is an example of a valid impact metric, expressed in SQL: ```sql - -- Sum of fork events to a project in the last 6 months - SELECT + select project_id, - 'FORKED_6M' AS impact_metric, - SUM(amount) AS amount - FROM `oso.events_monthly_to_project` - WHERE + 'fork_count_6_months' as impact_metric, + sum(amount) as amount + from `oso.int_events_monthly_to_project` + where event_type = 'FORKED' - AND DATE(bucket_month) >= DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH) - GROUP BY project_id + and DATE(bucket_month) >= DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH) + group by project_id ``` ```json [{ "project_id": "jUda1pi-FdNlaUmgKq51B4h8x4wX3QTN2fZkKq6N0vw\u003d", - "impact_metric": "FORKED_6M", + "impact_metric": "'fork_count_6_months", "amount": "125.0" }, { "project_id": "wdBmT3yweChtV4g4qcc1NN1QtfnmMPAuCVxoPkiAowY\u003d", - "impact_metric": "FORKED_6M", + "impact_metric": "'fork_count_6_months", "amount": "57.0" }, { "project_id": "HNv9-2g63oZCrLD2jDAIygVq7vQkn2iV2QGmEfcFsYk\u003d", - "impact_metric": "FORKED_6M", + "impact_metric": "'fork_count_6_months", "amount": "13.0" }] ``` -### Forks in the Last 6 Months (Artifact Level) +### Gas Fees -Here is the same metric calculated at the **artifact** level: +Here's an example of an impact metric that calculates the sum of gas fees contributed by a project across all networks: ```sql - -- Sum of fork events to an artifact in the last 6 months - SELECT - artifact_id, - 'FORKED_6M' AS impact_metric, - SUM(amount) AS amount - FROM `oso.events_monthly_to_artifact` - WHERE - event_type = 'FORKED' - AND DATE(bucket_month) >= DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH) - GROUP BY artifact_id - ``` + select + project_id, + 'gas_fees' as metric, + SUM(amount / 1e18) as amount + from `oso.int_events_monthly_to_project` + where + event_type = 'CONTRACT_INVOCATION_DAILY_L2_GAS_USED' + group by project_id + ``` ```json [{ - "artifact_id": "vg-apoPrbCtAM4enk0Ar4tODrqTBi9ZDnNnTw-udjw4\u003d", - "impact_metric": "FORKED_6M", - "amount": "1.0" + "project_id": "vg-apoPrbCtAM4enk0Ar4tODrqTBi9ZDnNnTw-udjw4\u003d", + "impact_metric": "gas_fees", + "amount": "0.04893" }, { - "artifact_id": "7ZqXfQTqiCHHcRw-THEsTgz8W1T0apGyLe7rE8n0NCs\u003d", - "impact_metric": "FORKED_6M", - "amount": "5.0" + "projectt_id": "7ZqXfQTqiCHHcRw-THEsTgz8W1T0apGyLe7rE8n0NCs\u003d", + "impact_metric": "gas_fees", + "amount": "5.03892" }, { - "artifact_id": "PM9tS7Fp_-LmYT8B-dx8FFBz7xa8rEwm9RFAZ6JehcI\u003d", - "impact_metric": "FORKED_6M", - "amount": "1.0" + "project_id": "PM9tS7Fp_-LmYT8B-dx8FFBz7xa8rEwm9RFAZ6JehcI\u003d", + "impact_metric": "gas_fees", + "amount": "0.01178" }] ``` -### Days from First Commit to First Onchain Transaction +### Daily Active Addresses -Here's a more complex impact metric that uses a CTE to calculate the number of days from the first commit to a project to its first onchain transaction. Note that this query returns null for projects that have not yet had an onchain transaction. It also returns a negative value for projects where the first onchain transaction occurred before the first commit to a public repo. +Here's a more complex impact metric that uses a CTE to calculate the number of daily active addresses for a project: ```sql - -- Number of days from the first commit to a project to its first onchain transaction - WITH first_events AS ( - SELECT + with txns as ( + select project_id, - from_namespace AS namespace, - MIN(CASE WHEN event_type = 'COMMIT_CODE' THEN bucket_day END) AS first_commit, - MIN(CASE WHEN event_type = 'CONTRACT_INVOCATION_DAILY_COUNT' THEN bucket_day END) AS first_txn - FROM `oso.events_daily_to_project_by_source` - GROUP BY project_id + from_artifact_name, + bucket_day + from `oso.int_events_daily_to_project` + where + event_type = 'CONTRACT_INVOCATION_SUCCESS_DAILY_COUNT' + and bucket_day >= '2023-10-01' + ), + + daas as ( + select + project_id, + bucket_day, + COUNT(distinct from_artifact_name) as active_addresses + from txns + group by + project_id, + bucket_day + ), + + total_days as ( + select DATE_DIFF(max_day, min_day, day) + 1 as days + from ( + select + MIN(bucket_day) as min_day, + MAX(bucket_day) as max_day + from txns + ) ) - SELECT + + select project_id, - namespace, - 'DAYS_FROM_FIRST_COMMIT_TO_FIRST_TXN' AS impact_metric, - DATE_DIFF(first_txn, first_commit, DAY) AS days_from_first_commit_to_first_txn - FROM first_events + 'daily_active_addresses' as metric, + SUM(active_addresses) / (select days from total_days) as amount + from daas + group by + project_id ``` ```json [{ "project_id": "-A5N7DYgI4bDZZQaF_4SPw_qx6cv1BbfcoxJ7rZxig8\u003d", - "impact_metric": "DAYS_FROM_FIRST_COMMIT_TO_FIRST_TXN", - "namespace": "OPTIMISM", - "amount": "554" + "impact_metric": "daily_active_addresses", + "amount": "554.5" }, { "project_id": "-KeRKHB_H0HcNkR2_SLUC1vLy46YWoaZSMrjpiW3d5s\u003d", - "impact_metric": "DAYS_FROM_FIRST_COMMIT_TO_FIRST_TXN", - "namespace": "ARBITRUM", - "amount": "224" + "impact_metric": "daily_active_addresses", + "amount": "224.2" }, { "project_id": "-ceY1smj0ZhYH-JSMeK3opMBmLgc29Oe-m5cOd-KnjY\u003d", - "impact_metric": "DAYS_FROM_FIRST_COMMIT_TO_FIRST_TXN", - "namespace": "OPTIMISM", - "amount": "112" + "impact_metric": "daily_active_addresses", + "amount": "112.9" }, { "project_id": "-ceY1smj0ZhYH-JSMeK3opMBmLgc29Oe-m5cOd-KnjY\u003d", - "impact_metric": "DAYS_FROM_FIRST_COMMIT_TO_FIRST_TXN", - "namespace": "ARBITRUM", + "impact_metric": "daily_active_addresses", "amount": null }] ``` -### Issues Closed to Full-Time Contributors in the Last 90 Days - -Finally, here's an example of a metric that utilizes one of the `users` mart models to calculate the ratio of issues closed to full-time contributors to a project in the last 90 days: - - - - ```sql - -- Ratio of issues closed to full-time contributors to a project in the last 90 days - WITH full_time_contributors AS ( - SELECT - project_id, - SUM(amount) AS full_time_contributors - FROM `oso.users_monthly_to_project` - WHERE - user_segment_type = 'FULL_TIME_DEV' - AND DATE(bucket_month) >= DATE_ADD(CURRENT_DATE(), INTERVAL -90 DAY) - GROUP BY project_id - ) - , issues_closed AS ( - SELECT - project_id, - SUM(amount) AS issues_closed - FROM `oso.events_monthly_to_project` - WHERE - event_type = 'ISSUE_CLOSED' - AND DATE(bucket_month) >= DATE_ADD(CURRENT_DATE(), INTERVAL -90 DAY) - GROUP BY project_id - ) - SELECT - ftc.project_id, - ic.issues_closed / ftc.full_time_contributors AS issues_closed_to_full_time_contributors_90_days - FROM full_time_contributors ftc - LEFT JOIN issues_closed ic ON ftc.project_id = ic.project_id - ``` - - - ```json - [{ - "project_id": "--IMjbfK8bKS7MWr-7g9HHKcKRI6BMVb6FX0oIP2QXc\u003d", - "issues_closed_to_full_time_contributors_90_days": "12.6" - }, { - "project_id": "-A5N7DYgI4bDZZQaF_4SPw_qx6cv1BbfcoxJ7rZxig8\u003d", - "issues_closed_to_full_time_contributors_90_days": "0.6" - }, { - "project_id": "-GaxcdxeEp_4kF03wAjs-aHrRoiGwl0BrK0cYYEU0o0\u003d", - "issues_closed_to_full_time_contributors_90_days": null - }, { - "project_id": "-Ne1fzPhJNWO4DD07pTXuFqfLwl3A_Zyq397l5W73s4\u003d", - "issues_closed_to_full_time_contributors_90_days": "3.0" - }, { - "project_id": "-XW3PLKpOAQWReyRlL30Ho_aYRYzHm3kVAG_5JAEtOQ\u003d", - "issues_closed_to_full_time_contributors_90_days": "1.0" - }] - ``` - - - --- To contribute new metrics, please see our guide [here](../../contribute/impact-models).