From a3accd8d385e9caf538b366b38dbc6730cbf2720 Mon Sep 17 00:00:00 2001 From: Giuliano Francescangeli Date: Tue, 17 Sep 2024 03:17:28 +0800 Subject: [PATCH] Made changes to the how-to guides and references sections --- docs/SUMMARY.md | 10 +-- .../your-first-application/evm.md | 61 ++++++++++++++++-- .../intro-your-first-application.md | 29 --------- .../your-first-application/solana.md | 64 ++++++++++++++++--- docs/new/references/devcontainer-ref.md | 27 ++++++++ 5 files changed, 141 insertions(+), 50 deletions(-) delete mode 100644 docs/new/how-to-guides/your-first-application/intro-your-first-application.md create mode 100644 docs/new/references/devcontainer-ref.md diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index d0a57012..c672ab64 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -4,11 +4,10 @@ ## Documentation -* [How-To-Guides](new/getting-started/intro-how-to-guides.md) - * [Your First Substreams Application](new/how-to-guides/your-first-application/intro-your-first-application.md) - * [EVM](new/how-to-guides/your-first-application/evm.md) - * [Solana](new/how-to-guides/your-first-application/solana.md) - * [Injective](new/how-to-guides/your-first-application/injective.md) +* [How-To Guides](new/getting-started/intro-how-to-guides.md) + * [Initialize an EVM Substreams Project](new/how-to-guides/your-first-application/evm.md) + * [Initialize an Solana Substreams Project](new/how-to-guides/your-first-application/solana.md) + * [Initialize an Injective Substreams Project](new/how-to-guides/your-first-application/injective.md) * [Consume Substreams](new/consume/consume.md) * [Install the CLI](new/common/installing-the-cli.md) * [Authentication](new/common/authentication.md) @@ -88,6 +87,7 @@ ## Reference & Specs * [Chains and endpoints](new/references/chains-and-endpoints.md) +* [Development Container Reference](new/references/devcontainer-ref.md) * [Substreams CLI reference](new/references/command-line-interface.md) * [Manifests Reference](new/references/manifests.md) * [GUI Reference](new/references/gui.md) diff --git a/docs/new/how-to-guides/your-first-application/evm.md b/docs/new/how-to-guides/your-first-application/evm.md index 641167af..e841b696 100644 --- a/docs/new/how-to-guides/your-first-application/evm.md +++ b/docs/new/how-to-guides/your-first-application/evm.md @@ -1,12 +1,59 @@ -Check out the [Getting Started Guide](./intro-your-first-application.md) for more information on how to initialize your project. There are two options within `substreams init` to initialize your EVM Substreams: +In this guide, you'll learn how to initialize an EVM-based Substreams project. You’ll learn how to set up a simple project to extract raw data or filter EVM events and calls based on a smart-contract of interest. -- `evm-minimal`: creates a simple Substreams that extracts raw data from the block (generates Rust code). -- `evm-events-calls`: creates a Substreams that extracts EVM events and calls filtered by one or several smart contract addresses. +## Prerequisites -## EVM Foundational Modules +- Docker and VS Code installed and up-to-date. +- Visit the [Getting Started Guide](https://github.com/streamingfast/substreams-starter) to initialize your development environment. -The `evm-events-calls` codegen path relies on one of the [EVM Foundational Modules](https://github.com/streamingfast/substreams-foundational-modules/tree/develop/ethereum-common). A Foundational Module extracts the most relevant data from blockchain, so that you don't have to code it yourself. +## Step 1: Initialize Your EVM Substreams Project -Specifically, the `evm-events-calls` path uses the [filtered_events_and_calls](https://github.com/streamingfast/substreams-foundational-modules/blob/develop/ethereum-common/substreams.yaml#L142) module from the EVM Foundational Modules to retrieve all the events filtered by specific smart contract addresses. +1. Open your development environment and run the following command to initialize your project: + + ```bash + substreams init + ``` + +2. You will be given the option to choose between two EVM project options. Select the one that best fits your requirements: + - **evm-minimal**: Creates a simple Substreams that extracts raw data from the block and generates Rust code. + - **evm-events-calls**: Creates a Substreams that extracts EVM events and calls using the cached [EVM Foundational Module](https://substreams.dev/streamingfast/ethereum-common/v0.3.0), filtered by one or more smart contract addresses. + + +## Step 2: Visualize the Data + +1. Create your account [here](https://thegraph.market/) to generate an authentification token (JWT) and pass it as input to: + + ```bash + substreams auth + ``` + +2. Run the following command to visualize and itterate on your filtered data model: + + ```bash + Substreams Gui + ```` + +## Step 3: Customize your Project: + +After initialization, you can: + +- Modify your Substreams manifest to include additional filters or configurations. +- Implement custom processing logic in Rust based on the filtered data retrieved by the foundational module. + +For a deeper dive into use cases and details, refer to the [EVM Tutorials](../../tutorials/evm). + +## Additional Resources + +You may find these additional resources helpful for developing your first EVM application. + +### Development Container Reference + +The [development container reference](../../references/devcontainer-ref) helps you navigate the complete container and its common errors. + +### Gui Reference + +The [gui reference](../../references/gui) lets you explore the complete tool of the Pyth contract. + +### Manifests Reference + +The [manifests reference](../../references/manifests.md) helps you with editing the `substreams.yaml`. -
diff --git a/docs/new/how-to-guides/your-first-application/intro-your-first-application.md b/docs/new/how-to-guides/your-first-application/intro-your-first-application.md deleted file mode 100644 index ad587022..00000000 --- a/docs/new/how-to-guides/your-first-application/intro-your-first-application.md +++ /dev/null @@ -1,29 +0,0 @@ -Quickly scaffold your first project inside the [Substreams Development Environment](https://github.com/streamingfast/substreams-starter?tab=readme-ov-file) (”devcontainer”). Check out the Getting Started in the `README`to run remotely or clone the substreams-starter repository to run locally. Inside the devcontainer the `substreams init` command sets up a code-generated Substreams project, from which you can easily build either a subgraph or an SQL-based solution for handling data. - -{% hint style="info" %} -**Note:** Validate Docker and VS Code are up-to-date. -{% endhint %} - -Consult the relevant ecosystem guide to get started using real-time and historical indexed data: - -- [Solana](./solana.md) -- [EVM](./evm.md) -- [Injective](./injective.md) - -## Navigating the Devcontainer - -When entering the devcontainer, you can either insert your own `substreams.yaml` file and run `substreams build` to generate the associated Protobuf files, or choose from two auto-generated code-paths: - -- **Minimal**: Creates a simple Substreams that extracts raw data from the block. -- **Non-Minimal**: Extracts filtered data specific to the network and relies on the cache and Protobufs provided by the Foundational Modules. - -Complete your Substreams project to be fully queryable, either through a Subgraph or directly from your SQL database, ensuring seamless access to the data for analysis and application use: - -- `substreams codegen subgraph`: The generated project follows the standard subgraph structure. By default, the `schema.graphql` and `mappings.ts` files respectively include only a required input ID and the basic code to create one. It's up to you to decide what entities to create based on the data extracted by Substreams. For technical details on how to configure a Subgraph sink, [click here](https://substreams.streamingfast.io/documentation/consume/subgraph). -- `substreams codegen sql`: For technical details on how to configure a SQL sink, [click here](https://substreams.streamingfast.io/documentation/consume/sql). - -If your plan is to deploy a Subgraph you may choose to either run the `graph-node` locally with the `deploy-local` command or deploy to the Subgraph Studio by checking out the `deploy` command in the `package.json`. - -{% hint style="info" %} -**Note:** When running local, make sure to verify that all containers are running properly in the Docker tab and that there’s no errors in the logs. -{% endhint %} \ No newline at end of file diff --git a/docs/new/how-to-guides/your-first-application/solana.md b/docs/new/how-to-guides/your-first-application/solana.md index 359065e4..6c833c27 100644 --- a/docs/new/how-to-guides/your-first-application/solana.md +++ b/docs/new/how-to-guides/your-first-application/solana.md @@ -1,14 +1,60 @@ -Check out the [Getting Started Guide](./intro-your-first-application.md) for more information on how to initialize your project. There are two options within `substreams init` to initialize your Solana Substreams: +In this guide, you'll learn how to initialize an Solana-based Substreams project. You’ll learn how to set up a simple project to extract raw data or filter Solana transactions based on Program IDs and Account IDs. -- `sol-minimal`: Creates a simple Substreams that extracts raw data from the block. -- `sol-transactions`: Creates a Substreams that extracts Solana transactions filtered by one or more Program IDs and Account IDs. +## Prerequisites -{% hint style="info" %} -**Note**: The block model in your your generated project will not include Solana voting transactions. Excluding voting transactions reduces the costs and size of processing a Solana full-block by 75%. You can still access voting transactions by consuming a Solana full-block. -{% endhint %} +- Docker and VS Code installed and up-to-date. +- Visit the [Getting Started Guide](https://github.com/streamingfast/substreams-starter) to initialize your development environment. -**Solana Foundational Modules** +## Step 1: Initialize Your Solana Substreams Project -The `sol-transactions` codegen path uses [Solana Foundational Modules](https://substreams.dev/streamingfast/solana-common/v0.3.0) to simplify filtering. These modules are designed to extract critical blockchain data, sparing you the need to write custom code. Specifically, the [filtered_transactions_without_votes](https://github.com/streamingfast/substreams-foundational-modules/blob/develop/solana-common/substreams-v0.2.0.yaml#L49) module extracts key data and accepts a regular expression to filter transactions, saving you from writing custom code. +1. Open your development environment and run the following command to initialize your project: + + ```bash + substreams init + ``` + +2. You will be given the option to choose between two Solana project options. Select the one that best fits your requirements: + - **`sol-minimal`**: This option creates a simple Substreams project that extracts raw data directly from Solana blocks. + - **`sol-transactions`**: This option creates a Substreams project that filters Solana transactions based on one or more Program IDs and/or Account IDs, using the cached [Solana Foundational Module](https://substreams.dev/streamingfast/solana-common/v0.3.0). + + Note: The filtered_transactions_without_votes module extracts transactions while excluding voting transactions, reducing data size and costs by 75%. To access voting transactions, use a full Solana block. + +## Step 2: Visualize the Data + +1. Create your account [here](https://thegraph.market/) to generate an authentification token (JWT) and pass it as input to: + + ```bash + substreams auth + ``` + +2. Run the following command to visualize and itterate on your filtered data model: + + ```bash + Substreams Gui + ```` + +## Step 3: Customize your Project: + +After initialization, you can: + +- Modify your Substreams manifest to include additional filters or configurations. +- Implement custom processing logic in Rust based on the filtered data retrieved by the foundational module. + +For a deeper dive into use cases and details, refer to the [Solana Tutorials](../../tutorials/solana). + +## Additional Resources + +You may find these additional resources helpful for developing your first Solana application. + +### Development Container Reference + +The [development container reference](../../references/devcontainer-ref) helps you navigate the complete container and its common errors. + +### Gui Reference + +The [gui reference](../../references/gui) lets you explore the complete tool of the Pyth contract. + +### Manifests Reference + +The [manifests reference](../../references/manifests.md) helps you with editing the `substreams.yaml`. -
\ No newline at end of file diff --git a/docs/new/references/devcontainer-ref.md b/docs/new/references/devcontainer-ref.md new file mode 100644 index 00000000..8a7c0628 --- /dev/null +++ b/docs/new/references/devcontainer-ref.md @@ -0,0 +1,27 @@ +The Substreams Development Container (“devcontainer”) is a tool to help you build your first project. You can either run it remotely or clone the [substreams-starter repository](https://github.com/streamingfast/substreams-starter?tab=readme-ov-file) to run it locally. Inside the devcontainer, the `substreams init` command sets up a code-generated Substreams project, allowing you to easily build a subgraph or an SQL-based solution for data handling. + +**Prerequisites:** + +- Ensure Docker and VS Code are up-to-date. + +### Navigating the Devcontainer + +Upon entering the devcontainer, you can either insert your `substreams.yaml` file and run `substreams build` to generate Protobuf files or choose one of the auto-generated paths: + +- **Minimal**: Extracts raw data from the block. +- **Non-Minimal**: Extracts filtered data using network-specific cache and Protobufs from the Foundational Modules. + +### Building Your Project + +You can configure your Substreams project for querying either through a Subgraph or directly from your SQL database: + +- **Subgraph**: Run `substreams codegen subgraph`. This generates a project with a basic `schema.graphql` and `mappings.ts` file. You can customize these to define entities based on the data extracted by Substreams. For more information on configuring a Subgraph sink, see the [Subgraph documentation](https://substreams.streamingfast.io/documentation/consume/subgraph). +- **SQL**: Run `substreams codegen sql` for SQL-based queries. For more information on configuring a SQL sink, refer to the [SQL documentation](https://substreams.streamingfast.io/documentation/consume/sql). + +### Deployment Options + +To deploy a Subgraph, you can either run the `graph-node` locally using the `deploy-local` command or deploy to Subgraph Studio by using the `deploy` command from the `package.json` file. + +### Common Errors + +- When running locally, make sure to verify that all Docker containers are healthy by running the `dev-status` command.