Skip to content

Commit

Permalink
Streamingfast/docs (#765)
Browse files Browse the repository at this point in the history
* StreamingFast docs on Substreams-powered subgraphs

* added a title

* Add SpS documentation to the menu

* Fix format

* Addressing feedback

* editing the _meta.js

* fix

* trying to fix prettier

* fixing prettier

* Fix title

* `sps-intro` => `introduction`

* Duplicate new pages in every language

---------

Co-authored-by: Enol Álvarez <[email protected]>
Co-authored-by: benface <[email protected]>
  • Loading branch information
3 people authored Sep 16, 2024
1 parent d3d4099 commit 5fffe03
Show file tree
Hide file tree
Showing 100 changed files with 4,858 additions and 39 deletions.
5 changes: 5 additions & 0 deletions website/pages/ar/sps/_meta.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import meta from '../../en/sps/_meta.js'

export default {
...meta,
}
19 changes: 19 additions & 0 deletions website/pages/ar/sps/introduction.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: Introduction to Substreams-powered Subgraphs
---

By using a Substreams package (`.spkg`) as a data source, your subgraph gains access to a stream of pre-indexed blockchain data. This enables more efficient and scalable data handling, especially with large or complex blockchain networks.

There are two methods of enabling this technology:

Using Substreams [triggers](./triggers): Consume from any Substreams module by importing the Protobuf model through a subgraph handler and move all your logic into a subgraph. This method creates the subgraph entities directly in the subgraph.

Using [Entity Changes](https://substreams.streamingfast.io/documentation/consume/subgraph/graph-out): By writing more of the logic into Substreams, you can consume the module's output directly into graph-node. In graph-node, you can use the Substreams data to create your subgraph entities.

It is really a matter of where you put your logic, in the subgraph or the Substreams. Keep in mind that having more of your logic in Substreams benefits from a parallelized model, whereas triggers will be linearly consumed in graph-node.

Visit the following links for How-To Guides on using code-generation tooling to build your first end-to-end project quickly:

- [Solana](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/solana)
- [EVM](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/evm)
- [Injective](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/injective)
137 changes: 137 additions & 0 deletions website/pages/ar/sps/triggers-example.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
title: 'Tutorial: Set Up a Substreams-Powered Subgraph on Solana'
---

## Prerequisites

Before starting, make sure to:

- Complete the [Getting Started Guide](https://github.com/streamingfast/substreams-starter) to set up your development environment using a Dev Container.
- Be familiar with The Graph and basic blockchain concepts such as transactions and Protobufs.

## Step 1: Initialize Your Project

1. Open your Dev Container and run the following command to initialize your project:

```bash
substreams init
```

2. Select the "minimal" project option.
3. Replace the contents of the generated `substreams.yaml` file with the following configuration, which filters transactions for the Orca account on the SPL token program ID:

```yaml
specVersion: v0.1.0
package:
name: my_project_sol
version: v0.1.0

imports: # Pass your spkg of interest
solana: https://github.com/streamingfast/substreams-solana-spl-token/raw/master/tokens/solana-spl-token-v0.1.0.spkg

modules:
- name: map_spl_transfers
use: solana:map_block # Select corresponding modules available within your spkg
initialBlock: 260000082

- name: map_transactions_by_programid
use: solana:solana:transactions_by_programid_without_votes

network: solana-mainnet-beta

params: # Modify the param fields to meet your needs
# For program_id: TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA
map_spl_transfers: token_contract:orcaEKTdK7LKz57vaAYr9QeNsVEPfiu6QeMU1kektZE
```
## Step 2: Generate the Subgraph Manifest
Once the project is initialized, generate a subgraph manifest by running the following command in the Dev Container:
```bash
substreams codegen subgraph
```

You will generate a`subgraph.yaml` manifest which imports the Substreams package as a data source:

```yaml
---
dataSources:
- kind: substreams
name: my_project_sol
network: solana-mainnet-beta
source:
package:
moduleName: map_spl_transfers # Module defined in the substreams.yaml
file: ./my-project-sol-v0.1.0.spkg
mapping:
apiVersion: 0.0.7
kind: substreams/graph-entities
file: ./src/mappings.ts
handler: handleTriggers
```
## Step 3: Define Entities in `schema.graphql`

Define the fields you want to save in your subgraph entities by updating the `schema.graphql` file. Here is an example:

```graphql
type MyTransfer @entity {
id: ID!
amount: String!
source: String!
designation: String!
signers: [String!]!
}
```

This schema defines a `MyTransfer` entity with fields such as `id`, `amount`, `source`, `designation`, and `signers`.

## Step 4: Generate Protobuf Files

To generate Protobuf objects in AssemblyScript, run the following command:

```bash
npm run protogen
```

This command converts the Protobuf definitions into AssemblyScript, allowing you to use them in the subgraph's handler.

## Step 5: Handle Substreams Data in `mappings.ts`

With the Protobuf objects generated, you can now handle the decoded Substreams data in your `mappings.ts` file found in the `./src` directory. The example below demonstrates how to extract to subgraph entities the non-derived transfers associated to the Orca account id:

```ts
import { Protobuf } from 'as-proto/assembly'
import { Events as protoEvents } from './pb/sf/solana/spl/token/v1/Events'
import { MyTransfer } from '../generated/schema'
export function handleTriggers(bytes: Uint8Array): void {
const input: protoEvents = Protobuf.decode<protoEvents>(bytes, protoEvents.decode)
for (let i = 0; i < input.data.length; i++) {
const event = input.data[i]
if (event.transfer != null) {
let entity_id: string = `${event.txnId}-${i}`
const entity = new MyTransfer(entity_id)
entity.amount = event.transfer!.instruction!.amount.toString()
entity.source = event.transfer!.accounts!.source
entity.designation = event.transfer!.accounts!.destination

if (event.transfer!.accounts!.signer!.single != null) {
entity.signers = [event.transfer!.accounts!.signer!.single.signer]
} else if (event.transfer!.accounts!.signer!.multisig != null) {
entity.signers = event.transfer!.accounts!.signer!.multisig!.signers
}
entity.save()
}
}
}
```

## Conclusion

You’ve successfully set up a trigger-based Substreams-powered subgraph for a Solana SPL token. You can now further customize your schema, mappings, and modules to suit your specific use case.

For more advanced customization and optimizations, check out the official [Substreams documentation](https://substreams.streamingfast.io/tutorials/solana).
37 changes: 37 additions & 0 deletions website/pages/ar/sps/triggers.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: Substreams Triggers
---

Custom triggers allow you to send data directly into your subgraph mappings file and entities (similar to tables and fields), enabling full use of the GraphQL layer. By importing the Protobuf definitions emitted by your Substreams module, you can receive and process this data within your subgraph’s handler, ensuring efficient and streamlined data management within the subgraph framework.

> Note: If you haven’t already, visit one of the How-To Guides found [here](./introduction) to scaffold your first project in the Development Container.
The following code demonstrates how to define a `handleTransactions` function in a subgraph handler. This function receives raw Substreams bytes as a parameter and decodes them into a `Transactions` object. For each transaction, a new subgraph entity is created.

```tsx
export function handleTransactions(bytes: Uint8Array): void {
let transactions = assembly.eth.transaction.v1.Transactions.decode(bytes.buffer).trasanctions // 1.
if (transactions.length == 0) {
log.info('No transactions found', [])
return
}

for (let i = 0; i < transactions.length; i++) {
// 2.
let transaction = transactions[i]

let entity = new Transaction(transaction.hash) // 3.
entity.from = transaction.from
entity.to = transaction.to
entity.save()
}
}
```

Here's what you’re seeing in the `mappings.ts` file:

1. The bytes containing Substreams data are decoded into the generated `Transactions` object, this object is used like any other AssemblyScript object
2. Looping over the transactions
3. Create a new subgraph entity for every transaction

To go through a detailed example of a trigger-based subgraph, [click here](./triggers-example).
5 changes: 5 additions & 0 deletions website/pages/cs/sps/_meta.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import meta from '../../en/sps/_meta.js'

export default {
...meta,
}
19 changes: 19 additions & 0 deletions website/pages/cs/sps/introduction.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: Introduction to Substreams-powered Subgraphs
---

By using a Substreams package (`.spkg`) as a data source, your subgraph gains access to a stream of pre-indexed blockchain data. This enables more efficient and scalable data handling, especially with large or complex blockchain networks.

There are two methods of enabling this technology:

Using Substreams [triggers](./triggers): Consume from any Substreams module by importing the Protobuf model through a subgraph handler and move all your logic into a subgraph. This method creates the subgraph entities directly in the subgraph.

Using [Entity Changes](https://substreams.streamingfast.io/documentation/consume/subgraph/graph-out): By writing more of the logic into Substreams, you can consume the module's output directly into graph-node. In graph-node, you can use the Substreams data to create your subgraph entities.

It is really a matter of where you put your logic, in the subgraph or the Substreams. Keep in mind that having more of your logic in Substreams benefits from a parallelized model, whereas triggers will be linearly consumed in graph-node.

Visit the following links for How-To Guides on using code-generation tooling to build your first end-to-end project quickly:

- [Solana](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/solana)
- [EVM](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/evm)
- [Injective](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/injective)
137 changes: 137 additions & 0 deletions website/pages/cs/sps/triggers-example.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
title: 'Tutorial: Set Up a Substreams-Powered Subgraph on Solana'
---

## Prerequisites

Before starting, make sure to:

- Complete the [Getting Started Guide](https://github.com/streamingfast/substreams-starter) to set up your development environment using a Dev Container.
- Be familiar with The Graph and basic blockchain concepts such as transactions and Protobufs.

## Step 1: Initialize Your Project

1. Open your Dev Container and run the following command to initialize your project:

```bash
substreams init
```

2. Select the "minimal" project option.
3. Replace the contents of the generated `substreams.yaml` file with the following configuration, which filters transactions for the Orca account on the SPL token program ID:

```yaml
specVersion: v0.1.0
package:
name: my_project_sol
version: v0.1.0

imports: # Pass your spkg of interest
solana: https://github.com/streamingfast/substreams-solana-spl-token/raw/master/tokens/solana-spl-token-v0.1.0.spkg

modules:
- name: map_spl_transfers
use: solana:map_block # Select corresponding modules available within your spkg
initialBlock: 260000082

- name: map_transactions_by_programid
use: solana:solana:transactions_by_programid_without_votes

network: solana-mainnet-beta

params: # Modify the param fields to meet your needs
# For program_id: TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA
map_spl_transfers: token_contract:orcaEKTdK7LKz57vaAYr9QeNsVEPfiu6QeMU1kektZE
```
## Step 2: Generate the Subgraph Manifest
Once the project is initialized, generate a subgraph manifest by running the following command in the Dev Container:
```bash
substreams codegen subgraph
```

You will generate a`subgraph.yaml` manifest which imports the Substreams package as a data source:

```yaml
---
dataSources:
- kind: substreams
name: my_project_sol
network: solana-mainnet-beta
source:
package:
moduleName: map_spl_transfers # Module defined in the substreams.yaml
file: ./my-project-sol-v0.1.0.spkg
mapping:
apiVersion: 0.0.7
kind: substreams/graph-entities
file: ./src/mappings.ts
handler: handleTriggers
```
## Step 3: Define Entities in `schema.graphql`

Define the fields you want to save in your subgraph entities by updating the `schema.graphql` file. Here is an example:

```graphql
type MyTransfer @entity {
id: ID!
amount: String!
source: String!
designation: String!
signers: [String!]!
}
```

This schema defines a `MyTransfer` entity with fields such as `id`, `amount`, `source`, `designation`, and `signers`.

## Step 4: Generate Protobuf Files

To generate Protobuf objects in AssemblyScript, run the following command:

```bash
npm run protogen
```

This command converts the Protobuf definitions into AssemblyScript, allowing you to use them in the subgraph's handler.

## Step 5: Handle Substreams Data in `mappings.ts`

With the Protobuf objects generated, you can now handle the decoded Substreams data in your `mappings.ts` file found in the `./src` directory. The example below demonstrates how to extract to subgraph entities the non-derived transfers associated to the Orca account id:

```ts
import { Protobuf } from 'as-proto/assembly'
import { Events as protoEvents } from './pb/sf/solana/spl/token/v1/Events'
import { MyTransfer } from '../generated/schema'
export function handleTriggers(bytes: Uint8Array): void {
const input: protoEvents = Protobuf.decode<protoEvents>(bytes, protoEvents.decode)
for (let i = 0; i < input.data.length; i++) {
const event = input.data[i]
if (event.transfer != null) {
let entity_id: string = `${event.txnId}-${i}`
const entity = new MyTransfer(entity_id)
entity.amount = event.transfer!.instruction!.amount.toString()
entity.source = event.transfer!.accounts!.source
entity.designation = event.transfer!.accounts!.destination

if (event.transfer!.accounts!.signer!.single != null) {
entity.signers = [event.transfer!.accounts!.signer!.single.signer]
} else if (event.transfer!.accounts!.signer!.multisig != null) {
entity.signers = event.transfer!.accounts!.signer!.multisig!.signers
}
entity.save()
}
}
}
```

## Conclusion

You’ve successfully set up a trigger-based Substreams-powered subgraph for a Solana SPL token. You can now further customize your schema, mappings, and modules to suit your specific use case.

For more advanced customization and optimizations, check out the official [Substreams documentation](https://substreams.streamingfast.io/tutorials/solana).
Loading

0 comments on commit 5fffe03

Please sign in to comment.