Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 21 additions & 28 deletions website/pages/en/substreams.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,43 +4,36 @@ title: Substreams

![Substreams Logo](/img/substreams-logo.png)

Substreams is a powerful blockchain indexing technology developed for The Graph Network. It enables developers to write Rust modules, compose data streams alongside the community, and provide extremely high-performance indexing due to parallelization in a streaming-first approach.
Substreams is a powerful blockchain indexing technology developed to improve The Graph Network:
- Substreams allows you to speed up the indexing times of your subgraph.
- Substreams allows you to index non-EVM data (Solana, Injective, Starknet, Vara...)
Comment on lines +8 to +9
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be "Substreams allow you"?


With Substreams, developers can quickly extract data from different blockchains (Ethereum, BNB, Solana, ect.) and send it to various locations of their choice, such as a Postgres database, a Mongo database, or a Subgraph. Additionally, Substreams packages enable developers to specify which data they want to extract from the blockchain.
This is possible because your subgraph can use a Substreams package as a data source, thus creating a Substreams-powered Subgraph (SpS).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is possible because your subgraph can use a Substreams package as a data source, thus creating a Substreams-powered Subgraph (SpS).
This is possible because your subgraph can use a Substreams package as a data source, thus creating a Substreams-powered subgraph.

According to "The Graph Style Guide", we shouldn't capitalize "subgraph", even in the phrase "Substreams-powered subgraphs".

(@bermchain, should we make that document public, and maybe link to it from the docs' CONTRIBUTING guidelines?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh and I propose removing the acronym because it's not used anywhere else on this page, but maybe it's used somewhere else and I'm not aware?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed @benface with lowercase subgraph and not using the acronym unless it's recurring.

I like the idea of publicizing the style guide in the Docs! Let's do it -- I'll select sections that make sense to share.

Also, re: "Substreams is" vs. "Substreams are" -- I believe it's the former because Substreams is a collective noun for the technology (i.e., powered by Substreams tech, not powered by a Substream).

A Substreams package a binary file that contains all the definitions of the data that you want to extract from the blockchain (similar to the `mapping.ts` file of your subgraph).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A Substreams package a binary file that contains all the definitions of the data that you want to extract from the blockchain (similar to the `mapping.ts` file of your subgraph).
A Substreams package is a binary file that contains all the definitions of the data that you want to extract from the blockchain (similar to the `mapping.ts` file of your subgraph).


## How Substreams Works in 4 Steps
The following example shows a subgraph manifest (`subgraph.yaml`) that uses a Substreams package as data source, thus receiving the data extracted by the Substreams.
Then, you can use this data in your `mapping.ts` and perform all the transformations needed as a usual subgraph.
Comment on lines +14 to +15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The following example shows a subgraph manifest (`subgraph.yaml`) that uses a Substreams package as data source, thus receiving the data extracted by the Substreams.
Then, you can use this data in your `mapping.ts` and perform all the transformations needed as a usual subgraph.
The following example shows a subgraph manifest (`subgraph.yaml`) that uses a Substreams package as data source, thus receiving the data extracted by the Substreams.
Then, you can use this data in your `mapping.ts` and perform all the transformations needed as a usual subgraph.

I'm thinking this is why the Prettier check is failing, but it could be more than one thing. If you're using VS Code, I recommend installing the Prettier extension with "auto-format on save" enabled.


1. **You write a Rust program, which defines the transformations that you want to apply to the blockchain data.** For example, the following Rust function extracts relevant information from an Ethereum block (number, hash, and parent hash).
// example
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I just realized this was a WIP; sorry I didn't pay attention to the title. 😅 I'll stop reviewing now unless you want me to continue.


```rust
fn get_my_block(blk: Block) -> Result<MyBlock, substreams::errors::Error> {
let header = blk.header.as_ref().unwrap();
## Find Substreams packages

Ok(MyBlock {
number: blk.number,
hash: Hex::encode(&blk.hash),
parent_hash: Hex::encode(&header.parent_hash),
})
}
```
There are many Substreams packages already built that you can simply import into your subgraph.
[substreams.dev](https://substreams.dev) is a Substreams registry where you can explore different Substreams packages for several blockchain networks.

2. **You wrap up your Rust program into a WASM module just by running a single CLI command.**
// image

3. **The WASM container is sent to a Substreams endpoint for execution.** The Substreams provider feeds the WASM container with the blockchain data and the transformations are applied.
## Create Your Own Substreams Package

4. **You select a [sink](https://substreams.streamingfast.io/documentation/consume/other-sinks), a place where you want to send the transformed data** (a Postgres database or a Subgraph, for example).
If you can't find a Substreams package that fits your needs, you can create your own package.
Creating a Substreams involves a Rust function that selects which specific data you want to extract from the network.

## Substreams Documentation
However, the Substreams CLI allows you to auto-generate a Substreams project without writing any code.
You can import Solana transactions or Vara extrinsics in your subgraph by using the auto-generation tools provided by The Graph.

The official Substreams documentation is currently maintained by the StreamingFast team [on the StreamingFast website](https://substreams.streamingfast.io/).
In the folllowing example, you run the `substreams init` command (similar to the `graph init` command), where you can select which kind of project you want to initialize.
You can choose among a variety of project generations (Solana transactions, EVM events and calls, Starknet transactions...).

To learn about the latest version of Substreams CLI, which enables developers to bootstrap a Substreams project without any code, please check [Substreams Codegen](https://streamingfastio.medium.com/substreams-codegen-no-code-tool-to-bootstrap-your-project-a11efe0378c6).
// image

### Getting Started

- In order to develop and deploy a Substreams, [you must install the Substreams CLI](https://substreams.streamingfast.io/documentation/consume/installing-the-cli).
- Then, run your first Substreams by following the [Quickstart Tutorial](https://substreams.streamingfast.io/documentation/develop/init-project).

### Expand Your Knowledge

- Take a look at the [Ethereum Explorer Tutorial](https://substreams.streamingfast.io/tutorials/evm) to learn about the basic transformations you can create with Substreams.
From the generated project, you can run `substreams codegen subgraph` to convert the Substreams project into a ready-to-deploy subgraph project!
44 changes: 44 additions & 0 deletions website/pages/en/substreams/getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: Getting Started with Substreams
---

## Getting Started

The easiest way to get started with Substreams is by using one of the auto-generation tools. The Substreams CLI provides the `substreams init` command, which will allow you to initialize your Substreams project for several blockchains (EVM, Solana, Injective...).

The Substreams CLI is pretty similar to The Graph CLI: you install it in your computer and execution the different commands available.

### The Development Environment

To facilitate the development of Substreams-powered Subgraphs, a local development environment for VSCode is available, the [substreams-starter](https://github.com/streamingfast/substreams-starter) project.

The development environment is a VSCode extension that spins up all the necessary dependencies in Docker containers:
- A terminal-based environment with the Substreams CLI pre-installed, so you don't have to install
- A Graph Node instance, which you can uses to test your subgraph locally.
- A PostgreSQL instance, which is used by the Graph Node to store all the necessary data for your subgraphs.
- An IPFS instance, which is used by the Graph Node to store the _build_ files of the subgraph.

You can run the development environment locally (you need both VSCode and Docker) or remotely through GitHub codespaces.

#### Run the Development Environment Locally

To run it locally, simply open the [substreams-starter](https://github.com/streamingfast/substreams-starter) in your local VSCode application.

#### Run the Development Environment Remotely

To run it in GitHub codespace, open [this link](https://github.com/codespaces/new/streamingfast/substreams-starter?machine=standardLinux32gb) (`https://github.com/codespaces/new/streamingfast/substreams-starter?machine=standardLinux32gb`) in your browser. Log in with GitHub and you will get access to a VSCode instance in your browser.

### Initialize Your Substreams Project

In the development environment, you will have access to the `substreams init` command. Depending on the type of project that you want to initialize, the flow will be different. Choose which blockchain you want to index:



### Generate Your Subgraph Project

From the generated Substreams project, you can create a Substreams-powered Subgraph project that uses the Substreams as data source.

In the folder of the Substreams project, run `substreams codegen subgraph` and choose the module that you want your subgraph to consume. A new `subgraph` folder will be created with ready-to-deploy subgraph.



Loading