-
Notifications
You must be signed in to change notification settings - Fork 155
Publish composable subgraphs #919
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
export default { | ||
'subgraph-composition': '', | ||
'subgraph-debug-forking': '', | ||
near: '', | ||
arweave: '', | ||
|
132 changes: 132 additions & 0 deletions
132
website/src/pages/en/subgraphs/guides/subgraph-composition.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,132 @@ | ||
--- | ||
title: Aggregate Data Using Subgraph Composition | ||
sidebarTitle: 'Build a Composable Subgraph with Multiple Subgraphs' | ||
--- | ||
|
||
Leverage Subgraph composition to speed up development time. Create a base Subgraph with essential data, then build additional Subgraphs on top of it. | ||
|
||
Optimize your Subgraph by merging data from independent, source Subgraphs into a single composable Subgraph to enhance data aggregation. | ||
|
||
## Introduction | ||
|
||
Composable Subgraphs enable you to combine multiple Subgraphs' data sources into a new Subgraph, facilitating faster and more flexible Subgraph development. Subgraph composition empowers you to create and maintain smaller, focused Subgraphs that collectively form a larger, interconnected dataset. | ||
|
||
### Benefits of Composition | ||
|
||
Subgraph composition is a powerful feature for scaling, allowing you to: | ||
|
||
- Reuse, mix, and combine existing data | ||
- Streamline development and queries | ||
- Use multiple data sources (up to five source Subgraphs) | ||
- Speed up your Subgraph's syncing speed | ||
- Handle errors and optimize the resync | ||
|
||
## Architecture Overview | ||
|
||
The setup for this example involves two Subgraphs: | ||
|
||
1. **Source Subgraph**: Tracks event data as entities. | ||
2. **Dependent Subgraph**: Uses the source Subgraph as a data source. | ||
|
||
You can find these in the `source` and `dependent` directories. | ||
|
||
- The **source Subgraph** is a basic event-tracking Subgraph that records events emitted by relevant contracts. | ||
- The **dependent Subgraph** references the source Subgraph as a data source, using the entities from the source as triggers. | ||
|
||
While the source Subgraph is a standard Subgraph, the dependent Subgraph uses the Subgraph composition feature. | ||
|
||
## Prerequisites | ||
|
||
### Source Subgraphs | ||
|
||
- All Subgraphs need to be published with a **specVersion 1.3.0 or later** (Use the latest graph-cli version to be able to deploy composable Subgraphs) | ||
- See notes here: https://github.com/graphprotocol/graph-node/releases/tag/v0.37.0 | ||
- Immutable entities only: All Subgraphs must have [immutable entities](https://thegraph.com/docs/en/subgraphs/best-practices/immutable-entities-bytes-as-ids/#immutable-entities) when the Subgraph is deployed | ||
- Pruning can be used in the source Subgraphs, but only entities that are immutable can be composed on top of | ||
- Source Subgraphs cannot use grafting on top of existing entities | ||
- Aggregated entities can be used in composition, but entities that are composed from them cannot performed additional aggregations directly | ||
|
||
### Composed Subgraphs | ||
|
||
- You can only compose up to a **maximum of 5 source Subgraphs** | ||
- Composed Subgraphs can only use **datasources from the same chain** | ||
- **Nested composition is not yet supported**: Composing on top of another composed Subgraph isn’t allowed at this time | ||
- Aggregated entities can be used in composition, but the composed entities on them cannot also use aggregations directly | ||
- Developers cannot compose an onchain datasource with a Subgraph datasource (i.e. you can’t do normal event handlers and call handlers and block handlers in a composed Subgraph) | ||
|
||
Additionally, you can explore the [example-composable-subgraph](https://github.com/graphprotocol/example-composable-subgraph) repository for a working implementation of composable Subgraphs | ||
|
||
## Get Started | ||
|
||
The following guide provides examples for defining 3 source Subgraphs to create one powerful composed Subgraph. | ||
|
||
### Specifics | ||
|
||
- To keep this example simple, all source Subgraphs use only block handlers. However, in a real environment, each source Subgraph will use data from different smart contracts. | ||
- The examples below show how to import and extend the schema of another Subgraph to enhance its functionality. | ||
- Each source Subgraph is optimized with a specific entity. | ||
- All the commands listed install the necessary dependencies, generate code based on the GraphQL schema, build the Subgraph, and deploy it to your local Graph Node instance. | ||
|
||
### Step 1. Deploy Block Time Source Subgraph | ||
|
||
This first source Subgraph calculates the block time for each block. | ||
|
||
- It imports schemas from other Subgraphs and adds a `block` entity with a `timestamp` field, representing the time each block was mined. | ||
- It listens to time-related blockchain events (e.g., block timestamps) and processes this data to update the Subgraph's entities accordingly. | ||
|
||
To deploy this Subgraph locally, run the following commands: | ||
|
||
```bash | ||
npm install | ||
npm run codegen | ||
npm run build | ||
npm run create-local | ||
npm run deploy-local | ||
``` | ||
|
||
### Step 2. Deploy Block Cost Source Subgraph | ||
|
||
This second source Subgraph indexes the cost of each block. | ||
|
||
#### Key Functions | ||
|
||
- It imports schemas from other Subgraphs and adds a `block` entity with cost-related fields. | ||
- It listens to blockchain events related to costs (e.g. gas fees, transaction costs) and processes this data to update the Subgraph's entities accordingly. | ||
|
||
To deploy this Subgraph locally, run the same commands as above. | ||
|
||
### Step 3. Define Block Size in Source Subgraph | ||
|
||
This third source Subgraph indexes the size of each block. To deploy this Subgraph locally, run the same commands as above. | ||
|
||
#### Key Functions | ||
|
||
- It imports existing schemas from other Subgraphs and adds a `block` entity with a `size` field representing each block's size. | ||
- It listens to blockchain events related to block sizes (e.g., storage or volume) and processes this data to update the Subgraph's entities accordingly. | ||
|
||
### Step 4. Combine Into Block Stats Subgraph | ||
|
||
This composed Subgraph combines and aggregates the information from the source Subgraphs above, providing a unified view of block statistics. To deploy this Subgraph locally, run the same commands as above. | ||
|
||
> Note: | ||
> | ||
> - Any change to a source Subgraph will likely generate a new deployment ID. | ||
> - Be sure to update the deployment ID in the data source address of the Subgraph manifest to take advantage of the latest changes. | ||
> - All source Subgraphs should be deployed before the composed Subgraph is deployed. | ||
|
||
#### Key Functions | ||
|
||
- It provides a consolidated data model that encompasses all relevant block metrics. | ||
- It combines data from 3 source Subgraphs, and provides a comprehensive view of block statistics, enabling more complex queries and analyses. | ||
|
||
## Key Takeaways | ||
|
||
- This powerful tool will scale your Subgraph development and allow you to combine multiple Subgraphs. | ||
- The setup includes the deployment of 3 source Subgraphs and one final deployment of the composed Subgraph. | ||
- This feature unlocks scalability, simplifying both development and maintenance efficiency. | ||
|
||
## Additional Resources | ||
|
||
- Check out all the code for this example in [this GitHub repo](https://github.com/graphprotocol/example-composable-subgraph). | ||
- To add advanced features to your Subgraph, check out [Subgraph advanced features](/developing/creating/advanced/). | ||
- To learn more about aggregations, check out [Timeseries and Aggregations](/subgraphs/developing/creating/advanced/#timeseries-and-aggregations). |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be a relative URL.