Skip to content

Commit b575045

Browse files
teogebteogeb
authored
chore(network-subgraphs): Template for subgraph.yaml (#1016)
Extracted subgraph configs (mainly contract addresses and `startBlock` values) to a separate `subgraph-config.json` file. Now the actual `subgraph.yaml` is generated in `build.sh` by filling the config options to a Handlebars template. We don't need separate environment specific `subgraph_* yaml` files anymore. ## Implementation There is a small `fill-handlebars-template.mjs` script which reads the config from `stdin` and outputs the filled template to `stdout`. There is no custom logic in that script, and alternative we could use some ready-made external Handlebars utility instead of this script. ## Config file The config file was generated from the existing `subgraph_*.yaml` files by this script: ```js import { readFileSync } from 'fs' import yaml from 'js-yaml' const ENVIRONMENT_IDS = ['dev2', 'polygon', 'polygonAmoy', 'peaq', 'iotex'] const output = {} for (const environmentId of ENVIRONMENT_IDS) { const fileId = (environmentId === 'polygonAmoy') ? 'amoy' : environmentId const fileName = `subgraph_${fileId}.yaml` const fileContents = readFileSync(fileName, 'utf8') const yamlData = yaml.load(fileContents) const contracts = {} for (const dataSource of yamlData.dataSources) { const source = dataSource.source const key = (source.abi === 'NodeRegistry') ? 'StorageNodeRegistry' : source.abi contracts[key] = { address: source.address, startBlock: source.startBlock } } const networkId = (environmentId === 'polygon') ? 'matic' : ((environmentId === 'polygonAmoy') ? 'polygon-amoy' : environmentId) output[environmentId] = { networkId, hubContracts: (environmentId !== 'iotex'), contracts } } console.log(JSON.stringify(output, undefined, 4)) ``` ## Small changes to subgraph.yaml output The subgraph.yaml output generated by the `build.sh` matches almost exactly the current files. Exceptions: - removed some TODO comments - unified quotes to be single quotes - unified the description See three commits in this PR for the exact unifications. There was also test.sh script in commit [bda37c6](bda37c6) to check that the outputs match the current files. ## Notes about Iotex Thet addresses in the `subgraph-config.json` matches the addresses which were used in `subgraph_iotex.yaml`, but don't match the values of `@streamr/config`. Therefore this PR doesn't change any functionality. There is a separate PR about this #1010 issue. Also in that PR or in a follow-up PR we may add hub contracts to `iotex` config. If we do that we should remove the `hubContracts` property from the `subgraph-configs.json` (and the related if statement in `subgraph.yaml.hbs`) as the value for that is true in environment.s
1 parent ced4fd8 commit b575045

File tree

11 files changed

+260
-1263
lines changed

11 files changed

+260
-1263
lines changed

‎package-lock.json‎

Lines changed: 5 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/network-subgraphs/README.md‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,17 +21,15 @@ Troubleshooting tips:
2121

2222
## Mainnet deployment to the Arbitrum / decentralized service (indexing Polygon MATIC)
2323
1. Authenticate: Log into `https://thegraph.com/studio/subgraph/streamr/` using "Streamr subgraph owner" key from 1password. On the right hand side, look for "Auth & Deploy" and "Authenticate in CLI". Copy the command and run it in the terminal: `npx graph auth DEPLOY_KEY`
24-
1. `cp subgraph_matic.yaml subgraph.yaml`
25-
1. `npm run build` (optional)
24+
1. `./build.sh polygon`
2625
1. `npx graph deploy streamr`
2726
* check version number from browser UI, bump it when asked
2827
1. Follow progress and look at https://thegraph.com/studio/subgraph/streamr/logs for errors
2928

3029
## Testnet deployment to Studio (indexing Polygon Amoy)
3130
1. Log into: `https://thegraph.com/studio/subgraph/streamr-amoy-testnet/`
3231
1. `npx graph auth --studio <KEY>`
33-
1. `cp subgraph_amoy.yaml subgraph.yaml`
34-
1. `npm run build`
32+
1. `./build.sh polygonAmoy`
3533
1. `npx graph deploy --studio streamr-amoy-testnet`
3634
1. Follow progress and look at https://thegraph.com/hosted-service/subgraph/streamr-dev/network?version=pending&selected=logs for errors
3735
``

‎packages/network-subgraphs/build.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ fi
77

88
# TODO could build these to "dist" directory so that we don't need to clean up the subgraph.yaml in package.json (the "rm subgraph.yaml" command)
99
# - then we can also remove subgraph.yaml from .gitignore
10-
cp subgraph_$1.yaml subgraph.yaml
10+
jq '.'$1 subgraph-config.json | node scripts/fill-handlebars-template.mjs subgraph.yaml.hbs > subgraph.yaml
1111
./scripts/copyAbisFromContractsPackage.sh
1212
npx graph codegen
1313
npx graph build

‎packages/network-subgraphs/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
},
1313
"devDependencies": {
1414
"@graphprotocol/graph-cli": "0.89.0",
15+
"handlebars": "4.7.8",
1516
"matchstick-as": "0.5.0"
1617
},
1718
"dependencies": {

‎packages/network-subgraphs/scripts/fill-handlebars-template.mjs‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
import { readFileSync } from 'fs'
2+
import handlebars from 'handlebars'
3+
4+
const readFromStdin = async () => {
5+
let result = ''
6+
for await (const chunk of process.stdin) {
7+
result += chunk
8+
}
9+
return result
10+
}
11+
12+
const main = async () => {
13+
const template = readFileSync(process.argv[2], 'utf8')
14+
const compiled = handlebars.compile(template)
15+
const data = JSON.parse(await readFromStdin())
16+
process.stdout.write(compiled(data))
17+
}
18+
19+
main()

‎packages/network-subgraphs/subgraph-config.json‎

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
{
2+
"dev2": {
3+
"networkId": "dev2",
4+
"hubContracts": true,
5+
"contracts": {
6+
"StreamRegistry": {
7+
"address": "0x18E0937099660B82464475Ea2B7e6Af4f2BFE5F0",
8+
"startBlock": 0
9+
},
10+
"StorageNodeRegistry": {
11+
"address": "0x338090C5492C5c5E41a4458f5FC4b205cbc54A24",
12+
"startBlock": 0
13+
},
14+
"StreamStorageRegistry": {
15+
"address": "0xB73189CeBCc870bE6f9aa16764bbB3665e4B49B6",
16+
"startBlock": 0
17+
},
18+
"ProjectRegistryV1": {
19+
"address": "0xA9166376706611110f814D1333b8E80FEc5d8f10",
20+
"startBlock": 0
21+
},
22+
"MarketplaceV4": {
23+
"address": "0xb8678223183d560280a7BEF68daAbB0E3daBd97D",
24+
"startBlock": 0
25+
},
26+
"ProjectStakingV1": {
27+
"address": "0x73a9310C43621B853C508902bb8c1DA8f1240EaE",
28+
"startBlock": 0
29+
},
30+
"StreamrConfig": {
31+
"address": "0x00E680d549FE53a627a3db86a6F88fA2471CFfAa",
32+
"startBlock": 0
33+
},
34+
"SponsorshipFactory": {
35+
"address": "0x699B4bE95614f017Bb622e427d3232837Cc814E6",
36+
"startBlock": 0
37+
},
38+
"OperatorFactory": {
39+
"address": "0xD13D34d37e2c94cb35EA8D5DE7498Cb7830d26e0",
40+
"startBlock": 0
41+
}
42+
}
43+
},
44+
"polygon": {
45+
"networkId": "matic",
46+
"hubContracts": true,
47+
"contracts": {
48+
"StreamRegistry": {
49+
"address": "0x0D483E10612F327FC11965Fc82E90dC19b141641",
50+
"startBlock": 23586971
51+
},
52+
"StorageNodeRegistry": {
53+
"address": "0x080F34fec2bc33928999Ea9e39ADc798bEF3E0d6",
54+
"startBlock": 23562861
55+
},
56+
"StreamStorageRegistry": {
57+
"address": "0xe8e2660CeDf2a59C917a5ED05B72df4146b58399",
58+
"startBlock": 23586998
59+
},
60+
"ProjectRegistryV1": {
61+
"address": "0x496a6154da5aA6a021a3bd0DCd337DA80F48a6e1",
62+
"startBlock": 38700928
63+
},
64+
"MarketplaceV4": {
65+
"address": "0xdF8b74D735197dCD5C94bE933db080E69A958de6",
66+
"startBlock": 39591313
67+
},
68+
"ProjectStakingV1": {
69+
"address": "0xAA7a4BdBE91F143F3103206e48A8AfF21101B6DE",
70+
"startBlock": 38701364
71+
},
72+
"StreamrConfig": {
73+
"address": "0x869e88dB146ECAF20dDf199a12684cD80c263c8f",
74+
"startBlock": 50045923
75+
},
76+
"SponsorshipFactory": {
77+
"address": "0x820b2f9a15ed45F9802c59d0CC77C22C81755e45",
78+
"startBlock": 50046404
79+
},
80+
"OperatorFactory": {
81+
"address": "0x935734e66729b69260543Cf6e5EfeB42AC962183",
82+
"startBlock": 50046356
83+
}
84+
}
85+
},
86+
"polygonAmoy": {
87+
"networkId": "polygon-amoy",
88+
"hubContracts": true,
89+
"contracts": {
90+
"StreamRegistry": {
91+
"address": "0xE9C98bdE63248e58E9137Db8270D9675B9E34b93",
92+
"startBlock": 6550000
93+
},
94+
"StorageNodeRegistry": {
95+
"address": "0x02fdF917f4e6Ae8F7F1bBDd28179d819E2b76820",
96+
"startBlock": 6550000
97+
},
98+
"StreamStorageRegistry": {
99+
"address": "0x0f3671A9A92416E1aD32750faCf2AD4FA1b66f78",
100+
"startBlock": 6550000
101+
},
102+
"ProjectRegistryV1": {
103+
"address": "0xc5e1434d35c0c7291c7032Fd9C4096b4876C6823",
104+
"startBlock": 6550000
105+
},
106+
"MarketplaceV4": {
107+
"address": "0x6C8eaA8e0bF605469c15b6F9106387B4cEC99976",
108+
"startBlock": 6550000
109+
},
110+
"ProjectStakingV1": {
111+
"address": "0x3A27A16770477EbcFb4B81cE462F4f12591767A0",
112+
"startBlock": 6550000
113+
},
114+
"StreamrConfig": {
115+
"address": "0x835bC97D2a61bbF5d05932C5105Ca34e1b815F94",
116+
"startBlock": 6550000
117+
},
118+
"SponsorshipFactory": {
119+
"address": "0xb194a68b166f2e3074B551393fA61490D19c69f8",
120+
"startBlock": 6550000
121+
},
122+
"OperatorFactory": {
123+
"address": "0xE02E8E9fF5ea6a58F34D00C0e4B091e066B9fA81",
124+
"startBlock": 6550000
125+
}
126+
}
127+
},
128+
"peaq": {
129+
"networkId": "peaq",
130+
"hubContracts": true,
131+
"contracts": {
132+
"StreamRegistry": {
133+
"address": "0xD0C720e99Bd39311614f292d8B0B4e351Bde157c",
134+
"startBlock": 900000
135+
},
136+
"StorageNodeRegistry": {
137+
"address": "0xaCF9e8134047eDc671162D9404BF63a587435bAa",
138+
"startBlock": 900000
139+
},
140+
"StreamStorageRegistry": {
141+
"address": "0xB36CF8ee4219a3Ac9FBC3865C35E9a99353c26db",
142+
"startBlock": 900000
143+
},
144+
"ProjectRegistryV1": {
145+
"address": "0xB14205AE3D7672B937B972EBdf2b986fB0B8CfaA",
146+
"startBlock": 900000
147+
},
148+
"MarketplaceV4": {
149+
"address": "0x959684926448296ea72BD4B05593426A3cD3F0dF",
150+
"startBlock": 900000
151+
},
152+
"ProjectStakingV1": {
153+
"address": "0xd1cBD6b1Be710C41c01A9613802209843407b8b9",
154+
"startBlock": 900000
155+
},
156+
"StreamrConfig": {
157+
"address": "0xc81Fa3Cc3E81dF24D21bfc608f8FB06262Da4F8c",
158+
"startBlock": 900000
159+
},
160+
"SponsorshipFactory": {
161+
"address": "0xdaF0905e19C45b9Fea7E60D232061edF78E9F570",
162+
"startBlock": 900000
163+
},
164+
"OperatorFactory": {
165+
"address": "0x6d201374D7c28EF1c3cc1b6FA300f06110B597A3",
166+
"startBlock": 900000
167+
}
168+
}
169+
},
170+
"iotex": {
171+
"networkId": "iotex",
172+
"hubContracts": false,
173+
"contracts": {
174+
"StreamRegistry": {
175+
"address": "0x080F34fec2bc33928999Ea9e39ADc798bEF3E0d6",
176+
"startBlock": 33510000
177+
},
178+
"StorageNodeRegistry": {
179+
"address": "0xb9e70Bd9833dd091f06edc8073aa86a76EdeF46D",
180+
"startBlock": 33510000
181+
},
182+
"StreamStorageRegistry": {
183+
"address": "0xF79c40Fbc432f1f8B8B017441B644e0ac7bccea2",
184+
"startBlock": 33510000
185+
},
186+
"StreamrConfig": {
187+
"address": "0x23Dd2D1f39AD0f9d517ce56Ca490eF19B50e6f1A",
188+
"startBlock": 33510000
189+
},
190+
"SponsorshipFactory": {
191+
"address": "0xe4Ae8e6c5b6E85914738803ad73c111eF2618621",
192+
"startBlock": 33510000
193+
},
194+
"OperatorFactory": {
195+
"address": "0xbC1cC84912d54fCf7316A6dA2e7A884731a87935",
196+
"startBlock": 33510000
197+
}
198+
}
199+
}
200+
}

0 commit comments

Comments
 (0)