[Concept]: feat: introducing lazy chain loading #644

graham-chainlink · 2025-12-29T04:11:45Z

Chains can now be loaded lazily instead of eager, this means chains will only be loaded when it is being used.

Hidden under the feature toggle CLD_LAZY_BLOCKCHAINS , off by default.

Basically this solves a bunch of issues

avoid loading all the chains in the network config file even they are not used (increases loading time)
failures on loading certain chains that are not used will now not failed the changeset
users having to always specify chain overrides in input yaml in order to avoid loading all the chains (this can be tricky if the chain overrides is a long list of chains too)
basically deprecate the ChainsOverrides feature, simplifying the code

changeset-bot · 2025-12-29T04:11:48Z

🦋 Changeset detected

Latest commit: 52648b1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
chainlink-deployments-framework	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

graham-chainlink · 2025-12-29T04:12:34Z

engine/cld/environment/environment.go

-	// default - loads all chains from the networks config
-	chainSelectorsToLoad := cfg.Networks.ChainSelectors()
-
-	if loadcfg.chainSelectorsToLoad != nil {


We no longer care about what chains to load via chainSelectorsToLoad as we no longer load everything.

chain/lazy_blockchains.go

gustavogama-cll · 2025-12-31T04:02:41Z

chain/lazy_blockchains.go

+// Unlike EVMChains, this method returns an error if any chain fails to load.
+// The error may contain multiple chain load failures wrapped together.
+// Successfully loaded chains are still returned in the map.
+func (l *LazyBlockChains) TryEVMChains() (map[uint64]evm.Chain, error) {


I think you can use a generic implementation here and reduce a lot of the duplication in this file:

func tryChains[T any, PT *T](l *LazyBlockChains, family string) (map[uint64]T, error) { l.mu.RLock() selectors := make([]uint64, 0) for selector, f := range l.availableChains { if f == family { selectors = append(selectors, selector) } } l.mu.RUnlock() chains := make(map[uint64]T) var errs []error for _, selector := range selectors { chain, err := l.GetBySelector(selector) if err != nil { errs = append(errs, fmt.Errorf("failed to load chain %d: %w", selector, err)) continue } switch c := chain.(type) { case T: chains[selector] = c case PT: if c != nil { chains[selector] = *c } } } if len(errs) > 0 { return chains, errors.Join(errs...) } return chains, nil } // then func (l *LazyBlockChains) EVMChains() map[uint64]evm.Chain { chains, err := tryChains[evm.Chain](l, chainsel.FamilyEVM) if err != nil { l.lggr.Errorw("Failed to load one or more EVM chains", "error", err) } return chains } func (l *LazyBlockChains) SolanaChains() map[uint64]solana.Chain { chains, err := tryChains[solana.Chain](l, chainsel.FamilySolana) if err != nil { l.lggr.Errorw("Failed to load one or more Solana chains", "error", err) } return chains } // etc...

Oh! Good pick up, using generic cleaned it up alot, thanks! I had to use the type constraint for it to work

func tryChains[T any, PT interface { *T }](l *LazyBlockChains, family string) (map[uint64]T, error) {

gustavogama-cll · 2025-12-31T04:03:25Z

chain/lazy_blockchains.go

+	chains := make(map[uint64]evm.Chain)
+	var errs []error
+
+	for _, selector := range selectors {


I think this must be done in parallel for the lazy loading to become a viable option.

done! another good pick up, updated the other getters to be parallel too!

chain/lazy_blockchains.go

gustavogama-cll · 2025-12-31T05:01:27Z

engine/cld/environment/environment.go

-	blockChains, err := chains.LoadChains(ctx, lggr, cfg, chainSelectorsToLoad)
+	// Use lazy loading for chains - they will be initialized on first access
+	// All chains from the network config are made available, but only loaded when accessed
+	lazyBlockChains, err := chains.NewLazyBlockChains(ctx, lggr, cfg)


I think we should feature flag this initially. Nothing too complicated, just check for an environment variable -- say, CLD_LAZY_BLOCK_CHAINS=1.

Good idea, added a feature toggle.

Copilot

Pull request overview

This PR introduces lazy loading of blockchain chains as an opt-in feature controlled by the CLD_LAZY_BLOCKCHAINS environment variable. Chains are now loaded on-demand when accessed rather than eagerly at initialization, improving startup performance and allowing users to avoid providing secrets for unused chains.

Key changes:

Introduces LazyBlockChains type that implements on-demand chain loading with caching
Adds BlockChainCollection interface implemented by both BlockChains and LazyBlockChains
Updates the Environment type to use BlockChainCollection for flexibility
Adds comprehensive test coverage for the new lazy loading functionality

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
chain/lazy_blockchains.go	New implementation of lazy chain loading with thread-safe caching
chain/lazy_blockchains_test.go	Comprehensive test suite for lazy loading behavior
chain/blockchain.go	Adds BlockChainCollection interface to support both eager and lazy loading
engine/cld/chains/chains.go	Adds NewLazyBlockChains function and ChainLoader type alias
engine/cld/environment/environment.go	Adds lazy loading path with feature flag check
engine/cld/environment/environment_test.go	Tests for both lazy and eager loading modes
deployment/environment.go	Updates Environment type to use BlockChainCollection interface
engine/test/environment/environment_test.go	Updates type references to BlockChainCollection
engine/cld/legacy/cli/mcmsv2/mcms_v2.go	Updates type reference to BlockChainCollection
experimental/analyzer/evm_analyzer_test.go	Updates test fixtures to use empty BlockChains instances
.changeset/empty-words-tickle.md	Changeset documenting the new feature

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

engine/cld/environment/environment_test.go

DimitriosNaikopoulos · 2026-01-05T11:45:40Z

chain/lazy_blockchains.go

+	l.mu.RLock()
+	if chain, ok := l.loadedChains[selector]; ok {
+		l.mu.RUnlock()
+		return chain, nil
+	}
+	l.mu.RUnlock()


Maybe we could have this as a helper functions that Rlocks and defers the unlock instead of having to cover the unlock in 2 different paths 🤔

Thanks, i think since this is the only scenario, some verbosity is fine, if there are more usage, of this pattern, then i can refactor. This so far still seem readable.

DimitriosNaikopoulos · 2026-01-05T11:55:07Z

chain/lazy_blockchains.go

+
+// EVMChains returns a map of all EVM chains, loading them lazily.
+// If a chain fails to load, the error is logged and the chain is skipped.
+func (l *LazyBlockChains) EVMChains() map[uint64]evm.Chain {


Should these chain specific functions be pushed down to the chain implementation? Just thinking that if we want to make lazy load the default loading method we will need to have these function with every new chain family we integrate therefore keeps all that we need to integrate within the same /chain package.

Hmm , can you elaborate more? Each of these functions eventually calls tryChains which is a generic chain loading function, dont think we want to move that into the chain specific package?

need to have these function with every new chain family

yeah because the design is not to have a single interface that can generically represent all chains which are too different from each other.

DimitriosNaikopoulos · 2026-01-05T12:00:35Z

chain/blockchain.go

+	EVMChains() map[uint64]evm.Chain
+	SolanaChains() map[uint64]solana.Chain
+	AptosChains() map[uint64]aptos.Chain
+	SuiChains() map[uint64]sui.Chain
+	TonChains() map[uint64]ton.Chain
+	TronChains() map[uint64]tron.Chain


Is there a way to make this more generic? Just thinking every time we want to add a new chain family we will need to update the interface 🤔 Instead if we could have some Chains generic map that could accept any type of chain could be useful (although not sure how practical is this because of how different the <chain_family>.Chain structs are 🤔 )

good question, when i initially designed the blockchain type, this was one of the question, then we come to a conclusion where we shouldnt try to make it generic since each chain is so different.

I also remember there was a doc somewhere in Chainlink channel from Connor about the attempt to consolidate the chains, basically it says dont since they are so different.

Chains are loaded lazily instead of eager, this means chains will only be loaded when it is being used, this means users are not forced to provide any secrets for chains if they are not used upfront and avoid loading huge amount of chains that are never used. This change also means we can deprecate `ChainOverrides` feature as we no longer have to tell CLD what chains to load. This lazy loading is hidden under the feature flag CLD_LAZY_BLOCKCHAINS

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-09T05:25:52Z

engine/cld/environment/environment_test.go

+	assert.IsType(t, &fchain.LazyBlockChains{}, env.BlockChains, "Expected LazyBlockChains instance")
+}
+
+func Test_Load_WithEagerBlockchains(t *testing.T) {


The test function name uses inconsistent casing: 'Blockchains' should be 'BlockChains' to match the terminology used throughout the codebase.

cl-sonarqube-production · 2026-01-09T05:30:59Z

Quality Gate failed

Failed conditions
78.6% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

graham-chainlink commented Dec 29, 2025

View reviewed changes

graham-chainlink changed the title ~~feat: introducing lazy chain loading~~ [Concept]: feat: introducing lazy chain loading Dec 29, 2025

graham-chainlink force-pushed the ggoh/lazy-blockchain branch 3 times, most recently from 2c1b033 to a5255df Compare December 29, 2025 04:46

gustavogama-cll reviewed Dec 31, 2025

View reviewed changes

chain/lazy_blockchains.go Outdated Show resolved Hide resolved

gustavogama-cll reviewed Dec 31, 2025

View reviewed changes

chain/lazy_blockchains.go Show resolved Hide resolved

gustavogama-cll reviewed Dec 31, 2025

View reviewed changes

graham-chainlink force-pushed the ggoh/lazy-blockchain branch 2 times, most recently from b71011b to ec50b62 Compare January 2, 2026 01:11

graham-chainlink marked this pull request as ready for review January 2, 2026 01:22

graham-chainlink requested a review from a team as a code owner January 2, 2026 01:22

Copilot AI review requested due to automatic review settings January 2, 2026 01:22

Copilot AI reviewed Jan 2, 2026

View reviewed changes

engine/cld/environment/environment_test.go Show resolved Hide resolved

engine/cld/environment/environment_test.go Show resolved Hide resolved

DimitriosNaikopoulos reviewed Jan 5, 2026

View reviewed changes

graham-chainlink force-pushed the ggoh/lazy-blockchain branch from ec50b62 to 3ee45b9 Compare January 8, 2026 12:50

DimitriosNaikopoulos previously approved these changes Jan 8, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings January 9, 2026 05:24

graham-chainlink dismissed DimitriosNaikopoulos’s stale review via 52648b1 January 9, 2026 05:24

graham-chainlink force-pushed the ggoh/lazy-blockchain branch from 3ee45b9 to 52648b1 Compare January 9, 2026 05:24

Copilot AI reviewed Jan 9, 2026

View reviewed changes

gustavogama-cll approved these changes Jan 9, 2026

View reviewed changes

DimitriosNaikopoulos approved these changes Jan 9, 2026

View reviewed changes

[Concept]: feat: introducing lazy chain loading #644

Are you sure you want to change the base?

[Concept]: feat: introducing lazy chain loading #644

Uh oh!

Conversation

graham-chainlink commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

cl-sonarqube-production bot commented Jan 9, 2026

Quality Gate failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

graham-chainlink commented Dec 29, 2025 •

edited

Loading

changeset-bot bot commented Dec 29, 2025 •

edited

Loading