Skip to content

Conversation

@stwiname
Copy link
Contributor

@stwiname stwiname commented Nov 11, 2025

Description

Fixes an issue where blocks would not be indexed if fetching failed because of caching blocks.

Because there is a need to look ahead to the next block, to get the previous block hash there is a cache to optimise network requests.
When this cache was hit, it would mutate the block numbers input, this is fine when all network requests work but if there is a failure then it would retry with the same block numbers, except it was mutated to exclude the blocks from the cache.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • I have tested locally
  • I have performed a self review of my changes
  • Updated any relevant documentation
  • Linked to any relevant issues
  • I have added tests relevant to my changes
  • Any dependent changes have been merged and published in downstream modules
  • My code is up to date with the base branch
  • I have updated relevant changelogs. We suggest using chan

Summary by CodeRabbit

  • Refactor
    • More concurrent, cache-aware block fetching for faster, more reliable data retrieval.
  • Tests
    • Reorganized suite with per-dataset isolation and lifecycle hooks for improved reliability.
  • Bug Fixes
    • Fixed cache mutation that could skip blocks.
  • Chores
    • Updated changelog with the cache-fix entry.

@coderabbitai
Copy link

coderabbitai bot commented Nov 11, 2025

Walkthrough

Reorganized Algorand tests into per-dataset lifecycles and switched test typing to AlgorandApiService; refactored Algorand API by removing two batch helper methods and implementing cache-aware concurrent block fetching with Promise.all; added a CHANGELOG entry for a cache mutation fix.

Changes

Cohort / File(s) Summary
Test Reorganization & Type + Import Updates
packages/node/src/algorand/algorand.spec.ts
Restructured tests into nested describes for testnet and mainnet with describe-scoped setup/teardown. Moved tests into appropriate describes (pagination, cache, stringify, grouped transactions). Replaced ApiService type with AlgorandApiService and added AlgorandBlock typing and non-null assertions in block/tx refs.
API Refactor: fetchBlocks Concurrency & Helpers Removed
packages/node/src/algorand/api.algorand.ts
Removed fetchBlocksArray() and fetchBlocksBatches(). Reimplemented fetchBlocks() to use Promise.all() with cache-aware per-block lookup (blockInCache(num) ?? this.getBlockByHeight(num)). getBlockByHeight() now returns the constructed block directly (no local assignment).
Changelog
packages/node/CHANGELOG.md
Added Unreleased Fixed entry describing a cache mutation issue that could cause blocks to be skipped (references issue #165).

Sequence Diagram(s)

sequenceDiagram
    participant Caller
    participant FetchBlocks as fetchBlocks()
    participant Cache as blockInCache
    participant API as getBlockByHeight()
    rect rgb(245, 255, 240)
    Note over Caller,API: Concurrent, cache-aware block fetching (Promise.all)
    end
    Caller->>FetchBlocks: fetchBlocks([nums])
    par For each num
        FetchBlocks->>Cache: blockInCache(num)?
        alt cache hit
            Cache-->>FetchBlocks: cached Block
        else cache miss
            Cache-->>FetchBlocks: null
            FetchBlocks->>API: getBlockByHeight(num)
            API-->>FetchBlocks: AlgorandBlock
        end
    end
    FetchBlocks-->>Caller: Promise.all([blocks])
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

  • Inspect per-describe app creation/closure to ensure no cross-test leakage.
  • Confirm removed helper methods have no remaining callers.
  • Review Promise.all concurrency with cache checks for race/order concerns.
  • Verify test typing changes (AlgorandApiService, AlgorandBlock) match runtime behavior.

Poem

🐰 I hop through tests with tidy paws,
Split the nets and mind the laws.
Batches folded, promises race,
Cache gives warmth to every trace.
Ledger hops on, steady pace.

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main bug fix: preventing cached blocks from mutating input data that breaks retries.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix-cache-mutation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 11, 2025

Coverage report

Caution

Test run failed

St.
Category Percentage Covered / Total
🔴 Statements 33.55% 893/2662
🔴 Branches 56.72% 76/134
🔴 Functions 28.79% 38/132
🔴 Lines 33.55% 893/2662

Test suite run failed

Failed tests: 3/21. Failed suites: 2/4.
  ● dictionary v1 › successfully validates metatada

    ApolloError: Failed to parse URL from /graphql

      at new ApolloError (node_modules/@apollo/client/errors/index.js:59:28)
      at node_modules/@apollo/client/core/QueryManager.js:823:71
      at both (node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
      at node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
      at Object.then (node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
      at Object.error (node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
      at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
      at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
      at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
      at node_modules/@apollo/client/utilities/observables/iteration.js:11:68
          at Array.forEach (<anonymous>)
      at iterateObserversSafely (node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
      at Object.error (node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
      at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
      at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
      at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
      at handleError (node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
      at node_modules/@apollo/client/link/http/createHttpLink.js:148:17

    Cause:
    TypeError: Failed to parse URL from /graphql



    Cause:
    TypeError: Invalid URL

      146 |       chainId,
      147 |     );
    > 148 |     await dictionary.init();
          |                      ^
      149 |     return dictionary;
      150 |   }
      151 |

      at node_modules/@apollo/client/link/http/createHttpLink.js:130:13
      at new Subscription (node_modules/zen-observable/lib/Observable.js:197:34)
      at Observable.subscribe (node_modules/zen-observable/lib/Observable.js:279:14)
      at Object.complete (node_modules/@apollo/client/utilities/observables/Concast.js:111:43)
      at Concast.Object.<anonymous>.Concast.start (node_modules/@apollo/client/utilities/observables/Concast.js:152:23)
      at new Concast (node_modules/@apollo/client/utilities/observables/Concast.js:137:19)
      at QueryManager.Object.<anonymous>.QueryManager.getObservableFromLink (node_modules/@apollo/client/core/QueryManager.js:744:37)
      at QueryManager.Object.<anonymous>.QueryManager.getResultsFromLink (node_modules/@apollo/client/core/QueryManager.js:786:30)
      at resultsFromLink (node_modules/@apollo/client/core/QueryManager.js:1114:26)
      at QueryManager.Object.<anonymous>.QueryManager.fetchQueryByPolicy (node_modules/@apollo/client/core/QueryManager.js:1177:52)
      at fromVariables (node_modules/@apollo/client/core/QueryManager.js:856:41)
      at QueryManager.Object.<anonymous>.QueryManager.fetchConcastWithInfo (node_modules/@apollo/client/core/QueryManager.js:898:35)
      at QueryManager.Object.<anonymous>.QueryManager.fetchQuery (node_modules/@apollo/client/core/QueryManager.js:383:21)
      at QueryManager.Object.<anonymous>.QueryManager.query (node_modules/@apollo/client/core/QueryManager.js:481:21)
      at ApolloClient.Object.<anonymous>.ApolloClient.query (node_modules/@apollo/client/core/ApolloClient.js:276:34)
      at AlgorandDictionaryV1.init (node_modules/@subql/node-core/src/indexer/dictionary/v1/dictionaryV1.ts:54:21)
      at AlgorandDictionaryV1.create (packages/node/src/indexer/dictionary/v1/algorandDictionaryV1.ts:148:22)
      at Object.<anonymous> (packages/node/src/indexer/dictionary/v1/algorandDictionaryV1.spec.ts:72:45)

  ● dictionary v1 › build correct dictionary entries and remove undefined fields in filter

    ApolloError: Failed to parse URL from /graphql

      at new ApolloError (node_modules/@apollo/client/errors/index.js:59:28)
      at node_modules/@apollo/client/core/QueryManager.js:823:71
      at both (node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
      at node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
      at Object.then (node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
      at Object.error (node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
      at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
      at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
      at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
      at node_modules/@apollo/client/utilities/observables/iteration.js:11:68
          at Array.forEach (<anonymous>)
      at iterateObserversSafely (node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
      at Object.error (node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
      at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
      at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
      at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
      at handleError (node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
      at node_modules/@apollo/client/link/http/createHttpLink.js:148:17

    Cause:
    TypeError: Failed to parse URL from /graphql



    Cause:
    TypeError: Invalid URL

      146 |       chainId,
      147 |     );
    > 148 |     await dictionary.init();
          |                      ^
      149 |     return dictionary;
      150 |   }
      151 |

      at node_modules/@apollo/client/link/http/createHttpLink.js:130:13
      at new Subscription (node_modules/zen-observable/lib/Observable.js:197:34)
      at Observable.subscribe (node_modules/zen-observable/lib/Observable.js:279:14)
      at Object.complete (node_modules/@apollo/client/utilities/observables/Concast.js:111:43)
      at Concast.Object.<anonymous>.Concast.start (node_modules/@apollo/client/utilities/observables/Concast.js:152:23)
      at new Concast (node_modules/@apollo/client/utilities/observables/Concast.js:137:19)
      at QueryManager.Object.<anonymous>.QueryManager.getObservableFromLink (node_modules/@apollo/client/core/QueryManager.js:744:37)
      at QueryManager.Object.<anonymous>.QueryManager.getResultsFromLink (node_modules/@apollo/client/core/QueryManager.js:786:30)
      at resultsFromLink (node_modules/@apollo/client/core/QueryManager.js:1114:26)
      at QueryManager.Object.<anonymous>.QueryManager.fetchQueryByPolicy (node_modules/@apollo/client/core/QueryManager.js:1177:52)
      at fromVariables (node_modules/@apollo/client/core/QueryManager.js:856:41)
      at QueryManager.Object.<anonymous>.QueryManager.fetchConcastWithInfo (node_modules/@apollo/client/core/QueryManager.js:898:35)
      at QueryManager.Object.<anonymous>.QueryManager.fetchQuery (node_modules/@apollo/client/core/QueryManager.js:383:21)
      at QueryManager.Object.<anonymous>.QueryManager.query (node_modules/@apollo/client/core/QueryManager.js:481:21)
      at ApolloClient.Object.<anonymous>.ApolloClient.query (node_modules/@apollo/client/core/ApolloClient.js:276:34)
      at AlgorandDictionaryV1.init (node_modules/@subql/node-core/src/indexer/dictionary/v1/dictionaryV1.ts:54:21)
      at AlgorandDictionaryV1.create (packages/node/src/indexer/dictionary/v1/algorandDictionaryV1.ts:148:22)
      at Object.<anonymous> (packages/node/src/indexer/dictionary/v1/algorandDictionaryV1.spec.ts:72:45)


  ● ApiService › waits on pending block to fetch hash

    Unable to get block hash for round 55467697

      234 |       }
      235 |     }
    > 236 |     throw new Error(`Unable to get block hash for round ${round}`);
          |           ^
      237 |   }
      238 | }
      239 |

      at AlgorandApi.getBlockHash (packages/node/src/algorand/api.algorand.ts:236:11)
      at packages/node/src/algorand/api.algorand.ts:189:22
          at async Promise.all (index 0)
      at AlgorandApi.fetchBlocks (packages/node/src/algorand/api.algorand.ts:187:29)
      at fetchLatestBlock (packages/node/src/indexer/api.service.spec.ts:87:8)

  ● ApiService › waits on pending block to fetch hash

    Unable to get block hash for round 55467697

      234 |       }
      235 |     }
    > 236 |     throw new Error(`Unable to get block hash for round ${round}`);
          |           ^
      237 |   }
      238 | }
      239 |

      at AlgorandApi.getBlockHash (packages/node/src/algorand/api.algorand.ts:236:11)
      at packages/node/src/algorand/api.algorand.ts:189:22
          at async Promise.all (index 0)
      at AlgorandApi.fetchBlocks (packages/node/src/algorand/api.algorand.ts:187:29)
      at fetchLatestBlock (packages/node/src/indexer/api.service.spec.ts:87:8)
      at Object.<anonymous> (packages/node/src/indexer/api.service.spec.ts:91:19)

Report generated by 🧪jest coverage report action from d5fb94e

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
packages/node/src/algorand/algorand.spec.ts (1)

148-167: Excellent test for the cache mutation bug fix!

This test directly validates the PR objective by ensuring the cache doesn't mutate the input blockNums array. The test strategy is sound:

  1. Trigger lookahead caching by fetching block N-1
  2. Fetch block N twice (once potentially from cache, once fresh)
  3. Assert the input array remains unchanged
  4. Verify consistent results across fetches

This prevents the bug where mutated block numbers would cause blocks to be skipped during retries.

Minor suggestion: The comment on line 157 could be clearer. Consider:

-      // The cache should have cleared the previous block, so this fetch is fresh
+      // Fetch the same block again (cache may have expired, triggering a fresh request)
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fa48f08 and d5fb94e.

📒 Files selected for processing (2)
  • packages/node/src/algorand/algorand.spec.ts (2 hunks)
  • packages/node/src/algorand/api.algorand.ts (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/node/src/algorand/api.algorand.ts
🧰 Additional context used
🧬 Code graph analysis (1)
packages/node/src/algorand/algorand.spec.ts (2)
packages/node/src/algorand/api.service.algorand.ts (1)
  • api (63-65)
packages/types/src/interfaces.ts (1)
  • AlgorandBlock (184-199)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Coverage report
🔇 Additional comments (7)
packages/node/src/algorand/algorand.spec.ts (7)

12-12: LGTM!

The AlgorandBlock import provides proper typing for the test data and improves type safety.


38-66: LGTM!

The helper function is well-structured with proper dependency injection and explicit typing for AlgorandApiService. The factory pattern correctly initializes the service with required dependencies.


121-134: LGTM!

The per-dataset lifecycle pattern provides better test isolation by creating fresh app instances for each test in the testnet suite.


137-145: LGTM!

The pagination test correctly validates that large blocks are properly handled with multiple paginated requests.


170-183: LGTM!

The per-dataset lifecycle pattern for mainnet tests provides proper isolation and resource management. Fetching the test block in beforeEach is efficient since all tests use the same block.


185-196: LGTM!

The circular reference test appropriately validates that blocks and transactions with circular references (transaction.block) can be safely stringified and parsed without errors.


198-204: LGTM!

The grouped transactions test correctly validates the getTransactionsByGroup functionality. The non-null assertions are appropriate for testing against a known, deterministic mainnet block.

@stwiname stwiname merged commit 46c6584 into main Nov 11, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants