Skip to content

Conversation

@eytan-avisror
Copy link
Contributor

@eytan-avisror eytan-avisror commented Nov 20, 2025

What

Fix depth counter bug in BlocksRetriever that caused current_block_depth to accumulate incorrectly across sibling blocks with children.

The BlocksRetriever.read_records() method increments current_block_depth when processing blocks with children, but was never decrementing it after the recursive call completes. This causes the depth counter to keep growing instead of returning to the parent level.

Impact of the bug:

source INFO Reached max block depth limit. Exiting.
  • When processing multiple sibling blocks that have children, the depth counter would incorrectly accumulate
  • Could prematurely hit the MAX_BLOCK_DEPTH limit (30) and stop retrieving valid nested blocks
  • Example: Processing 30 sibling blocks with children would incorrectly reach depth 30, even though the actual nesting depth is only 1

How

Added self.current_block_depth -= 1 after the recursive yield from self.read_records() call in components.py line 88 to properly restore the depth counter when returning from child block processing.

Before (Buggy behavior):

Review guide

User Impact

Unable to properly ingest from notion, many blocks missing.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@CLAassistant
Copy link

CLAassistant commented Nov 20, 2025

CLA assistant check
All committers have signed the CLA.

@github-actions
Copy link
Contributor

👋 Welcome to Airbyte!

Thank you for your contribution from eytan-avisror/airbyte! We're excited to have you in the Airbyte community.

Helpful Resources

PR Slash Commands

As needed or by request, Airbyte Maintainers can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
  • /run-connector-tests - Runs connector tests.
  • /run-cat-tests - Runs CAT tests.
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).

If you have any questions, feel free to ask in the PR comments or join our Slack community.

Tips for Working with CI

  1. Pre-Release Checks. Please pay attention to these, as they contain standard checks on the metadata.yaml file, docs requirements, etc. If you need help resolving a pre-release check, please ask a maintainer.
    • Note: If you are creating a new connector, please be sure to replace the default logo.svg file with a suitable icon.
  2. Connector CI Tests. Some failures here may be expected if your tests require credentials. Please review these results to ensure (1) unit tests are passing, if applicable, and (2) integration tests pass to the degree possible and expected.
  3. (Optional.) BYO Connector Credentials for tests in your fork. You can optionally set up your fork with BYO credentials for your connector. This can significantly speed up your review, ensuring your changes are fully tested before the maintainers begin their review.

📝 Edit this welcome message.

@agarctfi agarctfi self-requested a review November 21, 2025 16:29
@agarctfi agarctfi moved this from New PRs to Shield Team: In Review in 🧑‍🏭 Community Pull Requests Nov 21, 2025
@agarctfi
Copy link
Contributor

@eytan-avisror Thanks for your contribution! Some of the tests were unable to run since the branch doesn't exist in the airbytehq repo. I've copied over the PR here so we can run all the tests.

Overall, the PR looks good! I just changed the unit test name and changelog date for this upcoming Monday, which is when we plan to ship this.

I'll leave this PR open till we get the other one merged and follow up if anything comes up! :)

@eytan-avisror
Copy link
Contributor Author

@eytan-avisror Thanks for your contribution! Some of the tests were unable to run since the branch doesn't exist in the airbytehq repo. I've copied over the PR here so we can run all the tests.

Overall, the PR looks good! I just changed the unit test name and changelog date for this upcoming Monday, which is when we plan to ship this.

I'll leave this PR open till we get the other one merged and follow up if anything comes up! :)

Sounds good, Thanks @agarctfi

@DanyloGL
Copy link
Collaborator

DanyloGL commented Nov 24, 2025

/run-connector-tests

Connector CI Tests Started

These tests will leverage Airbyte's integration test credentials.

Check job output.
✅ Connector CI Tests job completed successfully. See logs for details.

Copy link
Contributor

@agarctfi agarctfi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We tested this locally & on cloud and everything looks good!

@DanyloGL was able to get the tests running on this PR so we're gonna use it for the merge.

@github-project-automation github-project-automation bot moved this from Shield Team: In Review to Ready to Ship in 🧑‍🏭 Community Pull Requests Nov 24, 2025
@agarctfi agarctfi merged commit e7ce8a0 into airbytehq:master Nov 24, 2025
29 of 30 checks passed
@github-project-automation github-project-automation bot moved this from Ready to Ship to Done in 🧑‍🏭 Community Pull Requests Nov 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

5 participants