Skip to content

feat(source-hubspot): add list_memberships stream using V3 Lists API#75545

Open
devin-ai-integration[bot] wants to merge 4 commits intomasterfrom
devin/1774647328-add-list-memberships-stream
Open

feat(source-hubspot): add list_memberships stream using V3 Lists API#75545
devin-ai-integration[bot] wants to merge 4 commits intomasterfrom
devin/1774647328-add-list-memberships-stream

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot commented Mar 27, 2026

What

Adds a new list_memberships stream to source-hubspot, replacing the contacts_list_memberships stream that was removed in v5.0.0 (PR #59673) due to HubSpot's V1 Contacts API deprecation. The new stream uses HubSpot's V3 Lists API endpoint (GET /crm/v3/lists/{listId}/memberships).

Resolves https://github.com/airbytehq/oncall/issues/11812:

Related: #75544

How

  • Added list_memberships_stream as a substream of the existing contact_lists_stream using SubstreamPartitionRouter
  • The parent stream provides listId values via partition routing; the child stream hits GET /crm/v3/lists/{listId}/memberships for each list
  • Cursor-based pagination using the after parameter (reads paging.next.after from response)
  • listId is injected into each record via an AddFields transformation to form a composite primary key [listId, recordId]
  • Schema includes listId, recordId, and membershipTimestamp
  • Stream is full-refresh only (no incremental support)
  • Version bumped: manifest 6.52.0 → 6.53.0, dockerImageTag 6.3.3 → 6.4.0

Review guide

  1. airbyte-integrations/connectors/source-hubspot/manifest.yaml — New stream definition (lines ~1173-1224), stream registration (line ~2029), and inline schema (lines ~2547-2566). Key areas:
    • The SubstreamPartitionRouter wiring to contact_lists_stream
    • The paginator configuration (cursor-based with after)
    • The AddFields transformation that injects listId
  2. airbyte-integrations/connectors/source-hubspot/unit_tests/test_streams.py — Two new tests: basic multi-list read and pagination handling
  3. airbyte-integrations/connectors/source-hubspot/unit_tests/test_source.py — Updated stream count assertions (34→35, 35→36) to account for the new stream
  4. airbyte-integrations/connectors/source-hubspot/metadata.yaml — dockerImageTag bump
  5. docs/integrations/sources/hubspot.md — Changelog entry for 6.4.0

Items worth extra scrutiny:

  • The listId schema type is string, but the AddFields Jinja template ({{ stream_partition['list_id'] }}) may inject it as an integer depending on the parent record type. The unit tests use str() casts to work around this. Reviewer: please verify this won't cause schema validation issues in production. If it does, a | string Jinja filter should be added to the template.
  • No stop_condition on the paginator — it relies on cursor_value evaluating to None when paging.next is absent. This follows the existing cursor_paginator pattern in the manifest but is worth confirming.
  • Only unit-tested with mocked responses; no live API validation was performed. Consider running live/regression tests before promoting out of draft.

User Impact

Users gain a new list_memberships stream that provides list membership data (which record IDs belong to which lists) via HubSpot's V3 Lists API. This partially restores functionality lost when contacts_list_memberships was removed in v5.0.0. The new stream requires the crm.lists.read OAuth scope (same as the existing contact_lists stream).

This is a purely additive change — no existing streams or schemas are modified.

Can this PR be safely reverted and rolled back?

  • YES 💚

Link to Devin session: https://app.devin.ai/sessions/a0d512382922491b824193ec1ca56253

devin-ai-integration bot and others added 2 commits March 27, 2026 21:39
Add a new list_memberships stream as a substream of contact_lists_stream
using HubSpot's V3 Lists API (GET /crm/v3/lists/{listId}/memberships).

- Implements cursor-based pagination with after parameter
- Uses SubstreamPartitionRouter to iterate over parent contact lists
- Adds listId via AddFields transformation for composite primary key
- Includes unit tests for basic read and pagination
- Bumps version to 6.53.0 (manifest) / 6.4.0 (docker)

Co-Authored-By: bot_apk <apk@cognition.ai>
Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Co-Authored-By: bot_apk <apk@cognition.ai>
@github-actions
Copy link
Copy Markdown
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • 🛠️ Quick Fixes
    • /format-fix - Fixes most formatting issues.
    • /bump-version - Bumps connector versions, scraping changelog description from the PR title.
  • ❇️ AI Testing and Review (internal link: AI-SDLC Docs):
    • /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
    • /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
    • /ai-review - AI-powered PR review for connector safety and quality gates.
  • 🚀 Connector Releases:
    • /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
    • /bump-progressive-rollout-version - Bumps connector version with an RC suffix (2.16.10-rc.1) for progressive rollouts (enableProgressiveRollout: true).
      • Example: /bump-progressive-rollout-version changelog="Add new feature for progressive rollout"
  • ☕️ JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
  • 🐍 Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
  • ⚙️ Admin commands:
    • /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
      Example: /force-merge reason="CI is flaky, tests pass locally"
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 27, 2026

Deploy preview for airbyte-docs ready!

✅ Preview
https://airbyte-docs-69p3x95z6-airbyte-growth.vercel.app

Built with commit 6198f97.
This pull request is being automatically deployed with vercel-action

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 27, 2026

source-hubspot Connector Test Results

163 tests   155 ✅  11m 34s ⏱️
  2 suites    8 💤
  2 files      0 ❌

Results for commit 6198f97.

♻️ This comment has been updated with latest results.

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

↪️ Triggering /ai-prove-fix per Hands-Free AI Triage Project triage next step.

Reason: Draft PR with CI all green (36/36), adds new list_memberships stream to source-hubspot per oncall feature request. Advancing through pipeline.
https://github.com/airbytehq/oncall/issues/11812

Devin session

@octavia-bot
Copy link
Copy Markdown
Contributor

octavia-bot bot commented Mar 28, 2026

🔍 AI Prove Fix session starting... Running readiness checks and testing against customer connections. View playbook

Devin AI session created successfully!

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

devin-ai-integration bot commented Mar 28, 2026

Fix Validation Evidence

Outcome: Fix/Feature Proven Successfully

Evidence Summary

Regression tests comparing the PR version against baseline (6.3.3) passed all four commands (spec, check, discover, read). The new list_memberships stream successfully returned 3,970 membership records from the V3 Lists API. All 36 existing streams returned identical record counts between target and control, confirming zero regression. The target version correctly discovers 37 streams (36 existing + 1 new list_memberships).

Next Steps
  1. This PR appears ready for review and merge.
  2. For broader validation before release, consider running /ai-canary-prerelease to test on additional connections.
  3. The daily_hands_free_triage automation will monitor the release rollout after merge.

Connector & PR Details

Connector: source-hubspot
PR: #75545
Pre-release Version Tested: 6.4.0-preview.6198f97
Detailed Results: https://github.com/airbytehq/oncall/issues/11812#issuecomment-4147971395

Evidence Plan

Proving Criteria

  1. New list_memberships stream successfully returns membership records from V3 Lists API
  2. Records contain correct listId injection via AddFields transformation
  3. Cursor-based pagination works correctly (using after parameter)
  4. Composite primary key [listId, recordId] is properly set
  5. Existing streams show no regression (same behavior as baseline)

Disproving Criteria

  1. list_memberships stream fails to read or returns errors
  2. Empty results when HubSpot account has known list memberships
  3. listId injection is missing or incorrect
  4. Pagination fails or produces duplicate records
  5. Existing streams break or show behavioral changes

Cases Attempted

  1. Regression Tests (PASSED) — Comparison test via run_regression_tests
Pre-flight Checks
  • Viability: Fix adds a new list_memberships stream using V3 Lists API — addresses oncall issue requesting restoration of list membership functionality lost in v5.0.0
  • Safety: No malicious code, standard declarative manifest patterns, no suspicious external calls
  • Breaking Change: No breaking changes detected — purely additive (new stream only, no modifications to existing streams, no schema/spec/state changes)
  • Reversibility: Can be safely downgraded/reverted — minor version bump (6.3.3 → 6.4.0), no state/config incompatibilities
Detailed Evidence Log

2026-03-28T11:50Z — Initial status comment posted to PR
2026-03-28T11:52Z — Pre-flight checks completed (viability, safety, breaking change, reversibility all pass)
2026-03-28T11:55Z — Pre-release 6.4.0-preview.6198f97 published successfully
2026-03-28T11:58Z — Regression tests triggered (run ID: 23684716659)
2026-03-28T12:03Z — Evidence plan posted to PR and oncall issue
2026-03-28T12:05Z — Live testing approval requested via Slack escalation
2026-03-28T12:11Z — Regression tests completed: ALL PASSED

Regression Test Results:

Command Target (PR) Control (6.3.3) Result
spec success success PASS
check success success PASS
discover 37 streams 36 streams PASS (+1 new stream)
read 376,875 records 372,905 records PASS (+3,970 new records)

Note: Connection IDs and detailed logs are recorded in the linked private issue.


Devin session

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 28, 2026

Pre-release Connector Publish Started

Publishing pre-release build for connector source-hubspot.
PR: #75545

Pre-release versions will be tagged as {version}-preview.6198f97
and are available for version pinning via the scoped_configuration API.

View workflow run
Pre-release Publish: SUCCESS

Docker image (pre-release):
airbyte/source-hubspot:6.4.0-preview.6198f97

Docker Hub: https://hub.docker.com/layers/airbyte/source-hubspot/6.4.0-preview.6198f97

Registry JSON:

@octavia-bot octavia-bot bot marked this pull request as ready for review March 29, 2026 11:43
@octavia-bot
Copy link
Copy Markdown
Contributor

octavia-bot bot commented Mar 29, 2026

AI PR Review starting...

Reviewing PR for connector safety and quality.
View playbook

Devin AI session created successfully!

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

devin-ai-integration bot commented Mar 29, 2026

AI PR Review Report

Review Action: NO ACTION (NOT ELIGIBLE)

Gate Status
PR Hygiene PASS
Code Hygiene PASS
Code Security PASS
Per-Record Performance PASS
Breaking Dependencies PASS
Backwards Compatibility PASS
Forwards Compatibility PASS
Behavioral Changes PASS
Out-of-Scope Changes SKIP
CI Checks PASS
Live / E2E Tests PASS

📋 PR Details & Eligibility

Connector & PR Info

Connector(s): source-hubspot
PR: #75545
HEAD SHA: 6198f9731f90ac40e18002718a90865de18636f4
Session: https://app.devin.ai/sessions/c1ffb65bb2f34026810b61211afb2ff1

Auto-Approve Eligibility

Eligible: No
Category: not-eligible
Reason: PR adds a new list_memberships stream with functional code changes (new declarative stream definition, unit tests, schema). This is a feature addition, not a docs-only, additive-spec, patch/minor-deps, or comment/whitespace-only change.

Review Action Details

NO ACTION (NOT ELIGIBLE) — All gates pass but the PR is not eligible for auto-approval because it contains functional code changes (new stream). No PR review submitted. Human review required.

Note: This bot can approve PRs when all gates pass AND the PR is eligible for auto-approval (docs-only, additive spec changes, patch/minor dependency bumps, or comment/whitespace-only changes). PRs with other types of changes require human review even if all gates pass.

🔍 Gate Evaluation Details

Gate-by-Gate Analysis

Gate Status Enforced? Details
PR Hygiene PASS Yes Description present with What/How/Review Guide/User Impact sections. Changelog updated in docs/integrations/sources/hubspot.md. Semantic PR title correct.
Code Hygiene PASS WARNING Two new unit tests cover core functionality and pagination. Stream count assertions updated in test_source.py. No code smells detected.
Code Security PASS Yes No auth/credential patterns in diff. Stream uses existing base_requester ref. No hardcoded tokens or secrets.
Per-Record Performance PASS WARNING Standard declarative stream with page_size: 250. No custom Python code in record processing. SubstreamPartitionRouter is the standard CDK pattern.
Breaking Dependencies PASS WARNING Manifest version 6.52.0 → 6.53.0 (minor). Docker image 6.3.3 → 6.4.0 (minor — feature addition). No dependency file changes.
Backwards Compatibility PASS Blocks Auto-Approve Purely additive change — new stream only. No existing streams, schemas, specs, or state formats modified.
Forwards Compatibility PASS Blocks Auto-Approve New stream is full-refresh only with no state. Rollback safe — removing the stream leaves no orphaned state. No config migration needed.
Behavioral Changes PASS Blocks Auto-Approve No changes to rate limits, retries, timeouts, or error handling. No modifications to existing stream behavior.
Out-of-Scope Changes SKIP Skip All 5 changed files are within airbyte-integrations/connectors/source-hubspot/ or docs/integrations/sources/hubspot.md. No platform/CDK/CI changes.
CI Checks PASS Yes All core checks green: Connector CI Checks Summary ✓, Test source-hubspot Connector ✓, Lint source-hubspot Connector ✓, Format Check ✓, Enforce PR structure ✓, Check Changelog Updated ✓. 163 tests: 155 passed, 8 skipped, 0 failed.
Live / E2E Tests PASS Yes Pre-release 6.4.0-preview.6198f97 published and regression-tested. Regression tests passed all 4 commands (spec, check, discover, read). New list_memberships stream returned 3,970 records. All 36 existing streams showed identical record counts vs baseline (6.3.3).
📚 Evidence Consulted

Evidence

  • Changed files: 5 files
    • airbyte-integrations/connectors/source-hubspot/manifest.yaml — New stream definition, registration, schema
    • airbyte-integrations/connectors/source-hubspot/metadata.yaml — dockerImageTag 6.3.3 → 6.4.0
    • airbyte-integrations/connectors/source-hubspot/unit_tests/test_source.py — Stream count assertions 34→35, 35→36
    • airbyte-integrations/connectors/source-hubspot/unit_tests/test_streams.py — 2 new tests (basic read + pagination)
    • docs/integrations/sources/hubspot.md — Changelog entry for 6.4.0
  • CI checks: 38 passed, 1 pending (re-triggered Generate Connector Matrix — non-core), 11 skipped. All required checks passed.
  • PR labels: connectors/source/hubspot
  • PR description: Present with detailed What/How/Review Guide/User Impact sections
  • Existing bot reviews: None
  • Validation evidence: Pre-release published (6.4.0-preview.6198f97), regression tests passed via /ai-prove-fix

Devin session

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant