fix: Pin duckdb <1.4.0 to avoid unhashable DuckDBPyType error

[Aaron ("AJ") Steers (aaronsteers)]

Summary

Pins the duckdb dependency to <1.4.0 to work around a breaking change in DuckDB 1.4.0 that causes TypeError: unhashable type: '_duckdb.typing.DuckDBPyType' when initializing DuckDB cache.

This is a known upstream issue in duckdb-engine: Mause/duckdb_engine#1338

The error occurs because DuckDB 1.4.0 made DuckDBPyType objects unhashable, which breaks SQLAlchemy's result processor caching mechanism. Downgrading to DuckDB 1.3.x avoids the issue.

Closes #809

Review & Testing Checklist for Human

• Verify the upstream duckdb-engine issue (#1338) is still open and unresolved
• Test DuckDB cache initialization on Python 3.12: cache = ab.get_default_cache() should not raise TypeError
• Consider if pinning to <1.4.0 blocks any critical duckdb features needed by users

Recommended test plan: Run python examples/run_faker.py on Python 3.12 to verify the DuckDB cache works correctly.

Notes

• The TODO comment includes links to both the upstream issue and this PyAirbyte issue for future reference
• Once duckdb-engine releases a fix, the upper bound can be relaxed
• Verified that duckdb 1.3.2 has prebuilt wheels for Python 3.12

Requested by: AJ Steers (Aaron ("AJ") Steers (@aaronsteers))
Link to Devin run: https://app.devin.ai/sessions/5094f54055bd4fc68a357aa705357fce

Summary by CodeRabbit

• Chores
  • Updated dependency version constraints for improved stability and compatibility.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[devin-ai-integration]

Original prompt from AJ Steers

Received message in Slack channel #ask-devin-ai:

@Devin - Triage the latest 10 issues in the PyAirbyte repo that are from external community members / contributors. Don't comment on them directly but look into them and prepare for me a set of suggestions on how to disposition each.
Thread URL: https://airbytehq-team.slack.com/archives/C08BHPUMEPJ/p1765247038245949

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1765249536-fix-duckdb-compatibility' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1765249536-fix-duckdb-compatibility'

Helpful Resources

• PyAirbyte Documentation
• API Reference

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /fix-pr - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test-pr - Runs tests with the updated PyAirbyte

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

[coderabbitai]

📝 Walkthrough

Walkthrough

The duckdb dependency version constraint in pyproject.toml has been narrowed from the caret range ^1.1.0 to an explicit bounded range >=1.1.0,<1.4.0 with explanatory comments, addressing compatibility issues with duckdb versions 1.4.0 and later.

Changes

Cohort / File(s)	Summary
Dependency version constraint pyproject.toml	Updated duckdb dependency from ^1.1.0 to >=1.1.0,<1.4.0 with explanatory comments to prevent unhashable type errors in newer duckdb releases

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

• Single-line dependency update with explanatory comments
• Clear rationale tied to issue #809 (DuckDBType error)
• No code logic changes or architectural implications
• Straightforward version constraint modification

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: pinning duckdb to <1.4.0 to fix the unhashable DuckDBPyType error, which matches the single-line dependency constraint update in pyproject.toml.
Linked Issues check	✅ Passed	The PR successfully addresses issue #809 by pinning duckdb <1.4.0, which resolves the TypeError: unhashable type error that occurred during DuckDB cache initialization when running examples like run_faker.py.
Out of Scope Changes check	✅ Passed	All changes are narrowly scoped to pinning the duckdb dependency version range; no extraneous modifications to unrelated code, tests, or documentation are present.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch devin/1765249536-fix-duckdb-compatibility

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1b48476 and 9ca8d1b.

⛔ Files ignored due to path filters (1)

• poetry.lock is excluded by !**/*.lock

📒 Files selected for processing (1)

• pyproject.toml (1 hunks)

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2024-10-10T16:17:57.989Z

Learnt from: aaronsteers
Repo: airbytehq/PyAirbyte PR: 417
File: airbyte/cli.py:503-504
Timestamp: 2024-10-10T16:17:57.989Z
Learning: In the PyAirbyte project, support for Python versions earlier than 3.10 is not necessary, as the project requires Python 3.10 or newer.

Applied to files:

• pyproject.toml

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)

• GitHub Check: Pytest (All, Python 3.10, Ubuntu)
• GitHub Check: Pytest (All, Python 3.11, Windows)
• GitHub Check: Pytest (All, Python 3.11, Ubuntu)
• GitHub Check: Pytest (All, Python 3.10, Windows)
• GitHub Check: Pytest (Fast)
• GitHub Check: Pytest (No Creds)

🔇 Additional comments (1)

pyproject.toml (1)

24-27: Good approach—duckdb-engine compatibility confirmed.

The pinned version constraint correctly addresses the upstream issue while maintaining a clear path to removal once duckdb-engine is patched. The TODO and issue links provide helpful context for future maintainers.

Verified: duckdb-engine ^0.13.2 requires duckdb>=0.5.0 with no upper bound, so the new constraint >=1.1.0,<1.4.0 is fully compatible. The narrower upper bound is intentional and safe.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

PyTest Results (Fast Tests Only, No Creds)

348 tests  ±0   348 ✅ ±0   6m 2s ⏱️ +8s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 81a32f2. ± Comparison against base commit 725e262.

♻️ This comment has been updated with latest results.

[github-actions]

PyTest Results (Full)

416 tests  ±0   400 ✅ ±0   25m 8s ⏱️ -7s
  1 suites ±0    16 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 9ca8d1b. ± Comparison against base commit 1b48476.

[Aaron ("AJ") Steers (aaronsteers)]

Devin, suggest if this PR is still needed.

[devin-ai-integration]

Recommendation: This PR is no longer needed and should be closed.

I investigated the upstream issue and current state:

Upstream fix status:

• The bug was fixed in duckdb 1.4.1 via duckdb/duckdb-python#61
• duckdb 1.4.1, 1.4.2, and 1.4.3 are all available on PyPI with the fix

Current PyAirbyte main constraints:

• duckdb = "^1.1.0" allows 1.4.1+ (the fixed versions)
• Most users will get duckdb 1.4.3 (latest) which includes the fix

Why this PR would now be harmful:

• Merging <1.4.0 would prevent users from getting the fixed versions (1.4.1+)
• It would force users back to 1.3.x, which is now the wrong direction

The upstream issue Mause/duckdb_engine#1338 is still technically open, but only because the duckdb-engine maintainer hasn't closed it yet - the underlying duckdb bug is resolved.

I'd recommend closing this PR. If you want extra safety, a separate PR could add >=1.4.1 as a lower bound to explicitly exclude only the broken 1.4.0, but that's probably not necessary since pip/poetry will resolve to 1.4.3 anyway.

[Aaron ("AJ") Steers (aaronsteers)]

Closing. Issue was resolved in DuckDB 1.4.1