APP-5243: Migrate to `httpx` (goodbye `requests` 👋 ) #679

Aryamanz29 · 2025-07-14T12:13:29Z

Removed requirements files
Updated GitHub workflows for Docker image builds to use uv sync (without dev dependencies)
Breaking change: Dropped support for Python 3.8 (httpx-retries requires Python ≥ 3.9)

Other `v8` Release Tasks:

Add Python version in headers
Add Prefetch paginator
Reduce test build time & coverage report generation
(explore label-based test runs for async tests)
Fix SDK Sphinx docs build
Fix conda packages publish GitHub Action
Complete final SDK benchmarking
Create async dev docs (TBD)

✨ Description

Briefly explain the purpose of this PR and what it covers. Mention any related issues or Jira tickets.

Jira link:

🧩 Type of change

Select all that apply:

🚀 New feature (non-breaking change that adds functionality)
🐛 Bug fix (non-breaking change that fixes an issue) — please include tests! Refer testing-toolkit 🧪
🔄 Refactor (code change that neither fixes a bug nor adds a feature)
🧹 Maintenance (chores, cleanup, minor improvements)
💥 Breaking change (fix or feature that may break existing functionality)
📦 Dependency upgrade/downgrade
📚 Documentation updates

✅ How has this been tested? (e.g. screenshots, logs, workflow links)

Describe how the change was tested. Include:

Steps to reproduce
Any relevant screenshots, logs, or links to successful workflow runs
Details on environment/setup if applicable

📋 Checklist

My code follows the project’s style guidelines
I’ve performed a self-review of my code
I’ve added comments in tricky or complex areas
I’ve updated the documentation as needed
There are no new warnings from my changes
I’ve added tests to cover my changes
All new and existing tests pass locally

Copilot

Pull Request Overview

Migrates the SDK from requests to httpx (using httpx-retries) and drops Python 3.8 support, updating tests, dependencies, and CI workflows accordingly.

Refactored AtlanClient to use httpx.Client with RetryTransport and new timeout/streaming logic
Updated unit and integration tests to mock and assert against httpx behavior
Removed requests from dependencies, bumped requires-python to ≥ 3.9, and adjusted GitHub Actions to no longer test on 3.8

Reviewed Changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/unit/test_query_client.py	Adapted mocks to support `httpx` streaming and reading
tests/unit/test_file_client.py	Updated mocks and assertions for `httpx` file streaming
tests/unit/test_base_vcr_yaml.py	Replaced `requests` calls with `httpx` equivalents
tests/unit/test_base_vcr_json.py	Replaced `requests` calls with `httpx` equivalents
tests/integration/test_index_search.py	Swapped `requests` exceptions and imports for `httpx`
requirements-dev.txt	Removed legacy dev requirements file
pyproject.toml	Dropped Python 3.8, removed `requests`, added `httpx`
pyatlan/client/common.py	Switched retry import from `urllib3` to `httpx-retries`
pyatlan/client/atlan.py	Refactored client setup, API calls, timeout and streams
pyatlan/client/asset.py	Wrapped retry errors in `_wait_till_deleted`
.github/workflows/pyatlan-test-cron.yaml	Updated Python matrix to ≥ 3.9
.github/workflows/pyatlan-publish.yaml	Changed `uv sync` to omit dev dependencies
.github/workflows/pyatlan-pr.yaml	Dropped 3.8 from PR and vulnerability-scan inputs
.github/workflows/build_and_upload_conda_packages.yaml	Removed 3.8 from conda build matrix

Comments suppressed due to low confidence (1)

.github/workflows/pyatlan-pr.yaml:40

[nitpick] Changing the vulnerability-scan inputs to scan the entire workspace may include unintended files. Specify the exact dependency files (e.g., requirements lock files) to limit the scan surface and reduce noise.

            inputs: .

tests/unit/test_query_client.py

tests/unit/test_file_client.py

pyatlan/client/atlan.py

- Removed requirements files - Updated GitHub workflows for Docker image builds to use `uv sync` (without dev dependencies) - Breaking change: Dropped support for Python 3.8 (httpx-retries requires Python ≥ 3.9)

…≥ 3.9)

…ations

…replacing_cm)

- Fixed local imports due to recent refactoring.

…leaned up

… client

…d `AysncApiCaller` for all aio clients

… `workflow_client` tests

…_tag` tests

cursor · 2025-08-12T12:19:28Z

pyatlan/cache/common/custom_metadata_cache.py

+        attr_def = attr_cache_by_id.get(attr_id)
+        if not attr_def:
+            raise ErrorCode.CM_ATTR_NOT_FOUND_BY_ID.exception_with_parameters(attr_id)
+


Bug: Incorrect Return Type Annotations in Metadata Cache

The static methods get_attr_id_for_name and get_attribute_def in CustomMetadataCacheCommon have incorrect return type annotations. Both are declared to return a tuple[..., bool] but only return the first element of the tuple, leading to type checking errors and potential runtime issues.

Aryamanz29 requested a review from Copilot July 14, 2025 12:13

Aryamanz29 self-assigned this Jul 14, 2025

Aryamanz29 added feature New feature or request dependencies Pull requests that update a dependency file change Pyatlan change pull request cleanup breaking-change refactor labels Jul 14, 2025

This comment was marked as outdated.

Sign in to view

Aryamanz29 force-pushed the APP-5243 branch from 40e5acf to 1a0f464 Compare July 14, 2025 12:23

This comment was marked as outdated.

Sign in to view

Aryamanz29 force-pushed the APP-5243 branch from 2cefee5 to c348cab Compare July 15, 2025 08:58

This comment was marked as outdated.

Sign in to view

Aryamanz29 force-pushed the APP-5243 branch from c348cab to 46a97a7 Compare July 15, 2025 09:05

This comment was marked as outdated.

Sign in to view

Aryamanz29 force-pushed the APP-5243 branch 2 times, most recently from 4a1f567 to 82d2354 Compare July 15, 2025 09:21

This comment was marked as outdated.

Sign in to view

Aryamanz29 force-pushed the APP-5243 branch 2 times, most recently from 652804a to 3ba2a99 Compare July 15, 2025 11:14

This comment was marked as outdated.

Sign in to view

Aryamanz29 requested a review from Copilot July 15, 2025 11:17

Copilot AI reviewed Jul 15, 2025

View reviewed changes

tests/unit/test_query_client.py Show resolved Hide resolved

tests/unit/test_file_client.py Show resolved Hide resolved

pyatlan/client/atlan.py Show resolved Hide resolved

Aryamanz29 force-pushed the APP-5243 branch from 3ba2a99 to c7c2ff6 Compare July 15, 2025 11:33

This comment was marked as outdated.

Sign in to view

Aryamanz29 force-pushed the APP-5243 branch from c7c2ff6 to 8ac1df8 Compare August 1, 2025 11:25

This comment was marked as outdated.

Sign in to view

Aryamanz29 force-pushed the APP-5243 branch from b1f17a8 to 4ab3dea Compare August 4, 2025 21:37

Aryamanz29 force-pushed the APP-5243 branch from 4ab3dea to 278b626 Compare August 4, 2025 21:49

This comment was marked as outdated.

Sign in to view

Aryamanz29 added 9 commits August 7, 2025 17:41

APP-5243: Migrate to httpx (goodbye requests)

4f4e55f

- Removed requirements files - Updated GitHub workflows for Docker image builds to use `uv sync` (without dev dependencies) - Breaking change: Dropped support for Python 3.8 (httpx-retries requires Python ≥ 3.9)

[ci] Dropped support for Python 3.8 (httpx-retries requires Python …

7a5148f

…≥ 3.9)

[changes] Fixed AtlanClient event stream handling

98b51c8

[refactor] Migrated AssetClient/AsyncAssetClient to use shared oper…

5a09533

…ations

[ci:temp] Skip qa-checks job

14cb505

[refactor] Migrated other clients to use common operations

dfc0ac3

[tests/fix] Fixed failing cm test due to incorrect refactoring (save_…

22655d6

…replacing_cm)

[refactor] Migrated caches to use common operations

ce98b05

- Fixed local imports due to recent refactoring.

[refactor] Rebased, refactored (dq_template_config_cache) and did c…

bcb5fbe

…leaned up

Aryamanz29 force-pushed the APP-5243 branch from 9b635e8 to bcb5fbe Compare August 7, 2025 12:24

[refactor] Refactored translators, retranslators, async batch and AIO…

b091fe4

… client

This comment was marked as outdated.

Sign in to view

Aryamanz29 added 6 commits August 8, 2025 15:09

[deps] Added pytest-asyncio dependency for async testing

27e7266

[tests] Added async connection_cache tests

1a82ee6

[tests] Added async source_tag_cache tests

d4f866f

[refactor] Use @validate_arguments decorator in aio clients and use…

3bbbea0

…d `AysncApiCaller` for all aio clients

[tests] Added asyncio configuration for aio tests

00ecd03

[tests] Added async client, credential_client, file_client, and…

139d097

… `workflow_client` tests

This comment was marked as outdated.

Sign in to view

[tests] Added async cm, audit, query, and sso, task, `atlan…

5494461

…_tag` tests

cursor bot reviewed Aug 12, 2025

View reviewed changes

APP-5243: Migrate to httpx (goodbye requests 👋 ) #679

Are you sure you want to change the base?

APP-5243: Migrate to httpx (goodbye requests 👋 ) #679

Uh oh!

Conversation

Aryamanz29 commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Other v8 Release Tasks:

✨ Description

🧩 Type of change

✅ How has this been tested? (e.g. screenshots, logs, workflow links)

📋 Checklist

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

cursor bot Aug 12, 2025

Choose a reason for hiding this comment

Bug: Incorrect Return Type Annotations in Metadata Cache

Uh oh!

Uh oh!

APP-5243: Migrate to `httpx` (goodbye `requests` 👋 ) #679

APP-5243: Migrate to `httpx` (goodbye `requests` 👋 ) #679

Aryamanz29 commented Jul 14, 2025 •

edited

Loading

Other `v8` Release Tasks: