Skip to content
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
4f4e55f
APP-5243: Migrate to `httpx` (goodbye `requests`)
Aryamanz29 Jul 14, 2025
7a5148f
[ci] Dropped support for `Python 3.8` (httpx-retries requires Python …
Aryamanz29 Jul 14, 2025
98b51c8
[changes] Fixed `AtlanClient` event stream handling
Aryamanz29 Jul 15, 2025
5a09533
[refactor] Migrated `AssetClient/AsyncAssetClient` to use shared oper…
Aryamanz29 Aug 1, 2025
14cb505
[ci:temp] Skip `qa-checks` job
Aryamanz29 Aug 1, 2025
dfc0ac3
[refactor] Migrated other clients to use `common` operations
Aryamanz29 Aug 4, 2025
22655d6
[tests/fix] Fixed failing cm test due to incorrect refactoring (save_…
Aryamanz29 Aug 5, 2025
ce98b05
[refactor] Migrated `caches` to use `common` operations
Aryamanz29 Aug 6, 2025
bcb5fbe
[refactor] Rebased, refactored (`dq_template_config_cache`) and did c…
Aryamanz29 Aug 7, 2025
b091fe4
[refactor] Refactored translators, retranslators, async batch and AIO…
Aryamanz29 Aug 7, 2025
27e7266
[deps] Added `pytest-asyncio` dependency for async testing
Aryamanz29 Aug 8, 2025
1a82ee6
[tests] Added async `connection_cache` tests
Aryamanz29 Aug 8, 2025
d4f866f
[tests] Added async `source_tag_cache` tests
Aryamanz29 Aug 8, 2025
3bbbea0
[refactor] Use `@validate_arguments` decorator in aio clients and use…
Aryamanz29 Aug 11, 2025
00ecd03
[tests] Added asyncio configuration for aio tests
Aryamanz29 Aug 11, 2025
139d097
[tests] Added async `client`, `credential_client`, `file_client`, and…
Aryamanz29 Aug 11, 2025
5494461
[tests] Added async `cm`, `audit`, `query`, and `sso`, `task`, `atlan…
Aryamanz29 Aug 12, 2025
7c3a70e
Merge branch 'main' into APP-5243
Aryamanz29 Aug 14, 2025
f61c373
[uv] Resync uv lock dependencies
Aryamanz29 Aug 14, 2025
3e73176
[feat] Added client type to the sdk headers
Aryamanz29 Aug 14, 2025
0bff7e4
[tests:aio] Added async `test_client` integration tests
Aryamanz29 Aug 14, 2025
c8c71ff
[tests:aio] Added async `test_index_search` integration tests
Aryamanz29 Aug 15, 2025
e163551
[change] Used tenacity for `_retrieve_connection_with_retry`
Aryamanz29 Aug 15, 2025
f337035
[tests:aio] Added remaining tests: admin, batch, tag, ol, cm, conn, g…
Aryamanz29 Aug 17, 2025
0ef9249
[tests] Fixed unit tests + aio integration runtime warnings (test_cus…
Aryamanz29 Aug 18, 2025
6fdcc42
[fix] Fixed race condition in shared session headers
Aryamanz29 Aug 18, 2025
62eea05
Merge branch 'main' into APP-5243
Aryamanz29 Aug 18, 2025
7152c45
[feat] Added `AsyncAtlanClient.from_token_guid()` method
Aryamanz29 Aug 18, 2025
74d5cea
[tests:aio] Fixed `aio` tests suite (+ test_lineage)
Aryamanz29 Aug 18, 2025
f5c3f82
[ci] Added `pattern` and `label` based execution of `aio` integration…
Aryamanz29 Aug 18, 2025
719b047
[ci] Fixed changed files output for `check-aio-changes` job
Aryamanz29 Aug 18, 2025
8dda545
[cleanup] Fixed imports + aio tests collision (admin_test)
Aryamanz29 Aug 19, 2025
d67c3b7
[generator] Fixed generator as per latest async refactoring changes
Aryamanz29 Aug 19, 2025
ba9a30b
[generator] Generated latest typedef models
Aryamanz29 Aug 19, 2025
1d17747
[change] Fixed async cm handling in referenceable.py (json() method)
Aryamanz29 Aug 19, 2025
dbf74cc
[ci] Added `-vv` flag for pytest verbose output
Aryamanz29 Aug 19, 2025
a609595
[qa] Fixed all mypy violations
Aryamanz29 Aug 19, 2025
0a72ae8
[test:aio] Improved `test_asset_batch` test
Aryamanz29 Aug 19, 2025
d4477ac
[change] lets use consistent naming in FS (`execute_async()`)
Aryamanz29 Aug 20, 2025
896ae40
[ci:conda] changed python constraint to `< 4`
Aryamanz29 Aug 20, 2025
020033d
[ci] enabled qa checks
Aryamanz29 Aug 20, 2025
767cdee
[tests] Use `token_client` (retry=0) to avoid extra token generation
Aryamanz29 Aug 20, 2025
ff0f932
[qa] Fixed mypy violations
Aryamanz29 Aug 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .conda-envs/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ requirements:
- setuptools
- python
run:
- python >=3.9,<3.13
- python >=3.9,<4
- jinja2 ~=3.1.6
- lazy_loader ~=0.4
- pydantic ~=2.11.7
Expand Down
107 changes: 103 additions & 4 deletions .github/workflows/pyatlan-pr.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
name: Pyatlan Pull Request Build

# This workflow runs both sync and async integration tests intelligently:
# - Sync integration tests: Always run on every PR
# - Async integration tests: Only run when:
# 1. Changes detected in pyatlan/*/aio/ or tests/*/aio/ paths
# 2. PR has the "run-async-tests" label (manual trigger)
# This prevents adding 12+ minutes to every PR while ensuring async tests run when needed.

on:
pull_request:
workflow_dispatch:
Expand Down Expand Up @@ -39,11 +46,45 @@
vulnerability-service: osv
inputs: .

check-aio-changes:
runs-on: ubuntu-latest
outputs:
run-async-tests: ${{ steps.check-conditions.outputs.run-async-tests }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Check for AIO changes or manual trigger
id: check-conditions
run: |
# Check if PR has the run-async-tests label
if echo '${{ toJson(github.event.pull_request.labels.*.name) }}' | grep -q "run-async-tests"; then
echo "run-async-tests=true" >> $GITHUB_OUTPUT
echo "🏷️ Manual trigger: Found 'run-async-tests' label"
exit 0
fi

# Check for changes in AIO-related paths
if git diff --name-only origin/${{ github.event.pull_request.base.ref }}...HEAD | grep -E "(pyatlan/.*aio/|tests/.*aio/)"; then
echo "run-async-tests=true" >> $GITHUB_OUTPUT
echo "🔍 Change detection: Found AIO-related changes:"
aio_files=$(git diff --name-only origin/${{ github.event.pull_request.base.ref }}...HEAD | grep -E "(pyatlan/.*aio/|tests/.*aio/)")
echo "$aio_files" | head -20
total_count=$(echo "$aio_files" | wc -l)
echo "📊 Total AIO files changed: $total_count"
else
echo "run-async-tests=false" >> $GITHUB_OUTPUT
echo "⏭️ No AIO changes detected and no manual trigger label found"
fi

qa-checks-and-unit-tests:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
needs: [vulnerability-scan]
runs-on: ubuntu-latest
outputs:
files: ${{ steps.distribute-integration-test-files.outputs.files }}
aio-files: ${{ steps.distribute-aio-test-files.outputs.aio-files }}
strategy:
matrix:
# Specify version as a string
Expand All @@ -65,15 +106,15 @@
- name: Install dependencies
run: uv sync --group dev

- name: QA checks (ruff-format, ruff-lint, mypy)
run: uv run ./qa-checks
# - name: QA checks (ruff-format, ruff-lint, mypy)
# run: uv run ./qa-checks

- name: Run unit tests
env: # Test tenant environment variables
ATLAN_API_KEY: ${{ secrets.ATLAN_API_KEY }}
ATLAN_BASE_URL: ${{ secrets.ATLAN_BASE_URL }}
# Run with `pytest-sugar` for enhancing the overall test report output
run: uv run pytest tests/unit --force-sugar
run: uv run pytest tests/unit --force-sugar -vv

- name: Prepare integration tests distribution
id: distribute-integration-test-files
Expand All @@ -83,6 +124,22 @@
json_files=$(echo "${files[@]}" | jq -R -c 'split(" ")[:-1]')
echo "files=$json_files" >> $GITHUB_OUTPUT

- name: Prepare async integration tests distribution
id: distribute-aio-test-files
run: |
# Check if AIO test directory exists and has test files
if [ -d "tests/integration/aio" ]; then
aio_files=$(find tests/integration/aio -name "test_*.py" -o -name "*_test.py" | tr '\n' ' ')
if [ -n "$aio_files" ]; then
json_aio_files=$(echo "${aio_files[@]}" | jq -R -c 'split(" ")[:-1]')
echo "aio-files=$json_aio_files" >> $GITHUB_OUTPUT
else
echo "aio-files=[]" >> $GITHUB_OUTPUT
fi
else
echo "aio-files=[]" >> $GITHUB_OUTPUT
fi

integration-tests:
needs: [vulnerability-scan, qa-checks-and-unit-tests]
runs-on: ubuntu-latest
Expand Down Expand Up @@ -120,4 +177,46 @@
timeout_minutes: 10 # Maximum time per test job; otherwise, the job will fail
# Run the integration test file using `pytest-timer` plugin
# to display only the durations of the 10 slowest tests with `pytest-sugar`
command: uv run pytest ${{ matrix.test_file }} -p name_of_plugin --timer-top-n 10 --force-sugar
command: uv run pytest ${{ matrix.test_file }} -p name_of_plugin --timer-top-n 10 --force-sugar -vv


async-integration-tests:
needs: [vulnerability-scan, qa-checks-and-unit-tests, check-aio-changes]
runs-on: ubuntu-latest
# Only run if AIO changes detected or manual trigger
if: needs.check-aio-changes.outputs.run-async-tests == 'true'
strategy:
fail-fast: false
matrix:
test_file: ${{fromJson(needs.qa-checks-and-unit-tests.outputs.aio-files)}}
concurrency:
group: async-${{ matrix.test_file }}

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
# Specify version as a string
# https://github.com/actions/setup-python/issues/160"
python-version: "3.9"

- name: Install uv
uses: astral-sh/setup-uv@v6

- name: Install dependencies
run: uv sync --group dev

- name: Run async integration tests
env: # Test tenant environment variables
ATLAN_API_KEY: ${{ secrets.ATLAN_API_KEY }}
ATLAN_BASE_URL: ${{ secrets.ATLAN_BASE_URL }}
uses: nick-fields/retry@v3
with:
max_attempts: 3
timeout_minutes: 15 # Async tests may take longer, increased timeout
# Run the async integration test file using `pytest-timer` plugin
# to display only the durations of the 10 slowest tests with `pytest-sugar`
command: uv run pytest ${{ matrix.test_file }} -p name_of_plugin --timer-top-n 10 --force-sugar -vv
2 changes: 1 addition & 1 deletion .github/workflows/pyatlan-test-cron.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@ jobs:
# to display only the durations of the 25 slowest tests with `pytest-sugar`
run: |
uv run pytest tests/unit
uv run pytest tests/integration -p name_of_plugin --timer-top-n 25
uv run pytest tests/integration -p name_of_plugin --timer-top-n 25 -vv
10 changes: 10 additions & 0 deletions docs/asset/appworkflowrun.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. _appworkflowrun:

AppWorkflowRun
==============

.. module:: pyatlan.model.assets
:no-index:

.. autoclass:: AppWorkflowRun
:members:
10 changes: 10 additions & 0 deletions docs/asset/databricksnotebook.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. _databricksnotebook:

DatabricksNotebook
==================

.. module:: pyatlan.model.assets
:no-index:

.. autoclass:: DatabricksNotebook
:members:
10 changes: 10 additions & 0 deletions docs/asset/microstrategycolumn.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. _microstrategycolumn:

MicroStrategyColumn
===================

.. module:: pyatlan.model.assets
:no-index:

.. autoclass:: MicroStrategyColumn
:members:
10 changes: 10 additions & 0 deletions docs/asset/notebook.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. _notebook:

Notebook
========

.. module:: pyatlan.model.assets
:no-index:

.. autoclass:: Notebook
:members:
10 changes: 10 additions & 0 deletions docs/asset/qlikcolumn.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. _qlikcolumn:

QlikColumn
==========

.. module:: pyatlan.model.assets
:no-index:

.. autoclass:: QlikColumn
:members:
7 changes: 5 additions & 2 deletions docs/assets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ You can interact with all of the following different kinds of assets:
asset/anomalo
asset/anomalocheck
asset/app
asset/appworkflowrun
asset/application
asset/applicationfield
asset/asset
Expand Down Expand Up @@ -124,6 +125,7 @@ You can interact with all of the following different kinds of assets:
asset/databricks
asset/databricksaimodelcontext
asset/databricksaimodelversion
asset/databricksnotebook
asset/databricksunitycatalogtag
asset/dataverse
asset/dataverseattribute
Expand Down Expand Up @@ -204,6 +206,7 @@ You can interact with all of the following different kinds of assets:
asset/metric
asset/microstrategy
asset/microstrategyattribute
asset/microstrategycolumn
asset/microstrategycube
asset/microstrategydocument
asset/microstrategydossier
Expand Down Expand Up @@ -232,6 +235,7 @@ You can interact with all of the following different kinds of assets:
asset/multidimensionaldataset
asset/namespace
asset/nosql
asset/notebook
asset/objectstore
asset/persona
asset/powerbi
Expand Down Expand Up @@ -260,6 +264,7 @@ You can interact with all of the following different kinds of assets:
asset/qlik
asset/qlikapp
asset/qlikchart
asset/qlikcolumn
asset/qlikdataset
asset/qliksheet
asset/qlikspace
Expand Down Expand Up @@ -366,5 +371,3 @@ You can interact with all of the following different kinds of assets:
asset/view
asset/workflow
asset/workflowrun
asset/alpha_dqrule
asset/alpha_dqruletemplate
2 changes: 1 addition & 1 deletion pyatlan/cache/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: Apache-2.0
# Copyright 2022 Atlan Pte. Ltd.
# Copyright 2025 Atlan Pte. Ltd.
# Based on original code from https://github.com/apache/atlas (under Apache-2.0 license)
2 changes: 1 addition & 1 deletion pyatlan/cache/abstract_asset_cache.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# Copyright 2024 Atlan Pte. Ltd.
# Copyright 2025 Atlan Pte. Ltd.
from __future__ import annotations

import threading
Expand Down
33 changes: 33 additions & 0 deletions pyatlan/cache/aio/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# SPDX-License-Identifier: Apache-2.0
# Copyright 2025 Atlan Pte. Ltd.
"""
Async cache modules for Atlan.

This module provides async versions of all cache functionality
with the same API as the sync versions, just requiring await.

Pattern: All async cache methods reuse shared business logic from pyatlan.cache.common
to ensure identical behavior with sync cache implementations.
"""

from .atlan_tag_cache import AsyncAtlanTagCache
from .connection_cache import AsyncConnectionCache
from .custom_metadata_cache import AsyncCustomMetadataCache
from .dq_template_config_cache import AsyncDQTemplateConfigCache
from .enum_cache import AsyncEnumCache
from .group_cache import AsyncGroupCache
from .role_cache import AsyncRoleCache
from .source_tag_cache import AsyncSourceTagCache
from .user_cache import AsyncUserCache

__all__ = [
"AsyncAtlanTagCache",
"AsyncConnectionCache",
"AsyncCustomMetadataCache",
"AsyncDQTemplateConfigCache",
"AsyncEnumCache",
"AsyncGroupCache",
"AsyncRoleCache",
"AsyncSourceTagCache",
"AsyncUserCache",
]
Loading
Loading