Update Python Client to 3.0.0 (#128)

This brings in the new Airflow API v2 to the Python client

As part of this change the following breaking changes have occurred:

- The API returns 422 status code instead of 400 for validation errors.

  For instance when the request payload, path params, or query params are invalid.

- When listing a resource for instance on GET ``/dags``, ``fields`` parameter is not supported anymore to obtain a partial response.

  The full objects will be returned by the endpoint. This feature might be added back in upcoming 3.x versions.

- Passing list in query parameters switched from ``form, non exploded`` to ``form, exploded``
  i.e before ``?my_list=item1,item2`` now ``?my_list=item1&my_list=item2``

- ``execution_date`` was deprecated and has been removed. Any payload or parameter mentioning this field has been removed.

- Datetime format are RFC3339-compliant in FastAPI, more permissive than ISO8601,
  meaning that the API returns zulu datetime for responses, more info here fastapi/fastapi#7693 (comment).
  Both ``Z`` and ``00+xx`` are supported for payload and params.

  This is due FastAPI and pydantic v2 default behavior.

- PATCH on ``DagRun`` and ``TaskInstance`` are more generic and allow in addition to update the resource state and the note content.

  Therefore the two legacy dedicated endpoints to update a ``DagRun`` note and ``TaskInstance`` note have been removed.

  Same for the set task instance state, it is now handled by the broader PATCH on task instances.

- ``assets/queuedEvent`` endpoints have moved to ``assets/queuedEvents`` for consistency.

- dag_parsing endpoint now returns a 409 when the DagPriorityParsingRequest already exists. It was returning 201 before.

- ``clearTaskInstances`` endpoint default value for ``reset_dag_runs`` field has been updated from ``False`` to ``True``.

- Pool name can't be modified in the PATCH pool endpoint anymore. Pool name shouldn't be updated via pool PATCH API call.

- Logical date is now a nullable. In addition it is a nullable required payload field for Triggering a DagRun endpoint.

Author: kaxil

.github/workflows/ci.yml

@@ -15,7 +15,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: 3.8
+          python-version: 3.9
           cache: 'pip'
       - name: Install hatch
         run: |

CHANGELOG.md

@@ -17,6 +17,58 @@
  under the License.
  -->
 
+# v3.0.0
+
+This is the first release of the **Airflow 3.0.0** Python client. It introduces compatibility with the new [Airflow 3.0 REST API](https://airflow.apache.org/docs/apache-airflow/3.0.0/stable-rest-api-ref.html), and includes several **breaking changes** and behavior updates.
+
+Below is a list of important changes. Refer to individual endpoint documentation for full details.
+
+- API v1 (`/api/v1`) has been dropped and replaced with API v2(`/api/v2`).
+
+- **422 Validation Errors (instead of 400)**
+
+  The API now returns `422 Unprocessable Entity` for validation errors (e.g. bad payload, path params, or query params), instead of `400 Bad Request`.
+
+- **Partial response support removed (`fields` parameter)**
+
+  Endpoints like `GET /dags` no longer support the `fields` query param for partial responses. Full objects are returned by default. This feature may return in a future 3.x release.
+
+- Passing list in query parameters switched from ``form, non exploded`` to ``form, exploded`` i.e before ``?my_list=item1,item2`` now ``?my_list=item1&my_list=item2``
+
+- **`execution_date` has been removed**
+
+  The previously deprecated `execution_date` parameter and fields are now fully removed. Use `logical_date` instead.
+
+- **Datetime format updated to RFC3339-compliant**
+
+  Datetimes returned are now in [RFC3339](https://datatracker.ietf.org/doc/html/rfc3339) format (e.g. `2024-10-01T13:00:00Z`). Both `Z` and `+00:00` forms are accepted in inputs.
+  → This change comes from FastAPI & Pydantic v2 behavior.
+  [More info](https://github.com/fastapi/fastapi/discussions/7693#discussioncomment-5143311)
+
+- PATCH on ``DagRun`` and ``TaskInstance`` are more generic and allow in addition to update the resource state and the note content.
+
+  Therefore, the two legacy dedicated endpoints to update a ``DagRun`` note and ``TaskInstance`` note have been removed.
+
+  Same for the set task instance state, it is now handled by the broader PATCH on task instances.
+
+- ``assets/queuedEvent`` endpoints have moved to ``assets/queuedEvents`` for consistency.
+
+- **`dag_parsing` returns 409 for duplicates**
+
+  If a `DagPriorityParsingRequest` already exists, `POST /dag_parsing` now returns `409 Conflict` instead of `201 Created`.
+
+- **Default value change in `clearTaskInstances`**
+
+  The `reset_dag_runs` field now defaults to `true` instead of `false`.
+
+- **Pool name is no longer editable**
+
+  `PATCH /pools/{pool_name}` can no longer be used to rename a pool. Pool names are immutable via the API.
+
+- **`logical_date` is now a required nullable field**
+
+  When triggering a DAG run (`POST /dags/{dag_id}/dagRuns`), `logical_date` is now required but can explicitly be set to `null`.
+
 # v2.10.0
 
 ## Major changes:

INSTALL

@@ -1,6 +1,6 @@
 # INSTALL / BUILD instructions for Apache Airflow Python Client
 
-This ia a generic installation method that requires a number of dependencies to be installed.
+This is a generic installation method that requires a number of dependencies to be installed.
 
 Depending on your system you might need different prerequisites, but the Python3.6 or above is a must.
 

LICENSE

@@ -247,7 +247,6 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
 
     (BSD 3 License) d3 v5.16.0 (https://d3js.org)
     (BSD 3 License) d3-shape v2.1.0 (https://github.com/d3/d3-shape)
-    (BSD 3 License) cgroupspy 0.2.1 (https://github.com/cloudsigma/cgroupspy)
 
 ========================================================================
 See licenses/LICENSES-ui.txt for packages used in `/airflow/www`

README.md

@@ -182,7 +182,7 @@ If you want to check which auth backend is currently set, you can use
 
 ```bash
 $ airflow config get-value api auth_backends
-airflow.api.auth.backend.basic_auth
+airflow.providers.fab.auth_manager.api.auth.backend.basic_auth
 ```
 
 The default is to deny all requests.
@@ -248,7 +248,7 @@ For more information, please visit [https://airflow.apache.org](https://airflow.
 
 ## Requirements.
 
-Python >=3.8
+Python >=3.9
 
 ## Installation & Usage
 
@@ -282,7 +282,7 @@ Please follow the [installation procedure](#installation--usage) and then run th
 
 ```python
 import time
-from airflow_client import client
+import airflow_client.client
 from pprint import pprint
 from airflow_client.client.api import config_api
 from airflow_client.client.model.config import Config
@@ -343,15 +343,15 @@ Class | Method | HTTP request | Description
 *DAGRunApi* | [**get_dag_run**](docs/DAGRunApi.md#get_dag_run) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id} | Get a DAG run
 *DAGRunApi* | [**get_dag_runs**](docs/DAGRunApi.md#get_dag_runs) | **GET** /dags/{dag_id}/dagRuns | List DAG runs
 *DAGRunApi* | [**get_dag_runs_batch**](docs/DAGRunApi.md#get_dag_runs_batch) | **POST** /dags/~/dagRuns/list | List DAG runs (batch)
-*DAGRunApi* | [**get_upstream_dataset_events**](docs/DAGRunApi.md#get_upstream_dataset_events) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamDatasetEvents | Get dataset events for a DAG run
+*DAGRunApi* | [**get_upstream_asset_events**](docs/DAGRunApi.md#get_upstream_asset_events) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamAssetEvents | Get asset events for a DAG run
 *DAGRunApi* | [**post_dag_run**](docs/DAGRunApi.md#post_dag_run) | **POST** /dags/{dag_id}/dagRuns | Trigger a new DAG run
 *DAGRunApi* | [**set_dag_run_note**](docs/DAGRunApi.md#set_dag_run_note) | **PATCH** /dags/{dag_id}/dagRuns/{dag_run_id}/setNote | Update the DagRun note.
 *DAGRunApi* | [**update_dag_run_state**](docs/DAGRunApi.md#update_dag_run_state) | **PATCH** /dags/{dag_id}/dagRuns/{dag_run_id} | Modify a DAG run
 *DagWarningApi* | [**get_dag_warnings**](docs/DagWarningApi.md#get_dag_warnings) | **GET** /dagWarnings | List dag warnings
-*DatasetApi* | [**get_dataset**](docs/DatasetApi.md#get_dataset) | **GET** /datasets/{uri} | Get a dataset
-*DatasetApi* | [**get_dataset_events**](docs/DatasetApi.md#get_dataset_events) | **GET** /datasets/events | Get dataset events
-*DatasetApi* | [**get_datasets**](docs/DatasetApi.md#get_datasets) | **GET** /datasets | List datasets
-*DatasetApi* | [**get_upstream_dataset_events**](docs/DatasetApi.md#get_upstream_dataset_events) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamDatasetEvents | Get dataset events for a DAG run
+*AssetApi* | [**get_asset**](docs/DatasetApi.md#get_asset) | **GET** /assets/{uri} | Get an asset
+*AssetApi* | [**get_asset_events**](docs/DatasetApi.md#get_asset_events) | **GET** /assets/events | Get asset events
+*DatasetApi* | [**get_assets**](docs/DatasetApi.md#get_assets) | **GET** /assets | List assets
+*DatasetApi* | [**get_upstream_asset_events**](docs/DatasetApi.md#get_upstream_asset_events) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamAssetEvents | Get dataset events for a DAG run
 *EventLogApi* | [**get_event_log**](docs/EventLogApi.md#get_event_log) | **GET** /eventLogs/{event_log_id} | Get a log entry
 *EventLogApi* | [**get_event_logs**](docs/EventLogApi.md#get_event_logs) | **GET** /eventLogs | List log entries
 *ImportErrorApi* | [**get_import_error**](docs/ImportErrorApi.md#get_import_error) | **GET** /importErrors/{import_error_id} | Get an import error
@@ -402,6 +402,11 @@ Class | Method | HTTP request | Description
  - [ActionCollection](docs/ActionCollection.md)
  - [ActionCollectionAllOf](docs/ActionCollectionAllOf.md)
  - [ActionResource](docs/ActionResource.md)
+ - [AssetCollection](docs/AssetCollection.md)
+ - [AssetCollectionAllOf](docs/AssetCollectionAllOf.md)
+ - [AssetEvent](docs/AssetEvent.md)
+ - [AssetEventCollection](docs/AssetEventCollection.md)
+ - [AssetEventCollectionAllOf](docs/AssetEventCollectionAllOf.md)
  - [BasicDAGRun](docs/BasicDAGRun.md)
  - [ClassReference](docs/ClassReference.md)
  - [ClearDagRun](docs/ClearDagRun.md)
@@ -426,17 +431,12 @@ Class | Method | HTTP request | Description
  - [DAGRun](docs/DAGRun.md)
  - [DAGRunCollection](docs/DAGRunCollection.md)
  - [DAGRunCollectionAllOf](docs/DAGRunCollectionAllOf.md)
- - [DagScheduleDatasetReference](docs/DagScheduleDatasetReference.md)
+ - [DagScheduleAssetReference](docs/DagScheduleAssetReference.md)
  - [DagState](docs/DagState.md)
  - [DagWarning](docs/DagWarning.md)
  - [DagWarningCollection](docs/DagWarningCollection.md)
  - [DagWarningCollectionAllOf](docs/DagWarningCollectionAllOf.md)
  - [Dataset](docs/Dataset.md)
- - [DatasetCollection](docs/DatasetCollection.md)
- - [DatasetCollectionAllOf](docs/DatasetCollectionAllOf.md)
- - [DatasetEvent](docs/DatasetEvent.md)
- - [DatasetEventCollection](docs/DatasetEventCollection.md)
- - [DatasetEventCollectionAllOf](docs/DatasetEventCollectionAllOf.md)
  - [Error](docs/Error.md)
  - [EventLog](docs/EventLog.md)
  - [EventLogCollection](docs/EventLogCollection.md)
@@ -481,7 +481,7 @@ Class | Method | HTTP request | Description
  - [TaskInstanceCollectionAllOf](docs/TaskInstanceCollectionAllOf.md)
  - [TaskInstanceReference](docs/TaskInstanceReference.md)
  - [TaskInstanceReferenceCollection](docs/TaskInstanceReferenceCollection.md)
- - [TaskOutletDatasetReference](docs/TaskOutletDatasetReference.md)
+ - [TaskOutletAssetReference](docs/TaskOutletAssetReference.md)
  - [TaskState](docs/TaskState.md)
  - [TimeDelta](docs/TimeDelta.md)
  - [Trigger](docs/Trigger.md)
@@ -534,11 +534,11 @@ that uses the API to run the tests. To do that, you need to:
 
 ```ini
 [api]
-auth_backend = airflow.api.auth.backend.session,airflow.api.auth.backend.basic_auth
+auth_backend = airflow.providers.fab.auth_manager.api.auth.backend.session,airflow.providers.fab.auth_manager.api.auth.backend.basic_auth
 ```
 
 You can also set it by env variable:
-`export AIRFLOW__API__AUTH_BACKENDS=airflow.api.auth.backend.session,airflow.api.auth.backend.basic_auth`
+`export AIRFLOW__API__AUTH_BACKENDS=airflow.providers.fab.auth_manager.api.auth.backend.session,airflow.providers.fab.auth_manager.api.auth.backend.basic_auth`
 
 * configure your airflow webserver to load example dags
   In the `[core]` section of your `airflow.cfg` set:
@@ -555,11 +555,11 @@ You can also set it by env variable: `export AIRFLOW__CORE__LOAD_EXAMPLES=True`
   In the `[webserver]` section of your `airflow.cfg` set:
 
 ```ini
-[webserver]
+[api]
 expose_config = True
 ```
 
-You can also set it by env variable: `export AIRFLOW__WEBSERVER__EXPOSE_CONFIG=True`
+You can also set it by env variable: `export AIRFLOW__API__EXPOSE_CONFIG=True`
 
 * Configure your host/ip/user/password in the `test_python_client.py` file