fix(security): upgrade Apache Airflow to 3.2.1

[akashverma0786]

…resolve CVEs

Describe your changes:

Fixes #

I worked on ... because ...

Type of change:

• Bug fix
• Improvement
• New feature
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation

High-level design:

N/A — small change.

Tests:

Use cases covered

Unit tests

Backend integration tests

Ingestion integration tests

Playwright (UI) tests

Manual testing performed

UI screen recording / screenshots:

Not applicable.

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• My PR is linked to a GitHub issue via Fixes #<issue-number> above.
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
• For UI changes: I attached a screen recording and/or screenshots above.
• I have added tests (unit / integration / Playwright as applicable) and listed them above.

---

Summary by Gitar

• Security Upgrades:
  • Upgraded apache-airflow to 3.2.1 and Flask to 3.1.3 to resolve associated CVEs.
• Refactoring:
  • Updated deploy.py to import escape from markupsafe instead of flask.
  • Replaced requests.utils with urllib.parse for URL parsing and quoting across multiple ingestion modules.
• Compatibility Fixes:
  • Updated metadata.py and operator.py to use new Airflow 3 SDK paths for SerializedDAG and Context.
  • Added a null check for settings.Session in connection.py to prevent RuntimeError on Airflow 3.x.
• Infrastructure Changes:
  • Updated setup.py dependencies for gcsfs and s3fs to ~=2026.3.
• Maintenance:
  • Cleaned up copyright headers in test_lookml_bitbucket_reader.py.

_{This will update automatically on new commits.}

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

🟡 Playwright Results — all passed (14 flaky)

✅ 4056 passed · ❌ 0 failed · 🟡 14 flaky · ⏭️ 103 skipped

Shard	Passed	Failed	Flaky	Skipped
✅ Shard 1	299	0	0	4
🟡 Shard 2	742	0	8	25
🟡 Shard 3	783	0	1	7
🟡 Shard 4	789	0	1	18
🟡 Shard 5	708	0	1	41
🟡 Shard 6	735	0	3	8

🟡 14 flaky test(s) (passed on retry)

• Features/BulkEditEntity.spec.ts › Glossary (shard 2, 1 retry)
• Features/ColumnBulkOperations.spec.ts › should navigate through pages (shard 2, 1 retry)
• Features/DataQuality/TestCaseImportExportE2eFlow.spec.ts › Admin: Complete export-import-validate flow (shard 2, 1 retry)
• Features/DataQuality/TestCaseResultPermissions.spec.ts › User with only VIEW cannot PATCH results (shard 2, 1 retry)
• Features/KnowledgeCenterList.spec.ts › Knowledge Center List - Test infinite scroll/pagination (shard 2, 1 retry)
• Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
• Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
• Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
• Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
• Pages/Entity.spec.ts › Tag and Glossary Selector should close vice versa (shard 4, 1 retry)
• Pages/ExplorePageRightPanel_KnowledgeCenter.spec.ts › Should remove user owner for knowledgeCenter (shard 5, 2 retries)
• Features/AutoPilot.spec.ts › Create Service and check the AutoPilot status (shard 6, 1 retry)
• Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
• Pages/UserDetails.spec.ts › Create team with domain and verify visibility of inherited domain in user profile after team removal (shard 6, 1 retry)

📦 Download artifacts

How to debug locally

# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

[gitar-bot]

Code Review ✅ Approved 4 resolved / 4 findings

Upgrades Apache Airflow to 3.2.1 and Flask to 3.1.3 to resolve vulnerabilities, while addressing dependencies, SDK compatibility, and import shadowing. All identified issues regarding version constraints, documentation, and code structure have been resolved.

✅ 4 resolved

✅ Quality: upath workaround lacks comment and ticket reference

📄 ingestion/Dockerfile.ci:140
Line 140 in Dockerfile.ci adds RUN pip uninstall upath -y && pip install "universal-pathlib==0.3.10" without any comment explaining why this workaround is needed. Per project guidelines, workarounds should include a comment with a ticket reference so the team knows when it can be removed. Additionally, this workaround is absent from ingestion/Dockerfile, which may cause inconsistency between the two images.

✅ Edge Case: Unbounded version constraints for gcsfs and s3fs

📄 ingestion/setup.py:31 📄 ingestion/setup.py:73
The version specifiers for gcsfs and s3fs were changed from ~=2023.12.1 (which constrains to >=2023.12.1, ==2023.12.*) to >=2023.12.1 (no upper bound). Both libraries use calendar versioning and may introduce breaking API changes in future major releases. This risks build or runtime failures when a new incompatible version is published.

If the goal is to allow newer versions needed by Airflow 3.2.1, consider using a bounded range (e.g., >=2023.12.1,<2026.0.0) or a compatible-release operator at the year level (~=2023.12) to allow patch/minor updates while preventing major breakage.

✅ Bug: Duplicate imports shadow each other in three files

📄 ingestion/src/metadata/ingestion/source/database/snowflake/models.py:18 📄 ingestion/src/metadata/ingestion/source/database/snowflake/models.py:21 📄 ingestion/src/metadata/ingestion/ometa/utils.py:20 📄 ingestion/src/metadata/ingestion/ometa/utils.py:23 📄 ingestion/src/metadata/utils/entity_link.py:18 📄 ingestion/src/metadata/utils/entity_link.py:24
The PR adds from urllib.parse import ... lines but leaves existing imports of the same name from requests.utils / requests.compat intact. In Python, the later import shadows the earlier one, making the added import dead code. While functionally harmless today (both resolve to the same stdlib function), this creates confusion and linting noise:

• models.py: from urllib.parse import quote (line 18) is immediately shadowed by from requests.utils import quote (line 21)
• utils.py: from urllib.parse import quote as url_quote (line 20) is shadowed by from requests.utils import quote as url_quote (line 23)
• entity_link.py: from urllib.parse import unquote_plus (line 18) is shadowed by from requests.compat import unquote_plus (line 24)

If the intent is to migrate away from requests.utils/requests.compat re-exports, the old imports should be removed. If the intent is a fallback, only one should remain.

✅ Quality: Test file entirely commented out instead of deleted or fixed

📄 ingestion/tests/unit/topology/dashboard/test_lookml_bitbucket_reader.py:1-15
The entire test file test_lookml_bitbucket_reader.py has been commented out (59 lines). Per project guidelines, commented-out code is not allowed — code should either be deleted or fixed. If the test is failing due to the Airflow upgrade and needs to be temporarily disabled, it should be deleted with a TODO referencing a ticket for re-enabling it, or the test should be updated to work with the new dependencies.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
20.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud