Skip to content

[1/3] sdks/python: refactor Milvus-related utilities as preparation step for Milvus Sink I/O integration #35708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
damccorm merged 37 commits into from
Nov 12, 2025
Merged

[1/3] sdks/python: refactor Milvus-related utilities as preparation step for Milvus Sink I/O integration #35708

Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
5df9628
sdks/python: replace the deprecated testcontainer max tries
mohamedawnallah Oct 30, 2025
91266a7
sdks/python: handle transient testcontainer startup/teardown errors
mohamedawnallah Oct 30, 2025
fa6d2f0
sdks/python: bump `testcontainers` py pkg version
mohamedawnallah Oct 31, 2025
9445aaa
sdks/python: integrate milvus sink I/O
mohamedawnallah Oct 31, 2025
f3a0b88
sdks/python: fix linting issues
mohamedawnallah Oct 31, 2025
4cbe014
sdks/python: add missing apache beam liscense header for `test_utils.py`
mohamedawnallah Oct 31, 2025
461c8fe
notebooks/beam-ml: use new refactored code in milvus enrichment handler
mohamedawnallah Oct 31, 2025
9d41879
CHANGES.md: update release notes
mohamedawnallah Oct 31, 2025
c64e9c9
sdks/python: mark milvus itests with `require_docker_in_docker` marker
mohamedawnallah Oct 31, 2025
825bf30
sdks/python: override milvus db version with the default
mohamedawnallah Oct 31, 2025
e6569ba
sdsk/python: add missing import in rag utils
mohamedawnallah Oct 31, 2025
281ea3f
sdks/python: fix linting issue
mohamedawnallah Oct 31, 2025
5a350b5
rag/ingestion/milvus_search_itest.py: ensure flushing in-memory data …
mohamedawnallah Nov 1, 2025
ee64600
sdks/python: fix linting issues
mohamedawnallah Nov 1, 2025
dab040a
sdks/python: fix formatting issues
mohamedawnallah Nov 1, 2025
deef266
sdks/python: fix arising linting issue
mohamedawnallah Nov 2, 2025
b4e31e8
rag: reuse `retry_with_backoff` for one-time setup operations
mohamedawnallah Nov 2, 2025
795ed60
sdks/python: fix linting issues
mohamedawnallah Nov 2, 2025
9d35585
sdks/python: fix py docs CI issue
mohamedawnallah Nov 2, 2025
119108f
sdks/python: fix linting issues
mohamedawnallah Nov 2, 2025
cfc44f6
sdks/python: fix linting issues
mohamedawnallah Nov 2, 2025
599c7f4
sdks/python: isolate milvus sink integration to be in follow-up PR
mohamedawnallah Nov 5, 2025
2ba2b33
CHANGES.md: remove milvus from release notes in the refactoring PR
mohamedawnallah Nov 5, 2025
894ab28
sdks/python: remove `with_sparse_embedding_spec` column specs builder
mohamedawnallah Nov 5, 2025
21ce084
sdks/python: fix linting issues
mohamedawnallah Nov 5, 2025
ffa5d2b
Revert "notebooks/beam-ml: use new refactored code in milvus enrichme…
mohamedawnallah Nov 6, 2025
732ae31
Merge remote-tracking branch 'upstream/master' into sinkWithMilvusIO
mohamedawnallah Nov 6, 2025
0c00044
sdks/python: fix linting issues
mohamedawnallah Nov 6, 2025
c9f4b6c
sdks/python: fix linting issues
mohamedawnallah Nov 6, 2025
17ba353
sdks/python: fix linting issues
mohamedawnallah Nov 6, 2025
90c43ea
Merge remote-tracking branch 'upstream/master' into sinkWithMilvusIO
mohamedawnallah Nov 6, 2025
560f926
sdks/python: fix linting issues
mohamedawnallah Nov 6, 2025
f72c2e1
CI: fix import errors in CI
mohamedawnallah Nov 6, 2025
0ef2c7d
sdks/python: fix linting issues
mohamedawnallah Nov 6, 2025
690fd72
sdks/python: fix linting issues
mohamedawnallah Nov 6, 2025
f01a7e5
sdks/python: fix linting issues
mohamedawnallah Nov 10, 2025
3145561
sdks/python: fix linting issues
mohamedawnallah Nov 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@
* X feature added (Java/Python) ([#X](https://github.com/apache/beam/issues/X)).
* Python examples added for Milvus search enrichment handler on [Beam Website](https://beam.apache.org/documentation/transforms/python/elementwise/enrichment-milvus/)
including jupyter notebook example (Python) ([#36176](https://github.com/apache/beam/issues/36176)).
* Milvus sink I/O connector added (Python) ([#36702](
https://github.com/apache/beam/issues/36702)). Now Beam has full support for
Milvus integration including Milvus enrichment and sink operations.

## Breaking Changes

Expand Down
Loading
Loading