feat: add delete by filter and update by filer to OpenSearchDocumentStore #2407
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
update_by_filter()anddelete_by_filter()operations toOpenSearchDocumentStore#2327Proposed Changes:
Added
delete_by_filter()andupdate_by_filter()methods toOpenSearchDocumentStoreto enable bulk operations on filtered documents.Implementation details:
delete_by_queryAPI to efficiently delete documents matching filter criteria without retrieving them firstupdate_by_queryAPI with Painless scripts to update metadata fields for matching documentsUse cases:
Files changed:
How did you test it?
test_delete_by_filter: Verifies selective deletion based on metadata filterstest_update_by_filter: Verifies metadata updates for filtered documentstest_delete_by_filter_async: Async version of delete testtest_update_by_filter_async: Async version of update testNotes for the reviewer
delete_by_filter()andupdate_by_filter()delete_by_queryandupdate_by_queryAPIs for efficiency and performanceupdate_by_filter()method uses Painless scripting language to update metadata:ctx._source.metadata.{key} = params.{key}normalize_filters()function to ensure consistency withfilter_documents()methodresult.get("deleted")andresult.get("updated"))time.sleep(2)after operations to ensure changes are reflected (OpenSearch eventual consistency)Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.