Skip to content

Conversation

@rishabh6788
Copy link
Collaborator

Description

This PR added looped mode parameter for bulk operations. Using this users will be able run ingestion even when the corpus is not big enough to sustain long term indexing at any specified rate.

If not time-period is defined and looped mode is set to true then ingestion would run indefinitely.

Example:

{
  "name": "bulk-index",
  "operation": {
    "operation-type": "bulk",
    "bulk-size": 5000,
    "looped": true
  },
  "time-period": 900,
  "clients": 10
}

Issues Resolved

[List any issues this PR will resolve]

Testing

  • New functionality includes testing

[Describe how this change was tested]


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Rishabh Singh <[email protected]>
Copy link
Collaborator

@gkamat gkamat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be useful for "looped" to be an integer instead, which would indicate the number of iterations to loop. That would operate as in iteration-based mode and the user could specify "1000" for a thousand iterations, and "0" for infinite -- the current behaviour.

@rishabh6788 rishabh6788 merged commit 0522e68 into opensearch-project:main Oct 19, 2025
11 of 12 checks passed
@opensearch-trigger-bot
Copy link

The backport to 1.16 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-1.16 1.16
# Navigate to the new working tree
pushd ../.worktrees/backport-1.16
# Create a new branch
git switch --create backport/backport-968-to-1.16
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0522e68f577d2e99de76a516321932eef53313e5
# Push it to GitHub
git push --set-upstream origin backport/backport-968-to-1.16
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-1.16

Then, create a pull request where the base branch is 1.16 and the compare/head branch is backport/backport-968-to-1.16.

rishabh6788 added a commit that referenced this pull request Oct 23, 2025
Signed-off-by: Rishabh Singh <[email protected]>
(cherry picked from commit 0522e68)
rishabh6788 added a commit that referenced this pull request Oct 23, 2025
Signed-off-by: Rishabh Singh <[email protected]>
(cherry picked from commit 0522e68)
rishabh6788 added a commit that referenced this pull request Oct 23, 2025
Signed-off-by: Rishabh Singh <[email protected]>
(cherry picked from commit 0522e68)
rishabh6788 added a commit that referenced this pull request Oct 23, 2025
Signed-off-by: Rishabh Singh <[email protected]>
(cherry picked from commit 0522e68)
rishabh6788 added a commit that referenced this pull request Oct 23, 2025
Signed-off-by: Rishabh Singh <[email protected]>
(cherry picked from commit 0522e68)
IanHoang pushed a commit that referenced this pull request Oct 30, 2025
Signed-off-by: Rishabh Singh <[email protected]>
(cherry picked from commit 0522e68)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants