Skip to content

Update EIS sparse and dense embedding max batch size to 16 #132646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jaybcee merged 4 commits into from
Aug 11, 2025
Merged

Update EIS sparse and dense embedding max batch size to 16 #132646

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e3c9927
Update EIS sparse embedding max batch size to 16
jaybcee Aug 11, 2025
258443d
Mirror for dense embeddings and add comments
jaybcee Aug 11, 2025
6763085
Merge branch 'main' into jbc-reduce-eis-batch-size
jaybcee Aug 11, 2025
dc3834a
Update docs/changelog/132646.yaml
jaybcee Aug 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...main/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ public class ElasticInferenceService extends SenderService {
public static final String NAME = "elastic";
public static final String ELASTIC_INFERENCE_SERVICE_IDENTIFIER = "Elastic Inference Service";
public static final Integer DENSE_TEXT_EMBEDDINGS_DIMENSIONS = 1024;
public static final Integer SPARSE_TEXT_EMBEDDING_MAX_BATCH_SIZE = 512;
public static final Integer SPARSE_TEXT_EMBEDDING_MAX_BATCH_SIZE = 16;

private static final EnumSet<TaskType> IMPLEMENTED_TASK_TYPES = EnumSet.of(
TaskType.SPARSE_EMBEDDING,
Expand Down