# RFC: Virtual Shards for Elastic Index Scaling in OpenSearch

[atris]

Is your feature request related to a problem? Please describe

---

Summary

This RFC proposes adding Virtual Shards to OpenSearch — a lightweight logical partitioning layer between documents and physical shards. Instead of routing directly to a fixed physical shard, documents are first assigned to one of many virtual shards, which are then mapped to physical shards at runtime.

This enables live rebalancing, elastic scaling, and temporal locality handling, without reindexing or changes to clients.

---

Motivation and Value

Virtual shards improve OpenSearch’s flexibility and long-term scalability by decoupling routing from physical shard allocation. This allows more granular routing and makes it easier to isolate hotspots, scale incrementally, and optimize storage tiering without operational disruption.

They offer a clean way to:

Capability	How Virtual Shards Help
Elastic Growth	Add physical shards and remap vShards without reindexing
Hotspot Isolation	Move only hot vShards (e.g. recent time range or tenant)
Temporal Locality	Co-locate recent vShards on fast nodes and archive older ones
Operational Simplicity	Start with many vShards, scale physical shards only when needed
Autoscaling Ready	Enable future rebalancing based on traffic/load
Index Flexibility	Avoid over/undersharding, future-proof long-lived indices

This design complements existing shard functionality and prepares OpenSearch for large-scale, dynamic, multi-tenant workloads.

---

Describe the solution you'd like

Design Overview

Core Idea

document → virtual_shard → physical_shard

• Each index declares number_of_virtual_shards (e.g. 1024)
• Routing uses hash(routing_key) % num_vshards to pick a vShard
• A vShard-to-physical-shard map is stored in cluster state
• Documents are routed to the mapped physical shard internally

---

Index Configuration Example

{
  "settings": {
    "number_of_shards": 4,
    "number_of_virtual_shards": 1024
  }
}

• The index stores data in 4 physical shards
• Routing logic uses 1024 logical vShards
• Internal mapping allows reassignment without index rebuild

---

What This Enables

Use Case	Benefit
Live Scale-Out	Add a physical shard, remap only impacted vShards, no reindexing
Tenant Isolation	Route specific tenants to their own vShards and scale independently
Temporal Querying	Route by timestamp, isolate hot vShards, optimize performance
Cold Tiering	Move older vShards to cold storage without splitting the index
Long-Running Index	Evolve layout over time without changing index name or structure

---

Moving a vShard

To relocate a vShard:

1. Identify and copy documents that hash to that vShard
2. Replay translog updates for the vShard
3. Update the vShard-to-shard mapping in cluster state (atomic)
4. Remove old segments via merge or compaction

The process uses existing replication and fencing mechanisms to ensure safety.

---

Admin APIs

Endpoint	Description
GET /_vshard/state	View current vShard → shard mapping
POST /_vshard/move	Manually trigger a vShard relocation
GET /_vshard/stats	View metrics like document count, size, QPS

These APIs are internal/admin only — not exposed to client applications.

---

Compatibility

Feature	Status
ILM	Compatible; vShards can follow tier policies
Replication	Works with existing shard replication
Routing Key	Fully supported (impacts vShard assignment)
Search APIs	No client-side changes
Indexing	Adds routing layer before shard resolution

---

Use Case: Temporal Data (e.g. Box-like Ingestion)

A system indexing user folders and files over time benefits from this model:

• Routing key includes folder ID and timestamp (e.g. folder:2024-07)
• Recent folders map to hot vShards
• Hot vShards are routed to fast SSD-backed physical shards
• Older vShards are moved to cold nodes
• No rollovers or reindexing required

---

Implementation Plan

Phase 1 – Core Support

• Add support for number_of_virtual_shards as an index setting
• Compute vShard ID per document using routing hash
• Maintain vShard → physical shard mapping in cluster state
• Update indexing and search paths to route through vShards

Phase 2 – Manual Movement

• Add admin API to move vShards
• Support atomic map update + translog replay per vShard
• Enable parallel movement for disjoint vShards

Phase 3 – Optional Rebalancer

• Track per-vShard stats: size, document count, query rate
• Add load-aware background balancer (pluggable)
• Future: autoscaler hooks for vShard-based elasticity

---

Fault Tolerance

• vShard mapping is stored and versioned in cluster metadata
• Routing is atomic and consistent across replicas
• Movement reuses fencing, recovery, and replication infrastructure
• System prevents routing inconsistency during transitions

---

Defaults and Compatibility

Item	Behavior
Default value	number_of_virtual_shards = null
Existing indices	No change
Backward compatibility	Full
Client-facing API changes	None

---

Observability

• Optional: Add vShard ID to slow logs and request tracing
• Expose /vshard/stats for introspection and tuning
• Enable future integrations with load balancers and allocators

---

Clarifying Relationship to Dynamic Shard Splitting and Shrinking

This proposal for Virtual Shards is not competing with dynamic shard splitting; it operates at a different layer and addresses a different set of challenges. The two proposals are complementary and can co-exist to enhance OpenSearch’s flexibility.

Feature	Dynamic Shard Split/Shrink	Virtual Shards (This RFC)
Unit of Change	Physical Shard	Logical Virtual Shard (vShard)
Purpose	Change physical shard count post-creation	Introduce logical indirection layer for routing flexibility
Operation Cost	Involves data copying / merges / reindexing	Lightweight remapping + selective relocation
Target Use Cases	Undersharded or oversharded static indices	Hotspot isolation, temporal skew, multi-tenant scaling
Reindexing Required?	Yes (implicitly or explicitly)	No
Long-Running Index Support	Limited (splits are disruptive)	Yes (vShards remap live)
Routing Granularity	Physical shard	Logical vShard (e.g., 1024 vShards → 4 shards)

---

Summary

• Shard split/shrink is ideal for permanent, structural changes to physical shard layout post-index-creation.
• Virtual Shards provide fine-grained routing flexibility and live rebalancing for dynamic or long-lived workloads — without requiring index restarts or client changes.

---

Integration Potential

Virtual Shards can complement dynamic shard splitting:

• You could split an overloaded shard and then rebalance only its affected virtual shards across new shards.
• Or use virtual shards to delay or avoid physical splits by remapping hotspots to different nodes.

We’re happy to align these mechanisms so they serve orthogonal purposes:

• Shard splitting: structural, coarse-grained change
• Virtual shards: logical, fine-grained elastic routing

Both together enable OpenSearch to scale cleanly across a broader set of real-world indexing patterns.

Final Notes

Virtual shards bring elastic scaling, dynamic isolation, and long-term adaptability to OpenSearch. They are designed to solve real-world challenges like tenant hotspots, temporal skew, and uneven document growth — while keeping the system stable, simple, and compatible.

They integrate seamlessly with existing indexing and search infrastructure, and open up new capabilities without introducing client complexity.

This RFC proposes making virtual sharding a core abstraction within OpenSearch’s routing and indexing path.

Related component

ShardManagement:Placement

Describe alternatives you've considered

No response

Additional context

No response

[vikasvb90]

@atris I have few doubts around your approach

remap only impacted vShards

What exactly does this mean? Does it mean rehashing all documents of the currently mapped physical shard to multiple virtual shards within the set of virtual shards? If yes, then I think it is no longer a lightweight process. In this approach you don't really need virtual shards. Based on your answer I have some follow up questions.

Document ids are not always controlled by OpenSearch. If you are thinking of dynamically re-assigning a physical shard to a different virtual shard then how do search or get queries for a document work since documents will no longer be found in the newly mapped physical shard. You mentioned that you will only keep mapping of virtual to physical shard but that doesn't explain how a document routing for a "remapped" shard would work. If you plan to fan out such queries to all physical shards within a virtual shard group then it does impact search queries by a great extent.

Add load-aware background balancer (pluggable)

How does this fit specifically for scaling and not in general for other hot shards?

Multi-tenant use cases where rollover is not an option get into uneven shards problem on an index. This is a known problem in OpenSearch. We have opened a RFC earlier where we discussed some approaches where one of the approaches was to bring up empty shards within the same key space but we found that search latencies were heavily degraded in this approach.
There's also a design out to split the shard in-place i.e. without blocking any write traffic which caters to DocRep, SegRep and SegRep+Remote store clusters. #13923

[vigyasharma]

Interesting idea. I'm curious about a few details. First, to make sure I understood correctly, documents are mapped to a virtual shard via their routing hash. And virtual shards are mapped to physical shards by some intelligent placement logic.
So in an index with 4 shards and 1024 vShards, each document goes to hash(routing_key) % num_vshards, but a single physical shard may have more or less than 256 vShards. And cluster state holds specific vShard -> physicalShard mapping.

Since documents are still hashed uniformly across vShards, how does this setup provide temporal locality or hotspot isolation? As a concrete example, say my document routing field was a timestamp. My docs would still get spread across all the vShards. How do we ensure that docs close to each other (temporally or logically) go to the same vShard? Also since docs are spread uniformly, why would only a specific vShard be hot?

To be clear, I think that uniform distribution is important. Otherwise you can have a hot potato (single hot vShard) that no one can hold. What is not obvious to me, is how temporal locality or hotspot isolation emerge from this setup.

...

The plan to "move" a vShard is to "Identify and copy documents that hash to that vShard". It would help to get more details on the copy process? Say I update my cluster state to map vShard:426 from shard:0 to shard:3, which is placed on a different node. How do we move the slice of docs that belong to vShard:426?

Would we reindex them to a different vShard using the _source field? In my experience, segment copy during shard relocation is usually not the bottleneck. Things get stuck during translog replay, which can sometimes take forever to catch up. If "copying" documents involves reindexing docs into target shard, I would worry that we are trading away a simpler process (segment copy) for a heavier one (reindex).

As an extension to above, how does scale out work when we add a physical shard? Also, what happens if we didn't keep the source field?

...

On second thoughts, is the idea here that each virtual shard is internally a lucene index? That would explain the ability to relocate segments of just the virtual shard, and scale out dynamically when physical shards are added. And I suppose you get finer control for hotspots, though I'm not sure about temporal/logical locality. However, if they are separate Lucene indexes, then why not just create a higher number of shards in the first place?

[atris]

@vikasvb90

Thanks for the detailed questions — let me clarify each of these.

“remap only impacted vShards” — What exactly does this mean? Does it imply rehashing documents of a currently mapped physical shard to multiple vShards?

No, we don’t rehash or move any existing documents. “Remap” here refers to updating the virtual-to-physical shard routing for future traffic only. It’s a metadata-level change that shifts where incoming writes and queries for a vShard go — without altering anything already indexed. So this avoids reindexing entirely.

“Document IDs are not always controlled by OpenSearch. If you’re dynamically re-assigning physical shards, how will search or get work for old documents?”

This depends on routing mode. When vShard-based routing is explicitly used by the client, the mapping ensures GETs and queries are routed to the correct shard. If the client doesn’t provide that, fallback options exist:
• For a short period after remap, we can allow bounded fan-out reads across the affected shard group (e.g., just 2 or 3 shards max).
• Alternatively, clients can dual-route during the remap window if stronger consistency is needed.

This is a configurable tradeoff depending on use case — not a mandatory system-wide fan-out.

“How does the load-aware background balancer fit specifically for scaling and not general hot shards?”

This is tied to routing scalability, not internal shard health. It operates at the vShard level and remaps high-QPS vShards to cooler shards in the pool. Unlike traditional rebalancing, which works on actual shard movement, this changes traffic flow at a logical level without I/O overhead.

It’s meant for dynamic multi-tenant or high-ingestion environments where traffic hotspots shift over time and the underlying shard layout is static.

“We explored empty shards within a keyspace but faced degraded latencies. How is this different?”

In that approach, GETs and writes need to search across multiple physical shards permanently for the same keyspace. Virtual shards don’t do that. We maintain a one-to-one mapping from vShard to shard, and remap only when needed.

Fan-out (if any) is localized and temporary, used only in narrow edge cases like non-routed GETs during a remap window.

“There’s a design to split shards in-place — how is this different?”

It’s complementary. That work addresses physical shard size, Lucene doc count limits, and storage constraints. Virtual shards focus purely on traffic-level imbalance.

Put simply:
• In-place split solves data allocation and Lucene scaling.
• Virtual shards solve routing flexibility and ingest/query scaling.

You can use both. For example, a shard might first be load-shed via vShard remap, and later physically split if it outgrows its size limits.

Virtual shards don’t solve historical imbalance. They’re meant to prevent pressure from accumulating. A typical pattern is: a high-write customer or key gets mapped to a vShard. If that vShard starts causing load on its current shard, we remap it. All new traffic gets routed elsewhere, and the existing data stays untouched.

This gives operational flexibility in environments where write or query pressure shifts fast, and you want to scale out without blocking ingestion or doing full index splits.

[atris]

@vigyasharma

Thanks for the thoughtful questions — I’ll walk through them point by point.

So in an index with 4 shards and 1024 vShards, each document goes to hash(routing_key) % num_vshards, but a single physical shard may have more or less than 256 vShards. And cluster state holds specific vShard -> physicalShard mapping.

Yes, that's correct. Routing is via hash(routing_key) % vShardCount, and the virtual→physical shard mapping is centrally maintained and pluggable.

---

How does this setup provide temporal locality or hotspot isolation?

It doesn’t out of the box — virtual sharding decouples routing from physical placement, but doesn’t enforce locality. However, it enables it. For example, if you define a routing key like user_id + timestamp_day, documents from a temporal window will hash into a predictable vShard bucket. That vShard can be placed together on a physical shard (or moved later).

Even if routing is uniformly hashed, access patterns are rarely uniform. A single vShard can become hot based on query behavior (e.g., recent files, popular customers, etc.). So hotspot isolation is enabled not by tight locality, but by placement control at the vShard level — something not possible today without reindexing.

---

How would remapping work? Say vShard:426 is moved from shard:0 to shard:3.

We don’t reindex from _source. We copy documents from shard:0 to shard:3 using segment-level iteration based on vShard ID, or metadata-based filtering if available. Once the copy is done and acknowledged, the mapping is updated, and shard:0 drops its local copy.

This avoids translog replay and full shard relocation, which is part of the motivation here.

---

What if we didn’t store the _source field?

As mentioned above, reindexing from _source is a fallback, not the default. The system will use segment-level iteration where possible (based on vShard mod hash), or leverage retained doc metadata. If neither is possible, fallback to _source is used as a last resort.

---

What happens during scale-out?

When a new physical shard is added, we update the vShard → physicalShard mapping to shift some vShards to the new destination. Documents corresponding to those vShards are then migrated incrementally, using the approach above. No rehashing or index duplication is needed.

---

Are vShards just tiny Lucene indexes?

No. Virtual shards are not separate Lucene indexes — they are logical routing units. Physical shards still contain full Lucene segments. This avoids inflating cluster state or overloading file handles. The abstraction sits above the Lucene layer and offers lightweight control over placement and load distribution.

---

Why not just start with 1024 physical shards?

Because shard inflation leads to high cluster state overhead, wasted resources, poor segment merging behavior, and operational pain. The point of vShards is to get the flexibility of high routing cardinality without the cost of large shard counts. It also allows us to add/move shards without needing a full reindex or _split.

---

Note: This layer is not meant to replace existing shard-splitting infrastructure. It's complementary — useful in cases where workload evolution or routing imbalance occurs over time, and where reindexing or write-blocking aren't viable. It also opens up future directions like shard-as-a-function or tenant-aware placement strategies that are currently harder to implement cleanly.

[vikasvb90]

@atris Your reply clarified some doubts on how you are planning to use virtual sharding. I am still not clear on how you are avoiding the need of scanning all shards in a virtual shard group in get or update queries.

This depends on routing mode. When vShard-based routing is explicitly used by the client, the mapping ensures GETs and queries are routed to the correct shard. If the client doesn’t provide that, fallback options exist:
• For a short period after remap, we can allow bounded fan-out reads across the affected shard group (e.g., just 2 or 3 shards max).

In "vshard" mode, can you explain with an example of a doc ID of an ingested document in a virtual shard which was earlier mapped to a different physical shard, how a get or update request for a doc ID will be served successfully without full scan of shards within the virtual shard group?

[atris]

This hinges on how routing is handled in vshard mode.

Let’s take an example:

• You have an index with 512 virtual shards (vShards), and they are initially distributed across 4 physical shards (pShards), say p0 to p3.
• Document with routing key "customer123-day20250724" hashes to vShard 426, which is initially mapped to pShard p1.

Ingestion

• At ingestion, doc ID abc123 with routing key "customer123-day20250724" is routed to vShard 426 → pShard p1. The doc is stored in p1.

Remap event

• Later, due to load balancing, vShard 426 is remapped from p1 → p3. A background copy moves the documents associated with vShard 426 from p1 to p3.
• Once the migration completes, the central vShard→pShard mapping is updated to reflect that 426 now lives on p3.

GET / UPDATE in vshard routing mode

• A GET or UPDATE for abc123 with the same routing key will compute vShard 426 and consult the latest mapping.
• Since vShard 426 now maps to p3, the request is directly routed to p3 — no fan-out happens.
• The document is found on the correct shard.

GET / UPDATE without routing key

• If the client omits the routing key (not recommended in vshard mode), we can't compute the vShard, so fallback is needed.
• In that case, bounded fan-out to the small subset of shards involved in recent vShard remaps (typically just 1–2) is used.
• This fallback window is time-bounded and can be eliminated once client adoption of routing is high.

Summary

As long as clients provide the routing key (same as during indexing), the vShard hash is stable, and lookups are routed precisely to the current physical shard. There’s no need to scan all physical shards in the vShard group. This is why we emphasize vshard mode assumes routing is part of the request path — similar to how parent/child joins or custom routing work today.

[vikasvb90]

Got it! So basically even though mapping has changed, documents are still present in the mapped shard so that solve the document routing problem.
Also, I think it is a known fact that if ingestion happened via a particular routing key then search should also happen via same. Otherwise, it is not meant to work. That is not really a concern.

I have follow up questions, pardon me for asking too many questions.

1. How are you accomplishing the following since p3 is an existing physical shard?

Later, due to load balancing, vShard 426 is remapped from p1 → p3. A background copy moves the documents associated with vShard 426 from p1 to p3.

2. As per your explanation there were some different set of virtual shards pointing to p3 so my next question is what happens if high traffic now switches from p1 to p3?

[atris]

Thanks for the questions — clarifying below. Please keep them coming

1. How are you accomplishing the following since p3 is an existing physical shard?
   "A background copy moves the documents associated with vShard 426 from p1 → p3."

We're not injecting documents directly into p3’s existing Lucene index structure. Instead, physical shards are treated as multi-tenant containers, where each virtual shard is logically isolated. Internally, a vShard can correspond to a bounded doc space (e.g., keyed by a _vshard_id) or routed into a dedicated sub-index abstraction within the physical shard.

When vShard 426 is reassigned from p1 to p3, the background copy process selectively transfers only the documents tagged to vShard 426. These are appended into the isolated subspace on p3. There’s no merging of unrelated segments or mutation of p3’s other vShards.

Query routing resolves vshard → physical shard → logical space, so the subspace remains addressable post-remap.

---

2. What happens if high traffic now switches from p1 to p3?

That’s expected — when a hot vShard is remapped to p3, its traffic follows. If p3 is already serving other active vShards, we might shift imbalance, not remove it.

To address this:

1. Placement decisions take into account physical shard utilization. If p3 is already under load, it’s excluded from remap targets.
2. Remap granularity helps. Since vShards are small, we can relocate others away from p3 to even out the distribution, without needing to move entire shards.

The model doesn't just react to hotspots — it gives us precise control over them. Instead of moving entire shards, we can rebalance fine-grained units (vShards) with minimal disruption, enabling faster recovery and sustained cluster stability.

Happy to expand on copy coordination or shard state tracking if helpful.

[kkewwei]

I'm not sure if I understand this correctly:

1. During the remapping, we will copy segments to the new physical shard. If a newly generated physical shard is composed of multiple virtual shards, and these virtual shards may be distributed across all old physical shards, will we then need to copy segments from all old physical shards to this new physical shard?
2. Is there another use case involving dynamic shard expansion without _id data being written?

[atris]

@kkewwei Thanks for looking.

1. We never remap multiple vShards at once into a single new physical shard, so we avoid multi-source segment copying. A vShard is always remapped in isolation, ensuring copy is single-source → single-target, keeping the process bounded and predictable.
   2. On dynamic expansion without primary-key routing: yes, this is common in multi-tenant indexes where tenant IDs aren’t used as explicit routing keys. vShard indirection allows the system to rebalance traffic or isolate hotspots at a finer granularity even when client-side routing isn’t used. This is one of the key motivations behind the design.