feat: Add HybridOfflineStore for supporting multiple offline store backends #5542
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
June 21, 2025 09:58
- Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView. - Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection. - Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment. - added documentation - fixed linter raised issues Signed-off-by: r0b0fyi <[email protected]>
- Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView. - Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection. - Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment. - added documentation - fixed linter raised issues Signed-off-by: r0b0fyi <[email protected]>
The HybridOfflineStore allows routing operations to different offline store implementations based on data source type. This enables a single Feast deployment to work with multiple offline storage backends simultaneously, improving flexibility for organizations with heterogeneous data infrastructure. Key features: - Configure multiple offline stores in a single Feast deployment - Automatic routing of operations to the appropriate offline store - Support for all existing offline store operations - Transparent to feature views and other Feast components Signed-off-by: r0b0fyi <[email protected]>
|
Really appreciate your contribution. I made the first version of the hybrid store and already in the codebase now. Do you mind taking a look see if there is something missing and you want to optimize? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
This PR adds a new HybridOfflineStore implementation that can route operations to different offline stores based on data source type. This enables organizations with heterogeneous data infrastructure to use a single Feast deployment while storing features in multiple different storage systems.
Key features:
Which issue(s) this PR fixes:
Fixes #5541
Misc
This implementation follows the existing patterns in Feast's offline store design. It introduces a new configuration type
HybridOfflineStoreConfigthat contains a list of offline store configurations.Testing: