Skip to content

Commit 8f0c042

Browse files
committed
fix: Updated readme list of supported stores
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
1 parent 18ede88 commit 8f0c042

File tree

10 files changed

+313
-16
lines changed

10 files changed

+313
-16
lines changed

README.md

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -182,32 +182,51 @@ The list below contains the functionality that contributors are planning to deve
182182
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/data-sources/postgres)
183183
* [x] [Spark (contrib plugin)](https://docs.feast.dev/reference/data-sources/spark)
184184
* [x] [Couchbase (contrib plugin)](https://docs.feast.dev/reference/data-sources/couchbase)
185+
* [x] [Athena (contrib plugin)](https://docs.feast.dev/reference/data-sources/athena)
186+
* [x] [Clickhouse (contrib plugin)](https://docs.feast.dev/reference/data-sources/clickhouse)
187+
* [x] [Oracle (contrib plugin)](https://docs.feast.dev/reference/data-sources/oracle)
185188
* [x] Kafka / Kinesis sources (via [push support into the online store](https://docs.feast.dev/reference/data-sources/push))
186189
* **Offline Stores**
187190
* [x] [Snowflake](https://docs.feast.dev/reference/offline-stores/snowflake)
188191
* [x] [Redshift](https://docs.feast.dev/reference/offline-stores/redshift)
189192
* [x] [BigQuery](https://docs.feast.dev/reference/offline-stores/bigquery)
190-
* [x] [Azure Synapse + Azure SQL (contrib plugin)](https://docs.feast.dev/reference/offline-stores/mssql.md)
193+
* [x] [DuckDB](https://docs.feast.dev/reference/offline-stores/duckdb)
194+
* [x] [Dask](https://docs.feast.dev/reference/offline-stores/dask)
195+
* [x] [Remote](https://docs.feast.dev/reference/offline-stores/remote-offline-store)
196+
* [x] [Azure Synapse + Azure SQL (contrib plugin)](https://docs.feast.dev/reference/offline-stores/mssql)
191197
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
192198
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/offline-stores/postgres)
193-
* [x] [Trino (contrib plugin)](https://github.com/Shopify/feast-trino)
199+
* [x] [Trino (contrib plugin)](https://docs.feast.dev/reference/offline-stores/trino)
194200
* [x] [Spark (contrib plugin)](https://docs.feast.dev/reference/offline-stores/spark)
195201
* [x] [Couchbase (contrib plugin)](https://docs.feast.dev/reference/offline-stores/couchbase)
196-
* [x] [In-memory / Pandas](https://docs.feast.dev/reference/offline-stores/file)
202+
* [x] [Athena (contrib plugin)](https://docs.feast.dev/reference/offline-stores/athena)
203+
* [x] [Clickhouse (contrib plugin)](https://docs.feast.dev/reference/offline-stores/clickhouse)
204+
* [x] [Ray (contrib plugin)](https://docs.feast.dev/reference/offline-stores/ray)
205+
* [x] [Oracle (contrib plugin)](https://docs.feast.dev/reference/offline-stores/oracle)
197206
* [x] [Custom offline store support](https://docs.feast.dev/how-to-guides/customizing-feast/adding-a-new-offline-store)
198207
* **Online Stores**
199208
* [x] [Snowflake](https://docs.feast.dev/reference/online-stores/snowflake)
200209
* [x] [DynamoDB](https://docs.feast.dev/reference/online-stores/dynamodb)
201210
* [x] [Redis](https://docs.feast.dev/reference/online-stores/redis)
211+
* [x] [Dragonfly](https://docs.feast.dev/reference/online-stores/dragonfly)
202212
* [x] [Datastore](https://docs.feast.dev/reference/online-stores/datastore)
203213
* [x] [Bigtable](https://docs.feast.dev/reference/online-stores/bigtable)
204214
* [x] [SQLite](https://docs.feast.dev/reference/online-stores/sqlite)
205-
* [x] [Dragonfly](https://docs.feast.dev/reference/online-stores/dragonfly)
215+
* [x] [Remote](https://docs.feast.dev/reference/online-stores/remote)
216+
* [x] [Postgres](https://docs.feast.dev/reference/online-stores/postgres)
217+
* [x] [HBase](https://docs.feast.dev/reference/online-stores/hbase)
218+
* [x] [Cassandra / AstraDB](https://docs.feast.dev/reference/online-stores/cassandra)
219+
* [x] [ScyllaDB](https://docs.feast.dev/reference/online-stores/scylladb)
220+
* [x] [MySQL](https://docs.feast.dev/reference/online-stores/mysql)
221+
* [x] [Hazelcast](https://docs.feast.dev/reference/online-stores/hazelcast)
222+
* [x] [Elasticsearch](https://docs.feast.dev/reference/online-stores/elasticsearch)
223+
* [x] [SingleStore](https://docs.feast.dev/reference/online-stores/singlestore)
224+
* [x] [Couchbase](https://docs.feast.dev/reference/online-stores/couchbase)
225+
* [x] [MongoDB](https://docs.feast.dev/reference/online-stores/mongodb)
226+
* [x] [Qdrant (vector store)](https://docs.feast.dev/reference/online-stores/qdrant)
227+
* [x] [Milvus (vector store)](https://docs.feast.dev/reference/online-stores/milvus)
228+
* [x] [Faiss (vector store)](https://docs.feast.dev/reference/online-stores/faiss)
206229
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
207-
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/online-stores/postgres)
208-
* [x] [Cassandra / AstraDB (contrib plugin)](https://docs.feast.dev/reference/online-stores/cassandra)
209-
* [x] [ScyllaDB (contrib plugin)](https://docs.feast.dev/reference/online-stores/scylladb)
210-
* [x] [Couchbase (contrib plugin)](https://docs.feast.dev/reference/online-stores/couchbase)
211230
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/customizing-feast/adding-support-for-a-new-online-store)
212231
* **Feature Engineering**
213232
* [x] On-demand Transformations (On Read) (Beta release. See [RFC](https://docs.google.com/document/d/1lgfIw0Drc65LpaxbUu49RCeJgMew547meSJttnUqz7c/edit#))

docs/SUMMARY.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,8 @@
9999
* [Azure Synapse + Azure SQL (contrib)](reference/data-sources/mssql.md)
100100
* [Couchbase (contrib)](reference/data-sources/couchbase.md)
101101
* [Oracle (contrib)](reference/data-sources/oracle.md)
102+
* [Athena (contrib)](reference/data-sources/athena.md)
103+
* [Clickhouse (contrib)](reference/data-sources/clickhouse.md)
102104
* [Offline stores](reference/offline-stores/README.md)
103105
* [Overview](reference/offline-stores/overview.md)
104106
* [Dask](reference/offline-stores/dask.md)
@@ -114,6 +116,7 @@
114116
* [Clickhouse (contrib)](reference/offline-stores/clickhouse.md)
115117
* [Ray (contrib)](reference/offline-stores/ray.md)
116118
* [Oracle (contrib)](reference/offline-stores/oracle.md)
119+
* [Athena (contrib)](reference/offline-stores/athena.md)
117120
* [Remote Offline](reference/offline-stores/remote-offline-store.md)
118121
* [Online stores](reference/online-stores/README.md)
119122
* [Overview](reference/online-stores/overview.md)
@@ -126,6 +129,7 @@
126129
* [Bigtable](reference/online-stores/bigtable.md)
127130
* [Remote](reference/online-stores/remote.md)
128131
* [PostgreSQL](reference/online-stores/postgres.md)
132+
* [HBase](reference/online-stores/hbase.md)
129133
* [Cassandra + Astra DB](reference/online-stores/cassandra.md)
130134
* [Couchbase](reference/online-stores/couchbase.md)
131135
* [MySQL](reference/online-stores/mysql.md)
@@ -134,6 +138,9 @@
134138
* [SingleStore](reference/online-stores/singlestore.md)
135139
* [Milvus](reference/online-stores/milvus.md)
136140
* [MongoDB](reference/online-stores/mongodb.md)
141+
* [Elasticsearch](reference/online-stores/elasticsearch.md)
142+
* [Qdrant](reference/online-stores/qdrant.md)
143+
* [Faiss](reference/online-stores/faiss.md)
137144
* [Registries](reference/registries/README.md)
138145
* [Local](reference/registries/local.md)
139146
* [S3](reference/registries/s3.md)

docs/reference/data-sources/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,10 @@ Please see [Data Source](../../getting-started/concepts/data-ingestion.md) for a
5858
[clickhouse.md](clickhouse.md)
5959
{% endcontent-ref %}
6060

61+
{% content-ref url="athena.md" %}
62+
[athena.md](athena.md)
63+
{% endcontent-ref %}
64+
6165
{% content-ref url="oracle.md" %}
6266
[oracle.md](oracle.md)
6367
{% endcontent-ref %}
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Athena source (contrib)
2+
3+
## Description
4+
5+
Athena data sources are AWS Athena tables or views.
6+
These can be specified either by a table reference or a SQL query.
7+
8+
## Disclaimer
9+
10+
The Athena data source does not achieve full test coverage.
11+
Please do not assume complete stability.
12+
13+
## Examples
14+
15+
Defining an Athena source:
16+
17+
```python
18+
from feast.infra.offline_stores.contrib.athena_offline_store.athena_source import (
19+
AthenaSource,
20+
)
21+
22+
driver_stats_source = AthenaSource(
23+
name="driver_hourly_stats",
24+
table="driver_hourly_stats",
25+
database="my_database",
26+
data_source="AwsDataCatalog",
27+
timestamp_field="event_timestamp",
28+
created_timestamp_column="created",
29+
)
30+
```
31+
32+
The full set of configuration options is available [here](https://rtd.feast.dev/en/master/#feast.infra.offline_stores.contrib.athena_offline_store.athena_source.AthenaSource).
33+
34+
## Supported Types
35+
36+
Athena data sources support standard Athena types mapped through the AWS Athena API.
37+
For a comparison against other batch data sources, please see [here](overview.md#functionality-matrix).

docs/reference/offline-stores/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,15 @@ Please see [Offline Store](../../getting-started/components/offline-store.md) fo
5353
{% content-ref url="oracle.md" %}
5454
[oracle.md](oracle.md)
5555
{% endcontent-ref %}
56+
57+
{% content-ref url="athena.md" %}
58+
[athena.md](athena.md)
59+
{% endcontent-ref %}
60+
61+
{% content-ref url="clickhouse.md" %}
62+
[clickhouse.md](clickhouse.md)
63+
{% endcontent-ref %}
64+
65+
{% content-ref url="remote-offline-store.md" %}
66+
[remote-offline-store.md](remote-offline-store.md)
67+
{% endcontent-ref %}
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Athena offline store (contrib)
2+
3+
## Description
4+
5+
The Athena offline store provides support for reading [AthenaSources](../data-sources/athena.md).
6+
* Entity dataframes can be provided as a SQL query or can be provided as a Pandas dataframe.
7+
8+
## Disclaimer
9+
10+
The Athena offline store does not achieve full test coverage.
11+
Please do not assume complete stability.
12+
13+
## Getting started
14+
In order to use this offline store, you'll need to run `pip install 'feast[aws]'`.
15+
16+
## Example
17+
18+
{% code title="feature_store.yaml" %}
19+
```yaml
20+
project: my_project
21+
registry: data/registry.db
22+
provider: local
23+
offline_store:
24+
type: athena
25+
data_source: AwsDataCatalog
26+
region: us-east-1
27+
database: my_database
28+
workgroup: primary
29+
online_store:
30+
path: data/online_store.db
31+
```
32+
{% endcode %}
33+
34+
The full set of configuration options is available in [AthenaOfflineStoreConfig](https://rtd.feast.dev/en/master/#feast.infra.offline_stores.contrib.athena_offline_store.athena.AthenaOfflineStoreConfig).
35+
36+
## Functionality Matrix
37+
38+
The set of functionality supported by offline stores is described in detail [here](overview.md#functionality).
39+
Below is a matrix indicating which functionality is supported by the Athena offline store.
40+
41+
| | Athena |
42+
| :----------------------------------------------------------------- |:-------|
43+
| `get_historical_features` (point-in-time correct join) | yes |
44+
| `pull_latest_from_table_or_query` (retrieve latest feature values) | yes |
45+
| `pull_all_from_table_or_query` (retrieve a saved dataset) | yes |
46+
| `offline_write_batch` (persist dataframes to offline store) | no |
47+
| `write_logged_features` (persist logged features to offline store) | yes |
48+
49+
Below is a matrix indicating which functionality is supported by `AthenaRetrievalJob`.
50+
51+
| | Athena |
52+
| ----------------------------------------------------- |--------|
53+
| export to dataframe | yes |
54+
| export to arrow table | yes |
55+
| export to arrow batches | no |
56+
| export to SQL | yes |
57+
| export to data lake (S3, GCS, etc.) | no |
58+
| export to data warehouse | no |
59+
| export as Spark dataframe | no |
60+
| local execution of Python-based on-demand transforms | yes |
61+
| remote execution of Python-based on-demand transforms | no |
62+
| persist results in the offline store | yes |
63+
| preview the query plan before execution | yes |
64+
| read partitioned data | yes |
65+
66+
To compare this set of functionality against other offline stores, please see the full [functionality matrix](overview.md#functionality-matrix).

docs/reference/online-stores/README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,10 @@ Please see [Online Store](../../getting-started/components/online-store.md) for
3838
[postgres.md](postgres.md)
3939
{% endcontent-ref %}
4040

41+
{% content-ref url="hbase.md" %}
42+
[hbase.md](hbase.md)
43+
{% endcontent-ref %}
44+
4145
{% content-ref url="cassandra.md" %}
4246
[cassandra.md](cassandra.md)
4347
{% endcontent-ref %}
@@ -69,3 +73,19 @@ Please see [Online Store](../../getting-started/components/online-store.md) for
6973
{% content-ref url="singlestore.md" %}
7074
[singlestore.md](singlestore.md)
7175
{% endcontent-ref %}
76+
77+
{% content-ref url="elasticsearch.md" %}
78+
[elasticsearch.md](elasticsearch.md)
79+
{% endcontent-ref %}
80+
81+
{% content-ref url="qdrant.md" %}
82+
[qdrant.md](qdrant.md)
83+
{% endcontent-ref %}
84+
85+
{% content-ref url="milvus.md" %}
86+
[milvus.md](milvus.md)
87+
{% endcontent-ref %}
88+
89+
{% content-ref url="faiss.md" %}
90+
[faiss.md](faiss.md)
91+
{% endcontent-ref %}
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Faiss online store
2+
3+
## Description
4+
5+
The [Faiss](https://github.com/facebookresearch/faiss) online store provides support for materializing feature values and performing vector similarity search using Facebook AI Similarity Search (Faiss). Faiss is a library for efficient similarity search and clustering of dense vectors, making it well-suited for use cases involving embeddings and nearest-neighbor lookups.
6+
7+
## Getting started
8+
In order to use this online store, you'll need to install the Faiss dependency. E.g.
9+
10+
`pip install 'feast[faiss]'`
11+
12+
## Example
13+
14+
{% code title="feature_store.yaml" %}
15+
```yaml
16+
project: my_feature_repo
17+
registry: data/registry.db
18+
provider: local
19+
online_store:
20+
type: feast.infra.online_stores.faiss_online_store.FaissOnlineStore
21+
dimension: 128
22+
index_path: data/faiss_index
23+
index_type: IVFFlat # optional, default: IVFFlat
24+
nlist: 100 # optional, default: 100
25+
```
26+
{% endcode %}
27+
28+
**Note:** Faiss is not registered as a named online store type. You must use the fully qualified class path as the `type` value.
29+
30+
The full set of configuration options is available in [FaissOnlineStoreConfig](https://rtd.feast.dev/en/master/#feast.infra.online_stores.faiss_online_store.FaissOnlineStoreConfig).
31+
32+
## Functionality Matrix
33+
34+
The set of functionality supported by online stores is described in detail [here](overview.md#functionality).
35+
Below is a matrix indicating which functionality is supported by the Faiss online store.
36+
37+
| | Faiss |
38+
|:----------------------------------------------------------|:------|
39+
| write feature values to the online store | yes |
40+
| read feature values from the online store | yes |
41+
| update infrastructure (e.g. tables) in the online store | yes |
42+
| teardown infrastructure (e.g. tables) in the online store | yes |
43+
| generate a plan of infrastructure changes | no |
44+
| support for on-demand transforms | yes |
45+
| readable by Python SDK | yes |
46+
| readable by Java | no |
47+
| readable by Go | no |
48+
| support for entityless feature views | yes |
49+
| support for concurrent writing to the same key | no |
50+
| support for ttl (time to live) at retrieval | no |
51+
| support for deleting expired data | no |
52+
| collocated by feature view | yes |
53+
| collocated by feature service | no |
54+
| collocated by entity key | no |
55+
| vector similarity search | yes |
56+
57+
To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix).
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# HBase online store
2+
3+
## Description
4+
5+
The [HBase](https://hbase.apache.org/) online store provides support for materializing feature values into an Apache HBase database for serving online features in real-time.
6+
7+
* Each feature view is mapped to an HBase table
8+
* Connects to HBase via the Thrift server using [happybase](https://happybase.readthedocs.io/)
9+
10+
## Getting started
11+
In order to use this online store, you'll need to run `pip install 'feast[hbase]'`.
12+
13+
## Example
14+
15+
{% code title="feature_store.yaml" %}
16+
```yaml
17+
project: my_feature_repo
18+
registry: data/registry.db
19+
provider: local
20+
online_store:
21+
type: hbase
22+
host: localhost
23+
port: "9090"
24+
connection_pool_size: 4 # optional
25+
protocol: binary # optional
26+
transport: buffered # optional
27+
```
28+
{% endcode %}
29+
30+
The full set of configuration options is available in [HbaseOnlineStoreConfig](https://rtd.feast.dev/en/master/#feast.infra.online_stores.hbase_online_store.hbase.HbaseOnlineStoreConfig).
31+
32+
## Functionality Matrix
33+
34+
The set of functionality supported by online stores is described in detail [here](overview.md#functionality).
35+
Below is a matrix indicating which functionality is supported by the HBase online store.
36+
37+
| | HBase |
38+
| :-------------------------------------------------------- | :---- |
39+
| write feature values to the online store | yes |
40+
| read feature values from the online store | yes |
41+
| update infrastructure (e.g. tables) in the online store | yes |
42+
| teardown infrastructure (e.g. tables) in the online store | yes |
43+
| generate a plan of infrastructure changes | no |
44+
| support for on-demand transforms | yes |
45+
| readable by Python SDK | yes |
46+
| readable by Java | no |
47+
| readable by Go | no |
48+
| support for entityless feature views | yes |
49+
| support for concurrent writing to the same key | no |
50+
| support for ttl (time to live) at retrieval | no |
51+
| support for deleting expired data | no |
52+
| collocated by feature view | yes |
53+
| collocated by feature service | no |
54+
| collocated by entity key | no |
55+
56+
To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix).

0 commit comments

Comments
 (0)