Skip to content

Commit a9c5c68

Browse files
authored
Merge pull request #256 from fbsolo-ms1/update-for-LT
Requested updates to cover CLI-related content . . .
2 parents 97b2eb5 + b30480f commit a9c5c68

File tree

2 files changed

+58
-7
lines changed

2 files changed

+58
-7
lines changed

articles/machine-learning/reference-yaml-feature-store.md

Lines changed: 54 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: reference
99
author: fbsolo-ms1
1010
ms.author: franksolomon
1111
ms.reviewer: qiax
12-
ms.date: 05/23/2023
12+
ms.date: 09/12/2024
1313
ms.custom: cliv2, build-2023
1414
---
1515

@@ -21,17 +21,19 @@ ms.custom: cliv2, build-2023
2121

2222
## YAML syntax
2323

24-
2524
| Key | Type | Description | Allowed values | Default value |
2625
|--|--|--|--|--|
2726
| $schema | string | The YAML schema. If you use the Azure Machine Learning VS Code extension to author the YAML file, including $schema at the top of your file enables you to invoke schema and resource completions. | | |
2827
| name | string | **Required.** Name of the feature store. | | |
2928
| compute_runtime | object | The compute runtime configuration used for materialization job. | | |
30-
| compute_runtime.spark_runtime_version | string | The Azure Machine Learning Spark runtime version. | 3.2 | 3.2 |
29+
| compute_runtime.spark_runtime_version | string | The Azure Machine Learning Spark runtime version. | 3.4 | 3.4 |
3130
| offline_store | object | | | |
3231
| offline_store.type | string | **Required** if offline_store is provided. The type of offline store. Only data lake gen2 type of storage is supported. | azure_data_lake_gen2 | |
3332
| offline_store.target | string | **Required** if offline_store is provided. The datalake Gen2 storage URI in the format of `/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.Storage/storageAccounts/<account>/blobServices/default/containers/<container>`. | | |
34-
| materialization_identity | object | The user-assigned managed identity that used for the materialization job. This identity needs to be granted necessary roles to access Feature Store service, the data source and the offline storage. | | |
33+
| online_store | object | | | |
34+
| online_store.type | string | **Required** if online_store is provided. The type of online store. Only redis cache is supported. | redis | |
35+
| online_store.target | string | **Required** if online_store is provided. The Redis Cache URI in the format of `/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.Cache/Redis/<redis-name>`. | | |
36+
| materialization_identity | object | The user-assigned managed identity that used for the materialization job. This identity needs to be granted necessary roles to access Feature Store service, the data source, and the offline storage. | | |
3537
| materialization_identity.client_id | string | The client ID for your user-assigned managed identity. | | |
3638
| materialization_identity.resource_id | string | The resource ID for your user-assigned managed identity. | | |
3739
| materialization_identity.principal_id | string | the principal ID for your user-assigned managed identity.| | |
@@ -48,7 +50,7 @@ You can include other [workspace properties](reference-yaml-workspace.md).
4850
The `az ml feature-store` command can be used for managing Azure Machine Learning feature store workspaces.
4951
## Examples
5052

51-
Examples are available in the [examples GitHub repository](https://github.com/Azure/azureml-examples/tree/main/cli). Some common examples are shown in below.
53+
Examples are available in the [examples GitHub repository](https://github.com/Azure/azureml-examples/tree/main/cli). Some common examples are shown here:
5254

5355
## YAML basic
5456

@@ -82,6 +84,53 @@ tags:
8284
foo: bar
8385
```
8486
87+
## Configure the online store in the CLI with YAML
88+
89+
```yaml
90+
$schema: http://azureml/sdk-2-0/FeatureStore.json
91+
name: mktg-feature-store
92+
93+
compute_runtime:
94+
spark_runtime_version: 3.4
95+
96+
online_store:
97+
type: redis
98+
target: "/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Cache/Redis/<redis-name>"
99+
100+
materialization_identity:
101+
client_id: 00001111-aaaa-2222-bbbb-3333cccc4444
102+
principal_id: aaaaaaaa-bbbb-cccc-1111-222222222222
103+
resource_id: /subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<uai-name>
104+
105+
# Many of workspace parameters will also be supported:
106+
location: eastus
107+
display_name: marketing feature store
108+
tags:
109+
foo: bar
110+
```
111+
112+
## Configure the online store in the CLI with Python
113+
114+
```python
115+
redis_arm_id = f"/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}/providers/Microsoft.Cache/Redis/{redis_name}"
116+
online_store = MaterializationStore(type="redis", target=redis_arm_id)
117+
118+
fs = FeatureStore(
119+
name=featurestore_name,
120+
location=location,
121+
online_store=online_store,
122+
)
123+
124+
# wait for feature store creation
125+
fs_poller = ml_client.feature_stores.begin_create(fs)
126+
127+
# move the feature store to a YAML file
128+
129+
yaml_path = root_dir + "/featurestore/featurestore_with_online.yaml"
130+
fs.dump(yaml_path)
131+
132+
```
133+
85134
## Next steps
86135

87136
- [Install and use the CLI (v2)](how-to-configure-cli.md)

articles/machine-learning/tutorial-online-materialization-inference.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,8 @@ The feature store needs the Azure Cache for Redis as an attached resource, for u
109109
> - Online store target
110110
> - Offline store target
111111
112+
For an example that shows how to do this with the SDK, visit the [Tutorial: Different Approaches for Provisioning a Managed Feature Store](https://github.com/Azure/azureml-examples/blob/main/sdk/python/featurestore_sample/notebooks/sdk_and_cli/4.Provision-feature-store.ipynb) resource.
113+
112114
## Materialize the `accounts` feature set data to online store
113115

114116
### Enable materialization on the `accounts` feature set
@@ -166,11 +168,11 @@ You can explore the feature materialization status for a feature set from the **
166168
- Your data can have a maximum of 2,000 *data intervals*. If your data contains more than 2,000 *data intervals*, create a new feature set version.
167169
- You can provide a list of more than one data statuses (for example, `["None", "Incomplete"]`) in a single backfill job.
168170
- During backfill, a new materialization job is submitted for each *data interval* that falls in the defined feature window.
169-
- A new job is not submitted for a *data interval* if a materialization job is already pending, or is running for a *data interval* that hasn't yet been backfilled.
171+
- A new job isn't submitted for a *data interval* if a materialization job is already pending, or is running for a *data interval* that hasn't yet been backfilled.
170172

171173
### Updating online materialization store
172174
- If an online materialization store is to be updated at the feature store level, then all feature sets in the feature store should have online materialization disabled.
173-
- If online materialization is disabled on a feature set, the materialization status of the already-materialized data in the online materialization store will be reset. This renders the already-materialized data unusable. You must resubmit your materialization jobs after you enable online materialization.
175+
- If online materialization is disabled on a feature set, the materialization status of the already-materialized data in the online materialization store is reset. This renders the already-materialized data unusable. You must resubmit your materialization jobs after you enable online materialization.
174176
- If only offline materialization was initially enabled for a feature set, and online materialization is enabled later:
175177
- The default data materialization status of the data in the online store will be `None`.
176178
- When the first online materialization job is submitted, the data already materialized in the offline store, if available, is used to calculate online features.

0 commit comments

Comments
 (0)