Skip to content

Commit 6f1b0cc

Browse files
franciscojavierarceodevin-ai-integration[bot]
franciscojavierarceo
and
devin-ai-integration[bot]
authored
feat: Adding permissions to UI and refactoring some things (feast-dev#5320)
* Feature: Add permissions display to Feast UI lineage visualization Co-Authored-By: Francisco Javier Arceo <[email protected]> * Format code with Prettier Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix TypeScript errors in getEntityPermissions function Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add permissions for zipcode_features, zipcode_source, and model_v1 Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add permissions page and display permissions on feature service page Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add mock permissions data for development Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add permissions display to feature view pages Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add permissions display to entity pages Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add permissions display to data source pages Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix permissions implementation with separate apply_permissions.py script Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Comment out NPM_TOKEN requirement in .npmrc for easier local development Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Add permissions display to features page and fix icon loading errors Co-Authored-By: Francisco Javier Arceo <[email protected]> * Update: EUI package to fix icon loading errors Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Remove theme import to resolve icon loading errors Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Update package dependencies to resolve icon loading errors Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add Home | Lineage link and update permission utils Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add RAG project with feature views and permissions Co-Authored-By: Francisco Javier Arceo <[email protected]> * Update feature_store.yaml for RAG project Co-Authored-By: Francisco Javier Arceo <[email protected]> * Update: Move Home and Lineage links to sidebar navigation Co-Authored-By: Francisco Javier Arceo <[email protected]> * Update: Remove Home | Lineage links from top of page Co-Authored-By: Francisco Javier Arceo <[email protected]> * Add: RAG data files for document embeddings and metadata Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Update sidebar navigation to show Home | Lineage as a single item Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Update Home | Lineage navigation to appear as a single line with two links Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Update sidebar navigation to fix React Hook error Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Update Lineage link to use feature service demo tab URL format Co-Authored-By: Francisco Javier Arceo <[email protected]> * Fix: Update Lineage link to properly redirect to the lineage tab Co-Authored-By: Francisco Javier Arceo <[email protected]> * Update: Add data files to .gitignore Co-Authored-By: Francisco Javier Arceo <[email protected]> * Update: Move Lineage to its own page under Resources in sidebar Co-Authored-By: Francisco Javier Arceo <[email protected]> * Update: Remove Home hyperlink and Lineage tab from home page Co-Authored-By: Francisco Javier Arceo <[email protected]> * Format: Run yarn format to ensure code follows project standards Co-Authored-By: Francisco Javier Arceo <[email protected]> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent d8a695b commit 6f1b0cc

28 files changed

+1073
-119
lines changed

‎ui/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
src/protos.d.ts
22
src/protos.js
3+
feature_repo/data/*.parquet

‎ui/.npmrc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
1+
# //registry.npmjs.org/:_authToken=${NPM_TOKEN}

‎ui/feature_repo/apply_permissions.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
from feast import FeatureStore
2+
from features import (
3+
zipcode_features_permission,
4+
zipcode_source_permission,
5+
model_v1_permission,
6+
risky_features_permission,
7+
document_embeddings_permission,
8+
document_metadata_permission,
9+
rag_model_permission,
10+
)
11+
12+
store = FeatureStore(repo_path=".")
13+
14+
store.apply([
15+
zipcode_features_permission,
16+
zipcode_source_permission,
17+
model_v1_permission,
18+
risky_features_permission,
19+
document_embeddings_permission,
20+
document_metadata_permission,
21+
rag_model_permission,
22+
])
23+
24+
print("Permissions applied successfully!")
25+
print("Current permissions:", store.list_permissions())

‎ui/feature_repo/apply_rag_data.py

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
import pandas as pd
2+
import numpy as np
3+
from datetime import datetime, timedelta
4+
5+
now = datetime.now()
6+
embeddings = []
7+
for i in range(10):
8+
embeddings.append({
9+
'document_id': f'doc_{i}',
10+
'embedding': np.random.rand(768).astype(np.float32),
11+
'event_timestamp': now - timedelta(days=i),
12+
'created_timestamp': now - timedelta(days=i, hours=1)
13+
})
14+
df_embeddings = pd.DataFrame(embeddings)
15+
df_embeddings.to_parquet('data/document_embeddings.parquet', index=False)
16+
17+
metadata = []
18+
for i in range(10):
19+
metadata.append({
20+
'document_id': f'doc_{i}',
21+
'title': f'Document {i}',
22+
'content': f'This is the content of document {i}',
23+
'source': 'web',
24+
'author': f'author_{i}',
25+
'publish_date': (now - timedelta(days=i*30)).strftime('%Y-%m-%d'),
26+
'event_timestamp': now - timedelta(days=i),
27+
'created_timestamp': now - timedelta(days=i, hours=1)
28+
})
29+
df_metadata = pd.DataFrame(metadata)
30+
df_metadata.to_parquet('data/document_metadata.parquet', index=False)
31+
32+
print('Created RAG data files successfully!')

‎ui/feature_repo/data/document_embeddings.parquet

45.8 KB
Binary file not shown.

‎ui/feature_repo/data/document_metadata.parquet

6.14 KB
Binary file not shown.

‎ui/feature_repo/feature_store.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@ online_store:
55
type: sqlite
66
offline_store:
77
type: file
8+
entity_key_serialization_version: 2

‎ui/feature_repo/features.py

Lines changed: 155 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
11
from datetime import timedelta
22

33
import pandas as pd
4+
import numpy as np
45

56
from feast import Entity, FeatureService, FeatureView, Field, FileSource
67
from feast.data_source import RequestSource
78
from feast.on_demand_feature_view import on_demand_feature_view
8-
from feast.types import Bool, Int64, String
9+
from feast.permissions.action import AuthzedAction, READ
10+
from feast.permissions.permission import Permission
11+
from feast.permissions.policy import RoleBasedPolicy
12+
from feast.types import Bool, Int64, String, Float32, Array
913

1014
zipcode = Entity(
1115
name="zipcode",
@@ -199,3 +203,153 @@ def transaction_gt_last_credit_card_due(inputs: pd.DataFrame) -> pd.DataFrame:
199203
tags={"owner": "[email protected]", "stage": "dev"},
200204
description="Location model",
201205
)
206+
207+
zipcode_features_permission = Permission(
208+
name="zipcode-features-reader",
209+
types=[FeatureView],
210+
name_patterns=["zipcode_features"],
211+
policy=RoleBasedPolicy(roles=["analyst", "data_scientist"]),
212+
actions=[AuthzedAction.DESCRIBE, *READ],
213+
)
214+
215+
zipcode_source_permission = Permission(
216+
name="zipcode-source-writer",
217+
types=[FileSource],
218+
name_patterns=["zipcode"],
219+
policy=RoleBasedPolicy(roles=["admin", "data_engineer"]),
220+
actions=[AuthzedAction.CREATE, AuthzedAction.UPDATE, AuthzedAction.WRITE_OFFLINE],
221+
)
222+
223+
model_v1_permission = Permission(
224+
name="credit-score-v1-reader",
225+
types=[FeatureService],
226+
name_patterns=["credit_score_v1"],
227+
policy=RoleBasedPolicy(roles=["model_user", "data_scientist"]),
228+
actions=[AuthzedAction.DESCRIBE, AuthzedAction.READ_ONLINE],
229+
)
230+
231+
risky_features_permission = Permission(
232+
name="risky-features-reader",
233+
types=[FeatureView, FeatureService],
234+
required_tags={"stage": "prod"},
235+
policy=RoleBasedPolicy(roles=["trusted_analyst"]),
236+
actions=[AuthzedAction.READ_OFFLINE],
237+
)
238+
239+
document = Entity(
240+
name="document_id",
241+
description="Document identifier for RAG system",
242+
tags={
243+
"owner": "[email protected]",
244+
"team": "rag",
245+
},
246+
)
247+
248+
document_source = FileSource(
249+
name="document_embeddings",
250+
path="data/document_embeddings.parquet",
251+
timestamp_field="event_timestamp",
252+
created_timestamp_column="created_timestamp",
253+
)
254+
255+
document_metadata_source = FileSource(
256+
name="document_metadata",
257+
path="data/document_metadata.parquet",
258+
timestamp_field="event_timestamp",
259+
created_timestamp_column="created_timestamp",
260+
)
261+
262+
document_embeddings_view = FeatureView(
263+
name="document_embeddings",
264+
entities=[document],
265+
ttl=timedelta(days=365),
266+
schema=[
267+
Field(name="embedding", dtype=Array(Float32, 768)),
268+
Field(name="document_id", dtype=String),
269+
],
270+
source=document_source,
271+
tags={
272+
"date_added": "2025-05-04",
273+
"model": "sentence-transformer",
274+
"access_group": "[email protected]",
275+
"stage": "prod",
276+
},
277+
online=True,
278+
)
279+
280+
document_metadata_view = FeatureView(
281+
name="document_metadata",
282+
entities=[document],
283+
ttl=timedelta(days=365),
284+
schema=[
285+
Field(name="title", dtype=String),
286+
Field(name="content", dtype=String),
287+
Field(name="source", dtype=String),
288+
Field(name="author", dtype=String),
289+
Field(name="publish_date", dtype=String),
290+
Field(name="document_id", dtype=String),
291+
],
292+
source=document_metadata_source,
293+
tags={
294+
"date_added": "2025-05-04",
295+
"access_group": "[email protected]",
296+
"stage": "prod",
297+
},
298+
online=True,
299+
)
300+
301+
# Define a request data source for query embeddings
302+
query_request = RequestSource(
303+
name="query",
304+
schema=[
305+
Field(name="query_embedding", dtype=Array(Float32, 768)),
306+
],
307+
)
308+
309+
# Define an on-demand feature view for similarity calculation
310+
@on_demand_feature_view(
311+
sources=[document_embeddings_view, query_request],
312+
schema=[
313+
Field(name="similarity_score", dtype=Float32),
314+
],
315+
)
316+
def document_similarity(inputs: pd.DataFrame) -> pd.DataFrame:
317+
"""Calculate cosine similarity between query and document embeddings."""
318+
df = pd.DataFrame()
319+
df["similarity_score"] = 0.95 # Placeholder value
320+
return df
321+
322+
rag_model = FeatureService(
323+
name="rag_retriever",
324+
features=[
325+
document_embeddings_view,
326+
document_metadata_view,
327+
document_similarity,
328+
],
329+
tags={"owner": "[email protected]", "stage": "prod"},
330+
description="Retrieval Augmented Generation model",
331+
)
332+
333+
document_embeddings_permission = Permission(
334+
name="document-embeddings-reader",
335+
types=[FeatureView],
336+
name_patterns=["document_embeddings"],
337+
policy=RoleBasedPolicy(roles=["ml_engineer", "data_scientist"]),
338+
actions=[AuthzedAction.DESCRIBE, *READ],
339+
)
340+
341+
document_metadata_permission = Permission(
342+
name="document-metadata-reader",
343+
types=[FeatureView],
344+
name_patterns=["document_metadata"],
345+
policy=RoleBasedPolicy(roles=["ml_engineer", "content_manager"]),
346+
actions=[AuthzedAction.DESCRIBE, *READ],
347+
)
348+
349+
rag_model_permission = Permission(
350+
name="rag-model-user",
351+
types=[FeatureService],
352+
name_patterns=["rag_retriever"],
353+
policy=RoleBasedPolicy(roles=["ml_engineer", "app_developer"]),
354+
actions=[AuthzedAction.DESCRIBE, AuthzedAction.READ_ONLINE],
355+
)

‎ui/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
"dependencies": {
2626
"@elastic/datemath": "^5.0.3",
2727
"@elastic/eui": "^95.12.0",
28+
"@elastic/eui-theme-borealis": "1.0.0",
2829
"@emotion/css": "^11.13.0",
2930
"@emotion/react": "^11.13.3",
3031
"@types/dagre": "^0.7.52",

‎ui/src/FeastUISansProviders.tsx

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
import React from "react";
22

3-
import "@elastic/eui/dist/eui_theme_light.css";
43
import "./index.css";
54

65
import { Routes, Route } from "react-router-dom";
@@ -23,6 +22,8 @@ import FeatureServiceInstance from "./pages/feature-services/FeatureServiceInsta
2322
import DataSourceInstance from "./pages/data-sources/DataSourceInstance";
2423
import RootProjectSelectionPage from "./pages/RootProjectSelectionPage";
2524
import DatasetInstance from "./pages/saved-data-sets/DatasetInstance";
25+
import PermissionsIndex from "./pages/permissions/Index";
26+
import LineageIndex from "./pages/lineage/Index";
2627
import NoProjectGuard from "./components/NoProjectGuard";
2728

2829
import TabsRegistryContext, {
@@ -144,6 +145,8 @@ const FeastUISansProvidersInner = ({
144145
path="data-set/:datasetName/*"
145146
element={<DatasetInstance />}
146147
/>
148+
<Route path="permissions/" element={<PermissionsIndex />} />
149+
<Route path="lineage/" element={<LineageIndex />} />
147150
</Route>
148151
</Route>
149152
<Route path="*" element={<NoMatch />} />

0 commit comments

Comments
 (0)