Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions knowledge_base/app_with_database/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.databricks/
build/
dist/
__pycache__/
*.egg-info
.venv/
54 changes: 54 additions & 0 deletions knowledge_base/app_with_database/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Databricks app with OLTP database

This example demonstrates how to define a Databricks app backed by
an OLTP Postgres in a Databricks Asset Bundle.

It includes and deploys an example application that uses Python and Dash and a database instance.
When application is started it provisions its own schema and demonstration data in the OLTP database.

For more information about Databricks Apps see the [documentation](https://docs.databricks.com/aws/en/dev-tools/databricks-apps).
For more information about Databricks database instances see the [documentation](https://docs.databricks.com/aws/en/oltp/).

## Prerequisites

* Databricks CLI v0.267.0 or above

## Usage

1. Deploy the bundle:
```
databricks bundle deploy -t dev
```
Please note that after this bundle gets deployed, the database instance starts running immediately, which incurs cost.

2. Run the app:
```
databricks bundle run my_app
```

3. Open the app:
Run the following command to navigate to the deployed app in your browser:
```
databricks bundle open my_app
```

Alternatively, run `databricks bundle summary` to display its URL.

4. Query the app data:

Run the following command to display the data generated by the app:
```
databricks psql example-database-instance -- --dbname example_database -c "select * from holidays.holiday_requests"
```

5. Explore the app data:
Run the following command to navigate to the Unity Catalog in your browser:
```
databricks bundle open my_catalog
```

## Clean up
To remove the provisioned resources run
```
databricks bundle destroy
```
10 changes: 10 additions & 0 deletions knowledge_base/app_with_database/app/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from dash import Dash
from layout import make_app_layout
from callbacks import register_callbacks

app = Dash(__name__, title="Holiday Request Manager", suppress_callback_exceptions=True)
app.layout = make_app_layout()
register_callbacks(app)

if __name__ == "__main__":
app.run(debug=True)
1 change: 1 addition & 0 deletions knowledge_base/app_with_database/app/app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
command: ["python", "app.py"]
44 changes: 44 additions & 0 deletions knowledge_base/app_with_database/app/callbacks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
from dash import Input, Output, State, callback_context
from lakebase_connector import get_holiday_requests, update_request_status
import pandas as pd


def register_callbacks(app):
@app.callback(
Output("holiday-table", "data"),
[
Input("refresh-interval", "n_intervals"),
Input("submit-action", "n_clicks"),
Input("action-feedback", "children"),
],
)
def refresh_table(n_intervals, n_clicks, action_feedback):
holiday_request_df = get_holiday_requests()
return holiday_request_df.to_dict("records")

@app.callback(
[
Output("action-feedback", "children"),
Output("action-radio", "value"),
Output("manager-comment", "value"),
],
Input("submit-action", "n_clicks"),
State("holiday-table", "selected_rows"),
State("holiday-table", "data"),
State("action-radio", "value"),
State("manager-comment", "value"),
prevent_initial_call=True,
)
def submit_holiday_request_review(
n_clicks, selected_row_nr, table_data, selected_action, manager_comment
):
if (selected_action is None) | (selected_row_nr is None):
return "Please select a request and approve/decline."
row_idx = selected_row_nr[0]
selected_row = table_data[row_idx]
request_id = selected_row["request_id"]

update_request_status(
request_id=request_id, status=selected_action, comment=manager_comment
)
return f"You {selected_action.lower()} request {request_id}.", None, ""
94 changes: 94 additions & 0 deletions knowledge_base/app_with_database/app/lakebase_connector.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
import os

import pandas as pd
from databricks.sdk import WorkspaceClient
from sqlalchemy import create_engine, event, text

workspace_client = WorkspaceClient()
user = workspace_client.current_user.me().user_name

postgres_username = user
postgres_host = os.getenv("PGHOST")
postgres_port = os.getenv("PGPORT")
postgres_database = os.getenv("PGDATABASE")

print("postgres_username", postgres_username)
print("postgres_host", postgres_host)
print("postgres_port", postgres_port)
print("postgres_database", postgres_database)

postgres_pool = create_engine(
f"postgresql+psycopg://{postgres_username}:@{postgres_host}:{postgres_port}/{postgres_database}"
)


@event.listens_for(postgres_pool, "do_connect")
def provide_token(dialect, conn_rec, cargs, cparams):
"""Provide the App's OAuth token. Caching is managed by WorkspaceClient"""
cparams["password"] = workspace_client.config.oauth_token().access_token


def get_holiday_requests():
"""
Fetch all holiday requests from the database.

Returns:
pd.DataFrame: DataFrame containing holiday requests.
"""
df = pd.read_sql_query("SELECT * FROM holidays.holiday_requests;", postgres_pool)
return df


def update_request_status(request_id, status, comment):
"""Update the status and manager note for a specific holiday request."""
with postgres_pool.begin() as conn:
conn.execute(
text("""
UPDATE holidays.holiday_requests
SET status = :status, manager_note = :comment
WHERE request_id = :request_id
"""),
{"status": status, "comment": comment or "", "request_id": request_id},
)


def initialize_schema():
with postgres_pool.begin() as conn:
# create schema:
conn.execute(text("""CREATE SCHEMA IF NOT EXISTS holidays"""))
print("schema created (or already exists)")
# grant privileges
conn.execute(text("""GRANT USAGE ON SCHEMA holidays TO PUBLIC"""))
conn.execute(
text("""GRANT SELECT ON ALL TABLES IN SCHEMA holidays TO PUBLIC""")
)
print("Read permissions granted to all users")

# Create the table within the schema:
conn.execute(
text("""
CREATE TABLE IF NOT EXISTS holidays.holiday_requests (
request_id SERIAL PRIMARY KEY,
employee_name VARCHAR(255) NOT NULL,
start_date DATE NOT NULL,
end_date DATE NOT NULL,
status VARCHAR(50) NOT NULL,
manager_note TEXT
)
""")
)
print("table created (or already exists)")
# Insert demo values:
conn.execute(
text("""
INSERT INTO holidays.holiday_requests (employee_name, start_date, end_date, status, manager_note)
VALUES
('Joe', '2025-08-01', '2025-08-20', 'Pending', ''),
('Suzy', '2025-07-22', '2025-07-25', 'Pending', ''),
('Charlie', '2025-08-01', '2025-08-05', 'Pending', '')
""")
)
print("demo data is inserted")


initialize_schema()
80 changes: 80 additions & 0 deletions knowledge_base/app_with_database/app/layout.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
from dash import html, dcc, dash_table


def make_app_layout():
return html.Div(
[
html.H1("Holiday Request Manager", className="main-title"),
html.P(
"Review, approve, or decline holiday requests from your team.",
className="subtitle",
),
dash_table.DataTable(
id="holiday-table",
columns=[
{"name": "Request ID", "id": "request_id"},
{"name": "Employee", "id": "employee_name"},
{"name": "Start Date", "id": "start_date"},
{"name": "End Date", "id": "end_date"},
{"name": "Status", "id": "status"},
{"name": "Manager Comment", "id": "manager_note"},
],
data=[],
row_selectable="single",
style_table={
"margin": "24px 0",
"width": "100%",
"borderRadius": "8px",
"overflow": "hidden",
},
style_cell={
"textAlign": "center",
"fontFamily": "Lato, Arial, sans-serif",
"fontSize": "1rem",
},
style_header={
"backgroundColor": "#1B5162",
"color": "white",
"fontWeight": "bold",
"fontSize": "1.05rem",
},
style_data_conditional=[
{
"if": {"filter_query": '{status} = "Approved"'},
"backgroundColor": "#d4edda",
"color": "#155724",
},
{
"if": {"filter_query": '{status} = "Declined"'},
"backgroundColor": "#f8d7da",
"color": "#721c24",
},
],
),
html.Div(
[
html.H3("Action", className="section-title"),
dcc.RadioItems(
id="action-radio",
options=[
{"label": "Approve", "value": "Approved"},
{"label": "Decline", "value": "Declined"},
],
className="action-radio",
),
dcc.Textarea(
id="manager-comment",
placeholder="Add a comment (optional)...",
className="manager-comment",
),
html.Button(
"Submit", id="submit-action", n_clicks=0, className="submit-btn"
),
html.Div(id="action-feedback", className="feedback"),
],
className="action-panel",
),
dcc.Interval(id="refresh-interval", interval=10 * 1000, n_intervals=0),
],
className="container",
)
6 changes: 6 additions & 0 deletions knowledge_base/app_with_database/app/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
databricks-sdk>=0.60.0
pandas
psycopg[binary]
psycopg-pool
sqlalchemy==2.0.41
dash
11 changes: 11 additions & 0 deletions knowledge_base/app_with_database/databricks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# This is a Databricks asset bundle definition for app_with_database.
bundle:
name: app_with_database_example

include:
- resources/*.yml

targets:
dev:
default: true
mode: development
14 changes: 14 additions & 0 deletions knowledge_base/app_with_database/resources/myapp.app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
resources:
apps:
my_app:
name: "app-with-database"
source_code_path: ../app
description: "A Dash app that uses an OLTP Database"
# The resources which this app has access to:
resources:
- name: "app-db"
description: "A database for the app to be able to connect to and query"
database:
database_name: ${resources.database_catalogs.my_catalog.database_name}
instance_name: ${resources.database_catalogs.my_catalog.database_instance_name}
permission: "CAN_CONNECT_AND_CREATE"
11 changes: 11 additions & 0 deletions knowledge_base/app_with_database/resources/mydb.database.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
resources:
database_instances:
my_instance:
name: example-database-instance
capacity: CU_1
database_catalogs:
my_catalog:
database_instance_name: ${resources.database_instances.my_instance.name}
database_name: "example_database"
name: example_database_catalog
create_database_if_not_exists: true