Skip to content

Commit c1b619e

Browse files
Updated lakehouse app readme to match docs site
1 parent c00efec commit c1b619e

File tree

1 file changed

+117
-84
lines changed

1 file changed

+117
-84
lines changed

lakehouse_app/README.md

Lines changed: 117 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,117 @@
1-
# DLT-META Lakehouse App Setup
2-
3-
Make sure you have installed/upgraded the latest Databricks CLI version (e.g., 0.244.0) and configured workspace access where the app is being deployed.
4-
5-
## Create App and Attach Source to Databricks Apps
6-
7-
### Step 1: Create a Custom App ("empty") Using the CLI
8-
For example, if the app name is `demo-dltmeta`:
9-
```bash
10-
databricks apps create demo-dltmeta
11-
```
12-
Wait for the command execution to complete. It will take a few minutes.
13-
14-
### Step 2: Checkout Project from DLT-META Git Repository
15-
```bash
16-
git clone https://github.com/databrickslabs/dlt-meta.git
17-
```
18-
19-
### Step 3: Navigate to the Project Directory
20-
```bash
21-
cd dlt-meta/lakehouse_app
22-
```
23-
24-
### Step 4: Sync the DLT-META App Code to Your Workspace Directory
25-
Run the command below to sync the code (replace `testapp` with your desired folder name):
26-
```bash
27-
databricks sync . /Workspace/Users/<user1.user2>@databricks.com/testapp
28-
```
29-
30-
### Step 5: Deploy Code to the App Created in Step 1
31-
```bash
32-
databricks apps deploy demo-dltmeta --source-code-path /Workspace/Users/<user1.user2>@databricks.com/testapp
33-
```
34-
35-
### Step 6: Open the App in the Browser
36-
- Open the URL from the Step 1 log, or
37-
- Go to the Databricks web page, click **New > App**, click back on **App**, search for your app name, and click on the URL to open the app in the browser.
38-
39-
---
40-
41-
## Run the App Locally
42-
43-
### Step 1: Checkout Project from DLT-META Git Repository
44-
```bash
45-
git clone https://github.com/databrickslabs/dlt-meta.git
46-
```
47-
48-
### Step 2: Navigate to the Project Directory
49-
```bash
50-
cd dlt-meta/lakehouse_app
51-
```
52-
53-
### Step 3: Install the Required Dependencies
54-
```bash
55-
pip install -r requirements.txt
56-
```
57-
58-
### Step 4: Configure Databricks
59-
```bash
60-
databricks configure --host <your-databricks-host-url> --token <your-token>
61-
```
62-
63-
### Step 5: Run the App
64-
```bash
65-
python App.py
66-
```
67-
68-
### Step 6: Access the App
69-
Click on the URL link: [http://127.0.0.1:5000](http://127.0.0.1:5000)
70-
71-
---
72-
73-
## Databricks App Username
74-
75-
Databricks creates a unique username for each app, which can be found on the Databricks app page.
76-
77-
### Step 1: Configure the DLT-META Environment
78-
After launching the app in the browser, click the button **"Setup DLT-META Project Environment"** to configure the DLT-META environment on the app's remote instance for onboarding and deployment activities.
79-
80-
### Step 2: Onboard a DLT Pipeline
81-
Use the **"UI"** tab to onboard and deploy DLT pipelines based on your pipeline configuration.
82-
83-
### Step 3: Run Available Demos
84-
Navigate to the **"Demo"** tab to run the available demos.
1+
# DLT-META Lakehouse App
2+
3+
## Prerequisites
4+
5+
### System Requirements
6+
- Python 3.8.0 or higher
7+
- [Databricks CLI](https://docs.databricks.com/en/dev-tools/cli/tutorial.html) (latest version, e.g., 0.244.0)
8+
- Configured workspace access
9+
10+
### Initial Setup
11+
1. Authenticate with Databricks:
12+
```commandline
13+
databricks auth login --host WORKSPACE_HOST
14+
```
15+
16+
2. Setup Python Environment:
17+
```commandline
18+
git clone https://github.com/databrickslabs/dlt-meta.git
19+
cd dlt-meta
20+
python -m venv .venv
21+
source .venv/bin/activate
22+
pip install databricks-sdk
23+
```
24+
25+
## Deployment Options
26+
27+
### Deploy to Databricks
28+
29+
1. Create Custom App:
30+
```commandline
31+
databricks apps create demo-dltmeta
32+
```
33+
> Note: Wait for command completion (a few minutes)
34+
35+
2. Setup App Code:
36+
```commandline
37+
cd dlt-meta/lakehouse_app
38+
39+
# Replace testapp with your preferred folder name
40+
databricks sync . /Workspace/Users/<user1.user2>@databricks.com/testapp
41+
42+
# Deploy the app
43+
databricks apps deploy demo-dltmeta --source-code-path /Workspace/Users/<user1.user2>@databricks.com/testapp
44+
```
45+
46+
3. Access the App:
47+
- Open URL from step 1 log, or
48+
- Navigate: Databricks Web UI → New → App → Back to App → Search your app name
49+
50+
### Run Locally
51+
52+
1. Setup Environment:
53+
```commandline
54+
cd dlt-meta/lakehouse_app
55+
pip install -r requirements.txt
56+
```
57+
58+
2. Configure Databricks:
59+
```commandline
60+
databricks configure --host <your databricks host url> --token <your token>
61+
```
62+
63+
3. Start App:
64+
```commandline
65+
python App.py
66+
```
67+
Access at: http://127.0.0.1:5000
68+
69+
## Using DLT-META App
70+
71+
### App User Setup
72+
73+
![App User Example](../docs/static/images/app_cli.png)
74+
75+
The app creates a dedicated user account that:
76+
- Handles onboarding, deployment, and demo execution
77+
- Requires specific permissions for UC catalogs and schemas
78+
- Example username format: "app-40zbx9_demo-dltmeta"
79+
80+
### Getting Started
81+
82+
1. Initial Setup:
83+
- Launch app in browser
84+
- Click "Setup dlt-meta project environment"
85+
- This initializes the environment for onboarding and deployment
86+
87+
2. Pipeline Management:
88+
- Use "UI" tab to onboard and deploy pipelines
89+
- Configure pipelines according to your requirements
90+
91+
**Onboarding Pipeline:**
92+
93+
![Onboarding UI](../docs/static/images/app_onboarding.png)
94+
95+
*Pipeline onboarding interface for configuring new data pipelines*
96+
97+
**Deploying Pipeline:**
98+
99+
![Deploy UI](../docs/static/images/app_deploy_pipeline.png)
100+
101+
*Pipeline deployment interface for managing and deploying pipelines*
102+
103+
3. Demo Access:
104+
- Available demos can be found under "Demo" tab
105+
- Run pre-configured demo pipelines to explore features
106+
107+
![App Demo](../docs/static/images/app_run_demos.png)
108+
109+
*Demo interface showing available example pipelines*
110+
111+
4. Command Line Interface:
112+
- Access CLI features under the "CLI" tab
113+
- Execute commands directly from the web interface
114+
115+
![CLI UI](../docs/static/images/app_cli.png)
116+
117+
*CLI interface for command-line operations*

0 commit comments

Comments
 (0)