Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# User-specific MCP config
.mcp.json
.cursor/mcp.json
.claude/mcp.json

# Installed skills (in user projects)
.cursor/rules/
.claude/skills/
.cursor
.claude

# Python
__pycache__/
*.pyc
*.pyo
.venv/
venv/
*.egg-info/

# IDE
.idea/
.vscode/

# OS
.DS_Store
Thumbs.db
86 changes: 38 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,15 @@ This template enables AI-assisted development in Databricks by leveraging the Da

This template enables AI assistants to:
- ✅ Run and test code directly on Databricks clusters via the Command Execution API
- ✅ Auto-select clusters - no need to specify a cluster ID, automatically connects to a running cluster
- ✅ Create Databricks Asset Bundles (DABs) projects
- ✅ Deploy pipelines to multiple environments (dev/staging/prod)
- ✅ All from natural language prompts!

**Just describe what you want → AI builds, tests the code on Databricks, and deploys the complete pipeline.**

> 💡 **Smart Cluster Selection**: If no `cluster_id` is provided, the MCP server automatically finds a running cluster in your workspace (prioritizing clusters with "interactive", "general", or "all-purpose" in the name).

---

### ⚙️ Prerequisites - Set Up Environment Variables
Expand Down Expand Up @@ -47,85 +50,72 @@ cd databricks-exec-code-mcp
#### 2. Install Requirements

```bash
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
pip install -r requirements.txt
```

#### 3. Start the MCP Server
#### 3. Configure Your AI Client

```bash
python mcp_tools/tools.py
```
Add the MCP server to your configuration. Use the path to your cloned repo:

The server runs at:
```
http://localhost:8000
```

#### 4. Configure Your AI Client

Add the following to your MCP configuration:

**For Cursor** (`~/.cursor/mcp.json`):
**For Cursor** (`.cursor/mcp.json` in your project, or global `~/.cursor/mcp.json`):
```json
{
"mcpServers": {
"databricks-dev-mcp": {
"type": "http",
"url": "http://localhost:8000/message"
"databricks": {
"command": "/path/to/databricks-exec-code-mcp/.venv/bin/python",
"args": ["/path/to/databricks-exec-code-mcp/mcp_tools/tools.py"]
}
}
}
```

**For Claude Code** (`~/.config/claude/mcp.json`):
**For Claude Code** (`.claude/mcp.json` in your project):
```json
{
"mcpServers": {
"databricks-dev-mcp": {
"type": "http",
"url": "http://localhost:8000/message"
"databricks": {
"command": "/path/to/databricks-exec-code-mcp/.venv/bin/python",
"args": ["/path/to/databricks-exec-code-mcp/mcp_tools/tools.py"]
}
}
}
```

**For Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
```json
{
"mcpServers": [
{
"id": "databricks-dev-mcp",
"type": "http",
"url": "http://localhost:8000/message"
}
]
}
```
> 💡 **Tip**: Replace `/path/to/databricks-exec-code-mcp` with the actual path to your cloned repo.

Restart your AI client after configuration.

#### 5. Prepare Your Credentials
#### 4. Install Skills

You'll need:
- **Databricks Workspace URL**: `https://dbc-xxxxx.cloud.databricks.com`
- **Databricks PAT Token**: Generate from User Settings → Developer → Access Tokens
- **Unity Catalog Name**: e.g., `my_catalog`
- **Cluster ID**: For testing code via Command Execution API
Install the Databricks skills to teach your AI assistant how to work with Databricks:

```bash
# Install for both Cursor and Claude Code
./install_skills.sh --all

# Or install for a specific client
./install_skills.sh --cursor
./install_skills.sh --claude
```

#### 6. Include Context for AI
This installs skills to:
- **Cursor**: `.cursor/rules/`
- **Claude Code**: `.claude/skills/`

**For Cursor:**
Add the `claude.md` file to your project:
1. Place it in `.cursor/claude.md` in your project root, OR
2. Go to Settings → Rules and Commands → Toggle to include claude.md
#### 5. Start Building!

**For Claude Code:**
Navigate to the directory containing the `claude.md` file.
Just describe what you want in natural language:

**Data Engineering:**
> "Build a Data Engineering pipeline using Medallion Architecture on the NYC Taxi dataset and deploy it with DABs"

#### 7. Example prompts
**Machine Learning:**
> "Train a classification model on the Titanic dataset, register it to Unity Catalog, and deploy as a DAB job"

To start chatting with the AI-assistant, you can also leverage some examples in [`example_prompts`](./example_prompts).
**Quick Test:**
> "Run a SQL query to show the top 10 tables in my catalog"

---

Expand Down
56 changes: 0 additions & 56 deletions claude.md

This file was deleted.

40 changes: 0 additions & 40 deletions example_prompts/data_engineering_pipeline.md

This file was deleted.

42 changes: 0 additions & 42 deletions example_prompts/ml_pipeline.md

This file was deleted.

Loading