Skip to content

Commit eff619d

Browse files
mcgrawiamcgrawia
authored
Increment arthur-engine version to 2.1.355
2 parents 968f373 + 1117426 commit eff619d

File tree

162 files changed

+14540
-4189
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

162 files changed

+14540
-4189
lines changed

‎.claude/settings.local.json‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"permissions": {
3+
"allow": [
4+
"Bash(grep:*)",
5+
"Bash(find:*)",
6+
"Bash(poetry run python:*)",
7+
"Bash(PYTHONPATH=src poetry run python:*)",
8+
"Bash(poetry env use:*)",
9+
"Bash(poetry install:*)",
10+
"Bash(GENAI_ENGINE_SECRET_STORE_KEY=\"test_key\" PYTHONPATH=src poetry run python:*)",
11+
"Bash(yarn type-check:*)",
12+
"Bash(yarn install:*)",
13+
"Bash(yarn generate-api:clean)",
14+
"Bash(PYTHONPATH=\"src:$PYTHONPATH\" GENAI_ENGINE_SECRET_STORE_KEY=\"test_key\" python:*)",
15+
"Bash(yarn lint:*)",
16+
"Bash(PYTHONPATH=\"src:$PYTHONPATH\" GENAI_ENGINE_SECRET_STORE_KEY=\"test_key\" poetry run pytest:*)",
17+
"Bash(docker compose:*)",
18+
"Bash(curl:*)",
19+
"Bash(export POSTGRES_USER=postgres)",
20+
"Bash(export POSTGRES_PASSWORD=changeme_pg_password)",
21+
"Bash(export POSTGRES_URL=localhost)",
22+
"Bash(export POSTGRES_PORT=5432)",
23+
"Bash(export POSTGRES_DB=arthur_genai_engine)",
24+
"Bash(export GENAI_ENGINE_SECRET_STORE_KEY=\"some_test_key\")",
25+
"Bash(export PYTHONPATH:*)",
26+
"Bash(poetry run alembic:*)",
27+
"Bash(export POSTGRES_SSL_REQUIRED=false)",
28+
"Bash(export POSTGRES_USE_SSL=false)",
29+
"Bash(lsof:*)",
30+
"Bash(ls:*)",
31+
"Bash(poetry run black:*)",
32+
"Bash(cd:*)",
33+
"Bash(poetry run isort:*)",
34+
"Bash(PGPASSWORD=changeme_pg_password psql:*)",
35+
"Bash(gh api:*)",
36+
"Bash(yarn format:check:*)",
37+
"Bash(yarn format:*)",
38+
"Bash(poetry update:*)",
39+
"Bash(PYTHONPATH=src GENAI_ENGINE_SECRET_STORE_KEY=changeme_secret_store_key poetry run generate_changelog:*)",
40+
"Bash(PYTHONPATH=src poetry run mypy:*)",
41+
"Bash(git checkout:*)"
42+
]
43+
}
44+
}

‎.claude/skills/setup-genai-dev/SKILL.md‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
---
2+
name: setup-genai-dev
3+
description: Set up the GenAI Engine development environment. Use when starting work on the project for the first time, or when environment needs to be reset. Handles Poetry, PostgreSQL, migrations, and environment variables.
4+
allowed-tools: Bash, Read, Write
5+
---
6+
7+
# Setup GenAI Engine Development Environment
8+
9+
## Prerequisites Check
10+
11+
Before setup, verify these are installed:
12+
1. Python 3.12 (`python3 --version`)
13+
2. Docker is running (`docker ps`)
14+
3. Poetry (`poetry --version`)
15+
16+
If any are missing, inform the user and stop.
17+
18+
## Setup Steps
19+
20+
Execute these steps in order:
21+
22+
### 1. Navigate to Project Directory
23+
```bash
24+
cd ./genai-engine
25+
```
26+
27+
### 2. Configure Poetry Environment
28+
```bash
29+
poetry env use 3.12
30+
poetry install --with dev,linters
31+
```
32+
33+
### 3. Start PostgreSQL Database
34+
```bash
35+
docker compose up -d db
36+
```
37+
38+
Wait for healthy status:
39+
```bash
40+
sleep 5
41+
docker compose ps
42+
```
43+
44+
### 4. Set Environment Variables
45+
46+
Export ALL of these environment variables:
47+
```bash
48+
export POSTGRES_USER=postgres
49+
export POSTGRES_PASSWORD=changeme_pg_password
50+
export POSTGRES_URL=localhost
51+
export POSTGRES_PORT=5432
52+
export POSTGRES_DB=arthur_genai_engine
53+
export POSTGRES_USE_SSL=false
54+
export PYTHONPATH="src:$PYTHONPATH"
55+
export GENAI_ENGINE_SECRET_STORE_KEY="some_test_key"
56+
export GENAI_ENGINE_ENVIRONMENT=local
57+
export GENAI_ENGINE_ADMIN_KEY=changeme123
58+
export GENAI_ENGINE_INGRESS_URI=http://localhost:3030
59+
export GENAI_ENGINE_ENABLE_PERSISTENCE=enabled
60+
export ALLOW_ADMIN_KEY_GENERAL_ACCESS=enabled
61+
```
62+
63+
### 5. Run Database Migrations
64+
```bash
65+
cd ./genai-engine
66+
poetry run alembic upgrade head
67+
```
68+
69+
### 6. Verify Setup
70+
Check database is running:
71+
```bash
72+
docker compose ps
73+
```
74+
75+
Check migrations applied:
76+
```bash
77+
poetry run alembic current
78+
```
79+
80+
## Output
81+
82+
Report success/failure for each step.
83+
84+
### LLM Configuration
85+
86+
Check for `OPENAI_API_KEY` in the `.env` file in the `./genai-engine` directory. If present, load it:
87+
```bash
88+
source ./genai-engine/.env
89+
export OPENAI_API_KEY
90+
```
91+
92+
If not present, inform the user they need to add their OpenAI API key to the `.env` file for LLM features to work.
93+
94+
## Troubleshooting
95+
96+
- If Docker fails: ensure Docker Desktop is running
97+
- If Poetry fails: try `poetry env remove 3.12` then retry
98+
- If migrations fail: check PostgreSQL is healthy with `docker compose logs db`

‎.claude/skills/start-genai-backend/SKILL.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
name: start-genai-backend
3+
description: Start the GenAI Engine backend server and frontend UI. Use when you need to launch the API server at localhost:3030 and the frontend at localhost:3000.
4+
allowed-tools: Bash, Read
5+
---
6+
7+
# Start GenAI Engine Backend Server
8+
9+
## Pre-flight Checks
10+
11+
### 1. Verify PostgreSQL is running
12+
```bash
13+
cd ./genai-engine
14+
docker compose ps db
15+
```
16+
17+
If not running or unhealthy, start it:
18+
```bash
19+
docker compose up -d db
20+
sleep 3
21+
docker compose ps db
22+
```
23+
24+
## Environment Variables
25+
26+
Set ALL of these environment variables before starting:
27+
28+
```bash
29+
# Database
30+
export POSTGRES_USER=postgres
31+
export POSTGRES_PASSWORD=changeme_pg_password
32+
export POSTGRES_URL=localhost
33+
export POSTGRES_PORT=5432
34+
export POSTGRES_DB=arthur_genai_engine
35+
export POSTGRES_USE_SSL=false
36+
37+
# Application
38+
export PYTHONPATH="src:$PYTHONPATH"
39+
export GENAI_ENGINE_SECRET_STORE_KEY="some_test_key"
40+
export GENAI_ENGINE_ENVIRONMENT=local
41+
export GENAI_ENGINE_ADMIN_KEY=changeme123
42+
export GENAI_ENGINE_INGRESS_URI=http://localhost:3030
43+
export GENAI_ENGINE_ENABLE_PERSISTENCE=enabled
44+
export ALLOW_ADMIN_KEY_GENERAL_ACCESS=enabled
45+
```
46+
47+
## LLM Provider Configuration
48+
49+
Load LLM configuration from the `.env` file:
50+
```bash
51+
source ./genai-engine/.env
52+
export OPENAI_API_KEY
53+
export GENAI_ENGINE_OPENAI_PROVIDER
54+
export GENAI_ENGINE_OPENAI_GPT_NAMES_ENDPOINTS_KEYS
55+
```
56+
57+
If `OPENAI_API_KEY` is not set in `.env`, inform the user they need to add it for LLM features to work.
58+
59+
## Start Backend Server
60+
61+
Run the server in the background so it doesn't block:
62+
```bash
63+
cd ./genai-engine
64+
poetry run serve
65+
```
66+
67+
Server will be available at:
68+
- **API**: `http://localhost:3030`
69+
- **Swagger Docs**: `http://localhost:3030/docs`
70+
- **Health Check**: `http://localhost:3030/health`
71+
72+
## Start Frontend
73+
```bash
74+
cd ./genai-engine/ui
75+
yarn install
76+
yarn dev
77+
```
78+
79+
Frontend will be available at: `http://localhost:3000`
80+
81+
## Authentication
82+
83+
To make API requests, use the admin key as a Bearer token:
84+
```
85+
Authorization: Bearer changeme123
86+
```
87+
88+
## Verification
89+
90+
After starting, verify the server is running:
91+
```bash
92+
curl -s -H "Authorization: Bearer changeme123" http://localhost:3030/health
93+
```
94+
95+
If successful, report the server is ready. If it fails, check the server logs for errors.
96+
97+
## Troubleshooting
98+
99+
- If port 3030 is in use: `lsof -i :3030` to find the process
100+
- If database connection fails: verify PostgreSQL is running with `docker compose ps`
101+
- If import errors: ensure `PYTHONPATH` includes `src`

‎.github/workflows/arthur-engine-workflow.yml‎

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -57,14 +57,13 @@ jobs:
5757
continue-on-error: true
5858
run: |
5959
poetry -C genai-engine run black --check src
60-
# TODO: Uncomment this when mypy is fixed
61-
# - name: Run mypy
62-
# id: mypy
63-
# continue-on-error: true
64-
# run: |
65-
# poetry -C genai-engine run mypy src
60+
- name: Run mypy
61+
id: mypy
62+
continue-on-error: true
63+
run: |
64+
poetry -C genai-engine run mypy src
6665
- name: Check linter results
67-
if: steps.isort.outcome == 'failure' || steps.autoflake.outcome == 'failure' || steps.black.outcome == 'failure'
66+
if: steps.isort.outcome == 'failure' || steps.autoflake.outcome == 'failure' || steps.black.outcome == 'failure' || steps.mypy.outcome == 'failure'
6867
run: |
6968
echo "One or more linters failed"
7069
exit 1
@@ -685,6 +684,17 @@ jobs:
685684
docker push arthurplatform/genai-engine-models-k8s:latest
686685
fi
687686
687+
build-gcp-model-upload-docker-image:
688+
uses: ./.github/workflows/build-gcp-model-upload.yml
689+
needs: [check-models-updates]
690+
if: |
691+
contains(github.event.head_commit.message, 'Increment arthur-engine version') &&
692+
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev')
693+
with:
694+
push: true
695+
image: arthurplatform/arthur-model-upload
696+
secrets: inherit
697+
688698
push-all-cfts:
689699
needs: [build-genai-engine-docker-images, build-ml-engine-docker-images]
690700
environment: shared-protected-branch-secrets

‎.github/workflows/build-gcp-model-upload.yml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: Build GCP Model Upload Image
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
push:
7+
description: "Push image after build"
8+
type: boolean
9+
required: false
10+
default: true
11+
image:
12+
description: "Image name (e.g. arthurplatform/arthur-model-upload)"
13+
type: string
14+
required: false
15+
default: "arthurplatform/arthur-model-upload"
16+
workflow_call:
17+
inputs:
18+
push:
19+
description: "Push image after build"
20+
type: boolean
21+
required: false
22+
default: true
23+
image:
24+
description: "Image name (e.g. arthurplatform/arthur-model-upload)"
25+
type: string
26+
required: false
27+
default: "arthurplatform/arthur-model-upload"
28+
29+
jobs:
30+
build-gcp-model-upload:
31+
environment: shared-protected-branch-secrets
32+
runs-on: ubuntu-latest
33+
steps:
34+
- uses: mathio/gha-cleanup@v1
35+
with:
36+
remove-browsers: true
37+
verbose: true
38+
- uses: actions/checkout@v5
39+
- uses: ./.github/workflows/composite-actions/setup-git
40+
with:
41+
safe-directory: ${{ runner.workspace }}
42+
- uses: ./.github/workflows/composite-actions/set-version
43+
with:
44+
version-prefix: "genai-engine-"
45+
- name: Use test suffix for manual runs
46+
if: github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main' && github.ref != 'refs/heads/dev'
47+
shell: bash
48+
run: |
49+
echo "VERSION=${VERSION%-dev}-test" >> $GITHUB_ENV
50+
- name: Set up Docker Buildx
51+
uses: docker/setup-buildx-action@v3.11.1
52+
- name: Log in to Docker Hub
53+
if: inputs.push
54+
uses: docker/login-action@v3.6.0
55+
with:
56+
username: ${{ secrets.DOCKER_HUB_USERNAME }}
57+
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
58+
- name: Build (and optionally push) docker image
59+
uses: docker/build-push-action@v6.18.0
60+
with:
61+
context: "{{defaultContext}}:deployment/gcp/model-upload"
62+
file: Dockerfile
63+
tags: |
64+
${{ inputs.image }}:${{ env.VERSION }}-gcp
65+
${{ github.ref == 'refs/heads/main' && format('{0}:latest-gcp', inputs.image) || '' }}
66+
${{ github.ref == 'refs/heads/dev' && format('{0}:latest-dev-gcp', inputs.image) || '' }}
67+
push: ${{ inputs.push }}
68+
platforms: linux/amd64

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,8 @@ local.env
2727
.env.local
2828
.env
2929

30-
.vscode/
30+
# Ignore .vscode contents but allow settings.json (per-folder Python interpreters)
31+
**/.vscode/*
3132

3233
.pre-commit-config-no-pytest-check.yaml
3334
oasdiff*
@@ -42,3 +43,4 @@ ml-engine/src/genai_client/**
4243
.cursor/
4344

4445
models/
46+
arthur-engine.code-workspace

‎.pre-commit-config.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,13 @@ repos:
6868
pass_filenames: false
6969
always_run: false
7070
files: ^genai-engine/src/
71+
- id: genai-engine-mypy-check
72+
name: "genai-engine: mypy type checking"
73+
entry: poetry -C genai-engine run mypy src
74+
language: system
75+
types: [python]
76+
pass_filenames: false
77+
files: ^genai-engine/src/
7178
- id: genai-engine-api-changelog-check
7279
name: "genai-engine: api changelog"
7380
entry: bash -c "export GENAI_ENGINE_SECRET_STORE_KEY=changeme_secret_store_key && poetry -C genai-engine run generate_changelog"

‎deployment/cloudformation/start-cpu-stack.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
<a href="https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://arthur-cft.s3.us-east-2.amazonaws.com/arthur-engine/templates/2.1.335/root-arthur-engine-cpu.yml&stackName=arthur-engine&param_MLEngineClientId=yourClientIdHere">
1+
<a href="https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://arthur-cft.s3.us-east-2.amazonaws.com/arthur-engine/templates/2.1.355/root-arthur-engine-cpu.yml&stackName=arthur-engine&param_MLEngineClientId=yourClientIdHere">
22
<img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch">
33
</a>

‎deployment/cloudformation/start-gpu-stack.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
<a href="https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://arthur-cft.s3.us-east-2.amazonaws.com/arthur-engine/templates/2.1.335/root-arthur-engine-gpu.yml&stackName=arthur-engine&param_MLEngineClientId=yourClientIdHere">
1+
<a href="https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://arthur-cft.s3.us-east-2.amazonaws.com/arthur-engine/templates/2.1.355/root-arthur-engine-gpu.yml&stackName=arthur-engine&param_MLEngineClientId=yourClientIdHere">
22
<img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch">
33
</a>

0 commit comments

Comments
 (0)