Add support for local OpenAI-compatible server, fix local hot reloading, add docs on local dev (#1148)

* Add support for local server and local dev docs

* Add test

Author: pamelafox

README.md

@@ -276,6 +276,8 @@ You can only run locally **after** having successfully run the `azd up` command.
 2. Change dir to `app`
 3. Run `./start.ps1` or `./start.sh` or run the "VS Code Task: Start App" to start the project locally.
 
+See more tips in [the local development guide](docs/local.md).
+
 ## Using the app
 
 * In Azure: navigate to the Azure WebApp deployed by azd. The URL is printed out when azd completes (as "Endpoint"), or you can find it in the Azure portal.

app/backend/app.py

@@ -299,6 +299,8 @@ async def setup_clients():
             azure_endpoint=f"https://{AZURE_OPENAI_SERVICE}.openai.azure.com",
             azure_ad_token_provider=token_provider,
         )
+    elif OPENAI_HOST == "local":
+        openai_client = AsyncOpenAI(base_url=os.environ["OPENAI_BASE_URL"], api_key="no-key-required")
     else:
         openai_client = AsyncOpenAI(
             api_key=OPENAI_API_KEY,

app/frontend/vite.config.ts

@@ -27,6 +27,7 @@ export default defineConfig({
         proxy: {
             "/content/": "http://localhost:50505",
             "/auth_setup": "http://localhost:50505",
+            "/.auth/me": "http://localhost:50505",
             "/ask": "http://localhost:50505",
             "/chat": "http://localhost:50505",
             "/config": "http://localhost:50505"

docs/localdev.md

@@ -0,0 +1,67 @@
+# Local development of Chat App
+
+You can only run locally **after** having successfully run the `azd up` command. If you haven't yet, follow the steps in [Azure deployment](../README.md#azure-deployment) above.
+
+1. Run `azd auth login`
+2. Change dir to `app`
+3. Run `./start.ps1` or `./start.sh` or run the "VS Code Task: Start App" to start the project locally.
+
+## Hot reloading frontend and backend files
+
+When you run `./start.ps1` or `./start.sh`, the backend files will be watched and reloaded automatically. However, the frontend files will not be watched and reloaded automatically.
+
+To enable hot reloading of frontend files, open a new terminal and navigate to the frontend directory:
+
+```shell
+cd app/frontend
+```
+
+Then run:
+
+```shell
+npm run dev
+```
+
+You should see:
+
+```shell
+> [email protected] dev
+> vite
+
+
+  VITE v4.5.1  ready in 957 ms
+
+  ➜  Local:   http://localhost:5173/
+  ➜  Network: use --host to expose
+  ➜  press h to show help
+```
+
+Navigate to the URL shown in the terminal (in this case, `http://localhost:5173/`).  This local server will watch and reload frontend files. All backend requests will be routed to the Python server according to `vite.config.ts`.
+
+Then, whenever you make changes to frontend files, the changes will be automatically reloaded, without any browser refresh needed.
+
+
+## Using a local OpenAI-compatible API
+
+You may want to save costs by developing against a local LLM server, such as
+[llamafile](https://github.com/Mozilla-Ocho/llamafile/). Note that a local LLM
+will generally be slower and not as sophisticated.
+
+Once you've got your local LLM running and serving an OpenAI-compatible endpoint, set these environment variables:
+
+```shell
+azd env set OPENAI_HOST local
+azd env set OPENAI_BASE_URL <your local endpoint>
+```
+
+For example, to point at a local llamafile server running on its default port:
+
+```shell
+azd env set OPENAI_BASE_URL http://localhost:8080/v1
+```
+
+If you're running inside a dev container, use this local URL instead:
+
+```shell
+azd env set OPENAI_BASE_URL http://host.docker.internal:8080/v1
+```

tests/test_app.py

@@ -54,6 +54,24 @@ async def test_missing_env_vars():
                 test_app.test_client()
 
 
+@pytest.mark.asyncio
+async def test_app_local_openai(monkeypatch):
+    with mock.patch.dict(os.environ, clear=True):
+        monkeypatch.setenv("AZURE_STORAGE_ACCOUNT", "test-storage-account")
+        monkeypatch.setenv("AZURE_STORAGE_CONTAINER", "test-storage-container")
+        monkeypatch.setenv("AZURE_SEARCH_INDEX", "test-search-index")
+        monkeypatch.setenv("AZURE_SEARCH_SERVICE", "test-search-service")
+        monkeypatch.setenv("AZURE_OPENAI_CHATGPT_MODEL", "gpt-35-turbo")
+        os.environ["OPENAI_HOST"] = "local"
+        os.environ["OPENAI_BASE_URL"] = "http://localhost:5000"
+
+        quart_app = app.create_app()
+
+        async with quart_app.test_app():
+            assert quart_app.config[app.CONFIG_OPENAI_CLIENT].api_key == "no-key-required"
+            assert quart_app.config[app.CONFIG_OPENAI_CLIENT].base_url == "http://localhost:5000"
+
+
 @pytest.mark.asyncio
 async def test_index(client):
     response = await client.get("/")