Adding support for non-Azure openai instances (#507)

* Adding openai non-azure endpoint support

* Editing infra files

* Adding missing args for the prepdocs script

* Rolling back indentation changes

* rolling back indentiation edits

* removing unnecessary parenthesis

* Adding OPENAI_EMB_MODEL parameter to bicep config

* Adding conditional args to reduce if statements

* Rm notebooks again

* OpenAI api type

* Tests pass

* Fix bicep issue

* Prepdocs fix

* Add main.bicep changes

* Add main.bicep changes

* Add missing api_type

* Parameterize tests

* Rm comma

* Making chatGptModelName param conditional

* Conditional system identity

* Make Azure_openai_service conditional

* fix inconsistencies across emb model param naming

* Fix formatting after merge from main

* Script fix

---------

Co-authored-by: Pamela Fox <[email protected]>
Co-authored-by: Pamela Fox <[email protected]>

Author: carlotta94c

README.md

@@ -49,7 +49,7 @@ The repo includes sample data so it's ready to try end to end. In this sample ap
 **IMPORTANT:** In order to deploy and run this example, you'll need:
 
 * **Azure account**. If you're new to Azure, [get an Azure account for free](https://azure.microsoft.com/free/cognitive-search/) and you'll get some free Azure credits to get started.
-* **Azure subscription with access enabled for the Azure OpenAI service**. You can request access with [this form](https://aka.ms/oaiapply).
+* **Azure subscription with access enabled for the Azure OpenAI service**. You can request access with [this form](https://aka.ms/oaiapply). If your access request to Azure OpenAI service doesn't match the [acceptance criteria](https://learn.microsoft.com/legal/cognitive-services/openai/limited-access?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext), you can use [OpenAI public API](https://platform.openai.com/docs/api-reference/introduction) instead. Learn [how to switch to an OpenAI instance](#switching-from-an-azure-openai-endpoint-to-an-openai-instance).
 * **Azure account permissions**: Your Azure account must have `Microsoft.Authorization/roleAssignments/write` permissions, such as [Role Based Access Control Administrator](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview), [User Access Administrator](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator), or [Owner](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#owner).
 
 ## Azure deployment
@@ -176,6 +176,20 @@ either you or they can follow these steps:
 
 ## Enabling optional features
 
+#### Using a non-Azure OpenAI instance
+
+To use an existing non-Azure OpenAI account, follow these steps before running `azd up`
+
+1. Run `azd env set OPENAI_HOST openai`
+2. Run `azd env set OPENAI_ORGANIZATION {Your OpenAI organization}`
+3. Run `azd env set OPENAI_API_KEY {Your OpenAI API key}`
+4. Run `azd up`
+
+You can retrieve your OpenAI key by checking [your user page](https://platform.openai.com/account/api-keys) and your organization by navigating to [your organization page](https://platform.openai.com/account/org-settings).
+Learn more about creating an OpenAI free trial at [this link](https://openai.com/pricing).
+Do *not* check your key into source control.
+
+
 ### Enabling Application Insights
 
 To enable Application Insights and the tracing of each request, along with the logging of errors, set the `AZURE_USE_APPLICATION_INSIGHTS` variable to true before running `azd up`
@@ -258,6 +272,17 @@ to production. Here are some things to consider:
 * [Revolutionize your Enterprise Data with ChatGPT: Next-gen Apps w/ Azure OpenAI and Cognitive Search](https://aka.ms/entgptsearchblog)
 * [Azure Cognitive Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search)
 * [Azure OpenAI Service](https://learn.microsoft.com/azure/cognitive-services/openai/overview)
+* [Comparing Azure OpenAI and OpenAI](https://learn.microsoft.com/en-gb/azure/cognitive-services/openai/overview#comparing-azure-openai-and-openai/)
+
+## Clean up
+
+To clean up all the resources created by this sample:
+
+1. Run `azd down`
+2. When asked if you are sure you want to continue, enter `y`
+3. When asked if you want to permanently delete the resources, enter `y`
+
+The resource group and all the resources will be deleted.
 
 ### Note
 

app/backend/app.py

@@ -139,6 +139,8 @@ async def chat_stream():
 
 @bp.before_request
 async def ensure_openai_token():
+    if openai.api_type != "azure_ad":
+        return
     openai_token = current_app.config[CONFIG_OPENAI_TOKEN]
     if openai_token.expires_on < time.time() + 60:
         openai_token = await current_app.config[CONFIG_CREDENTIAL].get_token(
@@ -155,10 +157,17 @@ async def setup_clients():
     AZURE_STORAGE_CONTAINER = os.environ["AZURE_STORAGE_CONTAINER"]
     AZURE_SEARCH_SERVICE = os.environ["AZURE_SEARCH_SERVICE"]
     AZURE_SEARCH_INDEX = os.environ["AZURE_SEARCH_INDEX"]
-    AZURE_OPENAI_SERVICE = os.environ["AZURE_OPENAI_SERVICE"]
-    AZURE_OPENAI_CHATGPT_DEPLOYMENT = os.environ["AZURE_OPENAI_CHATGPT_DEPLOYMENT"]
-    AZURE_OPENAI_CHATGPT_MODEL = os.environ["AZURE_OPENAI_CHATGPT_MODEL"]
-    AZURE_OPENAI_EMB_DEPLOYMENT = os.environ["AZURE_OPENAI_EMB_DEPLOYMENT"]
+    # Shared by all OpenAI deployments
+    OPENAI_HOST = os.getenv("OPENAI_HOST", "azure")
+    OPENAI_CHATGPT_MODEL = os.environ["AZURE_OPENAI_CHATGPT_MODEL"]
+    OPENAI_EMB_MODEL = os.getenv("AZURE_OPENAI_EMB_MODEL_NAME", "text-embedding-ada-002")
+    # Used with Azure OpenAI deployments
+    AZURE_OPENAI_SERVICE = os.getenv("AZURE_OPENAI_SERVICE")
+    AZURE_OPENAI_CHATGPT_DEPLOYMENT = os.getenv("AZURE_OPENAI_CHATGPT_DEPLOYMENT")
+    AZURE_OPENAI_EMB_DEPLOYMENT = os.getenv("AZURE_OPENAI_EMB_DEPLOYMENT")
+    # Used only with non-Azure OpenAI deployments
+    OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
+    OPENAI_ORGANIZATION = os.getenv("OPENAI_ORGANIZATION")
 
     KB_FIELDS_CONTENT = os.getenv("KB_FIELDS_CONTENT", "content")
     KB_FIELDS_SOURCEPAGE = os.getenv("KB_FIELDS_SOURCEPAGE", "sourcepage")
@@ -181,14 +190,19 @@ async def setup_clients():
     blob_container_client = blob_client.get_container_client(AZURE_STORAGE_CONTAINER)
 
     # Used by the OpenAI SDK
-    openai.api_base = f"https://{AZURE_OPENAI_SERVICE}.openai.azure.com"
-    openai.api_version = "2023-05-15"
-    openai.api_type = "azure_ad"
-    openai_token = await azure_credential.get_token("https://cognitiveservices.azure.com/.default")
-    openai.api_key = openai_token.token
-
-    # Store on app.config for later use inside requests
-    current_app.config[CONFIG_OPENAI_TOKEN] = openai_token
+    if OPENAI_HOST == "azure":
+        openai.api_type = "azure_ad"
+        openai.api_base = f"https://{AZURE_OPENAI_SERVICE}.openai.azure.com"
+        openai.api_version = "2023-05-15"
+        openai_token = await azure_credential.get_token("https://cognitiveservices.azure.com/.default")
+        openai.api_key = openai_token.token
+        # Store on app.config for later use inside requests
+        current_app.config[CONFIG_OPENAI_TOKEN] = openai_token
+    else:
+        openai.api_type = "openai"
+        openai.api_key = OPENAI_API_KEY
+        openai.organization = OPENAI_ORGANIZATION
+
     current_app.config[CONFIG_CREDENTIAL] = azure_credential
     current_app.config[CONFIG_BLOB_CONTAINER_CLIENT] = blob_container_client
 
@@ -197,33 +211,43 @@ async def setup_clients():
     current_app.config[CONFIG_ASK_APPROACHES] = {
         "rtr": RetrieveThenReadApproach(
             search_client,
+            OPENAI_HOST,
             AZURE_OPENAI_CHATGPT_DEPLOYMENT,
-            AZURE_OPENAI_CHATGPT_MODEL,
+            OPENAI_CHATGPT_MODEL,
             AZURE_OPENAI_EMB_DEPLOYMENT,
+            OPENAI_EMB_MODEL,
             KB_FIELDS_SOURCEPAGE,
             KB_FIELDS_CONTENT,
         ),
         "rrr": ReadRetrieveReadApproach(
             search_client,
+            OPENAI_HOST,
             AZURE_OPENAI_CHATGPT_DEPLOYMENT,
+            OPENAI_CHATGPT_MODEL,
             AZURE_OPENAI_EMB_DEPLOYMENT,
+            OPENAI_EMB_MODEL,
             KB_FIELDS_SOURCEPAGE,
             KB_FIELDS_CONTENT,
         ),
         "rda": ReadDecomposeAsk(
             search_client,
+            OPENAI_HOST,
             AZURE_OPENAI_CHATGPT_DEPLOYMENT,
+            OPENAI_CHATGPT_MODEL,
             AZURE_OPENAI_EMB_DEPLOYMENT,
+            OPENAI_EMB_MODEL,
             KB_FIELDS_SOURCEPAGE,
             KB_FIELDS_CONTENT,
         ),
     }
     current_app.config[CONFIG_CHAT_APPROACHES] = {
         "rrr": ChatReadRetrieveReadApproach(
             search_client,
+            OPENAI_HOST,
             AZURE_OPENAI_CHATGPT_DEPLOYMENT,
-            AZURE_OPENAI_CHATGPT_MODEL,
+            OPENAI_CHATGPT_MODEL,
             AZURE_OPENAI_EMB_DEPLOYMENT,
+            OPENAI_EMB_MODEL,
             KB_FIELDS_SOURCEPAGE,
             KB_FIELDS_CONTENT,
         )

app/backend/approaches/chatreadretrieveread.py

@@ -50,16 +50,20 @@ class ChatReadRetrieveReadApproach:
     def __init__(
         self,
         search_client: SearchClient,
+        openai_host: str,
         chatgpt_deployment: str,
         chatgpt_model: str,
         embedding_deployment: str,
+        embedding_model: str,
         sourcepage_field: str,
         content_field: str,
     ):
         self.search_client = search_client
+        self.openai_host = openai_host
         self.chatgpt_deployment = chatgpt_deployment
         self.chatgpt_model = chatgpt_model
         self.embedding_deployment = embedding_deployment
+        self.embedding_model = embedding_model
         self.sourcepage_field = sourcepage_field
         self.content_field = content_field
         self.chatgpt_token_limit = get_token_limit(chatgpt_model)
@@ -86,8 +90,9 @@ async def run_until_final_call(
             self.chatgpt_token_limit - len(user_q),
         )
 
+        chatgpt_args = {"deployment_id": self.chatgpt_deployment} if self.openai_host == "azure" else {}
         chat_completion = await openai.ChatCompletion.acreate(
-            deployment_id=self.chatgpt_deployment,
+            **chatgpt_args,
             model=self.chatgpt_model,
             messages=messages,
             temperature=0.0,
@@ -103,7 +108,8 @@ async def run_until_final_call(
 
         # If retrieval mode includes vectors, compute an embedding for the query
         if has_vector:
-            embedding = await openai.Embedding.acreate(engine=self.embedding_deployment, input=query_text)
+            embedding_args = {"deployment_id": self.embedding_deployment} if self.openai_host == "azure" else {}
+            embedding = await openai.Embedding.acreate(**embedding_args, model=self.embedding_model, input=query_text)
             query_vector = embedding["data"][0]["embedding"]
         else:
             query_vector = None
@@ -181,7 +187,7 @@ async def run_until_final_call(
             + msg_to_display.replace("\n", "<br>"),
         }
         chat_coroutine = openai.ChatCompletion.acreate(
-            deployment_id=self.chatgpt_deployment,
+            **chatgpt_args,
             model=self.chatgpt_model,
             messages=messages,
             temperature=overrides.get("temperature") or 0.7,

app/backend/approaches/readdecomposeask.py

@@ -7,7 +7,7 @@
 from langchain.agents import AgentExecutor, Tool
 from langchain.agents.react.base import ReActDocstoreAgent
 from langchain.callbacks.manager import CallbackManager
-from langchain.llms.openai import AzureOpenAI
+from langchain.llms.openai import AzureOpenAI, OpenAI
 from langchain.prompts import BasePromptTemplate, PromptTemplate
 from langchain.tools.base import BaseTool
 
@@ -20,16 +20,22 @@ class ReadDecomposeAsk(AskApproach):
     def __init__(
         self,
         search_client: SearchClient,
+        openai_host: str,
         openai_deployment: str,
+        openai_model: str,
         embedding_deployment: str,
+        embedding_model: str,
         sourcepage_field: str,
         content_field: str,
     ):
         self.search_client = search_client
         self.openai_deployment = openai_deployment
+        self.openai_model = openai_model
         self.embedding_deployment = embedding_deployment
+        self.embedding_model = embedding_model
         self.sourcepage_field = sourcepage_field
         self.content_field = content_field
+        self.openai_host = openai_host
 
     async def search(self, query_text: str, overrides: dict[str, Any]) -> tuple[list[str], str]:
         has_text = overrides.get("retrieval_mode") in ["text", "hybrid", None]
@@ -41,7 +47,8 @@ async def search(self, query_text: str, overrides: dict[str, Any]) -> tuple[list
 
         # If retrieval mode includes vectors, compute an embedding for the query
         if has_vector:
-            embedding = await openai.Embedding.acreate(engine=self.embedding_deployment, input=query_text)
+            embedding_args = {"deployment_id": self.embedding_deployment} if self.openai_host == "azure" else {}
+            embedding = await openai.Embedding.acreate(**embedding_args, model=self.embedding_model, input=query_text)
             query_vector = embedding["data"][0]["embedding"]
         else:
             query_vector = None
@@ -74,7 +81,7 @@ async def search(self, query_text: str, overrides: dict[str, Any]) -> tuple[list
                 vector_fields="embedding" if query_vector else None,
             )
         if use_semantic_captions:
-            results = [
+            self.results = [
                 doc[self.sourcepage_field] + ":" + nonewlines(" . ".join([c.text for c in doc["@search.captions"]]))
                 async for doc in r
             ]
@@ -114,11 +121,18 @@ async def search_and_store(q: str) -> Any:
         cb_handler = HtmlCallbackHandler()
         cb_manager = CallbackManager(handlers=[cb_handler])
 
-        llm = AzureOpenAI(
-            deployment_name=self.openai_deployment,
-            temperature=overrides.get("temperature") or 0.3,
-            openai_api_key=openai.api_key,
-        )
+        if self.openai_host == "azure":
+            llm = AzureOpenAI(
+                deployment_name=self.openai_deployment,
+                temperature=overrides.get("temperature", 0.3),
+                openai_api_key=openai.api_key,
+            )
+        else:
+            llm = OpenAI(
+                model_name=self.openai_model,
+                temperature=overrides.get("temperature", 0.3),
+                openai_api_key=openai.api_key,
+            )
         tools = [
             Tool(
                 name="Search",

app/backend/approaches/readretrieveread.py

@@ -6,7 +6,7 @@
 from langchain.agents import AgentExecutor, Tool, ZeroShotAgent
 from langchain.callbacks.manager import CallbackManager, Callbacks
 from langchain.chains import LLMChain
-from langchain.llms.openai import AzureOpenAI
+from langchain.llms.openai import AzureOpenAI, OpenAI
 
 from approaches.approach import AskApproach
 from langchainadapters import HtmlCallbackHandler
@@ -51,16 +51,22 @@ class ReadRetrieveReadApproach(AskApproach):
     def __init__(
         self,
         search_client: SearchClient,
+        openai_host: str,
         openai_deployment: str,
+        openai_model: str,
         embedding_deployment: str,
+        embedding_model: str,
         sourcepage_field: str,
         content_field: str,
     ):
         self.search_client = search_client
         self.openai_deployment = openai_deployment
+        self.openai_model = openai_model
         self.embedding_deployment = embedding_deployment
+        self.embedding_model = embedding_model
         self.sourcepage_field = sourcepage_field
         self.content_field = content_field
+        self.openai_host = openai_host
 
     async def retrieve(self, query_text: str, overrides: dict[str, Any]) -> Any:
         has_text = overrides.get("retrieval_mode") in ["text", "hybrid", None]
@@ -72,7 +78,8 @@ async def retrieve(self, query_text: str, overrides: dict[str, Any]) -> Any:
 
         # If retrieval mode includes vectors, compute an embedding for the query
         if has_vector:
-            embedding = await openai.Embedding.acreate(engine=self.embedding_deployment, input=query_text)
+            embedding_args = {"deployment_id": self.embedding_deployment} if self.openai_host == "azure" else {}
+            embedding = await openai.Embedding.acreate(**embedding_args, model=self.embedding_model, input=query_text)
             query_vector = embedding["data"][0]["embedding"]
         else:
             query_vector = None
@@ -143,11 +150,19 @@ async def retrieve_and_store(q: str) -> Any:
             suffix=overrides.get("prompt_template_suffix") or self.template_suffix,
             input_variables=["input", "agent_scratchpad"],
         )
-        llm = AzureOpenAI(
-            deployment_name=self.openai_deployment,
-            temperature=overrides.get("temperature") or 0.3,
-            openai_api_key=openai.api_key,
-        )
+        if self.openai_type == "azure":
+            llm = AzureOpenAI(
+                deployment_name=self.openai_deployment,
+                temperature=overrides.get("temperature", 0.3),
+                openai_api_key=openai.api_key,
+            )
+        else:
+            llm = OpenAI(
+                model_name=self.openai_model,
+                temperature=overrides.get("temperature", 0.3),
+                openai_api_key=openai.api_key,
+            )
+
         chain = LLMChain(llm=llm, prompt=prompt)
         agent_exec = AgentExecutor.from_agent_and_tools(
             agent=ZeroShotAgent(llm_chain=chain), tools=tools, verbose=True, callback_manager=cb_manager

app/backend/approaches/retrievethenread.py

@@ -40,15 +40,19 @@ class RetrieveThenReadApproach(AskApproach):
     def __init__(
         self,
         search_client: SearchClient,
-        openai_deployment: str,
+        openai_host: str,
+        chatgpt_deployment: str,
         chatgpt_model: str,
         embedding_deployment: str,
+        embedding_model: str,
         sourcepage_field: str,
         content_field: str,
     ):
         self.search_client = search_client
-        self.openai_deployment = openai_deployment
+        self.openai_host = openai_host
+        self.chatgpt_deployment = chatgpt_deployment
         self.chatgpt_model = chatgpt_model
+        self.embedding_model = embedding_model
         self.embedding_deployment = embedding_deployment
         self.sourcepage_field = sourcepage_field
         self.content_field = content_field
@@ -63,7 +67,8 @@ async def run(self, q: str, overrides: dict[str, Any]) -> dict[str, Any]:
 
         # If retrieval mode includes vectors, compute an embedding for the query
         if has_vector:
-            embedding = await openai.Embedding.acreate(engine=self.embedding_deployment, input=q)
+            embedding_args = {"deployment_id": self.embedding_deployment} if self.openai_host == "azure" else {}
+            embedding = await openai.Embedding.acreate(**embedding_args, model=self.embedding_model, input=q)
             query_vector = embedding["data"][0]["embedding"]
         else:
             query_vector = None
@@ -117,8 +122,9 @@ async def run(self, q: str, overrides: dict[str, Any]) -> dict[str, Any]:
         message_builder.append_message("user", self.question)
 
         messages = message_builder.messages
+        chatgpt_args = {"deployment_id": self.chatgpt_deployment} if self.openai_host == "azure" else {}
         chat_completion = await openai.ChatCompletion.acreate(
-            deployment_id=self.openai_deployment,
+            **chatgpt_args,
             model=self.chatgpt_model,
             messages=messages,
             temperature=overrides.get("temperature") or 0.3,