openai: adds examples and flattens tests (#46)

* openai: adds examples and flattens tests

This adds an examples directory which include two main features this
instrumentation supports: chat and embeddings

It also flattens test recordings and infrastructure around unit tests.
Notably, instrumentation code doesn't change depending on if azure is
used or not. Unit testing only openai (platform) simplifies tests and
reduces infrastructure significantly.

Signed-off-by: Adrian Cole <[email protected]>

* Add missing license headers

* Apply suggestions from code review

* prune

Signed-off-by: Adrian Cole <[email protected]>

* Introduce ollama after using OpenAI

---------

Signed-off-by: Adrian Cole <[email protected]>
Co-authored-by: Riccardo Magliocchetti <[email protected]>

Author: codefromthecrypt

instrumentation/elastic-opentelemetry-instrumentation-openai/README.md

@@ -22,32 +22,49 @@ pip install elastic-opentelemetry-instrumentation-openai
 
 This instrumentation supports *zero-code* / *autoinstrumentation*:
 
+Set up a virtual environment with this package, the dependencies it requires
+and `dotenv` (a portable way to load environment variables).
+```
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r test-requirements.txt
+pip install python-dotenv[cli]
+```
+
+Create a `.env` file containing the OpenAI API key:
+
+```
+echo "OPENAI_API_KEY=sk-..." > .env
 ```
-opentelemetry-instrument python use_openai.py
 
-# You can record more information about prompts as log events by enabling content capture.
-OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true opentelemetry-instrument python use_openai.py
+Run the script with telemetry setup to use the instrumentation.
+
+```
+dotenv run -- opentelemetry-instrument python examples/chat.py
 ```
 
-Or manual instrumentation:
+You can record more information about prompts as log events by enabling content capture.
+```
+OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true dotenv run -- \
+opentelemetry-instrument python examples/chat.py
+```
 
-```python
-import openai
-from opentelemetry.instrumentation.openai import OpenAIInstrumentor
+### Using a local model
 
-OpenAIInstrumentor().instrument()
+[Ollama](https://ollama.com/) may be used to run examples without a cloud account. After you have set it up
+need to install the models in order to run the examples:
 
-# assumes at least the OPENAI_API_KEY environment variable set
-client = openai.Client()
+```
+# for chat
+ollama pull qwen2.5:0.5b
+# for embeddings
+ollama pull all-minilm:33m
+```
 
-messages = [
-    {
-        "role": "user",
-        "content": "Answer in up to 3 words: Which ocean contains the canarian islands?",
-    }
-]
+Finally run the examples using [ollama.env](ollama.env) variables to point to Ollama instead of OpenAI:
 
-chat_completion = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
+```
+dotenv run -f ollama.env -- opentelemetry-instrument python examples/chat.py
 ```
 
 ### Instrumentation specific environment variable configuration
@@ -110,20 +127,22 @@ response without querying the LLM.
 
 ### Azure OpenAI Environment Variables
 
-Azure is different from OpenAI primarily in that a URL has an implicit model. This means it ignores
-the model parameter set by the OpenAI SDK. The implication is that one endpoint cannot serve both
-chat and embeddings at the same time. Hence, we need separate environment variables for chat and
-embeddings. In either case, the `DEPLOYMENT_URL` is the "Endpoint Target URI" and the `API_KEY` is
-the `Endpoint Key` for a corresponding deployment in https://oai.azure.com/resource/deployments
-
-* `AZURE_CHAT_COMPLETIONS_DEPLOYMENT_URL`
-  * It should look like https://endpoint.com/openai/deployments/my-deployment/chat/completions?api-version=2023-05-15
-* `AZURE_CHAT_COMPLETIONS_API_KEY`
-  * It should be in hex like `abc01...` and possibly the same as `AZURE_EMBEDDINGS_API_KEY`
-* `AZURE_EMBEDDINGS_DEPLOYMENT_URL`
-  * It should look like https://endpoint.com/openai/deployments/my-deployment/embeddings?api-version=2023-05-15
-* `AZURE_EMBEDDINGS_API_KEY`
-  * It should be in hex like `abc01...` and possibly the same as `AZURE_CHAT_COMPLETIONS_API_KEY`
+The `AzureOpenAI` client extends `OpenAI` with parameters specific to the Azure OpenAI Service.
+
+* `AZURE_OPENAI_ENDPOINT` - "Azure OpenAI Endpoint" in https://oai.azure.com/resource/overview
+  * It should look like `https://<your-resource-name>.openai.azure.com/`
+* `AZURE_OPENAI_API_KEY` - "API key 1 (or 2)" in https://oai.azure.com/resource/overview
+  * It should look be a hex string like `abc01...`
+* `OPENAI_API_VERSION` = "Inference version" from https://learn.microsoft.com/en-us/azure/ai-services/openai/api-version-deprecation
+  * It should look like `2024-10-01-preview`
+* `TEST_CHAT_MODEL` = "Name" from https://oai.azure.com/resource/deployments that deployed a model
+  that supports tool calling, such as "gpt-4o-mini".
+* `TEST_EMBEDDINGS_MODEL` = "Name" from https://oai.azure.com/resource/deployments that deployed a
+  model that supports embeddings, such as "text-embedding-3-small".
+
+Note: The model parameter of a chat completion or embeddings request is substituted for an identical
+deployment name. As deployment names are arbitrary they may have no correlation with a real model
+like `gpt-4o`
 
 ## License
 

instrumentation/elastic-opentelemetry-instrumentation-openai/examples/chat.py

@@ -0,0 +1,39 @@
+# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+# or more contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+import openai
+
+CHAT_MODEL = os.environ.get("TEST_CHAT_MODEL", "gpt-4o-mini")
+
+
+def main():
+    client = openai.Client()
+
+    messages = [
+        {
+            "role": "user",
+            "content": "Answer in up to 3 words: Which ocean contains Bouvet Island?",
+        }
+    ]
+
+    chat_completion = client.chat.completions.create(model=CHAT_MODEL, messages=messages)
+    print(chat_completion.choices[0].message.content)
+
+
+if __name__ == "__main__":
+    main()

instrumentation/elastic-opentelemetry-instrumentation-openai/examples/embeddings.py

@@ -0,0 +1,65 @@
+# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+# or more contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+import numpy as np
+import openai
+
+EMBEDDINGS_MODEL = os.environ.get("TEST_EMBEDDINGS_MODEL", "text-embedding-3-small")
+
+
+def main():
+    client = openai.Client()
+
+    products = [
+        "Search: Ingest your data, and explore Elastic's machine learning and retrieval augmented generation (RAG) capabilities."
+        "Observability: Unify your logs, metrics, traces, and profiling at scale in a single platform.",
+        "Security: Protect, investigate, and respond to cyber threats with AI-driven security analytics."
+        "Elasticsearch: Distributed, RESTful search and analytics.",
+        "Kibana: Visualize your data. Navigate the Stack.",
+        "Beats: Collect, parse, and ship in a lightweight fashion.",
+        "Connectors: Connect popular databases, file systems, collaboration tools, and more.",
+        "Logstash: Ingest, transform, enrich, and output.",
+    ]
+
+    # Generate embeddings for each product. Keep them in an array instead of a vector DB.
+    product_embeddings = []
+    for product in products:
+        product_embeddings.append(create_embedding(client, product))
+
+    query_embedding = create_embedding(client, "What can help me connect to a database?")
+
+    # Calculate cosine similarity between the query and document embeddings
+    similarities = []
+    for product_embedding in product_embeddings:
+        similarity = np.dot(query_embedding, product_embedding) / (
+            np.linalg.norm(query_embedding) * np.linalg.norm(product_embedding)
+        )
+        similarities.append(similarity)
+
+    # Get the index of the most similar document
+    most_similar_index = np.argmax(similarities)
+
+    print(products[most_similar_index])
+
+
+def create_embedding(client, text):
+    return client.embeddings.create(input=[text], model=EMBEDDINGS_MODEL, encoding_format="float").data[0].embedding
+
+
+if __name__ == "__main__":
+    main()

instrumentation/elastic-opentelemetry-instrumentation-openai/ollama.env

@@ -0,0 +1,10 @@
+# Env to run the integration tests against a local Ollama.
+OPENAI_BASE_URL=http://127.0.0.1:11434/v1
+OPENAI_API_KEY=notused
+
+# These models may be substituted in the future with inexpensive to run, newer
+# variants.
+TEST_CHAT_MODEL=qwen2.5:0.5b
+TEST_EMBEDDINGS_MODEL=all-minilm:33m
+
+OTEL_SERVICE_NAME=elastic-opentelemetry-instrumentation-openai