Instrumentation weaviate spans

instrumentation-genai/opentelemetry-instrumentation-weaviate/README.rst

@@ -0,0 +1,50 @@
+OpenTelemetry Weaviate Instrumentation
+====================================
+
+|pypi|
+
+.. |pypi| image:: https://badge.fury.io/py/opentelemetry-instrumentation-weaviate.svg
+   :target: https://pypi.org/project/opentelemetry-instrumentation-weaviate/
+
+This library allows tracing vector database operations made by the
+`Weaviate Python Client <https://pypi.org/project/weaviate-client/>`_. It also captures
+the duration of the operations and usage metrics.
+
+Installation
+------------
+
+If your application is already instrumented with OpenTelemetry, add this
+package to your requirements.
+::
+
+    pip install opentelemetry-instrumentation-weaviate
+
+Usage
+-----
+
+.. code:: python
+
+    from weaviate import Client
+    from opentelemetry.instrumentation.weaviate import WeaviateInstrumentor
+
+    WeaviateInstrumentor().instrument()
+
+    client = Client("http://localhost:8080")
+    # Your Weaviate operations will now be traced
+
+Configuration
+~~~~~~~~~~~~~
+
+You can configure the instrumentation using environment variables:
+
+- ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT``: Set to ``true`` to capture query content and results (default: ``false``)
+
+Supported Operations
+~~~~~~~~~~~~~~~~~~~~
+
+This instrumentation supports tracing the following Weaviate client operations:
+
+# TODO: Update this to the actual operations supported by the Weaviate client
+
+API
+---

instrumentation-genai/opentelemetry-instrumentation-weaviate/examples/Dockerfile

@@ -0,0 +1,12 @@
+# Use the official Ollama base image
+FROM ollama/ollama:latest
+
+# Run ollama pull commands to download the models during the image build process
+# The 'ollama serve' is needed for 'ollama pull' to work during build
+# We run it in the background, pull models, then kill it using its PID.
+RUN ollama serve & \
+    OLLAMA_PID=$! && \ 
+    sleep 5 && \       
+    ollama pull nomic-embed-text:latest && \
+    ollama pull llama3.2:latest && \
+    kill $OLLAMA_PID   

instrumentation-genai/opentelemetry-instrumentation-weaviate/examples/docker-compose.yml

@@ -0,0 +1,48 @@
+---
+services:
+  weaviate:
+    command:
+    - --host
+    - 0.0.0.0
+    - --port
+    - '8080'
+    - --scheme
+    - http
+    image: cr.weaviate.io/semitechnologies/weaviate:1.31.2
+    ports:
+    - 8080:8080
+    - 50051:50051
+    - 2112:2112
+    volumes:
+    - weaviate_data:/var/lib/weaviate
+    restart: on-failure:0
+    environment:
+      QUERY_DEFAULTS_LIMIT: 25
+      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
+      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
+      ENABLE_API_BASED_MODULES: 'true'
+      ENABLE_MODULES: 'text2vec-ollama,generative-ollama'
+
+      DEFAULT_VECTORIZER_MODULE: 'text2vec-ollama'
+      TEXT2VEC_OLLAMA_API_ENDPOINT: 'http://ollama:11434'
+      TEXT2VEC_OLLAMA_MODEL: 'nomic-embed-text:latest'
+      GENERATIVE_OLLAMA_API_ENDPOINT: 'http://ollama:11434'
+      GENERATIVE_OLLAMA_MODEL: 'llama3.2:latest'
+
+      CLUSTER_HOSTNAME: 'node1'
+      PROMETHEUS_MONITORING_ENABLED: 'true'
+    depends_on: 
+      - ollama
+
+  ollama:
+    build:
+      context: . 
+      dockerfile: Dockerfile 
+    image: my-ollama-with-models:latest 
+    ports:
+      - "11434:11434"
+    restart: on-failure
+
+volumes:
+  weaviate_data:
+  ollama_models: 

instrumentation-genai/opentelemetry-instrumentation-weaviate/examples/manual/.env

@@ -0,0 +1,4 @@
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
+OTEL_EXPORTER_OTLP_PROTOCOL=grpc
+
+OTEL_SERVICE_NAME=opentelemetry-python-weaviate

instrumentation-genai/opentelemetry-instrumentation-weaviate/examples/manual/README.rst

@@ -0,0 +1,42 @@
+OpenTelemetry Weaviate-Client Manual Instrumentation Example
+======================================================
+
+This is an example of how to instrument weaviate-client manually,
+using `opentelemetry-instrument`.
+
+When :code:`example_v{3,4}.py <main.py>`_ is run, it exports traces to an OTLP-compatible endpoint.
+Traces include details such as the span name and other attributes.
+
+Note: :code:`.env <.env>`_ file configures additional environment variables:
+- :code:`OTEL_LOGS_EXPORTER=otlp` to specify exporter type.
+- :code:`OTEL_EXPORTER_OTLP_ENDPOINT` to specify the endpoint for exporting traces (default is http://localhost:4317).
+
+Setup
+-----
+
+Anytime a bash example has {3,4}, choose which version of weaviate-client you mean to use.
+
+Run the docker environment contained in ../docker-compose.yml to start a Weaviate instance and ollama with built in embeddings.
+
+An OTLP compatible endpoint should be listening for traces http://localhost:4317.
+If not, update :code:`OTEL_EXPORTER_OTLP_ENDPOINT` as well.
+
+Next, set up a virtual environment like this:
+
+::
+
+    python3 -m venv .venv
+    source .venv/bin/activate
+    pip install "python-dotenv[cli]"
+    pip install -r requirements_v{3,4}.txt
+
+Run
+---
+
+Run the example like this:
+
+::
+
+    dotenv run -- python example_v{3,4}.py
+
+You should see spans around article creation, retrieval, and deletion. There should be an example span on near_text search as well