feat(openai): add OpenAI integration [backport #5488 to 1.13] (#5674)

Add an integration for the [OpenAI
library](https://github.com/openai/openai-python). This integration
provides tracing for the completion, embeddings and chat completion
endpoints along with cost estimation metrics and prompt/completion
sampling logs.

Each log, metric and trace are tagged with service, env, version, OpenAI
model, OpenAI endpoint and OpenAI organization.

[Docs

preview](https://output.circle-artifacts.com/output/job/fe3599b8-952e-4ceb-ac4f-0f15503e9c0d/artifacts/0/tmp/docs/integrations.html#openai)

## Design

### Logs

A new log writer implementation is added to submit logs. Logs are
submitted direct to intake following a similar approach that
[kyle-verhoog/datadog-python](https://github.com/Kyle-Verhoog/datadog-python/blob/main/datadog/_logging.py)
and the [.NET
tracer](DataDog/dd-trace-dotnet#2240) have taken
already.

### Metrics

A statsd client is used specifically for the OpenAI integration.


## Testing

Testing is done using VCR to record requests made to OpenAI to ensure
ease, consistency and reliability in test cases.

Logs and metrics are tested using mocking of the clients.

Several integration tests using snapshots and subprocess testing ensure
that the integration works in a real world OpenAI application.

A manual test app was also used:
https://gist.github.com/Kyle-Verhoog/1f263ed0aade076b313167d1ba3bfa16


## Risk

- Currently, logs, traces and metrics are all collected, buffered and
sent individually through their respective pipelines. Due to this, there
is risk that disparity occurs between the tagging and submission of the
data. There is also a performance risk as this data is not aggregated or
batched when submitted.
- Prompts and completions are captured on spans by default with a
default limit on the length of the data. This limit only applies to each
prompt/completion individually but requests can contain several prompts
and completions. If there are many prompts and completions of great
length then there is a risk of performance overhead of encoding and
transmitting the data.
- Logs and metrics clients are specified specifically for this
integration. If another integration were to introduce logs then there
would be a need for another log writer. Having several log writers could
induce thread contention and high memory usage.


Co-authored-by: lievan <[email protected]>
Co-authored-by: Kari Halsted <[email protected]>
Co-authored-by: Federico Mon <[email protected]>

Author: 4 people

.circleci/config.yml

@@ -372,6 +372,14 @@ jobs:
       - run_test:
           pattern: "debugger"
 
+  openai:
+    <<: *machine_executor
+    parallelism: 4
+    steps:
+      - run_test:
+          pattern: 'openai'
+          snapshot: true
+
   opentracer:
     <<: *contrib_job
     steps:
@@ -1072,6 +1080,7 @@ requires_tests: &requires_tests
     - mongoengine
     - mysqlconnector
     - mysqlpython
+    - openai
     - opentracer
     - opentelemetry
     - psycopg
@@ -1171,6 +1180,7 @@ workflows:
       - mongoengine: *requires_base_venvs
       - mysqlconnector: *requires_base_venvs
       - mysqlpython: *requires_base_venvs
+      - openai: *requires_base_venvs
       - opentracer: *requires_base_venvs
       - opentelemetry: *requires_base_venvs
       - psycopg: *requires_base_venvs

.riot/requirements/11fafab.txt

@@ -0,0 +1,60 @@
+#
+# This file is autogenerated by pip-compile with Python 3.10
+# by the following command:
+#
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/11fafab.in
+#
+aiohttp==3.8.4
+aiosignal==1.3.1
+async-timeout==4.0.2
+attrs==23.1.0
+certifi==2022.12.7
+charset-normalizer==3.1.0
+contourpy==1.0.7
+coverage[toml]==7.2.3
+cycler==0.11.0
+et-xmlfile==1.1.0
+exceptiongroup==1.1.1
+fonttools==4.39.3
+frozenlist==1.3.3
+hypothesis==6.45.0
+idna==3.4
+iniconfig==2.0.0
+joblib==1.2.0
+kiwisolver==1.4.4
+matplotlib==3.7.1
+mock==5.0.2
+multidict==6.0.4
+numpy==1.24.3
+openai[embeddings]==0.27.2
+openpyxl==3.1.2
+opentracing==2.4.0
+packaging==23.1
+pandas==2.0.1
+pandas-stubs==2.0.0.230412
+pillow==9.5.0
+plotly==5.14.1
+pluggy==1.0.0
+pyparsing==3.0.9
+pytest==7.3.1
+pytest-asyncio==0.21.0
+pytest-cov==4.0.0
+pytest-mock==3.10.0
+python-dateutil==2.8.2
+pytz==2023.3
+pyyaml==6.0
+requests==2.29.0
+scikit-learn==1.2.2
+scipy==1.10.1
+six==1.16.0
+sortedcontainers==2.4.0
+tenacity==8.2.2
+threadpoolctl==3.1.0
+tomli==2.0.1
+tqdm==4.65.0
+types-pytz==2023.3.0.0
+tzdata==2023.3
+urllib3==1.26.15
+vcrpy==4.2.1
+wrapt==1.15.0
+yarl==1.9.2

.riot/requirements/1522688.txt

@@ -0,0 +1,58 @@
+#
+# This file is autogenerated by pip-compile with Python 3.11
+# by the following command:
+#
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/1522688.in
+#
+aiohttp==3.8.4
+aiosignal==1.3.1
+async-timeout==4.0.2
+attrs==23.1.0
+certifi==2022.12.7
+charset-normalizer==3.1.0
+contourpy==1.0.7
+coverage[toml]==7.2.3
+cycler==0.11.0
+et-xmlfile==1.1.0
+fonttools==4.39.3
+frozenlist==1.3.3
+hypothesis==6.45.0
+idna==3.4
+iniconfig==2.0.0
+joblib==1.2.0
+kiwisolver==1.4.4
+matplotlib==3.7.1
+mock==5.0.2
+multidict==6.0.4
+numpy==1.24.3
+openai[embeddings]==0.27.3
+openpyxl==3.1.2
+opentracing==2.4.0
+packaging==23.1
+pandas==2.0.1
+pandas-stubs==2.0.0.230412
+pillow==9.5.0
+plotly==5.14.1
+pluggy==1.0.0
+pyparsing==3.0.9
+pytest==7.3.1
+pytest-asyncio==0.21.0
+pytest-cov==4.0.0
+pytest-mock==3.10.0
+python-dateutil==2.8.2
+pytz==2023.3
+pyyaml==6.0
+requests==2.29.0
+scikit-learn==1.2.2
+scipy==1.10.1
+six==1.16.0
+sortedcontainers==2.4.0
+tenacity==8.2.2
+threadpoolctl==3.1.0
+tqdm==4.65.0
+types-pytz==2023.3.0.0
+tzdata==2023.3
+urllib3==1.26.15
+vcrpy==4.2.1
+wrapt==1.15.0
+yarl==1.9.2

.riot/requirements/16cc6d6.txt

@@ -0,0 +1,58 @@
+#
+# This file is autogenerated by pip-compile with Python 3.11
+# by the following command:
+#
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/16cc6d6.in
+#
+aiohttp==3.8.4
+aiosignal==1.3.1
+async-timeout==4.0.2
+attrs==23.1.0
+certifi==2022.12.7
+charset-normalizer==3.1.0
+contourpy==1.0.7
+coverage[toml]==7.2.3
+cycler==0.11.0
+et-xmlfile==1.1.0
+fonttools==4.39.3
+frozenlist==1.3.3
+hypothesis==6.45.0
+idna==3.4
+iniconfig==2.0.0
+joblib==1.2.0
+kiwisolver==1.4.4
+matplotlib==3.7.1
+mock==5.0.2
+multidict==6.0.4
+numpy==1.24.3
+openai[embeddings]==0.27.4
+openpyxl==3.1.2
+opentracing==2.4.0
+packaging==23.1
+pandas==2.0.1
+pandas-stubs==2.0.0.230412
+pillow==9.5.0
+plotly==5.14.1
+pluggy==1.0.0
+pyparsing==3.0.9
+pytest==7.3.1
+pytest-asyncio==0.21.0
+pytest-cov==4.0.0
+pytest-mock==3.10.0
+python-dateutil==2.8.2
+pytz==2023.3
+pyyaml==6.0
+requests==2.29.0
+scikit-learn==1.2.2
+scipy==1.10.1
+six==1.16.0
+sortedcontainers==2.4.0
+tenacity==8.2.2
+threadpoolctl==3.1.0
+tqdm==4.65.0
+types-pytz==2023.3.0.0
+tzdata==2023.3
+urllib3==1.26.15
+vcrpy==4.2.1
+wrapt==1.15.0
+yarl==1.9.2

.riot/requirements/17ab8b6.txt

@@ -0,0 +1,62 @@
+#
+# This file is autogenerated by pip-compile with Python 3.8
+# by the following command:
+#
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/17ab8b6.in
+#
+aiohttp==3.8.4
+aiosignal==1.3.1
+async-timeout==4.0.2
+attrs==23.1.0
+certifi==2022.12.7
+charset-normalizer==3.1.0
+contourpy==1.0.7
+coverage[toml]==7.2.3
+cycler==0.11.0
+et-xmlfile==1.1.0
+exceptiongroup==1.1.1
+fonttools==4.39.3
+frozenlist==1.3.3
+hypothesis==6.45.0
+idna==3.4
+importlib-resources==5.12.0
+iniconfig==2.0.0
+joblib==1.2.0
+kiwisolver==1.4.4
+matplotlib==3.7.1
+mock==5.0.2
+multidict==6.0.4
+numpy==1.24.3
+openai[embeddings]==0.27.3
+openpyxl==3.1.2
+opentracing==2.4.0
+packaging==23.1
+pandas==2.0.1
+pandas-stubs==2.0.0.230412
+pillow==9.5.0
+plotly==5.14.1
+pluggy==1.0.0
+pyparsing==3.0.9
+pytest==7.3.1
+pytest-asyncio==0.21.0
+pytest-cov==4.0.0
+pytest-mock==3.10.0
+python-dateutil==2.8.2
+pytz==2023.3
+pyyaml==6.0
+requests==2.29.0
+scikit-learn==1.2.2
+scipy==1.10.1
+six==1.16.0
+sortedcontainers==2.4.0
+tenacity==8.2.2
+threadpoolctl==3.1.0
+tomli==2.0.1
+tqdm==4.65.0
+types-pytz==2023.3.0.0
+tzdata==2023.3
+urllib3==1.26.15
+vcrpy==4.2.1
+wrapt==1.15.0
+yarl==1.9.2
+zipp==3.15.0

.riot/requirements/180d9c4.txt

@@ -0,0 +1,62 @@
+#
+# This file is autogenerated by pip-compile with Python 3.9
+# by the following command:
+#
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/180d9c4.in
+#
+aiohttp==3.8.4
+aiosignal==1.3.1
+async-timeout==4.0.2
+attrs==23.1.0
+certifi==2022.12.7
+charset-normalizer==3.1.0
+contourpy==1.0.7
+coverage[toml]==7.2.3
+cycler==0.11.0
+et-xmlfile==1.1.0
+exceptiongroup==1.1.1
+fonttools==4.39.3
+frozenlist==1.3.3
+hypothesis==6.45.0
+idna==3.4
+importlib-resources==5.12.0
+iniconfig==2.0.0
+joblib==1.2.0
+kiwisolver==1.4.4
+matplotlib==3.7.1
+mock==5.0.2
+multidict==6.0.4
+numpy==1.24.3
+openai[embeddings]==0.27.2
+openpyxl==3.1.2
+opentracing==2.4.0
+packaging==23.1
+pandas==2.0.1
+pandas-stubs==2.0.0.230412
+pillow==9.5.0
+plotly==5.14.1
+pluggy==1.0.0
+pyparsing==3.0.9
+pytest==7.3.1
+pytest-asyncio==0.21.0
+pytest-cov==4.0.0
+pytest-mock==3.10.0
+python-dateutil==2.8.2
+pytz==2023.3
+pyyaml==6.0
+requests==2.29.0
+scikit-learn==1.2.2
+scipy==1.10.1
+six==1.16.0
+sortedcontainers==2.4.0
+tenacity==8.2.2
+threadpoolctl==3.1.0
+tomli==2.0.1
+tqdm==4.65.0
+types-pytz==2023.3.0.0
+tzdata==2023.3
+urllib3==1.26.15
+vcrpy==4.2.1
+wrapt==1.15.0
+yarl==1.9.2
+zipp==3.15.0

.riot/requirements/1a7e69a.txt

@@ -0,0 +1,63 @@
+#
+# This file is autogenerated by pip-compile with Python 3.9
+# by the following command:
+#
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/1a7e69a.in
+#
+aiohttp==3.8.4
+aiosignal==1.3.1
+async-timeout==4.0.2
+attrs==23.1.0
+certifi==2022.12.7
+charset-normalizer==3.1.0
+contourpy==1.0.7
+coverage[toml]==7.2.3
+cycler==0.11.0
+et-xmlfile==1.1.0
+exceptiongroup==1.1.1
+fonttools==4.39.3
+frozenlist==1.3.3
+hypothesis==6.45.0
+idna==3.4
+importlib-resources==5.12.0
+iniconfig==2.0.0
+joblib==1.2.0
+kiwisolver==1.4.4
+matplotlib==3.7.1
+mock==5.0.2
+multidict==6.0.4
+numpy==1.24.3
+openai[embeddings]==0.26.5
+openpyxl==3.1.2
+opentracing==2.4.0
+packaging==23.1
+pandas==2.0.1
+pandas-stubs==2.0.0.230412
+pillow==9.5.0
+plotly==5.14.1
+pluggy==1.0.0
+pyparsing==3.0.9
+pytest==7.3.1
+pytest-asyncio==0.21.0
+pytest-cov==4.0.0
+pytest-mock==3.10.0
+python-dateutil==2.8.2
+pytz==2023.3
+pyyaml==6.0
+requests==2.29.0
+scikit-learn==1.2.2
+scipy==1.10.1
+six==1.16.0
+sklearn==0.0.post4
+sortedcontainers==2.4.0
+tenacity==8.2.2
+threadpoolctl==3.1.0
+tomli==2.0.1
+tqdm==4.65.0
+types-pytz==2023.3.0.0
+tzdata==2023.3
+urllib3==1.26.15
+vcrpy==4.2.1
+wrapt==1.15.0
+yarl==1.9.2
+zipp==3.15.0