Skip to content

Commit fb38602

Browse files
mrnicegyu11mrnicegyu11
authored
✨ Add: Tracing via opentelemetry (#6168)
1 parent 7f466b5 commit fb38602

File tree

152 files changed

+3059
-1069
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

152 files changed

+3059
-1069
lines changed

‎.env-devel‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -307,9 +307,11 @@ SESSION_COOKIE_SECURE=False
307307
SIMCORE_VCS_RELEASE_TAG=latest
308308
STUDIES_ACCESS_ANONYMOUS_ALLOWED=0
309309
STUDIES_DEFAULT_SERVICE_THUMBNAIL=https://via.placeholder.com/170x120.png
310-
TRACING_ENABLED=1
311-
TRACING_THRIFT_COMPACT_ENDPOINT=http://jaeger:5775
312-
TRACING_ZIPKIN_ENDPOINT=http://jaeger:9411
310+
TRACING_OPENTELEMETRY_COLLECTOR_BATCH_SIZE=2
311+
TRACING_OPENTELEMETRY_COLLECTOR_PORT=4318
312+
TRACING_OPENTELEMETRY_COLLECTOR_ENDPOINT=http://opentelemetry-collector
313+
TRACING_OPENTELEMETRY_COLLECTOR_SAMPLING_PERCENTAGE=100
314+
TRACING_OPENTELEMETRY_COLLECTOR_EXPORTER_ENDPOINT=http://jaeger:4318
313315
TRAEFIK_SIMCORE_ZONE=internal_simcore_stack
314316
TWILIO_ACCOUNT_SID=DUMMY
315317
TWILIO_AUTH_TOKEN=DUMMY

‎.github/dependabot.yml‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,6 @@ updates:
1212
assignees:
1313
- pcrespov
1414
ignore:
15-
- dependency-name: aiozipkin
16-
versions:
17-
- ">= 1.a, < 2"
1815
- dependency-name: docker-compose
1916
versions:
2017
- 1.28.2
@@ -41,9 +38,6 @@ updates:
4138
assignees:
4239
- pcrespov
4340
ignore:
44-
- dependency-name: aiozipkin
45-
versions:
46-
- ">= 1.a, < 2"
4741
- dependency-name: openapi-core
4842
versions:
4943
- "> 0.12.0, < 1"

‎Makefile‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -310,18 +310,19 @@ TableWidth=140;\
310310
printf "%24s | %90s | %12s | %12s\n" Name Endpoint User Password;\
311311
printf "%.$${TableWidth}s\n" "$$separator";\
312312
printf "$$rows" "oSparc platform" "http://$(get_my_ip).nip.io:9081";\
313-
printf "$$rows" "oSparc web API doc" "http://$(get_my_ip).nip.io:9081/dev/doc";\
314313
printf "$$rows" "oSparc public API doc" "http://$(get_my_ip).nip.io:8006/dev/doc";\
315-
printf "$$rows" "Postgres DB" "http://$(get_my_ip).nip.io:18080/?pgsql=postgres&username="$${POSTGRES_USER}"&db="$${POSTGRES_DB}"&ns=public" $${POSTGRES_USER} $${POSTGRES_PASSWORD};\
316-
printf "$$rows" "Portainer" "http://$(get_my_ip).nip.io:9000" admin adminadmin;\
317-
printf "$$rows" "Redis" "http://$(get_my_ip).nip.io:18081";\
314+
printf "$$rows" "oSparc web API doc" "http://$(get_my_ip).nip.io:9081/dev/doc";\
318315
printf "$$rows" "Dask Dashboard" "http://$(get_my_ip).nip.io:8787";\
319316
printf "$$rows" "Docker Registry" "http://$${REGISTRY_URL}/v2/_catalog" $${REGISTRY_USER} $${REGISTRY_PW};\
320317
printf "$$rows" "Invitations" "http://$(get_my_ip).nip.io:8008/dev/doc" $${INVITATIONS_USERNAME} $${INVITATIONS_PASSWORD};\
318+
printf "$$rows" "Jaeger" "http://$(get_my_ip).nip.io:16686";\
321319
printf "$$rows" "Payments" "http://$(get_my_ip).nip.io:8011/dev/doc" $${PAYMENTS_USERNAME} $${PAYMENTS_PASSWORD};\
320+
printf "$$rows" "Portainer" "http://$(get_my_ip).nip.io:9000" admin adminadmin;\
321+
printf "$$rows" "Postgres DB" "http://$(get_my_ip).nip.io:18080/?pgsql=postgres&username="$${POSTGRES_USER}"&db="$${POSTGRES_DB}"&ns=public" $${POSTGRES_USER} $${POSTGRES_PASSWORD};\
322322
printf "$$rows" "Rabbit Dashboard" "http://$(get_my_ip).nip.io:15672" admin adminadmin;\
323-
printf "$$rows" "Traefik Dashboard" "http://$(get_my_ip).nip.io:8080/dashboard/";\
323+
printf "$$rows" "Redis" "http://$(get_my_ip).nip.io:18081";\
324324
printf "$$rows" "Storage S3 Minio" "http://$(get_my_ip).nip.io:9001" 12345678 12345678;\
325+
printf "$$rows" "Traefik Dashboard" "http://$(get_my_ip).nip.io:8080/dashboard/";\
325326

326327
printf "\n%s\n" "⚠️ if a DNS is not used (as displayed above), the interactive services started via dynamic-sidecar";\
327328
echo "⚠️ will not be shown. The frontend accesses them via the uuid.services.YOUR_IP.nip.io:9081";

‎api/tests/requirements.txt‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,6 @@ aiohttp==3.10.5
66
# -r requirements.in
77
aiosignal==1.3.1
88
# via aiohttp
9-
async-timeout==4.0.3
10-
# via aiohttp
119
attrs==24.2.0
1210
# via
1311
# aiohttp
@@ -23,8 +21,6 @@ coverage==7.6.1
2321
# via
2422
# -r requirements.in
2523
# pytest-cov
26-
exceptiongroup==1.2.2
27-
# via pytest
2824
frozenlist==1.4.1
2925
# via
3026
# aiohttp
@@ -118,10 +114,6 @@ six==1.16.0
118114
# rfc3339-validator
119115
termcolor==2.4.0
120116
# via pytest-sugar
121-
tomli==2.0.1
122-
# via
123-
# coverage
124-
# pytest
125117
urllib3==2.2.2
126118
# via
127119
# -c ../../requirements/constraints.txt

‎docs/coding-conventions.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,3 +136,7 @@ Have a look at `ESLint`'s configuration files [.eslintrc.json](.eslintrc.json) a
136136
[mypy]:https://www.mypy-lang.org/
137137
[pep257]:https://peps.python.org/pep-0257/
138138
[pylint]:https://pylint.readthedocs.io/en/latest/
139+
140+
# My first osparc-simcore PR: common pitfalls
141+
142+
- Make sure to run `make mypy` and `make pylint`, as the associated github-actions are required to pass. If you include new dependencies in `requirements/*.in`, make sure to run `make touch && make reqs "upgrade=NAME_OF_YOUR_NEW_DEPENDENCY"`. It is best to do this inside a reproducible environment, for this purpose a shell inside a docker container can be used: Go to `osparc-simcore/requirements/tools` and run `make shell`. Inside the new shell the osparc-simcore repo is placed in `~`. Run `make reqs` from inside this shell.

‎packages/aws-library/requirements/_base.txt‎

Lines changed: 81 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,25 @@ botocore==1.34.131
5656
# s3transfer
5757
botocore-stubs==1.35.2
5858
# via types-aiobotocore
59+
certifi==2024.7.4
60+
# via
61+
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
62+
# -c requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
63+
# -c requirements/../../../packages/service-library/requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
64+
# -c requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
65+
# -c requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
66+
# -c requirements/../../../requirements/constraints.txt
67+
# requests
68+
charset-normalizer==3.3.2
69+
# via requests
5970
click==8.1.7
6071
# via typer
72+
deprecated==1.2.14
73+
# via
74+
# opentelemetry-api
75+
# opentelemetry-exporter-otlp-proto-grpc
76+
# opentelemetry-exporter-otlp-proto-http
77+
# opentelemetry-semantic-conventions
6178
dnspython==2.6.1
6279
# via email-validator
6380
email-validator==2.2.0
@@ -70,11 +87,20 @@ frozenlist==1.4.1
7087
# via
7188
# aiohttp
7289
# aiosignal
90+
googleapis-common-protos==1.65.0
91+
# via
92+
# opentelemetry-exporter-otlp-proto-grpc
93+
# opentelemetry-exporter-otlp-proto-http
94+
grpcio==1.66.0
95+
# via opentelemetry-exporter-otlp-proto-grpc
7396
idna==3.7
7497
# via
7598
# anyio
7699
# email-validator
100+
# requests
77101
# yarl
102+
importlib-metadata==8.0.0
103+
# via opentelemetry-api
78104
jmespath==1.0.1
79105
# via
80106
# boto3
@@ -93,6 +119,45 @@ multidict==6.0.5
93119
# via
94120
# aiohttp
95121
# yarl
122+
opentelemetry-api==1.26.0
123+
# via
124+
# -r requirements/../../../packages/service-library/requirements/_base.in
125+
# opentelemetry-exporter-otlp-proto-grpc
126+
# opentelemetry-exporter-otlp-proto-http
127+
# opentelemetry-instrumentation
128+
# opentelemetry-instrumentation-requests
129+
# opentelemetry-sdk
130+
# opentelemetry-semantic-conventions
131+
opentelemetry-exporter-otlp==1.26.0
132+
# via -r requirements/../../../packages/service-library/requirements/_base.in
133+
opentelemetry-exporter-otlp-proto-common==1.26.0
134+
# via
135+
# opentelemetry-exporter-otlp-proto-grpc
136+
# opentelemetry-exporter-otlp-proto-http
137+
opentelemetry-exporter-otlp-proto-grpc==1.26.0
138+
# via opentelemetry-exporter-otlp
139+
opentelemetry-exporter-otlp-proto-http==1.26.0
140+
# via opentelemetry-exporter-otlp
141+
opentelemetry-instrumentation==0.47b0
142+
# via opentelemetry-instrumentation-requests
143+
opentelemetry-instrumentation-requests==0.47b0
144+
# via -r requirements/../../../packages/service-library/requirements/_base.in
145+
opentelemetry-proto==1.26.0
146+
# via
147+
# opentelemetry-exporter-otlp-proto-common
148+
# opentelemetry-exporter-otlp-proto-grpc
149+
# opentelemetry-exporter-otlp-proto-http
150+
opentelemetry-sdk==1.26.0
151+
# via
152+
# -r requirements/../../../packages/service-library/requirements/_base.in
153+
# opentelemetry-exporter-otlp-proto-grpc
154+
# opentelemetry-exporter-otlp-proto-http
155+
opentelemetry-semantic-conventions==0.47b0
156+
# via
157+
# opentelemetry-instrumentation-requests
158+
# opentelemetry-sdk
159+
opentelemetry-util-http==0.47b0
160+
# via opentelemetry-instrumentation-requests
96161
orjson==3.10.7
97162
# via
98163
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
@@ -105,6 +170,10 @@ orjson==3.10.7
105170
# -r requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/_base.in
106171
pamqp==3.3.0
107172
# via aiormq
173+
protobuf==4.25.4
174+
# via
175+
# googleapis-common-protos
176+
# opentelemetry-proto
108177
pydantic==1.10.17
109178
# via
110179
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
@@ -151,6 +220,8 @@ referencing==0.29.3
151220
# -c requirements/../../../packages/service-library/requirements/./constraints.txt
152221
# jsonschema
153222
# jsonschema-specifications
223+
requests==2.32.3
224+
# via opentelemetry-exporter-otlp-proto-http
154225
rich==13.7.1
155226
# via
156227
# -r requirements/../../../packages/service-library/requirements/../../../packages/settings-library/requirements/_base.in
@@ -162,6 +233,8 @@ rpds-py==0.20.0
162233
# referencing
163234
s3transfer==0.10.2
164235
# via boto3
236+
setuptools==74.0.0
237+
# via opentelemetry-instrumentation
165238
sh==2.0.7
166239
# via -r requirements/_base.in
167240
shellingham==1.5.4
@@ -197,6 +270,7 @@ typing-extensions==4.12.2
197270
# via
198271
# aiodebug
199272
# faststream
273+
# opentelemetry-sdk
200274
# pydantic
201275
# typer
202276
# types-aiobotocore
@@ -212,10 +286,16 @@ urllib3==2.2.2
212286
# -c requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
213287
# -c requirements/../../../requirements/constraints.txt
214288
# botocore
289+
# requests
215290
wrapt==1.16.0
216-
# via aiobotocore
291+
# via
292+
# aiobotocore
293+
# deprecated
294+
# opentelemetry-instrumentation
217295
yarl==1.9.4
218296
# via
219297
# aio-pika
220298
# aiohttp
221299
# aiormq
300+
zipp==3.20.1
301+
# via importlib-metadata

‎packages/aws-library/requirements/_test.txt‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,13 +36,16 @@ botocore-stubs==1.35.2
3636
certifi==2024.7.4
3737
# via
3838
# -c requirements/../../../requirements/constraints.txt
39+
# -c requirements/_base.txt
3940
# requests
4041
cffi==1.17.0
4142
# via cryptography
4243
cfn-lint==1.10.3
4344
# via moto
4445
charset-normalizer==3.3.2
45-
# via requests
46+
# via
47+
# -c requirements/_base.txt
48+
# requests
4649
click==8.1.7
4750
# via
4851
# -c requirements/_base.txt
@@ -214,6 +217,7 @@ regex==2024.7.24
214217
# via cfn-lint
215218
requests==2.32.3
216219
# via
220+
# -c requirements/_base.txt
217221
# docker
218222
# jsonschema-path
219223
# moto
@@ -231,8 +235,10 @@ s3transfer==0.10.2
231235
# via
232236
# -c requirements/_base.txt
233237
# boto3
234-
setuptools==73.0.1
235-
# via moto
238+
setuptools==74.0.0
239+
# via
240+
# -c requirements/_base.txt
241+
# moto
236242
six==1.16.0
237243
# via
238244
# -c requirements/_base.txt

‎packages/aws-library/requirements/_tools.txt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,8 +68,9 @@ pyyaml==6.0.2
6868
# pre-commit
6969
ruff==0.6.1
7070
# via -r requirements/../../../requirements/devenv.txt
71-
setuptools==73.0.1
71+
setuptools==74.0.0
7272
# via
73+
# -c requirements/_base.txt
7374
# -c requirements/_test.txt
7475
# pip-tools
7576
tomlkit==0.13.2

‎packages/dask-task-models-library/requirements/_test.txt‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,6 @@ coverage==7.6.1
44
# via
55
# -r requirements/_test.in
66
# pytest-cov
7-
exceptiongroup==1.2.2
8-
# via pytest
97
faker==27.0.0
108
# via -r requirements/_test.in
119
flexcache==0.3
@@ -67,10 +65,6 @@ six==1.16.0
6765
# python-dateutil
6866
termcolor==2.4.0
6967
# via pytest-sugar
70-
tomli==2.0.1
71-
# via
72-
# coverage
73-
# pytest
7468
typing-extensions==4.12.2
7569
# via
7670
# -c requirements/_base.txt

‎packages/dask-task-models-library/requirements/_tools.txt‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -70,22 +70,12 @@ ruff==0.6.1
7070
# via -r requirements/../../../requirements/devenv.txt
7171
setuptools==73.0.1
7272
# via pip-tools
73-
tomli==2.0.1
74-
# via
75-
# -c requirements/_test.txt
76-
# black
77-
# build
78-
# mypy
79-
# pip-tools
80-
# pylint
8173
tomlkit==0.13.2
8274
# via pylint
8375
typing-extensions==4.12.2
8476
# via
8577
# -c requirements/_base.txt
8678
# -c requirements/_test.txt
87-
# astroid
88-
# black
8979
# mypy
9080
virtualenv==20.26.3
9181
# via pre-commit

0 commit comments

Comments
 (0)