Skip to content

Commit 21053f7

Browse files
github-actions[bot]brettlangdon
github-actions[bot]
and
brettlangdon
authored
ci: add benchmark for ddtrace.internal.packages.update_imported_dependencies [backport 2.21] (#12374)
Backport 3470338 from #12318 to 2.21. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) Co-authored-by: Brett Langdon <[email protected]>
1 parent 4bc96f4 commit 21053f7

File tree

4 files changed

+304
-1
lines changed

4 files changed

+304
-1
lines changed

‎benchmarks/base/requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,5 +4,5 @@ pyperf
44
pyyaml
55
attrs
66
httpretty==1.1.4
7-
tenacity==8.0.0
7+
tenacity~=8.0
88
viztracer

‎benchmarks/packages_update_imported_dependencies/config.yaml‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
import_one: &import_one
2+
use_cache: False
3+
include_sys_modules: False
4+
imported_deps:
5+
- aiohttp
6+
import_one_stdlib: &import_one_stdlib
7+
<<: *import_one
8+
imported_deps:
9+
- logging
10+
import_one_unknown: &import_one_unknown
11+
<<: *import_one
12+
imported_deps:
13+
- made_up_module_name_that_doesnt_exist
14+
import_one_cache:
15+
<<: *import_one
16+
use_cache: True
17+
import_one_stdlib_cache:
18+
<<: *import_one_stdlib
19+
use_cache: True
20+
import_one_unknown_cache:
21+
<<: *import_one
22+
use_cache: True
23+
import_many: &import_many
24+
<<: *import_one
25+
imported_deps:
26+
- aiohttp
27+
- aiosignal
28+
- anthropic
29+
- anyio
30+
- attrs
31+
- boto3
32+
- botocore
33+
- httpcore
34+
- httpx
35+
- langchain
36+
- numpy
37+
- openai
38+
- opentracing
39+
- orjson
40+
- pandas
41+
- pluggy
42+
- pytz
43+
- six
44+
- tzdata
45+
- urllib3
46+
- vcrpy
47+
- yarl
48+
- zipp
49+
import_many_unknown: &import_many_unknown
50+
<<: *import_one
51+
imported_deps:
52+
- vkdhddreuo
53+
- ytxrdiivqn
54+
- nwunnqugfn
55+
- rfrmygohvb
56+
- xbwiuxpkvc
57+
- kntwvlgxsw
58+
- htdwaabgbd
59+
- sbtytehntv
60+
- yksoozwdeu
61+
- iylkyqtwaz
62+
- gxxzmynpox
63+
- tmmrarwxvw
64+
- hsymhsjcdm
65+
- niuoavhyjl
66+
- tbgjliuzje
67+
- xaftiibbee
68+
- nqbprpmqmf
69+
- hvdgkpcqhp
70+
- kwyimfrsam
71+
- chrkdaxenj
72+
- uksjgwzheh
73+
- hezilzavbj
74+
- qqfhnyfogb
75+
- morqdnjyba
76+
- mgpgfnswpu
77+
- qtccjvwaqd
78+
- xbogvdsmfx
79+
- dtshlffojt
80+
- nubduwftan
81+
- lumxogqtwf
82+
- yexpbwevnk
83+
- efibxffkaq
84+
- pyhcllqhol
85+
- niklokhmbz
86+
- dhzybycdxt
87+
- posnydtbrk
88+
- drshkcybdy
89+
- vgsbkpzobj
90+
- uxqeteebxn
91+
- wyookkhxxg
92+
- finnybsqih
93+
- gtdliydysv
94+
- lfibrspulf
95+
- auextzkcks
96+
- wtfappvbom
97+
- cesvtlfpar
98+
- fcobjvtyqx
99+
- zbkfjgfliu
100+
- rkqnmrehxt
101+
- ufjhfvmjpe
102+
- cvrrfkronk
103+
- timtmseroq
104+
- sascxaxzsr
105+
- hsreaxzjfl
106+
- qzrualbxrw
107+
- munkkwwogu
108+
- zaipilmoyv
109+
- gdchrmvpww
110+
- ebbignpbtp
111+
- xfogobyazm
112+
- wkwnvirvqt
113+
- yfixwsczul
114+
- fuusbgqujy
115+
- yjiigvgpvk
116+
- efyacaicht
117+
- jyhmocodtz
118+
- ewchftonkf
119+
- qauopafaxy
120+
- mtszxfzjuh
121+
- xcasjvrteo
122+
- ckdtjtxpkm
123+
- wnyvdueczd
124+
- xarajldokg
125+
- aifrpfocki
126+
- zwqnrwkggf
127+
- wlcwigswvq
128+
- wvubwdpdjr
129+
- jggghyfsyd
130+
- wfjuqhpelk
131+
- iwfkbhpsvt
132+
- ibeikdbtex
133+
- yjjeqjnedf
134+
- okgxdpsosi
135+
- agnzpqzgcz
136+
- csildvwwrv
137+
- sroinswkaa
138+
- qbwhmgtepw
139+
- iafalhccur
140+
- jmwncfmgxj
141+
- hrvmtfidat
142+
- ztsdooxbft
143+
- bmantrhugw
144+
- nzyxcxzxlv
145+
- ejoivpknor
146+
- cylqvpaenu
147+
- axftwwlabn
148+
- kfgirmnkpn
149+
- rnaodfsqtv
150+
- auaemnhznh
151+
- iyfgdvyqqr
152+
import_many_stdlib: &import_many_stdlib
153+
<<: *import_many
154+
include_sys_modules: True
155+
imported_deps: []
156+
import_many_cached:
157+
<<: *import_many
158+
use_cache: True
159+
import_many_unknown_cached:
160+
<<: *import_many_unknown
161+
use_cache: True
162+
import_many_stdlib_cached:
163+
<<: *import_many_stdlib
164+
use_cache: True

‎benchmarks/packages_update_imported_dependencies/requirements_scenario.txt‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Borrowed from langchain .riot/requirements/a311bc2.txt
2+
ai21==3.0.1
3+
ai21-tokenizer==0.12.0
4+
aiohttp==3.9.5
5+
aiosignal==1.3.1
6+
annotated-types==0.7.0
7+
anthropic==0.40.0
8+
anyio==4.7.0
9+
async-timeout==4.0.3
10+
attrs==24.2.0
11+
boto3==1.35.76
12+
botocore==1.35.76
13+
certifi==2024.8.30
14+
charset-normalizer==3.4.0
15+
cohere==5.13.3
16+
coverage[toml]==7.6.9
17+
dataclasses-json==0.6.7
18+
defusedxml==0.7.1
19+
distro==1.9.0
20+
exceptiongroup==1.2.2
21+
fastavro==1.9.7
22+
filelock==3.16.1
23+
frozenlist==1.5.0
24+
fsspec==2024.10.0
25+
greenlet==3.0.3
26+
h11==0.14.0
27+
httpcore==1.0.7
28+
httpx==0.27.2
29+
httpx-sse==0.4.0
30+
huggingface-hub==0.26.5
31+
hypothesis==6.45.0
32+
idna==3.10
33+
importlib-metadata==8.5.0
34+
iniconfig==2.0.0
35+
jiter==0.8.0
36+
jmespath==1.0.1
37+
jsonpatch==1.33
38+
jsonpointer==3.0.0
39+
langchain==0.3.10
40+
langchain-anthropic==0.3.0
41+
langchain-aws==0.2.9
42+
langchain-cohere==0.3.3
43+
langchain-community==0.3.10
44+
langchain-core==0.3.22
45+
langchain-experimental==0.3.3
46+
langchain-openai==0.2.11
47+
langchain-pinecone==0.2.0
48+
langchain-text-splitters==0.3.2
49+
langsmith==0.1.147
50+
marshmallow==3.23.1
51+
mock==5.1.0
52+
multidict==6.1.0
53+
mypy-extensions==1.0.0
54+
numexpr==2.8.5
55+
numpy==1.26.4
56+
openai==1.57.0
57+
opentracing==2.4.0
58+
orjson==3.10.12
59+
packaging==24.2
60+
pandas==2.2.3
61+
parameterized==0.9.0
62+
pinecone-client==5.0.1
63+
pinecone-plugin-inference==1.1.0
64+
pinecone-plugin-interface==0.0.7
65+
pluggy==1.5.0
66+
propcache==0.2.1
67+
psutil==6.1.0
68+
pydantic==2.10.3
69+
pydantic-core==2.27.1
70+
pydantic-settings==2.6.1
71+
pytest==8.3.4
72+
pytest-asyncio==0.23.7
73+
pytest-cov==6.0.0
74+
pytest-mock==3.14.0
75+
pytest-randomly==3.10.1
76+
python-dateutil==2.9.0.post0
77+
python-dotenv==1.0.1
78+
pytz==2024.2
79+
pyyaml==6.0.2
80+
regex==2024.11.6
81+
requests==2.32.3
82+
requests-toolbelt==1.0.0
83+
s3transfer==0.10.4
84+
sentencepiece==0.2.0
85+
six==1.17.0
86+
sniffio==1.3.1
87+
sortedcontainers==2.4.0
88+
sqlalchemy==2.0.36
89+
tabulate==0.9.0
90+
tenacity==8.5.0
91+
tiktoken==0.8.0
92+
tokenizers==0.21.0
93+
tomli==2.2.1
94+
tqdm==4.67.1
95+
types-requests==2.31.0.6
96+
types-urllib3==1.26.25.14
97+
typing-extensions==4.12.2
98+
typing-inspect==0.9.0
99+
tzdata==2024.2
100+
urllib3==1.26.20
101+
vcrpy==5.1.0
102+
wrapt==1.17.0
103+
yarl==1.18.3
104+
zipp==3.21.0

‎benchmarks/packages_update_imported_dependencies/scenario.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import sys
2+
3+
import bm
4+
5+
from ddtrace.internal.packages import get_module_distribution_versions
6+
from ddtrace.internal.packages import get_package_distributions
7+
from ddtrace.internal.telemetry.data import update_imported_dependencies
8+
9+
10+
class PackagesUpdateImportedDependencies(bm.Scenario):
11+
imported_deps: list[str]
12+
use_cache: bool
13+
include_sys_modules: bool
14+
15+
def clear_caches(self, use_cache: bool = False):
16+
if not use_cache:
17+
get_module_distribution_versions.cache_clear()
18+
if hasattr(get_package_distributions, "cache_clear"):
19+
get_package_distributions.cache_clear()
20+
elif hasattr(get_package_distributions, "__callonce_result__"):
21+
del get_package_distributions.__callonce_result__
22+
23+
def run(self):
24+
# Clear any initial caches
25+
self.clear_caches()
26+
27+
if self.include_sys_modules:
28+
self.imported_deps.extend(sys.modules.keys())
29+
30+
def _(loops):
31+
for _ in range(loops):
32+
self.clear_caches(self.use_cache)
33+
update_imported_dependencies({}, self.imported_deps)
34+
35+
yield _

0 commit comments

Comments
 (0)