-
-
Notifications
You must be signed in to change notification settings - Fork 988
Open
Labels
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is neededstatus:needs-investigationIssue needs investigation to determine scopeIssue needs investigation to determine scope
Description
Hi, I've been using the SDKs for my LLMs (openai, google.genai, etc.) so far which I use in a fastapi application I am working on which aims to support multiple LLM providers.
But recently got introduced to instructor which I was hoping can reduce some of my boiler plate code.
When I tried instructor I found my RAM utilization was spiking. When I debugged, I found that instructor was the cause.
Basically RAM stats when I import:
sys(python builtin) -> 15MBinstructor-> ~290MB (19x more)google.genai-> ~90MBopenai-> ~54MBlanggraph/langchain-> ~15MB
Any other library doesn't seem to be taking more than ~90MB making instructor 3x more RAM intensive than others
Reproducible snippets:
$ /usr/bin/time venv/bin/python -c "import sys"
0.02user 0.00system 0:00.03elapsed 91%CPU (0avgtext+0avgdata 15104maxresident)k
0inputs+0outputs (0major+1724minor)pagefaults 0swaps
$ /usr/bin/time venv/bin/python -c "import numpy"
1.16user 0.02system 0:00.21elapsed 547%CPU (0avgtext+0avgdata 30680maxresident)k
0inputs+0outputs (0major+3966minor)pagefaults 0swaps
$ /usr/bin/time venv/bin/python -c "import instructor"
7.64user 1.23system 0:08.89elapsed 99%CPU (0avgtext+0avgdata 290912maxresident)k
0inputs+0outputs (0major+83542minor)pagefaults 0swaps
$ /usr/bin/time venv/bin/python -c "import google.genai"
0.99user 0.11system 0:01.11elapsed 99%CPU (0avgtext+0avgdata 90708maxresident)k
0inputs+0outputs (0major+18901minor)pagefaults 0swaps
$ /usr/bin/time venv/bin/python -c "import openai"
0.80user 0.07system 0:00.87elapsed 99%CPU (0avgtext+0avgdata 54152maxresident)k
0inputs+0outputs (0major+10622minor)pagefaults 0swaps
$ /usr/bin/time venv/bin/python -c "import langgraph"
0.03user 0.00system 0:00.03elapsed 94%CPU (0avgtext+0avgdata 14976maxresident)k
0inputs+0outputs (0major+1723minor)pagefaults 0swaps
$ /usr/bin/time venv/bin/python -c "import langchain"
0.02user 0.01system 0:00.05elapsed 71%CPU (0avgtext+0avgdata 15104maxresident)k
8inputs+8outputs (0major+1744minor)pagefaults 0swapsVersions being used:
- Ubuntu 24.04 with WSL2 on Windows 11
- Python 3.11.14
- instructor 1.13.0 (same with 1.14.5 also) - Installed with
instructor[anthropic,vertexai,bedrock,google-genai]
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is neededstatus:needs-investigationIssue needs investigation to determine scopeIssue needs investigation to determine scope