Skip to content

RAM Spike when I import instructor #2205

@AbdealiLoKo

Description

@AbdealiLoKo

Hi, I've been using the SDKs for my LLMs (openai, google.genai, etc.) so far which I use in a fastapi application I am working on which aims to support multiple LLM providers.
But recently got introduced to instructor which I was hoping can reduce some of my boiler plate code.

When I tried instructor I found my RAM utilization was spiking. When I debugged, I found that instructor was the cause.

Basically RAM stats when I import:

  1. sys (python builtin) -> 15MB
  2. instructor -> ~290MB (19x more)
  3. google.genai -> ~90MB
  4. openai -> ~54MB
  5. langgraph/langchain -> ~15MB

Any other library doesn't seem to be taking more than ~90MB making instructor 3x more RAM intensive than others

Reproducible snippets:

$ /usr/bin/time venv/bin/python -c "import sys"
0.02user 0.00system 0:00.03elapsed 91%CPU (0avgtext+0avgdata 15104maxresident)k
0inputs+0outputs (0major+1724minor)pagefaults 0swaps

$ /usr/bin/time venv/bin/python -c "import numpy"
1.16user 0.02system 0:00.21elapsed 547%CPU (0avgtext+0avgdata 30680maxresident)k
0inputs+0outputs (0major+3966minor)pagefaults 0swaps

$ /usr/bin/time venv/bin/python -c "import instructor"
7.64user 1.23system 0:08.89elapsed 99%CPU (0avgtext+0avgdata 290912maxresident)k
0inputs+0outputs (0major+83542minor)pagefaults 0swaps

$ /usr/bin/time venv/bin/python -c "import google.genai"
0.99user 0.11system 0:01.11elapsed 99%CPU (0avgtext+0avgdata 90708maxresident)k
0inputs+0outputs (0major+18901minor)pagefaults 0swaps

$ /usr/bin/time venv/bin/python -c "import openai"
0.80user 0.07system 0:00.87elapsed 99%CPU (0avgtext+0avgdata 54152maxresident)k
0inputs+0outputs (0major+10622minor)pagefaults 0swaps

$ /usr/bin/time venv/bin/python -c "import langgraph"
0.03user 0.00system 0:00.03elapsed 94%CPU (0avgtext+0avgdata 14976maxresident)k
0inputs+0outputs (0major+1723minor)pagefaults 0swaps

$ /usr/bin/time venv/bin/python -c "import langchain"
0.02user 0.01system 0:00.05elapsed 71%CPU (0avgtext+0avgdata 15104maxresident)k
8inputs+8outputs (0major+1744minor)pagefaults 0swaps

Versions being used:

  • Ubuntu 24.04 with WSL2 on Windows 11
  • Python 3.11.14
  • instructor 1.13.0 (same with 1.14.5 also) - Installed with instructor[anthropic,vertexai,bedrock,google-genai]

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is neededstatus:needs-investigationIssue needs investigation to determine scope

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions