Skip to content

Commit 815bfa5

Browse files
authored
huggingface[major]: Reduce disk footprint by 95% by making large dependencies optional (#31268)
**Description:** `langchain_huggingface` has a very large installation size of around 600 MB (on a Mac with Python 3.11). This is due to its dependency on `sentence-transformers`, which in turn depends on `torch`, which is 320 MB all by itself. Similarly, the depedency on `transformers` adds another set of heavy dependencies. With those dependencies removed, the installation of `langchain_huggingface` only takes up ~26 MB. This is only 5 % of the full installation! These libraries are not necessary to use `langchain_huggingface`'s API wrapper classes, only for local inferences/embeddings. All import statements for those two libraries already have import guards in place (try/catch with a helpful "please install x" message). This PR therefore moves those two libraries to an optional dependency group `full`. So a `pip install langchain_huggingface` will only install the lightweight version, and a `pip install "langchain_huggingface[full]"` will install all dependencies. I know this may break existing code, because `sentence-transformers` and `transformers` are now no longer installed by default. Given that users will see helpful error messages when that happens, and the major impact of this small change, I hope that you will still consider this PR. **Dependencies:** No new dependencies, but new optional grouping.
1 parent ae3551c commit 815bfa5

File tree

1 file changed

+7
-3
lines changed

1 file changed

+7
-3
lines changed

libs/partners/huggingface/pyproject.toml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,6 @@ requires-python = ">=3.9"
99
dependencies = [
1010
"langchain-core<1.0.0,>=0.3.59",
1111
"tokenizers>=0.19.1",
12-
"transformers>=4.39.0",
13-
"sentence-transformers>=2.6.0",
1412
"huggingface-hub>=0.30.2",
1513
]
1614
name = "langchain-huggingface"
@@ -23,6 +21,12 @@ readme = "README.md"
2321
"Release Notes" = "https://github.com/langchain-ai/langchain/releases?q=tag%3A%22langchain-huggingface%3D%3D0%22&expanded=true"
2422
repository = "https://github.com/langchain-ai/langchain"
2523

24+
[project.optional-dependencies]
25+
full = [
26+
"transformers>=4.39.0",
27+
"sentence-transformers>=2.6.0",
28+
]
29+
2630
[dependency-groups]
2731
test = [
2832
"pytest<8.0.0,>=7.3.0",
@@ -53,7 +57,7 @@ target-version = "py39"
5357

5458
[tool.ruff.lint]
5559
select = ["E", "F", "I", "T201", "UP"]
56-
ignore = [ "UP007", ]
60+
ignore = ["UP007"]
5761

5862
[tool.coverage.run]
5963
omit = ["tests/*"]

0 commit comments

Comments
 (0)