Skip to content

Conversation

@takeshi-yoshimura
Copy link
Collaborator

  • track memory usage for bounce buffers and model buffers
  • add __version__
  • fix unittests to check memory usages
  • fix get_device_numa_node
  • fix missing bounce buffer release at SafeTensorsFileLoader.close()

hopefully this resolves #26
after all tests passe, I will update the version to v0.1.15 in pyproject.xml

Signed-off-by: Takeshi Yoshimura <[email protected]>
Signed-off-by: Takeshi Yoshimura <[email protected]>
Signed-off-by: Takeshi Yoshimura <[email protected]>
Signed-off-by: Takeshi Yoshimura <[email protected]>
Signed-off-by: Takeshi Yoshimura <[email protected]>
Signed-off-by: Takeshi Yoshimura <[email protected]>
Signed-off-by: Takeshi Yoshimura <[email protected]>
@takeshi-yoshimura takeshi-yoshimura merged commit 2385b26 into foundation-model-stack:main Jul 18, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extra GPU memory is used after loading model weights in distributed

1 participant