Skip to content

[Feature] out of tree torch models using torchax #870

@samos123

Description

@samos123

Checklist

Motivation

We have a custom torch model that we wish to run on TPU. the vLLM tpu-inference project uses torchax and similarly torchax may be used for sglang-jax to support torch models.

Why?

  • Support a wider range of models out of the box OSS models without having to port them to jax.
  • Ability to users to run their private torch models on TPU using sglang-jax.

Related resources

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions