Repositories list

• llm-inference-handbook

  Public

  bentoml/llm-inference-handbook’s past year of commit activity
  Everything you need to know about LLM inference

  inference-optimizationllmllm-inference+ 2

  inference-optimizationllmllm-inferenceinference-handbookinference-infrastructure

  TypeScript

  •

  Apache License 2.0

  •23•271•3•1•Updated Mar 20, 2026Mar 20, 2026

• yatai-schemas

  Public

  bentoml/yatai-schemas’s past year of commit activity
  Go

  •2•1•1•2•Updated Mar 20, 2026Mar 20, 2026

• helm-charts

  Public

  bentoml/helm-charts’s past year of commit activity
  0•3•0•0•Updated Mar 19, 2026Mar 19, 2026

• yatai-image-builder

  Public

  bentoml/yatai-image-builder’s past year of commit activity
  🐳 Build OCI images for Bentos in k8s

  dockerkubernetesdocker-image+ 3

  dockerkubernetesdocker-imagecontainerk8scontainer-image

  Go

  •12•20•6•13•Updated Mar 19, 2026Mar 19, 2026

• OpenLLM

  Public

  bentoml/OpenLLM’s past year of commit activity
  Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

  llamamistralfine-tuning+ 15

  llamamistralfine-tuningmlopsbentomlvicunallmmodel-inferencellmopsllm-serving+ 8

  Python

  •

  Apache License 2.0

  •802•12k•4•7•Updated Mar 16, 2026Mar 16, 2026

• BentoML

  Public

  bentoml/BentoML’s past year of commit activity
  The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

  pythonmachine-learningdeep-learning+ 12

  pythonmachine-learningdeep-learningmodel-servingmultimodalmlopsml-engineeringai-inferencellmgenerative-ai+ 5

  Python

  •

  Apache License 2.0

  •933•8.5k•138•9•Updated Mar 16, 2026Mar 16, 2026

• ai-gateway

  Public

  bentoml/ai-gateway’s past year of commit activity
  Manages Unified Access to Generative AI Services built on Envoy Gateway

  Go

  •

  Apache License 2.0

  •195•0•0•2•Updated Mar 13, 2026Mar 13, 2026

• BentoSGLang

  Public

  bentoml/BentoSGLang’s past year of commit activity
  Python

  •1•2•0•1•Updated Mar 12, 2026Mar 12, 2026

• yatai-common

  Public

  bentoml/yatai-common’s past year of commit activity
  Go

  •6•1•0•2•Updated Mar 7, 2026Mar 7, 2026

• sglang

  Public

  bentoml/sglang’s past year of commit activity
  SGLang is a fast serving framework for large language models and vision language models.

  Python

  •

  Apache License 2.0

  •4.9k•0•0•1•Updated Mar 5, 2026Mar 5, 2026

• openai_emulator

  Public
  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Mar 5, 2026Mar 5, 2026

• .github

  Public
  ✨🍱🦄️

  5•1•0•0•Updated Mar 5, 2026Mar 5, 2026

• BentoVLLM

  Public
  Self-host LLMs with vLLM and BentoML

  Python

  •

  Apache License 2.0

  •22•170•3•5•Updated Mar 3, 2026Mar 3, 2026

• bentocloud-homepage-news

  Public
  2•2•0•0•Updated Feb 10, 2026Feb 10, 2026

• BentoSpacy

  Public
  Python

  •0•0•0•2•Updated Jan 26, 2026Jan 26, 2026

• BentoTRTLLM

  Public
  Python

  •1•10•2•2•Updated Jan 22, 2026Jan 22, 2026

• mise-nebius

  Public
  Shell

  •

  MIT License

  •0•0•0•0•Updated Jan 22, 2026Jan 22, 2026

• BentoLMDeploy

  Public
  Self-host LLMs with LMDeploy and BentoML

  Python

  •2•22•1•1•Updated Dec 26, 2025Dec 26, 2025

• BentoFunctionCalling

  Public
  Python

  •1•9•0•2•Updated Dec 26, 2025Dec 26, 2025

• lago

  Public
  Open Source Metering and Usage Based Billing API ⭐️ Consumption tracking, Subscription management, Pricing iterations, Payment orchestration & Revenue analytics

  Go

  •

  GNU Affero General Public License v3.0

  •555•0•0•1•Updated Nov 20, 2025Nov 20, 2025

• comfy-pack

  Public
  A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.

  ai-artcomfyuigenai+ 4

  ai-artcomfyuigenaicomfyui-managercomfyui-workflowcomfyui-nodescomfyui-api

  Python

  •

  Apache License 2.0

  •29•211•11•2•Updated Nov 10, 2025Nov 10, 2025

• BentoOCR

  Public
  Turn any OCR models into online inference API endpoint 🚀 🌖

  ocrmodel-deploymentmodel-serving+ 2

  ocrmodel-deploymentmodel-servingocr-pythonai-applications

  Python

  •4•59•1•0•Updated Oct 29, 2025Oct 29, 2025

• pipelines-workflows

  Public
  Gruntwork Pipelines for BentoML

  25•0•0•0•Updated Oct 23, 2025Oct 23, 2025

• lago-go-client

  Public
  Lago Go Client

  Go

  •

  MIT License

  •24•0•0•0•Updated Oct 15, 2025Oct 15, 2025

• BentoYolo

  Public
  BentoML service of YOLO v8

  Python

  •

  Apache License 2.0

  •4•12•1•1•Updated Sep 17, 2025Sep 17, 2025

• llm-optimizer

  Public
  Benchmark and optimize LLM inference across frameworks with ease

  Python

  •

  Apache License 2.0

  •17•173•10•0•Updated Sep 12, 2025Sep 12, 2025

• kantoku

  Public
  A Process & Socket Manager built with zmq

  Python

  •

  Other

  •261•1•0•0•Updated Aug 28, 2025Aug 28, 2025

• byoc

  Public
  Shell

  •1•0•0•2•Updated Aug 28, 2025Aug 28, 2025

• BentoProphet

  Public
  BentoML with Facebook Prophet

  Python

  •0•1•0•0•Updated Aug 27, 2025Aug 27, 2025

• XGBoostDemo

  Public
  Python

  •0•2•0•0•Updated Aug 19, 2025Aug 19, 2025