Skip to content
@bentoml

BentoML

The easiest way to build fast and reliable AI serving systems

Welcome to BentoML 👋 Twitter Follow Slack

github banner

What's cooking? 👩‍🍳

🍱 BentoML: The Unified Serving Framework for AI/ML Systems

BentoML is a Python library for building online serving systems optimized for AI apps and model inference. It supports serving any model format/runtime and custom Python code, offering the key primitives for serving optimizations, task queues, batching, multi-model chains, distributed orchestration, and multi-GPU serving.

🎨 Examples: Learn by doing!

A collection of examples for BentoML, from deploying OpenAI-compatible LLM service, to building voice phone calling agents and RAG applications. Use these examples to learn how to use BentoML and build your own solutions.

🦾 OpenLLM: Self-hosting Large Language Models Made Easy

Run any open-source LLMs (Llama, Mistral, Qwen, Phi and more) or custom fine-tuned models as OpenAI-compatible APIs with a single command. It features a built-in chat UI, state-of-the-art inference performance, and a simplified workflow for production-grade cloud deployment.

☁️ BentoCloud: Unified Inference Platform for any model, on any cloud

BentoCloud is the easist way to build and deploy with BentoML, in our cloud or yours. It brings fast and scalable inference infrastructure into any cloud, allowing AI teams to move 10x faster in building AI applications with ML/AI models, while reducing compute cost - by maxmizing compute utilization, fast GPU autoscaling, minimimal coldstarts and full observability. Sign up today!.

Get in touch 💬

👉 Join our Slack community!

👀 Follow us on X @bentomlai and LinkedIn

📖 Read our blog

Pinned Loading

  1. BentoML BentoML Public

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Python 7.5k 825

  2. OpenLLM OpenLLM Public

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    Python 11k 699

  3. BentoDiffusion BentoDiffusion Public

    BentoDiffusion: A collection of diffusion models served with BentoML

    Python 358 29

  4. comfy-pack comfy-pack Public

    A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.

    Python 125 13

  5. BentoVLLM BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    Python 93 15

  6. BentoVoiceAgent BentoVoiceAgent Public

    Build Phone Calling Voice Agent fully powered by open source models.

    Python 29 9

Repositories

Showing 10 of 104 repositories
  • BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    bentoml/BentoVLLM’s past year of commit activity
    Python 93 Apache-2.0 15 0 2 Updated Mar 19, 2025
  • BentoVoiceAgent Public

    Build Phone Calling Voice Agent fully powered by open source models.

    bentoml/BentoVoiceAgent’s past year of commit activity
    Python 29 9 1 0 Updated Mar 18, 2025
  • BentoOCR Public

    Turn any OCR models into online inference API endpoint 🚀 🌖

    bentoml/BentoOCR’s past year of commit activity
    Python 54 4 1 0 Updated Mar 18, 2025
  • BentoMLflow Public
    bentoml/BentoMLflow’s past year of commit activity
    Python 1 1 0 0 Updated Mar 18, 2025
  • OpenLLM Public

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    bentoml/OpenLLM’s past year of commit activity
    Python 10,990 Apache-2.0 699 0 0 Updated Mar 17, 2025
  • BentoML Public

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    bentoml/BentoML’s past year of commit activity
    Python 7,487 Apache-2.0 825 157 3 Updated Mar 18, 2025
  • BentoDiffusion Public

    BentoDiffusion: A collection of diffusion models served with BentoML

    bentoml/BentoDiffusion’s past year of commit activity
    Python 358 Apache-2.0 29 9 0 Updated Mar 17, 2025
  • BentoXTTSStreaming Public

    xtts with streaming endpoint

    bentoml/BentoXTTSStreaming’s past year of commit activity
    Python 5 1 1 1 Updated Mar 17, 2025
  • bentoml/openllm-models’s past year of commit activity
    Python 35 12 0 1 Updated Mar 17, 2025
  • yatai-deployment Public

    🚀 Launching Bento in a Kubernetes cluster

    bentoml/yatai-deployment’s past year of commit activity
    Go 17 14 7 3 Updated Mar 17, 2025