bentoml

BentoML: Unified Model Serving

Overview
BentoML is an open-source framework for building reliable machine learning serving endpoints. It solves the "It works on my notebook" problem by packaging the model, dependencies, and API logic into a standard format called a Bento.

Workflow

1. Save Model

import bentoml
bentoml.sklearn.save_model("my_clf", clf_obj)

2. Define Service (`service.py`)

import bentoml
from bentoml.io import NumpyNdarray

runner = bentoml.sklearn.get("my_clf:latest").to_runner()
svc = bentoml.Service("classifier", runners=[runner])

@svc.api(input=NumpyNdarray(), output=NumpyNdarray())
def predict(input_series):
    return runner.predict.run(input_series)

3. Build & Serve

bentoml build
bentoml serve service.py:svc

Why BentoML?
- Containerization: Automatically generates the `Dockerfile` for you.
- Adaptive Batching: Automatically groups API requests to maximize throughput.
- Yatai: A Kubernetes-native dashboard to manage deployments.
- Integration: Works with standard tools (MLflow) and deploys anywhere (AWS Lambda, SageMaker, Heroku, K8s).