Skip to main content

BentoML

Layout​

A minimal BentoML model package has the following layout:

├── bentofile.yaml
├── requirements.txt
├── packages.txt
└── service.py

Environment​

The BentoML deployment environment is based on buildpack-deps:jammy (Ubuntu 22.04) and runs Python 3.10. The unpacked model is placed under /user/model.

Aptitude​

You can install packages with the apt-get Linux package manager. List packages to install one per line in packages.txt.

ffmpeg
libopenal1

Packages listed in packages.txt are installed with:

xargs sudo apt-get --no-install-recommends -y install < packages.txt

Virtualenv​

The environment-prep script only installs pip packages listed in requirements.txt. Make sure bentoml itself is included.

pip install --no-cache-dir -r requirements.txt

To list everything installed in the current environment, run:

pip freeze > requirements.txt

BentoML​

In the runtime container, BentoML is started with the following command. You can verify locally that the same command runs successfully:

bentoml serve /user_model --host 0.0.0.0 --p 8080

For details on writing BentoML-based models, see the BentoML docs.