Skip to main content

Shell

Layout​

A Shell model package has the following basic layout:

├── main.py
├── prepare.sh
└── run.sh
  • main.py: the main application file for the model server
  • prepare.sh: environment setup and dependency installation script
  • run.sh: model server startup script

Environment​

The Shell deployment environment is based on buildpack-deps:jammy (Ubuntu 22.04) and runs the run.sh script you provide. Unpacked model files are placed under the /user_function directory.

prepare.sh​

Use prepare.sh to set up the environment needed to run your model. It installs system and Python packages and performs any other required initial setup.

Example prepare.sh:

#!/bin/bash
set -e

if [ -f packages.txt ]; then
    xargs sudo apt-get update && apt-get --no-install-recommends -y install < packages.txt
fi

if [ -f requirements.txt ]; then
    pip install --no-cache-dir -r requirements.txt
fi

Execution​

In the runtime container, the Shell model is started with the following command. You can verify locally that the same command runs successfully:

/user_function/run.sh

run.sh​

The run.sh script starts the model server.

Example run.sh:

#!/bin/bash
set -e

python main.py

Server Readiness​

The model must expose a /readyz endpoint. This endpoint is used as a health check to verify that the model is ready to serve requests. A server is considered healthy when it returns HTTP 200.

Example readyz endpoint (FastAPI):

from fastapi import FastAPI
import uvicorn

app = FastAPI()

@app.get('/readyz')
async def readiness():
    """
    Return HTTP 200 once the server is ready to handle requests.
    """
    return {"status": "ready"}

if __name__ == '__main__':
    uvicorn.run(app, host='0.0.0.0', port=8080)