Skip to main content

Model Svc

The model service can start, stop AI models across multiple runtimes (eg. Docker) and maintains a database of available models on the platform.

This page is a high level overview of the Model Svc. For more details, please see the Model Svc API documentation.

Responsibilities​

  • Start and stop models
  • Maintain a database of models and other related information such as the default model

Dependencies​

  • Docker Svc to start containerized AI models (eg. Llama, Stabel Diffusion etc.)

Current Limitations​

  • Stop model endpoint is missing