Model Svc
The model service can start, stop AI models across multiple runtimes (eg. Docker) and maintains a database of available models on the platform.
The model service can start, stop AI models across multiple runtimes (eg. Docker) and maintains a database of available models on the platform.
The prompt service provides an easy to use interface to prompt LLMs and use AI models. Aims to serve humans and machines alike with its resilient queue based architecture.