Add Prompt
POST/prompt-svc/prompt
Adds a new prompt to the prompt queue and either waits for the response (if sync
is set to true), or returns immediately.
Request
- application/json
Body
required
Add Prompt Request
Id is the unique ID of the prompt.
MaxRetries specified how many times the system should retry a prompt when it keeps erroring.
ModelId is just the Singulatron internal ID of the model.
Prompt is the message itself eg. "What's a banana?
Sync drives whether prompt add request should wait and hang until the prompt is done executing. By default the prompt just gets put on a queue and the client will just subscribe to a Thread Stream. For quick and dirty scripting however it's often times easier to do things syncronously. In those cases set Sync to true.
Template of the prompt. Optional. If not present it's derived from ModelId.
ThreadId is the ID of the thread a prompt belongs to.
Clients subscribe to Thread Streams to see the answer to a prompt,
or set prompt.sync
to true for a blocking answer.
Responses
- 200
- 400
- 401
- 500
OK
- application/json
- Schema
- Example (from schema)
Schema
prompt
object
CreatedAt is the time of the prompt creation.
Error that arose during prompt execution, if any.
Id is the unique ID of the prompt.
LastRun is the time of the last prompt run.
MaxRetries specified how many times the system should retry a prompt when it keeps erroring.
ModelId is just the Singulatron internal ID of the model.
Prompt is the message itself eg. "What's a banana?
RunCount is the number of times the prompt was retried due to errors
Possible values: [scheduled
, running
, completed
, errored
, abandoned
, canceled
]
Sync drives whether prompt add request should wait and hang until the prompt is done executing. By default the prompt just gets put on a queue and the client will just subscribe to a Thread Stream. For quick and dirty scripting however it's often times easier to do things syncronously. In those cases set Sync to true.
Template of the prompt. Optional. If not present it's derived from ModelId.
ThreadId is the ID of the thread a prompt belongs to.
Clients subscribe to Thread Streams to see the answer to a prompt,
or set prompt.sync
to true for a blocking answer.
UpdatedAt is the last time the prompt was updated.
UserId contains the ID of the user who submitted the prompt.
{
"answer": "string",
"prompt": {
"createdAt": "string",
"error": "string",
"id": "string",
"lastRun": "string",
"maxRetries": 10,
"modelId": "huggingface/TheBloke/mistral-7b-instruct-v0.2.Q3_K_S.gguf",
"prompt": "What's a banana?",
"runCount": 0,
"status": "scheduled",
"sync": true,
"template": "[INST]{prompt}[/INST]",
"threadId": "string",
"updatedAt": "string",
"userId": "string"
}
}
Invalid JSON
- application/json
- Schema
- Example (from schema)
Schema
{
"error": "string"
}
Unauthorized
- application/json
- Schema
- Example (from schema)
Schema
{
"error": "string"
}
Internal Server Error
- application/json
- Schema
- Example (from schema)
Schema
{
"error": "string"
}