Inference¶

Five paths spend model quota: three chat dialects that share one routing decision, and two typed endpoints. All five require a bearer and an accepted transport (transport and identity).

The three inference dialects¶

POST /v1/chat/completions, POST /v1/messages and POST /v1/responses are one workflow. Identity, allowlist, alias resolution, selector semantics, billing ownership and the error contract are identical; only the request and answer shapes differ.

Each requires a model naming an allowed alias, a canonical provider/model route, or a selector. The alias names exactly one route, that route is attempted once, and its answer is the answer to the request; when it refuses, the refusal is what the caller receives and no other route is attempted. An alias that cannot serve is refused before any provider is contacted, with the 503 alias_unserviceable body in the refusal envelope.

Shared validation¶

• A request with no model is 400 missing field `model`, and a bare vendor name is 400 model must be a canonical provider/model route or a supported selector (both captured).
• max_tokens reaches the provider only when the request states one, and a stated zero is refused with max_tokens must be at least one when it is set. An omitted value is omitted on wires that do not require it. On the Anthropic Messages wire, Brama sends the model's stated output limit: on a subscription route, the limit the provider's own listing states in the published discovery inventory, and only when the listing states none (or discovery has not completed or failed) the one in the model table this build carries — the provider's statement wins because a provider changes a limit after a build is cut. On a direct route the table is the source. No numeric substitute is invented: a request no source sizes is refused before dispatch with 400, naming the provider/model, what the listing said and that the table holds no limit, and asking for max_tokens, instead of the provider's max_tokens: Field required. A table that does not parse, or a row missing a field, refuses sizing with 503 naming the row. An explicitly supplied max_tokens does not require this limit lookup.
• Subscription output limits are indexed by provider and model when discovery is published. A request reads one immutable publication, including its discovery outcome; it neither scans the full model inventory nor joins provider discovery. Inventory copying and replacement cleanup happen outside the publication lock. Repeated entries keep the first advertised limit, including an absent limit.
• temperature reaches the provider only when the request states one; an omitted temperature is omitted on the wire too, so the provider applies its own default. A gateway that invents a temperature for every request without one gets Claude's newest models refusing it — `temperature` is deprecated for this model — for callers that never asked. A stated value is refused unless it is finite and no greater than two, with temperature must be finite and between zero and 2.
• An empty conversation is refused with messages must not be empty.

The real subscription qualification is tests/subscriptions/journeys/unlisted_output_limit.sh, with BRAMA naming the binary, AGENT the paying agent and MODEL a provider/model absent from the embedded catalog. Before any provider request, it requires a readable catalog, the provider's array and a successful boolean absence result. Missing tooling, unreadable or malformed metadata cannot count as an unlisted model. The journey then checks the cost-acknowledgement refusal and the actual provider answer, retaining commands, exit statuses and complete output under target/real-tests/subscriptions. An admission failure is not a provider test.

POST /v1/chat/completions¶

Accepts exactly model, messages, max_tokens, temperature, tools, tool_choice, billingTarget (providerId/accountId/subscriptionId) and stream. An unknown field is refused with the deserializer's own sentence (captured):

400 invalid JSON: unknown field `frequency_penalty`, expected one of `model`,
`messages`, `max_tokens`, `temperature`, `tools`, `tool_choice`,
`billingTarget`, `stream` at line 1 column 93

Success (captured):

{"choices":[{"finish_reason":"stop","index":0,
  "message":{"content":"...","role":"assistant"}}],
 "id":"chatcmpl-...","model":"<route>","object":"chat.completion",
 "usage":{"completion_tokens":7,"prompt_tokens":9,"total_tokens":16}}

The model in the answer is the route that served the request, which is the one route the named alias resolves to. usage is the provider's stated meter; an answer whose provider stated no usage has no usage member, never zeros, and a stated count that is not a whole number is refused as a provider failure. The same holds for the buffered /v1/messages and /v1/responses answers below.

With "stream": true the answer is a text/event-stream of chat.completion.chunk frames: a role delta first, then content deltas, a finish_reason frame, a usage frame, closed by data: [DONE]. finish_reason is the provider's stop in this vocabulary (stop, length, tool_calls, content_filter); a reason it has no word for is sent as the provider stated it, and null when the provider stated none. The usage frame is sent only when the provider stated both counts.

POST /v1/messages¶

The Anthropic Messages dialect over the same routing decision. It accepts the Anthropic request shape: model, max_tokens, messages, system, tools and streaming. Anthropic fields the provider-neutral request cannot hold — stop sequences, cache-control hints, non-function tool types — are accepted and have no effect on the attempt; no approximation of them is sent in their place. Success (captured):

{"content":[{"text":"...","type":"text"}],"id":"msg_...",
 "model":"<route>","role":"assistant","stop_reason":"end_turn",
 "stop_sequence":null,"type":"message",
 "usage":{"input_tokens":9,"output_tokens":7}}

Streaming emits message_start, content_block_*, message_delta and message_stop events, with no [DONE] sentinel. message_start carries a meter of nothing counted yet; message_delta carries the provider's stop as end_turn, stop_sequence, max_tokens, tool_use or refusal (a content filter), any other reason as the provider stated it, and its usage only when the provider stated one.

POST /v1/responses¶

The OpenAI Responses dialect, with input in place of messages. Reasoning options and stored-response identifiers are accepted and have no effect on the attempt. Success (captured):

{"created_at":...,"id":"resp_...","model":"<route>","object":"response",
 "output":[{"content":[{"annotations":[],"text":"...","type":"output_text"}],
   "id":"msg_...","role":"assistant","status":"completed","type":"message"}],
 "status":"completed",
 "usage":{"input_tokens":9,"output_tokens":7,"total_tokens":16}}

Streaming emits response.* events closed by response.completed, or by response.incomplete with incomplete_details.reason when the provider stopped short (an output limit, a content filter, or the reason it stated); the terminal response carries usage only when the provider stated one.

Streaming commit rule¶

Everything that can change about a request — which credential redeems it, which account is billed — is settled before the first byte reaches the caller. Once a byte has been written, the generation is committed: a stream that ends without its terminal event was cut after commit, and Brama has already stopped. It does not resume a committed generation anywhere, and the caller sees the truncated stream and decides what to do about it.

Streaming a model reachable only through the shared catalogue is refused as 400 invalid_request before any provider is contacted.

POST /v1/embeddings¶

A typed, alias-only endpoint on the wisent-backend/embeddings route. It accepts exactly model, input, encoding_format (float or base64), dimensions and user; its shape refusals are invalid embedding input and invalid embedding encoding format. Success is the OpenAI embeddings shape: object: "list", data[].embedding, usage.

On a deployment whose registry declares no embeddings alias, the captured answer is 500 {"code":"internal_error","message":"embedding alias missing"}. Declaring the alias is a registry write (PUT /v1/admin/routes); an existing registry document is brought to the current shape by brama routes migrate.

POST /v1/moderations¶

A typed, alias-only endpoint on wisent-backend/moderation. It requires model and input, and answers invalid moderation input on a shape error. Success is the OpenAI moderations shape.

POST /v1/decisions¶

The endpoint that answers instead of generating. It accepts exactly model, state and questions, and exactly two model names: decision-model, on a route this deployment pays for, and best-decision-model, whose route may be best — in which case the caller's own signed identity selects the subscription that pays, exactly as it does for a chat completion. Everything the shape means is in decision.

{
  "model": "decision-model",
  "state": "Help! My payouts have been failing for 3 days.",
  "questions": {
    "is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"},
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {"billing": "Payments, invoicing, refunds", "technical": "Bugs, outages, integrations"}
    }
  }
}

Success is one answer per question, beside the route and engine that answered and the tokens it cost: {"model", "route", "engine", "answers", "usage"}. A noul answers a probability and a confidence; a choice answers one of the labels the question declared, its distribution and a confidence; a score answers the probability-weighted level index, its distribution, a confidence and the legend it was graded against.

The refusals are its own. A malformed question is 400 invalid_request naming the question — question `mood` declares unsupported type `vibe`: a question is a noul, a choice or a score — and nothing is sent to a provider. A model name that is not one of the two is 400 model must be `decision-model` or `best-decision-model`. An alias with no serviceable route is 503 alias_unserviceable carrying details.state and details.route. An answer outside the schema the caller declared is 502 decision_contract_violated, whose message names the route that answered and why, carrying details.alias, details.route (what the alias resolves to), details.answered_by (the route that answered: for best-decision-model, the subscription model it selected), details.reason and details.reply (the reply as the model wrote it): the provider answered, and what it answered cannot be used. The usage latency_ms of an answer is the measured value, unrounded.

A decision alias is refused on the chat endpoints by name, and a canonical route belonging to a decision-only provider is refused there too — route `typesafe/jev-latest` serves typed decisions only; call POST /v1/decisions.

The three media endpoints¶

POST /v1/images/generations takes {"model", "prompt"} with the optional n, size, quality, background, output_format, response_format and user, and answers the provider's image document with model rewritten to the route the caller named. Input images ride in image as data: URLs, with an optional mask beside them. POST /v1/images/edits is the OpenAI edit contract for the same request: a multipart/form-data form whose text fields are those options and whose files are the input images (image, or image[] for several) and an optional mask. A provider that declares an edit endpoint (OpenAI) is sent the edit there; Gemini and Seedream read the images in their generation request. A form with no input image, a file that is not an image, or a body that is not a form is 400 naming what was wrong, and JSON sent to the edit path is pointed at /v1/images/generations. POST /v1/videos takes {"model", "prompt"} with size and seconds, and answers a job — an id and a status — which GET /v1/videos/{id}?model=… reads back from the provider that issued it. POST /v1/audio/speech takes {"model", "input", "voice"} with response_format and speed, and answers the encoded audio itself, with the provider's own Content-Type: it is the one endpoint here whose body is not JSON.

Each accepts the deployment's media alias — image-model, video-model, voice-model — or a canonical route whose provider serves that shape and which the catalogue lists as that kind. A chat model is refused before any credential is redeemed (`openai/gpt-4o-mini` is a text model: name `image-model` or a provider/model route the catalogue lists as image), an unrouted media alias is 503 naming the command that declares it, and a status read with no model is 400 a video status read names the model that started the job: GET /v1/videos/{id}?model=…. Media is paid by the deployment's own capability: no subscription carries an image, video or voice quota, so best cannot stand behind a media alias (media).

Which aliases the presenting bearer may name, and which of them can serve right now, is GET /v1/aliases. Which concrete models the caller can execute is GET /v1/models.