Model Gateway

The Model Gateway is the only path from an agent turn to a model. It resolves the active provider, adapts the request to the model family, streams the response, and degrades when a provider is unhealthy.

Path

Agent turn
  → activeProvider / activeModel (or a routing rule match)
  → resolveProvider(id): config → credentials → environment → default base URL
  → OpenAI-compatible client (Vercel AI SDK v6)
  → model-family adaptation (reasoning models, Anthropic)
  → streaming call
  → text deltas + tool calls back to the loop

One adapter, many providers

Every remote provider is called through the same OpenAI-compatible adapter. This is what makes providers interchangeable: a provider is a base URL and a key, and its model list is data rather than a code path.

Loop control

The agent loop is the Vercel AI SDK core loop, not a hand-written while:

• prepareStep decides per step which tools exist and whether the turn may continue
• loop breakers (repeated identical calls, no-progress detection) and the step budget (defaults.maxSteps, capped at 100) bound the turn
• timeouts: streamTimeout per call, turnTimeout per turn

Mid-run message injection into a hand-written loop no longer exists; POST /api/runs/:id/message returns 410.

Streaming

Responses stream by default:

• Text appears as it is generated
• TTS can speak sentence-by-sentence
• Tool calls are executed as soon as a step yields them

Health and fallback

A provider that fails twice in a row is put into a 20-second cooldown with exponential backoff on repeat failures. While cooling down, calls skip it instead of waiting for the same timeout. The first success resets the counter.

Counting cost

Token usage is recorded when the provider reports it. Executors driven over ACP cannot be metered (the protocol has no usage update), so cost figures for those runs are explicitly estimates rather than provider counts.

Next Steps

• Model Providers — configure providers
• Configure Local Models — Ollama
• Reliability Evaluation — why pass rate alone is not enough