v1.1.0 release - Contributors, Sponsors and Enquiries are most welcome 😌

Gateway

Route across providers with failover, circuit breaking, and real token streaming.

agentsea_gateway picks a provider per request by strategy, trips a :fuse circuit breaker on repeated failures, and fails over to the next candidate. API on HexDocs.

Configure & route

elixir
alias AgentSea.Gateway
alias AgentSea.Gateway.{Config, Provider}

config = %Config{
  providers: [
    %Provider{name: :opus,  module: AgentSea.Providers.Anthropic, model: "claude-opus-4-8"},
    %Provider{name: :haiku, module: AgentSea.Providers.Anthropic, model: "claude-haiku-4-5"}
  ],
  strategy: AgentSea.Gateway.Router.Failover
}

{:ok, gw} = Gateway.start_link(config)

messages = [%{role: :user, content: "Summarize OTP in a sentence."}]
{:ok, response, served_by} = Gateway.completion(gw, messages)

response.content   # the answer
served_by          # which provider handled it

Strategies

  • AgentSea.Gateway.Router.Failover — try in order, fall through on error.
  • AgentSea.Gateway.Router.RoundRobin — spread load evenly.
  • AgentSea.Gateway.Router.CostOptimized — cheapest viable model first.
  • AgentSea.Gateway.Router.LatencyOptimized — fastest by observed latency.

Each provider has a :fuse circuit breaker: after repeated failures the breaker blows and that provider is skipped until it resets.

Streaming

stream/3 returns a lazy stream of normalized events from the first streamable provider:

elixir
{:ok, stream, _provider} = Gateway.stream(gw, messages)

Enum.each(stream, fn
  {:content, delta} -> IO.write(delta)
  {:thinking, _t}   -> :ok
  :done             -> IO.puts("")
  _other            -> :ok
end)

OpenAI-compatible endpoint

agentsea_web exposes POST /v1/chat/completions backed by the gateway — point any OpenAI client (or curl) at it. With stream: true it forwards each provider event as a real SSE chunk.

bash
curl http://localhost:4002/v1/chat/completions \
  -H "content-type: application/json" \
  -d '{
    "model": "agentsea",
    "messages": [{"role": "user", "content": "hello"}],
    "stream": true
  }'

Configure which gateway backs the endpoint via config :agentsea_web, :gateway, MyApp.Gateway. The same app also serves a Phoenix LiveView dashboard that renders agent, tool, crew, gateway, and guardrail telemetry live.

Next

Add retrieval with Embeddings & RAG, or wire tools and guardrails on the Agents page.