Skip to content

A provider's error reaches the adapter whatever the transport does with its body, so a context overflow over a real connection is compacted, and a person reads what the provider said, trimmed and without the key.

Decision 791 · accepted · 2026-10-05 · issue #436

Both HTTP adapters stream with Req's into: fn ..., and Req hands that function the body of every response, whatever its status: its Finch step (finch_stream_into_fun) puts the status and the headers on the response and passes each {:data, chunk} on. So on a 4xx the adapter's %Req.Response{status: 400, body: body} read "", and the provider's error had gone to the event-stream parser, which found no event in it. A context overflow read as a bare 400, so 659's compact-and-retry never fired and the turn ended with "the provider answered 400"; the OpenAI adapter's max_completion_tokens retry (658) and 780's wording for refused thinking, which read the same body, never applied; and every 4xx a person saw said only its status.

  • The status, checked in into. Each adapter's handle_chunk feeds the parser only for a 200; any other status's chunks are kept on the response's body (Provider.collect_error/2, up to 64 KiB, more than any provider's error and a bound on a proxy's page), where the error clauses of post read it. Req is not asked to decode it (decode_body: false) and the adapter decodes it itself, so a body that is not the JSON its content type says, or one cut at the bound, is still read as text rather than turned into a decoding exception the adapter would hand on as the failure. Req has no option that hands into only a 2xx's body, and the status read after the request is too late: the body has gone by then.
  • What a person reads. The message comes out of the JSON as before (error.message for both, and now a top-level message too, which is vLLM's), else the text, at most 400 characters; it is trimmed, and the key the request was sent with is masked wherever it appears, as troupe config shows a key (Config.mask/1, in Provider.error_text/2). A provider, or a gateway in front of one, may say a key it refused back in its message, and the sentence goes into llm_error.reason, into the log and into the conversation as the note (693), and so to the provider again. A key shorter than eight characters is a placeholder a local server takes (vLLM's EMPTY) and is left alone, since masking it would only garble the message. Provider.classify/1 and describe_error/1 are as they were: they have the provider's words to go on now.

Governs: apps/troupe_core/lib/troupe/llm/providers/anthropic.ex, apps/troupe_core/lib/troupe/llm/providers/openai.ex, apps/troupe_core/lib/troupe/llm/provider.ex