File failed LLM generations from Langfuse as Linear issues
A Langfuse + Linear agent flow
An LLM generation that errors or times out leaves a trace in Langfuse and a quiet degradation for the user. The trace is searchable, but searching it is a thing someone has to decide to do. This recipe has an agent watch the traces and turn the failures into work. It calls listObservations filtered to generations with an error level, reads each failed call, and files a Linear issue with create_issue: the prompt or trace name in the title, the error and model in the description, and a link back to the Langfuse trace. Langfuse returns observations when queried rather than pushing them, so the agent polls on a cadence and tracks the last observation id it filed so a recurring failure becomes one issue, not a flood.
The flow
listObservationsFind observations such as generations, spans, and events with filtering.
create_issueCreates a new issue or sub-issue with title, description, and workflow metadata.
Step by step
- Query failed observations
The agent calls listObservations with a filter for the generation type and an error or warning level, scoped to recent traces. The response is the matching observations with their model, latency, status, and trace ids.
- Group and dedupe
Many errors share a root cause, like one prompt template failing across requests. The agent keys on the trace name or error signature and the last observation id it filed, so a repeated failure updates or skips rather than spawning a new issue every poll.
- Create the Linear issue
For each new failure group the agent calls create_issue: title from the prompt or trace name, description with the error, model, and latency, a Langfuse trace link, and a label like llm-error so the team can triage these together.
Tell your agent
Every fifteen minutes, call Langfuse listObservations for generations at error level in the last hour, group them by trace name, and for each new group create a Linear issue with the error and model in the description and a link to the Langfuse trace.
Setup
This flow needs both servers connected to your agent. Follow each install guide:
Worth knowing
- listObservations returns observations matching your filter, not a pre-aggregated error report; the agent does the grouping, so decide up front whether to key on trace name, model, or error message.
- Observations span generations, spans, and events. Filter to the generation type when you only care about model calls, or the query also returns retrieval spans and tool events you may not want to file.
Questions
- Can it file on latency, not just errors?
- Yes. listObservations exposes latency, so the agent can flag generations above a threshold and open a Linear issue for slow calls the same way it does for failed ones.
- Will an intermittent error reopen issues forever?
- No, as long as you dedupe on the trace name or error signature. The agent files once per distinct failure and can comment on the existing issue when the count climbs rather than creating duplicates.