Research date · May 6, 2026
Generated with AI

Error Handling in AI Agent Frameworks

How LangChain, LangGraph, CrewAI, Google ADK, and Microsoft Semantic Kernel deal with the four ways an agent run breaks in production: tools throwing, LLMs returning rubbish, models hallucinating, and contexts blowing past the window. Plus what each framework does on its own to retry.

Four failure modes What each looks like LangChain LangGraph CrewAI Google ADK Semantic Kernel Side-by-side Cheat sheet

Four failure modes the frameworks design around

In a production deployment, agent runs fail for many reasons — provider rate-limits, content-moderation refusals, expired auth, infra outages, cyclic handoffs, cost ceilings, and more. This page focuses on the four failure modes the frameworks themselves ship dedicated machinery for, and where each framework draws the line between "we handle this for you" and "you handle this yourself." For everything else (provider outages, infra, auth, budgets, compliance), every framework defers to your operational tooling.

LLM picks tool + args, OR returns a final answer may hallucinate, malform, run long Tool runs the picked function (API, code, retrieval) may throw, time out, 500 tool_call result 2 3 4 1 Retry harness — max_attempts · backoff · timeouts User / next agent / persisted output downstream consumer of the agent's answer final answer
1Tool calls fail
The LLM's chosen tool throws — network 500, schema mismatch, missing API key, rate-limited downstream. The framework's job: catch it, surface it back to the LLM as an error message it can react to, and decide whether to retry.
2LLM sends incorrect response
Malformed JSON, wrong tool name, missing required arg, prose where a function call was expected. The framework's job: parse-with-recovery, validate against the schema, and either re-prompt with the validation error or hand it to a fixer.
3Hallucination
The LLM confidently invents a tool that doesn't exist, an argument that doesn't fit, a fact that isn't true. Most frameworks have no built-in detection — this is on you (validators, guardrails, factual tools).
4Context overflow
The accumulated messages grow past the model's context window. The framework's job: trim, summarize, or compact. Some have first-class hooks; some leave it entirely to the user.

What each failure looks like in practice

A plain-English walkthrough of how each of the four failure modes shows up in a real production run, before the framework-by-framework comparisons that follow.

1Tool calls fail

The model picks a tool, hands it some arguments, and the tool throws — a network 500 from an external API, a 401 because the downstream service rotated its credentials, a Pydantic schema mismatch because the model hallucinated an extra argument, a rate-limit error from a third-party data provider. To the agent loop this looks like a Python exception bubbling up from inside tool.invoke().

The framework has three choices. Crash the run and propagate the exception to the caller. Catch it, format the error as a synthetic tool-result message, feed it back to the model, and let the model decide what to do (often: try a different tool, or apologise to the user). Or retry the tool call automatically with backoff in the hope it was transient. Most frameworks default to "catch and feed back" — so the model gets a chance to react — and offer a configurable retry policy on top of that.

2LLM sends incorrect response

The model is supposed to produce a structured output: a function call, a JSON object matching a schema, or text in a specific format. It produces something that doesn't parse. Examples: malformed JSON missing a closing brace, a tool name that isn't in the registered tool list, missing a required argument, returning prose ("I think we should call the search tool with…") where a function-call structure was expected.

The framework's recovery options are limited but well-defined. A parser-with-recovery looks at the malformed output and tries to extract intent (e.g. "the model wrote action: search in YAML instead of a JSON tool call — assume the user meant to call search"). A schema validator catches argument-shape violations and re-prompts the model with the validation error attached. A "fixer" agent can be invoked as a sub-step to clean up the output. Whichever path is taken, the goal is to avoid burning the cycle and to get the model back on schema with minimal extra tokens.

3Hallucination

The model confidently produces something that looks plausible but isn't. It calls a tool that doesn't exist (weatherLookup when only search and get_weather are registered). It passes an argument value that's invalid (a customer ID format that has never existed). It states a fact that's wrong but surface-coherent. Hallucination is fundamentally different from the previous two modes because nothing throws — the call is well-formed, the output is grammatical, the schema validates. The error is semantic, not syntactic.

That makes it the hardest of the four to detect inside the framework. Most frameworks have no built-in hallucination detection — they trust the model's tool selection and argument shape. Defence falls outside the loop: factual tools (RAG over an authoritative source), output guardrails (a classifier or LLM-judge that flags suspicious answers), strict tool-name enums in the schema so the model can't invent tools, and post-hoc human review for high-stakes decisions.

4Context overflow

Every model has a context-window limit — the total number of tokens it can read plus generate in a single call. A long-running agent run accumulates messages: every user turn, every model response, every tool call, every tool result. After enough iterations, the prompt grows past the limit and the next call fails with an overflow error from the provider.

Recovery means shrinking the message list before the next call. Three strategies show up across the catalog: trim (drop the oldest messages, keep the system prompt and recent turns), summarise (replace older messages with an LLM-generated summary), and compact (a more sophisticated summary that preserves named entities and tool-call structure). Some frameworks ship a first-class hook for this; others leave it entirely to the developer to wire up before each call. Whichever is used, the trade-off is the same: shrinking saves tokens but loses information, and the model may forget context that was important three turns back.

How each framework handles the four modes

For each card: a diagram of the framework's recovery surfaces, the four-mode coverage, and a code-grounded explanation. Auto-retry behaviour is called out in each.

Lc

LangChain · classic handle_* flags + v1 middleware

Classic AgentExecutor exposes per-error flags; the new v1 package has dedicated ToolRetryMiddleware, ModelRetryMiddleware, SummarizationMiddleware, etc.
AgentExecutor — outer guard: max_iterations=15 · max_execution_time LLM Chain model.invoke(prompt) may return rubbish Output Parser handle_parsing_errors: bool · str · Callable + OutputFixingParser / RetryOutputParser Tool.invoke() Tool.handle_tool_error: bool · str · Callable → feeds error back to LLM langchain v1 (langchain.agents.middleware) — modular middleware stack ToolRetryMiddleware retries failed tool calls · backoff ModelRetryMiddleware retries failed model calls ModelFallbackMiddleware switch to backup model on fail SummarizationMiddleware trims chat history ContextEditing ClearToolUsesEdit · …
"Classic exposes a flag for every kind of failure. v1 promotes them to first-class middleware classes."

How it works

Classic AgentExecutor. Three flags do most of the work. handle_parsing_errors: bool | str | Callable (line 1042 of agent.py) controls what happens when the output parser blows up — set to True and the framework re-prompts the LLM with the parse error attached. Each tool has Tool.handle_tool_error with the same three-shape signature; on a tool exception the value (or the result of the callable) is fed back to the LLM as the tool's "result" string. Outer guards max_iterations (default 15) and max_execution_time stop runaway loops.

Retry-on-bad-output parsers. OutputFixingParser wraps a base parser and, on failure, calls a "fixer" LLM with the original output and the parser error. RetryOutputParser goes further, also passing the original prompt so the LLM can correct itself in context.

v1 middleware stack. The new langchain.agents.middleware package promotes each concern to its own AgentMiddleware subclass — ToolRetryMiddleware (retries with exponential backoff, default 2 retries, configurable should_retry), ModelRetryMiddleware, ModelFallbackMiddleware, SummarizationMiddleware, ContextEditingMiddleware (with edit policies like ClearToolUsesEdit).

Mode coverage

1 · Tool fails

Native. handle_tool_error on each Tool, plus ToolRetryMiddleware with backoff in v1.

2 · LLM incorrect

Native. handle_parsing_errors, OutputFixingParser, RetryOutputParser, ModelRetryMiddleware.

3 · Hallucination

No built-in detector. Schema validation in tool args is your best lever.

4 · Large context

trim_messages (in langchain_core.messages.utils), ConversationSummaryBufferMemory, v1 SummarizationMiddleware.

Auto-retry defaults

None at the executor level — you opt in by setting flags or adding middlewares. v1 ToolRetryMiddleware defaults: 2 retries · exponential backoff. OutputFixingParser retries once with the fixer LLM.

Code anchor

libs/langchain/langchain_classic/agents/agent.py · handle_parsing_errors (line 1042), max_iterations (line 1023)
libs/langchain/langchain_classic/output_parsers/{fix,retry}.py
libs/langchain_v1/langchain/agents/middleware/{tool_retry,model_retry,model_fallback,summarization,context_editing}.py

Lg

LangGraph · per-node RetryPolicy + ToolNode error handling

Every node can carry its own retry policy. ToolNode has a typed handle_tool_errors. Trim and validate via dedicated nodes.
StateGraph — outer guard: recursion_limit pre_model_hook trim_messages summarize · drop · … agent (LLM) RetryPolicy(max=3, backoff=2.0, retry_on=…) ValidationNode re-validates tool_calls against schemas ToolNode handle_tool_errors: bool · str · tuple[Exc] · Callable RetryPolicy is a NamedTuple — applied to ANY node RetryPolicy(initial_interval, backoff_factor=2.0, max_attempts=3, retry_on=…) error → ToolMessage(status="error") → re-enter agent
"Recovery is just another node — or a NamedTuple you bolt onto the node you already have."

How it works

RetryPolicy. A NamedTuple at langgraph/types.py:406 with max_attempts: int = 3, backoff_factor: float = 2.0, initial_interval, and retry_on (an exception type, a tuple of types, or a predicate). Pregel's runner accepts a Sequence[RetryPolicy] so you can stack them per-node — first policy that matches the exception wins.

ToolNode error handling. The prebuilt ToolNode takes handle_tool_errors: bool | str | tuple[type[Exception], ...] | Callable (default: a built-in _default_handle_tool_errors that turns the exception into a friendly string). On error, the tool produces a ToolMessage(status="error") that the next agent step sees — the LLM gets a chance to correct itself.

Validation as a node. ValidationNode (tool_validator.py:47) re-validates the LLM's tool calls against the schemas; you can route validation failures back to the agent node for a re-prompt.

Context editing. create_react_agent exposes pre_model_hook; the canonical move is to call trim_messages from langchain_core.messages there.

Mode coverage

1 · Tool fails

Native. Typed handle_tool_errors on ToolNode; RetryPolicy on the node.

2 · LLM incorrect

Native. ValidationNode for schema checks; response_format for structured output.

3 · Hallucination

No detector. Schema validation catches the worst (made-up tools / wrong args).

4 · Large context

pre_model_hook + trim_messages. Patterns documented; no built-in summarizer.

Auto-retry defaults

No retries by default — you opt in by attaching a RetryPolicy. When attached: 3 attempts · 2.0× exponential backoff.

Code anchor

libs/langgraph/langgraph/types.py · RetryPolicy (line 406)
libs/prebuilt/langgraph/prebuilt/tool_node.py · ToolNode, handle_tool_errors
libs/prebuilt/langgraph/prebuilt/tool_validator.py · ValidationNode
libs/prebuilt/langgraph/prebuilt/chat_agent_executor.py · pre_model_hook

Cr

CrewAI · max_retry_limit, guardrails, respect_context_window

Built-in retry counter on the agent loop, callable guardrails per task, and a token-aware context-window handler that auto-summarizes.
CrewAgentExecutor — outer guards: max_iter · max_rpm · max_retry_limit handle_context_length respect_context_window? → summarize_messages LLM parsed by agents/parser.py · parse() Tool tool error → AgentFinish retry counted vs max_retry_limit Task.guardrail / .guardrails callable validators on output fail → re-execute task Agent / Task defaults that act as silent retry budgets: Agent.max_iter (default 25 in core) · Agent.max_retry_limit (default 2) · Agent.max_rpm · Task.guardrail_max_retries
"Set a retry budget on the agent. Set a guardrail on the task. Let the executor count strikes."

How it works

Tool failures. When a tool throws inside CrewAgentExecutor's loop, the executor formats the exception, appends it as the tool's "result" so the LLM can react, and increments an internal counter against max_retry_limit. After self._times_executed > self.max_retry_limit the executor raises (crew_agent_executor.py:674).

LLM bad output. The ReAct-style parse() function in agents/parser.py handles malformed responses — it raises OutputParserError on bad shape, the executor catches it and surfaces a corrective message back to the LLM. max_iter caps the total iterations; if the agent keeps failing to parse it eventually raise RuntimeError(...) at line 435 / 1247 of crew_agent_executor.py.

Guardrails. A Task takes a single guardrail or a list of guardrails — each is a callable that runs on the task's output before propagation. Failure causes the executor to re-run the task. This is CrewAI's closest thing to a hallucination-mitigation hook: you write the validator, the framework calls it.

Context window. respect_context_window: bool on Agent / CrewAgentExecutor. When the LLM raises a context-length error, handle_context_length (utilities/agent_utils.py:709) dispatches to summarize_messages; with the flag off, the run aborts.

Mode coverage

1 · Tool fails

Native. Counted against max_retry_limit; error string fed back to LLM.

2 · LLM incorrect

Native. parse() in agents/parser.py + corrective re-prompt; capped by max_iter.

3 · Hallucination

Task.guardrail(s) are first-class callable validators.

4 · Large context

Native. respect_context_window → auto-summarize via handle_context_length.

Auto-retry defaults

max_iter default 25 (per task). max_retry_limit default 2 (tool / parse retries on a single agent invocation). max_rpm off by default — when set it throttles. Guardrail re-runs are bounded by guardrail_max_retries.

Code anchor

lib/crewai/.../agents/crew_agent_executor.py · max_retry_limit, handle_context_length
lib/crewai/.../utilities/agent_utils.py · handle_context_length (709), summarize_messages (921)
lib/crewai/.../task.py · guardrail, guardrails
lib/crewai/.../agents/parser.py · parse() in agents/parser.py

Ad

Google ADK · on_*_error_callback hooks + token-threshold compaction

Both the model call and every tool call have dedicated error-callback chains; large-context handling is its own request processor.
BaseLlmFlow — every transition observable by callbacks & plugins CompactionRequestProcessor flows/llm_flows/compaction.py token-threshold based LLM call before_model_callback after_model_callback on_model_error_callback Tool call before_tool_callback after_tool_callback on_tool_error_callback PluginManager runs error callback FIRST if it returns a response, that's used; else agent's Live API · session_resumption_handle on disconnect: increment attempt counter, reconnect with saved handle, replay deltas base_llm_flow.py · run_live() loop Response processors _nl_planning · _code_execution _output_schema_processor (validates) parse function_calls before dispatch
"Every error path has a callback chain — agent-level + plugin-level. Pick where you want to recover."

How it works

Tool failures. LlmAgent exposes on_tool_error_callback: OnToolErrorCallback | list[…] (line 450 of llm_agent.py). When a tool call raises, flows/llm_flows/functions.py calls _run_on_tool_error_callbacks: it first runs the plugin manager's error callback; if that returns a function_response, that becomes the tool result; otherwise it cycles through the agent's callbacks. If none handles it, the original exception propagates.

LLM call failures. Symmetrical: on_model_error_callback (line 405) is invoked from base_llm_flow.py:347 in _run_on_model_error_callbacks. Plugin first, then agent. Same "return a response to suppress, else re-raise" semantics. Used in the live (streaming) loop to recover transient failures without bringing down the run.

Live-API session resumption. The run_live loop in base_llm_flow.py:475 tracks attempt and live_session_resumption_handle; on disconnect it reconnects with the saved handle so the model can resume from the same conversation point.

Large context. CompactionRequestProcessor (flows/llm_flows/compaction.py:32) sits in the request-processor chain; if the configured token-threshold is exceeded it calls _run_compaction_for_token_threshold_config to compact the events before the LLM call. Hallucination-wise: the _output_schema_processor validates structured outputs so a made-up shape gets caught at the response stage.

Mode coverage

1 · Tool fails

Native. on_tool_error_callback chain (plugin then agent).

2 · LLM incorrect

Native. on_model_error_callback chain + _output_schema_processor.

3 · Hallucination

No detector. Output-schema validation traps malformed structured responses; the rest is on you.

4 · Large context

Native. Token-threshold CompactionRequestProcessor in the per-turn pipeline.

Auto-retry defaults

No general retry counter. Live-API: automatic reconnect with session-resumption handle; attempt counter increments per disconnect (no hard cap in the loop — relies on the underlying client's connect timeout).

Code anchor

src/google/adk/agents/llm_agent.py · on_model_error_callback (405), on_tool_error_callback (450)
src/google/adk/flows/llm_flows/base_llm_flow.py · _run_on_model_error_callbacks, run_live
src/google/adk/flows/llm_flows/functions.py · _run_on_tool_error_callbacks
src/google/adk/flows/llm_flows/compaction.py · CompactionRequestProcessor

Sk

Microsoft Semantic Kernel · filters + reducers + pluggable retry

Filters wrap every function call. Chat history reducers (truncate / summarize) handle context. Retry is a pluggable mechanism; the default is "no retry."
Kernel — filters + reducers + reliability extension ChatHistoryReducer (abstract) ChatHistoryTruncationReducer ChatHistorySummarizationReducer applied before the model call Kernel FunctionChoiceBehavior .Auto(...) loop RetryMechanismBase (pluggable) AutoFunctionInvocation Filter try { await next(context) } except: set context.function_result or context.terminate = True async middleware @kernel_function runs only if filter calls await next(context) exceptions propagate up Default reliability behaviour: PassThroughWithoutRetry — connector-level retries are off by default; bring your own RetryMechanismBase impl to enable.
"Filters catch tool errors; reducers shrink the chat; retries you wire yourself."

How it works

Tool failures. Auto-function-invocation filters wrap every plugin call. The filter can try { await next(context) } except and either set context.function_result to recover (so the model sees a clean result), set context.terminate = True to stop the function-call loop, or re-raise. Plain function-invocation filters give you the same hook for non-auto invocations.

LLM bad output. Same filter pattern at the auto-invocation boundary — the filter can swap or null out a malformed function call before it ever runs. Beyond that, output validation is on the caller.

Large context. ChatHistoryReducer is the abstract base; concrete implementations ChatHistoryTruncationReducer and ChatHistorySummarizationReducer shrink the history before each model call.

Retries. The reliability extension is pluggable — RetryMechanismBase in reliability/ is the contract. The default ships as PassThroughWithoutRetry (i.e. none). For real retries you supply your own implementation (typically wrapping tenacity or polly-style policies).

Mode coverage

1 · Tool fails

Filter try / except around await next(); set function_result or terminate.

2 · LLM incorrect

Auto-function-invocation filter sees the proposed call; can null it out or short-circuit.

3 · Hallucination

No detector. Filters can spot blatant cases (unknown function name) but that's it.

4 · Large context

Native. ChatHistoryTruncationReducer / ChatHistorySummarizationReducer.

Auto-retry defaults

PassThroughWithoutRetry — none. The reliability layer is pluggable but ships with a no-op default. You opt in by providing a RetryMechanismBase implementation.

Code anchor

python/semantic_kernel/filters/auto_function_invocation/auto_function_invocation_context.py · terminate, function_result
python/semantic_kernel/contents/history_reducer/{chat_history_truncation_reducer,chat_history_summarization_reducer}.py
python/semantic_kernel/reliability/{retry_mechanism_base,pass_through_without_retry}.py

Side-by-side · the four failure modes

A plain-English at-a-glance summary. Read down a column for one framework's whole story; read across to compare. For the exact APIs and source paths, see the per-framework cards above.

LangChain LangGraph CrewAI Google ADK Semantic Kernel
1 · When a tool throws an error Catches the exception and feeds it back to the model as a message it can read, so the model can try a different approach. Optional automatic retry. Catches the exception at the tool step. The step can retry itself automatically with exponential backoff. Catches the exception and counts it against the agent's retry budget, then either retries or feeds the error back to the model. Calls a developer-supplied error callback that decides how to recover — substitute a fake-success response, retry, or give up. A wrapping filter catches the exception and chooses what happens next.
2 · When the LLM returns bad output Built-in parsers try to repair the broken output; a "fixer" agent can rewrite a response that doesn't parse. A validation step re-prompts the model with the schema error attached. Structured output can be requested up front to avoid the problem. A parser recognises common mistakes and re-prompts the model with the error explained. Capped by the iteration limit. An error callback fires; the response can be validated against a schema before it's accepted. A filter intercepts the model's tool choice and can reject or replace it before the call runs.
3 · When the LLM hallucinates No detection of its own. Schema validation on tool arguments catches the obvious cases. No detection of its own. The same schema validation helps with malformed tool calls. The only framework with first-class output validators — you write a function, it runs after the task and rejects bad answers. No detection of its own beyond output-schema validation. No detection of its own. A filter can notice an unknown tool name and stop the call.
4 · When the conversation gets too long Built-in helpers either drop the oldest messages or summarise them with a separate model call. You wire a pre-model hook that trims or summarises messages before every call. DIY, but well documented. An opt-in setting auto-summarises when the context fills up. A built-in compaction step kicks in once the prompt crosses a token threshold. Pluggable "reducers" either truncate or summarise the chat history.
What happens automatically (retries) Off by default. The v1 tool-retry middleware ships a 2-attempt exponential-backoff policy you can enable. Off by default. When enabled, 3 attempts with 2× backoff between each. Built in — up to 25 reasoning iterations and 2 tool retries happen silently before anything bubbles up. Live-streaming sessions reconnect automatically. Everything else is opt-in. Nothing. The default is no retry at all.
What stops a runaway loop Caps total reasoning steps and total wall-clock time. Caps how deep the graph recursion can go. Caps reasoning steps, requests per minute, and tool retries. No flow-level cap. Relies on the per-call and per-tool timeouts you set yourself. Caps how many auto-function-call rounds happen inside one model turn.
How the error gets back to the LLM Formats the error as a fake tool result and feeds it back on the next turn. Wraps the error as a tool message marked as "error" and re-enters the agent step. Adds the error as a corrective message in the conversation history. The error callback can synthesise what looks like a successful response so the loop keeps going. The filter writes a substitute result directly, replacing the failed call.

Architect's cheat sheet

Six things to remember when production starts hurting.

DEFAULT

1 · Most defaults are "no auto-retry"

Only CrewAI retries silently out of the box (max_retry_limit=2, max_iter=25). LangGraph has retry as a NamedTuple but you have to attach it. SK ships PassThroughWithoutRetry. ADK retries only the live-API connection. LangChain v1 only retries if you add the middleware.

SHAPE

2 · Errors come back to the LLM as messages

The dominant pattern across all five: serialize the exception to a string and put it where the LLM expects the tool result. The model gets a chance to react — apologise, try a different tool, or escalate. This is also why hallucinations on error formats are common: the LLM sees plausible "human-friendly" error text.

VALIDATION

3 · Hallucination = your problem

None of the five ships a hallucination detector. The closest things are CrewAI's Task.guardrail(s) (callable validators on output) and tool-schema validation everywhere else. For factual hallucination you need retrieval-grounding or a separate verifier.

CONTEXT

4 · Context handling is mostly DIY

Only ADK's CompactionRequestProcessor and CrewAI's handle_context_length auto-trigger. SK gives you reducers but you attach them; LangChain offers trim_messages as a building block; LangGraph documents the pre_model_hook pattern but ships nothing pre-baked.

OBSERVABILITY

5 · The error path is your trace path

Every framework's error hooks are also where you'd plug telemetry. ADK's plugin-then-agent callback chain, LangGraph's RetryPolicy event stream, SK's filters, CrewAI's step callbacks — pick the framework whose error surface aligns with how you plan to trace.

RISK

6 · Silent retries can hide real bugs

The same auto-retry that smooths transient flakiness will also paper over genuine regressions in your tools or model. Whatever framework you pick, log every retry — and alert on retry rate, not just failure rate.