Research date · May 6, 2026
Generated with AI

How AI Agent Frameworks Orchestrate Tool Calls

A side-by-side conceptual tour of the agent harness inside LangChain, LangGraph, CrewAI, Google ADK, and Microsoft Semantic Kernel — what the loop looks like, who holds state, and how each framework decides when to stop.

The universal loop LangChain LangGraph CrewAI Google ADK Semantic Kernel Side-by-side Cheat sheet

The Universal Agent Loop

Strip away branding and every modern AI agent framework looks the same on the inside: a control loop that asks an LLM what to do, runs the tool the LLM picked, feeds the result back, and decides whether to stop. The frameworks differ in the shape of that loop and who owns the state.

MODEL PROVIDER — the LLM endpoint Reason — the LLM call picks a tool, or returns the final answer API boundary · HTTPS / JSON AGENT HARNESS — your process User Task "Book me a flight" Continue? tool_calls? Final Answer return to user Context + state user task · messages · scratchpad entry point · sent to the model Tools schemas + handlers Act — run the tool API · code · retrieval · sub-agent request context + tool schemas response tool_call OR final no yes dispatch append result (observe), loop
Reason runs at the model provider; the other steps run in the agent harness, which owns the loop, the state, and the tools (it sends the tool schemas to the model and dispatches to their handlers). Only the request and response cross the boundary.
ReasonLLM call
The model is shown the conversation so far plus the available tools, and asked to either emit a tool call or a final answer. Both styles run the same reason–act–observe loop; they differ in how the action is encoded — provider-native function-calling JSON (modern) or the free-text Thought / Action / Observation that ReAct originally parsed.
ActTool dispatch
The framework looks up the tool by name, validates arguments against a schema, and runs it. The tool can be a Python function, an HTTP call, code execution, retrieval — or another agent.
ObserveState update
The tool result is appended to whatever store the framework uses for memory: a scratchpad, a list of messages, a graph channel, a Session/Event log, or a chat history.
DecideContinue or stop
Stop when the LLM emits no more tool calls, when it returns an explicit "final answer" structure, or when a guardrail trips: max iterations, time budget, or a custom termination strategy.

Encoding the action: native function calling vs text-based ReAct

Both run the same reason–act–observe loop that ReAct introduced, and both end with the framework running a tool and feeding the result back. They differ only in how the action is encoded — and therefore in who has to parse it.

Native function callingProvider-structured JSON

The request carries the tool definitions, and the model returns the tool call — a tool name plus JSON arguments — in a dedicated response field, separate from its natural-language text. The framework reads that field directly instead of scanning the model's prose for an action. With strict / structured-output modes the provider can also guarantee the JSON is valid and matches your schema. You still parse the arguments, but you don't have to extract the call from free text or hope it is well-formed. This is what OpenAI, Anthropic, Gemini, and the other current providers emit natively. 

// assistant message field (OpenAI Chat Completions)
"tool_calls": [
  {
    "type": "function",
    "function": {
      "name": "get_weather",
      "arguments": "{\"city\":\"Paris\"}"
    }
  }
]
Source: OpenAI · Function calling · companion paper How AI Providers Support Tool Calls
Text-based ReActFree-text, framework-parsed

ReAct names the reason–act loop itself; in its original formulation the model was prompted to interleave reasoning and actions as plain text in a Thought / Action / Action Input / Observation format. The framework parses that text to extract the action and its input, runs the tool, and appends the observation. It works with any text model, but is more brittle: it depends on the model formatting its output exactly as instructed. Prompting the model to reply in JSON narrows the format gap, but without the provider's schema guarantee the framework must still locate the call in the text and defend against malformed or prose-wrapped output. 

Thought: I should look up the weather.
Action: get_weather
Action Input: {"city": "Paris"}
Observation: 15°C, cloudy
... (the loop repeats until a final answer) ...
Source: Yao et al., “ReAct” (ICLR 2023, arXiv:2210.03629) · project page

Five frameworks, five takes on the loop

Same building blocks; very different opinions about who runs the show. Below: each framework's orchestration model in one diagram and a few words.

Lc

LangChain · the classic AgentExecutor

A while-loop wrapped around an LLM and a parser.
while _should_continue(iter, time): ... LLM Chain Prompt + tool descriptions → raw text or tool_calls Output Parser → AgentAction or AgentFinish Tool.invoke() name → callable lookup returns Observation intermediate_steps List[(action, observation)] aka the "scratchpad" parse dispatch append re-prompt
"A while loop, an LLM call, a parser, and a dictionary of tools."

How orchestration works

AgentExecutor drives a single loop. On each turn it formats the prompt (system + tools + scratchpad), calls the LLM, and hands the response to an output parser. The parser produces either an AgentAction (tool name + args) or an AgentFinish (final answer). Actions are executed, the result becomes an Observation, and both get appended to the scratchpad before the next iteration.

How it stops

On AgentFinish, on a tool marked return_direct, or when the guardrail trips: max_iterations (default 15) or max_execution_time.

Mechanism

Single linear loop. Each step is recorded in intermediate_steps.

Tradeoffs

No native branching, parallelism, or human-in-the-loop primitives. Most net-new work has moved to LangGraph.

Lg

LangGraph · the loop as a stateful graph

Two nodes, one router, a shared message channel — and a checkpointer.
append ToolMessages → re-enter "agent" START agent model.bind_tools(tools) .invoke(state.messages) tool_calls? tools (ToolNode) runs every tool_call → ToolMessage[] END State channel · messages add_messages reducer · checkpointed yes no → END
"The agent loop, but expressed as a directed graph you can checkpoint and resume."

How orchestration works

create_react_agent(model, tools) builds a tiny StateGraph with two nodes — agent (LLM call) and tools (a ToolNode that fans out every tool_call). A conditional edge should_continue reads the last AIMessage: if it has tool_calls, route to tools; otherwise route to END. State is the messages channel, accumulated by an add_messages reducer.

What you get for free

Persistence (any checkpointer — memory, SQLite, Postgres), time-travel debugging, branching, parallel Send fan-out, human-in-the-loop via interrupt_before/interrupt_after, and streaming events at node-level.

Mechanism

Composable into multi-agent supervisors. State is explicit.

Tradeoffs

Requires modeling logic as nodes, edges, reducers, and channels rather than imperative scripts.

Cr

CrewAI · roles, tasks, and a Process that orchestrates them

Multiple agents collaborating like a team; each agent runs its own inner ReAct loop.
Crew · Process.sequential | hierarchical Researcher role · goal · backstory tools: [search, scrape] Task 1 → output Analyst tools: [calculator, sql] context = prev outputs Task 2 → output Writer tools: [doc_writer] Task 3 → final ↓ Inside each agent: CrewAgentExecutor._invoke_loop() LLM native tools? → JSON else → ReAct text tool_call? Tool + tool cache + guardrails Final AgentFinish yes no append observation → re-prompt
"Hire a team of role-played agents. Give each a goal and tools. Run them like a workflow."

Two layers of orchestration

Outer layer — the Crew. A Crew owns a list of Agents and a list of Tasks, executed by a Process that is either sequential (pass each task's output as context to the next) or hierarchical (a manager LLM decides who runs what next).

Inner layer — the Agent. Inside each agent, CrewAgentExecutor._invoke_loop() picks one of three modes based on the LLM's capabilities: _invoke_loop_native_tools (modern function-calling), _invoke_loop_react (text-based ReAct fallback), or _invoke_loop_native_no_tools. Each is a while True that calls the LLM, dispatches a tool if one was requested, appends the result, and repeats until AgentFinish or max_iter.

State + comms

Per-agent: a message list plus a tool-result cache. Cross-agent: in sequential Process the Crew automatically aggregates each finished task's output as context for the next; you can also wire explicit dependencies via Task.context. In hierarchical mode the manager agent gets AgentTools (a DelegateWorkTool per colleague) and chooses who runs what next.

Mechanism

Role-based "team of specialists" metaphor. Hierarchical Process uses two loops.

Tradeoffs

In Hierarchical Process mode, the manager LLM is invoked between every task, adding an LLM call per delegation.

Ad

Google ADK · a pipeline of processors and a tree of agents

An Event-streaming runtime where each LLM turn passes through an ordered list of request/response processors.
Runner — drives an InvocationContext, streams Events Agent composition root: LlmAgent SequentialAgent ParallelAgent LoopAgent · max_iters sub LlmAgent · transfer_to_agent BaseLlmFlow · per turn request: basic · instructions · identity + contents · NL planning · code_execution + agent_transfer (AutoFlow extends SingleFlow) LLM call response: NL planning · code_execution then handle_function_calls_async → Tool/Agent emit Event(s) → Session callbacks/plugins observe each step ↺ repeat until no function_calls
"Compose agents like Lego; inside each one, the LLM turn flows through a fixed processor pipeline."

Two orchestration axes

Composition. ADK ships a small algebra of agent shapes: LlmAgent (the reasoning unit), SequentialAgent, ParallelAgent, and LoopAgent. Compose them into a tree. An LlmAgent can also transfer control to a sibling/sub-agent by emitting a transfer_to_agent action.

Per-turn flow. A BaseLlmFlow — concretely SingleFlow for solo agents and AutoFlow (which adds agent_transfer) for multi-agent — runs each LLM turn through an ordered list of request processors (basic, instructions, identity, contents, NL planning, code execution, and several others), calls the model, runs response processors (_nl_planning, _code_execution), and finally invokes handle_function_calls_async to dispatch any function_calls on the response to the matching tool or sub-agent. Every step emits Events into the Session.

State + observability

State lives in a typed Session as an append-only stream of Events. Plugins and callbacks (before_model_callback, after_tool_callback, …) hook into every transition — the same hooks power tracing, eval, and policy.

Mechanism

Built-in composition. Observability/eval, streaming, and live (audio) are all native.

Tradeoffs

Larger API surface than LangChain. The processor pipeline has its own lifecycle and ordering rules.

Sk

Microsoft Semantic Kernel · the Kernel auto-invokes plugin functions

The agent delegates the loop to the model's function-choice behaviour; group chats add a Selection + Termination strategy.
ChatCompletionAgent.invoke() ChatHistory system + user + assistant + tool msgs (persisted in AgentThread) Kernel FunctionChoiceBehavior .Auto(max_invoke=N) if function_call → invoke kernel function Plugins @kernel_function native or prompt → exposed as "functions" ↺ Kernel loops internally until no function_call AgentGroupChat (multi-agent) Selection strategy → next agent Agent N .invoke() → message Broadcast → each agent's history Termination strategy? if not done → next round
"Plugins are functions, the Kernel calls them, and the model decides when to ask."

Single-agent: the Kernel does the looping

A ChatCompletionAgent is mostly a thin wrapper: it loads the ChatHistory from an AgentThread, sets a FunctionChoiceBehavior (typically Auto), and asks the Kernel to chat-complete. With Auto behaviour, the Kernel itself runs the function-call loop: model emits a function call → Kernel resolves it to a registered @kernel_function in some plugin → invokes it → appends the result to the chat → calls the model again — until no function calls are emitted or maximum_auto_invoke_attempts is hit.

Multi-agent: AgentGroupChat

For collaborations, AgentGroupChat uses a SelectionStrategy to pick which agent speaks next, runs that agent's invoke(), broadcasts the resulting message into every agent's history, then asks a TerminationStrategy whether to stop. The strategy can be a kernel function — i.e. another LLM judging the conversation.

Mechanism

.NET- and Python-friendly API. Filters, telemetry, and identity abstractions are built in. Tools are plugins registered with the Kernel.

Tradeoffs

Loop control is encapsulated inside the Kernel. Fine-grained intervention requires Kernel-level service registration rather than explicit graph nodes.

Side-by-side

Five views of the same loop. Read across rows to see how each framework answers the same question.

LangChain LangGraph CrewAI Google ADK Semantic Kernel
Loop shape Single while in AgentExecutor Two-node StateGraph with conditional edge Crew→Tasks (outer) + per-Agent while (inner) Agent tree + per-turn processor pipeline Kernel-driven function-choice loop
Tool dispatch Output parser → name lookup in name_to_tool_map ToolNode fans out every tool_call on the last AIMessage _handle_native_tool_calls (or ReAct text parse) → tool with caching/guardrails handle_function_calls_async after response processors Kernel auto-invokes the matching @kernel_function
State / memory intermediate_steps scratchpad (per run) messages channel + checkpointer (durable) Per-agent message list + cross-task context Session as an append-only Event stream ChatHistory persisted via AgentThread
Termination AgentFinish, return_direct, max_iterations, max_execution_time No tool_calls on last message → END (or recursion_limit) AgentFinish or max_iter; Crew ends when all tasks complete No more function_calls; LoopAgent uses max_iters No more function calls; maximum_auto_invoke_attempts; or TerminationStrategy in group chat
Multi-agent Manual — wrap an agent as a Tool Compose StateGraphs; supervisor patterns First-class — Crew + Process (sequential / hierarchical) Sequential / Parallel / Loop agents + transfer_to_agent AgentGroupChat with Selection + Termination strategies
Streaming & HITL Callback handlers; HITL via custom tools Native streaming; interrupt_before/after for HITL Step callbacks; built-in HITL via task.human_input=True Native event stream; request_confirmation processor for HITL Streaming via invoke_stream; filters/middleware for HITL
Code anchor libs/langchain/langchain_classic/agents/agent.py libs/prebuilt/.../chat_agent_executor.py · create_react_agent lib/crewai/.../agents/crew_agent_executor.py src/google/adk/flows/llm_flows/base_llm_flow.py python/semantic_kernel/agents/chat_completion/chat_completion_agent.py
Best fit Simple "LLM + tools" prototypes; teaching the loop Production agents with checkpointing, branching, HITL "Team of specialists" workflows; quick org-style prototypes Polyglot agent systems on Google infra; eval/observability heavy Enterprise .NET/Python apps with plugin-centric integration

Architect's cheat sheet

Five things to remember when you compare these frameworks at a whiteboard.

PATTERN

1 · It's the same loop, just different topologies

Reason → Act → Observe → Decide. Whether it's a while, a graph, a pipeline, or a kernel-driven auto-invoke — the four-step semantics are identical. Pick the topology that matches the property you need most: simplicity, durability, composition, or enterprise plumbing.

STATE

2 · "Where does the state live?" is the real differentiator

Scratchpad (LangChain), Channel + Checkpoint (LangGraph), Session/Event log (ADK), ChatHistory/Thread (SK), per-Agent message list (CrewAI). This drives durability, replay, time-travel, and how easy multi-agent gets.

CONTROL

3 · Implicit vs. explicit termination matters in production

LangGraph and ADK make the stop condition a first-class artifact (a router edge, a processor). LangChain and SK rely on the model+limit. For long-running, expensive, or regulated agents, prefer the explicit version.

SCALE

4 · Multi-agent isn't "one feature" — it's an architectural shift

CrewAI, ADK, and SK build multi-agent in. LangGraph composes graphs to do it. LangChain treats other agents as tools. Pick based on whether collaboration is core to your domain or an occasional pattern.

OBSERVABILITY

5 · Hooks decide whether you can run this in production

Look for: per-step callbacks (LangChain), node-level streaming (LangGraph), pre/post model + tool callbacks and plugins (ADK), filters + telemetry (SK), step callbacks (CrewAI). If you can't intercept every transition, you can't trace, eval, or guard.

MIGRATION

6 · The harness is the easy part to swap

Tools, prompts, evals, and guardrails are where the value compounds. Treat the orchestration framework as replaceable infrastructure — keep your tool definitions, prompts, and eval harness portable.

Mini-glossary

A few terms used above that mean different things in different frameworks.

Tool / Function / Plugin function — a callable the model can ask the framework to run, exposed by a JSON schema.
Tool call — the LLM's structured request to run a tool with named arguments (provider-native function calling).
ReAct — older pattern where the model emits "Thought / Action / Action Input / Observation" as plain text; the framework parses it.
Scratchpad — the running list of (action, observation) pairs the framework re-injects into the prompt each turn.
StateGraph / channel / reducer — LangGraph primitives; channels accumulate values via a reducer (e.g. add_messages).
Checkpointer — pluggable store (memory, SQLite, Postgres) that persists graph state at each step → resume / time-travel.
Process (CrewAI) — strategy that turns a list of Tasks into an executable plan: sequential or hierarchical (with a manager LLM).
InvocationContext / Session / Event (ADK) — the runtime container, the persisted conversation, and the typed entries that make it up.
FunctionChoiceBehavior (SK) — Auto / Required / None — tells the Kernel whether to let the model pick functions and to auto-invoke them.
SelectionStrategy / TerminationStrategy (SK) — pluggable policies for who speaks next and when the group chat ends.