The observability chapter asked whether spans exist and how they nest. This companion asks the question you only feel at 2 a.m.: when the agent gives a wrong answer, can you see the exact prompt it sent, the visible reasoning it produced, the tool it called with which arguments, and what came back — the full ReAct loop? And is that visibility free, hidden behind a flag, or locked to a vendor? A source-grounded infographic across the same ten frameworks.
The ten frameworks, unchanged from Chapter 1: LangChain, LangGraph, CrewAI, Google ADK, Microsoft Semantic Kernel (Python), Mastra (TypeScript), Microsoft Agent Framework (.NET), LangChain4j (Java), Genkit (Go), and Rig (Rust).
HEAD, with the exact commit SHAs listed there.
"I have tracing" can mean three very different things. Almost every framework gives you the first layer for free. The layer that actually lets you debug a wrong answer — the verbatim content — is a separate decision, and the layer that decides whether you're trapped is a third. Chapter 1 measured Layer 1. This page is about Layers 2 and 3.
Span shape, model name, token counts, latency, finish reason, error type. Tells you that an LLM call happened and how long it took — never what was said.
Chapter 1 lives here · on by default almost everywhereThe verbatim prompt, the model's completion (including any visible reasoning it emits), the tool name + arguments, and the tool's result. This is what you read to understand why the agent did the wrong thing.
Often gated behind a "sensitive data" flag · the subject of this pageIs the content emitted as vendor-neutral OpenTelemetry you can point anywhere, or is the only well-paved path a specific SaaS product you'll keep paying for?
Free vs paid & lock-in · §5 and §6
A framework can score "usable spans on day one" in Chapter 1 and still show you
not a single prompt without extra work. MS Agent Framework is the textbook case: you
get spans and token counts out of the box, but the prompt and response stay dark until you
flip a sensitive-data switch37. Semantic Kernel
is stricter still — its LLM spans stay off until you set one of its two diagnostics flags
(only a bare execute_tool span is emitted ungated), and content needs the
sensitive one34. That gap — between "I have
tracing" and "I can read what my agent actually said" — is, in our experience, one of the
most common production surprises.
Debugging an agent means replaying its reasoning loop. A single step has four things worth capturing — and a framework can capture any subset. "It logs LLM calls" usually means only the first and maybe the second. True depth means all four, every step, verbatim.
System instructions + user message + the entire prior conversation, exactly as serialized to the model.
Capture point 1 →The model's raw output text — including any visible "thought" it emits before an action. (Hidden reasoning tokens aren't returned by providers, so they can't be traced.)
Capture point 2 →Which tool the model chose and the actual arguments it passed — not just "a tool ran".
Capture point 3 →The verbatim content the tool returned, which feeds the next reasoning step and closes the loop.
Capture point 4In practice, many wrong answers aren't model failures — they're loop failures. The tool returned stale data (you need capture 4), or the model hallucinated an argument (capture 3), or the prompt template silently dropped a variable (capture 1), or the reasoning went sideways before the call (capture 2). Miss any one capture point and you're guessing about that class of bug. The matrix below scores each framework on whether it emits all four to a backend you control.
For each framework: are the four capture points actually emitted to your telemetry backend? The distinction that matters is not "does the data exist" — it nearly always exists in-process — but "does the official path put it where you can search it."
A note on "Completion": this column means the model's full output text, including any visible reasoning or "thought" it emits as part of that output. Hidden reasoning tokens from reasoning models are generally not returned by the provider's API at all, so no framework can trace them — what you get is the text the model actually returned.
| Framework | Prompt | Completion | Tool args | Tool result | Out-of-the-box state · where it lives | Sources |
|---|---|---|---|---|---|---|
| LangChainPython | ⚑ | ⚑ | ⚑ | ⚑ | Verbatim in callbacks for free; reasoning preserved in AgentAction.log. Export to a backend is opt-in (LANGSMITH_TRACING — hence the flags); content is not redacted unless you ask. |
123458 |
| LangGraphPython | ⚑ | ⚑ | ⚑ | ⚑ | Identical to LangChain — reuses langchain-core callbacks (same opt-in export). No separate content path. |
61 |
| CrewAIPython | ＋ | ＋ | ＋ | ＋ | Core telemetry.py is anonymized usage data — no content. Full content rides the event bus → reaches a backend only via a third-party instrumentor (OpenLIT / OpenInference), or via built-in tracing → CrewAI AMP. |
111213141771 |
| Google ADKPython | ✓ | ✓ | ✓ | ✓ | ON by default — ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS defaults true; full llm_request/llm_response/tool_call_args/tool_response on spans. (Defaults off on managed Cloud deploy unless you set it explicitly.) |
192021222325 |
| Semantic KernelPython | ⚑ | ⚑ | ⚑ | ⚑ | LLM telemetry off by default — spans + metrics need …ENABLE_OTEL_DIAGNOSTICS or …_SENSITIVE (either enables diagnostics); content needs …_SENSITIVE. Both default off, so the model-diagnostics decorators no-op until set; only a bare execute_tool span is ungated. Then emits gen_ai.*.message events + tool arg/result attrs. |
293031323334 |
| MastraTypeScript | ✓ | ✓ | ✓ | ✓ | No separate content flag — once observability is configured on the Mastra instance, input/output (incl. tool args + results) are captured by default; redaction / hide / sampling are opt-in. |
4344454648 |
| MS Agent Framework.NET | ⚑ | ⚑ | ⚑ | ⚑ | Spans + token metrics by default; content needs EnableSensitiveData / OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT. Standard Gen AI events. |
37383942 |
| LangChain4jJava | ＋ | ＋ | ＋ | ＋ | Official modules emit token/latency metadata only. Full messages live in ChatModelListener objects in-process — you write a listener that serializes them onto spans yourself. |
5657585960 |
| GenkitGo | ✓ | ✓ | ✓ | ✓ | ON by default — full verbatim JSON in genkit:input/genkit:output, untruncated in the local Dev UI. Cloud export truncates at 128k chars and is toggleable. |
5051525354 |
| RigRust | ✓ | ✓ | ✓ | ✓ | Prompt / completion / tool name+args / result recorded onto Gen-AI-semconv spans at INFO; full verbatim request/response JSON at TRACE. Appears once you attach a subscriber that admits INFO (tool-dispatch args need DEBUG). | 6364656667 |
Six frameworks emit the full loop to a backend you control with no per-field heroics: ADK, Mastra, Genkit and Rig do it on by default; MS Agent Framework does it the moment you flip the sensitive-data flag, and Semantic Kernel the moment you set its sensitive flag (which also switches on the LLM spans and metrics). LangChain / LangGraph also capture everything verbatim — export itself is opt-in (one env var), and the real question is the destination, covered next. The two genuine gaps are CrewAI (its native telemetry is anonymized; content needs an add-on or its own SaaS) and LangChain4j (official telemetry is metadata-only; content is yours to wire).
Same data, framed as a single question: how far can you debug straight out of the box, before writing custom code? Four tiers — from "attach an exporter and read everything" to "the content is on you."
The top tier costs you one exporter and nothing else — the words are already on the spans. The second tier is identical in fidelity; the only difference is a deliberate, sane default: Microsoft ships content off so you don't accidentally stream user PII to a dashboard, and you opt in per environment (Agent Framework still emits spans + metrics meanwhile; Semantic Kernel gates even its LLM spans behind a diagnostics flag — either flag enables them). The third tier captures everything too, but makes you pick a route — point LangChain at your own OTLP collector, or drop OpenLIT in front of CrewAI. Only LangChain4j leaves the content emission entirely to you.
The headline first, because it cuts against the reputations: capturing the content is free and open-source in all ten frameworks. You never pay to record the prompt and response. What's sometimes paid is the destination — the hosted UI you choose to send it to — and that is always optional.
Open-source instrumentation, in every framework
Convenient hosted UIs & evals — never required to see content
| Framework | Capturing content is free? | Optional paid product | Is the paid product required? | Sources |
|---|---|---|---|---|
| LangChainPython | Yes — Apache-2.0 callbacks | LangSmith | No — SDK emits OTLP to your own backend (since late 2024) | 7910 |
| LangGraphPython | Yes | LangSmith · LangGraph Platform | No — same OTLP path as LangChain | 610 |
| CrewAIPython | Yes — via OSS OpenLIT / OpenInference | CrewAI AMP | Only for the built-in tracing path; the add-on path is free | 1116171870 |
| Google ADKPython | Yes — standard OTel SDK | Google Cloud Trace | No — any OTLP backend works | 262728 |
| Semantic KernelPython | Yes — standard OTel SDK | Azure App Insights | No | 3536 |
| MastraTypeScript | Yes — local storage + Studio | Mastra Cloud | No | 4749 |
| MS Agent Framework.NET | Yes — std OTel + M.E.AI | Azure App Insights | No — added conditionally, alongside OTLP | 4041 |
| LangChain4jJava | Yes — Apache-2.0 | — (no first-party paid product) | — | 6162 |
| GenkitGo | Yes — Dev UI needs no cloud | Google Cloud / Genkit Monitoring | No, but cloud export needs a GCP project & truncates | 5355 |
| RigRust | Yes — MIT | — (no first-party paid product) | — | 6869 |
"LangChain means paying for LangSmith" and "CrewAI tracing means CrewAI's cloud" are both now
escapable. LangChain's SDK can send traces only to your own OTLP collector — its
LANGSMITH_OTEL_ONLY mode bypasses LangSmith79;
CrewAI's execution content can be exported to any OTLP backend through a free OSS instrumentor
like OpenLIT, which logs prompts and completions to span attributes by default1770.
The paid products buy you a polished UI and eval tooling — not access to your own telemetry.
"Pure telemetry" means the content is standard OpenTelemetry you can repoint at any backend on a config change. "Lock-in" means the well-paved path runs through one vendor and getting out is real work. Here's where each framework sits.
Standard gen_ai.* attributes / events (Mastra uses its own span format instead, but ships ~10 exporters in-tree, incl. OTel). Swap backends freely.
Content travels on valid OTel spans (vendor-neutral transport), so it goes anywhere — but on Google-specific attribute keys rather than the gen_ai.* conventions, so it renders best in Google Cloud. ADK uses gcp.vertex.agent.* keys19; Genkit uses genkit:input/genkit:output and ships only a GCP prod exporter, which truncates5052.
LangSmith is the path of least resistance, but an OTLP exit exists and content isn't captive7. You choose neutrality; you don't get it by default.
Lock-in here is about the default path, not a hard wall. Every framework's content can ultimately reach a neutral backend, because it all rides on OpenTelemetry primitives. The question is how much that costs you: nothing for the green group, a slightly less polished view for the Google pair, a deliberate config choice for the LangChain family, and an extra component you install for CrewAI.
Pick by the constraint you actually have.
ADK, Mastra, Genkit, Rig. Full ReAct content is on the spans by default — attach an exporter and you're reading the loop. Genkit's local Dev UI is the fastest path to an untruncated inspector.
MS Agent Framework, Semantic Kernel. Agent Framework ships spans + token metrics on, with content one explicit sensitive-data flag away. Semantic Kernel is stricter — even its LLM spans stay off until you enable a diagnostics flag — a posture some teams prefer for regulated workloads where nothing should emit unasked.
SK, MS Agent Framework, Mastra, Rig, LangChain4j. All exportable to self-hosted
Jaeger or Tempo with no SaaS in the path — SK, Agent Framework and Rig on standard
gen_ai.* OTel; Mastra via its in-tree OTel exporter (own span format).
LangChain4j is the only one where you must write the content listener yourself.
Genkit (local Dev UI, untruncated) and the LangChain family via the free LangSmith tier (~5K traces, 14-day retention)10 — with a documented OTLP escape hatch if you outgrow it.
CrewAI — core telemetry is anonymized; add OpenLIT or you'll see no content11. LangChain4j — official modules are metadata-only57. Genkit — cloud export truncates at 128k52. ADK — content defaults off on managed Cloud deploy unless you set the env var yourself24.
Capturing the full ReAct loop is free everywhere; six frameworks make it easy and vendor-neutral; the real differences are a flag (Microsoft), an add-on (CrewAI), or your own code (LangChain4j) — not a paywall.
Every claim above maps to a numbered reference here. Code references are GitHub permalinks pinned to a specific commit (so the file and line stay valid); doc references are official pages. Verified June 5, 2026; OpenLIT reference re-verified June 9, 2026.
cdafe60 · LangGraph 2b1abc8 ·
CrewAI 913a3ab · Google ADK 9670ce2 · Semantic Kernel
417d62f · MS Agent Framework dcc218d · Mastra d53cfc2 ·
Genkit 783fbf2 · LangChain4j fa449aa · Rig dab75e0 ·
OpenLIT 290e8ee.