Langfuse vs LangSmith vs Helicone: which LLM observability tool should you use?

What does LLM observability actually cover?

LLM observability tools record what your application sent to a model, what came back, how long it took and what it cost. For multi-step apps, they group those calls into traces so you can see a whole agent run or RAG query end to end.

Most tools now go further: prompt versioning, datasets of test cases, evaluations that score outputs, human annotation queues and dashboards. The differences are in how data is captured and where it is stored.

Langfuse, LangSmith and Helicone are three of the most widely used options, and they take noticeably different approaches to the same problem.

How do they compare side by side?

SDK tracing versus a proxy: why it matters

Langfuse and LangSmith mainly instrument your code. You wrap functions or use integrations, and the SDK sends trace data asynchronously. That captures rich structure, such as retrieval steps, tool calls and custom metadata, but requires code changes.

Helicone’s signature approach is a proxy: you change the model API base URL and add a header, and every request is logged. That is the fastest way to see costs and latency across an app, and it enables gateway features such as caching and rate limiting.

The trade-off is that a proxy sits in the critical path. If it is slow or down, your requests are affected, unless you use its async logging mode or run it yourself close to your app. Proxies also see only model calls, not the non-LLM steps between them.

Many mature teams end up with both patterns: a gateway for cost control and routing, plus SDK tracing for deep debugging of complex flows.

OpenTelemetry is becoming the common language between these approaches. Emerging semantic conventions for generative AI mean that instrumentation written once can, increasingly, be sent to more than one backend, which lowers the cost of switching tools later.

When is Langfuse the right choice?

Langfuse is a strong default when you want open source and the option to keep traces on your own infrastructure. Prompts and outputs often contain customer data, so self-hosting can simplify privacy reviews.

It is framework-agnostic, with integrations for popular SDKs and frameworks and OpenTelemetry support, so you can instrument a LangChain app, a LlamaIndex pipeline and plain API calls the same way.

Self-hosting at scale means running its storage components, including Postgres and an analytics database in recent versions, so budget operational time. Its managed cloud is an option if you prefer not to.

Its prompt management links each trace to the exact prompt version that produced it, which turns “the bot got worse last week” into a concrete diff. Evaluations can run on production traces or on curated datasets.

When is LangSmith the right choice?

If your app is built on LangChain or LangGraph, LangSmith gives you tracing with almost no setup. Runs, nested steps and graph state appear automatically, which is valuable when debugging agents.

Its evaluation workflow, with datasets, experiments that compare prompt or model versions, and annotation queues, is mature and designed around the same concepts as the frameworks.

It also works with non-LangChain code through its SDK. The main considerations are that it is proprietary and self-hosting is part of enterprise offerings, so data residency needs a conversation with the vendor.

The playground lets you rerun a traced call with a changed prompt or model, which shortens the loop between spotting a bad answer and testing a fix. For LangGraph users, seeing state at each node is often the fastest way to find where an agent went wrong.

When is Helicone the right choice?

Helicone suits teams that want visibility today: which features cost the most, which users make the most calls, and where latency spikes. Changing one base URL gets you there.

Because it is open source and self-hostable, you can keep logging inside your own infrastructure. Gateway features such as caching and retries can also reduce cost and improve reliability.

Caching deserves care: it helps for repeated identical requests, such as classification of common inputs, but can return stale or wrong answers where context matters. Enable it per route, not globally.

For deep debugging of multi-step agents, pure proxy logging shows less structure than SDK tracing. Helicone offers ways to group requests into sessions, which helps, but plan how you will link calls together.

Custom properties sent as headers, such as feature name, customer tier or experiment label, let you slice costs and latency in its dashboards. Doing this from day one makes pricing and optimisation decisions much easier later.

Which should you choose?

• Open source, self-hosting and framework-agnostic tracing plus evaluations: Langfuse.
• Building on LangChain or LangGraph and wanting zero-effort tracing and evals: LangSmith.
• Fast cost and latency visibility across all model calls, or a gateway with caching: Helicone.
• Strict data residency without an enterprise contract: Langfuse or Helicone self-hosted.
• Large team standardising on OpenTelemetry: prefer tools with solid OpenTelemetry ingestion, then compare dashboards.
• Solo developer prototyping: any of the three free tiers; switch once needs are clearer.

How to roll out observability in a week

The order matters. Cost and latency visibility pays for itself immediately, while evaluations pay off over months. Starting with logging builds the habit and the data you will later turn into test sets.

Whichever tool you choose, decide on a retention period and a redaction approach for personal data before production traffic arrives, not after the first privacy review.

• Day one: log every model call with model name, tokens, latency and a user or session identifier.
• Day two: group calls into traces per request and add metadata such as feature name and prompt version.
• Day three: move prompts into versioned prompt management so changes are traceable.
• Day four: build a dataset of twenty to fifty real questions with good answers.
• Day five: run an evaluation on that dataset for every prompt or model change before shipping.

Common mistakes

RepoLoot’s catalog tags observability and evaluation projects by licence and self-hosting difficulty, useful if you want to compare more open tools.

• Logging full prompts with personal data without redaction or a retention policy.
• Tracking cost but not quality, so cheaper changes quietly make answers worse.
• Trusting LLM-as-judge scores without spot-checking them against human judgement.
• Instrumenting only the model call and missing retrieval, which is often where RAG fails.
• Putting a hosted proxy in the request path without a fallback when it is unreachable.

Frequently asked questions

Is Langfuse really open source?

Langfuse’s core is released under the MIT licence and can be self-hosted, while some enterprise features are licensed separately. Check the repository’s licence file for the exact split, especially if you need features such as advanced access controls on a self-hosted instance.

Does LangSmith work without LangChain?

Yes. LangSmith has its own SDKs and supports tracing arbitrary functions and model calls, as well as OpenTelemetry. The integration is simply deepest with LangChain and LangGraph, where tracing is nearly automatic.

Does a proxy like Helicone add latency?

Any proxy adds a network hop, and the impact depends on where it runs relative to your app and the model provider. Helicone also supports async logging that keeps it out of the request path. Measure in your own environment before and after.

Can I use more than one of these tools?

Yes. A common pattern is a gateway or proxy for cost tracking and routing, plus SDK-based tracing for debugging and evaluations. Keep a single request identifier across both so you can connect a cost spike to the exact trace that caused it.