Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,12 @@ When documenting LLM setup or examples, ensure all three options are mentioned w
- `sdk/shared-snippets/how-to-run-example.mdx` - Shared snippet for running examples
- `sdk/guides/llm-subscriptions.mdx` - Dedicated guide for subscription login


## Documentation ownership patterns

- Keep `sdk/guides/observability.mdx` as the canonical tracing and OTEL reference for OpenHands SDK users, including Laminar links and generic backend configuration.
- Keep `enterprise/analytics.mdx` focused on OpenHands Enterprise deployment and admin-console setup, and link back to the SDK observability guide instead of duplicating OTEL reference material.

## Validation

### LLM pricing table validation
Expand Down
103 changes: 68 additions & 35 deletions enterprise/analytics.mdx
Original file line number Diff line number Diff line change
@@ -1,35 +1,30 @@
---
title: Analytics
description: Get started with LLM observability and tracing in OpenHands Enterprise.
description: Deploy Laminar for trace analysis in OpenHands Enterprise.
icon: rocket
---

This guide walks you through setting up and using Laminar for Analytics in OpenHands Enterprise.
You'll opt into Analytics and configure conversations to automatically send traces to Laminar.
This guide walks you through enabling Laminar in OpenHands Enterprise (OHE) so conversations automatically send traces for observability and analysis.

## Who This Is For

This guide is for users who want to explore analytics on their OpenHands Enterprise conversations.

### Why Laminar?
For SDK-level tracing concepts, OTEL environment variables, and non-Laminar backends, see [Observability & Tracing](/sdk/guides/observability).

[Laminar](https://laminar.sh/) is an open source observability platform for AI agents like OpenHands.
## Who This Is For

Use Laminar to view your conversation traces including prompts, tool calls, and answers. A trace is a record of what your agent did.
This guide is for users who want to deploy Laminar alongside OpenHands Enterprise and inspect traces from Enterprise conversations.

Laminar can help you see where the agent went wrong. From traces, you can create signals. A signal is a natural language instruction to extract structured data from traces. Use signals to analyze recurring behavior across traces. You can then create better situations for prompting and measure them in Laminar. You can also analyze and improve your skills.
## Why Laminar in OHE?

For example, you can view all conversation traces related to a specific skill.
Laminar helps you understand what your OpenHands deployment is doing in production:

Laminar can help you answer the following questions:
- On a trace, did the agent do a good job using the skill?
- On another trace, did the agent do a bad job?
- Inspect prompts, tool calls, answers, and nested agent behavior in Laminar's [trace views](https://laminar.sh/docs/platform/viewing-traces).

Check warning on line 19 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L19

Did you really mean 'Laminar's'?
- Use [session replay for browser agents](https://laminar.sh/docs/tracing/browser-agent-observability) when conversations drive browser automation.
- Define [signals](https://laminar.sh/docs/signals/introduction) to classify failures, measure outcomes, and monitor recurring patterns across many traces.

For more information on evaluating skills, see [Evaluating Agent Skills](https://www.openhands.dev/blog/evaluating-agent-skills).

### Prerequisites
## Prerequisites

Before you begin, make sure you completed the [Quick Start guide](/enterprise/quick-start).
Before you begin, complete the [Quick Start guide](/enterprise/quick-start).

## Enable Analytics

Expand All @@ -41,63 +36,92 @@

## Deploy

OpenHands will begin deploying. You can expect the deployment status to transition from
**Missing** to **Unavailable** to **Ready**. This typically takes 10-15 minutes.
OpenHands will begin deploying. You can expect the deployment status to transition from **Missing** to **Unavailable** to **Ready**. This typically takes 10-15 minutes.

![Deployment in progress](./images/laminar-deploy-in-progress.png)

Click **Details** next to the deployment status to monitor individual resources. Resources
shown in orange are still deploying -- wait until all resources are ready.
Click **Details** next to the deployment status to monitor individual resources. Resources shown in orange are still deploying, so wait until all resources are ready.

![Deployment status details](./images/laminar-deployment-status-details.png)

## Access Laminar UI
## Access the Laminar UI

Once the deployment status shows **Ready**, navigate to `https://analytics.app.<your-base-domain>`.

Click the **Continue with Keycloak** button:

Check warning on line 51 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L51

Did you really mean 'Keycloak'?

![Laminar Keycloak Auth](./images/laminar-keycloak-auth.png)

Check warning on line 53 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L53

Did you really mean 'Keycloak'?

## Create a Laminar project
If you want more background on Laminar Cloud versus self-hosting outside OHE, see Laminar's official [hosting options](https://laminar.sh/docs/hosting-options).

Check warning on line 55 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L55

Did you really mean 'Laminar's'?

## Create a Laminar Project

Create a project in the Laminar UI:

![Laminar Create Project](./images/laminar-create-project.png)

Once a project has been created, Laminar is ready to listen for traces.

![Laminar Listen Traces](./images/laminar-listen-traces.png)

## Create an ingest only API Key
## Create an Ingest-Only API Key

Important: Always use ingest API keys when deploying.
Always use ingest-only API keys when deploying OHE.

Create a key with the right permissions. Ingest only keys are recommended as they only have write access to write traces. They cannot be used to read data.
Ingest-only keys are recommended because OHE only needs permission to write traces. They cannot be used to read trace data.

![Configure Laminar Ingest Only Key](./images/laminar-ingest-only-key.png)

## Set Laminar Project API Key to enable automatic conversation traces
## Set the Laminar Project API Key

Set the ingest only key as the Laminar Project API Key in the Admin Console configuration:
Set the ingest-only key as the **Laminar Project API Key** in the Admin Console configuration. This is the same `LMNR_PROJECT_API_KEY` described in the [SDK observability guide](/sdk/guides/observability).

![Configure Laminar Project API Key](./images/laminar-configure-key.png)

Click **Save config**.

## Forward `LMNR_` and `LLM_` Environment Variables

In OHE, environment variables whose names start with `LMNR_` or `LLM_` are automatically forwarded to the SDK runtime. This lets you configure Laminar ingestion settings and the LLM settings used for Laminar-backed workflows from the Admin Console.

For example, you can point the runtime at the managed Laminar endpoint and use an ingest-only project key:

```yaml
LMNR_BASE_URL: "https://laminar-api.<your-base-domain>"
# Ingest-only API key, not a read-capable secret:
LMNR_PROJECT_API_KEY: ""
LMNR_FORCE_HTTP: "true"
```

You can also control which LLM Laminar uses for its AI features — chat-with-trace, SQL-with-AI, and [signals](https://laminar.sh/docs/signals/introduction) — by forwarding the standard `LLM_*` variables:

```yaml
LLM_PROVIDER: "openai"
LLM_BASE_URL: "https://llm-proxy.<your-base-domain>"
LLM_MODEL_SMALL: "gpt-5.4-mini"
LLM_MODEL_MEDIUM: "gpt-5.4-mini"
LLM_MODEL_LARGE: "gpt-5.5"
```

`LLM_PROVIDER` accepts `gemini` (Laminar's default), `openai`, or `bedrock`, and `LLM_MODEL_SMALL` / `LLM_MODEL_MEDIUM` / `LLM_MODEL_LARGE` are optional per-tier model overrides. Set `LLM_PROVIDER` to `openai` whenever you point `LLM_BASE_URL` at an OpenAI-compatible gateway (for example LiteLLM, OpenRouter, or vLLM), not just the public OpenAI API.

Check warning on line 106 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L106

Did you really mean 'Laminar's'?

For the full set of supported values, see Laminar's official [self-hosting configuration reference](https://laminar.sh/docs/self-hosting/configuration).

Check warning on line 108 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L108

Did you really mean 'Laminar's'?

## Deploy Updated Configuration

Deploy the config change after setting the Laminar Project API Key in the Admin Console.
Deploy the configuration change after setting the Laminar Project API Key in the Admin Console.

![Laminar Deploy Again](./images/laminar-deploy-again.png)

Wait for the deployment to complete.

## Start a conversation
## Start a Conversation

Navigate to the OpenHands UI at `https://app.<your-base-domain>`. Start a new conversation and try a prompt.

![Start a Conversation](./images/laminar-openhands-conversation.png)

Your conversations will now automatically send a trace to Laminar.
Your conversations will now automatically send traces to Laminar.

![Laminar Trace](./images/laminar-trace.png)

Expand All @@ -107,16 +131,25 @@
src="https://github.com/user-attachments/assets/0cdf1625-3246-4388-a989-765f00d33ffb"
></video>

## What to Do Next in Laminar

Once traces are flowing, use Laminar's official docs to go deeper:

Check warning on line 136 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L136

Did you really mean 'Laminar's'?

- [Viewing Traces](https://laminar.sh/docs/platform/viewing-traces) to inspect a single conversation in transcript, tree, or timeline views.
- [Signals](https://laminar.sh/docs/signals/introduction) to extract structured outcomes or failure modes across many traces.
- [Session replay for browser agents](https://laminar.sh/docs/tracing/browser-agent-observability) to debug browser-based automations.

Check warning on line 140 in enterprise/analytics.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

enterprise/analytics.mdx#L140

Did you really mean 'automations'?
- [Observability for OpenHands Software Agent SDK](https://laminar.sh/docs/tracing/integrations/openhands-sdk) for the OpenHands-specific tracing model.

## Next Steps

<CardGroup cols={2}>
<Card title="Observability & Tracing" icon="activity" href="/sdk/guides/observability">
Learn the full OpenHands tracing model, OTEL configuration options, and non-Laminar backends.
</Card>
<Card title="Prompting Best Practices" icon="lightbulb" href="/openhands/usage/tips/prompting-best-practices">
Get the most out of your AI coding agents with effective prompting techniques.
Get more reliable traces by improving the prompts you give your agents.
</Card>
<Card title="Contact Support" icon="headset" href="https://openhands.dev/contact">
Reach out to the OpenHands team for deployment assistance or questions.
</Card>
<Card title="OpenHands Documentation" icon="book" href="/overview/introduction">
Explore the full OpenHands documentation for usage guides and features.
</Card>
</CardGroup>
Loading
Loading