> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flow.engineer/llms.txt
> Use this file to discover all available pages before exploring further.

# Observe conversations

> Inspect replies, grades and failures across your agent's conversations.

Flow Observe shows you what your agent says to people, and which replies went wrong. It works on any agent, built with any framework or none.

* **Every conversation**, message by message, with how long each reply took.
* **Every reply graded.** Exact rules run first, then a grading model. Each reply gets a verdict: `good`, `wrong`, `unhelpful` or `unsafe`.
* **Failures grouped into issues** by a fixed rule (the check that failed, the verdict and the topic), ranked by how many conversations they touch. Each issue links to the replies that show it and comes with a suggested fix you can copy into your prompt or code.
* **The inside of each turn**, from your agent's traces: model calls (model, tokens), tool calls, retrieval, errors and retries, and how long each step took.
* **Stats and usage**: conversations, replies, reply time (median and 95th percentile), and the conversation count you are billed on.

> [!NOTE]
>
> Flow Observe is in **early access**. You add your agent and make its keys yourself on Observe's Setup page. Your account takes **test data only** until the Flow team enables it for production: send synthetic conversations, never real people's messages, until then.

## Two ways in

Pick the one that matches how your agent talks to people.

| | On Flow Messaging | On any other channel |
| - | - | - |
| Your agent talks through | [Flow Messaging](https://docs.flow.engineer/unified-messaging-api/quickstart.md) (Telegram, WhatsApp, iMessage, SMS) | Your own SMS provider, web chat, WhatsApp integration or anything else |
| What you add | Nothing. Optionally one tag on your traces | One OpenTelemetry trace per turn |
| Where conversations come from | Flow Messaging's event log | Your traces |
| The text graded | What Flow Messaging delivered and sent | What you report in `flow.user_message` and `flow.agent_reply` |
| The inside of each turn | When you also send traces | Yes, from the same traces |
| Test chats | [Simulated contacts](https://docs.flow.engineer/unified-messaging-api/concepts/test-and-live-mode.md#simulated-contacts-test-without-a-phone) today | `flow test` (early access, see [Test before you ship](https://docs.flow.engineer/unified-messaging-api/trust.md)) |

> **[On Flow Messaging](https://docs.flow.engineer/unified-messaging-api/observe/flow-messaging.md)**
>
> No code. Observe reads your app's conversations.

> **[On any other channel](https://docs.flow.engineer/unified-messaging-api/observe/traces.md)**
>
> Send one trace per turn. Python and TypeScript, with or without a framework.

## Works alongside your other tools

Observe takes standard OpenTelemetry over OTLP/HTTP. If you already send traces somewhere, add Observe as a second exporter: the same spans go to both, and nothing in your code is specific to Flow except the `flow.*` attributes. You can try Observe next to what you use today, and stop at any time by removing the exporter.

```python Python theme={null}
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(  # next to your existing processor
    endpoint="https://flow.engineer/observe/v1/traces",
    headers={"Authorization": f"Bearer {os.environ['FLOW_OBSERVE_TRACE_KEY']}"})))
```

```ts TypeScript theme={null}
spanProcessors: [
  new BatchSpanProcessor(yourExistingExporter),
  new BatchSpanProcessor(new OTLPTraceExporter({ url: "https://flow.engineer/observe/v1/traces",
    headers: { Authorization: `Bearer ${process.env.FLOW_OBSERVE_TRACE_KEY}` } })),
],
```

The TypeScript lines go in your provider's constructor (OpenTelemetry JS 2.x). The imports and the rest of the setup are on [Send traces from any agent](https://docs.flow.engineer/unified-messaging-api/observe/traces.md#1-set-up-the-exporter).

## Get access

**1. Sign in with your Flow login**

Open [flow.engineer/observe](https://flow.engineer/observe) and sign in with the same Flow login you use on flow\.engineer. Observe is in early access: if it isn't on for you yet, the page says so; ask the Flow team ([samyak@flow.engineer](mailto:samyak@flow.engineer) or [prabal@flow.engineer](mailto:prabal@flow.engineer)) to turn it on.

**2. Add your agent**

In the dashboard, add your agent. On Flow Messaging, choose Flow Messaging and paste a test key of your app (`fk_test_...`; a key made just for Observe is best). On any other channel, choose traces.

**3. Make your keys**

For traces, make a **trace key** (`otk_test_...`); for `flow test` in CI, a **CI key** (`oci_...`). Each is shown once. Keep a trace key in `FLOW_OBSERVE_TRACE_KEY` on your server, never in a browser or mobile app, and never in a chat with an AI assistant. The dashboard shows your test conversations, labelled as test data.

## Test data and production

* Keys come in two modes, as in Flow Messaging: `otk_test_...` and `otk_live_...` for traces, `fk_test_...` and `fk_live_...` for Flow Messaging apps. Data never crosses between modes.
* Until the Flow team enables your account for production, Observe accepts test keys and test data only. A live key is refused with `403`.
* Test conversations are never billed.

## How your data is kept

* Message text and the content of your traces (prompts, tool inputs and results) are encrypted at rest with a key per account.
* Observe deletes message text, including the text inside traces, 30 days after the message was sent (or after the period set for your account), and it is gone from Observe's database backups within 7 more days. Conversations, traces, verdicts, issues and counts keep no message text, only ids, times, labels and numbers, so they stay. Flow Messaging keeps its own copy of your messages under its own retention.
* Every time a person opens a conversation's text in the dashboard, it is logged.
* On other channels, the ids you send (conversation, contact, message) are stored only as keyed hashes, so a phone number used as an id is never stored as a phone number. On Flow Messaging, Observe keeps Flow's own ids (`conv_...`, `msg_...`, `ct_...`) and never the person's phone number or handle.
* The dashboard shows span details (names, models, tokens, timings, errors), not prompts or tool inputs.
* Dashboard pages load no analytics and no third-party scripts. Sign-in is with your Flow login, limited to the people on your account's list.

## Related

* [Send traces from any agent](https://docs.flow.engineer/unified-messaging-api/observe/traces.md)
* [Traces from agent frameworks](https://docs.flow.engineer/unified-messaging-api/observe/frameworks.md)
* [Test before you ship](https://docs.flow.engineer/unified-messaging-api/trust.md)
* [What changes when real people text your bot](https://docs.flow.engineer/unified-messaging-api/guides/real-people.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.