Skip to main content
Flow Observe shows you what your agent says to people, and which replies went wrong. It works on any agent, built with any framework or none.
  • Every conversation, message by message, with how long each reply took.
  • Every reply graded. Exact rules run first, then a grading model. Each reply gets a verdict: good, wrong, unhelpful or unsafe.
  • Failures grouped into issues by a fixed rule (the check that failed, the verdict and the topic), ranked by how many conversations they touch. Each issue links to the replies that show it and comes with a suggested fix you can copy into your prompt or code.
  • The inside of each turn, from your agent’s traces: model calls (model, tokens), tool calls, retrieval, errors and retries, and how long each step took.
  • Stats and usage: conversations, replies, reply time (median and 95th percentile), and the conversation count you are billed on.
Flow Observe is in early access. The Flow team creates your account and keys. Your account takes test data only until the Flow team enables it for production: send synthetic conversations, never real people’s messages, until then.

​
Two ways in

Pick the one that matches how your agent talks to people.

On Flow Messaging

No code. Observe reads your app’s conversations.

On any other channel

Send one trace per turn. Python and TypeScript, with or without a framework.

​
Get access

1

Sign in with your Flow login

Open flow.engineer/observe and sign in with the same Flow login you use on flow.engineer. Observe is in early access: if it isn’t on for you yet, the page says so; ask the Flow team (samyak@flow.engineer or prabal@flow.engineer) to turn it on.
2

Add your agent

In the dashboard, add your agent. On Flow Messaging, choose Flow Messaging and paste a test key of your app (fk_test_...; a key made just for Observe is best). On any other channel, choose traces.
3

Make your keys

For traces, make a trace key (otk_test_...); for flow test in CI, a CI key (oci_...). Each is shown once. Keep a trace key in FLOW_OBSERVE_TRACE_KEY on your server, never in a browser or mobile app, and never in a chat with an AI assistant. The dashboard shows your test conversations, labelled as test data.

​
Test data and production

  • Keys come in two modes, as in Flow Messaging: otk_test_... and otk_live_... for traces, fk_test_... and fk_live_... for Flow Messaging apps. Data never crosses between modes.
  • Until the Flow team enables your account for production, Observe accepts test keys and test data only. A live key is refused with 403.
  • Test conversations are never billed.

​
How your data is kept

  • Message text and the content of your traces (prompts, tool inputs and results) are encrypted at rest with a key per account, and deleted when your account’s retention period ends.
  • Every time a person opens a conversation’s text in the dashboard, it is logged.
  • On other channels, the ids you send (conversation, contact, message) are stored only as keyed hashes, so a phone number used as an id is never stored as a phone number. On Flow Messaging, Observe keeps Flow’s own ids (conv_..., msg_..., ct_...) and never the person’s phone number or handle.
  • The dashboard shows span details (names, models, tokens, timings, errors), not prompts or tool inputs.
  • Dashboard pages load no analytics and no third-party scripts. Sign-in is with your Flow login, limited to the people on your account’s list.