> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flow.engineer/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Flow Messaging is one HTTP API (https://api.flow.engineer) for AI agents to hold two-way conversations on Telegram and iMessage (replies only, on lines the Flow team connects), and on WhatsApp numbers the Flow team connects. If the user asks for WhatsApp, tell them WhatsApp numbers are arranged with the Flow team, and build and test on Telegram meanwhile: the code does not change per channel.
> Build the app on the REST API. MCP is not a runtime dependency: the app you build talks to Flow through the REST API (or the TypeScript or Python SDK, which wrap it) for everything it does at runtime. In TypeScript or JavaScript, the SDK is `npm install @flow-engineer/messaging`, then `new FlowMessaging()` (reads FLOW_MESSAGING_KEY). In Python, the SDK is `pip install flow-messaging`, then `from flow_messaging import Flow` and `Flow()` (reads FLOW_MESSAGING_KEY). The Go SDK is not published yet; in Go call the HTTP API directly.
> No key yet? If FLOW_MESSAGING_KEY is not set, get a test key with no account in one call: curl -X POST https://api.flow.engineer/v1/sandbox/keys (or npx @flow-engineer/messaging init, which writes it to .env). Save `key` as FLOW_MESSAGING_KEY and `claim_token` as FLOW_CLAIM_TOKEN; both are shown once. It allows 1 contact and 50 messages on the Telegram sandbox and expires after 7 days; a person signs in with `npx @flow-engineer/messaging login` (with a Flow account (flow.engineer: Google, GitHub or an email link)) to keep the app and get 3 contacts x 100 messages. Signed in, live keys (fk_live_...) are made in the dashboard for your own Telegram bot; iMessage lines are arranged with the Flow team.
> The hosted MCP server (https://api.flow.engineer/mcp) is optional, for testing during development. Adding the Flow MCP server to a developer's coding tools is the project owner's decision: suggest it and show them the command, but never add it to their config yourself. The command for Claude Code: claude mcp add --transport http flow https://api.flow.engineer/mcp --header "Authorization: Bearer $FLOW_MESSAGING_KEY" (run it with FLOW_MESSAGING_KEY set, then restart the client). If its tools are already available to you, use them for sandbox testing.
> Authenticate with `Authorization: Bearer fk_test_...` (sandbox) or `fk_live_...` (real contacts). Never put a live key in client-side code.
> Reply into a conversation (POST /v1/conversations/{conversation_id}/messages); never pick a channel per message. Switch on `error.type`, not the message text.
> Flow Observe (https://flow.engineer/observe, early access) records and grades every conversation of a chat agent. On Flow Messaging it needs no code. On any other channel, send one OpenTelemetry trace per turn over OTLP/HTTP to https://flow.engineer/observe/v1/traces with `Authorization: Bearer $FLOW_OBSERVE_TRACE_KEY`, and set `flow.conversation_id` (required), `flow.message_id`, `flow.user_message` and `flow.agent_reply` (the text actually sent) on the turn's span. Keys (trace key `otk_test_...`, CI key `oci_...`) are made by the user on Observe's Setup page at https://flow.engineer/observe after signing in with their Flow login; Observe is in early access, and if it is not on for them, they ask the Flow team (samyak@flow.engineer or prabal@flow.engineer). Never invent a key, and never ask the user to paste one into the chat. Test data only until the account is enabled for production. Guide: https://docs.flow.engineer/observe/traces.md
> Full documentation for agents: https://docs.flow.engineer/llms-full.txt. OpenAPI spec: https://raw.githubusercontent.com/flow-engineer/sdk/main/openapi/openapi.yaml.

# Test before you ship: test chats and a check in your CI

> Run test chats against your agent in CI and fail the build when a reply breaks a rule or misses what the test expects. Works with any agent on any framework, with flow test. Plus graded test conversations and issues from failures.

A change to a prompt, a model or a tool can fix one reply and break ten others. People rarely tell you; they stop writing. The goal of this step is that a version that makes your agent worse never reaches them.

## A check in your CI: `flow test`

<Note>
  Early access. Make your agent's **CI key** in the Observe dashboard ([Get access](/observe#get-access)); if Observe isn't on for you yet, write to the Flow team at [samyak@flow.engineer](mailto:samyak@flow.engineer) or [prabal@flow.engineer](mailto:prabal@flow.engineer). The `flow` binary is a public download (below).
</Note>

`flow test` runs your test chats against **your own agent**, sends the transcripts to Observe, and fails the build when a reply breaks a rule or misses what the test expects. Observe grades every reply and decides pass or fail by a fixed rule, so the same transcript always gets the same answer. Your agent can use any framework, or none: `flow test` only needs an HTTP endpoint or a command that answers one message.

```
your CI ──> flow test ──(each test message)──> your agent
                 │
                 └──(transcripts + expectations)──> Observe ──> pass / fail
```

### 1. Write test chats

Put them in `.flow/tests.yaml` in your repo. Each case is one conversation: the person's turns, and for each reply what it must and must not do.

```yaml theme={null}
version: 1
channel: sms
cases:
  - id: no-delivery-promise
    name: Doesn't promise a date it can't confirm
    turns:
      - user: "Can you get my order to me by Sunday?"
        expect: ["Doesn't promise a delivery date it can't confirm, and offers to check or to connect a person."]
        must_not: ["guaranteed"]
      - user: "ok thanks"
  - id: cancel
    turns:
      - user: "I want to cancel my order"
        includes: ["cancel"]
```

* `expect`: plain sentences a grader checks.
* `includes` and `must_not`: exact text, case-insensitive.
* `rubrics`: rule ids from your check set that must pass.

Every list is optional. A turn with none is still graded against your checks.

Test chats are test data you write. Never put a real person's conversation in a test file.

### 2. Let `flow test` reach your agent

Start your agent in the CI job, and give `flow test` one of two ways to send it a message:

* **`--http URL`**: `flow test` posts each turn as JSON:

  ```json theme={null}
  {"conversation_id": "ft_k3m9q2x7ab_cancel", "contact_id": "ft_k3m9q2x7ab_cancel",
   "channel": "sms", "message": {"text": "I want to cancel my order"},
   "test": {"case": "cancel", "turn": 0}}
  ```

  Your agent answers with `{"replies": [{"text": "..."}]}`, `{"reply": "..."}`, or a plain-text body. Keep the conversation's history by `conversation_id`, as you would for a real person.
* **`--cmd "..."`**: `flow test` runs your command once per turn, with the same JSON on its stdin, and reads the answer from its stdout.

Optionally, send your agent's [traces](/observe/traces) during the test too: in CI, give your agent the **CI key** as its trace key, and set `flow.conversation_id` to the `conversation_id` it was given. Each test chat then shows the inside of every turn in the dashboard, so you can see why a reply failed. With a CI key, Observe takes traces only for that agent's test chats; they never reach its real conversations.

### 3. Run it in CI

```yaml GitHub Actions theme={null}
- name: Get flow
  run: |
    V=v0.1.0
    base="https://github.com/flow-engineer/sdk/releases/download/flow-cli-$V"
    curl -fsSLO "$base/flow_${V}_linux_amd64"
    curl -fsSLO "$base/SHA256SUMS"
    grep " flow_${V}_linux_amd64\$" SHA256SUMS | sha256sum -c -
    install -m 755 "flow_${V}_linux_amd64" ./flow
- name: Start the agent
  env:
    FLOW_OBSERVE_TRACE_KEY: ${{ secrets.FLOW_CI_KEY }}   # optional: traces during the test
  run: |
    python3 my_agent_server.py &
    for i in $(seq 1 50); do curl -s -o /dev/null http://127.0.0.1:3000 && break; sleep 0.2; done
- name: flow test
  env:
    FLOW_CI_KEY: ${{ secrets.FLOW_CI_KEY }}
  run: ./flow test --http http://127.0.0.1:3000 --version "$GITHUB_SHA" --junit flow-test.xml
```

`flow` is published for Linux and macOS, on x86-64 and arm64, on the [SDK's releases page](https://github.com/flow-engineer/sdk/releases), with a `SHA256SUMS` file to check it against. Any other CI works the same way, as shell steps.

`flow test` prints a line per case, and for a failure, which expectation it missed and which rules failed:

```
FAIL   no-delivery-promise  Doesn't promise a date it can't confirm
       turn 1, reply 1 (verdict wrong):
         expect "Doesn't promise a delivery date it can't confirm, ..." failed
PASS   cancel
1 passed, 1 failed, 0 error: FAILED
```

It exits `0` when every case passes, `1` when one fails, and `2` when it couldn't run (your agent didn't answer, or Observe couldn't be reached). `--junit` writes a report your CI can show.

* **The key.** Use your agent's **CI key** (`oci_...`), kept in your CI's secret store as `FLOW_CI_KEY`. It works whether your agent is in test or live mode, and it can only run tests and send traces for them: it can't read or add to your agent's real conversations. Trace keys are refused. `flow test` never prints the key.
* **Limits.**
  * Up to 50 cases per run, 20 turns per case and 2,000 characters per message. Replies can be up to 8,000 characters.
  * A few runs per agent at once, and a daily budget of graded test replies.
  * When Observe is busy, `flow test` waits and retries.

Test runs aren't billed. They show in the dashboard labelled "Test run", kept apart from your production stats.

## Also today

**Every test conversation is graded.** Observe grades every reply your agent sends, test conversations included, with the same checks as production.

* **On Flow Messaging:** [simulated contacts](/concepts/test-and-live-mode#simulated-contacts-test-without-a-phone) play chats through the real send gate. Each reply's verdict shows in the [Observe dashboard](https://flow.engineer/observe).

  ```ts TypeScript theme={null}
  const conv = await flow.test.createConversation({ channel: "sms" });
  await flow.test.inbound(conv.id, { type: "text", text: "Can I change my delivery to Sunday?" });
  // your agent answers as usual; then:
  for await (const m of flow.test.transcript(conv.id)) console.log(m.direction, m.shown ?? m.content);
  ```
* **Failures become issues.** When replies fail the same check, Observe groups them into one issue under **Needs attention**, with the replies as evidence and a suggested fix. For each issue it also proposes a test case: the conversation up to the failing reply, and what the reply must and must not do. Copy it into `.flow/tests.yaml`.

## Coming

<Note>
  These are being built with the first early-access teams. Their shape may change before they open.
</Note>

* **Test cases in the dashboard**: write them there, or save one from a failing conversation in one click, and `flow test` runs them with your file.
* **`flow test` on Flow Messaging**: cases played through simulated contacts instead of an HTTP endpoint.

## Related

* [What Observe gives you](/observe)
* [Send traces from any agent](/observe/traces)
* [Simulated contacts](/concepts/test-and-live-mode#simulated-contacts-test-without-a-phone)
* [What changes when real people text your bot](/guides/real-people)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.