Chat Service

The chat is the conversation layer in front of the agent. The Agent Service is intentionally stateless — each POST /ask is an independent call that takes an optional history array but stores nothing. The chat is what turns that into a real chat product: it persists conversations, owns them per user, and feeds the recent history back into each agent call.

It is a Rust / Axum service over MongoDB, laid out as Ports & Adapters like the other Rust backend services. Keeping conversation storage out of the Python agent is deliberate: the agent stays a pure question-answering engine, while session persistence — a different concern with a different database and scaling profile — lives in its own service.

Source Layout

graph TD
    HTTP["adapters/driving/http_api\nclaims · controllers · SSE frames"] --> UC["service/conversations.rs\nownership, cap, exchange"]
    UC --> DOM["domain/\nvalidation, history, titling"]
    UC --> PORTS["service/ports.rs\nConversationStore · AgentClient"]
    PORTS -.-> MONGO["adapters/driven/persistence"]
    PORTS -.-> AG["adapters/driven/agent.rs"]
    MONGO --> DB[("chat-db")]
    AG -->|"x-gateway-claims forwarded"| AGENT["agent /ask"]

Data Model

A conversation is a single MongoDB document with its messages embedded rather than stored in a separate collection. A chat is loaded and saved as a whole, and message counts are bounded, so embedding avoids a join and makes each individual MongoDB save atomic.

Sending a message is not atomic end-to-end: it loads the conversation, waits for an external agent request, then appends the exchange. Concurrent sends can therefore use stale history. The append itself is a single $push + $set update rather than a read-modify-write, so two concurrent sends cannot lose each other’s messages, but the client still serialises sends from one widget and stronger server-side concurrency control remains future work.

pub struct Conversation {
    pub id: String,           // bare hex, never {"$oid": ...}
    pub user_id: String,      // owner; indexed { userId: 1 }
    pub title: String,        // defaults to "New chat"; auto-set from the first question
    pub messages: Vec<ChatMessage>, // embedded, capped at MAX_MESSAGES
    pub created_at: String,
    pub updated_at: String,   // the conversation list sorts on this
}

pub struct ChatMessage {
    pub id: String,           // every embedded message carries its own id
    pub role: Role,           // user | assistant
    pub content: String,      // 1..MAX_MESSAGE_LENGTH chars
    pub citations: Option<Vec<Citation>>, // grounding sources, from the agent's answer
    pub created_at: String,
}

Citation keeps snake_case field names (chunk_id, document_id, source, section_path): they are agent’s Python payload, stored and returned unchanged.

Citations are persisted on the assistant message so that reopening an old conversation shows the same grounded sources the agent originally returned, without re-querying.

The REST shapes are pinned in schemas/fixtures/chat-conversation.json, with the written rules in schemas/json/chat-conversation.schema.json (checked by claims-schema):

• chat’s domain/conversation.rs tests round-trip its types against the fixture.
• The frontend reads every reply only through src/utils/chat.ts, which throws on a shape it cannot read instead of showing an empty chat list.

HTTP API

Conversation routes require authentication. GET /health and GET /metrics are public operational endpoints. The service is exposed through Caddy at http://localhost/chat (and the same /chat ingress prefix in Kubernetes).

Every conversation route resolves the document by { _id, userId }, so one user can never read or mutate another user’s chat — an unmatched id returns 404 rather than 403, to avoid revealing that the conversation exists.

Sending A Message

POST /conversations/:id/messages is where the chat orchestrates the agent. It loads the conversation, takes the last HISTORY_MAX_MESSAGES turns as history, opens a stream to the agent, relays each token to the browser as it arrives, and persists the exchange once the stream completes.

The response is Server-Sent Events, not JSON:

sequenceDiagram
    autonumber
    participant C as Client (ChatWidget)
    participant CH as chat
    participant DB as chat-db
    participant A as agent
    C->>CH: POST /conversations/:id/messages { content }
    CH->>DB: load conversation by { _id, userId }
    Note over CH: validate, check the cap, take last HISTORY_MAX_MESSAGES turns
    CH->>A: POST /ask { question, history, stream:true } (+ x-gateway-claims header)
    A-->>CH: SSE: token …
    CH-->>C: SSE: token … (relayed, and accumulated in memory)
    A-->>CH: SSE: done { citations }
    CH->>DB: append user + assistant messages, bump updatedAt
    CH-->>C: SSE: done { message }

Design points worth calling out:

• Time-to-first-token, not total time. Streaming does not make generation faster; it makes the wait visible. The user sees words within milliseconds instead of a spinner for the whole generation. This is by far the largest perceived-latency win available to the chat path.

• Errors split by whether the response has begun. Validation, ownership, the message cap and an unreachable agent are all knowable before the first token, so they remain ordinary status codes (400, 404, 409, 502). After the stream opens the status line is already sent, so a failure arrives as a terminal error frame on a 200 instead.

• Nothing is persisted until the terminal frame. Tokens are accumulated in memory and the exchange is written once, atomically, when the agent signals completion. A dropped connection or a stream that stops early therefore leaves the conversation exactly as it was — no truncated answer, no half-applied title. A stream that ends without its done frame is reported as an invalid agent response.

• The edge must not buffer. A proxy that buffers the response defeats the whole change: Caddy needs flush_interval -1 on /chat/*.

• The caller’s x-gateway-claims header is forwarded to the agent verbatim, not a service account. This means the agent runs /ask under the user’s identity, so its permission-filtered retrieval only returns documents that user is allowed to see. The chat never widens access.

• History is truncated to the last HISTORY_MAX_MESSAGES turns before being sent. This bounds the prompt size (and therefore cost and latency) the agent has to process, and the agent already treats prior messages as untrusted context rather than instructions.

The first user message also seeds the conversation title (the prior default is "New chat"), so a freshly created chat gets a meaningful name without a separate request.

Configuration

MAX_MESSAGES (100), MAX_MESSAGE_LENGTH (8000), and MAX_TITLE_LENGTH (120) are fixed in-code limits that bound document growth and reject oversized input.

Authentication

A GatewayClaims extractor decodes the mesh-injected x-gateway-claims header — Istio’s RequestAuthentication already verified the gateway JWT once, at the ingress — and takes the sub claim as the owning userId. There is no signature verification here at all; chat trusts the header rather than re-checking a JWT itself. The raw header is kept and forwarded verbatim to the agent. A failure to reach the agent surfaces as a 502 Bad Gateway, distinguishing an upstream agent problem from a client error.

Client

In the frontend, the assistant is the floating ChatWidget.vue, driven by the useChatSessions composable, which calls these /chat/* routes. It appends the question and an empty assistant message optimistically, grows that message as token frames arrive, and replaces it with the saved one on done; a failure removes both again, so a broken stream never leaves a partial answer on screen. For what the assistant can and cannot answer from a user’s perspective.