/

Ambitwin · source-available, not open source · © 2026 Nicolò Ghignatti

Docker Compose

Keywords on this page
WordMeaningExample
Compose fileA YAML list of containers, their images, ports and volumesbackend/telemetry/docker-compose.yml
FragmentOne service’s own compose filebackend/chat/docker-compose.yml
Dev overrideA second file that changes a fragment for live editingdocker-compose.dev.yml
include:A Compose key that loads other compose files, each from its own foldercompose.runtime.yml
ProjectThe group of containers Compose starts together, under one nameambitwin-integration-tests
VolumeStorage that outlives the containertelemetry-data
WatchCompose copies changed files into a running container, or rebuilds itdevelop.watch
EdgeThe one entry point every browser request goes throughCaddy on port 80
forward_authCaddy asks another service “may this request pass?” before sending it onclaims-gateway /verify

Locally the whole platform runs under Docker Compose. No single file lists every service: each service keeps its own compose file beside its code, and a small script joins them into one on every start.


The files

FileHoldsExample
docker-compose.yml (root)Shared parts only: gateway (Caddy), redis, kafka, keycloak, keycloak-db, prometheus, grafana; includes docker/rust-base—
backend/<svc>/docker-compose.ymlThe service and its own databasetelemetry, telemetry-db
backend/<svc>/docker-compose.dev.ymlChanges for just stack dev: target: dev, watch rules, published database ports, database browserstelemetry-db-gui
backend/<svc>/docker-compose.test.ymlA throwaway stack for that service’s integration teststelemetry-test
backend/acceptance/docker-compose.*.ymlThe cross-service acceptance stackintegration-tests
docker/rust-base/docker-compose.ymlThe Rust build image, scale: 0rust-base
simulators/*/, devices/ap-collector/, frontend/Same base + dev pairclient

just stack dev, step by step

sequenceDiagram
    autonumber
    participant U as Developer
    participant J as just
    participant E as "scripts/env, one script per area"
    participant P as compose-run.py
    participant D as docker compose
    U->>J: just stack dev --no-agent
    J->>E: env recipe, 11 scripts in order
    E->>E: append missing secrets to the env file, rewrite RUST_VERSION
    J->>P: python3 scripts/compose/compose-run.py dev --no-agent
    P->>P: argparse, expand --no-agent to pattern agent
    P->>P: add docker-compose.yml
    P->>P: walk backend/, simulators/, devices/, skip excluded folders
    P->>P: add frontend
    P->>P: write compose.runtime.yml
    P->>D: config --format json
    P->>P: find stopped containers with a fixed name, docker rm -f them
    P->>D: config --services, drop names matching agent
    P->>D: up --watch --remove-orphans, with the kept service names
    D->>D: build images, start containers in depends_on order, watch files
StepWhat happensReal value
2create, vapid, token, config, langfuse, keycloak, controlplane, gateway-key, ingest-key, telemetry-db, rustEach one skips a key already in .env
3Only rust.sh rewrites a value every timeRUST_VERSION= the pin in .mise.toml
5--no-agent becomes the substring agentEXCLUDE_GROUPS["agent"] = ("agent",)
7Each folder adds docker-compose.yml, and in dev also docker-compose.dev.ymlbackend/agent skipped: printed excluded: backend/agent
9One include: entry per folder; a pair becomes one entry with two paths16 entries: root, 10 backend, 3 simulators, ap-collector, frontend
11A stopped container with a container_name would block up with a name clashambitwin-socket
12The root file is always included, so its services can only be dropped by name—
13COMPOSE_BAKE=true: builds go through BuildKit Bake, in parallel—

The generated file looks like this:

# Auto-generated by scripts/compose/compose-run.py — do not commit.
include:
  - path: docker-compose.yml
  - path:
      - backend/chat/docker-compose.yml
      - backend/chat/docker-compose.dev.yml
  - path:
      - backend/claims-gateway/docker-compose.yml
      - backend/claims-gateway/docker-compose.dev.yml

compose.runtime.yml is in .gitignore.

Why include: and not several -f flags

With -f a.yml -f b.yml, Compose resolves every relative path against the first file’s folder. build.context: ../.. in backend/telemetry/docker-compose.yml would then point two levels above the repo root. include: resolves each file against its own folder.

A pair of files is one entry because two separate entries may not define the same service: Compose stops with a conflict.

Leaving parts out

flowchart TD
    A["argument"] --> F{"--no-group flag?"}
    F -- yes --> G["its patterns from EXCLUDE_GROUPS"]
    F -- no --> S["the argument itself, as a substring"]
    G --> M{"mode is down or build?"}
    S --> M
    M -- yes --> N["ignored: everything is included"]
    M -- no --> C["folder name contains it: whole fragment left out"]
    C --> V["service name contains it: left out of the up list"]
FlagPatternsLeaves out
--no-agentagentbackend/agent, all 9 services
--no-metricsprometheus, grafanaTwo root services, by name
--no-simulatorssimulatorThe three simulator folders
--no-devicesap-collectordevices/ap-collector
bare worditselfAnything whose folder or service name contains it

Example. just stack dev chat:

  1. Folder backend/chat contains chat: its two files are not included.
  2. No other service name contains chat, so nothing else goes.

Example of a trap. just stack dev db drops every service with db in its name: twin-db, telemetry-db, keycloak-db… and the stack cannot start.

Modes

RecipeModeCommand
just stack devdevup --watch --remove-orphans
just stack dev --builddevthe same, plus --build
just stack startstartup --build -d --remove-orphans, production stages, in the background
just stack downdowndown --remove-orphans, always everything
just stack logs <svc>—docker compose -f compose.runtime.yml logs -f <svc>
--dry-runanyPrints the docker command and stops after writing the file

The edge: Caddy

gateway runs caddy:2.11.7 on ${BACKEND_PORT} (80), with the repo-root Caddyfile mounted read-only.

PathGoes toGatedPrefix removed
/gateway/*claims-gateway:3000noyes
/tenancy/*, /twin/*, /notification/*, /chat/*, /dashboard, /dashboard/*, /telemetry/*the serviceyesyes
/socket.io/*socket:3000yesno: Socket.IO expects its own path
/agent/*agent:3000no, and X-Gateway-Claims is removedyes
/telemetry/ingest, /telemetry/collector, /telemetry/observationstelemetry:3000no: the body is HMAC-signed and checked inside telemetry/telemetry only
/prometheus/*, /grafana/*the toolno/prometheus only
/<x>-db-gui/*the database browser of each Mongo service and telemetrynono
/langfuse*redirect to port 3030——

/chat/* adds flush_interval -1: answers stream as SSE, and buffering would deliver every token at the end.

A gated request

sequenceDiagram
    autonumber
    participant B as Browser
    participant C as Caddy
    participant G as claims-gateway
    participant T as digital-twin
    B->>C: GET /twin/buildings, cookie authentication_token
    C->>G: GET /verify, same headers
    G->>G: RequireAuthentication: extractToken
    G->>G: jwt.ParseWithClaims, RS256, issuer cv-gateway
    alt token missing or bad
        G-->>C: 401
        C-->>B: 401, the service is never called
    else valid
        G->>G: verifyHandler: json.Marshal(claims), base64
        G-->>C: 200, X-Gateway-Claims
        C->>C: copy_headers X-Gateway-Claims, strip /twin
        C->>T: GET /buildings, x-gateway-claims
        T->>T: decode the header, run Cedar
        T-->>B: 200
    end
StepData
1Cookie: authentication_token=eyJhbGciOiJSUzI1NiIs…
3extractToken reads Authorization: Bearer … first, then the cookie
4Signature checked against claims-gateway’s own public key; only RS256 accepted
8{"sub":"ada","memberships":[…],"exp":…} as standard base64
10Path /twin/buildings becomes /buildings

Why the edge checks once: the services behind it never verify a signature; they read the header. The cluster does the same with Istio, so a service runs unchanged in both places: Kubernetes & Istio.

gateway lists five services in depends_on with required: false: it starts even when they are left out, and finds them when they appear, because Caddy looks up each name per request.


.env: what just stack env writes

Every dev and start runs env first. Each script appends only what is missing, so a second run changes nothing except RUST_VERSION.

ScriptWrites
create.shAn empty .env, mode 600
vapid.shVAPID_PUBLIC_KEY, VAPID_PRIVATE_KEY, VITE_VAPID_PUBLIC_KEY, a P-256 key pair; checks the lengths 87 and 43
token.shEVAL_JWT_SECRET, 32 random bytes, local evals only
config.shBACKEND_PORT=80, FRONTEND_PORT=8080, DEV_URL=http://localhost; asks for OPENROUTER_API_KEY
langfuse.shLangfuse keys, salt, passwords
keycloak.shKEYCLOAK_ADMIN_PASSWORD, KEYCLOAK_DB_PASSWORD, empty Google client fields
controlplane.shINTERNAL_SIGNING_SECRET, TENANCY_DB_PASSWORD, REGISTRY_DB_PASSWORD
gateway-key.shsecrets/gateway-dev-key.pem, a 2048-bit RSA key, only if missing
ingest-key.shTELEMETRY_INGEST_SECRET, TELEMETRY_DEVICE_MASTER_KEY
telemetry-db.shTELEMETRY_DB_PASSWORD
rust.shRUST_VERSION, rewritten every run so a bump in .mise.toml reaches the build

The RSA key is kept on disk, not regenerated: a new key would make every signed-in cookie invalid.


Live reload

LanguageWatch ruleOn a source changeOn a dependency change
Rustsync ./srccargo watch -x run rebuilds the service crate onlyCargo.toml, Cargo.lock: rebuild the image
Gosync ./air rebuilds and restartsgo.mod: rebuild
agentsync ./appuvicorn --reloadpyproject.toml: rebuild
frontendsync ./srcVite dev server, npm run devpackage.json: rebuild

Published host ports

Only the dev overrides publish database ports. Two services on one host port stop the whole up.

PortService
80gateway
8080client (frontend)
5432, 5433, 5434, 5435telemetry-db, tenancy-db, registry-db, agent-db
27017, 27019, 27020, 27021twin-db, notification-db, dashboard-db, chat-db
6379, 9092, 8090redis, kafka, keycloak
3001, 3002, 3003sensor-simulator, aq-simulator, ap-simulator
3030langfuse-web

Test stacks

One service: just test <svc>-integration

sequenceDiagram
    autonumber
    participant J as just
    participant S as test-integration.sh
    participant D as docker compose
    participant T as telemetry-test
    J->>S: bash backend/telemetry/test-integration.sh
    S->>S: RUST_VERSION from the mise pin
    S->>S: trap down --volumes on EXIT
    S->>D: -p telemetry-integration-tests up --build --exit-code-from telemetry-test
    D->>D: start telemetry-db, telemetry-redis, telemetry-kafka
    D->>T: target dev, run scripts/test/rust-integration-tests.sh
    T-->>D: exit code
    D-->>S: same exit code
    S->>D: down --remove-orphans --volumes
ServiceProjectInfra
chatchat-integration-testsMongoDB
digital-twindigital-twin-integration-testsMongoDB
notificationnotification-integration-testsMongoDB, Redis, Kafka
socketsocket-integration-testsRedis
telemetrytelemetry-integration-testsTimescaleDB, Redis, Kafka
dashboarddashboard-integration-testsMongoDB, Redis

Acceptance: just test integration

backend/acceptance/run-integration-tests.sh writes compose.integration-tests.runtime.yml:

Included fileGives
docker/rust-base/docker-compose.ymlThe Rust build image
backend/acceptance/docker-compose.infra.ymlredis, kafka
backend/{telemetry,socket,dashboard}/docker-compose.ymlThe three real services
backend/acceptance/docker-compose.integration.ymlStubs for digital-twin and claims-gateway (nginx), the real Caddy gateway, and the integration-tests runner

Traps

Related: Docker Images, Kubernetes & Istio, Build System.