Docker Compose Orchestration

Locally, the entire platform runs under Docker Compose. There is no single hand-maintained compose file that lists every service; instead, each service owns its own compose fragment, and a small Node orchestrator assembles them into one runtime file on demand. This page explains that mechanism end to end, so a new contributor can predict exactly what just stack dev does.

The shape of the configuration

Compose configuration is split across three kinds of file.

The root file holds the pieces every run needs regardless of which services are included:

• gateway — the Caddy reverse proxy on port 80 (see Architecture Overview. Its depends_on entries are marked required: false, so the gateway still starts when a service is excluded and simply retries the connection when that service appears.
• redis — the publish/subscribe broker.
• kafka — event log for building-registration events between digital-twin and telemetry (single-node KRaft, no Zookeeper). Shared infra like redis, even though only those two services use it today.
• keycloak and keycloak-db — the hosted identity provider every login flows through, and its private Postgres. keycloak imports keycloak/realm-export.json on boot and exposes its own admin console directly on host port 8090 (not behind the gateway) — see Identity & Tenancy Architecture.
• prometheus and grafana — metrics and dashboards.
• The langfuse-*, redis-langfuse, clickhouse, and minio stack — the agent’s trace backend (see Setting Up the Environment for the generated Langfuse credentials and UI URL).

The shared Rust base image

All six Rust services build FROM rust-base, a build-only image defined in docker/rust-base/. It carries the toolchain, mold, build-essential, and pinned prebuilt cargo-chef and cargo-watch binaries, and sets RUSTFLAGS.

• Each service resolves it through build.additional_contexts: rust-base: service:rust-base, so Compose builds the base before anything that needs it.
• scale: 0 — it is a toolchain, never a running container, and up must not start one.
• The tool tarballs are checksum-pinned. cargo install pins each tool’s own lockfile but not its version, so two developers got different cargo-watch builds.
• RUST_VERSION comes from the single pin in .mise.toml: scripts/env/rust.sh rewrites it into .env on every just stack run.
• The per-service integration runners and the acceptance runner export RUST_VERSION themselves — they compose standalone projects that never read the root .env.
• Outside Compose, cd-registry publishes it to GHCR and hands it to buildx as a named context — see CI/CD Pipeline.

A cold build of every image (docker builder prune -af first) went from 1576s to 794s. cargo install of the two tools accounted for 3746 of the 10430 step-seconds before; it is now zero, and the Rust base image is pulled once rather than six times.

sccache on the debug dependency layer

The debug cargo chef cook runs under sccache, with RUSTC_WRAPPER set inline on that one step so nothing reaches the release stages or the running dev container.

• It pays off only when Docker must re-run the cook but the store is warm — a Cargo.toml or schemas/ change. Measured on dashboard: 744s to 19.9s, 241 hits, 0 misses.
• A cold build is worse: the store starts empty, so every call pays rustc plus sccache’s own hashing. Cold went 794s to ~1262s. Do not read that number alone.
• Crates are not shared between services. All six cook at once, so no service’s entries exist when the others need them. The reuse is across rebuilds of one service.
• sharing=shared is safe here, unlike a cargo registry mount: entries are keyed by a hash of the compiler inputs and written atomically. A shared registry mount corrupts instead — cargo’s lock lives at CARGO_HOME root, outside the mount, so concurrent builds do not see it.

The Go services’ air

The four Go Dockerfiles open with an identical tools stage that downloads a checksum-pinned air release; each dev stage copies the binary out. Identical text on an identical parent is what lets BuildKit build it once for all four — go install sat on per-service COPYs, so it compiled four times. No shared base image: one base tag and one tool do not need one.

The orchestrator

scripts/compose/compose-run.py (invoked by the Justfile, never directly) turns the scattered fragments into one runnable configuration.

flowchart TD
    J["just stack dev / start / down / …"] --> R["compose-run.py"]
    R --> W["Walk backend/, simulators/ and devices/<br/>apply exclude patterns"]
    W --> G["Write compose.runtime.yml<br/>(an include: list)"]
    G --> D["docker compose -f compose.runtime.yml (up | down | build)"]

Why a generated include: file, not multiple -f flags

Passing several files with repeated -f flags makes Compose resolve every relative path (such as build.context: .) against the first file’s directory, which breaks per-service Dockerfile lookups. Compose’s include: directive instead resolves paths relative to each included file, so build.context: . inside backend/digital-twin/docker-compose.yml correctly points at that service’s directory. The orchestrator therefore writes a small compose.runtime.yml whose body is an include: list:

# Auto-generated by scripts/compose/compose-run.py — do not commit.
include:
  - path: docker-compose.yml
  - path:
      - backend/digital-twin/docker-compose.yml
      - backend/digital-twin/docker-compose.dev.yml
  - path: backend/telemetry/docker-compose.yml
  # … one entry per included service …

A single-path entry becomes one sub-project; a multi-path entry merges a service’s base file with its dev override into one sub-project (two separate include entries cannot define the same service — that raises a conflict).

How the include list is built

1. Always include the root docker-compose.yml.
2. Walk backend/ (skipping tests/), simulators/ and devices/, adding each folder’s docker-compose.yml and — in development modes — its docker-compose.dev.yml if present.
3. Add frontend unless excluded.

Excluding services

In dev and start, services can be left out either by named flag or by raw substring:

just stack dev --no-agent                  # agent + its Langfuse/ClickHouse/MinIO stack
just stack dev --no-metrics                # prometheus, grafana
just stack dev --no-simulators             # aq-simulator, sensor-simulator, ap-simulator
just stack dev --no-devices                # ap-collector
just stack dev --no-agent --no-metrics     # combine freely
just stack dev redis-langfuse              # raw substring, for anything without a flag
just stack dev --dry-run                   # print the resolved docker command, run nothing
just stack dev --help                      # generated by argparse

A substring is matched against both a folder name and a compose service name. A folder match drops the whole fragment from include:; a service match drops that service from up’s explicit service list. The second form is the only way to omit something defined in the root docker-compose.yml, which is always included — that is why --no-metrics exists at all: prometheus and grafana live there and share no substring.

Each flag is generated from EXCLUDE_GROUPS at the top of scripts/compose/compose-run.py:

Adding a group is one entry in that dict; the flag appears automatically. Unknown flags are rejected with a usage message, so a typo fails loudly instead of silently matching nothing.

The down, integration, and build modes always operate on the full set and ignore exclude patterns.

Published host ports

Only the dev overrides publish database ports; in every other mode a service reaches its database by container name on its native port. Two services claiming one host port makes docker compose up abort the whole stack, so check this table before adding one.

The modes

The first argument to the orchestrator selects the docker compose invocation. Each maps to a just recipe.

The orchestrator runs docker compose with COMPOSE_BAKE=true, which delegates builds to BuildKit Bake for faster, parallel image builds.

The acceptance suite composes its own stack

just test integration does not go through the orchestrator. It runs backend/acceptance/run-integration-tests.sh, which writes its own runtime file under its own project name (crowdvision-integration-tests), so it never collides with a running just stack dev and tears down independently.

Fragments are listed in include: order. No fragment may redefine a service another one already declared — compose rejects that with services.<name> conflicts with imported resource, so anything a service needs only for this suite still has to live in that service’s own fragment:

Per-service integration suites compose their own too

A service whose tests/*.rs need real infra keeps that infra in its own docker-compose.test.yml, run by its own test-integration.sh under its own project name. just test <svc>-integration is the entry point; each tears itself down.

This is why just test all needs nothing running. Every Rust service’s moon test task is restricted to cargo test --lib (plus its architecture fitness test where it has one), so the unit leg touches no database — matching tpl-rust-ci.yml, which runs cargo test --lib and scripts/test/rust-integration-tests.sh as two separate steps. A service that skips the override runs its integration tests inside the unit task, which is what made just test all fail without a MongoDB until dashboard got one.

What is stubbed, and why

The gateway container is what makes backend/acceptance/edge/ possible: every other test in the suite addresses a service directly, so Caddy is otherwise never in the path.

Environment and the dev loop

Every stack-starting recipe (dev, start) depends on the env recipe, which regenerates .env (VAPID keys, the local-dev-only eval JWT secret, Langfuse secrets) before Compose runs — see Setting Up Your Environment.

Typical local loop:

just stack dev                 # full stack with hot-reload
just stack dev --no-agent      # lighter stack without the AI agent
just stack logs digital-twin   # follow one service's logs
just stack down                # tear it all down

just stack logs [service] follows logs against the generated compose.runtime.yml, so it targets exactly the services the last run started.

Each MongoDB runs with --wiredTigerCacheSizeGB 0.25. Unset, WiredTiger caps its cache at half of (host RAM − 1 GB) per instance — about 7 GB each on a 16 GB machine, for four instances. The cap is a ceiling, not an allocation, but a long-running stack grows toward it.

Each Rust dev container keeps its own <service>-cargo-registry volume. Do not merge them: cargo’s lock file lives at CARGO_HOME root, outside the volume, so two services rebuilding at once unpack the same crate into one directory and corrupt it.

The four Go dev containers share go-mod-cache (/go/pkg/mod) and go-build-cache (/root/.cache/go-build). air downloads and compiles at container start, not during the image build, so a build-time cache mount would never reach it. Sharing is safe here, unlike the cargo registry: Go takes its locks inside the cache directories themselves.

Image pins

Every third-party image is pinned to an exact version — no latest, no bare major — and the same image carries the same tag in compose, test compose, k8s, CI actions and testcontainers. A floating tag makes the stack you get depend on the day you pulled it.

• Postgres family stays on 17 (postgres:17.11-alpine, pgvector 0.8.6-pg17, timescaledb 2.30.0-pg17): Postgres 18 cannot open a 17 data directory.
• TimescaleDB 2.17.2 → 2.30.0 keeps working on existing volumes, because the image still ships the older library. ALTER EXTENSION timescaledb UPDATE; moves a database onto 2.30.0.
• Langfuse 4 needs ClickHouse ≥ 25.12, Postgres ≥ 15 and Redis ≥ 7. It rewrites the v3 ClickHouse data model with no rollback, so local v3 trace volumes were dropped, not migrated.
• cgr.dev/chainguard/minio is pinned by digest: Chainguard’s free images publish only latest.

Related operations

For the production-grade orchestration (StatefulSets, ingress, secrets), see Kubernetes Configuration; for the runtime topology and gateway routing, see the Architecture Overview.