CI/CD Pipeline

CrowdVision uses a fully automated CI/CD pipeline on GitHub Actions. Every pull request to master is gated by a suite of checks — builds, tests, linting, security audits, Docker and Kubernetes validation, and a guard that refuses to let any unreviewed path merge unchecked. Merging to master keeps a standing release pull request up to date; merging that release PR cuts a versioned release and pushes Docker images to GHCR. This page documents the workflows; the commit format and release mechanics that drive them are covered in Commits & Release Please.

Workflow naming

All workflow files live in \.github/workflows/ and use a three-prefix convention that mirrors their purpose.

The CI gate

Every pull request must pass CI Gate / CI Passed — the single aggregated check that covers all the fast CI. The gate (ci-gate.yml) detects what changed, fans out to the relevant checks in parallel, and reports one result. The only other check worth requiring in branch protection is CodeQL (the Static security analysis section below), which runs as a separate workflow because it needs wider permissions than the gate’s read-only token.

graph LR
    PR[Pull Request] --> CHANGES[changes<br/>path filter]
    CHANGES --> SVC[Service legs<br/>chat · agent · frontend<br/>contracts · notification<br/>sensor · socket · twin<br/>claims-gateway · tenancy-service<br/>registry-service · provisioner]
    CHANGES --> DOCKER[docker<br/>hadolint + build]
    CHANGES --> K8S[k8s<br/>kubeconform · pluto · kube-score]
    CHANGES --> LINT[workflows-lint<br/>actionlint]
    CHANGES --> INFRA[infra<br/>scripts + tooling]
    SVC --> PASSED[CI Passed]
    DOCKER --> PASSED
    K8S --> PASSED
    LINT --> PASSED
    INFRA --> PASSED
    PASSED --> MERGE[merge allowed]

How the gate decides

The changes job runs dorny/paths-filter once and emits a boolean per area. Every downstream leg is guarded by an if: on those booleans, so a leg runs only when a path it owns actually changed. Service legs and the cross-cutting checks are called as reusable workflows (uses: ./.github/workflows/ci-*.yml); the gate itself stays a thin orchestrator — detect, fan out, aggregate.

The final ci-passed job depends on every leg and runs with if: always(), so it reports even when a leg was skipped. Its verdict logic is deliberately strict:

• Whitelist, not blacklist. Each leg’s result must be success or skipped. Anything else — failure, cancelled, or any future result value — fails the gate. A blacklist that only checked for failure would let an unknown state slip through green.
• The fan-out root must have succeeded. If the changes job itself failed or was cancelled, every leg was skipped without ever being evaluated — a green gate there would be a false pass. So ci-passed hard-fails unless changes ended in success.

Service CI workflows

Each service has a dedicated reusable workflow that the gate calls when that service’s directory (or its own workflow file) changes. Each installs dependencies, lints, builds, runs tests, and audits its dependencies.

The Node services (chat, sensor, socket, notification) share the same shape: npm ci, install the shared tooling/eslint-config/, lint, build, test with coverage, then audit. The frontend splits into a quality job (lint, type-check, audit), a test job that shards Vitest across three parallel runners, and a coverage job that runs the suite unsharded to produce a single coverage report. The two Rust services (contracts-service, twin-service) both run cargo fmt --check and cargo clippy --all-targets -- -D warnings before cargo llvm-cov, then a rustsec/audit-check; twin-service’s job additionally installs mold and spins up a real mongo:7 service container, since its integration tests exercise MongoDB directly rather than mocking it. The agent runs ruff check, ruff format --check, and pyright in a quality job, with pytest (unit + integration, with --cov) in a separate tests job. The four Go services each run go vet (lint), go build, then go test -coverprofile — actions/setup-go’s go-version-file picks up each service’s own pinned version (provisioner is still on Go 1.23, the other three on 1.25); none of them run an audit step yet (see the note below). Coverage reporting is detailed in the Code coverage section below.

Code coverage

Every service CI workflow now measures code coverage as part of its test run and reports it on each pull request — collected with each stack’s native tooling, with no external coverage service involved.

The Node and Rust services write a coverage summary into the GitHub job summary (rendered from the report with jq); the agent surfaces its figures directly in the pytest log via --cov-report=term-missing.

Simulator CI

The two device simulators ship as GHCR images but are not application services, so they get a dedicated ci-simulators.yml — one job each, gated on simulators/.

Both were previously ungated despite shipping images until this workflow gave them a real bucket.

Cross-cutting checks

These legs validate properties that span the codebase rather than a single service. They are gated by their own path filters and join the same ci-passed aggregator.

CD workflows

cd-release

Triggered on every push to master. Runs googleapis/release-please-action, which reads commits since the last tag and either opens/updates a release pull request (CHANGELOG.md + every version manifest) or, if its own release PR was just merged, tags the repository and publishes a GitHub Release. A concurrency guard prevents duplicate runs. The full mechanics are in Commits & Release Please.

cd-registry

Triggered on push to master when source changes under backend/**, simulators/**, frontend/**, or the workflow file itself. It publishes Docker images to GHCR tagged both :latest and :<git-sha>, for all fourteen images — the ten TypeScript/Rust/Python services and simulators, plus claims-gateway, tenancy-service, registry-service, and provisioner.

The publish is gated by a vulnerability scan that runs before anything reaches the registry. Each image is built into the runner’s local daemon (push: false, load: true), scanned by Trivy, and pushed only if the scan passes — so a vulnerable image is never published. The push step rebuilds from the still-warm gha cache, so it costs no recompilation.

Selective builds: only changed services are pushed

The workflow does not rebuild every image on every push. A changes job detects which services changed and emits a dynamic build matrix; a build-and-push job (needs: changes) then builds only the selected images, and is skipped entirely when nothing matches. Two cases force a full rebuild: a change to cd-registry.yml itself, and a manual workflow_dispatch with service: all (or a comma-separated list to target specific images).

Scan scope

The Trivy scan is deliberately scoped to OS packages only (vuln-type: os). Application dependencies are not re-scanned here because they are already gated pre-merge by the per-service audits. The division of labour is: the service CI workflows own language dependencies at PR time; cd-registry Trivy owns the base-OS layer at push time — the part the dependency audits cannot see.

To keep the OS scan green, every production image upgrades its base-OS packages at build time (apk upgrade on Alpine images, apt-get upgrade on the Debian-based contracts, agent, and aq-simulator images). The Debian upgrade lines carry # hadolint ignore=DL3005. Any residual fixable CVE that cannot be patched in-build is waived in the repo-root \.trivyignore, where each entry must carry a justifying comment.

Static security analysis (CodeQL)

ci-codeql.yml runs GitHub’s SAST for javascript-typescript, python, rust, and go with the security-extended query suite. It runs automatically on every pull request to master, on every push to master (to refresh the Security tab’s default-branch baseline), and on a weekly schedule (Monday 03:00 UTC) to catch CVEs newly disclosed against unchanged code. Manual workflow_dispatch is retained for on-demand re-scans, and a concurrency guard cancels a scan superseded by a newer commit.

All four languages use build-mode: none — CodeQL extracts from source without a compile. For javascript-typescript and python that is the natural mode; for rust (contracts-service and twin-service) and go (claims-gateway, tenancy-service, registry-service, provisioner) it means no toolchain setup is needed for CodeQL’s own purposes, and each of the several independent crates/modules is found and extracted directly from source without a root Cargo.toml/go.work tying them together.

It runs as a standalone workflow rather than a leg of the gate, on purpose: CodeQL needs security-events: write to upload results, whereas the gate’s token is deliberately read-only, and a reusable workflow cannot hold more permission than its caller. Keeping CodeQL separate preserves the gate’s least-privilege token. Add CodeQL as a second required status check in branch protection (alongside CI Gate / CI Passed) to block merges on new high-severity findings.

Maintenance

maint-cleanup runs every Sunday at 02:00 UTC (or on manual dispatch). It removes old package versions from GHCR, keeping a minimum of three versions per image and protecting the :latest tag, so the registry does not grow unbounded.

Dependency updates

Dependency upgrades are made manually (there is no Dependabot). Bump the version in the relevant manifest, then verify and regenerate locks locally before opening a pull request:

just setup deps-check   # every lockfile in sync (npm ci / uv sync --locked / cargo check)
just setup audit        # no known vulnerabilities (npm / uv / cargo audit)

Both checks also run inside the per-service CI workflows on every pull request, so a drifted or vulnerable dependency blocks the merge regardless.

Docker image registry

All images are hosted at ghcr.io/nickghignatti/ and tagged both :latest and :<git-sha>, published automatically by cd-registry on every push to master that touches them. The Kubernetes deployment manifests reference :latest.

Reproducing the gates locally

Run the same checks CI runs before pushing, to avoid a slow round-trip through a failed pipeline:

just test all            # all unit tests (moon-cached); or just test affected
just lint affected   # ESLint / Ruff on changed projects
just setup deps-check      # lockfile sync
just setup audit           # vulnerability audit

Per-service tests are available as just test chat, just test twin, just test sensor, just test notification, just test socket, just test frontend, and just test agent (no per-service recipe for contracts-service or the four Go control-plane services — use just test affected/all, which cover every registered project via moon run :test). Full backend integration tests run via just test integration (it composes the full stack — see Docker Compose Orchestration.