Skip to content

CodeSynapse as a multi-tenant product

This is the target architecture for running CodeSynapse as an open-source, self-hostable SaaS product on Kubernetes. It is built in phases; each completed phase is committed, pushed and opened as its own pull request. Decisions below were agreed on 2026-10-22.

Who it is for

Engineering teams and the people around them (team leads, security and compliance, platform owners) who are not AI or data specialists. They know that model tokens cost money, that security issues and vulnerable dependencies matter, and that the wrong licence can be expensive. The product answers in that language: what needs attention, what it costs, and what to do next.

Concepts

Concept Meaning
User A person with an email address. Logs in with a password (argon2id) or single sign-on (OpenID Connect).
Organisation The tenant. Owns repositories, reviews, agent memory, statistics and settings. A user can belong to several.
Membership A user's role in an organisation: owner, admin, member, viewer.
API key A personal key (csk_…) bound to one user and one organisation. Agents (MCP), hooks and the CLI use it. Shown once, stored only as a SHA-256 hash, revocable, with last-used time.
Root The account that set up the installation. Governance only: settings, health, telemetry, audit and account recovery. It cannot do organisation work, whatever memberships, grants, keys or jobs it has; setup hands over to a standard owner. See root bootstrap.
Platform admin Operates the installation: sees cross-tenant operational statistics and manages users. Root is one; standard accounts can be granted the flag. Explicit organisation membership and repository permission are required for code access.
Repository An organisation's view of a code base (remote, branch, schedule, settings).
Graph The indexed code graph of a repository at a commit, in HugeGraph. Shared between organisations when allowed (below).

Data

  • PostgreSQL holds relational, tenant-owned data: users, organisations, memberships, sessions, API keys, invitations, settings, audit log, and (from phase 2) the repository registry and reviews.
  • HugeGraph keeps code graphs, agent memory and usage counters, scoped by organisation from phase 2; Milvus keeps memory vectors. (Neo4j did this until it was removed in October 2026.)
  • No source bodies are stored anywhere, as before (AGENTS.md).

De-duplication

The same repository is indexed once, not once per organisation:

  • A graph is keyed by its remote and commit. Organisations reference graphs through their repository entries.
  • Public repositories are shared by every organisation that adds them.
  • Private repositories are shared only between organisations that each proved access with their own token (a successful fetch of the same remote and commit). Otherwise each organisation gets its own graph.
  • Organisation-owned data (reviews, memory, decisions, schedules, settings) is never shared.

Authentication and authorisation

  • Sessions: login sets an HTTP-only, SameSite=Lax, Secure (behind TLS) cookie holding a random token; only its hash is stored. Sessions expire after 14 days and on logout or password change.
  • API keys: Authorization: Bearer csk_…. Every MCP connection, hook and CLI call uses one, so usage, memory and reviews are attributed to a user and organisation and never mix.
  • Service token: manages platform operations only in accounts mode; content calls require an authenticated organisation-scoped principal. Explicit single-tenant/offline mode remains supported.
  • Without a database (CODESYNAPSE_DATABASE_URL unset) the API runs single-tenant exactly as before.
  • Brute force: failed logins are rate limited per email and per client address.
  • Audit log (NIS2): every changing request (route, target, status, outcome) and every security event (sign-up, login success and failure, rate limiting, logout, password change, key creation and revocation, organisation and membership changes, refused requests) with actor, authentication method, key, organisation, client address, user agent and request ID. Entries are append-only and hash-chained (each hash covers the entry and the previous hash; GET /api/v1/admin/audit/verify checks the chain), exportable as JSON, JSON Lines or CSV (organisation admins: their organisation; platform admins: everything), written as structured audit log lines for SIEM collection, and kept CODESYNAPSE_AUDIT_RETENTION_DAYS (default 400).
  • Permissions: organisation administration is separate from repository access. Organisation defaults are READ_ONLY or NONE; direct user overrides take precedence over team grants, then organisation defaults. Repository READ permits navigation, WRITE adds reviews/memory writes, and ADMIN adds sync, schedules and access management. New and migrated organisations default to NONE. See the access-control implementation for migration and conflict rules.

Product structure (web app, phase 3)

The "Agent spark" product UI with light and dark themes: ink on white with lilac surfaces, one candy accent for the leading action and candy tints only for status. Plain language, an explanation next to every figure, and costs in money where possible.

  1. Overview: what needs attention now across the organisation (critical security findings, vulnerabilities you actually call, licence conflicts, stale repositories), and what agents saved.
  2. Repositories: list with health at a glance; each repository has Summary, Security, Licences, Dependencies, Code health, Reviews and Memory.
  3. Agents: connect an agent in three steps (create a key, copy the setup for Claude Code / Codex / Cursor, verify), then see sessions, tokens saved and memory.
  4. Settings: organisation (members and roles, Git tokens per host, AI models, licence policy, sync defaults) and personal (profile, password, API keys, sessions).
  5. Administration (platform admins): system health, organisations, users (suspend, platform admin), cross-tenant statistics and the audit log.
  6. Help: getting started and "what does this mean" pages linked from every screen.

Phases

  1. Accounts and audit (done): PostgreSQL with embedded migrations; users, organisations, memberships, sessions, API keys; the hash-chained audit log with export and verification; signup (first user is platform admin), login, logout, me, organisations and members, personal API keys; auth middleware accepting session, API key or the legacy service token; single-tenant mode unchanged without a database.
  2. Tenant scoping, in three parts:
  3. 2a (done): repository registry in PostgreSQL (repositories: organisation, graph, remote, visibility); every API route carries an authorisation policy; lists, overview, statistics and jobs show the organisation's repositories only; other organisations' repositories, reviews and jobs answer 404; an organisation adding an already indexed public repository is linked to the existing graph (de-duplication); organisations add public repositories only, cloned without the installation's tokens; deleting removes the link and the graph with the last link; organisation owners can link verified remote graphs (POST /api/v1/accounts/orgs/{org}/repos); phase 4d closes the old platform attachment bypass.
  4. 2b (done): the MCP server authenticates personal API keys (checked on every request, so revocation and removed memberships take effect at once) and scopes every tool to the key's organisation; agent memory is kept per organisation even on a shared graph (org|graph namespaces, also for the API's offload and recall); reviews carry their organisation; usage counters carry the organisation and the statistics show an organisation its own; calls the MCP server makes to the API on a user's behalf use that user's key.
  5. 2c (done): organisation Git tokens per host (PUT/DELETE /api/v1/accounts/orgs/{org}/git-tokens/{host}, owners and admins), sealed with AES-256-GCM under CODESYNAPSE_SECRET_KEY and bound to organisation and host; members see host and preview. Organisations add private repositories after proving access with their own token (one ls-remote); the clone, syncs, branch lists and fix-branch pushes use it, never the installation's tokens. A graph is shared only between organisations that each proved access, and only for the same remote. Every sync re-checks each organisation's access: a refused or removed token hides the graph from that organisation (audited) until a working token restores it; network failures change nothing. Also fixed on the way: organisations could not add public repositories without naming a branch, and a public URL registered under a private graph's name would have linked that graph. 2d. Relationship-based access (ReBAC) (done): Zanzibar-style relation tuples in PostgreSQL with a Go engine (no extra service). Teams in organisations (members and maintainers, with an organisation role for the whole team); a guest role that inherits nothing; per-repository grants to people and teams (viewer, editor, admin); legacy cross-organisation sharing (inactive after phase 4d until explicit membership and a grant exist); reviews and agent memory follow the repository's grant. The API and MCP compute access the same way; agents of viewers and share recipients are read-only; every grant change is audited. UI: Settings → Teams, and an Access page on each repository.
  6. New web app, in two parts:
  7. 3a (done): the calm design system (light and dark, one accent, status colours for meaning, validated chart palettes per theme) applied to every page through tokens; sign-in, sign-up and first-run setup; the web proxy forwards the user's session instead of the installation token (which made every visitor a platform admin) and reports the client address to the API through CODESYNAPSE_TRUSTED_PROXIES; organisation switcher; Settings (profile and password, API keys, members and roles, Git tokens); Agents (create a key, copy the setup for Claude Code, Codex, Cursor or any MCP client, see the first call arrive); Help (getting started and what every figure means).
  8. 3b (done): Overview is "what needs attention" (GET /api/v1/attention, also per repository with ?repo=): called vulnerabilities, critical and high findings, licence conflicts, private repositories that lost access, failing syncs, other vulnerabilities, repositories no security rule covers, stale indexes and missing update automation, ranked and each with a plain explanation and a link to where it is fixed; repositories with nothing flagged are listed as all clear. Repository pages are grouped as Repository (Summary with its own attention list, Security, Dependencies, Licences, Code health = Quality + Unused code, Access), Work (Reviews, Agent memory, Impact of a change, AI assistant) and Explore (Graph, Symbols, Kubernetes & Helm, Docs, Packages). The new Dependencies page lists every package with version, licence verdict, vulnerabilities with reachability and fixed version, and the Renovate and Dependabot findings.
  9. Administration, access isolation and retrieval:
  10. 4 (done): the Administration area for platform admins. Overview: health of the graph store, accounts database, live statistics and audit chain, with timings; counts of people, organisations and repositories; unassigned repository counts (phase 4d hides their names and removes platform assignment); tool calls by organisation, model cost and latency over 24 hours, 7 or 30 days. Organisations: members per role, repositories, repositories that lost access, teams. Users: memberships, active API keys, last agent use; suspend and resume accounts (a suspended account is signed out everywhere, cannot sign in and its keys stop working, while memberships and history stay) and grant or revoke platform admin, never leaving the installation without an active admin and never suspending yourself. Audit log: filters by kind of action, person, outcome and period, entry details, CSV and JSON Lines export of the filtered period, and chain verification. Every change is audited (user.suspend, user.resume, user.platform_admin.grant|revoke).
  11. 4b, privacy and model cost (done): personal data is masked in free text before it is stored or sent to a model (redact.Personal, and redact.Free for secrets plus personal data): payment card numbers (issuer prefix, length and Luhn), national identity numbers (US SSN with unissued ranges excluded; Nordic DDMMYY/YYMMDD/YYYYMMDD-NNNN with a real date, eight-digit dates not in the future) and phone numbers (international with +, North American, and eight digits in pairs), replaced by [CARD], [NATIONAL-ID] and [PHONE] so the sentence still reads. It applies to agent memory names, offloaded turns, review text, every model prompt and every embedding input. Only formatted or validated numbers match; hashes, versions (including semver build metadata), line numbers, timestamps, UUIDs, IP addresses, dates and diff hunks are left alone (tested, and fuzzed for idempotence). Ingest summaries keep secret redaction only. Anthropic calls mark the system prompt (always a constant) as the cache prefix, so repeated calls of a task read it from the prompt cache once it is longer than the model's minimum; cached tokens count in llm_tokens_saved_total and in the cost. Nearest-neighbour memory search returns 3 to 10 hits (8 by default; was up to 50) in every store (vector.ClampK).
  12. 4c, search (done): docs/adr/0001-search.md compares search inside the stores we run (Milvus BM25 and dense vectors with RRF, HugeGraph and Neo4j indexes) with a separate engine fed by change data capture (Elasticsearch, which is not Apache-2.0 compatible, or OpenSearch/Solr, which are) on footprint, freshness, tenant isolation and revocation, latency, answer size and licences, and keeps search in the stores. Agent memory recall is now hybrid in the proper sense: a BM25 leg over the namespace's entity names, aliases and types (MemoryLexical, one ranker for all three stores, about 5 ms at the 5,000-entity cap) and the vector leg are fused by reciprocal rank, deduplicated and cut to 3–8 entries, each saying which legs found it; both legs read only the caller's namespace, and answers never carry vectors. Before, vector and substring scores were mixed as if on one scale. Code search keeps exact identifier lookup first and graph traversal for structure (source is not embedded). The ADR names when to move a leg into Milvus' native BM25 (longer text such as doc sections and reviews, p95 above 50 ms, larger memory caps) and the only conditions under which a separate engine would be worth it.
  13. 4d, platform isolation and repository permissions (implemented): removed the API/MCP platform-admin and service-token content bypasses. Requires explicit organisation membership, supports user/team grants and organisation read/none defaults, and distinguishes repository read, write and admin actions. Includes a versioned access projection for Neo4j/HugeGraph without making stale graph data an authorization source. Implementation plan: schema mapping, Cypher/Gremlin candidates, authoritative Go resolver, HTTP/stdio enforcement, migration of existing shares and parity/security tests.
  14. 4d follow-up, bootstrap root separation (implemented): the first account is an immutable bootstrap root (one per installation) limited to governance. Setup creates the first organisation and a standard owner in one transaction and ends the root session. Every API route and MCP tool is classified; root is refused business work through memberships, grants, keys, execution sessions, queued jobs and schedules. Existing installations are classified at start-up from their setup history; ambiguity limits platform administrators until codesynapse-api designate-root. Plan and implementation.
  15. 4e, tenant telemetry and four dashboards (implemented): OTel metrics exported to VictoriaMetrics with authenticated tenant, accepted MCP tool and configured provider/model attribution. Dashboard groups: token/model cost, network/cloud bandwidth, database latency and memory/offloading. Implementation plan: metric types and current-name compatibility, measured versus estimated savings, bounded cardinality, fixed tenant-filtered MetricsQL, dependency review and integration checks. This replaces the earlier OTel deferral with the approved v1.47.0 SDK/exporter. Full race tests, lint, vet, web build and native VictoriaMetrics/dashboard integration checks passed. Cloud savings remain unmeasured without an equivalent fallback; missing usage/prices remain unknown.
  16. 4f, graph/vector context reduction (implemented): bounded structured responses, exact identifier priority, current-graph recall validation and tenant-safe hydration across all stores. Implementation and conditional Milvus migration; reproducible measurements. Shared BM25/RRF remains in Go: lexical p95 at 5,000 entities is 4.74 ms, below ADR 0001’s 50 ms migration trigger. The small fixture has negative aggregate token savings after full MCP framing; no production SLA improvement is claimed.
  17. 4g, immutable session context (done): one runtime-supplied primary ID per thread, registered to one tenant and participant, inherited by MCP, subprocesses, models and queued work. Missing/conflicting context fails closed. Separate session graph/vector memory, fixed expiry, replay tombstones and explicit promotion preserve durable repository memory. Contract and migration covers API/HTTP/stdio integration, limits and adversarial tests. Accounts and a runtime-aware client are required; the dashboard binds each signed-in browser session itself. Session IDs remain out of metric labels.
  18. 4h, repository configuration and annotation (implemented): strict repository-root .codesynapse.yaml parsing, shared ingestion filters, explicit exclusion approval, tenant-local finding suppressions, and bounded local annotation of indexed symbol metadata. Graph replacement is atomic; approvals and annotations live in PostgreSQL. The Indexing policy page shows the indexed configuration, coverage, approvals and current annotations. Configuration reference covers approval, runtime identity, queue limits and migration. Benchmarks record the policy digest.
  19. 4i, ingestion transparency (implemented): report files encountered, indexed and skipped with explicit reasons, annotation-job outcomes and estimated token savings. Implementation plan: exact per-run accounting, scanner versus whole-file exclusions, bounded opaque repository labels, tenant-owned Repository.metrics reports and dashboard/audit coverage. Depends on 4h policy decisions and 4e telemetry; session context follows 4g.
  20. 4j, code architecture audit and upgrade assessment (completed): Audit results inventory the current stack, reproduce parsing/edit-validation gaps, compare upstream packages and record profiles and reproducible benchmarks. The assessment plan is fulfilled by tests and documentation; production upgrades are proposed as separate 4j.1–4j.7 PRs. Priorities are edit validation, final prompt preflight and measured allocation reductions. New dependencies, native parsers and interactive-editor work remain conditional. The agent guide documents MCP capability/freshness checks and the missing-tool fallback.
  21. 4j.1, diff parsing and generated-edit validation (implemented): Validation policy 1 requires pinned current reviews, local generation, strict fenced output, valid Go syntax and changes confined to accepted function bodies. File/commit/revision checks precede publication. Diff parsing retains rename/copy metadata, decodes Git quoting and rejects malformed or oversized input. Final-prompt preflight follows in 4j.2.
  22. 4j.2, final prompt budgets (implemented): Budget policy checks the full redacted provider request before dispatch, enforces input/output/context and byte limits, and distinguishes conservative estimates from billing usage. Exact-only policy refuses unsupported counts; no tokenizer dependency was added. Task packing rejects oversized first items and preserves triage batch tails. The provider panel shows the policy.
  23. 4j.3, allocation reductions (implemented): Measurements: the memory store indexes folded names at ingest and stops searching at the limit, with the old implementation kept as a fuzzed reference. minimal_context on 10,000 nodes drops from 94 ms and 220 MB to 1.7 ms and 2.4 MB; ingest pays about 4 ms per 10,000 nodes. Parser rules skip their regexps on lines without required literals, and the TypeScript line-owner scan became a table. Graph copies stay until callers agree an ownership rule. Numeric budgets are proposed, not enforced.
  24. 4j.4, edge evidence and analysis completeness (implemented): Evidence fields label every resolution edge typed, syntactic or inferred (unknown on older graphs, never promoted), and versioned analysis records mark partial and truncated packages and files, build-tag exclusions, unresolved dynamic calls and bodies skipped by a per-package budget. Agents see both in symbol context, call graphs and outlines. Labelled Go and Python/TypeScript fixtures measure precision and recall. No migration; the next sync labels existing graphs. Optional SSA is deferred to 4j.4b.
  25. 4j.4b, optional data-flow call analysis (implemented): --dataflow / CODESYNAPSE_INGEST_DATAFLOW builds SSA for well-typed repository packages and runs VTA, adding CALLS edges with dataflow evidence for calls through function values. Off by default. All 16 edges it adds on this repository are correct, but it links only 6 of 4,389 dynamic sites, because most values come from outside the repository; extraction takes 39% longer. Bounded, cancellable between phases and skipped on type errors, size or failure.
  26. Neo4j removal (implemented): HugeGraph with Milvus is the only graph database. The Neo4j store, driver, flags, compose service and manifest are gone; a configuration that still asks for Neo4j stops with an explanation, and upgraded installations re-index. Kubernetes manifests are marked outdated until 6a adds HugeGraph and Milvus.
  27. Single sign-on: OpenID Connect login and just-in-time accounts, built on the completed tenant and session rules.
  28. 5a, API (implemented): one provider per organisation with sealed client secret, PKCE/nonce/browser-bound state, SSRF-guarded discovery and exchange, guarded email linking, passwordless just-in-time accounts, group-mapped roles and teams on provider-managed memberships, last-owner and root protection, full audit. Plan.
  29. 5b, dashboard (implemented): single sign-on on the sign-in page (with /login?sso=<org> links), the /sso/callback page, Settings → Single sign-on for owners and admins, linked identities on the profile; the proxy forwards the cs_sso binder on SSO paths only.
  30. 5c, MCP request audit trail (implemented): every MCP request is traceable afterwards to who made it, with which key, from which chat and what it asked for.
    • What is recorded: every request except ping and notifications, whether accepted, refused (authorization), failed or answered with a tool error. Requests refused earlier, for a bad key, root or a missing runtime session, keep their existing mcp.authentication_failed, root.denied and mcp.access_denied entries.
    • Fields per event:
    • action mcp.<method> (mcp.tools.call, mcp.prompts.get, mcp.initialize, …);
    • time and request ID;
    • user ID and email, organisation, and the API key's ID (never the key or its hash);
    • runtime (chat) session and MCP transport session;
    • client name and version (from initialize or the sessionless _meta);
    • client address (trusted proxies only) and user agent;
    • tool or prompt name and repository;
    • outcome, status (200, 400 for tool errors and refusals, 500 for errors) and the redacted first line of the error;
    • duration and response size.
    • Arguments:
    • Values pass redact.Free (secrets, card, national ID and phone numbers) and are capped at 400 characters and 32 keys.
    • diff, patch, code, content, body, text, turns, messages, triples and comments, and any value longer than the cap, keep only sha256:<hex> (<n> bytes).
    • A digest of the canonical arguments links a payload found later to its request.
    • Results are never stored.
    • Storage:
    • The same hash-chained events table, with a mcp jsonb column (migration 0012). MCP details join the hash only when present, so earlier entries verify unchanged.
    • Entries are written synchronously after the request. If the entry cannot be written, the result is withheld; tools that change something still audit before the change.
    • Batching appends is a follow-up if measurements show lock contention.
    • Who sees it:
    • Platform admins: Administration → Audit log.
    • Organisation owners and admins: the new Settings → Audit log tab.
    • Filters: what (including "Agent (MCP) requests"), who, outcome, chat session, tool, repository and period; clicking a session, tool or repository narrows to it.
    • Exports: CSV (with mcp_* columns) and JSON Lines.
    • Client address: RequestInfoHTTP sets it, with the request ID, from the connection and trusted proxies (app.ClientAddr, now shared with the API); copies a client sends are overwritten. Compose trusts no proxy for MCP unless CODESYNAPSE_MCP_TRUSTED_PROXIES is set.
    • Tests:
    • the store contract (round trip, caps, filters, chain verification on memstore and PostgreSQL);
    • TestEveryMCPRequestIsAudited: caller, key, session, client and address; redaction; bulk inputs digested; the key never logged; refusals recorded; filters; the chain intact; the result withheld while the audit store is down;
    • TestAuditTrail: API filters and the CSV export.
  31. 5d, CI/CD baseline (in progress): before 5d, the only workflows built the Homebrew formula and released the CLI on a tag.
    • Done, pull request and merge checks: .github/workflows/ci.yml runs just ci. The integration contracts run when stores or accounts change, and nightly. Accepted findings live in the tools' ignore files, with reasons. Actions are pinned by SHA, and Renovate updates the digests and the tools' versions.
    • Done, images: .github/workflows/images.yml builds and scans api, mcp and web on every relevant pull request, with read-only permissions. Merges to main build and scan only. Only a release publishes: a vX.Y.Z or vX.Y.Z-rcN tag on a commit on main (the workflow refuses any other tag). It publishes amd64 and arm64 images to ghcr.io with SBOM and provenance, signed with cosign (keyless). Release candidates are tagged :X.Y.Z-rcN only; final releases are tagged :X.Y.Z, :X.Y and :latest. Go and the web app are cross-built on the runner. The web runtime image drops npm, which was the source of its fixable vulnerabilities.
    • Next: the Helm checks arrive with the chart in 6a.
    • On every pull request (required checks; the same commands work locally through just):
    • go test -race (just test), golangci-lint with gosec (just lint), and govulncheck;
    • the Nuxt typecheck and build (just web-check), plus npm audit for production dependencies;
    • licence check of Go and npm dependencies against the Apache-2.0 allowlist (permissive only);
    • secret scanning (gitleaks), actionlint for workflows, Dockerfile lint (hadolint), shellcheck, yamllint;
    • Helm: helm lint, chart-testing (ct lint), kubeconform on rendered manifests;
    • kubesec and kube-linter on the rendered chart, failing on high findings;
    • Trivy config and filesystem scans;
    • the integration contracts (just test-postgres, just test-hugegraph) when stores or accounts change, and nightly otherwise.
    • On merge to main (prebuild):
    • build and scan all three images; nothing is published (only releases publish);
    • package the chart as a development version;
    • install it on a throwaway kind cluster and run a smoke test (health, sign-up, index a small public repository, an MCP repo_list through the bridge).
    • On a release (a vX.Y.Z or release-candidate vX.Y.Z-rcN tag on a commit on main):
    • versioned, signed images (cosign keyless, provenance attestation, CycloneDX SBOM); a release candidate gets only its exact version tag, a final release also X.Y and latest;
    • the chart published as an OCI artifact at oci://ghcr.io/astrona-io/charts/codesynapse;
    • the CLI binaries and Homebrew formula as today;
    • release notes generated from conventional commits.
    • Rules:
    • Workflows pin actions by commit SHA, run with least-privilege permissions, and never expose secrets to pull requests from forks.
    • Renovate keeps actions, tools and base images current, with approval for majors.
    • CI-only tools are not distributed: GPL tools such as hadolint and shellcheck are acceptable there, never inside images.

  32. 5e, service split by workload (planned; before 6a so the chart describes the final topology).

    • Why: today one api process serves HTTP and also runs every kind of work: clone and index jobs on untrusted repositories, AI tasks, fix branches that push, and eleven background loops (scheduler, annotations, offload extraction, access projections, audit and session-memory pruning, ingest-report recovery and metrics, memory-fact metrics, fix recovery). Jobs are in-memory, so a restart loses queued work. One slow ingest competes with every dashboard request for CPU and memory, the API image must carry the Go toolchain and git, and the process holding the accounts database also parses untrusted code.
    • Recommendation: split by workload, not by domain.
    • Accounts, reviews, memory and the catalog stay in the API. They share PostgreSQL transactions (policy, audit and outbox commit together) and the graph, so separate domain services would add network hops and failure modes without making anything faster or safer.
    • What scales, fails and is attacked differently gets its own process and image.
    Component Image Scales Holds
    api codesynapse-api, distroless or alpine, no Go toolchain or git horizontally, stateless HTTP API, dashboard backend, enqueues work
    mcp codesynapse-mcp horizontally, stateless MCP server, read-only graph access and the narrow audited writes
    web codesynapse-web horizontally, stateless dashboard
    worker (ingest role) codesynapse-worker with the Go toolchain and git by queue depth; memory sized from ingest peaks clone, sync, index, annotation; parses untrusted repositories, so it never holds push tokens or the SSO secrets
    worker (ai role) same image, --roles ai by queue depth AI tasks and offload extraction (model calls)
    worker (fix role) same image, --roles fix one or few fix branches; the only role given push credentials (CODESYNAPSE_ALLOW_PUSH)
    worker (system role) same image, --roles system one active (leader lease) scheduler, access projections, pruning, recovery, metrics export
    • Steps:

    • Durable job queue in PostgreSQL (no new dependency). Done, in process: jobs are rows before they run, and interrupted jobs are resumed after a restart. The separate worker follows in step 2.

      • Record: jobs (migration 0013) holds the kind, repository, organisation, requester (user, how they authenticated, API key ID), required role, execution identity, the parameters to rebuild the work (cleared at the end; no credentials), state, attempts, lease, timestamps, result and error. There is one queued or running job per repository (a unique index).
      • Runner: the API's runner adds the row before queueing, marks it running when the job gets its turn, renews this process's leases every 30 seconds (2-minute lease) and records the outcome.
      • Recovery: after a restart, resumeJobs claims jobs whose lease expired (FOR UPDATE SKIP LOCKED). It rebuilds the requester's context: session, account, API key, membership and repository permission are checked again, and tokens are derived again. Then it starts the work through the same function that built it (startSync, startCloneAuth, startSwitch, startAITask, startAutoAI), keeping the job ID.
      • Retries: a retried ingest gets its own run ID (<job>.<attempt>), so the interrupted run is reconciled, not overwritten. A job interrupted three times is failed. Fix jobs are recorded but never restarted; a second run could push twice.
      • History: job history lists from the store, so it survives restarts (newest 5,000 kept).
      • Without accounts: jobs stay in memory, as they also need accounts to run.
      • Tests: accountstest.RunJobs (memstore and PostgreSQL: one active job per repository, owner-only start and finish, orphan claims, the attempt limit, concurrent claims never doubling) and TestJobsSurviveRestart (an interrupted sync resumes and indexes; fix, revoked-session and exhausted jobs fail with a reason; new jobs record their requester and dashboard session).

      The original plan for this step: - A jobs table holds the kind, repository, organisation, serialized execution identity (never bearer tokens), state (queued, running, succeeded, failed, cancelled), attempts, lease owner and expiry, heartbeat, a trace context and an error. - Workers claim jobs with FOR UPDATE SKIP LOCKED, renew their lease while running, and requeue it if they die. - Retries keep the original primary session ID, as the session rules require. Every claim rechecks session, membership, key and repository permission before work and before persistence, as jobs do today. - The in-memory Jobs runner becomes one implementation of a Queue interface. Local development and the CLI keep it, as an in-process worker. 2. Workers (done; one image, roles). - Command: codesynapse-api worker --roles ingest,ai,fix (image target worker of services/api/Dockerfile, with the Go toolchain and git) claims queued jobs of its roles. Several workers share the work, claiming only while idle. The roles map to job kinds: ingest is sync, clone and switch; ai is ai and ai-auto; fix is fix. Only the fix role needs CODESYNAPSE_ALLOW_PUSH and push credentials. - API: with CODESYNAPSE_RUN_JOBS=false (image target api: Alpine with git, no toolchain, 81 MB instead of 321 MB) it records jobs without owner or lease, so the first worker of their kind claims them. Without workers (development, tests) the API still runs every job itself. - Rebuilding jobs: a worker rebuilds each job through the function that built it (startSync, startCloneAuth, startSwitch, startAITask, startAutoAI, and now startFix, with push credentials derived from the context). The requester's session, account, key, membership and permission are checked again. A fix that already started is still never restarted. - What stays in the API: the scheduler, pruning, access projections, ingest-report recovery and metrics export, conversation offload, and fix recovery. Schedules and the watched-organisation list live in a workspace file in API memory, so a worker never writes them; scheduled syncs show the outcome in the job list, not the schedule. Fix recovery now asks the job record: a fix is failed only once its job ended, never while a worker runs it. Annotation jobs run in the process that indexed the repository. - Compose: the stack runs api and worker (one worker with all roles; docker compose up --scale worker=3 adds more). The images workflow builds, scans and publishes codesynapse-worker alongside the others. - Follow-ups: - Done: schedules live in the accounts store (migration 0014). A schedules file from before moves in once and is renamed .imported. Workers report scheduled runs back. Runs and the missing-repository count update apart from the settings. - Done: the scheduler, pruning, access projections, ingest-report recovery and fix recovery run in one API process at a time, the holder of the api-loops lease (60 s, renewed every 20 s). They stop as soon as the lease can't be renewed. Several API replicas are now safe. - Remaining: the annotation queue stays in the process that queued it. Conversation offload is per process by design: turns are never stored, so they can't be shared. The watched-organisation list is single-tenant only. - Tests: TestWorkersRunWhatTheAPIRecords (the API records only; an ai-only worker leaves a sync alone; an ingest worker runs it, and the API shows the outcome). The contract covers claims by kind and unowned jobs.

      The original plan for this step: cmd/codesynapse-worker with --roles ingest,ai,fix,system, defaulting to all. The background loops move out of api.New. The system role takes a PostgreSQL advisory-lock leader lease, so loops never run twice; this also removes the single-API-replica limit noted for 6b. The API image drops the toolchain and git. Compose runs one worker with all roles, so just up stays simple. 3. Tracing and debugging (done; see traces): - W3C trace context crosses every hop: from web through api (HTTP headers) into the job row, then to the worker, the stores and model calls; MCP gets its own spans. - Logs carry trace_id, request_id and job_id. - The audit entry, the job and the trace share the request ID, so one ID finds everything. - Spans never carry prompts, code, arguments or repository names, following the telemetry rules. - Needs go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp, from the same Apache-2.0 project as the metrics exporter already in use; approval needed per AGENTS.md. Needs a trace store: VictoriaTraces or Jaeger (both Apache-2.0); Grafana Tempo is AGPL and excluded. 4. Isolation for the ingest role: - A read-only root filesystem with a writable workspace volume. - No route to PostgreSQL beyond what its checks need: a database role limited to job and permission reads plus ingest reports. - Network policies that allow only git hosts, osv.dev and deps.dev. - The fix role is the only one with push tokens, and the API never sees them. 5. Compose and the chart (6a) get one Deployment per component and role, each with its own resources, autoscaling on queue depth, and per-component dashboards. - Tests: - the queue contract (claim, lease expiry, requeue, cancel, the identity recheck, no double run) on PostgreSQL; - a worker killed mid-job is retried once by another; - the API image no longer contains go or git; - an end-to-end trace from a dashboard click to the graph write. - Not planned: a message broker (PostgreSQL is enough at this scale and keeps the transactional audit); splitting accounts, reviews, memory or the catalog into separate services; a sidecar per tenant. 6. Kubernetes deployment and releases (final phase): after the application features, access/session controls, telemetry, ingestion work and SSO are complete. Existing local development stays independent of Kubernetes, and nothing has been deployed to a production cluster. The earlier Kustomize work is parked on feat/k8s-production; it is reused as input for the chart, not shipped as is. - 6a, Helm chart and local build-and-test loop (done; see the chart README): - What's built: deploy/helm/codesynapse replaces the Kustomize tree. It has the four services, plus PostgreSQL, HugeGraph, Milvus, VictoriaMetrics and VictoriaTraces as StatefulSets, each replaceable by an external address. It uses one required Secret with no defaults, non-root pods (HugeGraph is the documented exception) and network policies. GOMEMLIMIT is set at 90% of each memory limit, there's an optional ingress (/mcp and the dashboard), and a helm test checks every service. - Checks: just chart-check (helm lint --strict, kubeconform, kube-linter, kubesec) runs on every pull request. just chart-e2e builds, loads, installs and tests on the throwaway kind-codesynapse-dev cluster, and the Chart end to end workflow runs it on chart changes, merges and on demand. Every command names its context explicitly. - Follow-ups: mount secrets as files (kubesec SecretsAsEnvironmentVariables), and measure ingest peaks for default memory limits. Backups, upgrade notes and versioning arrived with 6b.

    The original plan for this step: - A Helm chart in deploy/helm/codesynapse replaces the Kustomize overlays. - Values cover development and production. - The production data path is Apache-2.0 or permissive only (see the review below): HugeGraph and Milvus as the graph and vector stores, Milvus standalone on its own volume (no MinIO, which is AGPLv3), PostgreSQL (in-cluster StatefulSet, or a managed or CloudNativePG database by URL), VictoriaMetrics. Every secret the services need (CODESYNAPSE_SECRET_KEY, database, HugeGraph, metrics and API passwords) comes from one Secret with no defaults; HugeGraph refuses to start without its password; network policies let each store accept only the services that use it. TLS at the ingress with secure cookies and trusted proxies set. Memory requests and limits sized from measured ingest peaks, with GOMEMLIMIT so the API collects garbage before the kernel kills it. Backups (PostgreSQL dump, HugeGraph and Milvus volume snapshots, the audit export) and upgrade notes (migrations run on start; one API replica owns jobs and the scheduler). Tested on a throwaway kind cluster only. - Charts for the stores are optional dependencies or bring-your-own by URL. - The chart passes helm lint, ct lint, kubeconform, kubesec and kube-linter with no high findings. It has helm test hooks for health, sign-up and MCP reachability. - Local loop before anything ships, all through just: - build every image locally (image-build, multi-arch optional); - create a throwaway kind cluster with an explicit --context; - load the images and helm install the chart from the working tree; - run the same smoke test and lint/scan set as CI, then tear down. - The loop and CI share the same scripts, so a green local run predicts a green pipeline. - 6b, release pipeline (done; see Releasing and Upgrading): - What's built: a vX.Y.Z or vX.Y.Z-rcN tag on main publishes the four images (multi-arch, signed, SBOM and provenance) and the chart as a signed OCI artifact at oci://ghcr.io/astrona-io/charts/codesynapse. Chart, app and image versions are the tag without the v. A throwaway kind cluster then verifies the signatures, installs the published chart with the published images through a pull secret and runs helm test. Only after that does a final release get the images' :latest tag and become the latest GitHub release; release candidates never do. global.imagePullSecrets covers private registries, and just chart-install-published runs the same install locally. - Upgrade notes cover migrations on start (under an advisory lock), the api-loops lease, durable jobs across restarts, backups and rollback. - Follow-ups: ct lint and ct install --upgrade against the previous published chart, and Artifact Hub listing once the repository is public.

    The original plan for this step: - Images on ghcr.io: multi-arch, signed, with SBOM and provenance. - The chart as an OCI artifact, so anyone installs from GitHub with helm install codesynapse oci://ghcr.io/astrona-io/charts/codesynapse --version X.Y.Z. - Chart and app versions follow the release tag. - Upgrade notes cover migrations, which run on start, and the single API replica that owns jobs and the scheduler. - The kind smoke test runs against the published artifacts before the release is marked latest. - 6c, horizontal scaling of the services (done; see Scaling out): - What's built: login limits move from process memory to PostgreSQL (login_attempts, migration 0016, keys stored as SHA-256 hashes), so N API replicas still allow 5 attempts per email and 20 per client address a minute. A flood of distinct keys no longer resets them either. MCP gets --stateless (CODESYNAPSE_MCP_STATELESS, on in the chart and compose): no transport session between requests, so any replica answers any request, with the key and runtime session checked on each one. The chart adds optional CPU autoscaling for api, mcp and web, a disruption budget for any component with more than one pod, and preferred anti-affinity with zone spreading. The kind end-to-end run uses two API and two MCP replicas. - Already in place from 5e: durable jobs, workers by role, schedules in PostgreSQL and the api-loops lease. - Limits: annotation and memory extraction job status is per API replica (follow-up: a durable annotation queue); the shared workspace needs ReadWriteMany across nodes; MCP usage counters are read and rewritten in one Gremlin script, so two replicas flushing the same daily bucket at once may lose an increment (not yet measured). - Stores as subcharts (done): PostgreSQL, HugeGraph, Milvus, VictoriaMetrics and VictoriaTraces are subcharts of the chart (charts/<store>, condition: <store>.enabled), on and configured by default. Disabling one and linking an external store is the documented way to run production. The Secret, pull secrets and network policies are global values; shared helpers live in the common library chart. Store images are kept current by a Renovate rule for charts/*/values.yaml. - 6d, clustered stores. One PR per store. Done for a store when it survives the loss of a pod on kind while the services keep writing, with nothing lost. - 6d-1, PostgreSQL (done): run it highly available outside the chart and link it (postgresql.enabled=false, CODESYNAPSE_DATABASE_URL). just chart-failover-postgres (also in the Chart end to end workflow) installs CloudNativePG 1.30.1 (Apache-2.0, manifest pinned by SHA-256), points the chart at a three-instance cluster and kills the primary while the API writes: a new primary takes over in about 25 s, a couple of writes are refused, nothing restarts, and helm test passes. Two job fixes came out of it: a process that stops (SIGTERM during a rolling upgrade) no longer marks its running and queued jobs failed, and a process that cannot renew its job leases for 90 s (the database unreachable) stops its jobs before another worker may claim them (errLeaseLost). In both cases the record stays active and another worker resumes it once the lease runs out. - 6d-2, Milvus (done): link a Milvus cluster run by the zilliztech chart or the Milvus Operator (both Apache-2.0) instead of growing our subchart: milvus.enabled=false, milvus.externalURL (the proxy, port 19530) and, with authentication, MILVUS_TOKEN in the Secret (new: the chart passes it as CODESYNAPSE_MILVUS_TOKEN). The client now retries transient failures (connection errors, HTTP 5xx, Milvus "not ready/unavailable" codes) for up to 15 s; every call is safe to repeat. just chart-failover-milvus (also in the Chart end to end workflow) runs chart 5.0.30 (digest-pinned) with Milvus v3.0.4 in cluster mode: 3 etcd members, 2 coordinators in active-standby, 2 proxies, query and streaming nodes, 1 data node, authentication on, SeaweedFS 4.48 (Apache-2.0, digest-pinned) as the S3 store, and Woodpecker embedded in the streaming nodes as the write-ahead log, so no MinIO and no Pulsar. Killing a query node, a streaming node and a coordinator under load: Milvus absorbed it (no call needed a retry, none failed), every confirmed write was found afterwards, and helm test passed. Two findings for anyone running it: with Pulsar, Kafka and the separate Woodpecker off, the chart leaves the log type unset and cluster mode falls back to Pulsar, so set mq.type: woodpecker in extraConfigFiles.user.yaml; and that choice is stored in etcd at first start, so it cannot be changed on an existing cluster. - 6d-3, HugeGraph (waiting on upstream): distributed HugeGraph (PD + HStore with Raft) is still marked BETA in 1.7.0, its images are not ASF release artifacts, and the Helm chart on master targets 1.8.0 (which needs Java 17). Its Store and Raft ports have no authentication, so a deployment needs network policies around them, and server replicas need a shared token secret. Until 1.8 ships, the recommendation is the standalone server with volume snapshots; then test the store contract and parity against a 3 PD, 3 Store cluster.

Standing architecture review

Every phase is checked against these constraints, taken from the architecture and security review brief. The table says where each is enforced today and which phase closes what is still open.

Constraint Where it holds today Open, and where it is closed
Zero-trust tenancy. Tenant and repository come from the authenticated context, never from the model. The tenant is never a tool argument: API routes resolve it from the session or key (scopeOf, s.route policies), MCP from the key (Caller, memRepo). repo is an argument, but h.repo and visibleRepos accept only graphs in the caller's scope and answer 404 otherwise. Memory traversals start at tenant_id/ns; vector searches require a tenant (vector.ErrNoTenant) and drop foreign hits (vector.Isolate). Platform and service credentials cannot bypass content permissions. Repository-scoped readers reject foreign node IDs and traversals. Parent-session enforcement is implemented; final deployment still waits for the remaining application phases and client integration.
Apache-2.0 or permissive licences on the production data path. No SSPL or BSL. Go dependencies (cobra, the MCP SDK, x/tools, pgx, the Anthropic SDK), Nuxt, HugeGraph, Milvus, PostgreSQL and VictoriaMetrics are Apache-2.0, MIT, BSD or the PostgreSQL licence. Search stays out of Elasticsearch (docs/adr/0001-search.md). Milvus' usual object store, MinIO, is AGPLv3 (Neo4j Community, the former default store, was GPLv3 and has been removed). HugeGraph and Milvus (local storage) are the only stores, and 6a deploys them in production; 6c names object stores for clustered Milvus. A licence check of the Go and npm dependency trees runs in CI from 5d.
Graph and vector parity. HugeGraph (Gremlin) and the in-memory reference answer alike; Milvus serves vectors. storetest.Run is the contract every store passes; TestParityWithReference compares HugeGraph with the in-memory reference; vectortest.Run holds every vector index to tenant isolation. The lexical leg uses one ranker for all stores. 4f validates candidate identity and current graph evidence across all stores. Milvus-native BM25/RRF remains conditional on ADR 0001; the measured lexical latency does not trigger migration.
Token density and prefix caching. Short qualified names; full IDs only on request; static instructions first. Tools take and print qualified_id (store.(*Store).Get) and answer ambiguity with suggestions; composite tools (get_symbol_context, get_pr_context, minimal_context) cut round trips; recall returns 3–8 entries; system prompts are constants and cached (4b). 4f bounds composite structured answers and records omissions. Its benchmark compares full MCP responses with raw text using explicit evidence checks and cold/warm percentiles; model answer quality and production latency remain unmeasured.
Ingestion fits in memory. Batched writes; no OOM kills on large codebases. HugeGraph writes are size-bounded (MaxRequestBytes) and staged in chunks before one bounded transaction. Extraction loads and type-checks the whole program with go/packages and assembles the graph before writing, so peak memory grows with the repository, not the batch. 6a measures peaks on large repositories and sets limits and GOMEMLIMIT; loading per module or package group, writing each group as it is done, is the follow-up if the peaks are too high.
Secrets and personal data never reach the graph. redact.Summary on every stored summary; author e-mails stored as hashes; findings describe problems, not values; free text through redact.Free (4b). Nothing open.
Read-only retrieval tools. Every retrieval tool is ReadOnlyHint. Writes: the review tools (:Review only) and, as a discussed exception that is on by default, the agent-memory tools (:MemoryEntity/:FACT only, --allow-memory=false turns them off). Kept as decided; any new write tool needs the same discussion.
Telemetry without high-cardinality labels. Model calls, database latency, tool calls and tokens saved are exported to VictoriaMetrics with bounded labels only (TenantLabel, ToolLabel); prompts, code, paths and user IDs never become labels. Phase 4e uses an OTel SDK producer over the bounded registry and cumulative OTLP/HTTP export, with the existing Prometheus-text push path available as a rollback setting. 4e validated OTLP first samples, resets, replica separation, bounded recording and the four dashboard groups. The OTel v1.47.0 dependency set is approved. Distinguish cached tokens, avoided cloud payloads, returned graph rows and current durable facts.