Code architecture audit: phase 4j¶
Assessment completed 2026-10-23 against ddf78a3ce7bcddd4f162d3af70b99d183ce86eb0, after phase 4i. This change adds reproducible tests, measurements and an upgrade roadmap. It does not implement the proposed production fixes or approve dependencies.
Keep native Go type analysis, graph-based context selection and the existing store boundaries. Address generated-edit validation and final prompt limits first. Profiles justify improving structural-parser and in-memory search allocations; they do not establish that replacing the stack would improve production latency.
Working assumptions are batch indexing/MCP retrieval and a pure-Go default. The operator was asked about live editing, CGo, repository sizes and resource targets; those requirements remain unconfirmed. Live-editor support, new languages and native dependencies are conditional projects. The synthetic workloads below cannot establish a production SLA or capacity limit.
Follow-up: 4j.1 is implemented in fix validation policy 1. The findings and benchmark numbers below describe the audited baseline; regression tests for resolved gaps now assert the corrected behavior. 4j.2 is implemented in final prompt budgets using an explicit conservative estimate. Verified tokenizer/template adapters remain conditional. 4j.3 is implemented in allocation reductions: indexed in-memory search, a line-owner table and literal guards for parser rules. 4j.4 is implemented in edge evidence and its optional SSA/VTA mode in 4j.4b. The other proposed upgrades are pending.
1. Stack audit and gap analysis¶
The checked-in module uses Go 1.27.3, golang.org/x/tools v0.52.0, x/mod v0.42.0, yaml.v3 v3.0.1, MCP Go SDK v1.8.0, Neo4j driver v5.28.6 and OTel v1.47.0. There is no Tree-sitter, SCIP, text-diff or tokenizer dependency.
| Domain | Current choice and evidence | Pros and limitations | Verdict |
|---|---|---|---|
| Parsing | golang.Extract uses go/packages, AST and types. lang uses pure-Go structural parsing for Python, JS/TS and Vue. |
Go has typed declarations and partial recovery. Structural parsers are portable but do not provide compiler semantics or complete syntax diagnostics. Rust, Java and TypeScript declaration files are not indexed as source. | Keep native Go; upgrade structural diagnostics and measured hot paths. Consider a Tree-sitter adapter only for agreed language/editor requirements. |
| Indexing | Go static calls and interface candidates feed shared model, stores and identity.Reconcile. ingest.Run rebuilds the graph. |
Durable identities preserve evidence across revisions. Interface candidates can include implementations never instantiated; function-variable invocations are omitted. A one-file edit still performs full extraction. | Keep graph contracts and identity evidence; upgrade edge provenance before adding optional SSA or external indexes. Stable IDs alone do not provide incremental parsing. |
| Diffing and patching | Hardened gitx.Diff / ParseDiff, review.Analyze, full-file model replacement in fixes. |
Isolated worktrees, protected paths, permission checks and constrained pushes exist. Unified diff parsing misses rename-only changes; rewrite validation does not establish syntax, revision agreement or preservation of unrelated code. | Keep Git/worktree protections; upgrade metadata parsing and validation before changing edit representation. |
| Context tracing | agentctx selects exact identifiers and bounded graph evidence; budget.go counts serialized JSON using ceil(bytes/4). recall combines BM25, local vectors and Go RRF. |
Repository/session fencing, current graph hydration, omissions and source ranges are explicit. Estimates are not model token counts. Registry.Complete redacts and caps output but does not enforce MaxInputTokens against the final prompt. |
Keep semantic selection and reported billing usage; upgrade final-input preflight and profiled in-memory search. |
Reproduced correctness gaps¶
The TestAudit* cases document current limitations, including undesirable behavior. They are assessment baselines, not acceptance of that behavior: the follow-up fixes must replace those expectations with explicit rejection or correct recovery.
- Malformed Go: incomplete declarations and bodies retain useful symbols and valid siblings. Package diagnostics mark affected file stages failed, even when partial symbols survive. This is useful batch recovery, not a live-editor guarantee.
- Malformed Python/TypeScript: partial declarations may survive, but an unterminated Python multiline string or TypeScript block can hide later declarations without syntax diagnostics. A foreign
save()receiver can resolve to an unrelated sole local method by name. Such edges need an inferred-confidence label. - Call precision: typed static and generic calls resolve. Function-variable calls do not; assigned function references remain. Interface implementation edges are candidates rather than proven runtime targets.
- Diff metadata: rename-only input produces no changes. Git-quoted tab paths remain escaped instead of matching filesystem paths. Repeated equal text in distinct hunks retains separate line positions. An oversized line fails the whole parse; the existing comment describes different behavior. Hunk arithmetic also needs checked bounds (source inspection, not a measured overflow exploit).
- Generated edits: invalid Go, removed copyright and unrelated function replacement pass the current plausibility check. Multiple fenced blocks can be concatenated with intervening prose. A malformed rewrite and a review/base revision mismatch both reach the test runner's fake push callback. These tests use local fixtures and fake providers/pushes; no branch is published by them.
- Prompt limits: callers use byte approximations, and some task assembly accepts a first oversized part. Central output limits do not prevent an oversized final input. Provider-reported token usage and configured-price accounting already distinguish known from unknown usage and should remain authoritative for billing.
Evidence is in golang/audit_test.go, lang/audit_test.go, audit_diff_test.go and audit_rewrite_test.go.
2. Measurements and practical limits¶
Raw artifacts and reproduction commands record corpus hashes, policy fingerprints and harness hashes. Hardware: Apple M5 Pro, 48 GiB RAM, darwin/arm64, Go 1.27.3, Git 2.56.0. Runs use one request at a time, without the race detector. Indexing/context/retrieval use GOMAXPROCS=4; parser/diff microbenchmarks use the machine default of 18. Numbers from different harnesses must not be combined into an end-to-end estimate.
Full extraction and publication to the memory adapter¶
The generated repositories contain Go, Python, TypeScript, YAML and Markdown. Twenty samples per unchanged/edited series give nearest-rank p50/p95. Prior identity evidence is retained; extraction and graph replacement are repeated. No policy file means an empty effective policy digest and no configured exclusions.
| Corpus | First run, ms | Unchanged p50 / p95, ms | One-file edit p50 / p95, ms | Mean allocated per unchanged run |
|---|---|---|---|---|
| 51 files, 4,939 bytes, 224 nodes / 223 edges | 99.40 | 58.78 / 65.28 | 63.58 / 76.32 | 9.96 MB |
| 501 files, 50,209 bytes, 2,204 nodes / 2,203 edges | 155.97 | 124.39 / 131.11 | 130.34 / 167.66 | 96.02 MB |
These are small synthetic source bodies, even in the larger file-count case. First-run filesystem/toolchain cache state is unknown; it is not a cold-start benchmark. The profiled test process reached 59,621,376 bytes maximum RSS (56.9 MiB). Allocation counters exclude go list child heaps; this RSS observation is not an aggregate child/native-memory capacity measurement. Remote graph stores, authentication, models and network publication are excluded.
Across all 82 runs, sampled allocation space was 4,313 MB, including Graph.Nodes at 27.79% flat, docs.addDocument at 6.42% flat, memory ReplaceRepo at 20.93% cumulative and identity reconciliation at 6.72% cumulative. These are cumulative allocations, not live heap; the whole-process profile also includes fixture construction and graph assertions outside the timed section. The memory adapter copies data; its profile does not describe Neo4j/HugeGraph publication. Repeated graph materialization deserves a prototype before increasing workers.
Parser, diff and context microbenchmarks¶
| Workload | Observed time/op | Allocated bytes/op | Interpretation |
|---|---|---|---|
| Go extraction, 1,000 functions / 79,794 bytes | 42.99–45.02 ms | 12.69–12.70 MB | Includes package loading elapsed time; child allocations excluded |
| Python structural parse, 1,000 functions / 80,780 bytes | 18.63–25.11 ms | 1.24–1.26 MB | Excludes filesystem, graph publication and identity work |
| TypeScript structural parse, 1,000 functions / 95,780 bytes | 31.99–32.22 ms | 1.37 MB | Same narrow parser boundary; not comparable to complete Go extraction |
| Unified diff, 1,000 files | 167.66 µs | 440,373 | Median of three run means; Git process excluded |
| Rewrite envelope, 256 KiB | 271.56 µs | 753,669 | Parsing/plausibility only; no model, disk or push |
minimal_context, 100 common-prefix nodes |
2.187–2.320 ms | 3.39 MB | Exact target retained within 1,500 estimated tokens |
minimal_context, 10,000 common-prefix nodes |
97.035–101.066 ms | 219.60–219.67 MB | Broad-match stress case in memory adapter |
| Same context graph, escaping-heavy summaries | 102.596–108.546 ms | 220.57–220.64 MB | Structured budget still holds |
Ranges are benchmark run means, not request percentiles. The TypeScript CPU profile attributes 72.51% cumulative samples to regexp backtracking and 6.16% flat to the parser's line-owner search. A separate context profile at 18 processors attributes 98.34% cumulative allocation space to memory.Store.Search; repeated case folding and sorting are visible CPU costs. Profiles include one fixture setup, while timed benchmark loops exclude it. These findings support targeted parser/search changes, not a claim that CGo or a remote database would be faster. Concurrent contention and native parser memory were not measured.
Token reduction and retrieval¶
Reusing the phase 4f harness retained all required fixture evidence. Raw input was 3,108 estimated tokens and returned graph/MCP context was 4,523: (3108 - 4523) / 3108 * 100 = -45.53%, an expansion. Tiny files are smaller than the response metadata. There is no measured positive savings claim for this corpus.
The existing 5,000-name BM25 workload measured p50/p95/p99 of 4.587/4.799/4.851 ms over 100 samples. This remains below ADR 0001's 50 ms migration trigger. Successful-task warm totals span 0.0124–0.0989 ms at p95, but use in-process synthetic graph/vector inputs. They exclude authenticated HTTP, local embedding execution and remote fetches; they do not validate semantic embedding accuracy or a production SLA. Neo4j/HugeGraph lexical recall currently fetches the bounded namespace population before ranking in Go. Measure that fetch separately before deciding on native Milvus BM25/RRF.
3. Package comparisons¶
Primary upstream sources were checked on 2026-10-23. Candidate libraries were not installed or benchmarked. Integration effort below is relative scope, not a time estimate. Recheck pins, grammar licences and transitive dependencies before any approved prototype.
| Candidate | Benefits and limits | Maintenance/licence evidence | Effort and recommendation |
|---|---|---|---|
| Native Go AST/types and existing x/tools | Typed Go analysis; parser returns partial trees with errors. Does not parse other languages or provide incremental trees. | Go 1.27.3 / x/tools v0.52.0; BSD-3-Clause. Parser contract. | Low: keep; prototype SSA separately using the existing dependency. |
tree-sitter/go-tree-sitter |
Incremental syntax and error nodes; CGo/C11 runtime, explicit parser/tree ownership, separate grammar versions. Syntax alone does not resolve calls. | MIT; proxy resolves v0.24.0, tagged Go 1.22 module; repository unarchived, observed last push 2025-11-16. Tagged module, repository metadata. Rendered package documentation showed a conflicting newer version; verify a buildable pin before adoption. | High: preferred conditional multi-language syntax prototype, with native-resource and cross-build gates. |
smacker/go-tree-sitter |
Go bindings with bundled grammars and context-aware parsing; native allocations still need lifecycle management. | MIT, Go 1.20; proxy version v0.0.0-20240827094217-dd81d9e9be82; unarchived, last observed push 2024-08-27. Repository. Older activity is not proof of abandonment. |
High: comparison baseline; assess grammar update burden and API ownership against official bindings. |
| x/tools SSA, RTA and VTA | Better Go call analysis for selected workloads. RTA needs complete bodies and entry points; reflection and dynamic behavior remain incomplete. | Already pinned dependency; SSA, RTA, VTA (experimental API). | Medium: optional analysis mode with explicit edge provenance and build coverage. |
| SCIP | Portable definitions/references format; requires language-specific indexers and revision mapping. It is not a parser or live-editor server. | Project moved from sourcegraph to scip-code/scip; Apache-2.0, v0.10.0, observed September 2026 release and October activity. | High: start with bounded, local artifact import; do not execute arbitrary repository indexer commands. |
| LSP | Editor document versions and language-server semantics. Capabilities vary; long-lived external processes need isolation and cancellation. | LSP 3.17 specification; concrete servers have their own versions/licences. | High: only if live-editor requirements are confirmed; persist revision-specific graph evidence separately. |
sergi/go-diff v1.4.0 |
Pure-Go character diff and approximate patch matching. No AST semantics, revision validation or guaranteed comment preservation. | MIT; latest observed tag June 2025. Package, patch implementation. | Medium: optional text comparator; not a fix for current rewrite validation gaps. |
| Native AST/range validation; GumTree comparator | Exact byte ranges preserve untouched text; AST detects syntax/structural changes. Reprinting an AST need not preserve all formatting. GumTree computes structural matches, not safe application. | Go CommentMap; GumTree v4.1.0, Java 17 and LGPL-3.0. | Medium for Go validation; high for cross-language tree edits. GumTree is unsuitable for the planned permissive production path without licence review; optional offline research only. |
pkoukk/tiktoken-go v0.1.8 |
OpenAI BPE encoding coverage; default vocabulary loading downloads/cache assets. Offline loader requires explicit pinned assets. Not an exact Claude/local tokenizer. | MIT, Go 1.19; September 2025 release, tagged README. | Medium: tokenizer comparison baseline with offline assets. |
tiktoken-go/tokenizer v0.10.0 |
Pure Go, embedded vocabulary maps (~4 MB per README), count API and newer OpenAI mappings. Special-token handling remains incomplete. | MIT, Go 1.26; October 2026 release, tagged README. | Medium: preferred offline counting prototype for supported encodings; no measured speed superiority. |
tmc/langchaingo v0.1.15 |
TokenSplitter wraps pkoukk with generic overlapping windows. It does not replace graph scopes, session checks or redaction. Importing one package need not link the entire module. | MIT, Go 1.24.4; October 2026 release, splitter. | High for orchestration replacement: keep existing Registry/recall; do not adopt a framework solely for counting. |
4. Reviewable upgrade roadmap¶
Each row is a separate proposed implementation PR, with tests, documentation, commit and push before the next phase. Dependency approval is still required. Phase 4j itself changes no production interface or graph schema.
| Order | Scope and adapter boundary | Acceptance and comparative evidence | Rollout, migration and rollback |
|---|---|---|---|
| 4j.1, implemented | Harden existing gitx.ParseDiff and fixes.Run first. Parse Git extended headers/quoted names, check arithmetic/output limits, compare review/base/file hashes, accept one explicit rewrite envelope and validate supported syntax plus protected headers/unrelated edits. |
Turn audit gap fixtures into rejection/correctness tests. Include rename/move, repeated code, deletion-only hunks, CRLF, Unicode, oversized output, stale revisions and invalid syntax; rejection must occur before commit/push callbacks. Keep subprocess execution restrictions. Git format reference. | No graph schema change. Version queued request and commit evidence; fail old or mismatched jobs for review. Checks are enforced whenever the existing opt-in fix feature is enabled; rollback may disable automated fixes, never restore silent unsafe publication. |
| 4j.2, implemented estimate policy | Central final-prompt preflight in llm.Registry, using a small counter adapter with model/encoding/version and exact/estimated status. Count the final redacted envelope, including provider-specific message/tool framing for supported templates; retain byte limits. |
Unicode, code, escaping, special-token literals, long names, unknown models, output reservation and oversized first-part tests. Compare against provider/local tokenizer fixtures. Benchmark cold init, steady throughput, resident memory and concurrency; prohibit runtime vocabulary downloads. | The estimate policy is enforced now; verified exact adapters require a separate shadow comparison and supported-model policy. Unknown encodings require a documented estimated fallback, which cannot guarantee the model window, or refusal when a hard guarantee is required; never pretend an OpenAI encoding is exact for another model. Keep provider-reported billing usage. No graph migration. |
| 4j.3, implemented | Optimize memory.Store.Search using compact candidates, precomputed case-fold keys and bounded selection; reduce repeated graph copies only after an ownership review. Profile structural regexp/line-owner searches separately. |
Preserve exact-first deterministic ordering and repo fences under store contract/race tests. Compare common-prefix, no-match, mixed-kind and parser fixtures. Require a reproducible allocation/latency improvement without material small-input regression; set numeric budgets after target workloads are agreed. Add cancellation and concurrent contention measurements. | Same public store interface. Replace implementation per adapter behind existing contracts; retain old implementation for benchmark comparison. Remote adapters change only with their own profiles. No cross-tenant cache. |
| 4j.4, implemented without SSA | Add parse-completeness and edge-evidence fields before optional SSA. Distinguish typed, syntactic and name-inferred results; bound work per package and expose omissions. | Partial-code corpus, function variables, interfaces, generics and build tags; compare precision/recall against manually labelled fixtures. SSA build must retain offline go list defaults and cancellation. |
Version metadata in model/nodeprops/HugeGraph schema and API types, cover all store contracts. Legacy evidence is unknown, never promoted to typed. Additive readers tolerate mixed versions; rebuild projections with stable identity reconciliation. Roll back by disabling the optional analysis mode. |
| 4j.5, conditional | Prototype one Tree-sitter syntax adapter for an agreed unsupported language or partial-buffer workflow. Keep typed Go analysis. Use one parser per bounded worker; close trees/parsers and bound retained edits. | Compare malformed-code recovery, symbol ranges/comments, incremental edits and full parses. Measure CGo crossings, native RSS, allocator pressure, cancellation, races and cross-compilation. Verify runtime/grammar ABI and licence pins. | Explicit opt-in, shadow graph comparison, parser-version evidence and no uncertain identity merges. Approval before dependencies. On failure use the existing language path or report unsupported; do not silently mix parser revisions. |
| 4j.6, conditional | Import bounded SCIP artifacts through a semantic adapter; LSP only for confirmed editor needs. Later incremental indexing uses a dependency-aware invalidation planner. | Validate source revision, path confinement, language/indexer versions, imports/build tags/module changes, deletes/moves and policy digests. Compare full rebuild to incremental outputs and identity history. Test cancellation midway through publication. | Pin artifacts to repository revisions and reject stale evidence. Changed policy/toolchain/parser versions invalidate relevant caches. Atomic replacement retains prior committed graph on failure. Disable import/incremental mode to return to full rebuild. |
| 4j.7, conditional | Replace whole-file rewrites with versioned edit operations carrying base hash, symbol/range and exact old bytes. Structural diff is an optional validator, never authority to guess a match. | Reject ambiguous or overlapping edits; preserve bytes outside approved ranges, comments and licences. Compare exact-range and structural approaches across supported languages before adding libraries. Test replay, stale files and partial application rollback. | Separate worktree, human-reviewed draft PR and current auth checks remain. Version queued payloads; unknown versions fail closed. Keep whole-file mode only behind equivalent validation, or disable automated fixes. |
Before promising a service SLA, run representative repositories and concurrent tenant workloads through authenticated API/MCP, Neo4j/HugeGraph publication, local embedding, vector search, RRF and hydration. Measure p50/p95, cold initialization, aggregate process/native peaks and lock contention separately. The no-migration option remains valid if measured workloads meet agreed limits. Kubernetes work stays in the final deployment phase; bootstrap-root separation remains pending independently of this audit, followed by the existing SSO roadmap.
5. MCP capability and reuse assessment¶
| Capability | Observed in this agent session | Consequence |
|---|---|---|
| CodeSynapse symbol/context/export discovery | No CodeSynapse tools registered; .mcp.json names the HTTP URL |
No live symbol query or graph freshness claim |
| Transport/process health | API :8080/healthz and MCP :8090/healthz returned 200 |
Health alone does not establish authenticated tool execution |
| Personal key and immutable runtime identity | Required environment values absent from this shell | Do not invent sessions, reuse another chat's identity or extract credentials from files |
| Git history/status/diff | Local Git available; no complete codebase Git MCP workflow established | Read-only local inspection used; Git provenance remains inspectable |
| AST/reuse evidence | Local source search, tests and offline graph review | Adequate for assessment/test additions; disclose fallback before major production changes |
Reconnect the existing CodeSynapse HTTP integration with the supported runtime identity bridge before major graph-assisted work. An optional official Git MCP server exposes status/log/diff and also write tools; use a trusted repository restriction and client-enforced tool allowlist if read-only access is intended. Its documentation describes early development, so verify the installed version and capabilities. Do not assume a built-in read-only switch or install it implicitly. CodeSynapse supplies the symbol/context side; an external LSP-backed service needs a separate capability, licence and data-access review.
Worked reuse example: the indexing harness searches and reuses policyFiles, ingest.Run, ingestreport.New/Prepare/Finish, identity reconciliation and memory ReplaceRepo; it does not introduce a second extraction pipeline. Context tests reuse agentctx.Service and the existing phase 4f retrieval harness. Diff tests reuse ParseDiff, fixContext, setup and the fake push callback. These modules were queried locally, not via MCP.
The agent guide now specifies health/freshness checks, search before implementation, reuse and the unavailable-tool path. Its acceptance scenarios cover missing tools, unhealthy services, stale evidence, missing identity and foreign scope. Current environment observation exercises the missing-tools/identity path; the other connection scenarios are manual acceptance procedures, not claimed live results. Existing TestRuntimeHTTPBindingAndRevocation covers missing/forged/cross-tenant identity and revocation in an isolated MCP test server.