Mantis Structural Index

Structural Code Index Builder

This is an optional first-class stage in the Pass Lifecycle Contract. It runs immediately after the snapshot is pinned (Block D), before the first code-reading analysis stage (summarize/architecture). It only needs CODE_ROOT \+ SNAPSHOT_ID and must not depend on architecture/KB.

System Goal

Structural Code Index Builder. Builds a content-addressed semantic-unit index from source code using capability-based per-partition backend selection, degrading gracefully to grep. Provides find_callers(symbol), get_function_boundary(file, line), and call-site awareness to improve LLM reasoning quality during discovery — supplementing (never replacing) grep-based call-site discovery with structural data. The index is composed from independent semantic compilation units, persisted as a manifest + SQLite catalog, and exposed through a bounded query helper.

Command Definition

• Command: /mantis-structural-index
• Description: Build a content-addressed semantic-unit index from source code under CODE_ROOT.
• Arguments (optional; supplied by the orchestrator, consumed by Block A): --snapshot_root/--snapshot_id/--state_root. All absent → MODE-OFF/legacy mode (reads source from the current directory, writes index to ./workspace/kb/structural_index/).

Input/Output Contract

• Reads:
• workspace/.mantis_state.json (to read active_snapshot for provenance checking and snapshot-aware rebuild logic).
• workspace/kb/structural_index/manifest.json (to check snapshot_id for reuse-on-match idempotency — primary check).
• workspace/kb/structural_index.jsonl (backward-compat provenance check if manifest absent).
• workspace/kb/structural_index/units/ (content-addressed cache for incremental unit reuse).
• workspace/kb/structural_index/native/ and sidecar provenance.json files (prebuilt index attachments and metadata manifests).
• CODE_ROOT source files (via generated helper script — the script parses all source files under CODE_ROOT).
• Writes:
• workspace/kb/structural_index/manifest.json (STATE-RELATIVE — atomic commit point, written LAST).
• workspace/kb/structural_index/catalog.sqlite (STATE-RELATIVE — query-optimized serving store).
• workspace/kb/structural_index/units/ (STATE-RELATIVE — content-addressed immutable unit outputs).
• workspace/kb/structural_index/shards/ (STATE-RELATIVE — partitioned serving data for large corpora).
• workspace/kb/structural_index/native/ (STATE-RELATIVE — prebuilt index attachments: SCIP, Kythe, LSIF).
• workspace/kb/structural_index/tmp/ (STATE-RELATIVE — temporary objects during build).
• workspace/kb/structural_index.jsonl (STATE-RELATIVE — compatibility pointer; full export below threshold).
• workspace/helpers/build_structural_index.py (STATE-RELATIVE — the builder helper script).
• workspace/helpers/query_structural_index.py (STATE-RELATIVE — the query helper script).
• Preconditions:
• Source files must exist under CODE_ROOT. If CODE_ROOT is not resolved (MODE-OFF and no readable active_snapshot), build against the current directory with snapshot_id set to "unknown". Do NOT skip — this is the standalone-efficiency case.
• Inert until wired: This skill returns an empty index until a caller invokes it (the harness, mantis-plan, or mantis-researcher). It never fails — it simply returns an empty index if tools are unavailable or source cannot be parsed.
• Idempotency Guarantee:
• Read-only on CODE_ROOT. Writes only to STATE-RELATIVE paths. Re-running with the same CODE_ROOT and SNAPSHOT_ID reuses the existing index (manifest snapshot_id match) rather than rebuilding — except in MODE-OFF, where it always rebuilds. Individual semantic units are reused across snapshots when their content-addressed cache keys match (incremental rebuild).

Instructions

Step 0: Locator Resolution (run first)

LOCATOR RESOLUTION (before reading ANY target code or artifact):
0. ROLE: If this skill NEVER reads target source (report, calibrate, reflect),
   you are a FINDINGS-ONLY stage: skip steps 2-6; still read active_snapshot from
   state for provenance/annotation; NEVER stop merely because a code root is unset.
1. Determine CODE_ROOT, in this priority order:
   a. If --target_root is passed on THIS invocation, CODE_ROOT = --target_root.
      It is AUTHORITATIVE and OVERRIDES SNAPSHOT_ROOT and the state fallback
      (used when a caller hands you a prepared tree, e.g. a patched shadow).
   b. Else if --snapshot_root (or SNAPSHOT_ROOT) is passed, use it.
   c. Else read state_root/workspace/.mantis_state.json (state_root from
      --state_root if passed, else ./workspace/... relative to the current dir)
      -> active_snapshot.root / .snapshot_id / .snapshot_pinned.
   d. Else (no arg AND no readable active_snapshot): CODE_ROOT = current directory,
      treat snapshot_pinned = false (MODE-OFF). Do NOT stop.
2. SENTINEL CHECK (only if snapshot_pinned is true AND you did NOT take path 1a):
   verify CODE_ROOT/.mantis_snapshot_id exists and equals SNAPSHOT_ID. If missing
   or different -> STOP "snapshot sentinel mismatch". (A --target_root tree (1a) is
   deliberately mutated and is sentinel-EXEMPT.)
3. PATH FIELDS:
   - SNAPSHOT-RELATIVE (read under CODE_ROOT): code_paths entries; plan target_files
     that are file paths. Strip ONLY a trailing ":<digits>". A code_paths entry
     containing "://" is a URL/endpoint, NOT a file read. A code_paths entry that is
     NOT of the form <existing-path>:<integer> is a non-source LOCATOR
     (symbol/offset/endpoint): only check that the artifact/symbol exists; skip ALL
     line-range and line-existence logic.
   - STATE-RELATIVE (read/write under state_root/workspace, NEVER prefix CODE_ROOT):
     kb_references, repro_file_path, reattack_file_path, helper scripts, report
     files, and all state/findings JSON.
4. Never WRITE under CODE_ROOT when snapshot_pinned is true. Any command that
   compiles, generates, or writes artifacts MUST run in a PRIVATE SHADOW copy
   (mktemp -d from CODE_ROOT), never with cwd=CODE_ROOT. Read-only inspection may
   cd into CODE_ROOT.
5. VCS-METADATA CARVE-OUT: history-log extraction and any VCS diff/blame command
   run in the LIVE repository root (which still has .git/.hg/.repo), NOT CODE_ROOT
   (the snapshot copy strips VCS metadata). Do NOT stop merely because CODE_ROOT
   lacks .git/.hg/.repo.
6. Every shell command uses ABSOLUTE paths and sets its own working directory on
   that call. Do NOT assume the working directory persists between calls.

This is a CODE-READING stage — it reads target source files under CODE_ROOT via the helper script. Block A step 0's findings-only skip does NOT apply.

Step 1: Idempotency / Freshness Check (MANDATORY FIRST)

MANDATORY FIRST STEP: Before writing any build scripts, probing backends, or extracting symbols, check for MODE-OFF or an existing manifest:

1. MODE-OFF check (FIRST): In MODE-OFF (SNAPSHOT_ID is "unknown" or absent) → always rebuild. Do NOT reuse a previous "unknown" index, because the live tree is mutable and "unknown" is a constant (not a freshness signal). Skip directly to Step 2 (individual units whose content has not changed may still hit the content-addressed cache).
2. If workspace/kb/structural_index/manifest.json exists, read its snapshot_id field.
3. If snapshot_id matches the current SNAPSHOT_ID (and SNAPSHOT_ID is NOT "unknown") → reuse the index immediately and STOP. Do NOT invoke Step 2, probe backends, or write build scripts. This bounds cost across retries/crash-resume.
4. If manifest.json is absent but workspace/kb/structural_index.jsonl exists (backward compat), read its provenance header (_provenance, snapshot_id