Mantis Dedupe

Deduplicator (/mantis-dedupe)

System Goal

Duplicate Finding Merger. Evaluates lists of raw findings to cluster and consolidate identical or highly overlapping issues into singular, descriptive records.

Command Definition

• Command: /mantis-dedupe
• Description: Consolidates raw security findings to eliminate redundant reports.
• Arguments (optional; supplied by the orchestrator, consumed by Block A): --snapshot_root/--snapshot_id/--state_root. All absent -> MODE-OFF/legacy mode (behaves as today; snapshot gating disabled).

Input/Output Contract

• Reads:
• workspace/findings/ (raw finding JSON files, ignoring .trash/).
• workspace/archive/findings_pass_*/*.json and workspace/archive/loop*_findings/*.json (to skip findings already evaluated and triaged in previous passes).
• workspace/.mantis_state.json (to track current loop pass).
• Writes:
• Moves duplicate findings to workspace/findings/.trash/ after setting "status": "DUPLICATE" and "duplicate_of".
• Sets "possible_duplicate_of" (soft, non-terminal) on NOT_MATCHED matches and stamps "discovery_commit" on current findings that lack it. Reads active_snapshot/snapshot_pinned from .mantis_state.json.
• Appends transaction logs to workspace/.tx_log.jsonl.
• Generates/executes merging script workspace/helpers/merge_findings.py.
• Updates primary finding workspace/findings/<primary_id>.json (merges fields and history).
• Preconditions:
• workspace/findings/ must exist and contain finding files.
• Idempotency Guarantee:
• Cross-references against archived findings in workspace/archive/ to filter out any findings already processed in previous passes of this run. Snapshot-gated: a resolved archived finding on a differing snapshot is flagged as POSSIBLE REGRESSION (never silently filtered). Logs transactions to workspace/.tx_log.jsonl to support tracking and potential rollbacks. Deterministic merging rules implemented in merge_findings.py.

Instructions

Step 0: Locator Resolution (run first)

LOCATOR RESOLUTION (before reading ANY target code or artifact):
0. ROLE: If this skill NEVER reads target source (report, calibrate, reflect),
   you are a FINDINGS-ONLY stage: skip steps 2-6; still read active_snapshot from
   state for provenance/annotation; NEVER stop merely because a code root is unset.
1. Determine CODE_ROOT, in this priority order:
   a. If --target_root is passed on THIS invocation, CODE_ROOT = --target_root.
      It is AUTHORITATIVE and OVERRIDES SNAPSHOT_ROOT and the state fallback
      (used when a caller hands you a prepared tree, e.g. a patched shadow).
   b. Else if --snapshot_root (or SNAPSHOT_ROOT) is passed, use it.
   c. Else read state_root/workspace/.mantis_state.json (state_root from
      --state_root if passed, else ./workspace/... relative to the current dir)
      -> active_snapshot.root / .snapshot_id / .snapshot_pinned.
   d. Else (no arg AND no readable active_snapshot): CODE_ROOT = current directory,
      treat snapshot_pinned = false (MODE-OFF). Do NOT stop.
2. SENTINEL CHECK (only if snapshot_pinned is true AND you did NOT take path 1a):
   verify CODE_ROOT/.mantis_snapshot_id exists and equals SNAPSHOT_ID. If missing
   or different -> STOP "snapshot sentinel mismatch". (A --target_root tree (1a) is
   deliberately mutated and is sentinel-EXEMPT.)
3. PATH FIELDS:
   - SNAPSHOT-RELATIVE (read under CODE_ROOT): code_paths entries; plan target_files
     that are file paths. Strip ONLY a trailing ":<digits>". A code_paths entry
     containing "://" is a URL/endpoint, NOT a file read. A code_paths entry that is
     NOT of the form <existing-path>:<integer> is a non-source LOCATOR
     (symbol/offset/endpoint): only check that the artifact/symbol exists; skip ALL
     line-range and line-existence logic.
   - STATE-RELATIVE (read/write under state_root/workspace, NEVER prefix CODE_ROOT):
     kb_references, repro_file_path, reattack_file_path, helper scripts, report
     files, and all state/findings JSON.
4. Never WRITE under CODE_ROOT when snapshot_pinned is true. Any command that
   compiles, generates, or writes artifacts MUST run in a PRIVATE SHADOW copy
   (mktemp -d from CODE_ROOT), never with cwd=CODE_ROOT. Read-only inspection may
   cd into CODE_ROOT.
5. VCS-METADATA CARVE-OUT: history-log extraction and any VCS diff/blame command
   run in the LIVE repository root (which still has .git/.hg/.repo), NOT CODE_ROOT
   (the snapshot copy strips VCS metadata). Do NOT stop merely because CODE_ROOT
   lacks .git/.hg/.repo.
6. Every shell command uses ABSOLUTE paths and sets its own working directory on
   that call. Do NOT assume the working directory persists between calls.

[!NOTE] CURRENT-PASS CHECK (defensive; the binding guarantee is on the harness per `mantis-pipeline-adapter` Scenario 2): if active_snapshot is present AND active_snapshot.pass != state.pass_number, treat the snapshot as STALE for this pass — STOP "stale active_snapshot: pass mismatch" or degrade as HALT (snapshot_pinned effectively false: no authoritative verdicts, Block B NOT_MATCHED, reproduce not_attempted). This catches a custom harness that preserved active_snapshot across the Stage 15 pass increment without re-pinning. The reference meta-agent re-pins every pass, so this check never fires there. Block B itself cannot detect this (it is snapshot_id-only, not pass-aware).

Notes: workspace/findings/, workspace/archive/, workspace/.tx_log.jsonl, workspace/helpers/ and .mantis_state.json are STATE-RELATIVE (under --state_root). Any code snippet you inspect for a finding is SNAPSHOT-RELATIVE (under CODE_ROOT). Never write under CODE_ROOT.

Review a list of security findings and merge duplicate findings that refer to the exact same security flaw or adjacent code paths.

Execute your task as follows:

1. Load Raw Findings & Archived Findings Queue:

• List the contents of the directory and read the files in workspace/findings/. If the directory is empty or does not exist, notify the user and exit.
• Important: Ignore hidden files and directories (such as the .trash/ subdirectory) when listing or processing findings.
• Locate and load all archived finding JSON files from previous loop passes, if they exist, under workspace/archive/findings_pass_*/*.json and workspace/archive/loop*_findings/*.json. These files represent vulnerabilities that have already been fully evaluated, triaged, and potentially patched in previous passes.
• Important: Do NOT read or deduplicate against workspace/historical_learnings.jsonl (VCS history), as we want to catch regressions if old bugs were reintroduced.

1. Filter Loop Duplicates (snapshot-gated). First, stamp discovery_commit on any CURRENT finding that lacks it, using active_snapshot.snapshot_id from .mantis_state.json (skip when unpinned). Then, for each current finding that matches an archived finding (by code_paths+title similarity), run:

Signature-based candidate matching (Phase 3) — TIGHTENS, never replaces: signature may only PROMOTE a pair to "candidate for the pairwise snapshot check"; it may NEVER by itself cause a hard DUPLICATE/trash. A pair is a candidate for the Pairwise Snapshot Match Check below ONLY if it satisfies BOTH:

• it matches under today's code_paths + title similarity, comparing code_paths entries line-inclusively (WITH their trailing :line); AND
• (when both findings have a signature) their signature fields are equal. A signature match WITHOUT the code_paths + title agreement is NOT a duplicate — at most a soft possible_duplicate_of (keep the finding ACTIVE), never a trash. Rationale: signature strips the line number and