Anonymize Logs

Anonymize Logs

Sanitize customer-provided logs so they are safe to share or commit. Replace identity only; keep the original structure, delimiters, quoting, field order, and line breaks.

Input may be a file or a single syslog / KV / NDJSON line. Output must match that shape — do not pretty-print, wrap, truncate, or “fix” values as part of sanitization.

What you provide

Golden rule — never reformat the content

Only replace sensitive values. Do not touch anything else.

Parsers and anyone reviewing the log depend on exact whitespace, delimiters, quoting, and line structure.

• NDJSON (one JSON object per line): do not pretty-print, re-indent, or restructure. Each line must remain a single compact JSON object on one line.
• Syslog / CEF / key-value logs: do not add or remove spaces, change quoting, or normalize field order.
• Multiline logs: preserve line grouping exactly.
• Replace only the values that identify real people, systems, or organizations — preserve field names, delimiters, structural tokens, and everything else character-for-character.
• Do not add, remove, or rewrite a syslog PRI / RFC5424 header. Do not prefix a bare line with <134> (or any PRI) “to look more like syslog.”
• Truncation is never an anonymization strategy. Do not delete repeating fields, drop trailing KV pairs, or cut off the remainder of a line to “finish” sanitization. Use the two-pass map instead.

Workflow

Step 1 — Two-pass replacement

Do not replace as you read. Distinct originals that collapse to one placeholder, or a first-occurrence-only replace that leaves later copies, are both failures.

1. Scan the whole file/line and build a map: original value → placeholder for every sensitive token (including nested ones: URL query params, hex dwords inside compound IDs, path segments).
2. If two originals differ, they must get different placeholders. If they are identical, they must get the same placeholder.
3. Apply the map globally (all occurrences).
4. Scan again for leftovers (IPs inside URLs, hex IPs inside vendor IDs, hostnames that look like product names).

Cover at minimum:

• Authentication artifacts: API keys, bearer tokens, passwords, OAuth tokens, base64-encoded credentials, private keys and certs (PEM blocks, SSH private keys), TLS/SSH fingerprints (JA3/JA4 hashes, SSH host key fingerprints, certificate fingerprints), DHCP fingerprints, partial secrets (token_prefix, password_hash_prefix, hashed_token) — partial exposure still identifies the credential
• Personal identifiers: email addresses (including CC/BCC lists, delegate/owner/creator email variants), usernames, display names, employee IDs, phone numbers, principal names (e.g. user@tenant.onmicrosoft.com), email subjects and body text
• Organizational identifiers: company names, tenant IDs (including home_tenant_id, resource_tenant_id, aad_tenant_id variants), account IDs, subscription IDs, billing account IDs, org slugs embedded in paths or JSON fields, org unit paths (e.g. orgunit_path, org_unit_path), department names and IDs, cost center IDs, Windows SIDs (Security Identifiers) in pipe names, task names, or registry paths
• Infrastructure identifiers: internal hostnames, FQDNs, private IP addresses, MAC addresses, internal URLs (staging/prod hostnames, internal tool domains), cloud resource IDs (ARNs, S3 bucket names, GCP project IDs, Azure subscription/resource names), Kubernetes cluster names, node names, pod names, and namespace names, container names and IDs, database hostnames and names (including database.host, database.name, database_principal_name), Windows domain topology fields (domain controller hostnames, NT domain names, domain_controller_object_guid, domain_controller_object_sid)
• Device and hardware identifiers: serial numbers, hardware UUIDs, machine IDs, device UUIDs, BIOS/firmware version strings that are unique to a specific device, including values in device_id / serial / similar fields
• File system paths: process command lines, file paths, registry key paths, and log file paths that embed usernames, org names, or internal system structure (e.g. C:\Users\alice\, /home/bob/, HKLM\...\S-1-5-21-...)
• Connection strings: database URIs, Redis URLs, any connection string that includes credentials or internal hostnames
• Resource ownership: owner email, creator email, last-modified-by identity, delegate user email, assignee email, impersonator fields — any field that names a specific person as the actor on a resource
• Tracking identifiers: session IDs, request/correlation IDs, transaction IDs, or any long opaque string tied to a real entity
• Hash values: replace when they could be derived from sensitive input (password hashes, HMAC secrets) — preserve file hashes (MD5, SHA1, SHA256 of file content) and other content-addressable references (git SHAs, TLS cert hashes used as identifiers)
• Geographic specifics: precise GPS coordinates, real street addresses — city and country names in geo enrichment fields are generally safe to keep. City/site tokens in inventory or topology fields (location hierarchies, hostname building codes) are identifiers — replace those.

Also cover nested and compound identifiers (below).

Apply placeholder conventions and shape rules (see below) as you go.

Nested and compound identifiers

These are commonly missed. Include them in the map, including tokens nested inside URLs, compound IDs, and path segments.

IPv4 hidden in hex compound IDs. Some vendor IDs look like {0x00000001,0x1,0xcb007114,0x00000002}. A hex dword may be an IPv4 address (try both endiannesses). If you replace origin / src / dst IPs, decode each dword as IPv4. If either decoding is a plausible public/private IP and does not match the already-substituted origin, replace that dword with the hex encoding of the same documentation IP you used for origin (keep 0x + 8 hex digits). Keep the surrounding {0x…,0x…} shape; only the IP-bearing dword changes. Example: 0xcb007114 → 203.0.113.20 (big-endian); if origin was replaced with 198.51.100.10, use 0xc633640a.

Topology-encoded hostnames. Not a DNS name: SW-BLDG-01, CHI-FW-01, DC1-VPN-MGMT. In hostname / syslog-host / device-name fields, replace site/role/building codes. Do not replace vendor syslog message IDs like %NAME-6-302018 — those are product mnemonics, not hostnames. Placeholder: host-1.example.local or example-fw-01 matching original length class (short token vs FQDN).

Policy / ACL / access-group names. In policy, ACL, or access-group values, replace \d{8,14} date-like runs → 20240101000000 (same digit length); vs\d+ → vs01 (or drop vs/date suffix: example-policy); VLAN numbers in names → vlan100. Keep the token class (still looks like a policy name). Example: vlan200-20240115103000 → vlan100-20240101000000.

Interface names. ethN-MM.VLAN → eth1-01.100 (preserve eth + dotted subunit shape). IF-<org>-… / location words → IF-Example (keep an IF- prefix if present). Generic inside / outside / lan / wan may stay if they are interface roles, not site names. Prefer replacing when the token is a customer site or org.

Windows / SMB paths with org units. Replace directory components above the last two levels with generic segments (example-department, E01, EXAMPLE-UNIT). Keep depth, backslashes, and the final filename (Thumbs.db) if it is a generic OS file. Year folders like \2026\ can stay (not identifying).