Code Obfuscation Deobfuscation

SKILL: Code Obfuscation & Deobfuscation — Expert Analysis Playbook

AI LOAD INSTRUCTION: Expert techniques for identifying, classifying, and defeating code obfuscation in native binaries. Covers junk code, opaque predicates, SMC, control flow flattening, movfuscator, VM protectors (VMProtect/Themida/Code Virtualizer), string encryption, import hiding, and anti-disassembly tricks. Base models often conflate packing with obfuscation and miss the distinction between static and dynamic deobfuscation strategies.

0. RELATED ROUTING

• anti-debugging-techniques when the obfuscated binary also has anti-debug layers
• symbolic-execution-tools when using angr/Z3 for automated deobfuscation
• vm-and-bytecode-reverse for deep VM protector bytecode analysis

Quick identification picks

1. JUNK CODE & OPAQUE PREDICATES

1.1 Junk Code Insertion

Dead code that never affects program output, added to increase analysis time.

Identification:

• Instructions that write to registers/memory never read afterward
• Function calls whose return values are discarded and have no side effects
• Loops with invariant bounds that compute unused results

Removal strategy:

1. Compute def-use chains (IDA/Ghidra data flow analysis)
2. Mark instructions with no downstream use as dead
3. Verify removal doesn't change program behavior (trace comparison)

1.2 Opaque Predicates

Conditional branches where the condition is always true or always false, but this is non-obvious.

Deobfuscation:

• Abstract interpretation: prove the condition is constant
• Symbolic execution: Z3 proves ∀x: predicate(x) = True
• Pattern matching: recognize known opaque predicate families
• Dynamic: trace and observe the branch is never taken / always taken

import z3
x = z3.BitVec('x', 32)
s = z3.Solver()
s.add(x * (x + 1) % 2 != 0)
print(s.check())  # unsat → always true

2. SELF-MODIFYING CODE (SMC)

Runtime code patching: encrypted code is decrypted just before execution.

2.1 XOR Decryption Loop (Most Common)

lea esi, [encrypted_code]
mov ecx, code_length
mov al, xor_key
decrypt_loop:
    xor byte [esi], al
    inc esi
    loop decrypt_loop
    jmp encrypted_code  ; now decrypted

2.2 Analysis Strategy

1. Identify the decryption routine (look for XOR/ADD/SUB in loops writing to .text)
2. Set breakpoint AFTER the loop completes
3. At breakpoint: dump the decrypted memory region
4. Re-analyze the dumped code in IDA/Ghidra
5. For multi-layer: repeat for each decryption stage

2.3 Automated Unpacking via Emulation

from unicorn import *
from unicorn.x86_const import *

mu = Uc(UC_ARCH_X86, UC_MODE_32)
mu.mem_map(0x400000, 0x10000)
mu.mem_write(0x400000, binary_code)
mu.emu_start(decrypt_entry, decrypt_end)
decrypted = mu.mem_read(code_start, code_length)

3. CONTROL FLOW FLATTENING (CFF)

3.1 Structure

Original sequential blocks are transformed into a dispatcher loop:

Original:      A → B → C → D

Flattened:     ┌──────────────────┐
               │   dispatcher     │
               │   switch(state)  │◄─────┐
               ├──────────────────┤      │
               │ case 1: block A  │──────┤
               │ case 2: block B  │──────┤
               │ case 3: block C  │──────┤
               │ case 4: block D  │──────┘
               └──────────────────┘

Each block sets state = next_state before jumping back to the dispatcher.

3.2 Recovery Techniques

3.3 Symbolic Deflattening (angr approach)

import angr, claripy

proj = angr.Project('./obfuscated')
cfg = proj.analyses.CFGFast()

# Find dispatcher block (highest in-degree basic block)
dispatcher = max(cfg.graph.nodes(), key=lambda n: cfg.graph.in_degree(n))

# For each case block, symbolically determine successor
for block in case_blocks:
    state = proj.factory.blank_state(addr=block.addr)
    # ... solve state variable to find real successor

4. MOVFUSCATOR

4.1 Concept

All computation reduced to mov instructions only (Turing-complete via memory-mapped computation tables). Created by Christopher Domas.

4.2 Identification

• Function contains only mov instructions (no add, sub, xor, jmp, call)
• Large lookup tables in data section
• Memory-mapped flag registers

4.3 Demovfuscation

5. VM PROTECTION (VMProtect / Themida / Code Virtualizer)

5.1 VM Architecture

Protected code → bytecode compiler → custom bytecode
Runtime: VM entry (pushad/pushfd) → fetch → decode → execute → VM exit (popad/popfd)

5.2 VM Entry Point Identification

; Typical VMProtect entry
pushad                    ; save all registers
pushfd                    ; save flags
mov ebp, esp              ; VM stack frame
sub esp, VM_LOCALS_SIZE   ; allocate VM context
mov esi, bytecode_addr    ; bytecode instruction pointer
jmp vm_dispatcher         ; enter VM loop

5.3 Handler Table Extraction

1. Find dispatcher (large switch or indirect jump via table)
2. Each case/entry = one VM handler (implements one VM opcode)
3. Map handler addresses to operations by analyzing each handler:
   - Handler reads operand from bytecode stream (esi)
   - Performs operation on VM registers/stack
   - Advances bytecode pointer
   - Returns to dispatcher

5.4 Devirtualization Approaches

5.5 VMProtect Specifics

• Uses opaque predicates in dispatcher
• Handler mutation: same opcode, different handler code per build
• Multiple VM layers (VM inside VM)
• Integrates anti-debug and integrity checks

6. STRING ENCRYPTION

6.1 Common Patterns