Performance Benchmark Specialist

Performance Benchmark Specialist

Comprehensive performance benchmarking expertise for shell-based tools, focusing on rigorous measurement, statistical analysis, and actionable performance optimization using patterns from the unix-goto project.

When to Use This Skill

Use this skill when:

• Creating performance benchmarks for shell scripts
• Measuring and validating performance targets
• Implementing statistical analysis for benchmark results
• Designing benchmark workspaces and test environments
• Generating performance reports with min/max/mean/median/stddev
• Comparing baseline vs optimized performance
• Storing benchmark results in CSV format
• Validating performance regressions
• Optimizing shell script performance

Do NOT use this skill for:

• Application profiling (use language-specific profilers)
• Production performance monitoring (use APM tools)
• Load testing web services (use JMeter, k6, etc.)
• Simple timing measurements (use basic time command)

Core Performance Philosophy

Performance-First Development

Performance is NOT an afterthought - it's a core requirement from day one.

unix-goto Performance Principles:

1. Define targets BEFORE implementation
2. Measure EVERYTHING that matters
3. Use statistical analysis, not single runs
4. Test at realistic scale
5. Validate against targets automatically

Performance Targets from unix-goto

Achieved Results:

• Cached navigation: 26ms (74ms under target)
• Cache build: 3-5s (meets target)
• Cache hit rate: 92-95% (exceeds target)
• Speedup ratio: 8x (work in progress to reach 20x)

Core Knowledge

Standard Benchmark Structure

Every benchmark follows this exact structure:

#!/bin/bash
# Benchmark: [Description]

# ============================================
# Setup
# ============================================

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_DIR="$SCRIPT_DIR/.."

source "$SCRIPT_DIR/bench-helpers.sh"
source "$REPO_DIR/lib/module.sh"

# ============================================
# Main Function
# ============================================

main() {
    bench_header "Benchmark Title"

    echo "Configuration:"
    echo "  Iterations: 10"
    echo "  Warmup: 3 runs"
    echo "  Workspace: medium (50 folders)"
    echo ""

    benchmark_feature

    generate_summary
}

# ============================================
# Benchmark Function
# ============================================

benchmark_feature() {
    bench_section "Benchmark Section"

    # Setup
    local workspace=$(bench_create_workspace "medium")

    # Phase 1: Baseline
    echo "Phase 1: Baseline measurement"
    echo "─────────────────────────────────"

    # Warmup
    bench_warmup "command" 3

    # Run benchmark
    local baseline_stats=$(bench_run "baseline" "command" 10)

    # Extract statistics
    IFS=',' read -r min max mean median stddev <<< "$baseline_stats"

    # Print results
    bench_print_stats "$baseline_stats" "Baseline Results"

    # Phase 2: Optimized
    echo ""
    echo "Phase 2: Optimized measurement"
    echo "─────────────────────────────────"

    # Implementation of optimization
    optimize_feature

    # Warmup
    bench_warmup "optimized_command" 3

    # Run benchmark
    local optimized_stats=$(bench_run "optimized" "optimized_command" 10)

    # Extract statistics
    IFS=',' read -r opt_min opt_max opt_mean opt_median opt_stddev <<< "$optimized_stats"

    # Print results
    bench_print_stats "$optimized_stats" "Optimized Results"

    # Compare and analyze
    local speedup=$(bench_compare "$mean" "$opt_mean")
    echo ""
    echo "Performance Analysis:"
    echo "  Speedup ratio:                               ${speedup}x"

    # Assert targets
    bench_assert_target "$opt_mean" 100 "Optimized performance"

    # Save results
    bench_save_result "benchmark_name" "$baseline_stats" "baseline"
    bench_save_result "benchmark_name" "$optimized_stats" "optimized"

    # Cleanup
    bench_cleanup_workspace "$workspace"
}

# ============================================
# Execute
# ============================================

main
exit 0

Benchmark Helper Library

The complete helper library provides ALL benchmarking utilities:

#!/bin/bash
# bench-helpers.sh - Comprehensive benchmark utilities

# ============================================
# Configuration
# ============================================

BENCH_RESULTS_DIR="${BENCH_RESULTS_DIR:-$HOME/.goto_benchmarks}"
BENCH_WARMUP_ITERATIONS="${BENCH_WARMUP_ITERATIONS:-3}"
BENCH_DEFAULT_ITERATIONS="${BENCH_DEFAULT_ITERATIONS:-10}"

# ============================================
# Timing Functions
# ============================================

# High-precision timing in milliseconds
bench_time_ms() {
    local cmd="$*"
    local start=$(date +%s%N)
    eval "$cmd" > /dev/null 2>&1
    local end=$(date +%s%N)
    echo $(((end - start) / 1000000))
}

# Warmup iterations to reduce noise
bench_warmup() {
    local cmd="$1"
    local iterations="${2:-$BENCH_WARMUP_ITERATIONS}"

    for i in $(seq 1 $iterations); do
        eval "$cmd" > /dev/null 2>&1
    done
}

# Run benchmark with N iterations
bench_run() {
    local name="$1"
    local cmd="$2"
    local iterations="${3:-$BENCH_DEFAULT_ITERATIONS}"

    local times=()

    for i in $(seq 1 $iterations); do
        local time=$(bench_time_ms "$cmd")
        times+=("$time")
        printf "  Run %2d: %dms\n" "$i" "$time"
    done

    bench_calculate_stats "${times[@]}"
}

# ============================================
# Statistical Analysis
# ============================================

# Calculate comprehensive statistics
bench_calculate_stats() {
    local values=("$@")
    local count=${#values[@]}

    if [ $count -eq 0 ]; then
        echo "0,0,0,0,0"
        return 1
    fi

    # Sort values for percentile calculations
    IFS=$'\n' sorted=($(sort -n <<<"${values[*]}"))
    unset IFS

    # Min and Max
    local min=${sorted[0]}
    local max=${sorted[$((count-1))]}

    # Mean (average)
    local sum=0
    for val in "${values[@]}"; do
        sum=$((sum + val))
    done
    local mean=$((sum / count))

    # Median (50th percentile)
    local mid=$((count / 2))
    if [ $((count % 2)) -eq 0 ]; then
        # Even number of values - average the two middle values
        local median=$(( (${sorted[$mid-1]} + ${sorted[$mid]}) / 2 ))
    else
        # Odd number of values - take the middle value
        local median=${sorted[$mid]}
    fi

    # Standard deviation
    local variance=0
    for val in "${values[@]}"; do
        local diff=$((val - mean))
        variance=$((variance + diff * diff))
    done
    variance=$((variance / count))

    # Use bc for square root if available, otherwise approximate
    if command -v bc &> /dev/null; then
        local stddev=$(echo "scale=2; sqrt($variance)" | bc)
    else
        # Simple approximation without bc
        local stddev=$(awk "BEGIN {printf \"%.2f\", sqrt($variance)}")
    fi

    # Return CSV format: min,max,mean,median,stddev
    echo "$min,$max,$mean,$median,$stddev"
}

# Compare two measurements and calculate speedup
bench_compare() {
    local baseline="$1"
    local optimized="$2"

    if [ "$optimized" -eq 0 ]; then
        echo "inf"
        return
    fi

    if command -v bc &> /dev/null; then
        local speedup=$(echo "scale=2; $baseline / $optimized" | bc)
    else
        local speedup=$(awk "BEGIN {printf \"%.2f\", $baseline / $optimized}")
    fi

    echo "$speedup"
}

# Calculate percentile
bench_percentile() {
    local percentile="$1"