Video Processing Editing

Video Processing & Editing

Expert in FFmpeg-based video editing, processing automation, and export optimization for modern content creation workflows.

When to Use

✅ Use for:

• Automated video editing pipelines (script-to-video)
• Cutting, trimming, concatenating clips
• Adding transitions, effects, overlays
• Audio mixing and normalization
• Subtitle/caption handling
• Export optimization for platforms
• Batch video processing
• Color grading and correction

❌ NOT for:

• Real-time video editing UI (use DaVinci Resolve/Premiere)
• 3D compositing (use After Effects/Blender)
• Motion graphics animation (use After Effects)
• Basic screen recording (use OBS)

Technology Selection

Video Editing Tools

Decision tree:

Need automation? → FFmpeg
Need Python API? → MoviePy
Need frame-level control? → PyAV
Need manual editing? → DaVinci Resolve

Common Anti-Patterns

Anti-Pattern 1: Not Using Keyframe-Aligned Cuts

Novice thinking: "Just cut the video at any timestamp"

Problem: Causes artifacts, black frames, and playback issues.

Wrong approach:

# ❌ Cut at arbitrary timestamp (not keyframe-aligned)
ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 -c copy output.mp4

# Result: Black frames, artifacts, sync issues

Why wrong:

• Video codecs use keyframes (I-frames) every 2-10 seconds
• Non-keyframe cuts require re-encoding
• Using -c copy (stream copy) without keyframe alignment breaks playback
• GOP (Group of Pictures) structure depends on keyframes

Correct approach 1: Re-encode for precise cuts

# ✅ Re-encode for frame-accurate cutting
ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a aac -b:a 192k \
  output.mp4

# Frame-accurate, but slower (re-encoding)

Correct approach 2: Keyframe-aligned stream copy

# ✅ Fast cutting with keyframe alignment
# Step 1: Find keyframes near cut points
ffprobe -select_streams v -show_frames -show_entries frame=pkt_pts_time,key_frame \
  -of csv input.mp4 | grep ",1$" | awk -F',' '{print $2}'

# Step 2: Cut at nearest keyframes (fast, no re-encoding)
ffmpeg -i input.mp4 -ss 00:01:22.000 -to 00:02:46.000 -c copy output.mp4

# Blazing fast, no quality loss, but not frame-accurate

Correct approach 3: Two-pass for best of both worlds

# ✅ Fast seek + precise cut
ffmpeg -ss 00:01:20.000 -i input.mp4 \
  -ss 00:00:03.456 -to 00:01:25.678 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a aac -b:a 192k \
  output.mp4

# -ss BEFORE -i: Fast seek to keyframe (no decode)
# -ss AFTER -i: Precise trim (only decode needed portion)

Performance comparison: | Method | Time (1-hour video) | Accuracy | Quality | |--------|---------------------|----------|---------| | Stream copy (arbitrary) | 2s | ❌ Broken | ❌ Artifacts | | Stream copy (keyframe) | 2s | ±2s | ✅ Perfect | | Re-encode (simple) | 15min | ✅ Frame | ⚠️ Quality loss | | Two-pass (optimal) | 3min | ✅ Frame | ✅ Perfect |

Timeline context:

• 2010: FFmpeg required full re-encoding for cuts
• 2015: -c copy added for stream copying
• 2020: Two-pass cutting became best practice
• 2024: Hardware acceleration (NVENC) makes re-encoding viable

Anti-Pattern 2: Re-encoding Unnecessarily

Novice thinking: "Apply all edits in one FFmpeg command"

Problem: Multiple re-encodings cause cumulative quality loss.

Wrong approach:

# ❌ Re-encode for each operation (quality degradation)
# Operation 1: Trim
ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 \
  -c:v libx264 -crf 23 temp1.mp4

# Operation 2: Add audio
ffmpeg -i temp1.mp4 -i audio.mp3 -c:v libx264 -crf 23 \
  -map 0:v -map 1:a temp2.mp4

# Operation 3: Add subtitles
ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \
  -c:v libx264 -crf 23 output.mp4

# Result: 3x re-encoding = significant quality loss

Why wrong:

• Each re-encode is lossy (even with high CRF)
• Cumulative quality loss (generation loss)
• 3x encoding time
• Wasted disk I/O

Correct approach 1: Chain operations in single command

# ✅ Single-pass encoding with all operations
ffmpeg -ss 00:01:00 -i input.mp4 -i audio.mp3 \
  -to 00:04:00 \
  -vf "subtitles=subs.srt" \
  -map 0:v -map 1:a \
  -c:v libx264 -crf 18 -preset medium \
  -c:a aac -b:a 192k \
  output.mp4

# Single re-encode, all operations applied at once

Correct approach 2: Use stream copy when possible

# ✅ Lossless operations with stream copy
# Trim (stream copy)
ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 -c copy temp.mp4

# Add audio (stream copy video, encode audio)
ffmpeg -i temp.mp4 -i audio.mp3 \
  -map 0:v -map 1:a \
  -c:v copy -c:a aac -b:a 192k \
  temp2.mp4

# Burn subtitles (must re-encode video)
ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \
  -c:v libx264 -crf 18 -preset medium \
  -c:a copy \
  output.mp4

# Only 1 video re-encode (for subtitles)

Quality comparison: | Method | Encoding Passes | Quality (VMAF) | Time | |--------|-----------------|----------------|------| | 3x re-encode (CRF 23) | 3 | 82/100 | 45min | | Single pass (CRF 23) | 1 | 91/100 | 15min | | Stream copy + 1 encode | 1 | 95/100 | 18min | | All stream copy | 0 | 100/100 | 30s |

Anti-Pattern 3: Ignoring Color Space Conversions

Novice thinking: "Just concatenate videos together"

Problem: Color shifts, mismatched brightness, broken playback.

Wrong approach:

# ❌ Concatenate videos with different color spaces
# clip1.mp4: BT.709 (HD), yuv420p
# clip2.mp4: BT.601 (SD), yuvj420p (full range)
# clip3.mp4: BT.2020 (HDR), yuv420p10le

# Create concat list
echo "file 'clip1.mp4'" > list.txt
echo "file 'clip2.mp4'" >> list.txt
echo "file 'clip3.mp4'" >> list.txt

# Concatenate without color normalization
ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

# Result: Color shifts between clips, broken HDR metadata

Why wrong:

• Different color spaces (BT.601 vs BT.709 vs BT.2020)
• Different pixel formats (yuv420p vs yuvj420p)
• Different color ranges (limited vs full)
• Metadata conflicts

Correct approach:

# ✅ Normalize color space before concatenation

# Step 1: Analyze color space of each clip
ffprobe -v error -select_streams v:0 \
  -show_entries stream=color_space,color_transfer,color_primaries,pix_fmt \
  -of default=noprint_wrappers=1 clip1.mp4

# Step 2: Normalize all clips to common color space
# Target: BT.709 (HD), yuv420p, limited range

# Normalize clip1 (already BT.709)
ffmpeg -i clip1.mp4 -c copy clip1_normalized.mp4

# Normalize clip2 (BT.601 SD → BT.709 HD)
ffmpeg -i clip2.mp4 \
  -vf "scale=in_range=full:out_range=limited,colorspace=bt709:iall=bt601:fast=1" \
  -color_primaries bt709 \
  -color_trc bt709 \
  -colorspace bt709 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a copy \
  clip2_normalized.mp4

# Normalize clip3 (BT.2020 HDR → BT.709 SDR)
ffmpeg -i clip3.mp4 \
  -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=limited,format=yuv420p" \
  -color_primaries bt709 \
  -color_trc bt709 \
  -colorspace bt709 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a copy \
  clip3_normalized.mp4

# Step 3: Concatenate normalized clips
echo "file 'clip1_normalized.mp4'" > list.txt
echo "file 'clip2_normalized.mp4'" >> list.txt
echo "file 'clip3_normalized.mp4'" >> list.txt

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

Color space guide: | Standard | Color Space | Transfer | Primaries | Use Case | |----------|-------------|----------|-----------|----------| | BT.601 | SD | bt470bg | bt470bg | Old SD content |