Video Toolkit

Video Toolkit

Create professional explainer videos from a text brief. The toolkit uses open-source AI models on cloud GPUs (Modal or RunPod) for voiceover, image generation, music, and talking head animation. Remotion (React) handles composition and rendering.

CRITICAL: Toolkit Path

The toolkit lives at a fixed path. ALWAYS `cd` here before running any tool command.

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT

NEVER run tool commands from inside a project directory. Tools resolve paths relative to the toolkit root.

CRITICAL: Progress Reporting

ALWAYS add `--progress json` to every cloud GPU tool command. This gives you structured JSON Lines on stderr so you can monitor job status, detect stuck jobs, and report progress to the user in real-time.

# CORRECT — always include --progress json
uv run tools/music_gen.py --preset corporate-bg --duration 60 --output bg.mp3 --progress json

# WRONG — no visibility into job status
uv run tools/music_gen.py --preset corporate-bg --duration 60 --output bg.mp3

Tools that support --progress json: music_gen.py, qwen3_tts.py, flux2.py, upscale.py, sadtalker.py, image_edit.py, dewatermark.py, ltx2.py, chain_video.py.

See the Progress Reporting section below for output format and stage definitions.

CRITICAL: Long-Running Tasks — Use yieldMs, Not background:true

Any tool command that takes more than 30 seconds MUST use `exec` with `yieldMs` so you can report progress to the user live. This includes: batch FLUX generation, chain_video, SadTalker, music generation, and any multi-scene pipeline.

exec command:"cd ~/.openclaw/workspace/claude-code-video-toolkit && uv run tools/chain_video.py --output-dir /path/ --progress json ..." yieldMs:10000

The polling loop:

1. exec with yieldMs:10000 starts the command and returns control to you every 10 seconds
2. Read the --progress json output — look for "stage":"item" (scene complete) or "stage":"complete" (all done)
3. Report progress to the user ("Scene 05/30 complete, 17%")
4. Poll again: process action:poll sessionId:<id>
5. Repeat until "stage":"complete"

Why: Your agent run ends when you finish responding. If you use bash background:true, you lose the ability to report progress — the user sees silence until they nudge you. With yieldMs, you stay in the loop.

NEVER do this:

• bash background:true command:"long running thing" then promise to "monitor" — you can't, your run ends
• Break a batch into individual tool calls across separate messages — your run ends between each one
• Promise to "continue autonomously" — you literally cannot without an external trigger

Setup

Step 1: Check Current State

cd ~/.openclaw/workspace/claude-code-video-toolkit
uv run tools/verify_setup.py

If everything shows [x], skip to "Quick Test" below. Otherwise continue setup.

Step 2: Install Python Dependencies

cd ~/.openclaw/workspace/claude-code-video-toolkit
uv sync

Note: uv sync creates its own .venv/ from the lockfile, so it sidesteps Debian/Ubuntu's managed-Python restrictions (PEP 668) — no --break-system-packages needed. If uv is missing, install it first: curl -LsSf https://astral.sh/uv/install.sh | sh.

Step 3: Configure Cloud GPU Endpoints

The toolkit needs cloud GPU endpoint URLs in .env. Check if .env exists and has Modal endpoints:

cat ~/.openclaw/workspace/claude-code-video-toolkit/.env | grep MODAL

If Modal endpoints are configured, you're ready. If not, ask the user to provide Modal endpoint URLs or set up Modal:

uv sync --extra modal
uv run modal setup   # Opens browser for authentication

# Deploy each tool — capture the endpoint URL from output
cd ~/.openclaw/workspace/claude-code-video-toolkit
uv run modal deploy docker/modal-qwen3-tts/app.py
uv run modal deploy docker/modal-flux2/app.py
uv run modal deploy docker/modal-music-gen/app.py
uv run modal deploy docker/modal-sadtalker/app.py
uv run modal deploy docker/modal-image-edit/app.py
uv run modal deploy docker/modal-upscale/app.py
uv run modal deploy docker/modal-propainter/app.py
uv run modal deploy docker/modal-ltx2/app.py      # Requires: uv run modal secret create huggingface-token HF_TOKEN=hf_...

LTX-2 prerequisite: Before deploying LTX-2, create a HuggingFace secret and accept the Gemma 3 license:

uv run modal secret create huggingface-token HF_TOKEN=hf_your_read_access_token

Add each URL to .env:

ACEMUSIC_API_KEY=...                          # Free key from acemusic.ai/api-key (best music quality)
MODAL_QWEN3_TTS_ENDPOINT_URL=https://...modal.run
MODAL_FLUX2_ENDPOINT_URL=https://...modal.run
MODAL_MUSIC_GEN_ENDPOINT_URL=https://...modal.run
MODAL_SADTALKER_ENDPOINT_URL=https://...modal.run
MODAL_IMAGE_EDIT_ENDPOINT_URL=https://...modal.run
MODAL_UPSCALE_ENDPOINT_URL=https://...modal.run
MODAL_DEWATERMARK_ENDPOINT_URL=https://...modal.run
MODAL_LTX2_ENDPOINT_URL=https://...modal.run

Optional but recommended — Cloudflare R2 for reliable file transfer:

R2_ACCOUNT_ID=...
R2_ACCESS_KEY_ID=...
R2_SECRET_ACCESS_KEY=...
R2_BUCKET_NAME=video-toolkit

Step 4: Verify and Quick Test

cd ~/.openclaw/workspace/claude-code-video-toolkit
uv run tools/verify_setup.py

All tools should show [x]. Then run a quick test to confirm the GPU pipeline works:

cd ~/.openclaw/workspace/claude-code-video-toolkit
uv run tools/qwen3_tts.py --text "Hello, this is a test." --speaker Ryan --tone warm --output /tmp/video-toolkit-test.mp3 --cloud modal

If you get a valid .mp3 file, setup is complete. If it fails, check:

• .env has the correct MODAL_QWEN3_TTS_ENDPOINT_URL
• Run uv run tools/verify_setup.py --json and check modal_tools for which endpoints are missing

Cost: Modal includes $30/month free compute. A typical 60s video costs $1-3.

Creating a Video

Step 1: Create Project

cd ~/.openclaw/workspace/claude-code-video-toolkit
cp -r templates/product-demo projects/PROJECT_NAME
cd projects/PROJECT_NAME
npm install

Templates: product-demo (marketing/explainer), sprint-review, sprint-review-v2 (composable scenes).

Step 2: Write Config

Edit projects/PROJECT_NAME/src/config/demo-config.ts:

export const demoConfig: ProductDemoConfig = {
  product: {
    name: 'My Product',
    tagline: 'What it does in one line',
    website: 'example.com',
  },
  scenes: [
    { type: 'title', durationSeconds: 9, content: { headline: '...', subheadline: '...' } },
    { type: 'problem', durationSeconds: 14, content: { headline: '...', problems: ['...', '...'] } },
    { type: 'solution', durationSeconds: 13, content: { headline: '...', highlights: ['...', '...'] } },
    { type: 'stats', durationSeconds: 12, content: { stats: [{value: '99%', label: '...'}, ...] } },
    { type: 'cta', durationSeconds: 10, content: { headline: '...', links: ['...'] } },
  ],
  audio: {
    backgroundMusicFile: 'audio/bg-music.mp3',
    backgroundMusicVolume: 0.12,
  },
};

Scene types: title, problem, solution, demo, feature, stats, cta.

Duration rule: Estimate durationSeconds as ceil(word_count / 2.5) + 2. You will adjust this after generating audio in Step 4.

Step 3: Write Voiceover Script

Create projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

## Scene 1: Title (9s, ~17 words)
Build videos with AI. The product name toolkit makes it easy.

## Scene 2: Problem (14s, ~30 words)
The problem statement goes here. Keep it punchy and relatable.

Word budget per scene: (durationSeconds - 2) * 2.5 words. The -2 accounts for 1s audio delay + 1s padding.

Step 4: Generate Assets

CRITICAL: All commands below MUST be run from the toolkit root, not the project directory.