Login
calesthio/openmontage132 files

Music To Video

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

Use this skillView repositoryView on skills.sh
Specification
Skill ID
calesthio/openmontage/music-to-video
Publisher
calesthio
Repository
openmontage
Installs
0
Files
132
Synced
Sep 23, 2026
How to use it

Open any RiverX project, open the Skills panel in the chat, and search for this identifier. The files are fetched from the source repository at install time.

calesthio/openmontage/music-to-videoInstalls these files
  • SKILL.md
  • references/frame-skeleton.md
  • references/montage.md
  • references/motion-primitive-catalog.md
  • references/motion-primitives/3d-card-flip/index.html
  • references/motion-primitives/assets/gsap.min.js
  • references/motion-primitives/bg-flow-field/index.html
  • references/motion-primitives/binary-decrypt/index.html
  • references/motion-primitives/blur-resolve/index.html
  • references/motion-primitives/braam-punch/index.html
  • references/motion-primitives/chromatic-split/index.html
  • references/motion-primitives/chrome-sweep/index.html
  • references/motion-primitives/counting-punch/index.html
  • references/motion-primitives/crash-zoom-in/index.html
  • references/motion-primitives/datamosh-smear/index.html
  • references/motion-primitives/directional-fill/index.html
  • references/motion-primitives/dolly-zoom/index.html
  • references/motion-primitives/electric-arc/index.html
  • references/motion-primitives/flash-cut/index.html
  • references/motion-primitives/gooey-metaball/index.html
  • references/motion-primitives/hard-cut/index.html
  • references/motion-primitives/hypercut-whip/index.html
  • references/motion-primitives/iris-open/index.html
  • references/motion-primitives/kinetic-letter-in/index.html
  • references/motion-primitives/liquid-morph/index.html
  • references/motion-primitives/mask-reveal/index.html
  • references/motion-primitives/mosaic-pack/index.html
  • references/motion-primitives/neon-flicker/index.html
  • references/motion-primitives/outline-to-fill/index.html
  • references/motion-primitives/palette-flip/index.html
  • references/motion-primitives/particle-burst/index.html
  • references/motion-primitives/pixel-dissolve/index.html
  • references/motion-primitives/radial-burst-lines/index.html
  • references/motion-primitives/screen-shake/index.html
  • references/motion-primitives/slot-machine-reveal/index.html
  • references/motion-primitives/spotlight-sweep/index.html
  • references/motion-primitives/staggered-exit/index.html
  • references/motion-primitives/text-spectral-rays/USAGE.md
  • references/motion-primitives/text-spectral-rays/index.html
  • references/motion-primitives/text-wave-distort/index.html
  • references/motion-primitives/tile-mosaic/index.html
  • references/motion-primitives/typewriter-reveal/index.html
  • references/motion-primitives/word-grid-burst/index.html
  • references/planning.md
  • references/storyboard-format.md
  • references/template-catalog.md
  • references/templates/card-flyby/index.html
  • references/templates/card-flyby/program.json
  • references/templates/held-message-living-field/index.html
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/asphalt-031.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/bark-014.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/brick.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/bricks-075-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/bricks-101.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/bricks-102.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/bricks-104.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/carpet.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/concrete-034.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/concrete-042-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/concrete-046.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/concrete-047-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/concrete.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/diamond-plate-009.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/fabric-080.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/fabric-083.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/grass-001.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/grass-004.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/grass-005.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/ground-037.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/ground-054.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/ground-068.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/ground-080.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/ground-103.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/ground-104.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/lava.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/leather-037.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/marble-012.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/marble-016.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-032.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-038.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-041-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-046-b.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-048-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-049-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-055-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal-061-b.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/metal.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/onyx.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/painted-plaster-017.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/paving-stones-138.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/paving-stones-150.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/plaster-001.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/road-007.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/road-008-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/road-009-c.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/road-012-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/road-012-b.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/road-013-a.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/rock-058.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/rock-063.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/rock.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/snow-015.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/snow.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/tiles-138.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/travertine-009.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-049.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-051.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-058.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-066.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-092.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-094.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-floor-051.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-floor-064.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood-floor-070.png
  • references/templates/held-text-strobe-burst/assets/texture-mask-text/masks/wood.png
  • references/templates/held-text-strobe-burst/index.html
  • references/templates/intro-kinetic-cascade/index.html
  • references/templates/intro-kinetic-cascade/program.json
  • references/templates/logo-split-lockup-pulse/assets/fredoka-700.woff2
  • references/templates/logo-split-lockup-pulse/index.html
  • references/templates/poster-tile-mosaic/index.html
  • references/templates/poster-tile-mosaic/program.json
  • references/templates/roll-flipbook-word-cycle/index.html
  • references/templates/split-anchor-word-slot/index.html
  • references/templates/split-anchor-word-slot/program.json
  • references/templates/typewriter-phrase-keyword-shuffle/index.html
  • scripts/analyze-beatgrid.py
  • scripts/assemble-index.mjs
  • scripts/lib/storyboard.mjs
  • scripts/stage-assets.mjs
  • scripts/validate-plan.mjs
  • sub-agents/frame-worker.md

What this skill tells the agent

music-to-video — one music-grounded, beat-synced video workflow

Use this skill to turn a music track into a beat-synced HyperFrames video. You analyze the track once, lay out the frames, fill in a per-frame plan, and build each frame as a composition. The input is a music track plus optional user images or videos — there is no narration and no website capture. Typography and templates are the floor (a complete video needs zero assets); any media the user supplies is cut in on the same beat grid.

You are the orchestrator. Work in videos/<project>/. Run the steps in order and pass each Gate before moving on. Two steps need the user: Step 3 (plan approval) and Step 6 (render approval). Do every step yourself except Step 4, where you dispatch one sub-agent per frame. Keep design and motion rules out of this file — they live in references/ and the frame-worker sub-agent.

SKILL_DIR = this skill directory. PROJECT_DIR = videos/<project-name>/.

Workflow: Step 0 setup → hyperframes.json + assets/bgm.mp3; Step 1 analyze → audiomap.json; Step 2 skeleton → STORYBOARD.md (frames, groups TBD); Step 3 plan → complete STORYBOARD.md + frame.md; Step 4 build → compositions/frames/NN-*.html; Step 5 assemble → index.html; Step 6 render → renders/video.mp4.

Two ideas that shape everything

  • One analyzer, and you trust it. analyze-beatgrid.py is the only beat analyzer — never re-measure beats with another tool or by ear. Its energy / density / rolls / onsets / silences are always reliable. Its bpm and beats_sec are reliable only when the music is genuinely rhythmic; on calm music the grid is a metronome the tracker imposed, so pace by phrases and energy instead and never hard-cut to it. Deciding which case you're in is each frame's pacing (Step 2).
  • One frame = one file; groups live inside. Step 2 cuts the track into frames, and each frame becomes one composition file compositions/frames/NN-<frame_id>.html, built by one frame-worker. A frame can subdivide into groups (each a template or a motion-primitives combo). Extra density goes _inside_ a group, so frame count tracks distinct treatments, not beats — a fast track does not blow up the number of sub-agents.

Step 0: Setup, BGM, and inputs

Goal: Establish the music source, create the HyperFrames project, and note any user-supplied media.

The music is the spine — establish one track before anything else. This skill is tuned for fast, high-energy BGM: a strong beat grid drives the cuts (calm tracks work, but pace by phrase rather than beat). If the user gave you audio — a music file, or a video to pull the audio from — use it. If not, generate one: choose the mood from the user's description (e.g. "driving synthwave", "trap beat", "upbeat corporate") and produce a track via /hyperframes-media (references/bgm.md — HeyGen retrieval when credentialed, else local Lyria / MusicGen; ElevenLabs or another generator also works). Before generating, run npx hyperframes auth status and relay its output verbatim (don't paraphrase or rewrite it) — it shows whether BGM comes from HeyGen or local MusicGen and, if not signed in, how to sign in. If not signed in, STOP and wait for the user to choose — sign in, or continue offline with local MusicGen — before generating the track; don't write keys into a per-repo .env. (In autonomous mode, note the status and continue offline.) See /hyperframes-media → Preflight for the canonical guidance. Either way the track lands at assets/bgm.mp3. Stage any user-supplied images or videos so frames can weave them in on the beat grid; otherwise typography carries the whole video.

Initialize only if hyperframes.json is missing. Name <project> from the brief in kebab-case, such as midnight-drive-loop — never a timestamp. init checks the installed skills against the latest on GitHub and updates the global set if any are out of date.

npx hyperframes init "videos/<project>" --non-interactive --example=blank
mkdir -p "$PROJECT_DIR/assets" "$PROJECT_DIR/renders"
cp "<user-music>" "$PROJECT_DIR/assets/bgm.mp3"   # extract from a video first if needed
# only if the user gave you images/videos:
node <SKILL_DIR>/scripts/stage-assets.mjs --from <dir> --hyperframes "$PROJECT_DIR" --into public

The brand (font + palette) is chosen at Step 3, not here. Don't pick a genre or a track type up front — assets are just an optional ingredient, and the genre emerges from the per-frame choices.

Gate: hyperframes.json + assets/bgm.mp3 exist; aspect / length / fps and (if any) the asset inventory are noted.

Step 1: Analyze the music

Goal: Produce the one canonical timing analysis the whole video is built on.

analyze-beatgrid.py is the only beat analyzer — never re-measure beats with another tool or by ear. It reads the track once and writes audiomap.json: energy phases (level / density / feel), onsets + onset_rate, rolls, silences, hard_stops, key_moments, phrases, tempo / grid, and audio.duration_sec. It's deterministic — the same file always gives the same map. Most fields are reliable on any music; bpm and beats_sec are reliable only when the music is genuinely rhythmic, and judging that is the call you make at Step 2.

Prerequisites: Python 3 with librosa, numpy, and soundfile available. If import fails, install them into the active Python environment before running the analyzer:

python3 -m pip install librosa numpy soundfile
python3 <SKILL_DIR>/scripts/analyze-beatgrid.py "$PROJECT_DIR/assets/bgm.mp3" \
  -o "$PROJECT_DIR/audiomap.json" --print

Gate: audiomap.json exists; audio.duration_sec is known.

Step 2: Frame skeleton (structure only)

Goal: Read the music and lay out the frames — the skeleton of STORYBOARD.md.

Read `references/frame-skeleton.md`. Turn audiomap.json into the skeleton of STORYBOARD.md yourself — there is no intermediate JSON. Cut the track into frames at real musical changes (hard_stops, SURGE / DROP key_moments, the edges of a roll, a stretch with no onsets, a big energy jump), snapping every boundary to an audiomap anchor. For each frame set span_sec, pacing (the verdict from Step 1's trust call — beat_cut when the grid is real, phrase_flow when it's a metronome imposed on calm music), mood, and a one-line feel (the plain music situation Step 3 matches a template against). Only classify and lay out here: leave every frame's ### Groups as TBD (Step 3) and the frontmatter style blank — no templates, copy, color, or fonts. Expect ~1–6 frames.

Gate: frames tile the track (first at 0, last at duration_s); each carries span_sec + pacing + mood + feel; every ### Groups is TBD; no content anywhere.

Step 3: Fill the plan (user-gated)

Goal: Turn the skeleton into an approved, complete STORYBOARD.md.

Read `references/planning.md`, `storyboard-format.md`, `template-catalog.md`, `motion-primitive-catalog.md`, and `montage.md` (only if the user supplied assets). Editing the same file in place, do two things:

  1. Pick the brand. Choose one preset from ../hyperframes-creative/frame-presets/ using the table in ../hyperframes-creative/references/design-spec.md (match the track's mood; only its fonts and colors matter — templates own composition). Copy it into frame.md unmodified and fill the frontmatter style (font + a ≤4–6 swatch palette) from it.

More from calesthio

calesthio/openmontage

Acestep

AI music generation with ACE-Step 1.5 — background music, vocal tracks, covers, stem extraction for video production. Use when generating music, soundtracks, jingles, or working with audio stems. Triggers include background music, soundtrack, jingle, music generation, stem extraction, cover, style transfer, or musical composition tasks.

calesthio/openmontage

Agents

Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience.

calesthio/openmontage

AI Video Gen

|

Use this skill in a project

Open any RiverX project, search the Skills panel for calesthio/openmontage/music-to-video, and install it in one click.

Use this skill