Gemini Omni Flash API

Gemini Omni Flash Skill

This skill uses the Gemini Omni 1.1 Flash model (gemini-omni-1.1-flash) to perform text to video generation, image to video generation (first frame and last frame transitions), video extensions (up to 40s), and video editing.

[!WARNING] Important Regional Restrictions: Uploading videos to use for video edits or extensions is NOT available in the EEA, Switzerland, the United Kingdom, and some US states. If a video-to-video edit completes quickly with empty outputs (total_output_tokens: 0 or no video content), it is likely due to this restriction.

Core capabilities

1. Text to video: Generating videos from a text prompt.
2. First frame to video: Generating videos from a starting image (--first-frame).
3. First and last frame transition: Generating videos interpolating between a starting image and a final image (--first-frame and --last-frame; note: --last-frame must be used with --first-frame).
4. Video extensions: Extending existing videos by up to 10 seconds per turn, up to a total length of 40 seconds (--extend or --previous-interaction-id).
5. Video editing and refinement: Editing existing videos (maximum duration 10 seconds), applying stylistic changes, or performing inpainting/outpainting.
6. Image and video referenced generation: Using style, character, or object references from images or videos to guide video generation.

Workflow

1. Analyze request: Determine the target task (e.g., first-frame-to-video, first-and-last-frame transition, video extension, reference-guided editing) and identify any input media assets.
2. Run SDK scripts:

• Directly run the appropriate utility (scripts/video/generate_video.py or scripts/upload_file.py).
• Configure settings like --aspect-ratio (e.g. 16:9, 9:16), --resolution (360p, 720p, 1080p, 4k; default: 720p), and --duration (any integer between 3 and 10 seconds, e.g. 3, 5, 10). Note: `4k` requests take longer to generate.

1. Retrieve and process output: Outputs are saved to the local filesystem (e.g. media/). Report back the completed media path to the user.

Reference Documentation

• Interactions API: All operations and state management for the Gemini Omni 1.1 Flash model (gemini-omni-1.1-flash) are handled via the Interactions API.
• Files API: Input media files (such as reference images and videos) must be uploaded via the Files API first before being referenced in generations. The uploaded file URI and MIME type are then included in the interactions.create input parts array.
• [Gemini API Skill Reference](https://github.com/google-gemini/gemini-skills/blob/main/skills/gemini-api-dev/SKILL.md): Platform-wide guidelines, current model specifications, and SDK usage rules for the Gemini API.

Dependencies and Prerequisites

• Python SDK (`google-genai`): Requires google-genai >= 2.19.0 (Python) to support the interactions client and full video output resolution configuration (360p, 720p, 1080p, 4k). Install or upgrade using: ``bash pip install -U google-genai ``
• Python Runtime: Requires Python >= 3.10 (for compatibility with modern google-genai SDK types and methods).
• ffmpeg & ffprobe: prep_video.py, inspect_video.py, and generate_video.py (when stripping audio via --strip-audio) require ffmpeg and ffprobe binaries installed and available in your system PATH.
• API Key: Set the GEMINI_API_KEY environment variable: ``bash export GEMINI_API_KEY="your-api-key" ``

Available scripts

Use the following Python scripts to upload media with the Files API, prepare input videos with ffmpeg, and generate video outputs using the Interactions API.

1. [upload_file.py](scripts/upload_file.py): Uploads local media (images and videos) to the Files API and polls until ACTIVE. If uploading a video larger than 25MB, it prints an informative warning/tip highlighting that Gemini Omni Flash is optimized for editing 10s videos at 720p/24fps, and recommends pre-processing with prep_video.py first to speed up the upload.

   ./scripts/upload_file.py path/to/image.png

1. [generate_video.py](scripts/video/generate_video.py): Performs end-to-end video generation and downloads the output video. It detects and uploads local media references (images or videos) before calling the Interactions API. Large video assets (>25MB) will trigger informative pre-processing recommendations without blocking the upload.

• Text to video:

     ./scripts/video/generate_video.py "A close-up of a cat drinking tea" --output media/cat_tea.mp4

• Output resolution options (`--resolution`):

Gemini Omni 1.1 Flash natively supports four output resolutions across both landscape (16:9) and portrait (9:16) aspect ratios:

• 360p: 640x360 (16:9) or 360x640 (9:16)
• 720p: 1280x720 (16:9) or 720x1280 (9:16) — (default)
• 1080p: 1920x1080 (16:9) or 1080x1920 (9:16)
• 4k: 3840x2160 (16:9) or 2160x3840 (9:16)

     # High-definition (1080p)
     ./scripts/video/generate_video.py "A cinematic drone shot over misty mountains at sunrise" --resolution 1080p --output media/mountains_1080p.mp4

     # Ultra-high-definition 4K (Note: 4K requests take longer to generate; pass --timeout if needed)
     ./scripts/video/generate_video.py "A macro shot of a dewdrop on a flower petal in golden sunlight" --resolution 4k --timeout 900 --output media/flower_4k.mp4

• Configurable request timeouts (`--timeout`):

Default HTTP timeout is 600 seconds (10 minutes). For computationally intensive requests — such as extending a 30s video in 4K by 10s (up to the maximum 40s total video length) — generation can take several minutes. Use --timeout 900 (or 1200) to provide an extended execution budget.

• First frame to video:

     ./scripts/video/generate_video.py "The waves crash against the shore." --first-frame start.png --output media/waves.mp4

• First and last frame transition:

Provide a starting frame and an ending frame to generate a smooth transition between them (note: --last-frame must be used together with --first-frame):

     ./scripts/video/generate_video.py "A smooth timelapse from sunrise to sunset" --first-frame start.png --last-frame end.png --output media/interpolation.mp4

• Looping video (identical start and end frame):

     ./scripts/video/generate_video.py "A crystal orb spinning continuously in place" --first-frame orb.png --last-frame orb.png --output media/loop.mp4

• Image-referenced video generation:

     ./scripts/video/generate_video.py "A cybernetic warrior in the style of <IMAGE_REF_0>" --image reference.png --output media/warrior.mp4

• Video-referenced video generation:

Provide one or more reference videos (--video-reference / -vr) to guide character, object, or motion style (ideal duration is ~3s, up to 3 reference videos recommended):

     ./scripts/video/generate_video.py "A musician playing cello in the style of <VIDEO_REF_0>" --video-reference ref_dance.mp4 --output media/cello.mp4

• Video extension (extend an existing video):

Extend an existing video by up to 10 seconds (total duration up to 40 seconds):

     ./scripts/video/generate_video.py "The scene continues as the sun sets over the horizon" --extend media/sunset.mp4 --output media/sunset_extended.mp4

• Video extension with reference images and reference videos:

Prompt-based extension allows passing reference images and reference videos simultaneously: