A resumable local production pipeline that turns a song and creative direction into an organized AI-assisted music video project.
CineBraid Studio analyzes music, develops a director's treatment, creates an adaptive storyboard, generates stills through Gemini, processes them with ComfyUI, animates them through Runway on Syntx.ai, assembles a draft edit, and exports a DaVinci Resolve-ready timeline. Every stage is checkpointed so an interrupted project can continue without repeating completed work.
Important
This is an early Windows-first alpha. It automates third-party web interfaces, so provider UI changes, account limits, regional availability, and generation failures can still require maintenance or manual intervention. Use your own accounts and follow each provider's terms.
flowchart LR
A["Song + brief + vibe"] --> B["Audio analysis and transcription"]
B --> C["Director treatment"]
C --> D["Adaptive storyboard"]
D --> E["Gemini still images"]
E --> F["ComfyUI cleanup"]
F --> G["Motion prompts"]
G --> H["Syntx.ai / Runway video"]
H --> I["Draft montage"]
I --> J["MP4 + EDL + FCPXML"]
The seven resumable production stages are:
- Analysis - local audio structure, energy, timing, Faster Whisper transcription, and the director's treatment.
- Storyboard - visual bible, continuity rules, adaptive shot lengths, image prompts, and edit decisions.
- Images - one persistent Gemini project session, generation recovery, download verification, and per-scene checkpoints.
- Watermark cleanup - GPU-accelerated ComfyUI batch processing with the included workflow template.
- Motion - short image-to-video prompts derived from the finished frames and continuity plan.
- Video - Syntx.ai Runway automation, exact-card result tracking, retries, verified downloads, and optional semantic quality ranking.
- Montage - music-aware trims, restrained transitions, optional intro/outro branding, final MP4, EDL, timeline JSON, and DaVinci FCPXML.
- Local dashboard with dark mode, project queue, pause/resume, per-stage launch, full launch, reordering, and double-confirmed deletion.
- A single execution lane prevents several browser-heavy projects from fighting for the same profiles and GPU.
- Atomic project state, inter-process locks, retryable writes, and persistent queue recovery.
- User direction remains the creative source of truth; style rules extend the brief instead of replacing it.
- Optional anti-slop mode adds continuity, screen direction, subject identity, action-state, shot-family, and motion-complexity constraints.
- Verified image and video downloads, including static-video detection and regeneration, before a scene is marked complete.
- No Gemini, Runway, or OpenAI API key is required in the default subscription-based setup.
- Every deliverable is mirrored into a numbered, editor-friendly output folder.
The tested path is Windows 11 with an NVIDIA GPU.
- Install Python 3.11 or newer, Google Chrome, and Microsoft Edge.
- Install and sign in to the Codex CLI, or configure the documented local Ollama fallback where supported.
- Install the official ComfyUI Windows portable build and the ComfyUI Impact Pack.
- Put a compatible, properly licensed SDXL inpainting checkpoint in
ComfyUI/models/checkpoints. - Clone the repository and launch the bootstrap script:
git clone https://github.com/DenisGeide/CineBraid-Studio.git
cd CineBraid-Studio
.\start_pipeline.batOn first launch, the script creates local pipeline_config.json and config.json files from the safe examples and installs the Python dependencies. Before running a full project, follow the one-time ComfyUI and account setup in Installation.
The dashboard opens at http://127.0.0.1:8765. If the port is occupied, CineBraid selects the next free port and writes it to .dashboard-url.
The orchestration, transcription, audio analysis, ComfyUI processing, montage, queue, and dashboard run locally. The default workflow uses subscription-backed web sessions instead of separately billed APIs.
| Component | Required | Typical cost at release | Purpose |
|---|---|---|---|
| Codex CLI | Yes for the current full-quality planning path | ChatGPT Free may handle light use; Plus is listed at $20/month | Structured treatment, storyboard, motion planning, and optional video review |
| Gemini web app | Yes | Free limits exist; Google AI Pro is listed at $19.99/month in the US | Consistent still-image generation |
| Syntx.ai with Runway access | Yes for video generation | Provider-specific plan or credits | Browser-based Runway Gen-4.5 image-to-video bridge |
| ComfyUI | Yes | Free, local compute | Watermark-region inpainting and image cleanup |
| DaVinci Resolve | Optional | Free; Studio is listed at $295 | Manual finishing from the exported FCPXML/EDL |
Prices, limits, taxes, currencies, and regional availability change. See Costs And Services for source links and the important distinction between Syntx.ai credits and a direct Runway plan.
Each project is stored under projects/<project-id>/. The clean handoff is continuously mirrored to:
projects/<project-id>/output/
|-- 00_PROJECT/ treatment, storyboard, visual bible, EDL, FCPXML, title
|-- 01_FINAL_VIDEO/ assembled MP4
|-- 02_IMAGE_PROMPTS/ Gemini prompts
|-- 03_IMAGES_RAW/ original Gemini images
|-- 04_IMAGES_CLEAN/ ComfyUI-processed images
|-- 05_VIDEO_PROMPTS/ Runway motion prompts
|-- 06_VIDEO_SHOTS/ downloaded generated shots
|-- 07_AUDIO/ source track
|-- MANIFEST.json
`-- README.txt
Browser profiles, cookies, source music, generated media, local configs, logs, and model weights are intentionally ignored by Git.
- Installation
- Configuration
- Architecture
- Pipeline behavior
- Costs and external services
- Troubleshooting
- Privacy and account safety
- Development status
- Contributing
CineBraid Studio is licensed under the GNU Affero General Public License v3.0.
You may use, study, modify, and redistribute the project. If you distribute a modified version, or let users interact with a modified version over a network, the AGPL requires you to offer those users the corresponding source code under the same license. Private changes that are neither distributed nor offered as a network service do not automatically have to be published.
Third-party applications, models, generated media, and online services are not relicensed by this repository. Their own licenses and terms still apply. CineBraid Studio is not affiliated with Google, OpenAI, Runway, Syntx.ai, ComfyUI, Blackmagic Design, or their respective model providers.