Video production

Script to storyboard to approved shots to finished clips, and the pipeline that orchestrates it.

The video path is the longest chain in the product. It is designed so a user can go from an idea to finished clips without leaving chat, and without any dead end where something generates but has nowhere to go.

The chain

Scripts

ugc-script-writer covers 17 format archetypes: talking head, health hacks, emotional stories, character interviews, keynote speakers, whistleblowers, value and growth content, two stylized animation formats, the podcast two-hander, and the viral chibi sticker reaction. It requires no offer, because organic content is a first-class use case. The suggested ratio for organic creators is three growth videos per product video.

Every scene may name a speaker. It is optional for single-presenter formats and required for the podcast format, where it is what makes turns alternate between two people. The renderer shows it as a badge and carries the labels into the copied script. Viral chibi scenes carry realAction, chibiAction and sfx instead, and leave spokenScript empty - that format has no dialogue at all.

The ad format is a third axis, distinct from video type and model: lib/ad-formats.ts is the complete library of 22 formats across two worlds (“ugc” and “produced”). The host never invents or truncates format cards - the app builds the full tabbed panel in code from that file, and format_slug is a required argument on the script writer, so writing a script before the user picked a format is impossible.

The copy-ugc-scripts skill and the database agent prompt must stay in sync. They are two copies of the same thinking.

Storyboards

storyboard-director turns a script into shots. Its prompt enforces two disciplines:

• Element discipline. Verbatim element descriptions, explicit elementSlugs per shot, one continuous action per shot.
• Sound discipline. Sound effects only, never music.

elementSlugs travel end to end: storyboard, to the Creative Studio handoff, to the pipeline, where they become reference images on the scene image call.

The storyboard appears as a live previs card inside the chat run, with no reload, then opens in Creative Studio through a banner.

Two production routes

An approved script can become clips two ways. Both are supported and the user decides; the host does not pick silently.

Route A, storyboard. One Director pass plans every shot, then the batch renders. The previs card in chat produces the whole ad without leaving: “Generate scene images” fires the per-shot batch, then the review panel opens to approve shots and animate them.

Route B, scene by scene from the assets. No storyboard. Each shot is a still built from the cast that was already locked (character portrait and identity sheet, plus the element sheets for props, wardrobe, location, product), approved, then animated from its own first frame. The host works one scene at a time, in order, and never batches ahead.

Route B is the newer approach and the reason elements and characters exist as reusable sheets. Route A is version one, kept because a long ad with many shots is still faster to plan in a single Director pass.

Pipelines

A pipeline is the orchestration record for a multi-scene production. It runs on the app server using the existing image and video queues, so no new Cloudflare infrastructure was needed for it.

The frontend polls the GET endpoint and calls advance when images complete. Credits for the whole pipeline (N images plus N videos) are deducted upfront.

Review and animate

components/images/pipeline-review-sheet.tsx is the gate between images and clips. The user approves shots individually or with “Approve all completed”, then animates the approved set. The sheet flips to clip cards as they generate, and late approvals can be animated in a second pass.

Clips are saved with isSaved: true, foldered under {offerName}/{title}, and appear in the Videos tab of the media library.

Models

Video models are database rows, not code. The prompt box derives aspect ratios, modes, durations, frame slots, and the sound toggle from the selected model’s config. Talking head automatically switches to whichever model has features.talkingHead: true.

Adding a model means adding it in Admin Settings and writing a migration for its config JSON. The worker uses a generic createTask(apiModelPath, input, apiKey), with apiModelPath coming from that config through the queue message.

Video types

A video type is how a clip is produced: who is on screen, which agent turns the script into the model prompt, how many reference faces it needs. It is not the model (a database row) and not the ad format (the structure of the ad).

lib/video-types.ts is the single source of truth. Each entry carries a label, a prompt agent, a presenter count, and the model feature the type cannot run without.

Everything reads that file: the chat settings card, the tool schemas, the API validator, Creative Studio, and the generation branch that picks the engine. Adding a type (“3D demonstration”, “screen recording”) is one entry.

The prompt agent is dispatched from the registry, in lib/video-prompt-engine.ts. All three dispatch sites used to call the talking-head engine no matter what the entry said, which made promptAgent decorative - a new type with its own engine would have silently had talking-head prompts written for it.

Podcast is the two-hander. Turns alternate between exactly two speakers, each turn is its own single-speaker clip, and the script names a speaker on every scene. The Podcast Studio image preset holds one room, one mic style and one lighting setup across every frame so turn 6 looks like the same conversation as turn 1.

Viral Chibi is the mixed-media one: photoreal live-action footage with a flat 2D hand-drawn sticker of a real person composited on top, reacting. It is the only format that is a single clip - no storyboard, no per-scene stills, no image preset. The reference image is a photo of the person, and it is an identity source rather than a start frame: the model draws a chibi caricature from it and invents the live-action scene around the drawing, so passing it as first_frame_url fights the format. The prompt the model receives is sectioned ([STYLE], [SCENE], [CHARACTER], timed shot blocks, [HARD RULES]), and the hard-rules block is load-bearing: without it the model gives the drawing volume and lighting until it becomes a 3D cartoon in a real room. Nobody speaks - the comedy is expression, cartoon FX and sound effects.

The host infers the type from the request rather than interviewing the user about it. It only discusses the options when the user asks what is available, or when who is on screen is genuinely unclear. The settings card always renders the full list, so the user can override.

Aspect ratio

Decided once, at image time, and enforced in code rather than trusted to the host.

• generate_image rejects a call without aspect_ratio. There is no default. A still is the first frame of its clip, so a silently defaulted 1 becomes a 1 video nobody asked for.
• The user is asked once, before the first scene image, and the same answer is reused for every scene of that ad.
• When a still is animated, the clip inherits that still’s ratio (clamped to what the model supports). The ratio reaches the video provider without the host having to remember it.