Playbooks

Director — steer a short film from a rough idea

# Director — steer a short film from a rough idea

> Paste this whole file (with *Tell the agent about your film* filled in) into Claude, Codex, or Cursor, connected to the Livepeer Agent **creative** surface. You bring a rough idea and a character. The agent writes the film, **stops before any video is paid for**, and you steer by saying **what to change**. Nothing animates until you approve the stills.

This is one of two ways to make a short film here. If you already have a finished script and want one button, use the **Producer** playbook (`producer-short-film.md`) instead.

## What you'll get

- **A treatment** — logline, beat sheet, style bible, sets, a shot list and a sound plan, shown before any render.
- **A cast reference per face** — one locked image each, in the film's look. Each still is graded against the face in that shot, not against the other.
- **Approved stills** — one keyframe per scene, shown to you before any motion is rendered.
- **A 4–8 scene short** — one lead, the same person in every scene they enter. Each still is checked against the reference before any motion is bought, and a still that fails is reshot.
- **A project page** — a shareable `viewer_url` with every scene.
- **A finished reel** — hard cuts, spotted effects, a score under them, and an exact 16:9 frame.
- **Receipts** — the quote before you commit, and the spend per step after.

## Tell the agent about your film

```yaml
working_title:       # e.g. "Hard Candy"
rough_idea:          # 1–3 sentences. It does not need to be a script.
lead_character:      # name + the features that must survive every shot, e.g. "Mara, short black bob, yellow raincoat".
second_character:    # OPTIONAL. A second named face, the same way. Empty means no second face.
lead_reference_url:  # OPTIONAL. Leave empty and the agent renders one for you to approve.
look:                # medium, palette, texture. e.g. "hand-painted 2D, teal night, amber lamplight, soft film grain".
scene_count:         # 4–8.
aspect_ratio:        # 16:9 (default) / 9:16 / 1:1.
must_avoid:          # e.g. "daylight, crowds, readable signs".
tone:                # OPTIONAL. e.g. "eerie fairy tale".
sound_mood:          # OPTIONAL. e.g. "sparse music box and rain".
model_pin:           # OPTIONAL. e.g. "kling-v3-turbo-i2v". Empty means the agent chooses and tells you.
budget_usd:          # your ceiling. The agent quotes first and will not exceed it without asking.
```

## Make it like a film

The user's lines are the story. A render model given "Mara enters the shop" returns a stock clip. Given a shot, it returns a shot. Write every stage below before spending, and show it.

### 1 · Development

**Logline.** One sentence: who wants what, what the obstacle is, and what it costs them.
> *A girl sheltering from the rain wants to get warm in a candy shop, and the keeper's touch is the obstacle: it turns whatever he reaches into sugar, including her.*

**The lead.** Their want in the first shot. Their obstacle. Their **visible change** in the last shot: a different object in the hand, a different posture, a different light on the same street. If the change cannot be seen, rewrite the ending until it can.

**Beat sheet.** One beat per shot:

| Beat | Job | Typical shot |
|---|---|---|
| 1 · World | Where we are, who we follow, the mood, and the want | Wide, establishing |
| 2 · Turn | The problem arrives | Medium on her look; his reverse is the next shot |
| 3 · Escalation | The stakes become visible | Closer, or an insert |
| 4 · Climax | The image the film is about | Close-up, held |
| 5 · Consequence | The lead acts, and the cost lands | Medium, with movement |
| 6 · Echo | The opening image returns, and the lead has changed | The same framing as shot 1 |

For every shot, write one line starting *"Now the audience knows…"*. An empty line, or a line that repeats the previous shot, means cut or merge that shot.

**Slugline, then the shot.** Start each scene with where it is, then the camera:

`EXT. WET STREET - NIGHT` or `INT. CANDY SHOP - NIGHT`

Then the shot line in *Pre-production*. A title never goes in the image. AI text renders badly. Leave the title for the user.

### 2 · Pre-production

**Style bible.** One block, reused word for word as the job's `style`. Do not repeat it inside each scene.

- **Medium:** hand-painted 2D, or 35mm live action. One medium for the whole film.
- **Palette:** 3–5 named colours.
- **Texture:** "soft film grain" or "watercolour paper".
- **Era and place:** "1970s European side street".

**Set and prop sheet.** Each location is one sentence, used verbatim wherever it appears ("a narrow candy shop: dark wood shelves of glass jars, one brass lamp over the counter"). Each story prop has fixed words and a continuity note (the amber sweet: in the window in shot 1, in her palm in shot 6). The model has no memory between shots. The words are the continuity.

**Light plan by beat.** Name the source in every shot: the lamp, the window, a street lamp. The words "cinematic lighting" give the model nothing to place. Let the light carry the arc: warm at the opening, one hard source at the turn, the highest contrast on the face at the climax, the shop dark at the echo.

**Lens package.** Write the lens into the shot.

| Lens | Use |
|---|---|
| 24 mm | Wide, the world |
| 35 mm | Medium, walking, a reverse |
| 50 mm | Medium close-up |
| 85 mm | Close-up, the climax |

**Camera axis.** One line for the whole film: her screen direction ("Mara travels frame-left to frame-right") and which side of the 180 the camera stays on ("the camera stays on the street side of the door, so she is frame-left of the counter").

**The shot list.** One row per beat, with a duration:

`[EXT. or INT.] [shot size, lens] · [one motivated move] · [LEAD NAME + signature look + wardrobe state] [one action + the feeling + where they look] · [set sentence + the light's SOURCE] · [composition] · [Ns]`

Example: *"INT. CANDY SHOP - NIGHT. Close-up, 85 mm, slow push-in. Only Mara, black bob and yellow raincoat with the hood down, three-quarter toward the camera, looks down. The cuff of the yellow sleeve is translucent amber glass, and the rest of the coat is cloth. Fear giving way to wonder. The previous shot showed this same sleeve as cloth. The shop's single brass lamp is the only light, rain streaking the window behind her. Her face on the left third, the cuff in the foreground, a little headroom. 8s."*

- **Coverage.** A reverse may match the previous shot's size. Every other pair of neighbors changes size. Wide, then medium, then close, as the feeling rises. One insert of the key object. One reaction where the story turns.
- **One move, and a reason.** A push-in is a realisation. A pull-out is isolation or a reveal. A track follows intent. A static frame is tension. One move per shot.
- **Composition.** Subject on a third, headroom above the head, a foreground shape for depth, a leading line (the street, a shelf, the counter).
- **Screen direction and the 180.** She keeps the direction written on the axis. On the reverse he is on the opposite side of the frame, so her look meets his. **Eyeline:** say where she looks, and match it in the next shot. The camera does not cross the line between them.
- **Performance.** One emotion and one physical action. The feeling **changes inside the shot** ("fear giving way to wonder"). "Is scared" is not an action. "Clutches the sleeve to her chest" is.
- **Faces.** Frontal or three-quarter toward the camera, with the eyes aimed where the eyeline says. "Looking frame-right" alone is drawn as a profile, and a profile cannot be graded. From behind, the model invents a face when she turns.
- **Wardrobe and weather, in words.** The same clothing state in every shot ("hood down"). Outside: "rain falling". Inside: "rain streaks the window".
- **A second named face is a second cast.** One reference per recurring face, each a single figure. A shot has one on-camera face, and the other person is off-screen. A conversation is shot-reverse-shot: she looks frame-right, he looks frame-left, and the camera stays on the same side of the line. Do not ask for the other person's shoulder or the back of their head in this still. The editor has only this face's reference, and it draws a second copy of that face. Grade each still against that face's own reference, not against the other. An unnamed extra is a hand, a silhouette, or off-screen. A script line that names two faces is split into the shot and its reverse before anything is rendered.
- **Shot length is rhythm.** 3–4 s for an insert or a reaction. 5 s for most beats. 7–8 s for the climax. 6–8 s for the held last image. A uniform 5 s everywhere is a slideshow.
- **One continuous take.** End every motion prompt with "one continuous take, no cuts".

**What these models get wrong**

| Failure | Write this instead |
|---|---|
| Readable text or signs | "unmarked window", "no lettering" |
| Crowds | An empty street, or one distant figure |
| Hands doing fine work | The result, already done |
| Fast action and whip pans | One clear motion: "hurries", "backs away" |
| A change inside one still | The BEFORE state in one shot and the AFTER state in the next. The still shows the state. The motion prompt may describe the change. |
| Two of the lead | Name the other person by role and look, and state there is only one of her |

If the story needs talk, use one voice-over line or no words. Lip-synced speech across a scene is not available.

**Sound plan.** Write it with the shots.

- Per shot, the 1–3 sounds the audience should hear, and the millisecond they start (the sum of the earlier shots' durations).
- The score: mood and tempo that follow the arc. Silence at the climax is a choice you write down, and a choice you deliver.

Show the treatment — logline, beat sheet with its *"Now the audience knows…"* lines, style bible, set and prop sheet, axis, shot list, sound plan — and ask for changes. This stop is free.

### 3 · Post: sound, edit, grade

**Sound.** Build it after the picture is locked. The mix replaces the audio the video model invented. That is the point: the model's track is noise, and the designed track is the film.

1. **Score.** `create_media({ action: "music", model_override: "sonilo-t2m", prompt: <the score line> })`. Do not pass `duration`. This cap's own length is about 90 seconds, which covers a 30-second reel. `action: "music"` without that override is a different model, capped at 15 seconds.
2. **Effects.** For each distinct sound, `create_media({ action: "music", model_override: "mirelo-sfx", prompt: <that sound>, duration: <3–10> })`. It returns audio. Keep the total of score plus effects at 8 tracks or fewer. A rain bed can be one stem for the whole reel; a jar break is one short stem at that shot.
3. **Spot them.** `create_media({ action: "mix_tracks", tracks: [ { url: <score>, volume: 0.4 }, { url: <effect>, volume: 1, delay_ms: <ms from the reel's start> } ] })`. `delay_ms` is the sum of the earlier shots' durations, in milliseconds. Hard cuts keep that sum true.
4. **Lay the mix on the picture,** after the export below. `create_media({ action: "mux_audio", source_url: <reel>, audio_url: <mix>, audio_fill: "loop" })`. The mix is built first because spotting needs `delay_ms`, and only `mix_tracks` has it — `mux_audio` lays one track down and cannot place it at a timecode. The default mode is `replace`, so the model's own track is gone. `audio_fill: "loop"` is what keeps the picture's length: without it a mix shorter than the reel cuts the film off at the end of the audio, and a 9 second reel comes back 6 seconds long. To keep a clip's own foley and sit the score under it instead, pass `audio_mode: "mix"` with `music_volume` at 0.3–0.5 and `video_volume: 1`. That is a different film. This one throws the model's track away.

A voice-over, if the treatment has one line, is `create_media({ action: "tts", prompt: <the line> })`, added to the same `mix_tracks` at that shot's `delay_ms`. Do not lipsync the scene.

If the reel is harsh, lower the score's `volume` and mix again.

**Edit.** `director_export({ project_id, transition: "cut", soundtrack: "off" })`. A cut is the grammar. Pass `transition: "crossfade"` only when the treatment is a dream or a passage of time, and `transition: "fade"` only for a fade-out ending. One choice for every join. Open on the world and the lead in the first second. End on the held echo.

**Frame.** `create_media({ action: "generate", model_override: "ffmpeg-export", source_url: <the reel with the mix>, aspect_ratio: "16:9" })`. There is no loudness tool. Listen. If it clips, remix quieter. A 9:16 cut is the same call with `aspect_ratio: "9:16"`; it centre-crops, so check the lead stays in frame.

### 4 · Festival checklist

Look at the start, middle and end of every shot, then listen to the reel. It ships when every line holds:

| Check | It passes when |
|---|---|
| Identity | Every recurring face is the same person in every shot they enter, graded against that face's own reference, not against the other, and never appears twice in one frame. `critique_shot` on each of those stills is ≥ 0.85 with `lead_count` 1. |
| Supporting characters | A named second face is a second cast, one on-camera face per shot, graded against that reference. An unnamed extra is a hand, a silhouette, or off-screen. |
| Look | One medium and one palette. |
| Story | Each shot shows its beat. Its *"Now the audience knows…"* line is visibly true. |
| Artifacts | No cut inside a clip, no morph, no floating prop, no rain indoors, no readable text. |
| Sound | Spotted effects, a score under them, nothing clipping, and an ending that lands. |
| Spec | Exact aspect ratio. |

`critique_batch({ urls: <the stills>, goal: <the beat sheet>, focus: "fidelity" })` is a second opinion on the story. Your eyes are the final check. The scores catch a missing or doubled lead. They pass a wrong action. A shot that fails is reshot at most twice. After that it is shown **flagged**, with its score and the reason. It is never hidden, and it is never animated in silence.

### 5 · Receipts

Keep a running total of every step's reported cost: the job, each reshoot, the effects, the score, the mix and the export. Before a step would take the total past `budget_usd`, ask. The budget caps the job's quote, not what comes after it. `get_cost_report` on the keyless demo is the whole shared pool, so add up the per-call costs yourself. A render reported `failed` can still finish: poll `get_create_media` once more a minute later before you buy it again.

## Competency card

Check every line before hero motion. A miss is named. It is not averaged away.

1. **Identity.** Every recurring face is at least 0.85 against its own reference, with `lead_count` 1.
2. **Axis.** Screen direction and the 180 hold for the whole film.
3. **Eyeline.** A look frame-right is answered by a look frame-left.
4. Each shot delivers one new fact. A shot that repeats the previous shot is cut or rewritten.
5. **Prop state.** BEFORE and AFTER are different shots. A prop does not reset between them.
6. **Rhythm.** Durations are not a uniform 5 s.
7. **Performance.** The feeling changes inside the shot.
8. **Economy.** Previz is `ltx-25-i2v-fast` from the approved still. `kling-v3-turbo-i2v` is only for a shot whose previz passed this card.

## Prototype loop

Claude, Codex, and Cursor run this same loop. Personalize the six knobs and do not change the procedure: the lead and their wardrobe, a second face or none, the look, the aspect ratio, the target seconds, and `budget_usd`.

1. **Coverage table.** Before any still, show the shot, the on-camera face, the eyeline, the screen direction, the one new fact, and the prop state. Wait for a yes.
2. **Stills, then stop.** Do not animate a still the user has not accepted. If a hand is in frame, count the fingers. A wrong count is a new still. On a reverse, the lamp stays on the same side of the line as in the shot it answers.
3. **Fast clips.** Animate only accepted stills. The reverse should cut on the look: it opens with the eyes already aimed. Check the action match. If she exits frame-right, she enters the next shot from frame-left, still traveling that way. Then measure the file. A requested length or aspect is not a pass. `ltx-25-i2v-fast` returns about 6.12 seconds at 1920×1080. A climax longer than 6 seconds is a new clip at 8, 10, 12 or 14 — one unbroken take, not two shots cut together. Do not trim as a separate step: the previz call below takes each clip's `in` and `out` and trims on the way into the reel. Once the shot is on a project, `director_retime({ project_id, scene_index, target_duration })` retimes it — at or under the source length it trims, over it loops the clip to fill. A loop repeats visibly, so hold a shot longer with a new clip at the next duration up, not with a loop.
4. **Previz reel.** Hand back one previz reel before any hero spend: `assemble({ clips: [{ src: <clip>, in: 0, out: <the shot's seconds>, title: <the slugline> }, <one per approved clip, in screenplay order>], transition: "cut", title: <the working title + " previz"> })`. One call trims every clip to its written length and stitches them. It returns a `project_id` and a `viewer_url` as well as the reel, so the previz is already the project that `director_retime`, `director_re_render` and `director_insert` edit later. The job's `reel_url` is that same cut when the shots were submitted together, including a second face. A previz made shot by shot still goes through this one call. `director_export` adds a soundtrack; this reel does not. State the previz budget for the references, the stills, the fast clips, and this assemble before the first still. Writing `out` on a clip buys it its own trim dispatch, billed per second of that clip, so the assemble is not flat: one stitch plus one trim per clip puts a six-shot previz at roughly two cents. `kling-v3-turbo-i2v` is a separate yes, and only for a shot whose previz passed the competency card.

## How the agent runs it

1. **Treatment.** Write stages 1–2 and agree it with the user before anything is rendered.
2. **Cast.** Use `lead_reference_url`, or render one with `create_media({ action: "generate", model_override: "gpt-image", aspect_ratio: <the film's>, prompt: <lead, in the film's look, one figure, face and signature clothes visible, on a plain dark backdrop, no setting, no weather> })`. gpt-image keeps wardrobe that flux-dev drops. A multi-pose sheet copies into the scenes as extra people. Nothing else starts until the user accepts the reference. A second named face gets the same call, and its own accepted reference. Do not put both faces on one sheet.
   **Every shot, before the job.** A second face goes in the same job. Set that shot's `on_camera` to his name and put his reference in `casts`. The drift gate grades that shot against him. Build each shot, including a lead-only shot, as its own still: `create_media({ action: "generate", model_override: "kontext-edit", source_url: <that face's reference>, aspect_ratio: <the film's>, prompt: <the shot, the set sentence verbatim, one on-camera face, three-quarter toward the camera> })`. `critique_shot` against that same reference, threshold 0.85, passes only when `pass` is true and `lead_count` is 1. Animate the approved still, not the portrait: `create_media({ action: "animate", model_override: "ltx-25-i2v-fast", source_url: <the approved still>, cast: { reference_url: <that face's reference> }, duration: 6, prompt: <the motion, one continuous take, no cuts> })`. Motion is refused when that still is under 0.85 or `lead_count` is not 1. `ltx-25-i2v-fast` accepts 6, 8, 10, 12, 14 or `auto` here: the floor is 6, the step is 2, and `create_media` caps `duration` at 15. A 4 second ask is refused, and so is 16 and up — both before anything is dispatched. The previz call trims it to the shot length; check the result before moving on. Watch the start, middle and end. A shot moves to `kling-v3-turbo-i2v` only after the competency card passes, and only by animating that same approved still.
3. **Draft and stop.** This is the hero pass, and only after the previz reel is accepted. Put every shot in the job. Each scene sets `on_camera` to the one face in that shot. `casts` lists every face's reference. A shot is graded against that face, not against the lead. `character_anchor` stays the lead's reference. A still under 0.85, or whose `lead_count` is not 1, is not animated. The job returns `reel_url`: every shot, trimmed to its seconds, in screenplay order. It does not loop a short clip. The rest of the call is the shot-list fields that remain (`duration`, `style` set to the style bible, `aspect_ratio`, `shot_strategy: "keyframe_then_animate"`, `keyframe_review: true`, `global_model_override: "kling-v3-turbo-i2v"`, `budget_usd`). A job over the budget is refused, not trimmed. Leave `deliver` unset.
4. **Consistency check, before the user sees the stills.** `review_checkpoint` must list a still for every scene. For every still the lead is in, `critique_shot({ generated_url, reference_url, entity_name, threshold: 0.85 })` passes only when `pass` is true and `score.lead_count` is 1. `gradeable: false` is not a pass. An insert without the lead is looked at, not graded. Then `critique_batch` as in the festival checklist. Look at the action, the staging, the weather and the wardrobe yourself.
   Reshoot a failing still with `resume_from_checkpoint({ checkpoint_id, decision: "reshoot", scenes: [{ ref: "3" }] })`. To change what is in the shot, `decision: "edit"` with `scenes: [{ ref: "3", modification: "…" }]`. Each stop has a new `checkpoint_id`; read it from `get_creative_job`. Edits stack, so write each modification as a complete direction. Edits and reshoots together are at most twice per scene. Then show the still flagged.
5. **The user steers.** `review_checkpoint` shows the stills. Then `resume_from_checkpoint`:
   - One still: `decision: "edit"` with that scene's `modification`, or `decision: "reshoot"` to re-roll it.
   - The whole direction: `decision: "edit"` with a `modification` and no `scenes`. The job stops again.
   - Happy: `decision: "continue"`. The approved stills are animated as they are.
   - Stop spending: `decision: "abort"`.
   Every re-rendered still goes through step 4 again.
6. **After the motion.** Look at the start, middle and end of every clip. Fix one shot and leave the others:
   - **Same still, new motion.** `director_re_render({ project_id, scene_index, from_previous: true, action: "animate", model_override: <the film's motion model>, prompt: <the shot, plus the style bible> })`. If the clip is longer than the shot, `director_retime({ project_id, scene_index, target_duration: <seconds> })`.
   - **Wrong content.** A new still, then motion. `director_re_render({ project_id, scene_index, action: "generate", prompt: <the shot, plus the style bible>, cast: { name: <the on-camera face>, reference_url: <that face's reference> } })` replaces the clip with a still. Grade it as in step 4. Do not animate that still until the user accepts it. If it repeats the mistake, edit that still: `director_re_render({ project_id, scene_index, from_previous: true, action: "generate", prompt: <what to change, in one sentence> })`. Then `director_re_render({ project_id, scene_index, action: "animate", from_previous: true, model_override: <the film's motion model>, prompt: <the motion line> })`. Read `cast.applied`. A false means the reference was not used. Say so, and do not treat the shot as locked.
   - **A blank still twice** was refused by a content filter. Reword the beat to show the result rather than the touch.
   - **An insert without the lead** is made without `cast`. Edit a neighbouring still that has no other character in it: `create_media({ action: "generate", model_override: "kontext-edit", source_url: <that still>, prompt: <the insert>, aspect_ratio: <the film's> })`, then animate it onto the scene with `scene_index`.
   - **A back view** can pass `lead_count` 1. Reshoot it facing the camera.
   - **Add a scene.** `director_insert({ project_id, between: [N-1, N], prompt, cast: { name, reference_url } })` makes the still. `cast.applied` confirms the character was carried. Then animate it with `director_re_render` as above.
7. **Post.** Sound, edit and frame, exactly as in stage 3. The export uses `transition: "cut"` and `soundtrack: "off"`. The mix goes on after.
8. **Festival checklist.** Hand the reel over when every line passes, or name the line that does not, and why.

## Control

- A model the user names is the model that runs. `global_model_override` covers every scene. `keyframe_model_override` pins the stills; keep it an editor that can see the reference (`kontext-edit`). `strict: true` means only that model, and a clear failure instead of a substitute. `seed` makes a re-run reproducible where the provider honours seeds. A substitution is reported in `model_note`. Relay it.
- For a film you will show, pin `kling-v3-turbo-i2v` (about $0.12 per second). The fast default `ltx-25-i2v-fast` is a rough cut. Any `*-draft-*` model is a preview.
- With no pin, say what was chosen. Relay `warnings`, each scene's `model_note`, and `entity_routed` / `entity_skipped`.

## Known limits

- You cannot drop a scene at the gate. `abort` and resubmit without it, or `continue` and leave it out of the export.
- `character_anchor` is the lead. A scene's `on_camera` names the one face in that shot, and `casts` holds every face's reference. That shot is graded against that face. A prompt that names two faces, with no `on_camera`, is not rendered.
- A scene that deliberately shows the lead twice fails `lead_count`. Say so, and accept it.
- A still that cannot be rendered fails the scene. It does not fall back to a video model that cannot see the reference.
- `director_export` blends at every join of up to 12 scenes when the transition is not a cut. Each blend overlaps 0.3 seconds.
- The "cannot see your character reference" warning appears when a scene really rendered without the reference.
- There is no loudness normaliser. Listening is the check.

## FINAL OUTPUT

Hand back, in this order: the reel URL, the `viewer_url`, the approved stills, the total spend against the quote, and every substitution the server reported.

## AFTERCARE

- One shot: step 6. Never re-submit the whole job. That re-buys every finished scene.
- A 9:16 cut: `create_media` with `action: "generate"`, `model_override: "ffmpeg-export"`, the reel as `source_url`, `aspect_ratio: "9:16"`.
- The project persists. Keep the reference URL, the style bible and the motion model with it.

Copy linkCopy content