---
title: "Typographic social cut — captions, tracked overlays & on-video type, agent-native"
tier: hero
format: video-editing
theme: social | editorial | brand | agentic-editing
persona: social editor, brand studio, dev-rel, founder, growth marketer
duration: "one call per layer → a finished vertical cut in minutes"
budget_usd: "~$0.30–0.80 per 5–15s cut (talking-head/i2v dominates; captions + overlay + grade are ~$0.001/s each)"
caps: ["chatterbox-tts", "talking-head", "flux-dev", "transcribe", "whisper-word", "hyperframes-render", "ffmpeg-overlay", "ffmpeg-colorgrade", "list_capabilities", "describe_capability"]
skills: ["base", "finishing-quality", "cinematography"]
showcases: ["/chapters/typographic-social-cut-example.html"]
status: "live — the agentic-video-editing cut. Word-level captions, timed picture-in-picture at any x/y, pixel-perfect HTML/CSS typography over footage, and a finishing colour grade — every layer a single reproducible tool call, no NLE. Born from the a16z 'agentic editing' field test that these four capability fixes closed."
reliability: 4.3 # deterministic ffmpeg + hyperframes finishing; the talking-head / i2v base is the only latency-variable step
---

# Typographic social cut — the a16z "agentic editing" brief, now reachable

A creative-agency field test asked an agent to do what social editors do by hand every day:

> *"Add captions in condensed type; hit every word on the hero line; drop the attached clip in as an overlay that appears when I raise my hands; add molecular graphics in the a16z style; grade it."*

At the time the network **could not reach it** — the job fell back to local OpenCV + ffmpeg and shipped at **$0.00 network spend**. Not because the compute was missing, but because the capability cards didn't publish enough to *plan* or *call*. Four fixes closed that gap ([the analysis](/gtm-editing-gaps-analysis.html)); this playbook is the repeatable recipe that came out the other side.

## What you'll get

A finished **vertical (9:16) social cut** with, each as one reproducible call:

- **Word-by-word captions** that hit every word — real ASR word timings (`whisper-word`), styled fill / stroke / size / position.
- **Timed, positioned picture-in-picture** — an overlay clip or logo that appears only in a time window, at any `x/y` (not just a fixed corner).
- **Pixel-perfect on-video typography + graphics** — condensed type, kinetic captions, a decorative graphics layer (molecular network, ticker, lower-third…) composited from HTML/CSS over the footage via `hyperframes-render`.
- **A finishing colour grade** — a named look (`cinematic` / `warm` / `bw` …) or explicit brightness / contrast / saturation / gamma.

**No NLE, no local ffmpeg, no hand-keyed timings.** Every layer is an agent tool call you can re-run against a new take.

## Tell the agent about the cut

```yaml
# ── THE CUT ──────────────────────────────────────────────────────────
source_video:      # public URL of your footage (talking-head / product / b-roll). OR leave blank + fill source_still + vo_line to have the agent build a talking-head.
source_still:      # OPTIONAL — a presenter/product image URL, if you want the agent to generate the base clip.
vo_line:           # The spoken hero line, e.g. "We are bringing Livepeer Agent to market." (drives the word-level captions.)
aspect_ratio:      # 9:16 (default) | 1:1 | 16:9

# ── DESIGN SYSTEM (swap freely — this is the customization surface) ───
design_ref:        # OPTIONAL — a design.md / brand URL. The agent honors it (type, palette, "stillness", "no decoration"…).
palette:           # e.g. ["#F5E400", "#FF2D9B"] editorial, OR a neutral brand set. Empty = footage is the only colour.
type_style:        # e.g. "condensed all-caps, tight tracking" (editorial) | "sentence-case Inter, balanced wrap" (UI/system)
caption_style:     # e.g. "active word yellow slab, spoken white" | "active semibold, spoken muted — weight not colour"
graphics:          # e.g. "a16z molecular network" | "capability ticker" | "none — default to stillness"
grade:             # cinematic | warm | cool | vibrant | bw | none

# ── OVERLAYS (0-N timed picture-in-picture) ──────────────────────────
overlays:
  - clip:          # public URL of the overlay image/clip
    x:             # 0-1 fraction of frame (0=left, 1=right)
    y:             # 0-1 fraction of frame (0=top, 1=bottom)
    scale:         # 0.05-1.0 relative width (e.g. 0.30)
    start_sec:     # when it appears
    end_sec:       # when it leaves
```

## The recipe (what the agent runs)

Each step is one call; each output feeds the next. Skip any layer you don't need.

**0 · (optional) Build the base clip.** If you only have a still + a line: `chatterbox-tts` the `vo_line` → `talking-head` (still + audio) for a lip-synced base *with the speech track* (word-level captions read that track). Already have footage? Skip to step 1.

**1 · Word-level captions (RC-3).** `transcribe(source_url, granularity:"word", burn:true, font_size, font_color, outline, caption_position)`. Word granularity routes ASR to `whisper-word` for per-word cues — the "hit every word" effect — and the styling is forwarded to the burn. `granularity:"segment"` (default) stays phrase-level.

**2 · On-video typography + graphics (RC-1).** `hyperframes-render(background_video_url, html, aspect_ratio, duration)`. Write the `type_style` + `graphics` + `caption_style` as ordinary HTML/CSS (absolute-positioned layers, `@keyframes` for kinetic captions, inline SVG for a molecular network / lower-third). This is the only primitive that renders real typography over footage — pass it a `design_ref` and honor it.

**3 · Timed tracked overlays (RC-2).** For each entry in `overlays`: `overlay(source_url, image_url:clip, x, y, scale, start_sec, end_sec)`. The overlay appears only in `[start_sec, end_sec)` at your `x/y` — the "drops in when I raise my hands" beat. (For a subject-*following* overlay, see Roadmap.)

**4 · Colour grade (RC-4).** `run_capability("ffmpeg-colorgrade", { source_url, preset })` — or explicit `brightness/contrast/saturation/gamma`. The finishing look.

**5 · (optional) Aspect variants.** Re-run steps 2–4 with a different `aspect_ratio` for a 1:1 / 16:9 sibling.

## Customizing for similar use cases

This is a *template*, not a fixed pipeline — the same four calls cover a wide family of cuts:

| You want… | Change |
|---|---|
| **A different design system** (UI/neutral vs editorial) | Swap `type_style` / `palette` / `caption_style` / `graphics` in the brief. Pass a `design_ref` and the agent honors "sentence case", "use the neutral system", "default to stillness", "no decoration". The footage becomes the only colour when `palette` is empty. |
| **Product / e-commerce cut** | `source_video` = your product b-roll; `graphics` = price/spec lower-third; `overlays` = a detail crop PiP at the reveal beat; `grade: vibrant`. |
| **Explainer / breaking-news** | `graphics` = a ticker + kicker; `caption_style` = high-contrast; add a `manim-render` diagram as an overlay clip. |
| **Localized versions** | Re-run step 1 per language (`transcribe` auto-detects; word timings per language) — keep the same typography layer. |
| **Brand-locked repeatability** | Bind a `brand_kit` (palette + font intent + a `finishing_chain`) and drive steps 2–4 from it so every cut matches. |
| **Longer reel** | Longer `source_video` (or i2v/veo base) — the finishing layers are duration-agnostic. |

## Why this is possible now (the underlying fixes)

Every layer above maps to a capability that the field test proved *existed but was unreachable*. The fixes were mostly **contract, not compute**:

- **RC-1 — the capability contract.** Tool caps now publish a real input schema (so an agent can call `hyperframes-render` first-try instead of a probing round-trip) and reconcile price so you can cost before running. `describe_capability` is now honest.
- **RC-2 — timed / positioned overlay.** `ffmpeg-overlay` gained `x/y` + `start_sec/end_sec` — a timed PiP is one call, not 14 sliced scenes.
- **RC-3 — word-level captions.** `transcribe` gained `granularity:"word"` (routing to the live `whisper-word` cap) + styling passthrough — word timings come from the network, not local faster-whisper.
- **RC-4 — colour grade.** A new `ffmpeg-colorgrade` cap — the finishing grade the tool caps were missing, *stated* rather than discovered.

See the full write-up: [GTM editing-gaps analysis & proposal](/gtm-editing-gaps-analysis.html).

## Also shipped — subject-following overlay (RC-2 S2b)

Beyond a *timed + positioned* PiP, the overlay can now **follow a moving subject**. Pass `overlay` a `track` — a keyframed path `[{t,x,y}]` (x/y normalized 0–1) — or a `track_sidecar_url` (+ optional `track_class`) and it derives the path from a `yolo-detect` / `supervision-track` sidecar's box centers. Under the hood, `ffmpeg-overlay` writes an ffmpeg `sendcmd` script that re-positions the overlay per keyframe. This closes the field test's "overlay stuck to a moving subject" demo — the tracked-overlay anchor contract, not a general reframe engine.

## Roadmap (honest gaps)

- **Per-region / LUT grade** — `ffmpeg-colorgrade` is a global `eq` grade; a `.cube` LUT path is a natural follow-on.
- **Smooth (interpolated) tracking** — the tracked overlay steps between keyframes (holds each position until the next); dense keyframes read as smooth, but true per-frame interpolation is a follow-on.
- **Tool-verb parity** — the timed/tracked-overlay and word-caption params are live on the underlying caps and the `overlay` / `transcribe` verbs; a dedicated Studio UI toggle is polish, not a gap.