How to Use Claude to Edit Short-Form Videos: A Practical Workflow
A step-by-step look at using Claude, skills, and MCP tools like VidIQ and 11 Labs to script and auto-edit short-form talking-head videos.
What does it mean to edit video with Claude?
It means handing Claude raw, unedited talking-head footage and a set of connected tools, then letting it do the work a human editor usually does: cutting pauses, selecting B-roll, writing captions, adding music, and assembling a finished short. Claude doesn’t touch a timeline in Premiere or DaVinci Resolve. Instead, it runs the edit headlessly through FFmpeg, pulls supporting footage through tools like Hyperframes and yt-dlp, and uses VidIQ to figure out what kind of content and pacing tends to perform well before a single cut is made. The person demonstrating this workflow used it to produce Instagram and YouTube Shorts that pulled in anywhere from roughly 130,000 to over a million views, all edited by Claude rather than by hand.
TL;DR
- Claude Opus (via a Claude project with custom instructions) can take raw talking-head footage and return a cut, captioned, scored short-form video with minimal manual editing.
- The workflow combines Claude skills (reusable instructions Claude builds and follows) with MCP servers that give Claude access to outside tools and data.
- VidIQ’s MCP integration lets Claude research top-performing videos on a topic or channel, pull transcripts, and use that analysis to write a script and predict how well a draft will perform.
- FFmpeg handles the actual video editing behind the scenes, Hyperframes adds After Effects-style animation and motion graphics, and 11 Labs generates background music and sound effects.
- Claude edits using both the transcript and the audio waveform together, which helps it avoid cutting off words mid-sound, a common failure mode when relying on transcripts alone.
- The creator reports the AI-generated scripts work better as guidelines than final copy, since they can still read as noticeably AI-written if used verbatim.
- The same approach reportedly works with OpenAI’s Codex and GPT models (including Sora for generation), not just Claude, suggesting this is a pattern rather than a single-vendor trick.
● Reading src/server.ts
● Editing 3 files
+ import { limit } from 'rate'
- app.use(cors())
✓ Tests passed · ready for PR
How does the Claude video editing workflow actually work?
The process starts before any footage exists. You give Claude a source (a blog post, an announcement, a product page) and optionally a link to your own existing B-roll. Claude then uses VidIQ to search YouTube for other videos already covering that topic, pulls their transcripts, and identifies which ones performed best and why. That analysis feeds directly into a draft script, along with talking points pulled from top-performing videos and a rough performance score (the demonstrated example scored an 8 out of 10 from Claude’s own estimate).
From there, the human records the A-roll: a straightforward talking-head take, using the script as a loose guide rather than something to read word for word. That raw footage, pauses, stumbles, repeated lines and all, gets handed back to Claude along with a reference to the original source material.
Claude then works through a defined sequence: load the relevant skill, cut the A-roll, prepare B-roll and a music bed, write an edit plan, render, and do a quality pass before delivering the final file. In the demonstrated run, this took dozens of discrete steps, visible as a step-by-step execution log, including moments where Claude tried one approach (like controlling a browser directly), failed, and found a workaround on its own.
What tools and skills does it rely on?
The workflow is built around a Claude “project” with a few key pieces:
Skills. These are standing instructions Claude follows repeatedly, essentially a saved playbook. In this setup, Claude was asked to generate two skills: one for writing scripts likely to perform well, and one for editing raw footage into a finished video that matches the style of past successful content. Once built, these skills get reused on future videos without having to re-explain the process each time.
MCP servers. MCP (Model Context Protocol) is what lets Claude reach outside its own chat window and interact with external tools and services. Three were central to this setup:
- VidIQ: analyzes existing videos (your own channel or others) to find what performs well, both in script content and visual style, and can be queried for research while writing new scripts.
- Hyperframes: handles lightweight motion graphics and animation, similar in spirit to After Effects, used for things like generating a short clip of a UI being typed into when no real footage existed for it.
- 11 Labs: generates background music and sound effects dropped into the final cut.
FFmpeg and yt-dlp. FFmpeg does the actual cutting, trimming, and assembly, all without opening a GUI video editor. yt-dlp is used to download reference video from YouTube or Instagram so Claude can pull frames or clips for B-roll when needed.
Blender, mentioned as an optional addition, can be used to generate simple 3D objects or animations to drop into a video when relevant.
How do you set this up from scratch?
Setting it up doesn’t require coding. The process shown:
- Create a new Claude project (a Pro-tier Claude subscription is sufficient) and give it general instructions describing its role: a short-form video editor that writes scripts likely to perform well, cleans up raw A-roll, and adds B-roll, animation, subtitles, and audio.
- List which tools it’s allowed to use: VidIQ, Blender, Hyperframes, FFmpeg, yt-dlp.
- Under the project’s “customize” settings, connect the MCP integrations for VidIQ and Hyperframes, which involves a standard OAuth-style authorization in the browser.
- Point Claude at an existing YouTube or Instagram channel (or, lacking one, example channels to take style cues from) and ask it to analyze top-performing shorts using VidIQ, covering both script content and visual pacing.
- Ask Claude to generate two skills from that analysis: one for scriptwriting, one for editing, with explicit instructions to use both transcript and waveform data when deciding where to cut, so it doesn’t clip words mid-sound.
Once the skills exist, the day-to-day loop is short: send a source link, get a script back, record the A-roll, send the raw footage back, get an edited video.
Is this actually good enough to replace manual editing?
For short-form, talking-head content, the results shown are usable straight out of the process: subtitles added automatically, B-roll pulled both from supplied footage and from source material found online, generated background music, and a few AI-generated supplementary clips (via Hyperframes) for moments where no real footage existed. Several videos produced this way reportedly reached six and seven-figure view counts on Instagram and YouTube Shorts.
That said, scripts generated this way are described as better used loosely than verbatim, since AI-written scripts can still read as stilted if followed word for word. The workflow is also iterative: the first pass at building skills from channel analysis isn’t a one-and-done setup, it takes refinement to get the editing style to match. And this is specifically tuned for short-form, fast-cut, single-speaker content, not longer narrative edits or multi-camera production.
Frequently Asked Questions
Do I need to know how to code to set this up?
No. The entire setup happens through Claude’s project interface and chat, using plain-language instructions. The MCP connections (VidIQ, Hyperframes) are authorized through a browser login flow, similar to connecting any third-party app to an account.
Does this only work with Claude?
No. The creator notes the same approach works with OpenAI’s models, including Codex and Sora, for generation and editing tasks. Claude Opus was used for the demonstrated workflow, but the underlying pattern (skills plus MCP tool access) isn’t exclusive to one model provider.
What does VidIQ actually contribute to the process?
VidIQ’s MCP integration lets Claude search and analyze existing YouTube videos, whether from your own channel or competitors, pulling transcripts and performance data. Claude uses that to figure out which talking points and pacing tend to perform well before writing a script, and can also score a draft’s likely performance.
Why does Claude need both the transcript and the waveform to edit?
Relying on the transcript alone can cause Claude to cut a clip exactly where a word ends on paper, which sometimes falls before the actual sound has finished in the audio. Checking the waveform alongside the transcript helps it cut after the sound naturally ends, avoiding clipped or truncated words.
Can this generate B-roll, or does it only use footage I provide?
Both. Claude can pull B-roll from footage you supply, download relevant clips from source material online (via yt-dlp), and generate supplementary visuals itself using Hyperframes or Blender when no suitable real footage exists for a given moment in the script.
