It is late. The render queue is moving, the fans are spinning, and a tiny fictional world is slowly becoming a film. On screen, a camera sweeps past an orbital station. In another project, clay creatures wander through a hand-built-looking forest. Somewhere else, a puppet family watches the sea.
It looks like magic. But here is the thing: the most important part of FrameForge is not the final video model. It is everything I built around it.
FrameForge started as a simple question: could I turn one creative idea into a properly directed short film using AI? Not just a flashy four-second clip. Not a random montage. A film with a beginning, a change, a payoff, consistent characters, deliberate camera direction and audio that actually belongs to the scene.
That question took me much further than prompt engineering. I ended up building a small AI production system: a story director, structured quality checks, generated opening and closing frames, human approval gates, scene-by-scene video generation, durable background jobs, cost telemetry and a final assembly line.
My biggest lesson? Generative video stops being a prompting problem surprisingly quickly. It becomes a production-engineering problem.
Why “type a prompt, get a film” falls apart
A single prompt can produce a brilliant clip. But ask that clip to become part of a longer story and the cracks appear. A character changes clothes. An object jumps across the frame. The camera crosses the line. The emotional beat repeats instead of advancing. The narration needs 12 seconds, but the visual only lasts eight.
Those are not necessarily model failures. They are direction failures.
Traditional filmmaking has a script, storyboard, shot list, continuity notes, production schedule, edit and review process. My turning point was realising AI filmmaking needs many of the same disciplines. The tools are different, and the speed is incredible, but the need for intentional direction has not disappeared.
So FrameForge does not ask one model to be writer, director, cinematographer, animator, editor and sound designer all at once. It gives each model a clear job, passes structured information between stages, and puts the expensive work behind checks.
The model team behind FrameForge
The build uses Microsoft Foundry as the model and agent foundation. The exact model can evolve, but the roles are deliberately separated:
- A story director turns the creator’s brief into a structured storyboard. The agentic path can use web grounding for research-led projects, whilst a direct reasoning path keeps simpler jobs fast and deterministic.
- A reasoning model produces titles, scene beats, durations, camera direction, continuity notes, opening states, closing states and audio instructions as validated structured output.
- GPT Image creates the visual anchors: an opening keyframe and a closing keyframe for each scene.
- Sora generates each approved scene as a separate video clip rather than gambling the whole film on one generation.
- Azure Speech or scene audio supplies the soundtrack path, depending on the film style.
- FFmpeg handles the gloriously unglamorous final job of joining the approved pieces into a finished film.
That separation matters. It means I can improve one stage without rebuilding the entire system. I can swap a planning model, tune image generation, change the video engine or add a new audio path whilst preserving the production contract between them.
It is not just multi-model AI for the sake of it. It is a maintainable production line.
Structured output is where the magic becomes reliable
One of the least cinematic parts of the build made one of the biggest differences: the storyboard is data.
Each scene has a fixed duration, a causal story beat, a visible action, an opening composition, a changed closing composition, continuity notes and audio direction. That structure lets the application do something a chat box cannot do reliably on its own: inspect the plan before it spends money.
FrameForge checks whether the scene durations add up. It looks for vague or repeated story beats. It verifies that the opening and closing states are meaningfully different. It flags missing continuity anchors and narration that cannot fit into the available seconds. If a blocking issue appears, the planning layer gets one focused repair attempt.
This is a brilliant pattern for any AI application, not only video: use the model for creative or ambiguous work, then use deterministic code for rules you can state precisely.
The AI can decide what the lighthouse keeper discovers. Code can decide that 46 narration words will not fit comfortably into an eight-second scene.
Start frames, end frames and the continuity battle
Continuity was one of the hardest problems. Text-to-video is wonderfully creative, but creativity is not always what you want when scene two must inherit the exact state created by scene one.
My answer was to direct every scene between two visual anchors. The opening frame says, “this is the world at the start”. The closing frame says, “this is the changed world the next cut must inherit”. Motion direction then describes how the scene travels between those states.
It does not make continuity perfect. Nothing honest about generative video should promise that. But it turns continuity from a vague hope into an explicit part of the production design.
There is also a human approval gate before video generation. That is intentional. Video seconds are the expensive part, so the creator can inspect the story and keyframes first. If the character, composition or direction is wrong, it is far better to discover that before rendering every scene.
Content safety: adapt the direction, never bypass the guardrails
Content safety became one of the most interesting parts of the journey.
Creative prompts are messy. A harmless dramatic scene can use language that sounds much more severe when read without cinematic context. A request for a well-known character or recognisable person can cross rights or identity boundaries. Web-grounded planning also introduces a different risk: untrusted text can contain instructions that the agent must never follow.
The wrong response would be to weaken the safeguards until the prompt goes through. I did the opposite. FrameForge keeps mandatory platform protections active and adds an application-level adaptation layer before media generation.
- Visible people are described as original, fictional and non-identifiable.
- Brands and protected designs become original, unbranded creative direction.
- Sensitive story stakes are translated into implication, reaction, lighting, sound and consequence rather than graphic detail.
- Harmless wardrobe, relationships, movement and camera intent are preserved.
- Web-grounded research is separated from private reference media, and prompt-injection protection remains part of the planning boundary.
That distinction is massive. Safety adaptation is not censorship-by-keyword, and it is not a jailbreak. It is direction. You preserve the creative intent whilst expressing it in a form the production system can handle responsibly.
The result is usually a better prompt as well: clearer subjects, more original worlds, less accidental ambiguity and fewer failed generations.
The unglamorous engineering that saves the project
The demo moment is always the generated film. The production work is everything that stops a 20-minute render from becoming a very expensive spinner.
FrameForge publishes work scene by scene. Completed outputs are recorded durably. Background queues can resume. If scene five fails, the system should not forget scenes one to four. Usage is captured per operation, with pricing status and versioned retail estimates rather than one mysterious total at the end.
Private references are owner-scoped. When the story director uses web grounding, it receives privacy-filtered descriptions rather than raw private image bytes. Approval and update flows use concurrency checks so an older browser tab cannot silently overwrite a newer decision.
None of that looks as exciting as a clay forest. All of it is what turns a prototype into a product.
What does an AI film actually cost?
Costs vary by region, deployment type and date, so treat this as a transparent retail illustration, not a quote. The rates below are the public USD list-price assumptions captured by FrameForge on 30 August 2026, before tax and without private discounts.
| Stage | Retail assumption | Worked example |
|---|---|---|
| GPT-5.6 Sol planning | $5 per 1M input tokens; $0.50 cached input; $30 per 1M output tokens | 10,000 input + 5,000 output tokens = $0.20 |
| GPT Image 1.5 | $5 per 1M text input tokens; $8 per 1M image input tokens; $32 per 1M image output tokens | Actual cost depends on the reported image tokens, size, quality and number of keyframes |
| Sora 2 | $0.10 per generated video second | 60 generated seconds = $6.00 |
| Azure Speech | $15 per 1M characters | 1,000 narration characters = $0.015 |
| Agent model and web grounding | Rate unavailable in the captured retail feed | Excluded rather than guessed |
The lesson is obvious when you see the maths: video seconds dominate. A second full 60-second attempt adds another $6 of raw video generation before images, planning and speech. Twelve keyframes cost more than six. Regenerating only the failed scene is cheaper than rebuilding the film.
This is why quality gates, approval and scene-level retries are not just good user experience. They are cost architecture.
You can check current pricing on the Azure OpenAI pricing page and Azure Speech pricing page. The application records measured usage and the pricing version so estimates can be explained later.
Six worlds, one production system
This is the bit I love. The same production workflow can create orbital realism, product-film energy, tactile stop-motion, puppet theatre, watercolour motion and macro nature. The visual style changes wildly; the production discipline underneath does not.
That is the thought I want other builders to take away. Your moat is not one perfect prompt. Models will improve. Prices will move. New capabilities will arrive. The durable value is the system that captures intent, routes work, validates quality, protects people and data, controls cost, preserves completed outputs and keeps a human in charge of consequential decisions.
A practical blueprint for your own build
- Start with the creative contract. Define audience, duration, style, references and what “good” means before choosing models.
- Give each model one clear role. Planning, image, video and audio need different instructions and different evaluation.
- Use structured outputs. If the next stage depends on a value, make it data rather than prose you hope to parse.
- Validate deterministically. Duration, missing fields, repeated beats and narration speed do not need another model call.
- Approve before the expensive step. Let humans inspect the plan and visual anchors before generating video.
- Design safety into the creative workflow. Adapt prompts and preserve intent; never pretend platform safeguards do not exist.
- Make retries granular and resumable. A failed scene should not erase a successful production.
- Meter everything. Record usage, rate availability, pricing date and retry exposure.
If you are building with generative media, my encouragement is simple: go beyond the clever demo. Build the boring rails. Add the quality gate. Show the cost. Keep the approval step. Give every model a job description.
Example Videos: