Pavan Varahad V

Home / Blog / Video generator

Seven agents, one animated video

From a topic to the final cut: a multi-agent system, inspired by MetaGPT, that turns a topic into a narrated educational animation using Manim and the Gemini API.

  • Multi-agent
  • LLMs
  • Manim
  • Gemini API
  • Python

Give it a topic, and it hands back an MP4: an animated explanation with a voice-over, built from code. No video editor, no recording, no manual animation.

Instead of one big model doing everything, I designed seven specialised agents that collaborate like a small team. The interesting question was never "can an LLM write a script". It was how to make a chain of LLM steps reliable enough that the final video renders, and how to split the work so each step has one clear job. This post covers the design, and the part that mattered most: validating generated code before it ever reaches the renderer.

The demo, from my LinkedIn post. Open on LinkedIn

Why several agents instead of one prompt

One giant prompt that says "make me a video about X" produces something that is hard to debug. When it fails, you cannot tell whether the lesson was badly planned, the narration was off, or the animation code was broken.

Splitting the work into agents, in the style of MetaGPT, gives each stage a narrow task and a defined output that the next stage can rely on. A failure then points at one agent, and each agent's prompt can be improved on its own.

The pipeline

Seven agents, one pipeline

  1. 01Plannerlearning objectives and a scene breakdown
  2. 02Scriptscene-wise narration
  3. 03Code generationManim Python code per scene
  4. 04ValidationAST check and an execution test
  5. 05Voicetext-to-speech audio for the narration
  6. 06Composerputs the animation and the audio together
  7. 07Mergerjoins the result into the final MP4

The plan is the contract: every later agent works from the same scene breakdown.

1. Planner: decide what the video is about

The planner defines learning objectives and breaks the topic into scenes. This is the most leveraged step, because everything downstream inherits its structure. A vague plan produces vague narration and vague animation, no matter how good the later agents are.

2. Script: say it scene by scene

The script agent writes narration for each scene. Keeping narration tied to scenes, instead of one long monologue, means the audio can later be matched to the right animation.

3. Code generation: write the animation as code

The code agent synthesises Manim Python code for each scene. Manim is a good target for an LLM because animations are described programmatically: shapes, text and transformations are all code, so the model can generate them directly.

4. Validation: the step that makes it work

LLM-written code is plausible, not guaranteed correct. A missing import, a wrong method name or a bad LaTeX string will break a render. So the validation agent checks every scene before anything is rendered into the final video, in two passes:

Validation loop

Agent 3Code generationwrites Manim code for a scene
↓
Agent 4 · pass oneAST checkdoes the code even parse, and is its structure sound? Fast and cheap.
↓ passes
Agent 4 · pass twoExecution testdoes it actually run? Catches errors that only appear at runtime.
↓ passes
NextScene acceptedmoves on to voice and composition

When a check fails, the error goes back to the code agent so it can fix the scene and try again. That loop is what cuts the human effort per video.

The order is deliberate. Parsing the syntax tree costs almost nothing, so it goes first and weeds out the obvious breakage. Actually executing the scene is slower, so it only runs on code that already looks valid. Cheap checks first, expensive checks last.

5. Voice: narration to audio

The voice agent turns each scene's narration into audio with gTTS. A hosted text-to-speech service keeps the setup light: no model to download and nothing to run locally for this step.

6 and 7. Composer and Merger: put it together

The last two agents synchronise the rendered animation with the audio and produce the final MP4. (The project README describes this as one video composition stage; as a team of agents I count it as a Composer and a Merger.) The hard part is timing: narration length and animation length have to agree, which is why the plan and script are both organised per scene.

Setting it up

Install the Python packages:

pip install metagpt google-generativeai manim gTTS

Manim also needs system dependencies: FFmpeg and LaTeX. On Windows, make sure ffmpeg and manim are on your PATH. Missing LaTeX is the most common reason a first render fails.

The model is configured in config2.yaml:

llm:
  api_type: "gemini"
  model: "gemini-1.5-flash"
  api_key: "YOUR_GEMINI_API_KEY"

You can also set the GEMINI_API_KEY environment variable instead, which is the better habit: it keeps the key out of a file that might get committed. Then run python main.py.

What I learned

  • Give the pipeline a contract. The scene breakdown from the planner is the shared structure every other agent follows. Without it, the narration and animation drift apart.
  • Validate generated code like untrusted input. It is not malicious, but it is unverified. Parse it, run it, and feed failures back.
  • Make failures cheap and specific. A validation error that names the scene and the problem is something an agent can act on. A broken final video is not.
  • Pick a target the model can express. Animation described as code is a much better fit for an LLM than asking it to produce pixels.

Limits

  • Running is not the same as looking good. A scene can execute without errors and still be cluttered or confusing. Execution testing proves the code works, not that the lesson is clear.
  • Output quality depends on the model. The default configuration uses a small, fast Gemini model, so plans and scenes can be simple.
  • Heavy setup. FFmpeg and LaTeX make the first install harder than a pure-Python project.

What I would do next

These are ideas, not shipped work:

  • A visual check: render a frame or two per scene and ask a vision model whether the layout is readable, so the loop catches ugly scenes as well as broken ones.
  • Per-scene caching, so changing one scene does not re-render the whole video.
  • A human review step after the plan, where a person can approve or edit the scene breakdown before the expensive stages run.

Takeaways

  1. One job per agent makes failures easy to locate, which is why I split the work seven ways.
  2. Cheap checks first: AST before execution, execution before rendering.
  3. Close the loop. Send validation errors back to the agent that made them.
  4. Plan structure drives everything, so invest in the planner.

If you are building multi-agent pipelines, I would like to hear how you validate agent output before the next stage consumes it.

← All posts