How Open Source Films built an open-source VFX and agentic development pipeline on Runpod

About

LifeR, an independent feature film built around an entirely open-source, AI-assisted production pipeline, released by the not-for-profit Open Source Films

Industry

Media and entertainment, open-source R&D

Company size

Small core team currently expanding into dedicated production departments

The challenge

Turning generative video into assets a conventional VFX pipeline can use, and keeping multi-week agentic coding projects from losing context between sessions, without a studio budget or an infrastructure team

Most generative video tools produce an impressive clip and stop there. The clip isn't locked to the camera motion, depth of field, or color of the shot it's supposed to replace, so it can't drop into a real edit. That gap, between a demo and a usable VFX asset, is what Open Source Films spent the past year building against, on Runpod GPU pods that now run their generative VFX work and the agentic software development that built it.

open-source ComfyUI workflows built from production problems

public repos published from the production

90%

of the proof-of-concept scene's shots are VFX shots

Shot first, pipeline built second

The team shot its proof of concept first and built the pipeline second. In September 2025, roughly 20 cast and crew spent three days at a chapel north of Melbourne shooting a 15-minute scene from the feature, day for night, with half the building wrapped in scaffold and black plastic. Ninety percent of the resulting shots are VFX shots, most of them because a practical element in frame needed to be replaced or removed: flaming sconces, cable paint-outs, set fixes propagated through a moving shot from a single corrected frame. Finishing that footage required a VFX pipeline the team did not yet have, so the following year went into building one. The pipeline that resulted is now the pipeline for the feature.

‍

A hub-and-workers setup on Runpod Pods

The infrastructure layer comes first, because everything else depends on it. Rather than provisioning a fresh GPU pod from scratch for every job and re-downloading multi-gigabyte model checkpoints each time, the team built a hub-and-workers setup on Runpod Secure Cloud. One long-lived pod, which they call the mainvolume, holds a Runpod network volume with the shared ComfyUI models, footage, and output storage, and runs an SSH endpoint.

When they need generation capacity, they launch a GPU worker pod in the same datacenter as the mainvolume — most often an RTX PRO 6000, ranging up through RTX 5090s, RTX 4090s, H200s, and B300s depending on how much VRAM the job needs, with the team deliberately favoring lighter hardware where they can so the published tools stay accessible to anyone running less intense GPUs themselves, paste in a generated bootstrap block, and the worker installs its dependencies, clones the ComfyUI variants the production actually uses (a general install, WAN 2.2 image-to-video through WanVideoWrapper, VideoMaMa with SAM3 for matting, and an isolated EffectErase runtime, amongst numerous others), links them to one shared model directory so weights are downloaded once, pulls the SAM3 and WAN weights from the mainvolume before falling back to Hugging Face, and installs a handful of small commands for moving inputs and results back and forth over rsync.

The scripts are published as they ran in the team's orchestration repo, along with a deployment guide that documents the platform constraints they hit at the time, including an early window when B300 pods could be launched from the Runpod console but not yet through the REST API, a gap that has since closed.

We wrote tooling around worker setup and some of the practical problems we've encountered using newer GPU generations.

-- Daniel Connel, Open Source Lowtech Designer

Keeping agentic dev sessions coherent across weeks

On top of that infrastructure, the team runs Claude Code and other coding agents directly inside the persistent pod to do the development work that produced the tools described below. Long, complex projects tend to lose context between agent sessions: a new session picking up mid-project has no memory of what the last one tried, decided, or already ruled out. Open Source Films built BATON, an open-source tool that tracks project state, completed work, decisions, validation results, and known dead ends, so a new session continues rather than reconstructing the project from scratch.

It keeps track of project state, completed work, decisions, validation results and known traps, so a new agent session can continue from where the previous one left off rather than reconstructing the project from scratch or repeating failed work.

-- Daniel Connel, Open Source Lowtech Designer

From a generated clip to a usable VFX asset

That combination, persistent compute plus a way to keep agent sessions coherent across weeks, is what let a small team build production-grade tools instead of one-off scripts. The clearest example is the VFX pipeline itself. Platelock and Orbitturn take a photographed camera move, alter or reconstruct the environment around it, and propagate the new scene through the shot while preserving the original geometry, framing, focus progression, and camera movement.

Platelock registers the generated clean plate back onto the original photography and exports STMaps, the distortion maps a conventional compositor uses to warp one image onto another, so a generative clip becomes something a real compositing pipeline can consume. Automated matting and roto tools built on the same ComfyUI stack, using SAM3 and VideoMaMa, pull actors off their backgrounds in one click, work that would otherwise consume most of a VFX artist's time, leaving the team's artists for the handful of hero shots that still need a human eye.

A test lab for new models and hardware

The team also uses Runpod as a structured test lab for infrastructure decisions most productions never get to make deliberately: Blackwell/CUDA compatibility work, and comparisons of competing motion-generation models, including LTX and MiniMax H3, a newer open-weights video model Daniel says holds up well against proprietary options like Seedance and Kling, run against exactly the same production problem rather than a synthetic benchmark.

We've also been using Runpod as a practical test lab for emerging research.

-- Daniel Connel, Open Source Lowtech Designer

The proof-of-concept scene is now in final edit. The team shot that scene as a 15-minute proof of concept for an estimated AU$5,000 (US$3,500), including all VFX and post.

Conclusion

Over the past year, Open Source Films built a working, open-source production stack on Runpod: multi-pod orchestration tooling, a generative VFX pipeline, an agent-context system, and 16 published ComfyUI workflows. The proof-of-concept scene that drove all of it is in final edit. The team is now moving from building that infrastructure to using it, starting feature production with a shot Daniel calls the most technically ambitious in the entire film.

Results

16 open-source ComfyUI workflows built from real production problems, with custom nodes, pinned dependencies, and per-component licensing, published for anyone to use
A hub-and-workers pod architecture, published as the scripts that ran in production, that stands up a new B300 GPU ComfyUI worker against a shared network volume without re-downloading checkpoints
Automated matting and roto tooling removed most manual extraction work from a scene where 90% of shots carry VFX elements
Structured Blackwell/CUDA compatibility work and motion-generation model comparisons run against production problems, not synthetic benchmarks
Proof-of-concept scene shot for AU$5,000 (US$3,500), including all VFX and post