Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ __pycache__/
.env
runs/
.DS_Store
site/
48 changes: 43 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,14 @@ your own ComfyUI.
| **Disintegrate** | **Colossus** | **Zero Gravity** | **Frozen Rain** (beta) |
| ![](web/thumbs/disintegrate.jpg) | ![](web/thumbs/colossus.jpg) | ![](web/thumbs/zero-gravity.jpg) | ![](web/thumbs/frozen-rain.jpg) |

## Try it

* **In your browser, no key:** [vfx.nomastudio.ai](https://vfx.nomastudio.ai) plays every effect.
Pick one and open it as a Comfy Cloud app: sign in with your Comfy account, add a photo and run
it. The run bills your own Comfy account.
* **On your machine, everything:** the full app adds video mode, a second photo for the likeness, 9:16
and HQ. It runs with your own Comfy API key, see [Run it](#run-it).

## Two ways in

**From a photo.** Upload one or two photos of a person. Each preset builds its own film scene
Expand All @@ -25,11 +33,16 @@ start, and VFX 4 All continues the shot from that exact frame. The result is you
cut, then the effect, joined on the same frame so the cut is invisible. Clips from 2 to 60 seconds,
480p to 4K, up to 500 MB (`VIDEO_LIMITS` in `vfx4all/config.py`).

No photo or clip at hand? Each mode loads our own samples with one click (see [samples/](samples/)).
Under **More options**, an extra direction in English goes after the preset's effect prompt, and a
seed repeats a take.

**In Comfy Cloud, no key at all.** Every preset that animates a single first frame (41 of the 45)
can also run as a Comfy Cloud App Mode app: one graph that takes your photo, builds the scene and
renders the effect. You open it with your own
Comfy account and run it with your own credits; nobody hands an API key to anyone.
`tools/build_app_graph.py` builds those graphs from the presets.
`tools/build_app_graph.py` builds those graphs from the presets, and each preset keeps the share link
of its app in `comfy_app`.

## How it works

Expand All @@ -56,8 +69,8 @@ first frame --> Qwen Image Edit 2511 (or a mirror) --> last frame --> LTX-2.5 fi
repeats the uploads and downloads that time out, and splices videos with ffmpeg.
* `web/` is the interface: plain HTML, CSS and JavaScript, no build step. It shows the prompt that
was sent, keeps your results across reloads and compares a spliced clip with your original.
* `tools/build_app_graph.py` builds the App Mode graphs and `tools/batch_presets.py` renders test
versions of the presets; `tools/dev/` keeps the A/B scripts behind the choices in
* `tools/build_app_graph.py` builds the App Mode graphs, `tools/build_site.py` builds the static
catalog site and `tools/batch_presets.py` renders test versions of the presets; `tools/dev/` keeps the A/B scripts behind the choices in
[workflows/README.md](workflows/README.md).

## Run it
Expand All @@ -76,14 +89,17 @@ cp .env.example .env # then paste your COMFY_API_KEY
ffmpeg ships inside the `imageio-ffmpeg` wheel, so there is nothing else to install.
`requirements.txt` pins the exact versions the app is tested with.

No photo or clip at hand? `samples/` has two photos and two clips to try both modes; see
[samples/README.md](samples/README.md) for which effect to pair with each.

### Run the tests

```bash
.venv/Scripts/python -m unittest -v
```

The tests run offline in a few seconds. They check every preset and graph, the App Mode builder,
the upload limits, the ffmpeg steps (phone clips shot vertical included), the HTTP API with its
the static site build, the upload limits, the ffmpeg steps (phone clips shot vertical included), the HTTP API with its
hosted mode, access code and daily cap, the job queue, and that every model the graphs load is
listed in `THIRD_PARTY_NOTICES.md`. They never reach Comfy Cloud (the test package blanks
`COMFY_API_KEY` before the app loads), so they cost nothing. GitHub Actions runs them on Linux and
Expand All @@ -102,8 +118,14 @@ Every generation bills the Comfy Cloud workspace that owns the API key it runs w
uses it for that job only and never stores it. If they leave it empty, the server key from `.env`
is used, so read [Hosting it](#hosting-it) before you expose the app.
* **Your own GPU, no cloud credits:** the models are open weights, so the same app can drive your own
ComfyUI through [comfy-api-proxy](https://github.com/Comfy-Org/comfy-api-proxy). Set
ComfyUI through [comfy-api-proxy](https://github.com/Comfy-Org/comfy-api-proxy), with no key. Set
`COMFY_BASE_URL=http://127.0.0.1:8189` in `.env`.
* **A Comfy deployment:** a ComfyUI of your own on a dedicated cloud GPU, billed while it is up. Set
`COMFY_BASE_URL` to its URL.

[docs/RUN_ON_YOUR_COMFYUI.md](docs/RUN_ON_YOUR_COMFYUI.md) lists every model file the graphs load, with
its source, revision and download command, and the steps for the proxy and for a deployment. The top
bar of the app says where it runs.

Only open-weights models run in the workflows, so no partner API credits are used: Comfy Cloud
charges GPU time. In our tests on Comfy Cloud a scene took about 10 seconds of GPU and a 6 second
Expand Down Expand Up @@ -235,6 +257,22 @@ sends a strict Content Security Policy and serves the fonts itself, so the page
other sites. If you host it for others, the LTX license asks you to put its Section 4 and the
Acceptable Use Policy into your own terms (see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)).

### The catalog site

A public page that spends nothing does not need the server at all. `tools/build_site.py` writes
`site/`: the same interface in static mode, where every card plays its preview and opens the
preset's Comfy Cloud app (`comfy_app`), so each visitor runs it with their own Comfy account. The
studio and the key settings are left out, `presets.json` replaces the API, and `vercel.json` sends
the same security headers as the server.

```bash
.venv/Scripts/python tools/build_site.py --demo path/to/demo.mp4 # --demo is optional
cd site && vercel deploy # or any static host
```

On Vercel, the first deployment of a project goes to production and takes the project's domains,
even without `--prod`.

## Consent

Only use photos and videos of people who agreed to it. The app asks you to confirm that before
Expand Down
7 changes: 4 additions & 3 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ not licensed under MIT.
## Demo media

The thumbnails in `web/thumbs/`, the preview loops in `web/previews/` and the hero image and video in
`web/media/` were made with VFX 4 All from photos of members of the NoMa Studio AI team. They show
real people, so they are not licensed under MIT: do not reuse them outside this project. Use your
own media when you fork it.
`web/media/` were made with VFX 4 All from photos of members of the NoMa Studio AI team, and the
photos and clips in `samples/` show the same people. They show real people who agreed to this use,
so they are not licensed under MIT: do not reuse them outside this project. Use your own media when
you fork it.
142 changes: 142 additions & 0 deletions docs/RUN_ON_YOUR_COMFYUI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Run VFX 4 All on your own ComfyUI or on a Comfy deployment

VFX 4 All talks to the Comfy API v2 through `comfy-sdk`, so three places can run its graphs:

| Where | `COMFY_BASE_URL` in `.env` | `COMFY_API_KEY` | Who pays |
|---|---|---|---|
| Comfy Cloud (the default) | leave it out | your Comfy API key | your Comfy workspace, for the GPU time |
| Your own ComfyUI, through [comfy-api-proxy](https://github.com/Comfy-Org/comfy-api-proxy) | `http://127.0.0.1:8189` | leave it empty | nobody: it runs on your GPU |
| A Comfy Developer Platform deployment | `https://<your-deployment>.run.comfy.app` | the key of the workspace that owns the deployment | that workspace, while the deployment is up |

The top bar of the app says where it runs: "Comfy Cloud ready", "Your ComfyUI ready" or "Your Comfy
deployment ready". Every model is open weights, so the results are the same graphs wherever they run.

## What the graphs need

**ComfyUI:** a recent release. Every node the graphs use ships with ComfyUI itself, the LTX-2.5
keyframe nodes (`LTXV Add Generated Keyframes` and the rest) included, so no custom node pack is
needed. If your ComfyUI is older than the graphs, a job fails with the name of the missing node.

**Model files:** the graphs load them by the names below, so each file must sit in that folder of
`ComfyUI/models/` under exactly that name.

| File | Folder | Used by | Source (repository, file, revision) |
|---|---|---|---|
| `qwen_image_edit_2511_int8_convrot.safetensors` | `diffusion_models/` | scenes, last frames | `Comfy-Org/Qwen-Image-Edit_ComfyUI`, `split_files/diffusion_models/...`, `f68ace85e60b4a02a323e394253731947657b7d2` |
| `qwen_2.5_vl_7b_fp8_scaled.safetensors` | `text_encoders/` | scenes, last frames | `Comfy-Org/Qwen-Image_ComfyUI`, `split_files/text_encoders/...`, `1f12b17be14c89b026c51a91d67c32f84bb047bc` |
| `qwen_image_vae.safetensors` | `vae/` | scenes, last frames | `Comfy-Org/Qwen-Image_ComfyUI`, `split_files/vae/...`, `1f12b17be14c89b026c51a91d67c32f84bb047bc` |
| `ltx-2.5-22b-distilled-transformer-comfy-int8-convrot.safetensors` | `diffusion_models/` | every effect | `Lightricks/LTX-2.5`, `diffusion_models/...`, `49b093a39c19169a8a81721dcc541e07e945232f` |
| `gemma4-12b-with-proj-ltx-2.5-comfy-int8-convrot.safetensors` | `text_encoders/` | every effect | `Lightricks/LTX-2.5`, `text_encoders/...`, same revision |
| `ltx-2.5-video-vae-bf16.safetensors` | `vae/` | every effect | `Lightricks/LTX-2.5`, `vae/...`, same revision |
| `ltx-2.5-audio-vae-bf16.safetensors` | `vae/` | every effect | `Lightricks/LTX-2.5`, `vae/...`, same revision |
| `ltx-2.5-latent-spatial-upscaler-x2-bf16-1.0.safetensors` | `latent_upscale_models/` | every effect | `Lightricks/LTX-2.5`, `latent_upscale_models/...`, same revision |
| `ltx-2.5-latent-temporal-upscaler-x2-bf16-1.0.safetensors` | `latent_upscale_models/` | every effect | `Lightricks/LTX-2.5`, `latent_upscale_models/...`, same revision |
| `ltx-2.5-22b-ic-lora-refine-details-1.0.safetensors` | `loras/` | every effect | `Lightricks/LTX-2.5-22b-IC-LoRA-Refine-Details`, at the root, `4912c478b35dd96b6cdcc5f7c7ff9d72c2c57929` |
| `ltx-2.5-22b-distilled-transformer-bf16.safetensors` | `diffusion_models/` | only `VFX4ALL_EFFECT_PIPELINE=classic` | `Lightricks/LTX-2.5`, `diffusion_models/...`, same revision |

Five presets add a style LoRA. Their files have other names on Hugging Face, so they are renamed when
you download them:

| File the graph loads | Preset | Source (repository, file, revision) |
|---|---|---|
| `LTX_2.3_Cinematic_Sci-fi-Cyberpunk_Style.safetensors` | Hologram | `vrgamedevgirl84/LTX_2.3_Cinematic_Sci-fi-Cyberpunk_Style_LoRa`, `Cinematic_sci-fi-cyberpunk.safetensors`, `2d2e1796710c62882cba8cea4e51f5100e352c24` |
| `LTX_2.3_Clay_Mation_Style.safetensors` | Claymation | `vrgamedevgirl84/LTX_2.3_Clay_Mation_Style_LoRa`, `Claymation.safetensors`, `bbf7937db81244e7855adadc459ec7f1813b34ac` |
| `LTX2.3_Cozy_Felt_Style.safetensors` | Stop Motion | `vrgamedevgirl84/LTX2.3_Cozy_Felt_Style_LoRa`, `CozyFelt.safetensors`, `3a0b5ec60a63f039c24fc18daa388031a1ee30f4` |
| `LTX_2.3_Fantasy_Anime_Style.safetensors` | Anime Shift | `vrgamedevgirl84/LTX_2.3_Fantasy_Anime_Style_LoRa`, `Fantasy_Anime.safetensors`, `b4853b426fb37c7f4f0c28bf477c98a25aafeba3` |
| `LTX_2.3_Fantasy_Painterly_Style.safetensors` | Paint World | `vrgamedevgirl84/LTX_2.3_Fantasy_Painterly_Style_LoRa`, `Fantasy_Painterly.safetensors`, `a921cc65c53e3b46507165968801a1994f4f4803` |

The two LTX repositories are gated: accept their terms on Hugging Face and run `hf auth login` once.
Read the licenses in [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md) before you download, the
style LoRAs' in particular.

### Download them

Run this from the ComfyUI folder (the one that holds `models/`). The Comfy-Org repositories keep
their files under `split_files/`, so those land in a scratch folder first and move into place.

```bash
python -m pip install -U huggingface_hub
hf auth login

# LTX-2.5: the repository's folders match ComfyUI's
LTX=49b093a39c19169a8a81721dcc541e07e945232f
for f in diffusion_models/ltx-2.5-22b-distilled-transformer-comfy-int8-convrot.safetensors \
text_encoders/gemma4-12b-with-proj-ltx-2.5-comfy-int8-convrot.safetensors \
vae/ltx-2.5-video-vae-bf16.safetensors vae/ltx-2.5-audio-vae-bf16.safetensors \
latent_upscale_models/ltx-2.5-latent-spatial-upscaler-x2-bf16-1.0.safetensors \
latent_upscale_models/ltx-2.5-latent-temporal-upscaler-x2-bf16-1.0.safetensors; do
hf download Lightricks/LTX-2.5 "$f" --revision $LTX --local-dir models
done
hf download Lightricks/LTX-2.5-22b-IC-LoRA-Refine-Details ltx-2.5-22b-ic-lora-refine-details-1.0.safetensors \
--revision 4912c478b35dd96b6cdcc5f7c7ff9d72c2c57929 --local-dir models/loras

# Qwen Image Edit 2511 and its text encoder and VAE
hf download Comfy-Org/Qwen-Image-Edit_ComfyUI split_files/diffusion_models/qwen_image_edit_2511_int8_convrot.safetensors \
--revision f68ace85e60b4a02a323e394253731947657b7d2 --local-dir hf_tmp
hf download Comfy-Org/Qwen-Image_ComfyUI split_files/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors \
split_files/vae/qwen_image_vae.safetensors --revision 1f12b17be14c89b026c51a91d67c32f84bb047bc --local-dir hf_tmp
mv hf_tmp/split_files/diffusion_models/* models/diffusion_models/
mv hf_tmp/split_files/text_encoders/* models/text_encoders/
mv hf_tmp/split_files/vae/* models/vae/

# The five style LoRAs, under the names the graphs load
hf download vrgamedevgirl84/LTX_2.3_Cinematic_Sci-fi-Cyberpunk_Style_LoRa Cinematic_sci-fi-cyberpunk.safetensors --revision 2d2e1796710c62882cba8cea4e51f5100e352c24 --local-dir hf_tmp
hf download vrgamedevgirl84/LTX_2.3_Clay_Mation_Style_LoRa Claymation.safetensors --revision bbf7937db81244e7855adadc459ec7f1813b34ac --local-dir hf_tmp
hf download vrgamedevgirl84/LTX2.3_Cozy_Felt_Style_LoRa CozyFelt.safetensors --revision 3a0b5ec60a63f039c24fc18daa388031a1ee30f4 --local-dir hf_tmp
hf download vrgamedevgirl84/LTX_2.3_Fantasy_Anime_Style_LoRa Fantasy_Anime.safetensors --revision b4853b426fb37c7f4f0c28bf477c98a25aafeba3 --local-dir hf_tmp
hf download vrgamedevgirl84/LTX_2.3_Fantasy_Painterly_Style_LoRa Fantasy_Painterly.safetensors --revision a921cc65c53e3b46507165968801a1994f4f4803 --local-dir hf_tmp
mv hf_tmp/Cinematic_sci-fi-cyberpunk.safetensors models/loras/LTX_2.3_Cinematic_Sci-fi-Cyberpunk_Style.safetensors
mv hf_tmp/Claymation.safetensors models/loras/LTX_2.3_Clay_Mation_Style.safetensors
mv hf_tmp/CozyFelt.safetensors models/loras/LTX2.3_Cozy_Felt_Style.safetensors
mv hf_tmp/Fantasy_Anime.safetensors models/loras/LTX_2.3_Fantasy_Anime_Style.safetensors
mv hf_tmp/Fantasy_Painterly.safetensors models/loras/LTX_2.3_Fantasy_Painterly_Style.safetensors
```

LTX-2.5 is a 22B model. We render on Comfy Cloud and have not measured how much GPU memory it needs on
one consumer card; start with Draft.

## Your own ComfyUI, through comfy-api-proxy

The proxy puts the Comfy API v2 in front of a ComfyUI you run, so the app needs no change.

1. Start ComfyUI as usual, on `127.0.0.1:8188`.
2. Start the proxy. `--max-upload-mb 500` matches the app's own limit for clips:

```bash
pip install comfy-api-proxy
comfy-api-proxy --comfyui http://127.0.0.1:8188 --port 8189 --max-upload-mb 500
```

3. In `.env`, set `COMFY_BASE_URL=http://127.0.0.1:8189` and leave `COMFY_API_KEY` empty. The proxy
asks for no key while it stays on `127.0.0.1` without `--token`.
4. Start the app with `python run.py`. The top bar says "Your ComfyUI ready".

The first job loads the models into memory and takes longer than the ones after it. We have run the
app end to end on Comfy Cloud; this path follows the same API contract through the proxy, and we have
not yet timed it on a local GPU.

## A Comfy Developer Platform deployment

A deployment is a ComfyUI of your own on a dedicated cloud GPU. It also runs models and LoRAs that are
not in the Comfy Cloud catalog. Prepare a ComfyUI folder with the files above, then package it with the
Comfy CLI and publish a release:

```bash
pip install comfy-cli
comfy setup --where cloud
cd /path/to/ComfyUI
comfy build init --name vfx4all --models-dir ./models --custom-nodes-dir ./custom_nodes
comfy build status
comfy build push --release --target linux/nvidia
comfy deploy refs compute
```

Create a deployment from that release on the Developer Platform, then set `COMFY_BASE_URL` to its URL
and `COMFY_API_KEY` to the key of the workspace that owns it. A deployment bills for as long as it is
up, so stop or delete it when you finish.

## When a job says something is missing

The app reads ComfyUI's own validation error and names what is missing: "This Comfy server has no
X node" (update ComfyUI) or "This Comfy server does not have the model file X" (download it as above,
into the folder the tables give, then restart ComfyUI).
2 changes: 1 addition & 1 deletion presets/walk-of-fire.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,5 @@
"scene_prompt": "Create a new cinematic film still in {aspect_words} format. {references} The {subject} is the only person in the new still, with exactly {their} face, eyes, nose, jaw, hairline, hairstyle, hair colour{facial_hair}, skin tone and build{identity_notes}. The clothing, the place and the light come only from this description. The {subject} walks straight toward the camera down the middle of an empty industrial street at night, mid-stride, wearing {wardrobe}, hands loose at the sides. Red-brick warehouses line both sides, sodium streetlights cast pools of orange light, steam rises from a grate in the distance, and the wet asphalt holds long glossy reflections. Low-angle wide shot from knee height, symmetrical one-point perspective with the street vanishing behind {them}, the full body filling the middle third of the frame height. Cinematic night photography, anamorphic look, deep shadows with warm sodium highlights and a cool blue moonlight rim on the shoulders. Correct human anatomy, natural skin texture, sharp focus on the {subject}, plain unmarked brick walls throughout.",
"effect_prompt": "Use the provided start image as the first frame. The {subject} is ALREADY walking toward the camera in a calm, heavy, unhurried \"funeral march\" stride, weight landing on each heel, shoulders level, arms swinging loose, and keeps walking at exactly the same pace for the whole shot. On the second step, fire erupts from {their} shoulders and spreads over the whole body in a single roaring burst, and the {subject} walks on fully engulfed in tall orange flames, the fire streaming backward like a cape, while {their} face stays clear, calm and unharmed inside the blaze. Every footstep leaves a burning footprint on the wet asphalt, and the flames throw dancing orange light across the brick walls and long fire reflections across the puddles. {Their} eyes stay locked on a point just past the camera, one slow deliberate blink when the flames reach the collar, and the stride never breaks. The camera holds a low angle and slowly dollies backward at walking pace, keeping the {subject} centered and the same size in frame. Sound: a deep whoomph as the fire ignites, then a steady roaring blaze, heavy footsteps on wet asphalt and a low distant city hum. Photorealistic, cinematic night visual effects, fire with real weight and turbulence, sharp detail, stable picture.",
"generic_prompt": "Use the provided start image as the first frame. The {subject} in the frame carries on with exactly the same action and pace. Within the first second, fire erupts from {their} shoulders and spreads over {their} whole body in a single roaring burst, tall orange flames streaming upward and backward with real weight and turbulence, while {their} face stays clear, calm and unharmed inside the blaze. Embers drift upward, dark smoke rises, and the firelight throws flickering orange light across everything around {them}. The camera keeps the same framing and the same movement it already has. Sound: a deep whoomph as the fire ignites, then a steady roaring blaze and crackling embers. Photorealistic, cinematic visual effects, sharp detail, stable picture.",
"comfy_app": ""
"comfy_app": "https://cloud.comfy.org/?share=21dcb7ff424d"
}
Loading
Loading