mm-tools is a pruned, local-first multimedia workstation monorepo. It contains the runtime source, per-project setup entrypoints, and browser/CLI frontends used by this tool suite. It does not contain checkpoints, virtual environments, caches, generated media, or training/evaluation material.
Root-authored mm-tools code is distributed under the root AGPLv3 LICENSE.
Imported upstream attributions and revisions are recorded in the commit ledger.
The all-in-one frontend for local multimedia inference. Cutting out the cloud.
Runtime inference is local. Network access is needed only while installing or downloading the model artifacts required.
This repo was created for protecting open-source runtimes under a documentable heirarchy with AGPL.
| Directory | Runtimes |
|---|---|
vocals/tts, vocals/stt, vocals/singing |
Speech synthesis, transcription, singing |
music |
Composition, generation, score transcription |
motion-sim |
Character animation and motion reconstruction |
sculpting |
Mesh generation and editing |
world-gen |
World generation |
visual-modeling |
Image understanding and decomposition |
translate, ideogram, img2svg, video-compact, models, workflows, and
tests remain at the root. GIF/AVIF conversion lives in video-compact/gif-avif.
Install the host tools required by the projects you plan to run:
uvfor Python environments and packagesffmpegandffprobefor audio/video projectsbun- sandwich optional- Rust/Cargo for
img2svgand the optional CrisperWhisper launcher - a GPU or CPU, and likely a monitor
Then download the checkpoints once (the dependencies):
Due to the size of the model weights. Make sure you're sitting on a speedy drive.
cd ~ && git clone https://github.com/CommanderTurtle/mm-tools multimedia && cd multimedia
cd models
uv venv --python 3.12.10 --seed --managed-python
source .venv/bin/activate
uv pip install -U huggingface_hub hf_transfer gdown==5.2.0
uv run download_models.py
# Choose workers first, then select project bundles or press Enter for all.
# Non-interactive full install: uv run download_models.py all --yes --workers 24
deactivate # when doneThe downloader is resumable. Run it again after an interrupted transfer. New
source-import bundles are named 4d, lingbot, lingbot-5090, fire3d,
aukspeech, sculpting, worldsculpt, nvidia-sim, v2v, yue2, and
motionbricks, flowhmr, optional flowhmr-physics, and physag3d; use
uv run download_models.py --list for the complete current menu.
The source imports, single-GPU adapters, setup/start contracts, and private
browser studios are included here; model artifacts remain external and
resumable. LingBot deliberately separates its shared encoder/VAE bundle from
the official 1.3B single-5090 DiT and does not retain the unusable 14B DiT.
Model downloads remain governed by each model's native license and any access terms accepted for their usage. ARDY uses an ungated, pre-merged INT8 LLM2Vec encoder that preserves the exact Llama 3 + MNTP + supervised-adapter pipeline; the original gated BF16 assembly is not downloaded. 4DAnyone's neutral SMPL-X body model remains a separate manual download. See the model inventory for every included artifact, output path, and explicit exclusion. The ability to self-host a tool rivaling Adobe in 2026 was the purpose for this repo.
Assuming one has a nice GPU, you can host a wide variety of tools. Emulating the top-tier closed source software out there, solely with open-source runtimes. It wouldn't be without the community that building something like this would be possible.
~16-20gb vram recommended for most tools. The largest quality-first music paths benefit from 24-32 GB. Each project has its own upper bound, and none is authoritative if you run them one at a time. (only ~8-9gb for LongCat)
So. What can you do?
2-longcat.mp4
compatible with vox
- an in-house built translation pipeline with native web-ui. Compatibile with vox system-audio driver. (This exact fork)
compatible with vox
whisper.mp4
Motion authoring: MotionBricks studio provides SOMA and Unitree G1 inbetweening, Smart Primitive authoring, scene editing and interactive character preview through the pinned native runtime.
FlowHMR studio reconstructs SMPL-H motion from reference video, with person selection, reusable intermediates and an optional side-by-side PHC humanoid rollout.
PhysAg3D studio builds articulated objects from images through separately saved TRELLIS generation and OMP/Blender/MuJoCo stages, with part controls, motion playback and portable asset bundles.
Idol studio turns images into Gaussian splats, with the original Space presets and sampling controls, an orbit viewer and PLY export.
SceneBloom studio generates furnished rooms with local vision, saved stages, spatial inspection and textured GLB export.
Hy-Nomas studio generates continuous worlds with text, image starts and recorded camera motion, rollback, saved state and MP4 exports.
2-video-to-gif-avif.mp4
because who says ffmpeg is fine in cli?
view what a combined workflow looks like:
2-ideogram+img2svg-compact.1.mp4
2-redesign.mp4
2-muscriptor.mp4
(uses minimized, stripped comfyUI pipeline directly, but custom WebUI)
minimax.mp4
Run all at once.
The install sequence is simple for each project. Requiring WSL if on Windows.
Jump into a project folder, run its setup once, and start it. Setup creates the ignored .env and isolated .venv automatically.
cd ~/multimedia/PROJECT
./setupwithuv.sh
./startwithuv.shVox - Native Audio/Translate Routing for this repo.
See other projects by me!
https://github.com/CommanderTurtle/persephone just my integration of own internal config + continuity with omp. gpu & local-only for just about everything so far
-
Diogenes - hosting (linux, service manager), maintained fork of Odysseus for backends and control center — this can install and integrate the following if not already installed:
-
sandwich - node package management/auditing for all these package.jsons
-
libriarian - wikifier (makes wikis, dreaming agents)
-
retrieval - skill labrador that fetches necessary skills from archive on disk, rather than bloating agent prompt with full list
-
persephone - adds omp (pi agent) gateway layer
-
leetcoder - allows hermes to puppeteer pi’s for small coding tasks (delegation)
Lots of ideas from stablyai/orca, which does all these things, and where i got a lot of inspiration for the ecosystem
Still WIP. Doing lots of editing as I continue development. Especially with Diogenes automatically setting up my config upon install
- orc - a multisite/multiframework F# engine for modular websites in py, ts, c#, js, and ruby.
- reactor - a lightweight rust-poller that allows for a live fsharp repo in 7+ languages. Side-hosts vite dev server, zensical serve, jekyll serve, netdocs serving (C#), and more with preview
- tools - just some lightweight tools that help making maintaining websites easy in modular languages
- regedited - for databases
- macrohelp - for macros
- firebending - an Anything-MCP built on macrohelp (TBD)
- app/adspace - ad simulator
- app/countku - count in haiku
- app/webclip - scrape the web to markdown
extra
For Python projects, choose the setup entrypoint in that project only. No root setup wrapper exists.
Local Ideogram 4 generation source plus the fuzzy-mask editing browser: ObjectClear, BiRefNet, OpenCV, and optional private-Comfy Ideogram masked editing.
cd ~/multimedia/ideogram
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8174The standalone Ideogram generator is local_generate.py; add src to PYTHONPATH or install the local package when using that lane. The editor loads Ideogram only when explicitly selected. Its private engine reuses bundled Comfy code (not the Minimax service), supports local caption drafting and configurable crop processing, and preserves the original image outside the feathered mask. See Ideogram architecture and setup.
Native VTracer-based raster-to-SVG CLI and browser studio.
cd ~/multimedia/img2svg
./setupwithrust
./startwithrust # http://127.0.0.1:417Local LongCat voice synthesis and cloning. The HTTP service and browser workbench own independent model lifecycles.
cd ~/multimedia/vocals/tts/longcat
./setupwithuv.sh
./starthttp.sh # machine API: :8230
./startwithuv.sh # browser UI: :8231Native ACE-Step 1.5 music generation, editing, repainting, extension, and audio-to-audio controls.
cd ~/multimedia/music/acestep
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8250The browser process preloads the native XL-SFT DiT and 4B 5 Hz language model: the quality-first ACE combination for a 24 GB+ workstation. It reads the shared native ACE-Step model tree directly and releases all weights on Ctrl+C.
Standalone MiniMax Music 3 text-to-song production desk using the official graph: full FP16 DiT, pruned INT8 text encoder, DAV decoder, real CFG/top-k/sampler controls, and an animated lyrics-and-spectrum performance view. It does not require a separately running ComfyUI.
cd ~/multimedia/music/minimax
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8254The browser owns a private loopback song engine. Its separate Prompt Guide tab uses the bundled Comfy CLIPLoader(type=krea2) → TextGenerate path with the local SergiusFlavius/Qwen3-VL-4B-Instruct-heretic-NVFP4 checkpoint, but begins fully off and unloaded; enabling it starts a second loopback-only engine. A Guided Brief Lab adds one-click recipes plus expandable genre, instrumentation, key/mode, BPM, meter, groove, harmony, vocal, form, production, and listening-context vocabulary. It creates plain-English copypasta without invoking a model or silently changing controls. Guide results remain detached, with manual copy controls for MiniMax's Global Metadata, Vocal Details, Arrangement, and tuning notes. Load models and Unload control song-model residency explicitly, while Ctrl+C unloads all weights and closes the web app plus both private engines. Links in the studio hand covers/editing to ACE-Step, loops to Foundation-1, and audio reverse engineering to MuScriptor.
Foundation-1 text-to-audio with the enhanced Stable Audio Gradio controls, local T5 conditioning, MIDI analysis, and optional TorchAO controls.
cd ~/multimedia/music/stableaudio
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8251Two-stage symbolic composition: generate harmony MIDI, then orchestrate a MIDI sketch with baseline or GRPO checkpoints.
cd ~/multimedia/music/symphony
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8252The web process is model-free while idle. Each generation request owns and releases its model in a short-lived worker.
Prompt-conditioned local singing synthesis with the highest-quality Pro weights only, explicit load/unload controls, browser playback, and WAV downloads.
cd ~/multimedia/vocals/singing/vocalrender
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8253Audio-to-MIDI/score transcription with a browser piano roll.
cd ~/multimedia/music/muscriptor
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8222Single-file CLI:
./startwithuv.sh INPUT.mp3Sheet-music image/PDF encoding and SMT-backed beKern/MIDI/SVG conversion.
cd ~/multimedia/music/musvit
./setupwithuv.sh
./startwithuv.sh # browser studio: :8223
./startwithuv.sh SCORE_IMAGE_OR_PDF # direct CLIUse musvit_embed.py directly for the embedding lane.
Unlimited OCR 2 studio parses ordered PDF/image
pages into a structured document with layout overlays, tables, equations and
figure crops. Select unlimited-ocr2 in the downloader; serve on :8287.
Pixel Wizard uses WeVisOne-VL-4B for
grounding, detection, counting, pointing, OCR, captions, segmentation and alpha
matting. Select pixel-wizard in the downloader; serve on :8288.
Flat-image decomposition into editable layers using local detection, segmentation, inpainting, OCR, and Qwen Image Layered components.
cd ~/multimedia/visual-modeling/redesign
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8173Review .env before launch. Native Diffusers is the primary image lane; an existing private ComfyUI listener is optional.
Local EraX text translation, language classification, and a shared INT4 multimodal lane for image explanation, OCR translation, custom visual prompts, and spoken-language arbitration.
cd ~/multimedia/translate
./setupwithuv.sh
./starthttp.sh # http://127.0.0.1:8176
./startwithuv.sh # standalone browser UI: :8177The setup prompt accepts GPU or CPU; TRANSLATE_ACCELERATOR remains available for noninteractive runs. The browser UI owns an independent model lifecycle. Its explicit external-load button can attach to an already-loaded starthttp.sh instance, but it never falls back between them automatically.
Private VideoSmaller-style FFmpeg compression.
cd ~/multimedia/video-compact
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8240Local trim/crop/resize and animated GIF or AVIF conversion.
cd ~/multimedia/video-compact/gif-avif
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8241The host FFmpeg build must expose the requested GIF/AVIF encoder and AVIF muxer.
Local Nemotron 3.5 transcription, speaker timelines, karaoke, subtitles and audio questions.
uv run models/download_models.py nemotron
cd vocals/stt/nemotron
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8289Native NVFP4/FP8 on Blackwell; microphone, media files and explicit URL imports, word seeking, CPU vocal separation, saved conversations and timed exports.
Local Realistic Air speech, reference voices and paragraph takes.
uv run models/download_models.py audio8
cd vocals/tts/audio8
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8290Native BF16 and 44.1 kHz codec, microphone/file references, waveform seeking, saved sessions, WAV/FLAC and codec-token exports.
Local voice design, cloning, multilingual delivery and multi-speaker podcasts.
uv run models/download_models.py ming3-0.5b # dense, ~3 GB
uv run models/download_models.py ming3-tiny # optional MoE + podcast, ~18 GB
cd vocals/tts/ming3
./setupwithuv.sh
./startwithuv.sh # http://127.0.0.1:8291Separate native BF16 environments, reference recording, director controls, vocal tags, saved takes, waveform seeking and WAV/FLAC exports.
Local CrisperWhisper 2.0 transcription in normalized or literal mode. The HTTP service and browser workbench use separate ports and model lifecycles.
cd ~/multimedia/vocals/stt/whisper
./setupwithuv.sh
./starthttp.sh # machine API: :8172
./startwithuv.sh # browser UI: :8173ReDesign also defaults to 8173; change CW2_UI_PORT when both browser services run together.
English transcription with independent Bark 3.0, Plus, Clean and Pro loaders. Whisper-style studio, microphone recording, transcript comparison and TXT/JSON export.
uv run models/download_models.py underdog-pro # also: underdog-base, underdog-plus, underdog-clean
cd vocals/stt/underdog
./setupwithuv.sh
./startwithuv.sh # studio: :8293
./starthttp.sh # optional API: :8292, BARK_MODEL in .env| Port | Runtime |
|---|---|
417 |
img2svg |
8172 |
CrisperWhisper HTTP |
8173 |
CrisperWhisper UI or ReDesign |
8174 |
Ideogram/ObjectClear editor |
8176 |
Translate |
8177 |
Translate standalone UI |
8221 |
MotionBricks character studio |
8222 |
MuScriptor |
8223 |
MuSViT local score studio |
8230 |
LongCat HTTP |
8231 |
LongCat UI |
8240 |
Video Compact |
8241 |
Video to GIF/AVIF |
8250 |
ACE-Step 1.5 |
8251 |
Stable Audio Foundation-1 |
8252 |
SymphonyGen local studio |
8253 |
VocalRender local studio |
8254 |
MiniMax Music 3 local studio |
8281 |
FlowHMR reference-to-motion studio |
8282 |
PhysAg3D staged asset studio |
8283 |
Idol Gaussian studio |
8284 |
Hy3D mesh studio |
8285 |
SceneBloom scene studio |
8286 |
Hy-Nomas world studio |
8287 |
Unlimited OCR 2 document studio |
8288 |
Pixel Wizard vision studio |
8289 |
Nemotron transcription studio |
8290 |
Audio8 speech studio |
8291 |
Ming3 speech and podcast studio |
8292 |
Underdog Bark HTTP |
8293 |
Underdog Bark studio |
The new studios default to loopback; existing services retain their configured private-LAN interfaces. Keep network-exposed services behind the host firewall or set the supported bearer token where provided.
The current expansion uses one shared, dependency-light product surface with project-native adapters. Every studio has responsive dark/light themes, searchable task cards, complete native controls, presets and drafts, raw streaming uploads, a persistent single-GPU queue, cancellation, SSE progress, model load/unload health, an output library, side-by-side comparison, and API examples. There are no hosted assets, analytics, CORS opt-ins, or automatic model downloads at runtime.
| Project | Canonical surface | Default port | VRAM (approx) |
|---|---|---|---|
| AuK | 16 speech generation, cloning, editing, emotion, timing, and nonverbal workflows; Vox-compatible speech route | 8260 | ~10 GB |
| 4DAnyone | monocular 4D human reconstruction and reusable result inspection | 8261 | ~14 GB |
| TRELLIS.2 + Pixal3D | image/multiview sculpting, local refinement, and interactive GLB inspection | 8262 | ~20 GB |
| WorldSculpt | complete multi-object scene reconstruction and scene inspection | 8263 | ~18 GB |
| ID-V2V | identity-preserving transfer, normal/depth guidance, and reusable preprocessing | 8265 | ~22 GB |
| ARDY + SOMA-X | text-to-motion, motion inspection, parametric bodies, and hands | 8266 | ~8 GB |
| LingBot World V2 | 1.3B single-5090 world generation and visual camera-path authoring | 8267 | 32 GB |
| Fire3D | RGB/RGBD scene reconstruction, native-release evaluation, and GLB inspection | 8268 | ~20 GB |
| YuE2 | composition, planning, continuation/cover, score realization/editing, transcription, resume, and VAE decode | 8270 | ~24 GB |
This expansion folds thirteen additional runtimes into the same shared, dependency-light product surface with project-native adapters: motion repair and video-to-character animation, geometry, scene, and world generation, optical character recognition and natural-language image editing, plus speech synthesis, transcription, and podcast generation. Every studio keeps responsive dark/light themes, searchable task cards, complete native controls, presets and drafts, raw streaming uploads, the persistent single-GPU queue, cancellation, SSE progress, model load/unload health, an output library, side-by-side comparison, and API examples. Pinned upstream source snapshots now ride in-tree, so a fresh checkout restores any runtime with dependency installation alone; only checkpoint bytes route through the shared downloader.
| Project | Canonical surface | Default port | VRAM (approx) |
|---|---|---|---|
| Audio8 | speech synthesis | 8179 | ~8 GB |
| MotionBricks | motion repair, inpainting, outpainting, and trajectory steering | 8272 | ~10 GB |
| FlowHMR | video and reference to animated character meshes | 8273 | ~12 GB |
| PhysAg3D | physics-grounded 3D human scenes | 8275 | ~16 GB |
| Idol | video to 3D character generation and inspection | 8276 | ~22 GB |
| Hy3D | image and video to high-fidelity 3D geometry | 8277 | ~22 GB |
| SceneBloom | procedural 3D scene generation with Blender-MCP refinement | 8278 | ~12 GB |
| Hy-Nomas | interactive generated worlds with first-person play | 8279 | ~28 GB |
| Unlimited OCR 2 | document and scene text extraction | 8280 | ~12 GB |
| Pixel Wizard | natural-language image editing | 8281 | ~24 GB |
| Nemotron | long-form transcription | 8282 | ~6 GB |
| Ming3 | speech synthesis and podcast generation | 8291 | ~16 GB |
| Underdog | Bark speech behind plain HTTP and studio surfaces | 8292/8293 | ~6 GB |
sculpting-1-compact.mp4
The only Yue2 frontend with stem splitting!
- (Share with Friends via app.shel.sh/make & a.shel.sh superlinks, entirely local)
Another:
https://a.shel.sh/#media:rZ9jBas~g*mK]Q-()SB$IFDo]Xl-b5y[P,/@f/FwenhdLmh~Aaw





