LocalMind

Run Qwen3.6 35B-A3B, a 35-billion-parameter coding model, on your own computer. No cloud, no account and no GPU required.

LocalMind is a desktop app for chatting with the model and serving it to your other tools. It runs on the siphon.cpp engine. On a PC without a graphics card, the model runs on the CPU and streams its weights from the SSD, so a 21 GB model works on a 16 GB laptop.

Install

On Ubuntu 24.04+ or Debian 13+, paste this into a terminal:

curl -fsSL https://raw.githubusercontent.com/Manohar20006/LocalMind/v1.0.0/install.sh | bash

The installer:

1. Checks the computer: CPU, RAM, free disk space and whether the drive is an SSD.
2. Installs the system packages it needs. It asks for your password once (sudo).
3. Downloads LocalMind and the siphon.cpp engine, then builds the engine for your CPU (5-15 minutes).
4. Downloads the model: 21.1 GiB from Hugging Face, checksum-verified. If the download stops, run the command again and it resumes.
5. Adds LocalMind to your applications menu and a localmind command.

Then open LocalMind from the applications menu, pick Qwen3.6 MoE 35B and press Start.

What you need

	Minimum	Notes
System	Ubuntu 24.04 / Debian 13, x86_64	The desktop app needs GTK 4 and WebKitGTK 6
CPU	AVX2 (most CPUs since 2015)	Runs without AVX2, but very slowly
RAM	16 GB	Less works, but slowly
Disk	SSD with ~25 GB free	The model is read from disk while it runs; a hard disk is too slow
GPU	Not needed	With an NVIDIA GPU, see GPU builds

Speed on CPU: about 4-6 tokens per second with 16 GB RAM and an NVMe SSD. The first answer after starting is slower while the model warms up.

Options

Set these before bash, for example curl … | LOCALMIND_DIR=/data/LocalMind bash:

Variable	Default	Purpose
LOCALMIND_DIR	~/LocalMind	Install folder (engine in engine/, model in engine/models/)
LOCALMIND_MODEL	—	Use a Qwen3.6-35B-A3B-UD-Q4_K_M.gguf you already have instead of downloading
LOCALMIND_SKIP_MODEL	0	1 installs without the model; run the installer again later to download it
LOCALMIND_JOBS	auto	Parallel build jobs

Update or uninstall

• Update: run the install command again. Your settings and downloaded model are kept.
• Uninstall: run the rm -rf … command that the installer prints at the end.

Using it

• Chat: replies render as Markdown with highlighted, copyable code blocks. The model's thinking appears in a collapsible box.
• Local API: the API tab serves the model as an OpenAI- and Anthropic-compatible endpoint with scoped keys. One click connects coding tools such as OpenCode.
• More models: "Add a model folder" imports other GGUF models.

Details are in docs/USAGE.md.

GPU builds

The installer builds the CPU engine. With an NVIDIA GPU (6 GB VRAM or more) and the CUDA toolkit, siphon.cpp's GPU engine reaches about 20-25 tokens per second by caching hot experts in VRAM. See the siphon.cpp README to build it into engine/. LocalMind picks the GPU automatically when one is usable.

Development

LocalMind/
  bin/            launchers (localmind = desktop app, siphon = command-line tool)
  siphon/         the app (Python): desktop host, engine runner, API gateway, CLI
  siphon-desktop-dark-opendesign.html   the desktop UI
  profiles/       model profiles (settings saved from the app)
  tests/          unit tests (need the engine checkout in engine/)
  engine/         siphon.cpp checkout (created by the installer; git-ignored)

Run the tests with python3 tests/test-studio-<name>.py and node tests/test-studio-settings.js.

License

MIT. See LICENSE. The siphon.cpp engine is also MIT-licensed.