Skip to content

Add Vast.ai Mem0 Qwen3.5 launcher - #3

Draft
jinuk0211 wants to merge 2 commits into
OpenDataBox:mainfrom
jinuk0211:codex/vastai-mem0-qwen35
Draft

Add Vast.ai Mem0 Qwen3.5 launcher#3
jinuk0211 wants to merge 2 commits into
OpenDataBox:mainfrom
jinuk0211:codex/vastai-mem0-qwen35

Conversation

@jinuk0211

@jinuk0211 jinuk0211 commented Aug 19, 2026

Copy link
Copy Markdown

What changed

  • add a local OpenAI-compatible Mem0 preset for Qwen/Qwen3.5-4B
  • add a minimal benchmark dependency set that stays compatible with a recent vLLM runtime
  • add a Vast.ai launcher that creates the environment, starts chat and embedding servers, waits for health checks, generates the runtime YAML, and runs the benchmark
  • make LongMemEval the default benchmark and support overriding its sample cap at runtime
  • document smoke/full runs, GPU placement, and supported environment overrides
  • enforce LF line endings for shell scripts

Why

The existing Mem0 preset assumes already-running local endpoints and does not provide a reproducible Vast.ai bootstrap path. Qwen3.5 also requires a newer serving stack than the repository-wide pinned Transformers dependency, so the launcher keeps vLLM setup separate from the broad all-method requirements.

User impact

Users can clone this branch on a CUDA-enabled Vast.ai instance and start a one-query LongMemEval smoke run with a single script invocation. MAX_QUERIES=0 runs every query in the configured five-sample preset, while MAX_TEST_SAMPLES can expand that sample limit.

Validation

  • staged Bash script passes bash -n
  • script is committed with executable mode and LF line endings
  • agent and LongMemEval YAML parse and required fields were asserted
  • runtime agent/dataset YAML generation was exercised, including the sample-limit override
  • pip install --dry-run --ignore-installed -r requirements-mem0-vast.txt resolves successfully
  • git diff --cached --check passes

GPU model loading was not executed on the Windows development host because no local CUDA/WSL runtime is available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant