Skip to content

test(vela): keep root head checks and document the fixed L20 replay profile - #205

Draft
psv666 wants to merge 2 commits into
codex/l20-b-vela-executor-basefrom
codex/l20-b-vela-replay-supplement
Draft

psv666 wants to merge 2 commits into
codex/l20-b-vela-executor-basefrom
codex/l20-b-vela-replay-supplement

Conversation

@psv666

@psv666 psv666 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Required human contributor answers

The contributor confirmed the stage scope and these answers on 2026-11-19; the detailed confirmation is retained in the contributor's task record.

  1. Outcome: Bring assigned model work to reproducible, reviewable acceptance so supported models can be used reliably.
  2. Highest-leverage variable: Reliable model coverage and numerical consistency.
  3. How this moves it: Reuse existing implementations, resolve contract and evidence gaps, and preserve meaningful regression checks and reproducible setup.
  4. User and need: Model, application and agent developers who need stable interfaces, reliable outputs and reproducible deployment.

Is a PR needed? Yes. The contributor authorized necessary stage draft PRs and independent supplements to existing authors' work, preserving their commits.

Bug-fix evidence: This changes test placement and documentation. At #177 head, the three head-loader test bodies live under src/, contrary to CONTRIBUTING.md, and the parity-example command omits VELA_NATIVE_MANIFEST_SHA256, which its loader requires. No runtime numerical defect is claimed.

Summary

Keep Vela's head-loader regressions under root tests and document the independently reproduced L20 native-library profile with anchored release replay commands.

root head-loader checks
  + external manifest SHA in the parity example
  + fixed L20 CPU/head/response/GPU replay evidence

This is a stacked supplement to #175–#177. Its B-owned base is fixed to MrDongsls's #177 head 55ed516d302ad5bf1f5824f5ba53208236080a3b. Current head 5679f595e9ecd324326907ba913aa8f209a36a69: three files, +62/-43. Original author history is preserved. Keep draft; integration into main requires the existing stack to land and a fresh retarget/check.

Evidence

  • Before: Three private head-loader tests were implemented under src, and the parity-example command omitted its required manifest anchor. This real L20/toolkit profile lacked an independent replay.
  • After: All three checks are discovered from root tests and pass; examples supply the anchor and optimized test profile; all original 0.8B real-package checks passed on L20 with their unchanged assertions and gates.

Personally executed:

Check Result
Original #177 ordinary crate tests 63 passed, 0 failed, 5 opt-in tests ignored
Candidate discovery and ordinary crate tests All three moved tests present; same63/0/5
fmt, crate clippy, full workspace clippy/tests/release build PASS
Strict MkDocs, including latest recipe update PASS
Original token/structured-state package replay PASS
Original FP32 head replay, 5 cases PASS; max logit3.719e-5, answer7.252e-7, gates4e-5/1e-6
Original CPU response replay, 4 cases PASS; max1.490e-6, gate1.5e-6
Original GPU executor replay, 4 cases PASS; max0.3297, gate0.35

GPU stdout:

example-request: worst logit drift 0.1487
example-request-set-threshold: worst logit drift 0.0152
example-request-cjk: worst logit drift 0.3297
example-request-score: worst logit drift 0.0158
test the_gpu_backbone_replays_the_reference_readouts ... ok
test result: ok. 1 passed; 0 failed

Purpose

Resolve the root-test placement and runnable-example gaps while adding current target-hardware evidence. Related: #137, #169, #175, #176, #177.

Test Plan

  • Model/runtime/tokenizer: vllm-sr/Vela-2.0-0.8B@326b01d81f61fff7fd4ae570e54b87a49119e490. Full source shard LFS SHA256 c70237ecd579c60289c177fa9806e71c24840c2fc557b5c76af1395dc3081fa6. Original committed fixtures unchanged.
  • Export: original mixed tensors retained; uniform-BF16 backbone plus FP32 heads. External manifest anchor 1d789daa559b74a65c814b212ea5789d95d418d11c2f3ec67fa41267de9ecddb. Pre/post-export package gates pass.
  • Hardware: one NVIDIA L20 sm_89,46068MiB, driver580.178.04, GPU1 UUID GPU-8634ad01-28fd-6f5b-a574-bf1832a8dbc8; nvcc12.8.61, Rust1.100.0. Independent checkout/venv/target/export and GPU1 functional lock.
  • CUDA library built from unchanged vela: add the 0.8B native response finishing and backbone execution on CUDA #177 source, SHA256 47e918cbbaebff25d62f841c98b405e8325e80e17cb3b46d979b71de3d301db5. Release executor test binary SHA256 b9ef97773f086be82555f7ec5841d1a61082877f868e8d23b223e40f14d9bf0f.
  • CPU exporter dependencies: Python3.12.13, torch2.11.0+cpu, transformers5.19.0, tokenizers0.23.2, numpy2.5.3, safetensors0.8.0. Rust executes CUDA independently of PyTorch.
  • Real-package replay uses cargo test --release --locked as documented. All numerical override envs unset. No source, case, structural assertion or gate was changed.
  • An initial debug head replay was interrupted while CPU-bound before any case/assertion; it is retained as incomplete. Ordinary debug checks passed separately. The release replays completed; the GPU test's137.32s includes loading/verification and is not serving latency.

Test Result

This exact L20 native-library logit profile passes the original0.35 gate. CPU head/response and processing gates also pass. The code tested is the unchanged executor/backend at55ed516 plus the root-test supplement; the subsequent commit adds recipe text only.

Native HTTP, GPU end-to-end answers, positive long-window GPU parity, 4B/9B and other hardware remain separate. The original0.8B long-span head fixture has an empty expected answer; passing it does not establish a positive long-span GPU decision. Historical RTX profile descriptions remain attributed to their sources.

Demo / evidence

The raw excerpt above is this actual run. Complete stdout, interrupted debug attempt, model metadata, environment, hashes and protocol are retained in vela08-evidence.tar.gz, SHA256 9c681b1067b38abcbf1c3fa9a63a5235eea4fff9a53edd55e48f989c7665aeb7, on both contributor-local and L20 task storage. No generated run dump or archive is added to product Git.

No latency/throughput comparison or video claim. The runtime/kernel implementation is unchanged. #205's staging-base head returned no pull-request workflow run; the personally run checks above are the evidence.

Merge Danger

Door: two-way.

Blast Radius: tests.

Private test wiring and a focused recipe update only. Retarget after the existing stack lands and rerun relevant integration checks.

Self-review

Agent precheck reviewed the complete three-file supplement, registration/discovery, preserved assertions, actual profile/gates, artifact hygiene and strict docs. Human contributor checklist remains for review; no Ready/merge request.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on the stated purpose.
  • I have run the checks appropriate to this change and reported commands, results, and anything that I could not verify above.
  • I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

@psv666 psv666 changed the title test(vela): keep head-loader checks in root tests and anchor replay example test(vela): keep root head checks and document the fixed L20 replay profile Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant