[tawei-amd]

hy-memory on HRX: cap the prefill chunk at 1024

Problem. With the HRX runtime, a Hy-Memory prompt of about 2k tokens or more hangs. The engine's prefill overlay dispatches time out (amdxdna dispatch timed out: ert state 8, DEADLINE_EXCEEDED), the engine carries on with the failed result, and the request either never returns or returns garbage. Found while updating the IRON baseline: the HRX GSM8K bench stalled once (3 timeouts in a row), and the HRX perf bench recorded a nonsense decode speed (1100+ tok/s, TTFT 20 s at 1k) because the dispatches failed instantly.

Measured (HRX portable build b908208a, Strix Halo, local weights):

prompt	result
1,399 tokens	fine, prefill 112 tok/s, decode 17.6 tok/s
2,317-2,500 tokens	4-7 timeouts, request hangs (240-500 s client timeout)
2,317 tokens with prefill + attention on the host (HY_MEM_MM_HOST=1 HY_MEM_ATTN_HOST=1)	fine
2,317 tokens with either one alone on the host	still hangs
2,317 tokens with --prefill-chunk-len 1024	fine, no timeouts, prefill 97 tok/s, decode 17.2 tok/s
flm bench hy-memory:30b-a3b -c 4096	dispatch timeouts, bench never finishes

Fix. In HyMemory::insert, on FLM_USE_HRX builds only, clamp meta_info.max_prefill_len to 1024 (the value tested above). XRT keeps the catalog's 16384 (the engine splits at 4096): it is unaffected by this problem and is faster with big chunks (the MoE expert matmuls pad each expert to 256 rows). Verified on the XRT build: smoke and determinism pass, prefill 180 / 242 tok/s at 1k / 2k tokens, unchanged.

Not done / follow-up. The underlying cause is in the engine or the HRX runtime (a long-chunk dispatch of the prefill overlay). This only avoids it. The HRX baseline for hy-memory should be (re)generated after this lands; the XRT baseline is already merged (IRON#171).

🤖 Generated with Claude Code