Summary
_extract_with_adaptive_retry() (llm.py ~:2043 on v8) bisects a chunk on three signals: finish_reason == "length", a context-exceeded error (_looks_like_context_exceeded), or a hollow response. A per-chunk timeout is not one of them: _call_claude_cli uses subprocess.run(..., timeout=_resolve_api_timeout()), so a slow chunk raises subprocess.TimeoutExpired, the except Exception branch re-raises because it isn't a context-exceeded message, and the whole chunk fails with no output. The files are (correctly) left unstamped, but nothing makes them smaller, so on a corpus where the same file packs the same way next run they fail the same way.
For the claude-cli backend the timeout is the output-size signal in practice: a chunk whose output is heading past ~50-60k tokens takes longer than 900s to stream, so it dies on the clock before it can return stop_reason: max_tokens and be bisected.
Observed (0.9.45, claude-cli, --token-budget 25000, GRAPHIFY_API_TIMEOUT=900)
Full extract of a 1,900-file markdown corpus, 2026-08-17: 28/29 chunks succeeded, one failed with
[graphify] chunk 25/29 failed: Command '['claude', '-p', '--output-format', 'json', '--no-session-persistence', '--model', 'claude-sonnet-5', '--json-schema', ...]' timed out after 900.0 seconds
[graphify] WARNING: 1/29 semantic chunk(s) failed — see errors above. Partial results returned.
Chunks with entity-dense reference/index docs (many distinct names/paths/keys per KB) emit far more output per input token than narrative docs, which is what pushes a same-sized chunk over the clock. On the same corpus, chunks that returned finish_reason=length were bisected and recovered as designed; the timed-out one was not.
Suggested fix
Treat subprocess.TimeoutExpired (and the equivalent SDK timeout exceptions for the API backends) as a bisectable failure in _extract_with_adaptive_retry, subject to the same max_depth cap — i.e. route it through the same branch as _looks_like_context_exceeded(exc). A timeout on a single unsplittable unit should still fail as it does today. Workaround meanwhile: a smaller --token-budget and a larger GRAPHIFY_API_TIMEOUT for dense corpora.
Summary
_extract_with_adaptive_retry()(llm.py ~:2043 onv8) bisects a chunk on three signals:finish_reason == "length", a context-exceeded error (_looks_like_context_exceeded), or a hollow response. A per-chunk timeout is not one of them:_call_claude_cliusessubprocess.run(..., timeout=_resolve_api_timeout()), so a slow chunk raisessubprocess.TimeoutExpired, theexcept Exceptionbranch re-raises because it isn't a context-exceeded message, and the whole chunk fails with no output. The files are (correctly) left unstamped, but nothing makes them smaller, so on a corpus where the same file packs the same way next run they fail the same way.For the
claude-clibackend the timeout is the output-size signal in practice: a chunk whose output is heading past ~50-60k tokens takes longer than 900s to stream, so it dies on the clock before it can returnstop_reason: max_tokensand be bisected.Observed (0.9.45,
claude-cli,--token-budget 25000,GRAPHIFY_API_TIMEOUT=900)Full extract of a 1,900-file markdown corpus, 2026-08-17: 28/29 chunks succeeded, one failed with
Chunks with entity-dense reference/index docs (many distinct names/paths/keys per KB) emit far more output per input token than narrative docs, which is what pushes a same-sized chunk over the clock. On the same corpus, chunks that returned
finish_reason=lengthwere bisected and recovered as designed; the timed-out one was not.Suggested fix
Treat
subprocess.TimeoutExpired(and the equivalent SDK timeout exceptions for the API backends) as a bisectable failure in_extract_with_adaptive_retry, subject to the samemax_depthcap — i.e. route it through the same branch as_looks_like_context_exceeded(exc). A timeout on a single unsplittable unit should still fail as it does today. Workaround meanwhile: a smaller--token-budgetand a largerGRAPHIFY_API_TIMEOUTfor dense corpora.