Repository navigation
Conversation
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Score: 9/10. Summary: get_embeddings_parallel now caps in-flight batches at 10 and BuildSemanticIndex delegates to it; the OpenAI and Ollama async paths match their sync counterparts, and the new bound test fails on the base helper and passes here.
imbajin
left a comment
There was a problem hiding this comment.
Score: 7.0/10 · Confidence: high
Blocking: yes
Summary
Reusing the semantic index with a persistent async embedding client can fail on a subsequent update.
Evidence
The changed call creates a fresh event loop for each run while provider instances retain their async clients. A local HTTP/1.1 keep-alive reproduction using httpx succeeded on the first asyncio.run and raised RuntimeError: Event loop is closed on the second.
Validation notes
Controlled HTTP transport reproduction and static caller review; no live OpenAI or Ollama integration was run.
| if added_vids: | ||
| vids_to_process = self._extract_names(added_vids) if all_pk_flag else added_vids | ||
| added_embeddings = asyncio.run(self._get_embeddings_parallel(vids_to_process)) | ||
| added_embeddings = asyncio.run(get_embeddings_parallel(self.embedding, vids_to_process)) |
There was a problem hiding this comment.
When the same semantic-index operator or embedding instance handles another nonempty update, asyncio.run creates a new loop after closing the first one, but the OpenAI/Ollama async client retains its HTTP connection pool. With a reusable keep-alive connection, the second request can fail with RuntimeError: Event loop is closed, leaving the new vertices unindexed. I reproduced this lifecycle with an httpx.AsyncClient and a local HTTP/1.1 keep-alive server. Please keep the client and event loop lifetimes aligned (or retain the synchronous executor path), and cover two updates using the same instance.
There was a problem hiding this comment.
Root cause: the shared fetch ran every batch through embedding.async_get_texts_embeddings, so the instance-owned async client (and its pooled keep-alive connections) outlived the per-update loop that asyncio.run creates and closes. The same failure applied to all three consumers of the helper (BuildSemanticIndex, BuildGremlinExampleIndex, BuildVectorIndex).
Structural fix (0722aac): retained the synchronous executor path. Each batch now runs get_texts_embeddings on the loop's executor — the route BuildSemanticIndex used before this PR. The synchronous clients hold no loop-bound state, so client and loop lifetimes no longer need to match across updates. Order, batching (500) and the in-flight bound (10) are unchanged; the async fetch methods stay on the embedding API for direct use.
Validation:
test_two_updates_on_one_instance_keep_indexing: two nonempty updates on oneBuildSemanticIndexinstance through the real helper. Red before 0722aac withRuntimeError: Event loop is closedon the second update; both updates now index their new vertices. The embedding fake reproduces the reported lifecycle (one pooled connection bound to the loop that first used it) — it models the transport binding, not a live HTTP keep-alive exchange, so it does not establish real-client behavior.test_fetch_avoids_loop_bound_async_clientspins the fetch to the sync client.- Full suite: 1292 passed, 12 skipped, 2 failed — the same two pre-existing
test_external_config.pyWindows mode-bit failures as on main.ruff format/checkclean on touched files;ty check hugegraph_llm265 diagnostics before and after (zero new).
BuildSemanticIndex._get_embeddings_parallel duplicated get_embeddings_parallel with divergent batching (1000 vs 500) and concurrency (Semaphore(10) vs none), and the shared helper's docstring promised a semaphore it never had. Every index build now shares one bounded implementation: - get_embeddings_parallel caps in-flight batches at 10 (the bound BuildSemanticIndex already used), so a large input cannot fan out one request per batch - BuildSemanticIndex delegates to the shared helper; its private copy is removed and the build_gremlin_example_index.py FIXME is resolved - order preservation across batches is unchanged (asyncio.gather) Behavioral deltas for the semantic vid index: batches of 500 instead of 1000 (progress granularity only; embedding models batch internally per API call) and async_get_texts_embeddings instead of executor-wrapped sync get_texts_embeddings - the Ollama/OpenAI implementations build sync and async clients from the same config with identical batching and ordering semantics. Tests: new tests/utils/test_embedding_utils.py pins order, batching, empty input, error propagation and the concurrency bound (red before the fix: all 30 batches in flight at once). Semantic-index tests follow the unified batching and patch the shared entry point (red before the fix, green after). Co-Authored-By: Claude Code <noreply@anthropic.com>
…ient Review finding (imbajin, apache#424): index operators run get_embeddings_parallel through asyncio.run() once per update, so each nonempty update gets a fresh event loop after the previous one is closed, while the OpenAI/Ollama async client kept by the embedding instance retains its HTTP connection pool. With a reusable keep-alive connection, the second update reuses pooled state bound to the closed loop and fails with RuntimeError: Event loop is closed, leaving the new vertices unindexed. The same failure applied to every consumer of the shared helper (BuildSemanticIndex, BuildGremlinExampleIndex, BuildVectorIndex). The fetch path now retains the synchronous executor route BuildSemanticIndex used before this PR: each batch runs get_texts_embeddings on the loop's executor. The synchronous clients hold no loop-bound state, so client and loop lifetimes no longer need to match across updates. Order, batching (500) and the in-flight bound (10) are unchanged; the async fetch methods remain on the embedding API for direct use. Tests: test_two_updates_on_one_instance_keep_indexing covers two nonempty updates on one BuildSemanticIndex instance through the real helper (red before: RuntimeError: Event loop is closed on the second update, via an embedding fake whose pooled connection is bound to the loop that first used it). test_fetch_avoids_loop_bound_async_clients pins the fetch to the sync client. The tracking fake now records from both paths, keeping the order, concurrency-bound and error-propagation tests valid for either route. Co-Authored-By: Claude Code <noreply@anthropic.com>
d1d8ead to
0722aac
Compare
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Score: 9/10. Summary: get_embeddings_parallel caps in-flight batches at 10 and runs each batch through sync get_texts_embeddings on the loop's executor, so BuildSemanticIndex, BuildVectorIndex and BuildGremlinExampleIndex share one bounded, loop-agnostic path; the earlier event-loop-lifetime comment is fixed and covered by test_two_updates_on_one_instance_keep_indexing.
imbajin
left a comment
There was a problem hiding this comment.
Score: 7.5/10 · Confidence: high
Blocking: yes
Summary
Synchronous executor batches can continue issuing requests after a peer batch fails.
Evidence
A controlled standard-library executor reproduction showed ongoing work completing before asyncio.run returned. Source inspection confirms OpenAI batches issue sequential requests; no live provider was exercised.
Validation notes
Controlled executor reproduction and static source review; no live embedding provider tested.
| # Callers run this helper on a fresh loop per update, while the embedding | ||
| # instance (and its clients) outlives those loops. The synchronous client | ||
| # holds no loop-bound state, so repeated runs on one instance stay safe. | ||
| result = await loop.run_in_executor(None, embedding.get_texts_embeddings, batch) |
There was a problem hiding this comment.
If one batch fails while other batches are running, asyncio.run() waits for its executor threads to finish; cancelling the futures cannot stop their synchronous work. For Gremlin and vector indexing this replaces cancellable async requests, and an OpenAI batch can continue issuing up to 50 sequential requests after the overall operation has failed, delaying the error and incurring unnecessary API usage. Please make remaining batches cooperatively stop on failure or use cancellable requests with a client lifecycle scoped to the event loop.
Problem
Two divergent parallel-embedding implementations for the same job:
utils/embedding_utils.get_embeddings_parallel— batch 500, no concurrency bound, despite its docstring promising "a semaphore to limit maximum concurrency"BuildSemanticIndex._get_embeddings_parallel— batch 1000,Semaphore(10), sync method viarun_in_executorbuild_gremlin_example_index.pycarried a FIXME for this split ("we need keep the logic same with build_semantic_index.py"). The unbounded helper is a hazard on large inputs: 100k chunks -> 200 batch requests all in flight at once.What this changes
One bounded implementation (
get_embeddings_parallel) shared by all three index builds:Semaphore(10)per batch (the boundBuildSemanticIndexalready used), honoring the docstring contract that already existedBuildSemanticIndexdelegates to the helper; its private copy is removed and the FIXME is retiredasyncio.gather)Behavior deltas (for review)
get_texts_embeddings(see the review follow-up below)Review follow-up (0722aac)
imbajin's review caught a lifecycle regression in the unification: the helper ran batches through
embedding.async_get_texts_embeddings, while the index operators call it throughasyncio.runonce per update — a fresh loop per update against an instance-owned async client whose pooled keep-alive connections outlive the first loop. The second nonempty update on one instance could fail withRuntimeError: Event loop is closedand leave the new vertices unindexed; all three consumers of the helper shared the failure. The helper now retains the synchronous executor path: each batch runsget_texts_embeddingson the loop's executor, which holds no loop-bound state, so client and loop lifetimes no longer need to match. Order, batching (500) and the in-flight bound (10) are unchanged; the async fetch methods stay on the embedding API for direct use.Validation (executed locally)
SKIP_EXTERNAL_SERVICES=true uv run --extra dev pytest -q hugegraph-llm/src/tests-> 1292 passed, 12 skipped, 2 failed (after0722aac; 1290 passed at the first push) — the twotest_external_config.pyfailures (test_saved_complete_connection_survives_deployment_upgrade,test_single_file_mount_fallback_and_existing_mode) also fail on cleanorigin/main(verified via stash), pre-existing and unrelatedtest_bounds_concurrent_batchesfails before the fix (30 batches in flight vs bound 10) and passes after; the updated semantic-index tests fail against the old private implementation (4 failed) and pass after; after0722aac,test_two_updates_on_one_instance_keep_indexingis red before withRuntimeError: Event loop is closedon the second update and green after, andtest_fetch_avoids_loop_bound_async_clientspins the fetch to the sync clientuv run --extra dev ruff format --check+ruff checkclean on changed filesuv run --extra dev ty check hugegraph-llm/src/hugegraph_llmreports 265 diagnostics before and after0722aac(none introduced)test_build_vector_index.py/test_gremlin_example_index_query.py emit a "coroutine never awaited" RuntimeWarning (their patched async helper returns a coroutine the also-mockedasyncio.rundiscards) — reproduced on cleanorigin/mainRefs #107
🤖 Generated with Claude Code