Skip to content

perf(llm): batch gremlin neighbour queries - #430

Open
ZhYQ1 wants to merge 2 commits into
apache:ontogenyfrom
ZhYQ1:perf/gremlin-neighbor-batching
Open

ZhYQ1 wants to merge 2 commits into
apache:ontogenyfrom
ZhYQ1:perf/gremlin-neighbor-batching

Conversation

@ZhYQ1

@ZhYQ1 ZhYQ1 commented Nov 12, 2026

Copy link
Copy Markdown

Closes #429

Summary

The id-to-match subgraph branch queried neighbours with one gremlin().exec() per matched vid, so a large id set cost one server round-trip per id. Group the ids into batches of NEIGHBOR_QUERY_BATCH_SIZE (10) and issue one exec per batch, concatenating each batch's ["data"] in matched_vids order.

Changes

  • nodes/hugegraph_node/graph_query_node.py — batch the per-vid neighbour traversal; the full-vertex VERTEX_QUERY_TPL exec and the property-match branch are unchanged.
  • tests/nodes/test_graph_query_node.py — covers batch count (5 vids → 2 execs, 25 vids → 1 + ceil(25/10)), single vid, empty set, id split across batches, and cross-batch path ordering.

Query count

For N matched vids: 1 + N execs → 1 + ceil(N / 10) execs (e.g. N=25: 26 → 4).

Test plan

From hugegraph-llm:

ruff check src/hugegraph_llm/nodes/hugegraph_node/graph_query_node.py src/tests/nodes/test_graph_query_node.py
ruff format --check src/hugegraph_llm/nodes/hugegraph_node/graph_query_node.py src/tests/nodes/test_graph_query_node.py
PYTHONPATH=src pytest src/tests/nodes -q

Result: new tests pass, 9 passed in src/tests/nodes, ruff clean.

Notes

@ZhYQ1
ZhYQ1 force-pushed the perf/gremlin-neighbor-batching branch 3 times, most recently from 5c63ea7 to 1b9241e Compare November 12, 2026 17:34
@ZhYQ1

ZhYQ1 commented Nov 12, 2026

Copy link
Copy Markdown
Author

Rebased onto latest ontogeny (e3b98ec). The branch previously carried stale
fork history from earlier rebases; after this force-push it contains only this
change (1 commit: graph_query_node.py + test_graph_query_node.py). No semantic
changes vs. the earlier review state.

@github-actions github-actions Bot added the llm label Nov 12, 2026

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: batching the neighbour query shares the edge limit, dedup() and max_items cap across each batch of 10 vids, so the subgraph drops most neighbours of every vid after the first. Wrapping the per-vid traversal in local(...) keeps one round-trip per batch with per-vid results.

batch = matched_vids[start : start + batch_size]
gremlin_query = VID_QUERY_NEIGHBOR_TPL.format(
keywords=f"'{matched_vid}'",
keywords=", ".join(f"'{vid}'" for vid in batch),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important: bothE().limit({edge_limit}), dedup() and .limit({max_items}) in VID_QUERY_NEIGHBOR_TPL are global steps, so the vids in one batch share a single edge budget and dedup set; on TinkerGraph 3.5.1 with one edge label, 10 vids with 8 neighbours each return 8 paths, all from the first vid, instead of 80. Wrap the traversal after g.V(...) in local(...) so each vid keeps its own limits and dedup, and add a test that checks the generated query keeps that per-vid scope.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this — it was a real behaviour regression, not just an edge case.

Fixed in e3409f6. The traversal after g.V(...) is now wrapped in local(__. ...), so each matched vid runs the inner repeat ... path ... limit(max_items) in its own scope. bothE().limit(edge_limit), dedup() and .limit(max_items) are now per vid instead of sharing one edge budget / dedup set / cap across the batch, while the query count is still 1 + ceil(N / 10) (N=25: 4 execs, unchanged).

Generated query shape:

g.V('v1', ..., 'v10')
.local(
   __.repeat(
      bothE('edge').limit(edge_limit).otherV().dedup()
   ).times(max_deep).emit()
   .simplePath()
   .path()
   .by(project('label', 'id', 'props')...)
   .by(project('label', 'inV', 'outV', 'props')...)
   .limit(max_items)
)
.toList()

Semantics are back to the old per-vid loop: each vid contributes its own paths, capped independently by max_items, with no cross-vid contamination of the edge limit or dedup() set.

Tests added in src/tests/nodes/test_graph_query_node.py, both asserting the generated query keeps the per-vid scope:

  • test_neighbor_query_wraps_per_vid_traversal_in_local_scope — bothE(, dedup() and both .limit( steps are inside local(...), with the global .toList() outside.
  • test_neighbor_query_keeps_limits_inside_local_not_global — no limit( / dedup() leaks ahead of .local(, and both limits are scoped inside it.

Verification:

ruff check <both files>            -> All checks passed
ruff format --check <both files>   -> 2 files already formatted
PYTHONPATH=src pytest src/tests/nodes -q -> 11 passed

Branch was also rebased onto current upstream/ontogeny, so the diff is now just the two files above.

@ZhYQ1
ZhYQ1 force-pushed the perf/gremlin-neighbor-batching branch from 1b9241e to e3409f6 Compare November 13, 2026 05:14
@ZhYQ1 ZhYQ1 changed the title Perf/gremlin neighbor batching perf(llm): batch gremlin neighbour queries Nov 14, 2026

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: the branch sits on open #391, so merging it lands that PR's 68 commits and 383 files in ontogeny. The local(...) change resolves the earlier shared-budget comment: on HugeGraph 1.8.0 the batched query returns the same paths, in the same order, as the per-vid loop.

# Number of matched vids grouped per single gremlin ``exec`` for the neighbour
# traversal. One round-trip per group instead of one per matched vid; the batch
# stays small to keep each query's work bounded.
NEIGHBOR_QUERY_BATCH_SIZE = 10

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important: the head is based on #391's head d97afa1, not on ontogeny, so this PR carries #391's 68 commits and 383 extra files and merging it lands them without that PR's review. Rebase the two llm commits onto ontogeny so the diff holds only graph_query_node.py and its test.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bitflicker64 Done — rebased the two llm commits onto ontogeny and force-pushed perf/gremlin-neighbor-batching. The PR diff now contains only those 2 commits and graph_query_node.py + its test (test_graph_query_node.py).

The batched neighbour query ran repeat/bothE().limit()/dedup()/limit(max_items) as global steps, so all vids in a batch shared one edge budget, one dedup set and one result cap; on a batch of 10 vids only the first vid's neighbours survived. Wrap the traversal after g.V(...) in local(__. ...) so each vid keeps its own limits and dedup while still issuing one exec per batch. Add tests asserting the generated query keeps those steps inside the local() scope.
@ZhYQ1
ZhYQ1 force-pushed the perf/gremlin-neighbor-batching branch from e3409f6 to fb2dee7 Compare November 20, 2026 02:14

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Score: 9/10. Summary: Wrapping the neighbour traversal in local(...) keeps edge_limit, dedup() and max_items per vid while cutting exec calls from 1 + N to 1 + ceil(N/10), and the branch holds only graph_query_node.py and its test on ontogeny. The new tests in test_graph_query_node.py pass in the llm CI job.

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Score: 8.0/10 · Confidence: medium

Blocking: no

Summary

No new issue found in the per-vertex batching change. The earlier shared-limit and branch-history findings are addressed in this diff.

Evidence

Reviewed the traversal, call path, existing discussion and server ID lookup; the LLM CI job passed for this head.

Validation notes

The added tests mock Gremlin execution; I did not independently run a live server or validate other server versions and high-load behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants