Repository navigation
Conversation
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: the coercion and partial-import changes match #373, but the new skipped-endpoint check drops edges to vertices that were created when the same vertex id appears more than once in the input.
| continue | ||
|
|
||
| out_v, in_v = edge.get("outV"), edge.get("inV") | ||
| failed_endpoints = [endpoint for endpoint in (out_v, in_v) if endpoint in skipped_ids] |
There was a problem hiding this comment.
Important: extraction does not dedupe vertices across chunks, so when one copy of 1:Sarah is skipped and another is created, 1:Sarah sits in both skipped_ids and vid_mapping, and every edge to Sarah is dropped although the vertex exists (base writes the edge). Skip an endpoint only when it is in skipped_ids and not in vid_mapping, and add a test with a duplicated vertex.
There was a problem hiding this comment.
Good catch — pushed (ca011d2). Root cause: the endpoint check keyed only on skipped_ids, but extraction without cross-chunk dedupe can leave a duplicated input id in both skipped_ids (one copy skipped) and vid_mapping (another copy created), so every edge to that id was dropped although the vertex exists.
Structural fix: an endpoint is skipped only when it is in skipped_ids and no created vertex claimed it (endpoint not in vid_mapping). A duplicated id whose copy was created keeps resolving through vid_mapping to the created server id.
Test: test_load_into_graph_keeps_edges_when_a_duplicated_vertex_was_created — one copy of 1:Sarah skipped on a non-nullable coercion failure, the other created under the same id; the edge must run against the created copy's server id (asserted on the add_edge call). Red without the fix (the edge was dropped and add_edge never called).
There was a problem hiding this comment.
Score: 7.0/10 · Confidence: medium
Blocking: yes
Summary
Two correctness issues need to be addressed before merging:
- Lossy numeric conversion: DOUBLE coercion can silently change a property value.
- Hidden partial failures: callers receive a successful result without knowing which vertices were skipped.
Evidence
- Executed the numeric coercion methods from this PR:
9007199254740993is accepted as9007199254740992.0. - Exercised the import method with controlled graph I/O: one vertex succeeds, one is rejected, and the returned data has no partial-failure summary.
Validation notes
A live HugeGraph server was not exercised. The reproduced findings have high confidence; the overall score remains provisional.
| if data_type in (PropertyDataType.FLOAT.value, PropertyDataType.DOUBLE.value): | ||
| if isinstance(value, (int, float, str)): | ||
| try: | ||
| return [float(value)] |
There was a problem hiding this comment.
Both an integer and its string form can be accepted even when float() changes the value:
9007199254740993 -> 9007199254740992.0
"9007199254740993" -> 9007199254740992.0Impact: the graph write receives a different value while the conversion is reported as valid.
Suggested fix: reject conversions that lose information, and add regression coverage for large integers and numeric strings.
There was a problem hiding this comment.
Agreed — pushed (ca011d2). Root cause: the FLOAT/DOUBLE branch accepted float(value) without comparing the result to the value that was written, so IEEE-754 rounding (9007199254740993 and "9007199254740993" -> 9007199254740992.0) was reported as a valid conversion while the graph received a different value.
Structural fix: a conversion is lossless only when the written value and the converted value denote the same number (Decimal(str(value)) == Decimal(repr(converted))). Anything else is reported like an un-coercible value (nullable property dropped, non-nullable skips the vertex with a reason) instead of silently rounded. Out-of-range inputs (10**400, "1e400") no longer raise OverflowError or write inf — they report as un-coercible as well. Exactly representable boundaries still coerce (2**53 -> 9007199254740992.0, "3.50" -> 3.5).
Test: test_coerce_property_value_rejects_lossy_double_conversions covers the large-integer and numeric-string forms from the report, out-of-range values for DOUBLE and FLOAT, and the exact-representability boundary. Red without the fix (rounded values returned as valid; 10**400 crashed on OverflowError).
| skip_reports.append(f"{mapping_id or input_label}: server rejected the vertex") | ||
| if mapping_id: | ||
| skipped_ids.add(mapping_id) | ||
| continue |
There was a problem hiding this comment.
Example: the server creates one vertex but rejects another.
Actual result: this branch continues, and run() returns the original data, including the rejected vertex. The import flow returns successfully without identifying the skipped vertex or its dependent edges.
Expected: return skipped IDs and reasons through the flow/API result, as requested in #373. Cover the case with one successful and one rejected vertex in a flow-level test.
There was a problem hiding this comment.
Fixed — pushed (ca011d2). Root cause: load_into_graph only logged the skips and run() returned data untouched, so the flow/API result carried no record of the rejected vertex; and WkFlowState.assign_from_json drops keys that are not state members, so a report attached in run() would not have reached the result either.
Structural fix: load_into_graph now returns a structured import_summary — imported_vertices/imported_edges counts, skipped_vertices as [{id, reason}] (unknown label / missing primary key / non-nullable coercion failure / server rejection) and skipped_edges as [{label, outV, inV, reason}], including the dependent edges of skipped vertices — and run() attaches it to the data returned into the pipeline. WkFlowState gains an import_summary member (with setup() reset) so the report survives assign_from_json and appears in the flow/API result via ImportGraphDataFlow.post_deal.
Test: test_import_flow_reports_skipped_vertex_and_dependent_edges runs the real ImportGraphDataFlow end to end (server creates one vertex, rejects the other) and asserts the result names person:Bob with its reason and the dependent knows edge (label/outV/inV + reason). Red without the fix (import_summary missing from the result).
A single property value that failed the declared-type check used to
drop the whole vertex while logging as if only the property was
skipped, and edges referencing it were sent to the server with the raw
local id, failing with a cryptic "Invalid vertex id".
- Coerce values losslessly onto the declared type first (numeric
strings to numbers, numbers to text, scalars to single-element
collections, strict yyyy-MM-dd dates).
- An un-coercible nullable property is dropped on its own; an
un-coercible non-nullable property skips only that vertex, both
named in the logs and the import summary.
- Edges referencing skipped vertices are skipped with a reason instead
of being written with unresolved local ids; unknown endpoints are
passed through unchanged since they may reference pre-existing
server vertices.
- Import proceeds partially and reports a summary; it only raises
("Failed to create vertex") when no vertex could be imported at all,
before any edge write.
- bool no longer validates as INT (True is not a valid number), BLOB
validates as text to match its TEXT creation, unknown property keys
are dropped instead of crashing with KeyError, and unknown data
types are reported per property instead of raising.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Review round for apache#422 (Refs apache#373): - Skip an edge endpoint only when its id is in skipped_ids and no created vertex claimed it, so edges of a duplicated input id whose later copy was imported still run against the created server id. - Reject DOUBLE/FLOAT coercions that change the written value (9007199254740993 and "9007199254740993" no longer round to ...992.0) and report out-of-range values instead of raising OverflowError or writing inf. - Return import_summary through the flow/API result naming each skipped vertex id with its reason and each skipped edge (label, outV, inV, reason), including the dependent edges of skipped vertices. Validation: 3 new regressions red-green (duplicated-vertex edge kept, lossy-double rejection, flow-level one-created-one-rejected report); hugegraph-llm suite 1297 passed 12 skipped 2 failed (known test_external_config pair, pre-existing); ruff format/check clean; ty clean on changed files. Co-Authored-By: Claude Code <noreply@anthropic.com>
69e419c to
ca011d2
Compare
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Score: 9/10. Summary: lossless coercion, per-property and per-vertex skips, and the import_summary returned through ImportGraphDataFlow.post_deal fix #373, and the earlier duplicate-id, lossy DOUBLE and hidden partial-failure comments are resolved with tests that fail on the base.
imbajin
left a comment
There was a problem hiding this comment.
Score: 8.0/10 · Confidence: medium
Blocking: no
Summary
The current head reports partial imports, preserves usable duplicate IDs for edges, and rejects lossy numeric coercions. No new source defect was found.
Evidence
Read the full exact-head diff, all reviews and conversation, the Commit2Graph flow, and its call sites. The exact-head LLM, quality, CodeQL, and aggregate CI checks are successful; previous blocking findings are covered by the current implementation and tests.
Validation notes
The configured independent reviewer lanes are unavailable for this account, and the automation Python environment lacks pytest for a local rerun. No live HugeGraph server import was performed.
Refs #373 (stale claim comment from September had no linked PR - happy to yield if someone is actively working on it).
Problem (as observed in #373)
A single property value failing the declared-type check dropped the whole vertex while logging as if only the property was skipped, and edges referencing it were sent with the raw local id (e.g.
1:Sarah), failing withInvalid vertex id.Approach
yyyy-MM-dddates. FLOAT/DOUBLE conversions keep the written value:Decimal(str(value)) == Decimal(repr(converted))must hold, so9007199254740993and"9007199254740993"are reported instead of silently rounded to...992.0, and out-of-range inputs (10**400,"1e400") report instead of raisingOverflowErroror writinginf.skipped_idsand no created vertex claimed it: a duplicated input id whose later copy was created keeps its edges (they run against the created server id). Endpoints never seen in this import are passed through unchanged - they may legitimately reference pre-existing server vertices, so unknown stays unknown rather than being treated as invalid (deviation from the issue's strict rule, stated here for review).load_into_graphreturns animport_summary(imported counts; skipped vertex ids + reasons; skipped edges withlabel/outV/inV+ reasons, including the dependent edges of skipped vertices), whichrun()attaches to the data returned into the pipeline. AWkFlowState.import_summarymember carries it throughassign_from_jsoninto the flow/API result viapost_deal, so callers see what was skipped and why.Failed to create vertexraises only when no vertex could be imported at all, and before any edge write.Adjacent fixes in the same validation boundary:
boolno longer validates as INT (Trueis not a valid number), BLOB validates as text to match its TEXT creation mapping, unknown property keys are dropped instead of aKeyErrorcrash, and unknown data types are reported per property instead of raising out of the whole import.Validation
test_commit_to_hugegraph*.py; 3 from review roundca011d2: a duplicated vertex id keeps its edges against the created copy (test_commit_to_hugegraph_load_into_graph.py), lossy FLOAT/DOUBLE rejection covering large integers, numeric strings and out-of-range values (test_commit_to_hugegraph.py), and a flow-level one-created-one-rejected import that reports skipped ids, reasons and dependent edges throughImportGraphDataFlow.post_deal(tests/flows/test_import_graph_data.py).SKIP_EXTERNAL_SERVICES=true uv run pytest hugegraph-llm/src/tests-> 1297 passed, 12 skipped, 2 failed (the known pre-existing Windows mode-bit pair intest_external_config.py, unrelated).operators/hugegraph_op/+tests/flows/-> 59 passed (run separately).ruff format+ruff checkclean on touched files;ty checkclean on touched files (2 pre-existing transitive diagnostics unchanged; also fixed the pre-existing Match-vs-bool return in the DATE check that ty flagged in the rewritten method).Out of scope (flagged for maintainer decision)
🤖 Generated with Claude Code