Skip to content

fix(llm): align VectorStoreBase annotations and finish its docstrings - #414

Open
lplalbert wants to merge 4 commits into
apache:mainfrom
lplalbert:fix/vector-store-base-contract
Open

lplalbert wants to merge 4 commits into
apache:mainfrom
lplalbert:fix/vector-store-base-contract

Conversation

@lplalbert

@lplalbert lplalbert commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Refs #107 (Tiny Task: fix TODO/FIXME in LLM code)

What

VectorStoreBase carried 6 #TODO: finish comment docstrings and 2 annotations that
contradict all three backends (faiss/qdrant/milvus):

  • get_all_properties() -> list[str], but add(props: List[Any]) stores arbitrary values
    (faiss keeps list[Any])
  • clean() -> bool, but every backend deletes and returns nothing

Review follow-ups asked for two contracts the interface left implicit: what from_name
does when embed_dim disagrees with persisted data, and what add() /
get_all_properties() actually round-trip.

Changes

  • Wrote the six placeholder docstrings from the three backends' actual behavior.
  • Annotations: get_all_properties → list[Any], clean → None.
  • from_name documents the dimension-mismatch contract: reusing persisted vectors
    requires passing their existing dimension as embed_dim; a mismatch yields a fresh
    empty index with backend-specific stale-data disposal (the file store keeps its
    files, server-backed stores recreate the collection).
  • add() documents the per-backend storage forms for non-string properties and the
    read methods reference it: the file-backed store keeps the original Python values
    (pickle), server-backed JSON-payload stores keep JSON-representable values, and
    text-column backends store JSON text and return that text. Input APIs (add,
    remove) take original values; read APIs (search, get_all_properties) return
    the stored form. Review (4580997499) flagged that the first version also promised
    the string round-trip here before this head implemented it; those sentences now
    live in fix(llm): honor vector_dis_threshold and validate results in VectorIndexQuery #412 beside the implementing code (83ba325), and add() here covers only
    the non-string storage forms (trimmed in 4c9731d).
  • Added src/tests/indices/test_vector_store_base_contract.py (docstrings complete,
    annotations match the implemented contract).

Validation (actually run; offline only)

  • red→green (stash-demonstrated in the first round): 3 failed on the old contract →
    all green afterwards
  • SKIP_EXTERNAL_SERVICES=true uv run --extra dev pytest -q hugegraph-llm/src/tests/indices
    → 21 passed, 1 skipped (this head)
  • ruff format --check / ruff check → clean
  • ty check → no diagnostics in indices/vector_index/base.py (stash-compared; the
    old-contract diagnostics are gone)
  • Real-provider verification: none (offline tests only; non-string storage form stays
    documented-only, which the review accepts)

Refs #107

🤖 Generated with Claude Code

@github-actions github-actions Bot added the llm label Nov 9, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: Please clarify the index loading contract for dimension mismatches. Evidence: Static inspection of the FAISS and Qdrant implementations.

#TODO: finish comment
Return the index registered under the given name.

Loads persisted data when it exists; otherwise returns an empty index sized to embed_dim.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ The dimension is also used when an index already exists: FaissVectorIndex.from_name returns an empty index on a mismatch, and QdrantVectorIndex.from_name deletes and recreates the existing collection. A caller following this contract could therefore lose stored vectors. Please document the backend-specific mismatch behavior and the need to supply the existing dimension.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified in 182ddef. from_name now documents the mismatch path explicitly: reusing persisted vectors requires passing their existing dimension as embed_dim; vectors built with a different dimension cannot be reused, so the call returns a fresh empty index sized to embed_dim instead of failing the load, and disposal of the stale data is backend-specific (the file-backed store keeps its saved files, server-backed stores like Qdrant delete and recreate the collection). A caller that changes embed_dim must expect to re-index. Behavior is unchanged and stays pinned by test_load_with_different_dimension_starts_empty_index.

@lplalbert
lplalbert force-pushed the fix/vector-store-base-contract branch from 8689bff to 182ddef Compare November 10, 2026 09:49

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: Please align the property retrieval contract with the Milvus backend. Evidence: Static inspection of MilvusVectorIndex.add and get_all_properties.

Return all properties currently stored in the index.

Returns:
List[Any]: The values previously passed to add().

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This promises the values passed to add(), but MilvusVectorIndex.add() JSON-encodes non-string values and get_all_properties() returns those encoded strings while filtering out empty strings. For example, a stored dictionary is returned as a JSON string and an empty string is omitted. Please document these backend differences or make the implementation return the original values.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documented in 59db519, taking your "document these backend differences" branch.

Root cause: the docstring promised "the values previously passed to add()", but the read-back form is storage-defined — text-column backends (Milvus) keep the property as text, so a stored dict comes back as its JSON text, and its query path also dropped empty strings server-side. A verbatim promise could only hold for strings.

Structural fix: add() now carries the round-trip contract and the read methods reference it — string properties (including JSON-shaped text like 123 and empty strings) round-trip unchanged on every backend; non-string values come back in their backend storage form (file-backed pickle keeps the original values, server-backed JSON payloads keep JSON-representable values, text columns return the JSON text). Input APIs (add, remove) take the original values; read APIs (search, get_all_properties) return the stored form. Callers pass strings for exact round trip. The stated behavior for strings — including the empty-string and JSON-shaped cases — is implemented at the producing boundary in #412 (129cc48 removed Milvus' json.loads decode and its property != "" filter), so if this PR lands first the doc describes that head's behavior for a moment.

Validation: docstring-only change — ruff format --check / ruff check / ty check clean on the file, pytest hugegraph-llm/src/tests/indices 21 passed / 1 skipped. The string round trip is pinned per backend in tests (Qdrant real-client round trip here; Milvus JSON-shaped-text and empty-string round trip in #412's test_optional_vector_clients.py); non-string storage form stays documented-only, as this thread allows. Remaining cleanup: none in this scope.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Score: 9/10. Summary: The VectorStoreBase docstrings and the get_all_properties and clean annotations match the FAISS, Qdrant and Milvus backends, and 182ddef documents the from_name dimension-mismatch behaviour raised earlier.

lplalbert and others added 3 commits November 16, 2026 03:13
Six VectorStoreBase entry points still carried '#TODO: finish comment'
placeholders, and two annotations contradicted the implemented contract:
get_all_properties said list[str] although add() accepts List[Any] (the
FAISS store annotates list[Any]), and clean() said bool although every
backend deletes and returns nothing.

- document get_all_properties, save_index_by_name, get_vector_index_info,
  from_name, exist and clean from the three backend implementations
- get_all_properties -> list[Any]; clean -> None
- add contract tests covering the docstrings and both annotations

Co-Authored-By: Claude Code <noreply@anthropic.com>
Clarify what from_name returns when the persisted index dimension
differs from embed_dim: a fresh empty index instead of a failed load,
with backend-specific disposal of the stale data (file store keeps the
saved files, server-backed stores recreate the collection).

Co-Authored-By: Claude Code <noreply@anthropic.com>
get_all_properties() promised the values passed to add(), but the read
path is backend-defined for non-string values and text-column backends
decode nothing useful to promise verbatim. Spell out the contract:
string properties (including JSON-shaped text and empty strings)
round-trip unchanged on every backend, non-string values come back in
their backend storage form (pickle values, JSON payloads, or JSON text),
and input APIs take the original values while read APIs return the
stored form. Pass strings for exact round trip.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@lplalbert
lplalbert force-pushed the fix/vector-store-base-contract branch from 182ddef to 59db519 Compare November 15, 2026 19:15

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The add() round-trip contract contradicts MilvusVectorIndex at this head and holds only once #412 merges. The other docstrings and the get_all_properties and clean annotations match the three backends.

"""
Add a list of vectors and their corresponding properties to the store.

String properties round-trip verbatim through search() and

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important: at this head MilvusVectorIndex.search still runs json.loads on each property, so "123" returns as 123, and get_all_properties filters property != "", so empty strings vanish; this promise holds only after #412. Move the string round-trip sentences into #412 beside the code that implements them, or hold this PR until #412 merges.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved — 4c9731d takes your first option.

Root cause of the overpromise: I documented the string round-trip guarantee here while the code that establishes it for Milvus lives in #412 (129cc48), so this head promised behavior it does not yet implement.

Structural fix: the string round-trip sentences now sit beside that implementation — MilvusVectorIndex.search/get_all_properties in #412 (83ba325) state that property strings round-trip verbatim, including JSON-shaped text and empty strings. This PR now documents only what its own code establishes: the per-backend non-string storage forms (pickle values / JSON payloads / JSON text), the input-API vs read-API split, and the guidance to pass strings to avoid those differences.

Validation: docstring-only change — ruff format --check / ruff check / ty check clean on base.py. No holding on #412 is needed: the two PRs no longer claim overlapping behavior, so each head's docs match its own code.

The string round-trip guarantee does not hold at this head (the Milvus
decode and empty-string filter are fixed in apache#412), so its sentences move
to apache#412 beside the code that establishes it. This contract now documents
the per-backend non-string storage forms and the input/read split only.

Co-Authored-By: Claude Code <noreply@anthropic.com>

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The trimmed add() contract still describes #412's MilvusVectorIndex, not this head's: Milvus search decodes JSON, so it returns neither the stored text nor unchanged strings. The other docstrings and the get_all_properties and clean annotations match the three backends.

(pickle), server-backed JSON-payload stores keep JSON-representable
values, and text-column backends store JSON text and return that text.
Input APIs (add, remove) take the original values; read APIs (search,
get_all_properties) return the stored form. Pass string properties to

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important: at this head MilvusVectorIndex.search runs json.loads on each property, so a stored dict returns as a dict instead of "that text", and the string "123" returns as 123, which breaks "Pass string properties to avoid these per-backend differences". Limit the text-column and string claims to what this head does, or move them into #412 with the code that removes the decode.

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Score: 7.5/10 · Confidence: high

Blocking: no

Summary

The new contract overstates removal support for unhashable properties; the existing review also identifies inaccurate Milvus read behavior.

Evidence

Inspected exact-head backend remove implementations and Milvus read conversion.

Validation notes

Static inspection only; no live vector database was exercised.

(pickle), server-backed JSON-payload stores keep JSON-representable
values, and text-column backends store JSON text and return that text.
Input APIs (add, remove) take the original values; read APIs (search,
get_all_properties) return the stored form. Pass string properties to

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Qualify the removal contract for unhashable properties

add() accepts dictionaries and lists, but the backends convert a list of removal properties to a set before processing it. For example, remove([{"k": 1}]) raises TypeError instead of deleting the value. Please document the hashability restriction or make removal accept these original values.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants