Skip to content

fix(vectorize): include request kwargs in the embedding cache key - #853

Open
pcbeingused333 wants to merge 1 commit into
redis:mainfrom
pcbeingused333:fix/embed-cache-kwargs-c3168
Open

pcbeingused333 wants to merge 1 commit into
redis:mainfrom
pcbeingused333:fix/embed-cache-kwargs-c3168

Conversation

@pcbeingused333

@pcbeingused333 pcbeingused333 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

With an EmbeddingsCache attached, embed, embed_many, aembed and aembed_many look up and store vectors by content and model name only. Provider kwargs that change the output are ignored, so for Cohere (the pattern in the CohereTextVectorizer docstring) embed("x", input_type="search_document") followed by embed("x", input_type="search_query") returns the stored document vector for the query, with no error.

Changes

Request kwargs are part of the cache key

BaseVectorizer._cache_model_name(kwargs) returns self.model when a call has no request kwargs, and otherwise f"{self.model}:{hashify(json.dumps(kwargs, sort_keys=True, default=str))}". embed, embed_many, aembed, aembed_many and the four batch cache helpers use it for lookup and store, so a vector cached for one input_type is no longer returned for another. Entries cached by calls without kwargs keep their current key. metadata is popped before the key is computed, so it does not affect the key (it is still stored as metadata).

  • EmbeddingsCache and the concrete vectorizers are untouched.
  • tests/unit/test_base_vectorizer.py: a fake vectorizer whose output depends on input_type and an in-memory cache cover sync/async single and batch calls, plus the unchanged key without kwargs.

Release Notes

Embedding calls that pass provider kwargs (for example Cohere's input_type) now cache separately per kwarg value instead of returning a vector cached for different kwargs. Calls without kwargs use the same cache keys as before.

Reviewer Brief

Start here: redisvl/utils/vectorize/base.py, _cache_model_name, and its use in embed / embed_many.

Where it could fail

  • Kwargs are serialised with json.dumps(..., default=str); a kwarg whose str() is not stable across processes (for instance an object using the default repr) would miss the cache each run. Plain values (strings, numbers, lists, dicts) are stable.
  • embed_many now pops metadata before the cache lookup even when every item hits; it was previously only popped on a miss, with no other effect.

Not verified: a live Cohere/Voyage call and a real Redis-backed EmbeddingsCache (the tests use an in-memory stand-in); the new tests ran with --noconftest (no Docker Redis).

Beyond CI: with base.py restored to main, the 4 new kwargs tests fail and the other 2 pass; at this commit all 6 pass. black, isort and mypy are clean on the touched source file.

Reviewed 30f9779 · Claude Code (claude-sonnet-5-5) · lenses: correctness, tests

Prepared with the help of an AI coding assistant (Claude).


Note

Low Risk
Narrow caching-layer fix with backward-compatible keys for kwargs-free calls; main caveat is unstable str() for exotic kwarg values weakening cache hits, not corrupting results.

Overview
Fixes incorrect embedding cache hits when the same content is embedded with different provider kwargs (e.g. Cohere input_type), which previously shared one cache entry keyed only by content and base model name.

BaseVectorizer adds _cache_model_name, which keeps self.model when there are no request kwargs and otherwise appends a hashify of JSON-serialized kwargs. embed, embed_many, aembed, and aembed_many (plus batch cache helpers) use that value for get/set instead of self.model. metadata is popped before the key is built so it still affects stored metadata but not the key.

Unit tests add a kwargs-sensitive fake vectorizer and in-memory cache covering sync/async single and batch paths, including unchanged keys when no provider kwargs are passed.

Reviewed by Cursor Bugbot for commit 30f9779. Bugbot is set up for automated code reviews on this repo. Configure here.

embed/embed_many (and async) cached by content and model name only, so a
call with a different provider kwarg (e.g. Cohere input_type) returned the
vector stored for another kwarg. Kwargs are now hashed into the cache model
name; calls without kwargs keep the existing key.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant