Repository navigation
fix(vectorize): include request kwargs in the embedding cache key - #853
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
embed/embed_many (and async) cached by content and model name only, so a call with a different provider kwarg (e.g. Cohere input_type) returned the vector stored for another kwarg. Kwargs are now hashed into the cache model name; calls without kwargs keep the existing key.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
With an
EmbeddingsCacheattached,embed,embed_many,aembedandaembed_manylook up and store vectors by content and model name only. Provider kwargs that change the output are ignored, so for Cohere (the pattern in theCohereTextVectorizerdocstring)embed("x", input_type="search_document")followed byembed("x", input_type="search_query")returns the stored document vector for the query, with no error.Changes
Request kwargs are part of the cache key
BaseVectorizer._cache_model_name(kwargs)returnsself.modelwhen a call has no request kwargs, and otherwisef"{self.model}:{hashify(json.dumps(kwargs, sort_keys=True, default=str))}".embed,embed_many,aembed,aembed_manyand the four batch cache helpers use it for lookup and store, so a vector cached for oneinput_typeis no longer returned for another. Entries cached by calls without kwargs keep their current key.metadatais popped before the key is computed, so it does not affect the key (it is still stored as metadata).EmbeddingsCacheand the concrete vectorizers are untouched.tests/unit/test_base_vectorizer.py: a fake vectorizer whose output depends oninput_typeand an in-memory cache cover sync/async single and batch calls, plus the unchanged key without kwargs.Release Notes
Embedding calls that pass provider kwargs (for example Cohere's
input_type) now cache separately per kwarg value instead of returning a vector cached for different kwargs. Calls without kwargs use the same cache keys as before.Reviewer Brief
Start here:
redisvl/utils/vectorize/base.py,_cache_model_name, and its use inembed/embed_many.Where it could fail
json.dumps(..., default=str); a kwarg whosestr()is not stable across processes (for instance an object using the defaultrepr) would miss the cache each run. Plain values (strings, numbers, lists, dicts) are stable.embed_manynow popsmetadatabefore the cache lookup even when every item hits; it was previously only popped on a miss, with no other effect.Not verified: a live Cohere/Voyage call and a real Redis-backed
EmbeddingsCache(the tests use an in-memory stand-in); the new tests ran with--noconftest(no Docker Redis).Beyond CI: with
base.pyrestored tomain, the 4 new kwargs tests fail and the other 2 pass; at this commit all 6 pass.black,isortandmypyare clean on the touched source file.Reviewed
30f9779· Claude Code (claude-sonnet-5-5) · lenses: correctness, testsPrepared with the help of an AI coding assistant (Claude).
Note
Low Risk
Narrow caching-layer fix with backward-compatible keys for kwargs-free calls; main caveat is unstable
str()for exotic kwarg values weakening cache hits, not corrupting results.Overview
Fixes incorrect embedding cache hits when the same content is embedded with different provider kwargs (e.g. Cohere
input_type), which previously shared one cache entry keyed only by content and base model name.BaseVectorizeradds_cache_model_name, which keepsself.modelwhen there are no request kwargs and otherwise appends ahashifyof JSON-serialized kwargs.embed,embed_many,aembed, andaembed_many(plus batch cache helpers) use that value for get/set instead ofself.model.metadatais popped before the key is built so it still affects stored metadata but not the key.Unit tests add a kwargs-sensitive fake vectorizer and in-memory cache covering sync/async single and batch paths, including unchanged keys when no provider kwargs are passed.
Reviewed by Cursor Bugbot for commit 30f9779. Bugbot is set up for automated code reviews on this repo. Configure here.